This Page Intentionally Left Blank
Selections of the International Conference on Intelligent Robots and Systems I994, IROS 94, Munich, Germany, 12-i6 September i 9 9 4
E D I T E D BY
V o T.K~.R G RAE F E
Institut ffir Messtechnik Universitgt der B W Mi~nchen Neubiberg, Germany
1995 ELSEVIER A m s t e r d a m - Lausanne - New Y o r k - O x f o r d - Shannon - Tokyo
ELSEVIER SCIENCE B.V. Sara Burgerhartstraat 25 P.O. Box 2II, IOOO AE Amsterdam, The Netherlands
Library oF C o n g r e s s
Cataloging-In-Publication
Data
IEEE/RSJ/OI Internationa] ConFerence on I n t e l l i g e n t R o b o t s and Systems (1994 : M u n i c h , Oermanv) Inte]ligent r o b o t s and s y s t e m s : s e l e c t i o n s oF t h e I n t e r n a t i o n a ] C o n F e r e n c e on I n t e l l i g e n t R o b o t s and Systems 1994, IROS 9 4 , M u n i c h , Oerman#, 12-16 S e p t e m b e r 1994 / e d i t e d bv V o l k e r O r a e ~ e .
p. cm. Includes bibliographical references. ISBN 0-444-82250-X I. Robotics--Congresses. 2. Inte111gent control systems-Congresses. I. OraeFe, Volker, 1938. II. Title. TJ210.3.I447 629.8'92--dc20
1994a 95-36630 CIP
ISBN: o 444 82250 x 9 i995 Elsevier Science B.V. All rights reserved. No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, without the prior written permission of the publisher, Elsevier Science B.V., Copyright & Permissions Department, P.O. Box 52i, iooo AM Amsterdam, The Netherlands. Special regulations for readers in the U.S.A. - This publication has been registered with the Copyright Clearance Center Inc. (CCC), 222 Rosewood Drive, Danvers, MA, oi923. Information can be obtained from the CCC about conditions under which photocopies of parts of this publication may be made in the U.S.A. All other copyright questions, including photocopying outside of the U.S.A., should be referred to the copyright owner, Elsevier Science BV, unless otherwise specified. No responsibility is assumed by the publisher for any injury and/or damage to persons or property as a matter of products liability, negligence or otherwise, or from any use or operation of any methods, products, instructions or ideas contained in the material herein. This book is printed on acid-free paper. Printed in The Netherlands.
PREFACE In September 1994 the seventh International Conference on Intelligent Robots and Systems- IROS '94-- was held in Neubiberg, a suburb of Munich, Germany, on the premises of the German Armed Forces University Munich. Of the 300 papers presented during the conference I have selected 48 ones which, in my subjective opinion, are particularly significant and characteristic for the present state of the technology of intelligent robots and systems. This book contains the selected papers in a revised and expanded form. Robotics and intelligent systems constitute a very wide and truly interdisciplinary field. The papers in this book have been grouped into the following categories: Sensing and Perception Learning and Planning Manipulation Telerobotics and Space Robotics Multiple Robots - Legged Locomotion - Mobile Robot Systems Robotics in Medicine
-
-
-
-
-
-
They cover, however, many additional fields, e.g. control, navigation, and simulation, to mention only a few. When preparing the conference, and this book, I found that most researchers in robotics are now apparently interested in some combination of learning, mobile robots, and robot vision. Consequently, most of the articles in this book relate to at least one of these fields. A book like this is always the result of many persons' efforts. Special thanks are due to Mr Michael LiJtzeler who helped with the compilation of the book by creating and maintaining various data bases, and to my secretary, Mrs Sigrun Haussmann, who among other things, helped some of the authors to improve the style and grammatical correctness of their texts. Thanks also to Ms Drs. Eefke Smit &Elsevier Science BV for competently and patiently managing all matters related to the publishing of the book.
Neubiberg, March 1995 Volker Graefe
This Page Intentionally Left Blank
vii
TABLE OF CONTENTS
Preface
SENSING AND PERCEPTION
A new High-performance Multisonar System for Fast Mobile Robot Applications U.D. Hanebeck and G. Schmidt Proper Selection of Sonar and Visual Sensors in Vehicle Detection and Avoidance N. Moghaddam Charkari, K. Ishii and H. Mori
15
Selective Refinement of 3-D Scene Description by Attentive Observation for Mobile Robot H. Takizawa, Y. Shirai and J. Miura
29
Active Vision Inspired by Mammalian Fixation Mechanism T. Yagi, N. Asano, S. Makita and Y. Uchikawa
39
Realtime Range Imaging Using An Adjustment-free Photo VLSI A. Yokoyama, K. Sato, T. Yoshigahara and S. Inokuchi
49
A Pedestrian Discrimination Method Based on Rhythm S. Yasutomi, H. Mori and S. Kotani
61
Visual Recognition of Obstacles on Roads U. Regensburger and V. Graefe
73
Visual Collision Avoidance by Segmentation I. Horswill
87
LEARNING AND PLANNING
Using Perceptions to Plan Incremental Adaptions A.J. Hendriks and D.M. Lyons Robot Task Programming by Human Demonstration: Mapping Human Grasps to Manipulator Grasps Sing Bing Kang and Katsushi Ikeuchi Robot Learning By Nonparametric Regression S. Schaal and C.G. Atkeson
101
119
137
~ 1 7 6
VIII
Behavioral Control in Mobile-Robot Navigation Using a Fuzzy Decision Making Approach H.R. Beom, K.C. Koh and H.S. Cho Learning Emergent Tasks for an Autonomous Mobile Robot D. Gachet, M.A. Salichs and L. Moreno Efficient Reinforcement Learning of Navigation Strategies in an Autonomous Robot J. del R. Milldn and C. Torras
155
169
185
A Lifelong Learning Perspective for Mobile Robot Control S. Thrun
201
A Multilevel Learning Approach to Mobile Robot Path Planning F. Wallner, M. Kaiser, H. Friedrich and R. Dillmann
215
A Self-Organizing Representation of Sensor Space fi~r Mobile Robot Navigation B.J.A. KrOse and M. Eecen
229
Path Planning and Guidance Techniques for an Autonomous Mobile Cleaning Robot C. H ~ w r and G. Schmidt
241
MANIPULATION
Grasp Strategies for a Dextrous Robotic Hand K. Woelfl and F. Pfeiffer
259
Motion Scheme for Dextrous Manipulation in the Intelligent Cooperative Manipulation System - ICMS M. Buss and H. Hashimoto
279
Designing a Behavior of a Mobile Robot Equipped with a Manipulator to Open and Pass through a Door K. Nagatani and S. Yuta
295
The Development of Re-usable Software Systems for Intelligent Autonomous Robots in Industrial and Space Applications E. Freund and J. Roflmann
311
TELEROBOTICS AND SPACE ROBOTICS
Toward Integrated Operator Interface for Advanced Teleoperation under Time-Delay A.K. B~jczy, P. Fiorini, W.S. Kim and P.S. Schenker
327
Active Understanding of Human Intention by a Robot through Monitoring of Human Behavior T. Sato, K Nishida, J. Ichikawa, K Hatamura and H. Mizoguchi
349
Human/machine Sharing Control for Telerobotic Systems T.-J. Tarn, N. Xi, C. Guo and A.K. Bejczy
373
Task-Directed Programming of Sensor-Based Robots B. Brunner, K. Arbter and G. Hirzinger
387
Telerobotics with Large Time Delays - the ROTEX Experience G. Hirzinger, K. Landzettel and Ch. Fagerer
401
Feature-Based Visual Servoing and its Application to Telerobotics G.D. Hager, G. Grunwald and K. Toyama
415
Practical Coordination Control Between Satellite Attitude and Manipulator Reaction Dynamics Based on Computed Momentum Concept K. Yoshida
431
A 5-Axis Mini Direct Drive Robot for Teleoperation and Biotechnology M. Moreyra, P.-H. Marbot, S. Venema and B. Hannaford
445
A Laboratory Test Bed for Space Robotics: The VES II S. Dubowsl,y, W. DurJ~)e, T. Corrigan, A. Kuklinski and U. MOiler
463
MULTIPLE ROBOTS
Hierarchical Prediction Model for Intelligent Communication in Multiple Robotic Systems K. Sekiyama and T. Fukuda
477
Proposal for Cooperative Robot "Gunryu" Composed of Autonomous Segments S. Hirose, T. Shirasu and E.F. Fukushima
491
Unified Control for Dynamic Cooperative Manipulation K Kosuge, H. Yoshida, T. Fukuda, M. Sakai, K. Kanitani and K. Hariki
501
LEGGED LOCOMOTION Comparative Study of Adaptive Controllers for a Pneumatic Driven Leg M. Guihard
513
An Efficient Forward Gait Control for a Quadruped Walking Robot D.J. Pack and A.C. Kak
533
The Six-Legged TUM Walking Robot H.-J. Weidemann and F. Pfeiffer Vision-Based Adaptive and Interactive Behaviors in Mechanical Animals Using the Remote-Brained Approach M. Inaba, S. Kagami, T. Ishikawa, F. Kanehiro, K. Takeda and H. Inoue
549
559
MOBILE ROBOT SYSTEMS
Autonomous Sonar Navigation in Indoor, Unknown and Unstructured Environments W.D. Rencken
577
Control and Localisation of an Autonomous Mobile Vehicle M. Adams, N. Tschichold-Garman, S. Vestli and D. yon FlOe
589
The Robotic Travel Aid "HITOMI" H. Mori, S. Kotani and N. Kiyohiro
607
Vision-based Naviation for an Office Messenger Robot T. Gomi, K. Ide and P. Maheral
619
Trajectory Tracking Control for Navigation of the inverse Pendulum Type Self-Contained Mobile Robot Y.-S. Ha and S. Yuta
637
An Outdoor Robots System for Autonomous Mobile All-purpose Plattbrm T. Okui and Y. Shinoda
657
Mechanism and Control of Propeller Type Wall-Climbing Robot A. Nishi and H. Miyagi
669
Automated Guided Vehicles in Japan T. Tsumura
679
ROBOTICS IN MEDICINE Robotics in Medicine P. Dario, E. Guglielmelli and B. Allotta Planning, Calibration and Collision-Avoidance for Image-Guided Radiosurgery A. Schweikard, M. Bodduluri, R. Tombropoulos and J.R. Adler
691
721
Intelligent Robots and Systems V. Graefe (Editor) 1995 Elsevier Science B.V.
A new High Performance Multisonar System for Fast Mobile Robot Applications Uwe D. Hanebeck and Giinther Schmidt Department for Automatic Control Engineering (LSR) Technische Universit~t Miinchen 80290 Miinchen, Germany A new type of actively sensing multisonar system has been developed for obstacle detection, localization and map updating in indoor environments. It comprises 24 individual intelligent transmitter and receiver elements arranged in a horizontal plane around a high-speed motion platform. The physical sensors are electronically configured to form various types of virtual sensor arrays for specific tasks and situations. A method for high-frequency firing compensates for the low speed of sound in air. Various schemes for intelligent evaluation of the array outputs provide highly accurate localization estimates. Experiments with the developed multisonar system in structured and cluttered environments demonstrate improved perception capabilities compared to state-of-the-art approaches. 1. I n t r o d u c t i o n
Ultrasonic sensor systems in mobile robot applications serve three major objectives: 9 detecting collisions with unexpected, possibly mobile obstacles, 9 localizing known environmental structures, 9 updating an internal map with previously unknown objects. Different approaches have been reported. One class of systems uses one rotating time-offlight sonar sensing device to obtain a 360 degree scan of the environment [4]. The width of the sonar beam causes a blurred image. Attempts to narrow the beam by phased arrays [2], [6] or horn constructions [3] have also been undertaken. Mechanical positioning makes these systems slow and inadequate for collecting data during fast motion. In another class of systems several (20-30) individual time-of-flight sensors surround a robot. Fixed firing schemes independent of mission and situation are employed [7], [13]. Undesired mutual influences of the sensors result in slow firing rates and make error correction algorithms necessary [1]. Crosstalk effects are eliminated by identifying and rejecting pulses received from other sensors. The fastest system known to the authors achieves a sampling interval of 60 ms for each sensor with a maximum range of 2.5 m [1]. The separate evaluation of sensor outputs results in an unsatisfactory angular resolution. This feature complicates many sensory tasks, like determining the vehicle's own position with respect to a landmark or the position of a moving obstacle. While the second class of systems tries to avoid crosstalk effects, another class takes advantage of these phenomena. Sparse sensor arrays are used and the characteristics of sound propagation are taken into account. This approach permits determination of the normal directions of walls [10], I12] or tracking
of moving obstacles as discussed in [8]. The need for mechanical aiming renders these systems too slow for the intended application. The sensor system discussed in this paper is used with an omnidirectional locomotion platform as shown in Figure 6. For this application, omnidirectional measurement capabilities are required. In addition, the system is expected to concurrently perform various sensory tasks around the mobile robot. For example, localizing of a wall is performed while reliably scanning the current travelling direction. Data has to be provided at a sufficiently high rate to guide a fast (2 m/s) mobile robot with steering commands generated every 10 ms. The proposed system which adopts useful aspects of today's sensor systems is discussed in detail in Section 2. A method to alleviate complicated cross-talk error correction schemes is presented. High-frequency firing schemes compensate for the low speed of sound in air. Furthermore, it is shown how the system is dynamically configured during operation. Section 3 presents algorithms for extracting several relevant environmental features with the proposed system. Section 4 introduces a current prototypical implementation. Experiments with the multisonar system mounted on a fast motion platform are described in Section 5. Compared to state-of-the-art systems the new approach gives improved perception capabilities and simplifies subsequent algorithms like obstacle detection and localization. The techniques presented in this paper apply to what is called conformal arrays in radar technology, i.e. sensor arrays which conform with a given vehicle geometry. 2. S y s t e m O v e r v i e w The proposed multisonar system is designed for operation in office or laboratory type indoor environments. Unknown objects like humans, chairs or trash cans must be detected in a wide surveillance area around the mobile robot even during fast motion. The estimate of the obstacle positions should, nevertheless, be accurate enough to avoid blocking of narrow passages. Section 2.1 details on a method that allows combined firing of many sensors while avoiding cross-talk errors. We assume that static structures such as walls, edges and doorframes are stored in a 2D map as natural landmarks. These a-priori known features are used for localization estimation. A fast-firing method that takes advantage of this prior knowledge is introduced in Section 2.2. The sampling rate is increased far beyond the limits usually imposed by the low speed of sound in air. To allow for several tasks to be performed concurrently, only the most appropriate subsets of sensors are used. This form of dynamic resource management is discussed in Section 2.3. 2.1. M e t h o d of V i r t u a l Point Source Moving with high speed in a cluttered hallway requires early detection of the closest unknown obstacles and the determination of their positions. Usually several sensors are used separately for this purpose. Disturbing crosstalk effects require elaborate error correction schemes. A more appropriate configuration for this application uses one single transmitter with the desired beam form and a receiving array configured of several receivers in order to evaluate incoming echos as reported in [8]. A typical surveillance area is shown in Figure 1 a). Since the objective is to use only transmitters located on the
Figure 1. a) Typical surveillance area. b) Combining physical transmitters to form a virtual transmitter. vehicle surface, this beam shape cannot be achieved with a single element. A virtual transmitter with the desired characteristics is obtained by appropriately phasing several adjacent transmitter elements with the main axis intersecting at the origin, Figure 1 b). For developing an adequate firing scheme, we assume a virtual circular wave which is starting from the origin. When the virtual wave front passes a physical transmitter, an ultrasonic pulse is fired. For M transmitters involved, an index vector I can be defined such that CIi < CIi+l, i = 1, 2, . . . , M -
1,
(1)
with Ci the distance from transmitter i to the origin. The time differences ti between firing transmitters Ii and/i+1 are given by CIi+~ - CIi ti =
V~
, i-
1, 2, . . . , M -
1,
(2)
with V~ the velocity of sound in air. This idea result,, in the following firing scheme: FORi=I,2
..... M-1
~
Fire S e n s o r I i Wait t~ Fire S e n s o r I M
! i
Let Tj denote the time difference between firing of the first sensor 11 and detection of an echo at receiver j, j = 1, 2, . . . , N, where N is the number of receivers. The distances Rj from virtual transmitter to the object and back to receiver j are then given by n j : T j . Ys -Jr-CI1, j • 1, 2, . . . , N . (3)
Figure 2. Fast firing
combining windowing and interlacing.
The number M of neighbouring transmitters employed depends on the desired beam width. Since the physical sensors' main axis intersect at the origin, superposition of the pulses r o u g h l y a p p r o x i m a t e s one pulse of a single centrally located virtual transmitter. Since we assume only one single pulse stemming from the origin, no decision on which physical transmitter produced the pulse detected by an individual receiver is required. Rotation of the beam to another focus of attention is achieved by electronically shifting the sensor arrangement. 2.2. Fast F i r i n g It is common engineering practice to focus on a certain interval around a nominal or predicted measurement value. For example, prior knowledge from a map could be used in conjunction with an estimated position to a-posteriori discard inconsistent measurements. However, we propose to go one step further and use prior knowledge at an earlier stage, i.e. during the data gathering process. Echos are received only in certain precalculated time-slots. This scheme is also applicable to reflectors not represented in the map. In this case, for windowing the distance predicted from previous data is used. Missing an obstacle occluding the expected reflector is impossible because the firing scheme is changed when the predicted time-slot is empty for several times. Due to the low speed of sound in air, the elapsed time between transmitting a pulse and receiving the echo is approximately 3 ms per meter travelled. With longer distances, interlaced transmitting and receiving becomes more and more appropriate. Thus, windowing and interlacing are combined in a highly effective manner. After initializing with a standard firing method, i.e. determining the expected echo return time T~, we receive the echo from the previous pulse after the next pulse has been transmitted. A window of length Tv + TN, with Tv = TN, is centered around the expected time of arrival. The most straightforward scheme for one sensor is shown in Figure 2. The pause time Tpkus~ is given by
Tpkus~ - T~ - 2. Tdead- 2 . Tv -- T N .
(4)
The expected echo return time Tk is updated with the knowledge from every new sample. Td~d corresponds to the dead zone of the transmitter/receiver-combination. Tv = TN = 0.5 ms and Td~ad = 1.5 ms results in the firing rate 1/T~ with
T~ = T E - T d ~ d - Tv
~
d - - " 6 m s - 2 ms. m
(5)
N "1-
)
'~--~ 4 5 5 .i....,
250 C U_
loo i
45!
. . . .
t + _ _ •
~
L
i ..........
!....
0.7
i
1
*
2
>
4 Distance d / m
Figure 3. Firing rate for fast firing of first order as a function of distance.
The dependency of the firing rate on distance d is shown in Figure 3. The order of the fast firing scheme is defined according to the number of pulses interlaced in the pause times. With simple first order fast firing, sampling frequencies above 100 Hz are achieved for distances of up to 2 m. Thanks to windowing, the scheme is invulnerable to noise and secondary reflections. It is applicable for d > (2. Tdead -4- 2" Tv + TN)" V~/2, in this case approximately 70 cm. If the tracking process breaks down because the object is moving too fast, the system restarts with initialization measurements. Fast firing of higher order is achieved by interlacing more pulses during the pause times.
2.3. Dynamic Reconfiguration The 24 transmitter and receiver elements arranged on a horizontal plane around the fast mobile robot constitute a homogeneous multisonar system. They are, however, combined to a variety of virtual sensing arrays depending on tasks and situations. This type of operation is similar to a concept described in [5]. Since virtual sensors are aimed electronically, there is no need for moving parts. All virtual sensor arrays consist of N arbitrarily distributed receiving elements. W h e n the angular region of interest is wide, a virtual point source combining several physical transmitters is used to illuminate the scene. On the other hand, when prior knowledge on reflector positions is available, the angular region of interest is narrow and one physical transmitter may suffice. 3. O b j e c t L o c a l i z a t i o n Based on the concept of dynamic reconfiguration, virtual sensors are formed from several physical transmitting and receiving elements. In general, we assume a virtual point source located at the vehicle's origin and an array of N arbitrarily distributed receivers. For the sake of simplicity, only two types of reflecting surface primitives are considered here: i) line reflector ~ walls, objects with small curvature, ii) point reflector --~ edges, doorframes. Both reflectors represent classes of real targets within the 2D sensor plane.
Figure 4. Measurement model for a) line type reflector b) point type reflector.
Given a reflector hypothesis and N measured distances Ri, i -- 1, 2 , . . . , N, an advanced object localization scheme should 9 allow for arbitrary receiver distribution, 9 provide position estimates with high angular accuracy, 9 discard inconsistent measurements, 9 quantify the estimates' uncertainty, 9 use redundancy: N receivers -~ N equations, 9 be valid for near/far-field reflectors, 9 be simple, linear, and non-iterative. The derivation of schemes that fulfill the above set of requirements is given next. 3.1. Line R e f l e c t o r Assuming specular reflection and introducing the mirror image of the transmitter, the following relationship can be obtained from Figure 4 a). R 2 - (C[ - 2x) z 4 - ( C ~ - 2y) 2 , i - 1, 2 , . . . , i
(6)
with c; = c,
= e,
(7)
With r 2 = x 2 + y2 the set of Eqs. (6) is rewritten in vector/matrix form
z+4r21-H'(X)+e_ y
(8)
with
c~ - n~
c~ ci'
c~-n~ z
,H
-
4
1
C~ C~
-
.
c ~ - n~
.
,
1
-
c~ c~
.
(o)
,
i
where e denotes an error vector. In a first step, x, y and r 2 are assumed to be independent variables. Then a weighted least-squares estimate of 3, 9 in terms of 2 2 is given by 9
- ~+4
~
(10)
with a
-
a.z,
G
-
(HTE-IH)
(11) (12)
~-G.1 -1. H T. E -1.
E denotes the weighting matrix. With the relationship ~
,
(13)
the following quadratic equation for 2 2 is obtained 16flTz~ 4 + ( 8 a T f l - 1)~z + a T a -
0.
(14)
Substitution of 2 2 back into Eq. (10) gives the desired solution. The optimal choice for E is derived next by adopting a set-theoretic point of view. The errors Ai in the measurement distances R{, i - 1, 2 , . . . , N, are assumed to be located inside an ellipsoid defined by a symmetric, positive definite matrix SA, i.e. (A,, A2, . . . , AN) S~' (A1, A2, . . . , AN) T
<__1.
(15)
According to the appendix, the error e will then be constraint by an ellipsoid defined by (16)
E - D. SA. D T
with D ~ {I-1
8(x,0) G}. - 8(&,9)/~ - 1
RD,
RD --
--2 diag(/~,,/~2, .-
"'
/~N).
(17)
The noise free value of 9 is denoted as ~ and I is the identity matrix. The distance of the actual error from the surface of the error ellipsoid in Eq. (16) is given by d- 1-
[ z + 4r2_1- H ( ~ ~) ] f
E -1 [ z + 4r21_- H ( 3 ~) 1
.
(18)
If d is negative, the solution is not valid within the constraints given in Eq. (15) and may be discarded. Otherwise (0 <_ d < 1) the solution is valid and contained in an ellipsoid given by - d. (HTE-1H)
-1 ,
(19)
which is centered at 5, ~. Since the matrix D contains the true distances/~i and the true position (~,~), some approximations are required to arrive at a feasible solution for the weighting matrix E. The measured distances R i are used to approximate the unknown true distances /~i. In a first step, R D 9S A 9R D is used in place of E to solve for a, and then ~2. Next, the resulting ~2 is used as an approximation of the true ~2 in Eq. (17) to calculate D. Although the described process could be repeated to obtain even better solutions, iteration does not lead to significant improvements in practice. Special Case: Linear Array For a linear arrangement of receivers, a simpler solution may be obtained and may replace the above formulas. When the transmitter is also lined up with the receivers, this solution must be used to avoid singularity of G in Eq. (12). Without loss of generality the linear array is assumed to be arranged along the x-axis. Then Eq. (6) simplifies to C~ - R ~ -
4 C[ x + 4 y,
i -
l, 2,...,
N,
y -
C V y - r 2.
(20)
In matrix form, this reads as C~2 -
( )x
z-H
+e
z-
,
R~
C~- R~ ,
H-4
C~v- R~v
C~
1
~
o
C~1 C}
(21)
9
1
With ~2 _ 52 + ~2 an estimate ~ is found from ~2 _ CV?~+ ~ + 52 _ 0.
(22)
The error e is then contained in an ellipsoid given by E -
RD.
S A " RTD .
(23)
A straightforward first-order error propagation analysis provides A~
~ F.
A,)
, F-
(,
2~ cy-2~
0)
1 cy-26
9
(24)
and it follows that -
d. F.
( H T E -~
H) -1.
(25)
F T .
In the special case, the solution is obtained in one step. 3.2. P o i n t R e f l e c t o r From the measurement model of Figure 4 b) follows that
( n ~ - ~)~
__
(c~x - 9)2 + ( c ~ -
y)~, i - ~, 2 , . . . ,
g.
(26)
This can be written as
z+2rR-H
(x)
n,
+e
R-
n~ .
c i ~ c~'
,H-2
c~c~ . .
.
(27)
Again, x, y and r are first assumed to be independent variables. The estimate ~, ~ in terms of ~ is then given by
with ~
=
G -
G . z , ~ = G . R
(29)
(H TE -1H)-1. HT " E_I.
(30)
With Eq. (13), a solution for ~ is obtained from (4/3T/3 -- 1)~ 2 § 4aTl3i" + aTa = 0.
(31)
Using similar arguments as for the wall reflector, the error e will be located within an ellipsoid defined by
E = D. SA. D T
(32)
with D
~
(I -
h2(~,~)~_ ~
a]
+
(33)
Special Case: L i n e a r A r r a y When the receivers a n d the transmitter are lined up, G in Eq. (30) becomes singular. Without loss of generality, the array is again assumed to stretch along the x-axis. A solution for ~, ~ is then obtained from
c~ z-H(X] +e'H-2v
Cf.
-R1 -R2 (34)
c~ --RN and
(35) in a very similar fashion as for the line reflector. 3.3. D i s c r i m i n a t i n g R e f l e c t o r T y p e s Besides localizing objects for which the reflector type is a-priori known, it is often useful to identify reflector types based on actual measurement data. The additional surface information simplifies the determination of line endpoints and outlier elimination for subsequent segmentation algorithms. The hypothesis "line reflector" is accepted, if the line condition Eq. (18) is true. In analogy, the hypothesis "point reflector" is accepted, if the respective condition is true. No decision is possible, if both conditions are true. Both hypothesis are rejected, if no condition is true. For the prototype implementation considered, the measurement error is in the order of magnitude to the differences between the distances for the two reflector hypothesis. Numerous misinterpretations are the result. The amplitude of the returned signals is very well suited to support decisions in conflicting cases. Unfortunately, amplitude information is not yet available in the current prototype.
10
Figure 5. Overview of the ultrasonic sensing hardware.
4. I m p l e m e n t a t i o n Issues The prototype implementation comprises 24 piezo-ceramic emitter and receiver elements operating at 40 kHz. They are fixed in a horizontal plane around the mobile platform. The maximum range is about 4 m for most reflector types. The minimum range the dead zone is approximately 0.25 m. Each sensor is connected to an analog signal processing unit which generates pulses and detects the returned echos. A tandemarranged digital signal processing unit determines the elapsed time between transmitting a pulse and the returning echos. The evaluation units are fully decoupled. The digital electronic is mapped to the memory of a local transputer network consisting of four T805 transputers, Figure 5. This network takes care of configuration, the sensor firing, data gathering, and prefiltering. For experimental evaluation of the proposed multisonar system, the omnidirectional service robot ROMAN (ROving MANipulator) has been used. It is a full scale mobile robot, Figure 6, adapted to indoor requirements: width 0.63 m x depth 0.63 m x height 1.6 m. Three independently steerable wheel systems provide excellent maneuverability. Wheel diameters of 0.2 m allow travel across rough surfaces like carpeted floor. The maximum speed of 2 m/s has been chosen to correspond with human walking speeds and wheel-chair velocities. Steering commands are emitted every 10 ms based on actual position information. A lightweight 7 DOF manipulator with a maximum range of 80 cm is mounted in one corner of the platform. Besides the ultrasonic sensors, ROMAN employs a laser navigation system, a wheel based dead-reckoning system and a low-cost gyroscope. The local sonar transputer network is part of a larger distributed network which takes care of the sensor system. 5.
Applications
So far, a number of tools for enhancing the perception capabilities of sonar array sensors have been introduced. Their application depends on how structured the environment is, whether the objects are static or moving, and the degree of prior knowledge. Table 1 lists the rules that allow the system to automatically select sensing strategies depending on the current task and situation.
ll
Table 1 Rules for automatic selection of sensing methods. # of transmitters per array
# of concurrent arrays
# of receivers i per array i
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
data rate per array
fast firing . . . . .
_
virtual point source __+
.
.
.
.
.
.
.
.
I
j
Localization for navigation
1
typ. 2-3
cf. Fig. 3
5
X
i
i
i
I
Obstacle detection
!
6-10
1 .
updating
2-3
.
.
.
.
.
.
.
.
t
1
.
.
.
.
5
.
.
X
25 ms
6-10 .
!
25 ms i
5.1. E x p e r i m e n t s in Cluttered Environment In a cluttered environment, the multisonar system is used to detect and localize obstacles. Absolute localization of the vehicle is performed by the laser navigation system with respect to known artificial landmarks. To demonstrate the performance of the proposed system, the following experiment has been conducted: While following a given course, the robot encounters two obstacles of different size. A virtual point source consisting of 8 physical transmitters points to the travelling direction of the vehicle. 6 receiver elements are employed to detect the incoming echoes. For simplicity, it is assumed, that the reflecting surfaces may be approximated by lines within their 2D projection. Every 25 ms, 6 distances Ri corresponding to the first echoes are measured. The procedure from Section 3.1 is used in an order-recursive manner to combine these measurements. Inconsistent combinations are discarded. Valid combinations are kept and provide the reflector position in the vehicle frame. Via the known transformation vehicle ~ world, the object positions are converted to the world frame and plotted in Figure 7 without further processing. Only a few outliers are generated. For the small obstacle, however, the line reflector assumption leads to errors in the localization estimate.
5.2. E x p e r i m e n t s in Structured Environment In a structured environment, a few a-priori known natural landmarks such as walls and edges are tracked and used for updating the vehicle's position and orientation. The number of employed virtual sensors is equal to the number of simultaneously visible landmarks, typically 2 to 3. Every sensor array employs one physical transmitter and up to 5 receiving elements. At any instant of time, the transmitter pointing closest to the predicted landmark is chosen. Fast firing is used to exploit a-priori knowledge on the reflectors' positions. This leads to very high update rates. For demonstration purposes, the vehicle moved along a wall. Based on the measured distances, the wall position is calculated with respect to the vehicle frame with the scheme discussed in Section 3.1. Since the wall position is known, the vehicle position in world coordinates may be obtained. The resulting data is compared with localization data from the laser navigation system, Figure 8. A satisfactory correspondence of data from both measurement systems is achieved. The raw data is used subsequently to calculate the actual position estimate. The previous estimate is propagated by means of dead-reckoning
12
Figure 6. Mobile manipulator ROMAN.
and gyroscope and fused with localization data from every visible landmark. The respective uncertainties are again modeled based on a set-theoretic approach. Statistical modeling would not capture the systematic errors caused by coordinating the three wheels. According to this point of view, the fusing process results in uncertainty set intersection [11]. Going into more detail would take us too far afield.
6. C o n c l u s i o n s A new high performance multisonar system for application with fast (2 m/s) omnidirectional mobile robots has been developed. 24 transmitter and receiver elements are arranged in a horizontal plane around a mobile robot and constitute a homogeneous construction set for forming of transmitting and receiving arrays. Depending on mission and situation the individual elements are combined to form several heterogeneous virtual sensing arrays, each designed for carrying out a given task. Virtual sensors are aimed electronically, i.e. there is no need for moving parts. High frequency firing schemes exploit a-priori information about the current environment to purposefully collect data in a surveillance area of 8.6 m in diameter. This makes the proposed system superior to those with fixed data gathering schemes. The results of experiments underline the system's improved perception capabilities in detecting obstacles during motion. Other tests demonstrate sonar localization results that are in good accordance with data from a laser navigation system.
13
Figure 7. Obstacle detection during motion.
Figure 8. Sonar localization of the vehicle compared with a laser navigation system.
REFERENCES
1. Borenstein, J. and Koren, Y., "Noise rejection for Ultrasonic sensors in Mobile Robot Applications". Proceedings of the 1992 IEEE International Conference on Robotics and Automation, Nice, France, S. 1727-1732, May 1992. 2. Burks et al., "Autonomous Navigation, Exploration, and Recognition Using the HERMIES IIB Robot". IEEE Ezpert, Vol. 2, No. 4, S. 18-27, 1987. 3. Crowley, J. L., "Dynamic world modelling for an intelligent mobile robot using a rotating ultrasonic ranging device". Proceedings of the 1985 IEEE International Conference on Robotics and Automation, St. Louis, MO, 1985. 4. Drumheller, M., "Mobile Robot Localization Using Sonar". IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 9, No. 2, S. 325-332, 1987. 5. Fukuda, T. and Nakagawa, S., "Dynamically reconfigurable robotic systems". Proceedings of the 1988 IEEE International Conference on Robotics and Automation, Philadelphia, PA, S. 1581-1586, 1988. 6. Kampmann, P.; Freyberger, F.; Karl, G.; and Schmidt, G., "Real-Time Knowledge Acquisition and Control of an Experimental Autonomous Vehicle". Intelligent Autonomous Systems, S. 294-307, 1986.
14
.
10.
11.
12.
13.
Kortenkamp et al., "Integrated Mobile-Robot Design, Winning the AAAI '92 Robot Competition". IEEE Expert, Vol. 8, No. 4, S. 61-73, 1993. Kuc, R. and Barshan, B., "Bat-Like Sonar for Guiding Mobile Robots". IEEE Control Systems Magazine, Vol. 12, No. 4, S. 4-12, 1992. Kuc, R. and Siegel, M. W., "Physically Based Simulation Model for Acoustic Sensor Robot Navigation". IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 9, No. 6, S. 766-778, 1987. Nagashima, Y. and Yuta, S., "Ultrasonic sensing for a mobile robot to recognize an environment - Measuring the normal direction of walls -". Proceedings of the 1992 I E E E / R S J International Conference on Intelligent Robots and Systems, Raleigh, NC, S. 805-812, July 7-10, 1992. Sabater, A. and Thomas, F., "Set Membership Approach to the Propagation of Uncertain Geometric Information". Proceedings of the 1991 IEEE International Conference on Robotics and Automation, Sacramento, CA, S. 2718-2723, 1991. Sabatini, A. M., "Active Hearing for External Imaging Based on a Ultrasonic Transducer Array". Proceedings of the 1992 IEEE//RSJ International Conference on Intelligent Robots and Systems, Raleigh, NC, S. 829-836, July 7-10, 1992. Walter, S. A., "The Sonar Ring: Obstacle Detection for a Mobile Robot". Proc. IEEE International Conference on Robotics and Automation, Rayleigh, NC, March 31. - April 3., S. 1574-1579, 1987.
Appendix The error e in Eq. (8) may be written as ~ - s + ZXz + 4 ( ~ ~ + z x ~ ) l -
H
~
(36)
- ZXz + 4 z X ~ , ! .
From Eq. (14) follows that 16Z~Z(~ ~ + ~x~) ~ + ( s ( a + ~x.)~Z -
~)(~ +
~ x ~ ) + (a + ~x.)~(a + A . ) = 0.
(37)
If squared error terms are neglected, error At2 is given by 2(4~3 ~ +
a~)
Ar~ ~ - 8(4~2/~T~ + 5 T 3 ) _ 1
a.
= _ AO~
"
(3s)
From Eq. (11) it is concluded that An = G" Az and we get from Eq. (9)
Az ,-~ RD "
A1 A2 .
9
(39)
Intelligent Robots and Systems V. Graefe (Editor) 9 1995 Elsevier Science B.V. All rights reserved.
15
P r o p e r S e l e c t i o n of S o n a r a n d V i s u a l S e n s o r s in V e h i c l e D e t e c t i o n a n d A v o i d a n c e N. Moghaddam Charkari, Kazmmri Ishii and Hideo Xlori * Department of Electrical Engineering and Computer Science "~unanashi University
A general 1)aradigm for vehicle detection and avoidance by proper selection of visual and sonar sensors is 1)resented. The most appropriate sensor for detection and navigation of the robot is selected in accordance with the sensor fimctions and limitations. This strateg T is activated when one sensor can't obtain enough information for navigation of the robot because of sensor limitations. Five types of actions based on stereotyped motion are used for th(, navigation system: Moving Along Wall, Moving Along, Moving Toward, Tracking Motion and Danger Estimation. The first is sonar based and the rest are vision based a('tions. Visual detection of the vehicle is 1)ase(l on the locomotion strateg3' "Sign PattcrT~ based Stereotyped MotioTi'. Th(" sign pattern of the vehicle is the shadow underneath the vehMe, which is dark('r than any other parts of the paved roa(l. The rol)ot detects the sign pattern of a vehMe fi'om a (listance of 30 m and extracts its front shape 1)y video rate pro('essing.
1. Introduction One of the most important tasks for the navigation of mol)ile robots in outdoor envirom nents is the ability to detect and avoid moving or I)ark('(l vehicles. In this paper, sonar sensors are used in combination with vision to compensate for some of the difficulties and limitations encountered in the machine processing of visual information. The proper use of multiple sensors is becoming an in(,reasingly important area of research and appli('ation to improve the capabilities of intelligent machines, because of the advantages and the limitations of each type of sensor, most mobile robots use a combination of different sensors to perform some tasks. Each of the sensors in a defined task may be operated synchronously or asynchronously with the other sensors, depending on the Selection Process. Selecting the most appropriate sensor, or group of sensors, during the navigation of the mobile robot is also perform('d by the Selection Process. The strategy of selection dep('nds on the limitations of the s('nsors. For instance, a visual sensor observes an entire scene and can be used for dete('tion an(l re('ognition of objects that are distant, while tactile and sonar sensors are used to ol)tain inibrmation about a few objects that are nearl)v. *This work is parti~dly supported l)y Grmlt-in-Aid for Sci(,ntifi(" Research from the .Ministry of Education, No. 03650343 and fr(nn tile Okawa Institute of Information and Tel(wommunication No. 91-32.
15 Two (liffcr,'nt al)l)roachcs to the s('nsor Sclcctiou Process are Preselectiug during initialization and On.line selection in resl)OnSC to changing (,nvironment or syst(un conditions. Gaskell and Prol)crt [1] discussed the use of De,.cisiou Theoretic PTi'aciplcs for sensor management in multiscnsor systems. Their fi'amcwork i)r(~vidcs a way of rcl)rcsc:lting sensory data and choosing actions under uncertainty. Luo and Kay [2] surveyed a variety of recent al)l)roa('hes to the pro])lcm of multiscnsory information. G('neral multiscnsor fitsion m('tho(ls, sensor selection strategy, an(l some other apl)roachcs and applications are discussc(l in their report. In the NAVLAB paper [3] sonar, a lapser rangcfindcr and a color vision system are used for object detection as well as navigation in outdoor environment. The design of the architecture can support parallel processing and th(, development of multiscnsor integration has been the major goal of this research. h:imoto and "~]lta [4] dcvclol)cd a method based on the fusioll of internal sensors and ultrasonic sensors, ms external sensors, using an extended Kahnan filter for the navigation of the YAMABICO rol)ot. Information on the 1)lamlcd path and environmental data are the prior knowledge of the robot. Sonar, a laser rangcfin(lcr and two color vi(h'o camera.s have been Cml)loycd for roadfollowing and ol)jcct detection appli('ations on road environments in ALV [5]. Color video information is us('(l to locate the road, as the las('r rangcfindcr may siml)ly be confllscd if there is litth" diffcrcncc between the road au(l th(, surroundings. On the other hand, la.,~cr range information is used to ol)tain an accurate (lcscril)tion of the geometrical features of obstacles on the road. Visual ol)jcct detection is one of the main subj(,(,ts in robotic studies. Among such objects, vehicles are the :::()st troublesome ones in out(loor cnviromncnts. Several methods for vision 1)ascd vehicle detection have 1)con prol)oscd l)y researchers. A dynamic 2-D model has been iml)lcmcntc(l on VaMotLs [6]. This model is l)ascd on the notion that the world is structured, rather than monolithic, and consists of individual objects. It allows the determination of the relative Sl)ced and distance in real time and may guide appropriate maneuvers if necessary. In this method, the right and left. sides and the bottom of a car arc detected horizonta.lly and vertically by a local edge operator. Symmetry is another method which provides a useful feature since vehicles' rears are generally contour-symmetric and spatially gray-level-symmetric about a vertical centerline [7,8]. It means that an appropriately shifted mirror image of a vehicle overlaps itself. However, the error rate may increase when the mctho(1 is performed in a highly structured scene.
In our earlier article [9] we discussed a method 1)ascd on "Sign Pattern based Stereotyped Motiou" for vehicle detection. There the robot (lots not need to fully understand the obstacle as it reconstructs the 3-D shal)e of the obstacle in its memory. The robot only needs to obtain signs to such a degree that they inform it of the existence of an object in its way. The sign wa~s referred to as "sign 1)attcrn" in the earlier work. We found shadow underneath a vehicle to 1)c a rclial)lc sign 1)att('rn of the vehicle [10,11]. \Tchicle detection ba.,~ed on sign 1)attern is one of the lowcr-lcv('l COml)ctcn('ics of the rol)ot's information set of the cnviromncnt. Therefore, it does not nc('d highly computational image processing. Moreover, it cnal)les the s.x'stcm to overcome the tra.cking process, i.e. the ability to keep a vehicle in track l)y continuously sensing its sign 1)attcrn in real time.
17
Figure 1. The structure of HARUNOBU-6
In section 2, the structure of the HARUNOBU-6 robot and its explained. A simple reflection model of the road surface is discussed developments of the visual vehicle detection module can 1)e found in paradigm of proper sensor selection is discussed in section 5. Finally, the experimental results and conclusion respectively.
2.
sensory system are in section 3. Recent se(-tion 4. A general sections 6 and 7 are
S t r u c t u r e of the R o b o t
The work on developing the compact rol)ot HARUNOBU-6 (54 cm x 84 cm x 84 cm) has been undertaken in our laboratory since 1992. The structure of the robot is shown in Figure 1. The undercarriage is a motor-tricycle developed by Suzuki Co. Ltd.
2.1. Sensor S y s t e m Two types of sensors have been installed on the rol)ot: internal and external sensors. The internal sensors include encoders, a magnetic directional sensor and a gyroscope. The external sensors are a video camera and six sonar sensors. Sonar sensors are attached to the right, left and front of the robot. Two sonar sensors which are attached on the front of the robot are used for obstacle detection. The remaining sensors are used for moving along an object. Data from sonar sensors is sampled every 50 ms while the frequency of each sensor is about 40 kHz. The video camera is set at a height of 84 cm with a tilt angle of 12 ~ A VME image memory system (CPU 68040, 25 MHz) is used for ilnage processing and navigation. The communication between the image processing system and the vehicle motion control systern is performed through an RS232C connection. Data from internal and sonar sensors are fed to a UP P (universal pulse processor) for flu'ther processing. The UP P contains a 10-bit A / D converter, 1 kB RAM and a pulse modulator with counter. The control of the UPP is done by a single board control 1)ro('cssor (CPU V25, 8 MHz).
18 !
urce I Y
L _
(o
o•'4000
I
o Io
/'
//7
~ i~L ~' r /-/ J -- ~ ' ~ ' - :
vJow
|
$
E
C
un
.
oo E
D i f f u s e Light
_ . . I: ~
...
,~4,
: , 'IX
.:
9
3000
I """"'=,=
I
5
'
I~o
'
3~o angle
Figure 2. Schematic of specular and diffilsed light. Incident angle is Oi degrees in the x-y 1)lane. View position is defined by
(0~, 0,.).
~ r
Figure 3. Image intensity curve of the reflected light on the a.sphalt road when view position 0,. changes from 0 to 360 ~ and Oi =70 ~ and ~ = 60 ~
3. N o t e s o n t h e T e r m i n o l o g y
Tile terminologT employed in this stu(ly to describe the reflectance geometry and tile reflectance model is illustrated inFigure 2. A surfa~'e of an object is illuminated by the sun light I~,, and the sky light I~ky. The light s o u r c e is assumed to lie in the x-y plane with the zenith angle 0i. The light is reflected in the ((r , 4),.) direction, where 8~ denotes the angle between the normal vector of the surface and the reflected.light which is seen from the viewpoint of the video camera. ~ is the reflected light angle between the plane of I~,, and the normal vector. When the view position lies in the reflected direction (0i = 8,.), then the luminance of a surface is mostly evaluated by the specular component of the light source. Furthermore, if I,,,, becomes ahnost parMlel to the surface, the surface behaves like a mirror and all the light is reflected in the ~)i = 0~ direction. The reflected light is called "specular light" and represented by R,~. The specular component ha~s a spectrMenergy distribution close to that of the incident light. The directional characteristic of R~pr is called the "specular lobe" and is expressed by R~
= L,,,, " s i ,
e,. . cos4(t~i - 0 , ) . cos40,..
(1)
When I.,,,, is nearly perpendicular to the surface the light penetrates into the substance of the object and is partially scattered in many layers. The scattered light is called "diffused light" and is represented by RdL,. The (liffused light of a surface is given by Rd/s = P" h~,, " cos(Oi - 0r)(1 -- kCO.S" 20,.)/(1 -t-/,:),
(2)
where p is the reflectance factor of the sllrfa(,e and k denotes the fraction of diffuse light. Equations (1) and (2) show that surface reflectance not only depends on the material properties, but also on the viewing geometry. Figure 3 shows the image intensity curve of the reflected light on the road surface in one of our experiments. 0i and 0r are assumed to be almost equal (0i = 70 ~ , 0,. = 60 ~ while viewing position 0~ varies from 0 to 360 ~
19 Tal) le 1 Luminance of road surface on a clear day Clear weather Road surface type In specular light In diffused light Shadow of vehicle Underneath vehicle
Luminance (cd/m 2) Dry surface Wet surface 3600 to 4800 2020 to 3000 2600 to 3300 850 to 2200 400 to 900 275 to 500 100 to 200 60 to 150
Table 2 Luminance of road surface on a cloudy day Cloudy weather Road surface tyl)e Open parts Underneath vehMe
Lmninance (cd/m2) Dr3' surface Wet sm'fa,ce 2610 to 3500 2000 to 3000 105 to 140 80 to 100
On tile other hand, tile reflected light of the sun R.,,,,, is generally expressed by
R:,,, = R~p~ + Rd/.~.
(3)
The. total reflected light R, is given 1)3: R -
a l 9R , , , , + 0'.2. R , k y + c~a. R~,,v,
(4)
where at, (~'2and a3 are transparency factors equal to or less than 1.0. From the viewpoint of optical science, the sunny part of the asphalt road is illuminated by tile sun and the sky light (al - a2 = aa = 1) while the sha.d('d part is only illuminated by the blue sky light (a'l ~ 0, a2 = 1, aa - 1). The area underneath a vehicle is illuminated through the gap between the vehicle and the ground l)y the faint environment light R,,,,, (at - ~2 = 0, oa - 1). Equation 4 shows that although the luminance of the shadow underneath and the other shaded parts on the road apl)ear similar to the human visual system, this is an illusion of brightness constancy. To determine 1)recisely the above differences, the luminance of sunny regions, shadows, and the areas underneath a vehicle on dr3" and wet road surfa.ces were measured. Tables 1 and 2 show the result of the experiments on a clear day and during cloudy weather conditions respectively. It was found that the area ml(lerneath a vehicle on a ('lear day is markedly dm'ker than the shaded parts and wet patches in luminance. Similarly, tile area. underneath is nmch darker than the other open parts of the asl)halt road on a cloudy day when there is no shadow. Moreover, the luminance of the shadow underneath shows little change while those of sunny and shaded parts have no consistency because of the different sources of illunfinating light. Finally, the shadow underneath is not responsive to change in the viewing position and weather conditions.
20
Figure 4. Flow diagram of vehicle detection and tracking process
4. Vehicle D e t e c t i o n Algorithm Visual vehicle detection and tracking is (lecoml)osed into three layers. In the lowest layer the candidate vehicles are detected l)y the Selection sul)-module. In the middle layer, the pseudo-objects which arc wrongly selectte(l as vehicles are identified. In this l~'er, some additional features of vehicles are investigated by the Confirmation sul)-module. In the third layer, the type of motion is investigated and the candidate vehicles are tracked. Moreover, the trajectory of vehicles in motion and the motion direction, which are important in danger estimation, are found during the tracking process. The entire system structure is composed of the following sub-modules: Window Set-up, Searching Underneath, Confirmation, Motion Estimator, Danger Estimation, and Tracking and is shown in Figure 4. In the following the operation of ea~'h sub-module is discussed in brief. (1) W i n d o w Set-up: A fi'ame is digitized by its intensity and the right and left lane marks are detected by the line-fitting method. Then a thin window Wt, with its length equal to the road width, is set up at a position corresponding to a distance of 30m ahea~l of the robot on the image plane. (3) Searching U n d e r n e a t h : The vertical projection is performed on the window rWt. The candidate pixels are extracted and linke(l into the line segments by comparing the projection with the threshold value Trl. Then, the short line segments are eliminated and the collinear line segments are merged. The result is a new line segment L, = (~,, ui, ~, ui)T representing the a.rea underneath the vehicle.
21
Figure 5. Ilesult of vertical projection on a road partly covered by shadows. The area underneath a vehicle is clearly distinguished from other parts of the road surface when it enters the search window.
Figure 5 shows ml example of the vertical projection when the area underneath a vehicle enters the search window on a. road partly covered by shadows. As can be seen, the area underneath is clearly distinguished from other parts of the asphalt road by the intensity vahle. (4) Onc0ming/Parking Detection: Physiological studies suggest that the human visual system may use instantaneous motion information COmlmted fl'om two successive flames. In our work, once two feature points of t lw area underneath the vehicle ((~,, I~,)T and ((~,ui) T with length of Li are extracted in the search window 11,'1, then the next window II.) is set up at (ui + 7~e) as the next image. (ui + ~ite) indicates the shift of the search window on the 2-D image. Then, the corresl)onden('e of these points in successive images is determined. If the correspondence is estalJlished within an interval of time tl th('n the detected vehicle in lI'l is identified as an oncoming vehicle; otherwise it is a.s a stationary w'hicle. Since moving vehicles have a smooth and continuous and not a. transverse motion, we have emplyed a time interval tl which indicates the teml)oral 1)roperty of the area underneath when it is in motion. ( 5 ) C o n f i r m a t i o n : Moving vehicle detection based on two successive fl'anws may include some errors caused by distortion, random noise, 1)lurring, etc. The uncertainty can be considerably reduced by the following assumptions: (5.1) Number of Line Segments: The presence of at most two line segments in the vertical projection. When these are two line segments, one line segment corresponds to the shadow underneath all oncoming vehicle a~ld the other to that of a departing vehicle. (5.2) Length of the Line Segment: The length of the line segment should be greater
22 than the thresholcl value/Ji defined as tile metal value of vehicle width on the 2-D image. The threshold value increases with decrease in the depth of view. (5.3) Vertical Edge O p e r a t i o n : Vertical edge operation is performed on the t)eginning and the points of the line segment to detect the left and right sides of the vehicle. Confidence is defined to find the accuracy of measurement. It is represented 1)y the square root of the inverse of the vaa'iance of the candidate edge points. The smaller the variance, the higher the confidence of the measurement. (5.4) Vehicle Side S y m m e t r y O p e r a t i o n : The cletected eclges on tile left. and right windows belong to tile vehicle sides if they are symmetric about the vertical centerline defined by the line segment L,, = (g~,,,~'i,~,:~,,,(vi - Li/2)) T. ~,~,, is the mean of {, and ~ and Li denotes the length of the line segment expressed by ({~ - ~,). (5.5) G r a y Level S y m m e t r y : Window ITQ,,d of size (Li x K ) is set up at tile position of tile candidate. Two columns of pixels in Wu,,d are gray level symmetric al)out tile vertical eenterline Lv if
k
~-~[(a(eent-i)j-
I < A,.
(5)
j=l
where Ag indicates the threshold value during tile symmetry operation. This process is done for i = 1,2 ..... , ({+ - {,)/2. The extent of symmetry is found to be proportional to the number of pixels which are gray level symmetric. As the vehicle's underneath is only illuminated by faint light from the environment, the difference between two column pixels gradually increases from the vehicle's centerline to tile sides of tile vehicle, in tile small range of Ag. This feature is effectively use(l to clistinguish the area underneath the vehicle from pseudo-objects in tile mid-range distance of tile vehicle on tile road in tile vehicle detection module. (5.6) H o m o g e n e i t y : Homogeneity is one of the main properties of the area underneath a vehicle. As this area is not directly illuminated by tile sun or tile sky light, the intensity dispersion of pixels has little change. Therefore, the average intensity and tile standard deviation of this area can be efficiently used to distinguish the shadow underneath a vehicle from pseudo-objects, such as, low-contrast shaded parts of an asphalt road. If the confirmation is successfully completed then tile detected segment is assumed to be the sign pattern of a vehicle, otherwise it is noise. (6) Tracking Vehicles: Tile tracking process gives tile robot the ability to estimate the safe distance between the robot and the vehicle. A vehicle is detected and tracked in successive images by shifting the dynamic windows Wi (i - 3,4,...) to tile lower side of the image. Tile size and the position of tracking window Wi is determined by the length, center position, and ordinate of tile line segment in tile tracking window Wi-l. The length of the tracking window is given by
Wi = W~-I + AL
(6)
where A L is expressed by A L --
L i - l [ R i - ~ - Di • 2tan(aJ/2)] Di • 2 tan(a/2)
(7)
23 and a a n d / ? i - I are the field of view (F()\,') and the n,lative width of tile area subtended by the camera at distance Di-t respectively. L;-t denotes the length of line segment in window IV/_ I. Finally, depth, velocity, and width are determined by analyzing the time series of the ordinate of the area underneath the segment.
5. Proper Sensor Selection Two types of external sensors, visual and sonar, are currently used in vehMe detection and avoidance in HARUNOBU-6. The proper selection of sensors is the main purpose of the control s3"sWm of this robot. In the whoh, system, the most al)propriate group of sensors, or only one type of sensor, is selected to use as a response to changing conditions during the robot's motion. This means that all the sensors do not seem to l~e combined but rather, depending on the environmental conditions and the sensors' fimctionalities, the data fi'om one sensor tends to dominate. This helps to simplify the control program, while in the case of combining or filsion the complexity of the control sysWm grows exponentially [2]. Five types of actions are currently undertaken in HARUN()BU-6; Moving Along Wall (MAW), Tracking Motion (TM), Moving Toward (MT), Danger Estimation (DE), and Moving Along (MA). The first action is a sonar based and the remaining actions are vision based. The sign pattern of MA action is white lane marks or long edges along the ground, while a line segment found from the shadow underneath the vehicle is the sign pattern in TM, M T and DE actions. The principles of MA W and MA actions have been discussed in [9] and [12]. Each of these actions consists of simple processing whicll enables the robot to respond to sudden changes in its environment quickly. Selection of an action is determined by knowledge of tile limitations and the advantages of each type of sensor as well as by the current action of the rol)ot. The relevant action is reliable if its response to the enviromnent is positive. Two new actions M T and TM applicable to vehMe detection and avoidance are presented in this paper. Tile selection of TM and M T is done in the Motion Estimator module as shown in Figure 4. TM is used fox" tracking an oncoming vehMe. Distance 1)etween tile robot and tile vehicle as well as tile collision time are continuously estimated in this action. It is a useful action which ensures the robot maintains a salt' distance and finds a free route for its motion. On the other hand, M T is used fox" tracking a parked vehicle. Continuous detection and tracking of a parked vehicle helps the robot to evaluate its distance from that vehicle as it moves. Moreover, the width of the vehicle and the free path are estimated and fed to tile control system for the avoidance process. A free path in the road is found from knowledge of the road width and tile estimated wMth of the vehicle. It indicates whether tile path can be followed or not. Finally, DE is activated during the tracking process. The trajectories of vehicles as well as their direction of motion are found during this action. The output of this action indicates whether the route is safe or not. The result may be used to decide the next action of the rol)ot: for instance, the decision to ('ross an intersection, make an elnergency stop, or continue in motion, and so on. Tile detail of this action has been discussed in [13].
24
Figure 6. Stereotyped motion of the robot in avoiding a vehMe parked at the side of the roa(l
Fig~lre 7. The state transition diagram
5.1. O r d e r of P r o c e s s Figure 6 shows the Stereotyped Motion of the rol)ot in tile vehMe avoidance process. A general 1)aradignl for vehicle detection an(l avoidance l)y the l)roper use of visual and sonar sensors is shown in Figure 7. In the first phase, one of two actions, M A W or MA, is activated. In both cases the global information concerning the environmellt is ol)tained ])y the visual system. This information is ('omI)osed of the position an(l the angular directions of the left and right lane marks. If a vehMe is detected during the rol)ot's motion then one of two types of action, TM or MT. is started as a result ()f the motion estimator. At the same time, DE is activated to estimate the danger coefficient of vehM('s. The relevant action is continued until the visual sensor fails to provide sutfi('ient information about the vehicle. In other words, the uncertainty of visual sensory data progressively increases a,s the detection
25
fre(lllency decreases. Thus. the rehwan('e and the r('lial)ilitx" of this sensory data to the ('ulrent action is re(luce(l an(l the resl)()nse fl'()m ()ther t.X'l)eS of sensory data ten(ls to dolninate. Ill 1)ractice, this hal)l)ens at ('h)se l"ang(', less than 61n, when the visual sensor can't see the entire vehicle an(l fails to dete('t the sign 1)attern ()f the vehicl('. In this case, the reliaMlitv of the visual sensor reaches z('r() while that of the sonar sensor increases and 1nay reach unity. The estimat(,d width of the v(,hi('le, the al)l)roximate distance 1)etween the robot and the vehicle, and tile free path. are sent to the control system at this 1)oint. In the secon(l phase, the fl'ont sonar sensor is activated to prepare inforlnation on the position of the rear of the vehicle. The robot continues its motion until it is f meters from the vehicle, where distance f is provided by the front sonm" sensors. In the third phase, avoidall('e occurs. The avoidance lnaneuver is set up to sidestep to the right of tile 1)arked vehMe. The inforlnation ()l)tained fi'om the visual sensor, i.e. width and fl'ee path, as well as that fi'Oln the front sonar sensors, i.e. distance f, is used for ax'oiding the vehicle. The robot avoids the vehi('le 1)y following the sylnmetrical arc R. The interior angle 0 and the radius r of the arc ar(, given 1)y .)
II 2 -1- I',
-
~
(8)
,
4u II
0 - 2arctalt-,
(9)
U
where u and v are define(l as follows: u-
a
IV
+ s
~,-f+~,
+ -~-,
L
(10)
(11)
where L and W are the length and width of the rol)()t, a is equal to half of the vehicle width, and s is define(l as the distance between the rol)ot and the vehicle's side while it is moving along the vehicle. In the fourth phase, the M A W action is activated. MA can't be selected, as the left, lane mark is not in the visual field of the rol)()t. Xloving along the vehicle's side continues mgil the vehicle's side is not detected by the sonar sensors attached on the left side of the robot. In this case, the robot uses its locati()n 1)efor(, the avoidance process and returns to its position during the M T action. Finally, phw~e 1 will 1)e activated again.
6. E x p e r i m e n t a l R e s u l t s The vehicle detection and avoidance systenl was tested in the university carol)US. A vehicle was parked 30m in front of the rol)ot. Figure 8 shows the locus of the robot during motion and avoidance processes. The order of 1)ro('ess('s was as exl)lained in the previous section. The avoidance process was activated when the rol)ot reached 2m in front of the vehicle. The robot continued its motion, keeping l m fl'om the right side of the vehicle, using the ilffOrlnation received fl'om the s(mar sensors attached on the left side of the robot. Figure 9 shows the result of six frames of visual vehicle detection flom 30m to
26
Figure 8. The locus of the robot during motion
Figure 9. The result of vehicle detection in six images
5m in distance. Figure 10 shows tile result of estimating the vehicle width during M T action. Tile estimated width is 165(,m where tile real width of the vehicle is 160cm. The accuracy of more than 90% in many exl)eriments as sures tile robot of safe avoidance. Finally, Figure 11 shows the time series results of vehi('le detection by: vision-based and sonar based actions fl'om 30m to 2m in distan('e. The number of d('te('tions has been found in a defined time interval T. Because of the limitation of visual sensors in detecting tile sign pattern of the vehicle at a near distance, tile number of visual vehicle detections, shown by x, has reached 0 at a distance of 6m. Similarly tile number of sonar-1)ased vehicle detections, shown b v . , has increased at a near distance. 7. C o n c l u s i o n
A general paradignl for vehicle detection and avoidance by proper use of visual and sonar sensors has 1)een 1)resented. The 1)erfornmn(,e of the system has 1)een evaluated in
27
.-.. K
|
i
300
|
Vehicle Width Estimated Width
r
,,,
=
,,|
O (J (D
160 cm " 165 cm 9
60
Visual Sensor
Cl
"o E
O
200
d z Z
40
"-
r
uJ
1O0
20
5 2'o
lh
'
Ib
'
,
g"
30
20
Distance(m)
Figure 10. The estimated width of the vehicle. The result has been found from 20111 to 5m in distan('e during MT a ctioll.
10
a1
0
Distance(m)
Figur(, 11. The llUllll)el of v(,hMe dete('tions l)y visual and sonar s('nsors (hu'ing th(" l"(~l)ot's nlotion frOlll 3()m to 3111 in (list alwe. x indicates the nUlll])er of det(,('ti()ns 1)v visual sensor. ,, indicates the llllllll)('l" of (h't('('tions by sonar sensor.
Table 3 Results of vehMe detection
Road surface Clear weather (1)artly shadow) Clear weather (entirely shadow) Clear weather( llO shadow ) Cloudy weather
Detection success 99.8~! 100% 100% 100(~
()vet d(,t('('tion 0.4(/ ().8(/(~ 0.2(/: 0.6(/
Under detection 0.2(~ 0.0% 0.0% 0.0%
Total 11o. of vehMes 850 1050 880 860
tile university camlmS. The order of using s('ns()rs is sele('tcd according to the sensors' ('al)al)ilities. Tile lnethod of selection uses knowh'(lg(' of th(, functionalities and linfitations of each type of sensor a.s well as the outdoor cnvil-Olnn('nt situation. Five types of actions, originated froln Stereotyped Motion. have 1)ten al)l)li('(l for detection and a voidallce problems: they are lklovillg Along Wall, Moving Along, Tra('king NIotion, Nioving Toward, and Danger Estinmtion. Visual vehicle detection is discussed in 1)fief. Th(' method is 1)ased on Sign Pattcrn Stereotyped Motion. The Sign Pattern of the vchich' is the shadow underneath the vehicle. It. is unique to the time, shadow, weather, and light conditions. Tal)h ~ 3 shows the robustness of the shadow underneath for vchi('le d('t.ection in different conditions. This result has been obtained Kern one hour of videotal)e consisting of more than 3000 scenes of vehicles in different road and weather conditions. ()ver-det.ectioll occurs when pseudo-objects are detected as vehicles by mistake. Similarly, under-detection indicates the system is failure to detect, vehicles.
28
:s research on 1)rol)er selection of sensors is still ('ontimfing. hnl)roving the navigational system and multi-ol)ject detection are flltm'e work. REFERENCES
1. Alex Gask('ll and Penelope Probcrt, "Sensor Models and a Framework for Sensor Managenwnt", SPIE 2059, United States, Boston, pp. 2 1 3 , 1993. 2. R. luo and M. Kay, "Multisensor Integration and Fusion in Intellig('nt Systems", IEEE Trans. on Systems, Man, and Cyl)ernetics 19, No. 5, pp. 901- 931, 1989. 3. S.A. Shafer, A. Stentz and E. Thrope, "An Architecture for Sensory Fusion in a Mobile Rol)ot', Proc. of IEEE Int. Conf. Robotics and Automation, Raleigh NC, pp. 4 0 2 - 4 0 8 , 1987. 4. K. Kimoto and S. Yuta. "Sonar Ba~se(l Outdoor Navigation - Nax'igation using Natural Landnmrks ", IC'AR, Tokyo, Japan. Pl). 239 244. 1993. 5. M.A. Turk. D.G. ~Iorgenthaler, h:.D. Ch,'ml~an and M. Marra, "\'ideo Road Following for the Autonomous Land \,~,hich,", in Proc. of IEEE Int. Conf. Rol)oti('s and Automation, pp. 727-733, 1988. 6. U. Solder and V. Graefe, "O1)ject Detection in Real Time", in Pro(-. of the SPIE Symposiunl on Advances Intelligent System. Boston, 1)P- 104111, 1990. 7. A. Kuehnel. "Symmetry based Recognition of Vehicle Rears", Pattern Recognition Letters 12, North Holland, pp. 249--258, 1991. 8. T. Zielke, M. Braukman and W.Seelen. "Intensity and Edge-Based Symmetry Detection aI)I)lied to Car Following", Coml)ut('r Vision- ECCV '92, 1)1). 865-874, 1992. 9. H. Mori, M. Nakai, H. Chen, S. Nakayanm and T. '~51guchi, "A Mol~ile Robot Strategy - Stereo typed Motion by Sign Pattern - " , ~IIT press, pp. 161-172, 1990. 10. N. Moghadam and H. Xlori, "A New A1)proach for Real time VehMe Detection", IEEE Int. Conf. on Intelligent Robots and Systems IROS, Yokohanm, Japan, pl ). 273--278, 1993.
11. Hideo Mori. N. Moghadam Charkari and Takeshi ~Iat.sushita, "On-Line VehMe and Pedestrian Detections Based on Sign Pattern", IEEE Transaction on Industrial Electronics 41, No. 4, pp. 384-391, 1994. 12. H. Xlori, "Guide Dog Robot HAI:/UNOBU-5 - Stereotyped Motion and Navigation -", Springcr-\:erlag Press, Germany, Pl). 135 149, 1991. 13. Hideo Mori, N. Moghaddam Charkari and Shinji Kotani, "Danger Estimation of Vehicles at Intersection", IEEE Int. Conf. on Rol)oti('s and Automation, Nagoya, Japan, 1995.
Intelligent Robots and Systems V. Graefe (Editor) 9 1995 Elsevier Science B.V. All rights reserved.
Selective Refinement of 3-D Scene Description for Mobile Robot
29
by Attentive
Observation
Hotaka Takizawa ~, Yoshiaki Shirai and Jun Miura ~Osaka University, 2-1, Yamadaoka, Suita, Osaka 565, JAPAN This paper describes a method of constructing a 3-D scene description by attentive observation to find a path to a given destination. From low-resolution stereo images, features are extracted and a scene description is constructed with the constrained Delaunay triangulation. Then, a path is searched for in the description. If no reliable paths are found, an unknown image region on the most promising path is examined in zoomed-in stereo images and new features are extracted from the zoomed-in images. The new features are integrated to the original ones. A new description is similarly constructed of the integrated features, and a path is again searched for in the description. This procedure is repeated until a path is found or no more ambiguous regions are left. The experimental results for a real indoor scene including a chair, desks and computers are shown. 1. I n t r o d u c t i o n Vision is useful for an autonomous mobile robot to reach a given destination safely in an environment [1]. The cost of vision may be high if the robot has to observe the entire environment to make a detailed description [2] [31. There have been attempts to reduce the cost of making a scene description by selecting observation regions. Xie [4] proposed a method of viewpoint selection to determine whether or not the environment is different from a given map. Jasna et al. [5] proposed to observe an unknow environment (without maps) from several different viewpoints. The viewpoints are selected so that the robot observes the entire environment and that the number of the observations is minimized. Subhodev et al. [6] proposed to acquire range information of an environment with low-resolution stereo images to observe selected regions with high-resolution images. The observation regions are selected so that the fine description of the entire scene is constructed with minimum camera movement. These methods, however, are not effective for the task of finding a path to a given destination. In Blake et al. [7], the environment is first observed to find a destination. If it is not found, a freespace to pass through is searched for. The robot moves to the freespace and observes the environment again. The search of freespace may be inefficient because the robot has no global information of the environment. In our method, the robot selects the region on the most promising path out of the candidates obtained by low-resolution images. Then, the robot observes the region attentively. Fig. 1 shows a typical situation where the method is useful. If only the features Ti(i = 1 ,,~ 4) are detected from the low- resolution image, the movable area for the robot
30 is the movable area-1 because the feature points are regarded as lying on a surface. If the destination of the robot is behind the plane containing T2 and T3, the region corresponding to the area is observed attentively. If any textures on the back wall are observed, the movable area expands to the movable area-2 and the path to the destination might be found.
Figure 1. Movable areas for the robot.
Figure 2. Panther-VZ on a mobile robot.
In this paper, a 3-D scene description is constructed by attentive observation to find a path to a given destination. From low-resolution stereo images, features are extracted and a scene description is constructed. Then, a path is searched for in the description. If a safe path is found, the robot goes to the destination. If no reliable paths are found, impassable image regions and unknown regions are determined and an unknown region on the most promising path is examined in a stereo pair of zoomed-in images. From the zoomed-in images, new features are extracted. The new features are integrated to the original ones. A new description is similarly constructed of the integrated features and a path is again searched for. This procedure is repeated until a path is found or no more ambiguous regions are left. We use an autonomous mobile robot equipped with an Active Vision System (See Fig. 2), named Panther-VZ [8]. The camera system has four degrees of freedom: pan, tilt and vergence for the two cameras. A motorized lens is attached to each camera. Zoom, focus and aperture of the lens are controllable. 2. O v e r v i e w of t h e m e t h o d The procedure of the method to find a path to a given destination is as follows:
31 1. Take a pair of low-resolution stereo images and compute the range data. 2. Constructed a scene description from the data(described in Section 3.1). 3. If a safe path is found from the description, the robot goes to the destination and the procedure ends successfully. Otherwise, extract impassable regions and undefined regions (described in Section 3.2). 4. If the undefined regions are found, select a region to observe attentively. Zoom in the region and take a pair of high-resolution stereo images of the region(described in Section 4.1). Integrate the new range data with the original data (described in Section 4.2). Go to Step (2). 5. Otherwise. If there are no next viewpoints, the procedure fails. Otherwise, the robot goes to the nearest viewpoint. Go to Step (1).
3. Scene Interpretation 3.1. 3-D Scene Description with Constrained Delaunay Triangulation The 3-D positions of feature points(edges) are obtained by our feature-based stereo method [9]. The straight line segments are extracted by merging connected edges. The scene description is constructed by interpolating edges and segments with triangles. Delaunay triangulation is made use of to generate the triangles. It generates triangles such that nearer pairs of edges are preferably linked. In this description, a generated triangle may not correspond to a real surface. We will verify whether or not the triangles are real surfaces later.
3.1.1. Delaunay Triangulation The Delaunay triangulation is calculated from a Voronoi diagram [10]. The Voronoi diagram and the Delaunay triangulation on a 2-D plane are defined as follows(See Fig. 3). For a given set of points (called generators), Mi, on a plane, the corresponding Voronoi diagram consists of polygons, each of which includes a generator such that the polygon region is closer to the generator than to the other generators. The Delaunay triangulation draws line segments between adjacent generators in the Voronoi diagram. The Voronoi diagram and the Delaunay triangulation in a 3-D space is represented by polyhedra.
""/
"
M3 - ~ ~
M 4 ~ ~ . ~ i
Voronoi diagram
M2
....
it"............
~ M5 Delaunay triangulation
Figure 3. Voronoi diagram and Delaunay triangulation.
32
3.1.2. 3-D Scene Description with Constrained Delaunay Triangulation In indoor scenes, straight line segments are useful for 3-D scene description. Although the ordinary Delaunay triangulation is computed only with the generators, it is desirable to preserve straight line segments in the triangulation. The constrained Delaunay triangulation keeps the line segments as a part of edges of the final triangulation [11]. Floriani et al. [12] proposed to apply the 2-D constrained Delaunay triangulation to the 3-D description. First, the 2-D constrained Delaunay triangulation is generated from segments in an image. Then, the 2-D triangulation is backprojected to the 3-D space using the 3-D positions of the segments. We apply the method to segments and points. Given edges obtained by the stereo method, the straight line segments are extracted. The 2-D constrained Delaunay triangulation is constructed from both the segments and the remaining isolated points. Finally, the 2-D triangulation is backprojected to 3-D space. Fig. 4(a) shows an example of scene and the extracted segments (the white arrow indicates the destination). Fig. 4(b) shows the 3-D scene description with the constrained Delaunay triangulation.
Figure 4. Scene description.
3.2. Interpretation of Delaunay Triangulation The safe path to a given destination is calculated using the scene description. If the destination is in front of the triangles, the path is the straight line from the current position to the destination. Otherwise, the path is undefined because nothing is known about the space behind the triangles. For example, in Fig. 4(b), the robot can get no reliable paths, because the destination is behind the triangles. Some triangles of Delaunay triangulation, however, are generated on the plane which is not object surface. The robot can go into the space through those triangles. We search such regions. Triangles of the Delaunay triangulation are classified into the following two categories: 9 impassable: The robot cannot pass the triangle region.
33
9 undefined: The passability cannot be decided. The procedure is as follows. 1. Triangles which are higher than the height of the robot are eliminated. Triangles which are farther than the destination are also eliminated. 2. Triangles whose width is narrower than the robot are classified as impassable because the robot cannot pass the region. The other triangles are classified as undefined. Fig. 5 shows examples of the undefined triangles. 3. The undefined triangles are merged into a polygon if they lie on a plane. For every pair of neighboring triangles(or polygons), a plane is fitted by the least squares method. The pair is selected which has the smallest squared error. If the value is less than a threshold, the pair is merged. The merged pair is added to the undefined triangles(or polygons) set and the procedure is repeated until no more pair is merged. Fig. 6 shows the result of this process.
4. Undefined polygons are examined using the light intensity image to see if they are the surfaces of an object. If the angle between the normal vectors of two adjacent polygons is large, there must be a visible boundary between them. Thus, if there exist edges along a boundary between two polygons, at least one side of the boundary can be regarded as an object surface. (Broken lines in Fig. 6 indicate such boundaries.) If a robot cannot pass through either of the polygons without interfering the boundary, the both polygons are labeled impassable. The remaining polygons are labeled undefined. The image regions( See Fig. 7) which consist of the neighboring undefined polygons are the candidates for attentive observation to verify the passability. 4. A t t e n t i v e O b s e r v a t i o n 4.1. D e t e r m i n a t i o n of R e g i o n for A t t e n t i v e O b s e r v a t i o n The image region for attentive observation is selected from the candidates as the one which is closest to the promising path. The camera direction is set to the center of the region. The zoom value(magnification) is determined so that the zoomed-in image includes the region. It is also necessary that the zoomed-in image contains segments in various directions, because they are used to register the position of the zoomed-in image and the original image. The positional difference of two segments can be defined only in the perpendicular direction to the that of segments. For example, in Fig. 4(a), the destination is the left most computer on the desk. The right region shown in Fig. 7 is chosen for attentive observation, because the region is on the path which is straight line from the current position to the destination. Fig. 8 shows the zoomed-in image including the region for attentive observation and the extracted segments.
34
Figure 5. Undefined triangles.
Figure 6. Result of merging. The broken lines indicate object boundaries.
4.2. Integration of Observed Data This section describes how to integrate the base data and the zoom data. The base data are defined as the 3-D positions of the segments and the points which are extracted from the initial images. The zoom data are defined as the 3-D positions which are extracted from the zoomed-in images and are transformed into the coordinate system of the base data. The zoom data include errors from the pan and tilt angles of the camera platform. We correct the errors using the positional differences in the image plane between the corresponding segments in both data. The following procedure is repeated until the sum of the differences is less than a threshold. First, the matching pairs of segments are determined. Second, the difference vector (described in Section 4.2.2) in the image plane between both data is computed. Finally, the zoom data are modified with the difference vector. After the final difference vector is obtained, the segments and the points are integrated to the base data. We assume that the zoom value is always correctly controllable.
4.2.1. D e t e r m i n a t i o n of M a t c h i n g of S e g m e n t s We define the following evaluation values to be used for matching: 9 The average contrast is the average of the contrast value of the pixels which compose a segment. 9 The average direction is similarly defined. 9 The non-overlap-ratio is the ratio of the length of the shorter segment to the nonoverlapping length between the both segments. It is given as follows: N o v e r = (18 - lo)/l,,
(1)
where 18 is the length of the shorter segment and the lo is the overlap length as shown in Fig. 9.
35
Figure 7. The neighboring undefined polygons are the candidates for attentive observation.
Figure 8. Zoomed-in image.
9 The mean-squared-error is defined as follows:
Msqua-
1 ~)~ {e(t) }2 dt,
t 2 - t,
(2)
where e(t) is the distance from the point at t, on the shorter segment, to the longer segment, tl and t2 are the positions of the endpoints of the overlapping part along the longer segment as shown in Fig. 10. For a segment in the base data, the segment in the zoom data is selected which minimizes the following expression:
wlCnij + w2Drij + waNoverij + w4Msquaij,
(3)
where Cnij is the difference of the average contrast, Drij is the difference of the average direction between the i-th segment of the base data and the j-th segment of the zoom data. If the minimum value is smaller than a threshold, the selected segments in the zoom data correspond to the segment in the base data.
4.2.2. Computation of Difference Vector With a set of matched pairs of segments, we determine how much the zoom data should be translated on the image plane. The optimal amount of translation is called the difference vector Z. The difference vector is determined so that the sum of the mean-squarederror weighted by the overlapping length may be minimized, that is, Z is given by,
{ where errk is mean-squared-error of the k-th matched pair, lo,k is the overlapping length of the pair.
36
Figure 9. Non-overlap-ratio.
Figure 10. Mean-squared-error.
Let us consider Fig. 11 where the solid lines are in the base data, the broken lines are in the zoom data. The difference vector is determined to minimize the sum of the meansquared-error in the shaded area. The difference vector is computed by the contragradient method.
Figure 11. Computation of the difference vector.
Figure 12. Scene description with the integrated data.
5. Experimental Results The initial input image of a scene, shown in Fig. 4(a) is taken from the start point and the scene description is constructed with the constrained Delaunay triangulation. Fig. 4(b) shows the result of the triangulation. The triangles are merged into polygons and the polygons are classified into two categories: impassable and undefined. Fig. 6 shows the undefined polygons. The regions which include the undefined polygons are the candidates for attentive observation. Fig. 7 shows
37 the attentive region candidates. If the left most computer in Fig. 4(a) is the destination, the path is set to the straight line course from the current position to the destination. Since the right region in Fig. 7 is on the path to the destination, the region is selected for attentive observation. The zoomed-in image including the region is shown as Fig. 8. The constrained Delaunay triangulation with the integrated data is shown as Fig. 12. Fig. 13 shows the top views of the descriptions. With some of the newly obtained features, the surface composed of the edges of the chair and the edges of the desk is determined to be passable(See Fig. 13(b)). Finally, the path from the current position to the destination is found.
Figure 13. Top view of the description.
6. C o n c l u s i o n This paper describes a method of constructing a 3-D scene description to find a path to a given destination by attentive observation. The 3-D scene description is constructed with the constrained Delaunay triangulation and a path to a given destination is determined from the description. If the path is not found, the unknown image region which is on the most promising path is observed attentively. New data is obtained with higher-resolution. The new data, however, includes the position displacement by the errors from the pan and tilt angles of the camera platform. The both images are registrated by matching segments in both images and by computing the displacement. The new data is integrated with the original description, and a path is searched for in the new description.
38 Experiments have been performed for an indoor scene which includes a chair, desks, and computers and so on. The results have proved that the proposed method is valid for cluttered scenes.
Acknowledgment This research is partially supported by Mr.Fredric Solomon and by the Science and Technology Agency of Japan as one of the fundamental technical research project in their Sensor Fusion Program and by Grant-In-Aid for Scientific Research under Grant No.06555071 from the Ministry of Education, Science, and Culture, Japanese Government. The authors would like to thank those concerned for their cooperation and support.
REFERENCES 1. 2.
3.
4. 5.
6.
7. 8.
9.
10. 11. 12.
"Vision-Motion Planning with Uncertainty", J.Miura and Y.Shirai, Proc. International Conference on Robotics and Automation, pp.1772-1777, 1992. "Omni-Directional Stereo For Making Global Map", Hiroshi Ishiguro, Masashi Yamamoto and Saburo Tsuji, International Conference on Computer Vision, pp.540-547, 1990. "Map Building For a Mobile Robot Equipped With a 2D Laser Rangefinder", Javier Gonzalez, Anibal Ollero and Autonio Reina, Proc. IEEE International Conference on Robotics and Automation, vo13, pp.1904-1909, 1994. "View Planning for Mobile Robots", S.Xie, Proc. IEEE Int.Conf. on Robotics and Automation, pp.748-754, 1990. "Occlusions as a Guide for Planning the Next View", Jasna Maver and Ruzena Bajcsy, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.15, No.5, May 1993. "Multiresolution Image Acquisition and Surface Reconstruction", Subhodev Das and Narendra Ahuja, IEEE International Conference on Computer Vision, pp485-488, 1990. "Visual Exploration of Free Space", Andrew Blake, Andrew Zisserman and Roberto Cipolla, ACTIVE VISION(The MIT Press), pp.175-188. "Tracking a Moving Object by an Active Vision System: PANTHER-VZ", J.Miura, H.Kawarabayashi, M.Watanabe, T.Tanaka, M.Asada, Y.Shirai, Proc. International Symposium on Robotics, Mechatronics and Manufacturing System'92, pp.957-962, 1992. "A Multistage Stereo Method Gibing Priority to Reliable Matching", O.Nakayama, A.Yamaguchi, Y.Shirai, M.Asada, Proc. International Conference on Robotics and Automation, pp.1753-1758, 1992. "Voronoi Diagrams - A Survey of a Fundamental Geometric Data Structure", Franz Aurenhammer, ACM Computing Surveys, Vol.23, No.3, pp.345-405, 1991. "Representing Stereo Data with the Delaunay Triangulation", O.D.Faugeras, E.Le Bras-Mehlman, J.D.Boissonnat, Artificial Intelligence, vo144, pp.41-87, 1990. "Constrained Delaunay Triangulation for Multiresolution Surface Description", De Floriani L., Puppo E., Proc. International Conference on Pattern Recognition'88, 1988.
Intelligent Robots and Systems V. Graefe (Editor) 9 1995 Elsevier Science B.V. All rights reserved.
39
A C T I V E VISION INSPIRED BY M A M M A L I A N F I X A T I O N M E C H A N I S M
T. Yagi, N. Asano, S. Makita, and Y. Uchikawa Bio-Electronics Engineering Laboratory, Dept. of Information Electronics Nagoya University, Furo-cho, Chikusa-ku, Nagoya 464-01, Japan*
Focusing on the mammalian fixation mechanism, we propose a localization technique for object-detection in a scene, and implement it into an active vision system. In our technique, large and small camera movements are coordinated to detect an object precisely at the optical axis of the camera. Each type of movement is performed via two separate localization methods: approximate localization and adjustment localization. In the former localization, a preliminary fixation point is detected using the multi-resolution retina. After the optical axis of the camera is pointed in the direction of this point, the latter localization method determines a precise fixation point by means of visual feedback control, which minimizes the angle between the object's center and the optical axis of the camera. Experimental results show that the active vision system using our proposed technique succeeded in fixating objects one after another in a scene, even if several objects are placed within it. According to these results, we have confirmed that bottom-up processing using primitive visual features is an effective technique for localizing an object.
1. INTRODUCTION Active vision is expected to solve ill-posed problems in the understanding of 3-D scenes from 2-D images, and to be useful for more sophisticated visual information processing in industries [ 1][2][3]. One of the important components of active vision is "localization," which is the detection of an object and determination of its position. Conventional studies have used various techniques to localize an object in a scene; among them, segmentation by means of clustering several extracted features (such as intensity, color, texture, etc.) in a feature space, so called "feature-clustering" [4][9], is widely used. The problem inherent in this technique, however, is the long processing time required, which increases in proportion to the number of extracted features. Furthermore, it is difficult to apply this technique in scenes where each object has features that vary from the others only slightly. The other segmentation techniques, "region-growing" [15] or "split and merge procedure" [5], perform better in some cases; however, they are greatly affected by noise and sharp edges, as well as by lines which form disconnected boundaries. As a result, it is difficult to formulate simple criteria to decide whether or not they form true region boundaries. Of course, the "spatial-filtering" procedure [6] can be applied to find region boundaries; however, it also is sensitive to noise, and requires several other refining processes like noise-reduction or smooth-filtering in order to obtain clear and continuous edges. Consequently, conventional localization techniques have some problems in terms of processing time and noise. * E-mail:
[email protected]
40 In order to effect better and easier localization in active vision, we have for years focused on biological visual systems, and have studied the mammalian fixation mechanism from the computational point of view [12][13][14]. Fixation is the process of looking at a single point using eye movements [8][11]. Because spatial resolution in vision is the highest near the optical axis and decreases towards the periphery, only an object placed at the center of the retina is identifiable. Therefore, fixation is one of the inevitable functions of mammalian visual systems. Fixation is usually completed with a fast eye movement called saccadic eye movement; however, sometimes overshooting or undershooting of the object occurs. Such detection errors are compensated with a small saccade (catch-up saccade) and/or a slow eye movement known as smooth pursuit eye movement. In addition, fixation is generally considered to be driven by at least two different visual systems: the primary visual system (i.e. r e t i n a - lateral geniculate body - striate area), and the secondary visual system (i.e. r e t i n a - superior colliculus - pulvinar). Because identification of an object is carried out in the occipital lobe, fixation through the primary visual system seems to be related to top-down (or knowledgebased) visual information processing. On the other hand, some experimental results have shown that localization of an object can be achieved using only the secondary visual system, such as in lesion-induced malfunction of the primary visual system [7][10]. Therefore, fixation through the secondary visual system seems to be related to bottom-up visual processing. Moreover, it has been theorized that the secondary visual system in fact specializes in finding where objects are. Consequently, the understanding of this system is a key to accurate emulation of localization in active vision. In our previous psycho-physical and computational studies, we have hypothesized the importance of retinal signal processing in the secondary visual system, and revealed that certain fixation is correctly executed only if the mathematical model of the retina has linearly increased dendritic field size in the retinal ganglion cells [13][14]. Based on this hypothesis, in this paper we propose a localization technique which is similar to the mammalian fixation mechanism. Implementing this technique in an active vision system, we demonstrate that the active vision system is able to fixate objects one after another in a scene where several objects exist. In the last part of this paper, we discuss the advantages of our proposed technique and our future research work using this system.
2. B I O L O G I C A L L Y - I N S P I R E D L O C A L I Z A T I O N In order to fixate an object at the optical axis of a camera, the object must first be detected in a scene, then its location determined, prior to camera movements. Following two
Figure 1. The proposed localization method.
41 different types of localization, approximate localization and adjustment localization, work interactively for this purpose (Fig. 1). Whenever a camera is moved, a new image is sent to a host computer, then either type of localization is executed. The former localization directs large camera movements and determines a preliminary fixation point, which is approximately identical to the object's position. When overshooting or undershooting occurs, small camera movements are effected for compensation by the latter localization method until the object is foveated at the optical axis of the camera. Such coordination of large and small camera movements, which correspond to saccadic eye movements and smooth pursuit eye movements, respectively, is expected to perform precise localization.
2.1. Approximate localization Our previous studies have revealed that retinal signal processing is considered to play a great role in the secondary visual system [12]. Moreover, fixation points seem to be computed primarily at the retina [13][14]. Therefore, we use a multi-resolution retina "Eye Field" (Fig.2) in approximate localization. This Eye Field increases resolution near the optical axis and lowers it in the periphery. It is a simplified version of the mathematical retina model, and has non-overlapping rectangular-shaped receptive fields. (Although the mathematical model could be applied, we have modified the model in order to reduce computational load.)
Figure 2. Various types of Eye Fields. Fig.3 shows how a camera-taken image is converted into the image processed by the Eye Field. Every camera-taken image is split into smaller sizes near the optical axis than in the periphery (Fig.3a). In each split area, the intensity values of each pixel are added. The total intensity is divided by the split area to obtain the mean intensity (Fig.3b). Fig.3c indicates the mean intensity in each split area. The Eye Field is expected to intensify each object differently, corresponding to its location. In addition, it is supposed to reduce the effect of noise, especially at the peripheral region. From among several peripheral split areas, the one which has the maximum intensity value is selected, and the center of that split area is defined as a preliminary fixation point. The camera is then pointed in the direction of this
Figure 3. Approximate localization.
42 selected position (Fig.3d). These processes are repeated until the image part of the object is located at the optical axis of the camera. As shown in Fig.2, several types of Eye Fields could be proposed. Because the dendritic field size in the retinal ganglion cells greatly affects fixation [13][14], the localization ability of the Eye Field was hypothesized to depend on its splitting format. Therefore, we evaluated each type of Eye Field on a computer in terms of the following: suppose several objects are placed at random in a scene. When they are the same in size, intensity, shape, and color, it is known that human beings tend to fixate the object which is more closely located to the fovea. From the facts, the nearness of the object, otherwise termed the proximity of the visual stimulus to the fovea, is one of the factors which induce fixation. Focusing on this tendency, we simulated the localization ability of each Eye Field in various scenes. For quantitative analysis, we employed the "Proximity Index," which was defined in our previous studies [12]. Proximity Index refers to the normalized traveling distance of the optical axis of the camera in fixation. Assume the following distances: L from an initial fixation point to a detected fixation point, Lmax t o the farthest object, and Lmin t o the closest. Then, Proximity Index p is given as follows; p =
tmax m t
(1)
Lma x - Lmi n
As shown in this equation, this index gives a value of 1.0 if the detected object is located close to a fixation point, and a value of 0.0 if it is farther away; consequently, we can determine how far the detected object is located from an initial fixation point. In addition to Proximity Index analysis, we also checked if each localization was executed without overshooting or undershooting. In this analysis, we simply observed whether or not an object was detected at the central region of the Eye Field. Fig.4 is the histogram where the frequencies of Proximity Index in steps of 0.1 were graphed. It shows very high frequency in the category 0.9-1.0 in every type of Eye Field; therefore, it can be said that the object which was more closely located to the center of the Eye Filed tended to be detected. From this analysis, we confirmed that Type-I Eye Field resulted in slightly better localization than the others in terms of the proximity of the visual stimulus to the fovea. Also shown below, Table 1 indicates the percentage of localization without overshooting and undershooting. Note that 67% of localizations were performed without overshooting and undershooting in the case of Type-l, the highest score among the four Eye Fields. As a result of its superior performance, we have used Type-1 Eye Field in following experiments. Table 1 The percentage of the localization without overshooting or undershooting.
Figure 4. Proximity Index analysis.
Eye Field
Percentage
Type- 1 Type-2 Type-3 Type-4
67% 62% 35% 40%
43
2.2. Adjustment localization After approximate localization is carried our, adjustment localization takes place. Because approximate localization may result in overshooting or undershooting, it is necessary to compensate for such errors. In adjustment localization, only the central region of the Eye Field (the smallest split areas) is used for processing (Fig.5a). After approximate localization has occurred, the smallest central region is divided into horizontal and vertical halves (Fig.5b). The mean intensity values of each split area within regions A and B, or C and D are then added. In order to determine the traveling direction of the camera, we compare the total value in each region. If the total value in region A is greater than in region B, the camera is pointed in an upward direction. If not, the camera is moved downward. Likewise, comparison of the total values in regions C and D determines the horizontal traveling direction, left or right, respectively. The traveling distance of the camera is computed using the sum of the values of intensity in each pair of regions. Fig.5c shows an example of this procedure. The maximum traveling distance is defined as the distance from the center of the fovea to its edge. (It corresponds to 4 blocks in the figure.) As shown, region A is 14.50, and B is 4.25, hence the total is 18.75, and the difference 10.25. The ratio, therefore, between the total and the difference is 0.547. As for the horizontal case, the sum, difference and ratio are 18.75, 14.75, and 0.787 respectively. The traveling distance is defined as the multiplication of the sumdifference ratio by the maximum traveling distance. Therefore, in the case of Fig.5, the vertical traveling distance is computed as 0.787 x 4 (blocks), and the horizontal as 0.547 x 4 (blocks). These processes are repeated until the total values in regions A and B, as well as C and D, reach equality. When they become equal, an object is expected to be localized at the center of a camera-taken image.
Figure 5. Adjustment localization.
44
2.3. Subsequent object detection Based on the detected position of the object via the above, the camera is then aimed in such a way that the object is placed at the center of the Eye Field after the completion of localization. In the next phase, a new target is selected from the periphery again using approximate localization; however, the next target might be one which has been previously localized. In order to avoid such repetitive localization, we set the following rule: whenever an object is detected, its position is memorized. During approximate localization in the next phase, the position of the newly-detected split area's center is compared with stored positions. If it coincides to a previously memorized position within a c e r t a i n range, it is excluded as a candidate for the next fixation point. Accordingly, the area which has the second highest intensity is then selected.
3. EXPERIMENT 3.1. Experimental settings Our proposed localization technique was evaluated in the experimental settings described below. We applied the technique to an active vision system, called "Eye Mover" (Fig.6). In Eye Mover, a small CCD camera (EM-102II, produced by ELMO CO.) was incorporated into a universal joint linked to a motion driver via a sliding rod. Because the camera is able to rotate around the rotational center of the joint, it is possible to point the optical axis of the camera in any direction. An X-Y plotter (GRAPHTEC WX1000, produced by GRAPHTEC CO.) was used as the motion driver of the camera. Camera-taken images were transferred to a host computer (EPSON PC-386V, produced by EPSON, with an Intel 80386 20 MHz CPU, and 80387 co-processor) through an image digitizer (PHOTRON FDM 4-256, produce by PHOTRON, with 256 x 256 pixel resolution and 1/60 second sampling time). In Eye Mover, the rotational centers of the camera and the focusing point are identical and fixed at one position. In addition, the angle between the optical axis of the camera and the vertical axis of Eye Mover, is easily obtained from the camera-positioning data in the cameramotion driver. Therefore, the object's position in the 3D space could be geometrically computed using the angle and the height of the rotational center from the floor. The maximum traveling speed of our system was 180 (degrees/second), and its acuity was within 0.4 degrees. Unlike hand-eye systems, the size of our Eye Mover system was relatively small. In addition, the simple mechanism enabled the camera to execute movements within a wide rage of motion and to fixate an object quickly.
Figure 6. Active Vision System, "Eye Mover."
45
3.2. Results
The experiment was executed in a scene which contained several randomly placed objects on a plane background. Fig.7 shows a typical example of approximate localization (Fig.7a and 7b) and adjustment localization (Fig.7c and 7d). In the figure, camera-taken images are indicated on the left, and processed images on the fight. The camera-taken image at the initial fixation point is depicted in Fig.7a-left. Fig.7a-right shows the processed image using the Eye Field. Using the approximate localization process, the most intensified split area, the middle upper-fight in Fig.7a-fight, was selected. The center of the area was then defined as a preliminary fixation point, and the optical axis of the camera pointed in that direction. Fig.7b-left shows the camera-taken image after the camera movement. In this case, the object was correctly foveated using only approximate localization, and adjustment localization was skipped in order to select the next object for localization. Overshooting or undershooting, however, happened sometimes, as can be seen in Fig.7c. Because the large camera movement resulted in pinpointing only the edge of the ball (Fig.7c-left), adjustment localization was performed. Using the central region of the processed image (Fig.7c-right), a more accurate position was then computed. After the second small camera movement, the object was fixated precisely at the optical axis of the camera (Fig.7d-left).
Figure 7. Experimental results. (a)(b) Approximate localization. (c)(d) Adjustment localization.
Fig.8 indicates a locus of camera movements. Open triangles and open circles denote fixation points in approximate localization and adjustment localization, respectively. Starting from the center of the scene, Eye Mover pointed towards each object and foveated them one after another within a few small camera movements. In addition, it can be observed from this figure that repetitive detection was avoided. At the second to last fixation point (near the bottom of a can), the system detected the can because it was closely located to the camera's optical axis. The detected position, however, coincided within a pre-described range to the previously detected position of the can, so the can was consequently excluded as the next fixation target. Finally, the last object (a wooden piece) was localized.
46
Figure 8. Locus of camera movements.
4. DISCUSSION AND CONCLUSION The experimental results reveal that an active vision system implementing our proposed localization technique is able to achieve precise localization when simple scenes are displayed. The optical axis of the camera is directed close to each object on the first camera movement during approximate localization; each object is then correctly localized within a few small additional movements in the adjustment localization phase. In approximate localization, camera images are split into various sizes, and the intensity values of each pixel are successively added in order to compute the mean intensity in each split area. On this point, approximate localization is similar to the conventional "split and merge" procedure. One of differences between the two methods, however, is the splitting format. In the Eye Field, split areas are inhomogenenous. This makes resolution with our technique high at the center and lower at the periphery, while resolution is uniform in the "split and merge" procedure. Therefore, the proposed technique enables every object in a scene to be weighted differently according to the distance from the center of the scene to its location. Moreover, approximate localization is more applicable to a noisy image, because the effect of noise is reduced by means of blurring the peripheral region of the image. The second difference is the time required in the splitting procedure. Because the splitting format of the Eye Field is already determined before processing, the time in the approximate localization phase is always constant, while in the "split and merge" procedure it is variable. This constant time element allows us to estimate the processing time of approximate localization in advance. Consequently, the simplified computation in approximate localization overcomes the disadvantages experienced by conventional techniques in terms of noise and the processing time. Our proposed technique can be seen as bottom-up processing. Based on a low-level visual feature such as intensity, an object is localized without the assistance of top-down processing. Moreover, this technique does not require any noise-reduction or segmentation processing, thus consequently reducing processing time. Of course, some criticism can be leveled against such simple processing. Because low-level visual features mainly affect the determination of a fixation point, the localizing ability of the system is still at a primitive
47 level. The experimental results, however, show that our technique achieves adequate localization in a scene where several objects were placed. As we mentioned at the beginning of this paper, localization was realized without object identification. Hence it is suggested that such primitive processing is a key element of localization, and a key to understand "attention," which involves top-down processing. Based on this assumption, we have undertaken psycho-physical experiments on human eye movements, and are formulating a mathematical model of fixation and attention in order to reveal the relationship between bottom-up and top-down processing. Results will be published soon. In addition, we are attempting to apply our proposed active vision system to robot-manipulation tasks. In this study, a robot hand is manipulated by means of the positioning-data computed via our technique. After the position of an object is determined, the hand is moved to grasp the object. Preliminary analysis shows that the acuity of the detected position is precise enough to manipulate the hand. Gradually incorporating top-down processing into our active vision system, we will seek to further refine our technique in future studies.
REFERENCES
[1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12]
[ 13]
[14]
[15]
J. Aloimonos, and I. Weiss, and A. Bandyopadhyay, "Active vision," Proc. 1st Int. Conf. Computer Vision, pp. 35-54, 1989. R. Bajsy, "Active perception vs. passive perception," Proc. IEEE Workshop on Computer Vision, 1985. D . H . Ballard, "Reference Frame for Animate Vision," Proc. 11th Int. Joint Conf. Artificial Intelligence, pp. 1635-1641, 1989. G.B. Coleman and H. C. Andrews, "Image segmentation by clustering," Proc. IEEE, 67, 5, pp. 773-785, 1979. S.L. Horowitz and T. Pavlidis, "Picture segmentation by a directed split-and-merge procedure," Proc. 2nd Int. Joint Conf. on Pattern Recognition, pp. 424-433, 1974. D. Marr, Vision, W. H. Freeman and Company, New York, 1982. M. Mishkin, L. G. Ungerleider, and L. A. Macko, "Object vision and spatial vision: Two cortical pathways," Trends in Neurosciences, 6, pp. 414-417, 1983. D. Noton and L. W. Stark, "Scanpaths in eye movements during pattern perception," Science, 171, pp. 308-311, 1971. R. Ohlander, et. al., "Picture segmentation using a recursive region splitting method," CGIP, 8, 3, pp. 313-333, 1978. G.E. Schneider, "Two visual systems," Science, Vol. 163, pp. 895-902, 1969. A.L. Yabus, Eye movement and vision, Plenum Press, 1967. T. Yagi, K. Gouhara, and Y. Uchikawa, "An algorithm of eye movement in selective fixation," Proc. of IEEE International Conference on Neural Networks, Vol. 2, pp. 761765, 1993. T. Yagi, K. Gouhara, and Y. Uchikawa, "The influence of the inhomogeneous dendritic field size of the retinal ganglion cells on the fixation," Proc. of The European C o ~ on Artificial Neural Networks, Vol. 1, pp. 26-29, 1994. T. Yagi, K. Gouhara, and Y. Uchikawa, "A computational study on fixation and the effect of an inhomogeneous morphological feature in the retina," Proc. of IEEE International Conference on Systems, Man and Cybernetics, Vol. 1, pp. 777-781, 1994. S.W. Zucker, "Region growing: childhood and adolescence," Comput. Graph. Image Process., 5, pp. 382-399, 1976.
This Page Intentionally Left Blank
Intelligent Robots and Systems V. Graefe (Editor) 9 1995 Elsevier Science B.V. All rights reserved.
Realtime Range Imaging Using An Adjustment-free
49
PhotoVLSI
Atsushi Yokoyama ~ , Kosuke Sato b , Takayuki Yoshigahara ~ and Seiji Inokuchi ~ ~Production Technology Group, Sony Corporation, 6-7-35 Kitashinagawa Shinigawa-ku Tokyo, 141 Japan bNara Institute of Science and Technology, 8916-5, Takayama, Ikoma, 630-01 Japan CDepartment of Systems Engineering, Osaka University, Toyonaka, Osal~, 560 Japan We report a VLSI sensor which gives a range image of a scene with high robustness and high accuracy at a video frame rate. No adjustment is required. Photodiodes and operational circuits are integrated into each pixel of the VLSI sensor so they function as analog/digital micro-rangefinders. The 2D array of the micro-rangefinders acquires frames of range data in parallel manner as a lightstripe is swept across the scene. Complementary pairs of photodiodes in each cell reliably acquire range data without being disturbed by temperature drift, ambient light, manufacturing irregularities or other noise. A prototype system has been built using a 24 x 24 VLSI array of sensing elements. It gives us 24 x 24 range images at a video-frame rate. 1. I n t r o d u c t i o n Obtaining 3D information of a scene at a high speed and with high accuracy is important in factory automation tasks such as inspection and assembly. Numerous studies have been made of rangefinding techniques[ill2]. The lightstripe-rangefinding method is one of the most common and reliable methods of rangefinding. In this paper we report an adjustment-free range-imaging VLSI sensor which receives a complete range image consisting of lightstripe range data with high robustness at a video frame rate. The speed of a conventional lightstripe rangefinder using a CCD camera does not exceed one image/s because a lightstripe rangefinder needs hundreds of intensity images in order to construct a 2D range image. The conventional CCD camera captures intensity images of the scene on which the lightstripe is projected. The position of the projected lightstripe is extracted from each image and used to calculate the range data. For each position of the lightstripe, only a one-dimensional range image is acquired by triangulation. In order to acquire a complete set of the range data in the camera's field of view, these operations should be iterated as the lightstripe moves across the scene in steps, which requires more than one second. Cell-parallel lightstripe rangefinders have been reported by Araki et al. [3], Kida et al. [4] and Kanade et al. [5][6]. In these rangefinders, the lightstripe continuously sweeps
50 across the scene. During one sweep of the lightstripe, the time at which the image of the lightstripe crosses a cell indicates the range between the rangefinder and the scene along the line-of-sight of each cell. The time is detected by each cell and a complete set of range data in the camera's field of view is acquired during each sweep of the lightstripe. Araki et al. [3] have built a realtime rangefinder with 47 • 47 resolution with discrete phototransistors connected to separate signal processors. This rangefinder holds the timestamp by digital latches. Kida et al. [4] have built a rangefinder with 4 • 4 resolution with discrete photodiodes. This rangefinder holds the time-stamp in the shape of an analog sawtooth voltage signal by a sample-hold circuit. For large arrays, a high bandwidth link between each photosensor and signal processor is required and is provided by a 2D array of micro-rangefinders - each cell includes a photosensor and a signal processor. Kanade et al. [5] [6] constructed a VLSI sensor which consists of an array of cells, each of which contains a photodiode and signal processor. This sensor holds the time-stamp in the shape of a charge in a MOS capacitor with an analog sawtooth signal. One disadvantage of this sensor is that the time at which the image of the lightstripe crosses each cell is determined by comparing the photocurrent of the photodiode in each cell with the external threshold. Precise adjustment of an external threshold is therefore necessary, taking into account the ambient light, the reflectance of the object's surface and the operating temperature. Another disadvantage is that leakage of the holding capacitance makes it difficult to obtain accurate range data. In this paper we report a realtime, adjustment-free range-imaging VLSI sensor which gives a complete frame of robust range-image data at a video-frame rate. We call this new sensor a SRF(Silicon Range Finder) chip. In order to determine the time at which the image of the lightstripe crosses each cell with high robustness, a photo-comparator which consists of a complementary pair of photodiodes and analog/digital circuitry is integrated in each cell of the SRF chip. This has the advantage of allowing reliable 3D data to be obtained without being disturbed by noise. A prototype system using this SRF chip and a CCD chip has been developed. This system provides both a range image (from the SRF chip) and an intensity image (from the CCD chip). 2. C e l l - p a r a l l e l L i g h t s t r i p e R a n g e f i n d e r In this technique, each cell of the sensor array acquires range data while the lightstripe continuously sweeps across the scene. As shown in Figure 1, a lightstripe sweeps across the scene from right to left at a constant angular velocity of the scanning mirror. The image of the lightstripe on the sensor array moves left to right across each cell. Each cell detects the time at which the image of the lightstripe crosses it during one sweep of the stripe. The time-stamp signal is sampled and held, and converted to the orientation of the lightstripe, then the range data is calculated based on the triangulation principle. Figure 2 illustrates the cell-parallel lightstripe rangefinder geometry. The range data Zi,j of the cell Pi,j is given using baseline Bi,j and A, the direction of the line-of-sight/3i,j and the orientation of the lightstripe ai,j,
tan(~i,jtan/~i,j A Zi j = ( Bi,j + ), ' tan(~i,j + tan/3i,j tanc~i,j where/3i,i,
Bi,j, A
are constant.
(1)
51 The range Zc,r of each cell Pc,r on the sensor array is acquired during one sweep of the lightstripe by the detection of the orientation ac,r of the lightstripe. It is obvious that robust detection of the orientation of the lightstripe is essential for accurate range finding.
Figure 1. Cell-parallel Lightstripe Rangefinder
Figure 2. Cell-parallel Lightstripe Rangefinding Geometry
3. Silicon Range Finder The reliability and robustness of the range measurement of each cell depend on accurate determination of the timing. The VLSI sensor designed by Kanade et al. [5][6] determines the time by comparing the photocurrent of a photodiode with an external fixed threshold. For highly reliable and robust rangefinding, however, the time at which the image of the lightstripe crosses each cell must be determined without being affected by the ambient light, the reflectance of the object's surface or the operating temperature.
3.1. A N e w Timing D e t e c t i o n M e t h o d As shown in Figure 3, each cell of the SRF chip consists of an optical comparator capable of distinguishing minor differences in light intensity across the cell. The optical comparator has a complementary pair of adjacent photodiodes which generate photocurrents proportional to their incident light intensity. The photocurrents are fed into a current comparator which determines which photodiode is receiving the brighter light. As the lightstripe sweeps from right to left, the dominant photocurrent moves from right to left. The time at which the image of the lightstripe crosses each cell is determined by the transition of the dominant photocurrent. Even though the lightstripe is blurred due to defocus, the transition of the dominant photocurrent occurs at the exact center of the pair of photodiodes. In addition to this, the transition of the dominant photocurrent is not affected by ambient light, thermal drift or other noise.
52
Figure 3. Optical Comparator
Figure 4. Sensing element circuitry and Clock
3.2. Sensing Element Circuitry As shown in Figure 4, the circuit consists of five main components: photodiodes, current mirrors with gain, inverting voltage buffers, a clocked comparator, and comparator read circuitry. The photodiodes are made by N + doping in a P - substrate. They are surrounded by guard rings to ensure a solid ground connection and isolation fl'om adjacent cells. A complementary pair of photodiodes forms part of the optical comparator which determines which photodiode is receiving the brighter light. The size of the photodiode pair is slightly different to prevent disturbance from weak noises- the right photodiode is 3% bigger than the left one. The current mirror acts as a current amplifier and a buffer between the photodiodes and the inverting voltage buffers. It serves three purposes: first, it amplifies the photocurrent (a ratio of 2:1 is used); second, it isolates the photodiode cathode from the buffer input, which is susceptible to the influence of clock breakthrough during buffer clamping and unclamping; and third, it allows the back bias voltage on the photodiode to be independent of the large capacitance of the photodiodes. The buffer is a simple CMOS inverter with a CMOS switch between the input and output. Closing the CMOS switch clamps the buffer to its highest gain state.
53 A clocked comparator detects very weak differential voltage of the pair of buffers. The comparator is constructed using two back-to-back low drive CMOS inverters with a CMOS switch in parallel to allow initialization. The comparator is initialized during each sensing cycle before the buffer outputs are connected to the comparator inputs. The result of the comparison is converted to a digital value and sent to the output (Yn) via the inverting tri-state buffer selected by the output control signal Xn. 3.3.
Circuit
Operation
The sensing element is operated under three phase signals - r Figure 4. These signals iterate the following four phases:
r
r
-, as shown in
9P h a s e - A The input and output of the buffer are clamped together to place the buffer
in its highest gain state. The comparator holds the previous state. 9P h a s e - B The buffer clamping is released and the buffers are fl'ee to work. The stray capacitance at the buffer input nodes will be charged, via the current mirror, at a rate proportional to the photocurrents of the two photodiodes, and therefore proportional to the incident light intensity. As the charge of the photocurrent slightly changes the input voltage, the output voltage will be changed by the gain of the inverter. If the incident light intensity is different on the photodiodes, the resultant voltage change on each buffer output will be different. This differential voltage is compared by the following latched comparator. Sufficient time is allowed for PhaseB to ensure that enough voltage movement is achieved at the buffer outputs to switch the comparator. 9P h a s e - C The comparator is clamped into a cross-coupled neutral state in preparation for comparison. During this phase, the buffer continues to output. 9P h a s e - D The comparator is unclamped and the outputs fl'om the two buffers are switched to the comparator inputs. At the end of this phase, when the buffer drive is released, the comparator flips to a definite logic state, depending upon the polarity of the differential input voltage. This state will be read out near the end of Phase-B in the next cycle. A peripheral circuit addresses each column (X~) and reads the output (7~;,) in turn at a clock of 30kHz to 100kHz. When a pixel (X~,Yj) transits low to high, the corresponding digital latch in a frame buffer acquires the time-stamp count in 16 bits. Thus, the resolution of the rangefinding depends on both scan rates - the clock rate of addressing each column and the rate of the lightstripe scanning. For example, when the lightstripe sweeps at 30Hz and when the peripheral circuit scans the output (}2) at this clock rate, the resolution of rangefinding ranges from l@00 to ~ of the measurement range. 3.4. P e r i p h e r a l C i r c u i t r y The SRF chip has n-parallel column select lines (X~) and n-parallel row read lines (?~;,). Peripheral circuitry encodes the column address lines and decodes the row read lines, as shown in Figure 5. A time-stamp counter holds the current time. A scanning mirror controller resets the time counter, so that the time stamp is synchronous to the mirror scan. The frame buffer
54 is a 2D array of digital latches, with the same pixel resolution as the SRF chip. According to the trigger generated by the decoder module, the corresponding digital latch holds the time stamp. After the lightstripe was scanned a scene, all digital latches in the frame buffer hold the time stamp. An interface module transfers the time-stamp data to a computer. The computer then calculates the range data based on the triangulation principle.
.n~
C,o~ ~
7 \ q-
Eq i
i
,..,rror Con,,o,,.r
Line Xn
.
i "
---
Reset
"1o
" 0
II'
*
--
II -I1
SRF Chip
Scanning Mirror
rim e-S tam p
II
I
~' 'IC~ inter I
Row Read Line Yn
Trigger
~IL IlL IILI ~ ~ I L IlL I I Interface ]
~~IL
IlL
I
1
Computer Frame Buffer
Figure 5. Peripheral Circuitry
3.5. S e n s o r L a y o u t a n d F a b r i c a t i o n A prototype SRF chip has been fabricated in a 1.5# CMOS using a double metal, single polysilicon process. Figures 6 and 7 show the layout of a sensing element and a 24• element range sensor array with read-out circuitry, respectively. Each cell is 250pro • 250#m and the two photodiodes in each cell are 66#m • 100/tm and 64pm • 100pro. Figure 8 shows the SRF chip within the 84 pin ceramic PGA package, which includes 24 column-address pins, 24 row-read pins, and 3 comparator-control (phase inputs) pins.
4. P r o t o t y p e R a n g e f i n d e r A prototype rangefinder has been built using the new SRF chip. Figure 9 shows the prototype system. Figure 10 shows the configuration of the prototype rangefinder system, which includes a camera module, a range-image memory unit, a lightstripe generation module, and a host computer. The camera module consists of the SRF chip, a CCD chip, a lens, and a dichroic mirror which splits the incident light into visible ray and perpendicularly reflected infrared (Figure 11). Infrared focuses on the SRF chip and visible ray focuses on the CCD. The SRF chip and the CCD are mounted at the same optical distance from the lens. A range image and an intensity image in the camera's field of view are obtained at the same time.
55
Figure 6. Layout of Sensing Element
Figure 7. 24 • 24 SRF chip
Figure 8. SRF chip within the PGA package
The range-image memory unit consists of the frame buffer and a time-stamp counter. As mentioned before, this unit outputs the control signals to each cell on the SRF chip through 24 column address lines, and reads out the comparator state through 24 row read lines from the SRF chip in parallel. The counter value - the time-stamp data - at which the comparator changes its state from low to high is stored in the frame buffer. During one scan of the lightstripe, the counter values corresponding to each pixel are stored in the frame buffer. The lightstripe generation module consists of a 30roW infrared (780nm) laser diode, cylindrical lenses through which the laser beam goes to the lightstripe, a scanning mirror mounted on a galvanometer and a controller. This module outputs the reset signal to the time-stamp counter in the range-image memory unit. A host computer converts the counter values of each cell in the frame buffer into range data through a look-up table. In the next section, we see that the look-up table which maps the counter values into range data along all sensing element line-of-sight rays is generated by a calibration process.
56
Figure 9. The Prototype Rangefinder
Figure 10. System Configuration
5. C a l i b r a t i o n For accurate measurement, the system must be calibrated to reflect the exact relationship between the time-stamp count and the range data. The conversion function f ( C ) is derived by measuring four known parallel planes, as shown in Figure 12. In this figure, the lightstripe and the SRF chip are perpendicular to the plane of the figure, a reference plane SO is located at a known distance and the time-stamp counter Co of each cell is recorded. After measuring the distance of the reference plane SO, this process is repeated for planes S1 and $2, which are located at d and 2d from SO, and time-stamp count C1 and C2 are recorded. Using these data, the conversion function f ( C z ) which maps time-stamp count C z into range data Z from the reference plane along the line of sight is obtained as 2dT~(T~ - Tz) Z = f ( C z ) = TzT~ - 2TzT~ + T~T~'
(2)
where T~ =
tan(kC1 - kC2)
T~ = T~ =
tan(kCz - kC2) tan(kC0 - kC2), k = const.
The constant k in (2) is derived from measuring another plane, $3, which is parallel to the reference plane at the known position of Zx(r 0, d, 2d). In practice, a look-up table for each cell is generated using conversion function (2) for realtime conversion. p f -
57
Figure 11. The Camera Module
Figure 12. Calibration
6. E x p e r i m e n t s 6.1. R a n g e a c c u r a c y a n d r o b u s t n e s s In order to test range accuracy and the robustness of the prototype system, a white plate was positioned on a translation stage in the measurable area and range data were obtained by moving the white plate toward the sensor. Initially, the white plate on the stage was positioned 30cm away from the sensor and stepped 2500 times at 4pro pitch, with range data measured at each position. This experiment was done both in a dark room and under ambient light (50cm below a 30W fluorescent lamp) without thermal control. An experimental system using the SRF chip was built to the following specifications. B a s e line
920cm,
Focal l e n g t h of lens : 55mm, M e a s u r a b l e r a n g e : 28cm to 32cm, Theoretical maximum resolution S c a n r a t e of l i g h t s t r i p e
930Hz,
C l o c k r a t e of r o w s c a n : 50kHz, P o w e r of l a s e r : 30roW, W a v e l e n g t h of l a s e r : 780nm.
924#m,
s
.
o
.
.
.
.
.
.
.
.
]: ]!}
o -0.02 i l -0.04 0
,
,
5
,
'
10 15 Column
20
"
(a) under ambient light
o 8e-05
I
0
.
-o.o: .i
.
.
.
.
.
.
,
m*" -0.04
0
5
.
,
,
10 15 Column
,
.
20
(b) without ambient light
I under ambient I I light -I 1 without ambient ligh.t.:-..---
"= 4e-05
>
.
e
""
0
""-I
......
5
I''"
I0 Column
I
I
15
20
(c) variance of errors
Figure 13. Range Measurement Errors
Measured range data were compared to the actual range data. Averaged errors of 2500 measured range data are plotted in Figure 13(a) and (b). Figure 13(a) shows the averaged range measurement error of the each cell on the 14th row of the SRF chip under ambient light. Figure 13(b) shows the averaged measurement error in a dark room. The diamondoid points in the plot represent the mean errors and the vertical lines represent the ranges of the 2500 measurement errors. Figure 13(c) shows the variance of range measurement errors under each condition. The maximum range error under ambient light is - 2 4 # m to +10#m and that without ambient light is - 2 4 g i n to +14#m. This shows that the experimental error is less than the quantization error (+24#m). Because the accuracy and reliability of sensing data are the same in both cases, we conclude that a complementary pair of photodiodes - an optical comparator - is robust for the ambient light, and adjustments are not necessary. 6.2. R a n g e I m a g e of a Scene w i t h C o m p l e x V a r i a t i o n s in R a n g e We acquired a range image of a PCB mounting two QFP ICs, eight chip capacitors and two chip resistances. The heights varied between 0.5ram to 1.6mm. Figure 14(a) shows the PCB intensity image captured by a CCD. The PCB was positioned 180mm ahead of the SRF camera under the ambient light without thermal control. Figure 14(b) shows an elevation map of the 24 × 24 range image acquired by the SRF chip. In order to avoid occlusion, light generation modules were set at both the right and left side of the camera module. A lightstripe was projected from the right and left in turns and each range image was acquired at a video frame rate. The complete range image was generated by
59
Figure 14. ICs and Chip Parts on PCB
integrated two range images. The range image was a few pixels wider than the intensity image because the SRF chip is wider than the CCD chip. Although there are several different reflectances of the object's surface, such as mold surface and solder surface, the range image shows that the acquired range data are quite accurate. We also acquired a range image of a human face (Figure 15(a)). The face was positioned 50cm ahead of the SRF camera with a 20ram lens under ambient light without thermal control. Figure 15(b) shows an elevation map of 24 x 24 range image acquired by the SRF chip. The curved surface of the face is clearly visible and the lips, the nose and the cheeks are readily distinguishable.
7. C o n c l u s i o n \Ve have reported a new photo-sensing structure for a cell-parallel lightstripe rangefinder capable of reliably acquiring range data without disturbance from ambient light, the temperature drift of the environment, or the reflectance of the target object. We fabricated a smart vision VLSI sensor (SRF chip) based on this new structure. An experimental prototype system using the SRF chip demonstrates that it is able to measure a dynamic 24 x 24 range image with 99.9% resolution at a video fl'ame rate. Unlike other rangefinder chips, it is adjustment free. The prototype system also acquires intensity images by a CCD chip in the camera's field of view at a video flame rate. The combination of these range and intensity images of the scene should be useflll for various factory automation tasks, such as visual inspection and assembly.
50
Figure 15. Face
REFERENCES
1. Y. Nishikawa and S. Inokuchi, "Range Data Entry Techniques Using a Laser Spot Scanner and Two Linear Solid-State Image Sensors", IEEE Trans. on Instrumentation and Measurement, VOL.IM-30, No.4, pp.270-273, 1981. 2. K. Sato and S. Inokuchi, "Range-Image System Utilizing Nematic Liquid Crystal Mask ", 1st Int. Conf. of Computer Vision, pp.657-661, 1987. 3. K. Araki, Y. Sato and S. Parthasarathy, "High Speed Rangefinder", Proc. of SPIE, Vol.850, pp.184-188, 1987. 4. T. Kida, K. Sato and S. Inokuchi, "A Realtime Range Imaging Sensor", Proc. of 5th Sensing Forum, pp.91-96, (1988) in Japanese. 5. T. Kanade, A. Gruss and L. R. Carley, "A Very Fast Rangefinder", Proc. of IEEE 1991 Robotics & Automation, pp.1322-1329, April 1991. 6. A. Gruss, T. Kanade and L. R. Carley, "Integrated sensor and range-finding analog signal processor", IEEE Journal of Solid-State Circuit, vol.26, pp.184-191, March 1991. 7. K. Sato, A. Yokoyama and S. Inokuchi, "Silicon Range Finder- A Realtime Range Finding VLSr', Proc. of Custom Integrated Circuit Conference '94, pp.339-342, 1994.
Intelligent Robots and Systems V. Graefe (Editor) 9 1995 Elsevier Science B.V. All rights reserved.
51
A Pedestrian Discrimination Method Based on Rhythm S. Yasutolni , H. ~h)ri and S. Kotalfi Department of Electrical Engineering and Computer Science, Yamana.shi University 4-3-11, Takeda, Kofll, Yamanashi 400, Japan A new metho(l of fin(ling pedestrians in a time sequence of visual images has beell proposed, and its great 1)ra('ti(:al al)l)li('al)ility has ])een demonstrated. Walkers exhil)it 1)eriodic motion in (luite a different way from other moving ()l)jects. This 1)eriodicity, or in other words rhythm of walking, is the result of the walking action and can ])e ol)served in two kinds of perio(lic intensity change of the 1)e(lestrian's image. So, pe(lestrians can 1)e ibun(l if we l()()k for these p('rio(licities whi('h are (llfite unique to them. To realize this idea, we estimate the teml)oral-fre(llmn(.y and the spatial-period ()f the moving ()l)je(.ts. The temporal-fie(luen('y is the os(,illati()n of intensity against time. The spatial-1)erio(l represents the fluctuation of intensity along the traje('tory of the object, an(l is referred to as the stride of walking. A pe(lestrian detection meth()(l using these two oscillations has the following advantages. First, this metho(l is in(lel)en(lent of the al)solute intensity or contrast of the ol)je('ts. Secondly, clothes, hairstyle, shape of 1)ody, illumination and the distance fi'om the camera do not affect the relial)ility of detection. Finally, the algorithm is simple enollgh for real time operation. The (letaile(l algorithm is presented here and its vali(lity and 1)racti('al appli('ability are examined 1)3" an experiment on asphalt paved road. 1. I N T R O D U C T I O N
We focus on how to discriminate pedestrians flom other moving objects such as bicy(-les, animals and swaying plants. This technique is inq)ortant ill lnany fields: for example, detecting intruders for a. security,,' gual(ling systeln, a.n(l finding an avoidance-1)ath lnechanism for mobile rol)ots. However, variety,, of clothes, hairstyle and bodyshal)e make it. difficult to realize this end. So, we propose a new (l('te('tioll method which makes fldl use of periodicity in walking [1]. The recognitioll of an ol)je(-t in motion is (me of the tol)i('s of 1)sychol)hysics. ~\}' use the word "motiolF' here to refer to the challge of shape of the ol)ject and not to the change of its position. The moving light disI)lay (5ILD) nlctho(l proposed l)x" Johansson [2] is capable of perceiving the l)ody and the lnotioll of a walker. Kozlowski and Cutting [3] have demonstrated that the MLD can tell the sex of a walker using the patterll of the motion. These facts suggest that the motion of the ol)ject ("(mid 1)e the key to its recognition. R ashid [4] has reconstructed the structure of an ol)ject k o m the MLD kom a.n ellgineering point of view. So, given the structure of the lmlnan body, pedestrians can be selected from other moving ol)jects. However, this idea will not work because it. will be difficult
62 to know the in(livi(h:al ln(~tion of tit(' lmlts (if" th(, l~(,(l(,st:ian, i.('. arlns, f'(,('t, h('a(l an(l so ()lI. \\'(' can't ('Xl)('('t ('v(u'y 1)('(h'strial~ t(~ w('ar Sl)('cial marks at ('v(,ry j()int. P(>lana's a l)l)r()a('h [5] is s(mwwhat (lose to ours" It(' has tal:('n th(' m()ti(m int() a(-(-()lmt 1)y al)l)lying F F T and its 1)(uio(li('ity against tim('. A new ln('thod w(' 1)lOl)OSC in this 1)al)(U t(,lls 1)('d('st:ians h o m oth(u ln()ving ()l)j(,('ts 1)y utilizing two tyl)CS of 1)('rio(li('ity uni(lll{' t{) th(,m. Th(, first ()1:(' is the pitch (): 1)a('(, of walking. \V(' shall ('all this 1)(':i()(li('it)' against tim(' "t(unl)()ral-fr('q~:('ncy". Th(' oth('r tyl)(' is st ri(l(' of walking whi('h wc ('all "Sl)atial-1)('rio(l"" it is a n('w ('on('('l)t a(l(l('(l t() th(' apl)roach by Polana. The pace of walking at natural Sl)r has its o.wn Ul)l)(U' and low(u limits. So has the walking stride. Thus, t lwsc two featur(,s can represent all the motion of 1)c(h'strial~s. In order to ol)sc:vc t h('sc 1)('ri()di('iti('s in mono('ular images, w(' look at th(' intcmsity diffi'r('m'(' l)etwc(,l~ suc('essivc flam(,s. The l~e:'iodi(" ('hal~g(' of intel:sity l~rought al)out 1)y t h(' motion of t h(, f(,et can 1)(, ()l)s(uv(,(l and is ~s(,(l t() ('stimat(' t h(" t('ml~oralh'('qucnt'y. Tlw aml)lit~td(' of tlw os('illati(m will not aff'('('t th(' tcml)()ral-fr('(l~:(u~('y. ()m'(' the 1)a('(' aim dire('tiol: and distan('c of a m()ving ()l)j(,(-t a:(' knr th(, sl)atial-1)('ri(~(l ('an 1)e 1):'('(li('t('(l. Again, thcs(' 1):'o('c(hu('s a:'(' n()t aff'('('t('(l 1)y vis~al (listurl)am'(' as m('nti(m('(l a])ov('. Pedestrian (h,t(,('tio:~ 1)as(,d on t h(, a l)ove i(h'a is :('aliz('(l l)y tim,(' lm~('('ss('s: 1) m()vingol)j('ct (h't('ction, 2) the t ra('kil~g 1)l'O('('ss, 3) s('l('(ti(m l~y rhythln. In section 2, w(" (lcs('ril)(' t h(' m e t h o d of (l(,t('('ting a m()ving ol)j(,('t and its 1)()siti(m in successive h'alnc data. Then section 3 will l~(, (h,v(,t(,d to a tracking m(,th()(l l)asr on th(' kincmati(" model and ('onsi(h~latioll of its :vlial)ility. Cal('~:lation (,f t(,ml)oral-h('(l,:Cn(y and sl)atial-p('riod as wcll as a mod('l mat('hing t('('hniq~:c ::,ill 1)(, Dilly lnVs('nt('d in s(,('ti(m 4. The CXl)(Uiln('ntal results ol)tain(,d (m an asphalt lmV('d roa(l at(' (h,s('ril)(,(l in s(,('ti(m 5. Th(, final section gives some conch:(ling r(una:ks.
2
MOVING
OBJECT
DETECTION
A moving ol).jcct ('an 1)e detect(,d in m a n y ways such as sul)tra('ti(m of su(('(,ssiv(' frames [6], sul)tra('tion from background imag(', ol,ti('al flows [7] and so on. Sin('(" th(, sul)tractioll of su('c'r frames can 1)e dolw in the sh(n't(,st 1)ro('cssing t inw, w(, al)l)li('d this at the l~cginning of the data pro(cssing. The detail('d dctecti(m ln'O('cdm(' is as folh)ws. 1. Th(' illt('nsity ('hang(' 1)('tween su('('('ssiv(' flam('s is ('ah'ulatc(l and its al)solut(' vah:(,s arc t ransferr('d to the following lnOc('ss as sh()wll ill figlll(' 1. 2. To d(:'fin(" the moving-ol~jcct region a('('urat('ly, a vertical ('(lg('s arc sear('h('d fin and d a t a l~oints :cmoved whi('h hay(, h'ss int(,nsity than a t hrcsh(~M vahw. Th~,se lnOCeSs('s a.rc us(,ful bccaus(' imag('s ~)f"p(,d(,strians at(' mo:'~' likc,ly t.o hay(, vcrti~'al ('(lg('s than arc ()tlwr obj('cts. Th(' t hr(,sh(fl(l int(,nsiti(,s I/'t//el and Iv'nl~2 arc fix('(l so to cut off th(' 99 % of the intensity ('hange Inought h o m the 1)a('k-ground of t lw images. 3. To get th(' l~ott(ml position of t h(, ol)j(,(t :vgion, th(, v(,rti('al and h(n'iz(mtal lnoJ('('tions of the intensity a.rc ('al('ulatcd as shown in figure 2. The ol)je(,t-rcgion width
63
F i g u r e 1. D(,t(,ctit)n ()f a m o v i n g ol)ject
F i g l m ' 2. P o s i t i o n an(l wi(lth on t h e imag('
u'i(lt:xl(;(~.) is d(~t('rmined as t h a t of t h( ~ vcrti('al 1)roj(,('ti()n, a n d tit(, h o r i z o n t a l 1)osition u(A.) p o i n t s th(, (cnt(~r of th(' wi(lth. Th(' v('rti('al 1)()sition ~,(~.) is (h't('rmin(~(l as the 1)()ttt)m of the 1)rojti~(:tioll so t() fi)(.us 111)t)n th(, m()ti(m of t ht ~ t'eet. St'e figurt 92 t'()r (l(,tails of tht' r(qations 1)(~twt~en u'i(//.xtc,.(z.), u(~.) a n d ~'(~.). Her( ~ th(" in(l('x L" s t a n d s for tht, tim(, at whi('h th(' flamt, is ol)s(,rv(,tl. Th( ~ 1)ro('t'ss('s list t'd al)ov(, art, 1)el'fOrln('(| ()vt, r t h(' wh()l(' t)f th(' imagt~s ol)tain(~(l at t h(" v('ry 1)(~ginning of th(, d a t a 1)roct'ssillg. H()w(,v(~r, if tht, 1)t)siti()n of th(, ()l)jt,('t ca.n 1)(" 1)redi('t(~d in tit(' t't)urs(, of t r a c k i n g , only th( , (lata in a s m a l l t l a ( k i n g w i n d o w ('Clltt,lt,(l Olt t h(, ol)j(,('t art, 1)r()('t'ss(,(l. T h i s grt'atly r(~thl('('s 1)r()('('ssing cost a n(l nois(' t'a.llst,(| ])y ()th('r m o v i n g ol)j(~('ts.
3. T R A C K I N G Thtu(, art, tw() k i n d s ()fmt, t ht)(l tt) realiz(' t r a c k i n g . Tit(' first is tt) s('t'k th(" mfitllu 9shal)(" of the t)l)j(-'(-t. Gill)(,rt t't al. [8] h a v e (l(,mt)nstrat(,(l this mt, th()d in t r a c k i n g an airl)lan( ~ using its twt)()rth()g()nal 1)rojt'ctit)llS. Howt, vt'r. this mt,thotl rt,tluircs t ht~ ol)jt,('t to 1)t, rigid an(l is nt)t suital)h' for ()1lr l m r p o s ( '. T h t ~ s('('t)n(l att('tho(l is l)ased ()n t ht' ldnt'mati('s of the ()l).j(,(,t. Lt, gtt,rs anti Y ( m n g [9] hart, a i)l)lit'(l a m('tht)tl whi('h us(,s kintunati(" mo(h,ls an(l t.ra('killg filters to the sinmlatt~(| scent'. Alth()ugh thi.s ln('tho(| is us(~ful a n(l suital)h' f()r ()ur goal, it is n()t 1)ow('rflfl (m()ugh. \V(' h()l)(' th(, t r a c k i n g will (-ontinu(, (,Vt,ll if t h(, m('asur(un('nt ('n('()untt'rs ()utli('rs. Th(' ilnl)r()vt'(l r(~('()v('ry svst(un for th( ~ t ra('king will 1)(, stat('d in sul)s(,('tion 3.2.
3.1. K i n e m a t i c s a n d M e a s u r e m e n t M o d e l B('t'or(' s t a t i n g th(' m('asllr(un(mt or (lata h a n d l i n g t('('hni(llt(', th(' s('tting ()f t h(, vi(h,() ('amtua sht)lfl(l l)(' ('xl)laint'(l. \\-(' assure(' it tt)1)(, a 1)in-h()h' c a m t u a an(l alignt'(l 1)aralh'l tt) th(' r()ad. T h i s makt's it 1)()ssil)h ~ to st't, a m a x i m l m ~ mmfl)('r t)f 1)edt'strians in a vi(,w. Tit(' horiz()ntal axis ()f" vi(,w is also 1)aralh'l tt) th(' ,gr()lln(| s() as tt) ltlak(' 1)(,d('strians hay(, m()rt ' vtuti('al ('(lg('s as (letaih~d in the 1)rt'vi()lls s(,('ti()n. L('t .~'(a.)and !I(L.) giv(' th( ' 1)()siti()n
64
Z 9
,,0
/'-'-'-'-'-'-'-'O ~- / D l " " ",,
(~,,v(~)
.I
~ ~ 1 ',, ...--" )- ' 4 ~ ~ . / " , ,
ax,
~(X(k) ,y(k) )-Y
X"
Figure 3. ~Ieasurelnent model and ('OOl'(linat(' s3"stelns of a 1)edestrian at tim(' k. The state v(,('tor X(t.) is conll)osed of the position .r(~.) and y(k.) an(l their (lerivatives with respect to tilne .i'(/,,) and /1(i,.), i.e. velocity. Now we suppose a pedestrian to have an ahnost constant velo('ity. Then the estimated 1)osition of the pedestrian at time (k + 1)-Ys will 1)e
(i)
X(k+t) = FX(,.) + w(t.) where
X(t.)=
,i'(t.) y(k) !)(,.)
1 00
,F-
0 0 0
1 0 0
0 0 1 Ts' 0 1
,w(t.)-
i01 ,'t(t.) 0 ,"2(~.)
"
Here Ys is the nornlalized time interval and w(i,.) is calh'd system noise, w(t.) represents the fact that no pedestrian goes exactly straight, and it. is supposed to be a sequence with zero lnean, white noise and Gaussian with ('ovaiiallce Q. The Q is set to be 0.27Tm/s 2, as determined by experiment. The relation between the position in real-world coordinates with that in the linage plane Y(k) can l)e expressed using translation operator hit. ) as
(2)
Y(~'i = hIk'l(X(~'l) + z(k)
where Y(k)
=
~'(k)
, Zlk)
--
-'2(k)
9
The operator h(t.) defines the relative position of tile video camera and the ground (see figure 3) mid is read as
" fc :r(t.) uc + fc--Y(k) sin(O-Tr/2)+h cos(O-Tr/2) h(k.j(X(k.i) =
(3)
b fc (Y(t.) cos(O-Tc/2)+h s i n ( O - r / 2 ) ) fc-Y(k) sin(O-Tr/2)+h, cos(O-Tr/2)
65 where .f(:, h, and 0 correspond to focal length, height an(l depression angle of the ('amera. The optical axis is set at tile center of the image plane and its position is (u(,, vc). a and b are the aspect, ratios needed to correct the rati() 1)(,tween wi(lth and height of the image plane. Z(k) represents the measurement error and again is set to be zero mean, white noise and Gaussian with cova.rimlce R(k.). R(k) del)ends on the distance 1)etween pedestrian and tile video camera: the measurement noise becomes smaller when a 1)edestrian goes farther from the camera. Extended h:ahnan filter (EKF) is used to get the estimated state vector X(~.]~.) and predicted measurement Y(klk-1). X(klk) is the result of the incremental procedure of EKF. The predicted ~r(klk._l) points the center of the window at the next time step as is mentioned in section 2.
3.2. Checking the Validity of M e a s u r e m e n t The pedestrian may be hidden 1)3;other moving ol)jects and noise may lessen the validity of measurement. These facts reduce the reliability of measur('ment and will lead tracking to end in failure. For these reasons, we should make s~tr(, that observation is reliable. If the observation does not satisfy the following ('()nditions, th(' ol)serve(l l)osition Y(~.) is judged to 1)e wrong and should 1)e replaced with th(" 1)redi('ted 1)ositi()n ~r(~.l~._l). 1. Occlusion When the image of a pedestrian ix covered by another moving object, occlusion may arise. In this case, the 'mcas'urcd width of the object may not lie with in the range of the pedestrian width. So. th,c width of the object is a good measure of occlusion. The width of the object in real spacc wid(~.) is determined by applying an inverse perspective transformation to tt,i(11,~l(;(~.) at (u(~.), t'(~.)), and ch(:cki'ng by thc following condition. I'VI > u'id(k) > H2
(4)
where l't'l and H,~ dcnote the maximum and minimum widths of pedestrian respectively. .
Outlier When the noise cause an outlier in th,c measurement of position, the estimation may not be valid. We check the following condition for validity.
[X(k',/~'-1)-s
p(kl/,._l)-I[X(k',~,-l)-- s 2_
(5)
where ~( is a tentative estimation by the mcas'uTvmcnt "Y at I,. P(~I~-J) is the prediction error covariance determined by EKF. This condition means that wh,en the Mahalanobis distance between ~((k'l~'-l) and ~( is greater than 3.0 .) in the chisquare distribution of 4 degrees of freedom, then the measurement ~Y ix not reliable.
4. D I S C R I M I N A T I O N B A S E D ON R H Y T H M Tile rllythm of walking is caused 1)y a two-stage 1)il)edal action: first, a pedestrian stauds still for a relatively long time on both feet" second, oue of the feet steps forward rather swiftly. This is clearly seen in figure 4 as the 1)eriodi(' intensity change of the sul)tracted image data around the feet. In this section we will r the ways of placing a small window which is especially useful for detecting rhythln and ascribing it to walking.
66
Figure 4. Sul)tra('ted images at the fi,(,t of a 1)edestrian. Th(' ('(mtrast was enhan('ed fin. clarity
Fig~lr(, 5. \Vin(h)w fin d(,te(-tion of int(,nsitv ('hauge
4.1. D e t e c t i o n of R h y t h m A small win(low which in useful fin" (h,t(,('ting th(, rh.xthm ()f 1)('(h'strians in 1)1lt within the tra('king window as shown in figure 5. Th(' small win(low in l l l in wi(lth an(l 11~ in height. The actual values of 11, and ll~ will l)(, stated ill s('('ti(m 5. Th(' int('nsitv in this window will oscillate during walking lint th(' (listan('(' 1)('tw('en th(' camera and t h(, pedestrian aftb('ts its a nll)litude. Since the (listan('(' is inv(u'se 1)rol)ortional to the win(l()w size, the total int(msity within tit(" window in divide(l 1)3" the x,~:in(h)w size ill th(' imag(' plane. The norlnalized tilne se(tuen('c of total int(ulsity (~f"the window (l('ll()t,'(l by I(~.) in pa.rti('ularl.x,, suit al)le fl)r detecting the rh.x'thm. To calculate th(, 1)ower sl)e('trum P(.f) flOln I(~.) (1,: - 0 , . . . , / ~ ) for a d m a t i o n ()f :X' 9T.s. seconds, F F T is not slfital)le be('ause res(flllti()n is lilnit('d 1)y t.h(' numl)er of d a t a 1)r in a sampling interval. So, we use the n mxim, ml cntrol)y m(,thod (~IEM) [10] for its 1)etter fl'eqllency resolution. The final 1)l'('dicti(m Crl'()r in used to estimate the (n'(h'r 3I tbr the aut(~regressive lno(h'l for ~IE~I. The f'~uMaln('ntal t elnl)oral-fl(,(l~ten('y F of I(~.) ol)tain('(l in this way is related to the sl)atial-1)(ui()d T 1)y r -
D F - N . T,s,
(6)
where D iel)resents the distan('e of the ol)je('t at _Y. J-:.s'.
4.2. M o d e l M a t c h i n g Before 1)roceeding to the matchillg 1)roc('ss. a whit(, 1lois(, ('(mtl)(ment is r~unove(l flOln the power sl)e('truln ol)tained. Peaks slnalh,r than PrH/~ are set to 1hake no ~'ontril)uti(m. Walking a.t natm'al speed has its own limitatiollS fl)r th(' flmdamental t('ml)()ral-fl('q,Wll('y F and sl)atial-1)eriod T. Let F:~l.:l.\', F~ll:v and T.~I i.\', T~t/,x l)e the Ul)l)er and lower limits of these quantities. The dimensions of tiles(' 1)aramet('rs at(' presented in sul~section 5.1. The (letaih~d process in as follows: 1. To nornmliz(, the power Sl)ectrunl shal)e: a l l / } larg('r than P/'t11,' are smnme(l:
67 ,\:p
(7) /--1
H('r(' ~ is th(' p()w('r at th(, fl'('qll('ll('y f I (/ - 1 , . . . , A ~ , ) . If th('r(' is n() 1)()W('l ~/ gr('at('r than P~H~r no 1)('(lcstrian is in th(' win(low. S(), tit(, ('vahlati()n valu(' 5" is set to () and th(' 1)ro('('ss is lmltcd. 2. F()r ('a('h fr('(lu(ul('y _F!, th(' ('()rr('sl)()n(ling Sl)atial-1)('ri()(l T/ is ('ah'ulat('(l llsing th(' r('latioll 6. 3. The fr('qu(uwy F i is found at which Pi is a m a x i n m m and slwh that
F,~ttx < F/ < F.~t.~x, T,~llx < T / < T.~I ~.x-.
(s)
4. The evaluation valu(' E is t lw normaliz('(l 1)ow('r Pi, i.(,. E - P j / S . 10() 5. If E is grcat('r than t h(" giv(,n yah1(, E/'H t~, w(, (()n(ht(h ~ that th(u(' is a 1)(,(l(,strian in th(' window. ()n th(, oth(,r hand, if E is smalh'r than ErH/r t h(u'(" is n() 1)('(l('strian. ErH I~ is ('Xl)('rim(mtally s(,t so as t() nfininfiz(' th(, dis('riminati()n ('rr()r and th(, vahw is givcll in slll)s('ction 5.2. In th(, r(,al (mvir()nment, irwgular nois(:, will fr('(lll('ntly al)l)car whi('h has almost th(' sam(' Sl)c('trunl as that a ssigne(l t o 1)edestrians. This t('n(huwy is strong if' th(' ()l)s('rvati()n is ('arri(,d out fl)r only. a sh()lt. tiln(,. . To less(u1 th(, ('()ntril)lltion ('alls(,(l 1)v nois(', th(' . 11111111)o1 of o])s(uvations _V should l)e largc. In oth(,r words, the Salnl)lillg 1)(uio(l sh()uhl 1)(' long. Howcv('r, this will 1hake t lw d a t a 1)r()('('ssing tim(' l()ng. As a r(,sult ()i'a vari('tv of ('Xl)erim(ults. we t'oluld 64 d a t a 1)oints to 1)(, ('n()l@l t() lnak(, t h(' disturl)an(-(' smalhu than E/'Hfr 1)rovid('d only tw()('y('h:'s of 1)(u'i()(li(" ('hang(' wh()s(, fr(,(llWn(-y is th(, sam(, as that of walking, at(' a('cid(,ntally m(,rg('(l in th(, (lata.
5. E X P E R I M E N T S 5.1. E x p e r i m e n t a l S y s t e m The algorithm fi)r 1)('(l('strian dete(tion dcs('ril)(,(l in th(, 1)rcvious s('('ti()ns was t('stc(l using imag(' s('(lu(uwcs of an asl)halt paved road ()n th(' university ca nll),ls. Th(' vi(h'() c a m e r a S o ~ g C C D - TRI()00 was 1)la('('d at 1.()m h('ight, with 15.0 ~ (l('l)r('ssi()n angh'. In t lw RGB dcco(l(,(l vi(h'o signal, only tlw G (,Oml)()n('nt was us('d, whi('h has al)out 60(X: of th(" total hmfinosity. The A N D R O X ICS-400X~I9 imag(' 1)ro('('ssillg system ( C P U 68030, OS-9) was (Unl)loy('d with i b m digital signal 1)r()('('ss()rs. The saml)ling of th(' images and t la('king of the ('Xl)('rim('ntal s('t-ul) arc 1)r('s(ultc(l in tal)h,s 1 an(l 2. 5.2. E x p e r i m e n t a l R e s u l t s Two exanll)los of imag(, sequences arc shown in tigmc 6. Th(u(' are six 1)('(h'strians (Nil-El6), two 1)i('.x'('l('s (NIt, :~i~) and a dog (NI~)). The video ('anwra can look at ol)j(,('ts at distan('cs of 3.0 to 20.0m. Figure 7 plots t h(' cstimatvd traj('('tori('s and v(,lo('iti('s of t h(, ol).ic('ts. Th(' time illtcnsity an(l its power Sl)(,('trlml fl)l ('a(-h ol)j(,(t arc shown in figur(' 8. The int('nsity (l(,1)('n(ls on the a ctioll of walking (n ln()titm such as riding a 1)i('y('h,
68
Table 1 Parameters for the exl)eriments Threshold for moving object detection Number of intensity change data Order of AR model Threshold of power Pace model Stride model Pedestrian width model Ankle height model
Ir..i N.ll -
-
11 64
I r H R2 =
1?
15 P r H , =10.0 F.~/i,v = 1.50
F,~,.ux,
TMI,v =0.45
Tal..~x =0.99
ll~l= li,~--
14:2=
0.66 0.13
-2.60
Hz 111
0.13
Ill 111
Table 2 Examples of the standard deviation of measurement noise Distance frolll the video camera 3.0 5.0 10.0
20.0
m
/R(t:)ll
10.4
7.5
3.2
0.5
1)ixel
R(t,)22
4.9
3.9
2.2
0.7
1)ixel
o1" the running of a dog, as well as on the intensity diff('ren('(, l)etw(,en t lw ol)ject and the background. The pedestrians MI-.NI~i show 1)(,rio(lie change of intensity, but nonpedestrians lklr-M9 exhibit no periodicity. The estimated 1)ace, stride, evahmtion value and real pace for each object are talmlated in tal)h, 3. Th(' real pace for the pedestrialls is that of an average over 10 steps of walking. \Vith regard to non-pedestrian, pace and stride are estimated kom the frequency which has the maximum power. The estimated pace for non-pedestrian Mr is very (,lose to the lower ])omld of F, i.e. FM~A'. However, the stride is estimated to be about 2m which is not within the range of pedestrians (0.45-0.99m). So, we can tell Mr is not a pedestrian. Indeed, the evaluation value for 1~I7 is equal to zero.
The polmlation of the evaluation value E for 533 1)edestrians is shown in figure 9(a) and for 109 non-1)edestrians in figure 9(b). Of tht' 1)('d(,strians 82cZ~.wear trousers, 11% are in knee-length skirts or short 1)ants and 7(X, art' in long skirts. Of tht, non-pedestrians 807c, are bicycles, 18% (logs and 2c/(:. swaying 1)lants. Figure 9(a) shows that evaluation values E for 1)edestrians have a peak at the right side of the figure. On the other hand, for non-pedestrian there is a peak on the left. If the threshold value E'rHI~ is set to be 10, 94.0% of pedestrians and 95.40/(,, of non-pedestrians are corre('tly recognized. This m('ans that 94.2~: of all moving objects are correctly identified. The reasons for error will be summarized as follows. For pedestrians, 1) jittering of the observed images disturb the detection of the rhythnl of walking, 2) with swaying 1)lants often al)l)ear in the tra('king window, 3) walkers light-coh)red shoes diminish the contrast between the images of feet and the paved road. Reasons (1) and (2) are also true for n~ Jitters between successive fram('s is the main cause of error.
69
Figure 6. hnage sequences for.the eXl)eriments
6. C O N C L U D I N G
REMARKS
\Ve have I)roposed a new method for discrimination of pedestrians fl'om other moving objects based on the rhythm of walking. This method is achieved by the following three processes: moving-object detection in successive frames, tracking of moving objects using kinematic models and measurement applying extended 1,2ahnan filter, and searching for the walking rhythm in the power spectra of intensity of the images. These processes arc realized and executed in a realtime operation. The experiment has shown 94.2% of moving ol)jects to 1)e correctly
70
20
,
10
'
'
' .,
' ' ., ' ' '~ " '\ \~2" ,,. ,,,,~, M1 'x " 5.09 9 \\\' \, km/h ,, 4.71 \ \ "\, km/h \\ "". M3 \ ", ",, '", 6.33 \,\ "\ ", km/h ".,, \,, ""
\
-,4'
'
-'2
'
20'
'
',
';\ ' '\
M5 3.92 km/h
'
'
'
'
"\'x,\\,,X M6 \'", "'x"~ 4.04 ~ '\ \~ km/h "'\, Ix\
10
~r -4 i
20r
\,, \ M4 ',,X~,),,~: X 4.27 \ ~,, \ \ _ ,_ '~ \ . \ Km/n ""'\."*'\ X
~ i,
6 x(~) m
'
i
',
-'2
!
6 'x(~) m
I
, ,
\
',
\
M7 '" '" 10.54 ",, 101"-
km/h
\
\
i
'
9 "",
6.86 km/h .
\\
xx ~
M8 7.00
',
M9
~ -4.
I
\',,,~ ~kk\\
~.. !
i
. -'2.
\\
\\k m / h
"% N "',
"\, \
6
x(;0 m
Figure 7. Traj(,('tori(,s of l nOVillg ol)j('cts (1)ir(l's-(,y(, vi('w)
Tal)l(, 3 Estimat('d 1)a('('. st.ri(l(,, ('valuatioll Ol)j('('t lmznl)(u" ~Ii NI.2 Estilnat(,d 1)a('(' 1.92 1.91 Estimat('(l stri(l(' 0.74 0.68 Evaluation vahw E 48 92 R('al 1)a('(' 1.91 1.93
yah1(, Nla 1.94 0.91 73 2.00
and r('al 1)a('c of ea('h ()l)j(,ct Nil NI.~ Nlci NIt his 1.86 1.85 1.89 1.39 0.42 0.64 ().59 0.61 2.11 4.63 85 100 100 0 0 1.88 1.86 1.84
M!) 0.71 2.69 0
Hz m Hz
and 1)e(l('strians t() 1)(, re('ogniz('(l w('arillg various kin(Is of cl()th('s, and with diff(uing hairstyl('s and 1)()dyshal)('s. The exp(uim(,ntal results v('riI~" the vali(lity and r()l)ustness of th(, met h()(l. Th(" a(lwmtag('s of ()ur ln(,thod al'(': 1) the ('l( )t h('s, shal)(' of 1)()(ly, hairstyh' and s() on (l() not aff('('t th(, :(,suits, 2) th(' 1)a('( 9(t('lnl)()ral-h('(lll('ll('y) and th(' stride (Sl)at.ial-1)(ui()(l) at(' well ('()ml)incd tog('th(,r t.() make th(un 1)()w('rful (,n()ugh t() r(,m()v(, mils(, su('h as swaying plants, 3) the m('th()(l is simple and can 1~(, 1)(,rf'():m(,(l in r(,al time. Pedestrians in sha(low may not 1)(, (l(,t(,(,t(,(l 1)y ()lu 1):('s('nt m('tho(l. This 1):ol)l('n: should 1)(' ()v(u'('(mw in th(' :war fltt,u('. ACKNOWLEDGMENTS
\V(' at(' thankfld to Associat(' Prof. Dr. I(. Fujima fin valual)l(, sl:gg(,sti()n to iml)r()v(' this 1)at)(u". This work has 1)(,on 1)artia.lly SUl)l)()lt('(l 1)y Grant-Ai(l for S('i(,ntifi(" R('s('al('h flom ~Iinistrv of Edu('atioll No. 03650343. ()kawa Institut(' ()f hfformation and T('l('('ommuni('ati(m No. 91-32 and OMI1ON Co. Lt(l. REFERENCES
1.
H. 1[():'i, N.1I. Charkari and T. XIats,:shita. "()n-line \),hi('l(' an(l Pedestrian D('t(,cti(ms Based on Sign Pattern", IEEE Trans. In(h:s. El(,ctroni('s. \2)1. 41, No. 4,
71
>, 6 0 t l , , , , , , ,
,
60
oo 40
,
,
,
,
,
,
,
,
,
60
i
,
0 0
'
40
40
20
20
,
,
,
,
' 2
'
,
,
,
,
,
c-
(D
__E 20 0"()
'
601_,
'
' :2 '
, 1, ,
,
'
' 4s
,
, ,
0 0
o~ 4020 i-i, 0
2
(~[~)
4
601~ , , , , ,
,a'
Hz
,
0
~ 60
i'
2
,
__E 20
20
;
O"
' I-I
' '
~-o ~ 1000I
' 2
'
'
'
'
'
'
'
'
;,s'
",'
,
6
Hz
'
,
2
i
(~[,~)
0
(3 Hz
I'
'
'
'
'
'
'
0";
'
'
' 2 '
'
' ~,s I
::4~
'
, :
4
'
4
60
.'.',,., 2
'
'
,
,
,
4
(x[:~)
,
,
,
,
6
Hz
,
,
4s '
'
'
0
'
2
'
'
'
0 0
4~f,
0
2
4s
'
,-
4
'
I
I
t
O
6 Hz
(~[~-,)
'
'
'
I
I
I
2 ~-I
o
1000
140,
'
4s
100
20~
60
2
20
10I~,
Oh--'b-'~
0
~
2011
A t
0
40
0 0
'
,
~ ,
4
(x[._,)
40
(D
, i
' 4s
1 24~
0
6
--(oo-9 40
o~
60i,
'~176176I
0
60
4s
2 0 0 0 ~ ~ ~ r ~
B_
>,
2
60~,
2
,
,
4~
,
v 0
I
.
~
0
(~[.) ,
4
6 Hz
,
,
,
I
I
I
2 I
.I
I
I
,
4
1
4s I
J
11
&.
0
0
0
2
(~[:)
4
6 Hz
I- I
0
2
4
6 Hz
F i g u r e 8. I n t ( ' n s i t y ('haltgc a n d p o w e r Sl~(,('tllun ~ff' c a ( h ol~j(,(t
0
2
4
6 Hz
72
100 104 o
2oo7 1 94 OO/o
I [
100-
e-
t_ U.
0
10
50 Evaluation Value (E)
90 100
(a) Pedestrians
0 10
50 90 100 EvaluationValue (E) (b) Non-1)edestrians
Figure 9. Histograms of evaluation value
pp. 384-391, 1994. 2. G. Johansson, "S1)atio-Teml)oral Differentiation and Integration in Visual Motion Perception", Psych. tt,'s., Vol. 38, Pl). 379-383, 1976. 3. L.T. I(ozlowski and J.E. Cutting, "Recognizing the Sex of a Walker fi'om a Dynanfic Point-Light Disl)lay", Perception & Psychophysics, Vol. 21, No. 6, pp. 575-580, 1977. 4. R.F. Rashid, "Towards a. System for the hlterl)retation of Moving Light Disl)lays", IEEE Trans. Pattern Anal. Math. Intell., Vol. PAMI-2, No. 6, pp. 574581, 1980. 5. R. Polana, "Detecting Activities", Pro('. of IEEE Computer Soc. Conf. on Computer Vision and Pattern Recognition, pp. 2 7, New York, 1993. 6. M. Yachida, M. Asada and S. Tsuji, "Automatic Analysis of Moving hnages", IEEE Trans. Pattern Anal. ~[ach. Intell., Vol. PAMI-3, No. 1, pp. 12-20, 1981. 7. A. Shio and .I. Sklansky, "Segmentation of Peol)le in Motion", Proc. of IEEE Workshop on Visual Motion, pp. 325-333, 1991. 8. A.L. Gill)ert, M.G. Giles, G.M. Flachs, R.B. Rogers and Y.H. U, "A Real-Time Video Tracking System", IEEE Trans. Pattern Anal. Math. Intell., Vol. PAMI-2, No. 1, pp. 49-56, 1980. 9. G.R. Legters, Jr. and T.Y. Yomlg, "A Mathematical Model for Computer hnage Tracking", IEEE Trans. Pattern Anal. Math. Intell., Vol. PAMI-4, No. 6, pp. 583594, 1982. 10. D.F. Elliott (ed.), Handbook of Digital Signal Processing Engineering Applications, pp. 701-740, A('ademic Press, 1987.
Intelligent Robots and Systems V. Graefe (Editor) 9 1995 Elsevier Science B.V. All rights reserved.
73
Visual Recognition of Obstacles on Roads Uwe Regensburger and Volker Graefe Institut ftir MeBtechnik Universit~it der Bundeswehr Mtinchen Neubiberg, Germany Fax (+49 89) 6004-3074
As part of a driver support system for motor vehicles on freeways an obstacle recognition system was developed within the EUREKA project PROMETHEUS over the last four years. The obstacle recognition system uses a single monochrome TV camera and a multi-processor robot vision system. The recognition system includes a road tracker and an obstacle detector that were described in previous publications, and an object classification and tracking module presented here. The module is based on generic 2-D object models. It recognizes obstacles in real time and from distances of 200 to 300 m, sufficient for high-speed driving. The module was extensively tested in real-world scenes on the German Autobahn and on various other roads, including city streets with heavy traffic.
1. INTRODUCTION A driver support system for road vehicles is a great challenge for the technology of machine vision. The goal is to relieve the driver from monotonous tasks, to warn him of imminent danger (for instance, if he is about to fall asleep), and - possibly- to assume control of the vehicle in precarious situations. The necessary technology is being developed in numerous laboratories all over the world; there is hope that it will eventually lead to a significantly increased traffic safety [Masaki 1992]. Among the characteristics demanded of such a system are high reliability, low costs, and independence of any special infrastructure. Within the European research project PROMETHEUS the Institute of Measurement Science has developed real-time vision technology that may be used for a driver support system [Graefe 1993 ]. Freeways were chosen as the principal domain for testing and demonstrating the visual recognition of objects that are relevant for the understanding of traffic situations. The reason for choosing freeways, in spite of the typically high driving speeds, is that the complexity of the traffic situations and the variety of objects are much lower on freeways than on other roads. Obstacle recognition is an essential subtask of a driver support system. (In the sequel "obstacle" refers to any visible object on the road in front of the ego vehicle that is, or could possibly become, a danger for the ego vehicle.) According to a concept for obstacle recognition proposed by [Graefe 1989] the task should be decomposed into several largely independent subtasks, or modules, including the initial detection of possible obstacle candidates, the classification of obstacle candidates into false alarms (caused e.g. by shadows), and real physical objects, the tracking of obstacles, etc.. A module for detecting obstacles in distances adequate for high-speed
74 driving was introduced by [Solder 1992] and [Solder, Graefe 1993], and road modules by [Tsinas, Graefe 1992] and [Wershofen 1992]. A multitude of methods for recognizing obstacles in monocular image sequences has been described in the literature. Essentially three different classes of methods have been used. Gestalt approaches, e.g. [Davies 1990], aim at a direct reconstruction of the 3-D world as a basis for object recognition. Other approaches, e.g. [Enkelmann 1990], [Carlsson, Eklundh 1990] and [Young, G. et al. 1992] evaluate displacement vector fields and the differences in image motion between points within the road surface and points above the road surface. Due to the large amounts of computation required by these methods, they have not yet been demonstrated in real time and, thus, do not appear suitable for vehicle applications. The third class of methods is based on object models. According to this approach objects are recognized by certain combinations of visible features. This may be accomplished in a number of different ways. As one example, [Moil et al. 1989] and [Raboisson, Even 1990] first segment the image. Irregularities within the road segment are then taken as indications for objects. Additional criteria (edges, color, etc.) are used to minimize the frequency of false detections, while additional sensors, such as laser range finders [Alizon et al. 1990] or radars [Young, E. et al. 1992], may be used to measure the distance from an object. [Korn 1989] bases the tracking of vehicles on histograms of edge orientations. [Seitz et al. 1991] use a hierarchically structured multi-layer representation of edge directions for building object models and recognizing objects in images. [Hartmann, Mertsching 1992] describe object images by a "hierarchical structure code." [Ktihnle 1991] and [Zielke et al. 1992] track vehicles on the basis of edges and symmetry. [Graefe, Jacobs 1991], [Efenberger et al. 1992], [Solder 1992], and [Solder, Graefe 1993] detect and track vehicles and obstacles on roads by comparing generic 2-D object models with edges detected in image sequences. [Thomanek, Dickmanns 1992] and [Schmid, Thomanek 1993] use spatio-temporal models for tracking objects.
2. THE CONCEPT OF OBSTACLE RECOGNITION An object-oriented concept for dynamic robot vision has been proposed by [Graefe 1989; 1992]. It is based on the notion that the world is structured, rather than monolithic, and consists of individual objects. Only a few physical objects in a robot's environment affect the control of the robot in any instant. Following this concept, the task of object recognition should, therefore, be structured according to the relevant objects. This is done by associating one individual object process within the vision system with each relevant external object. The following object processes are needed for recognizing obstacles on roads (Figure 1): 9 Recognition and tracking of the road (because only objects that are actually sizuated on the road are relevant as obstacles) 9 Detection of obstacle candidates (the corresponding object is either a hypothetical object that has not yet been found in the image, or an object that has just been detected) 9 Tracking of an obstacle; multiple copies of this process may exist in the system to allow multiple obstacles to be tracked simultaneously.
75
Video section
V Road recognition
V
0otection II tr. ,o0
V Obstacle
tracking
L _ _ '_--'-'-- . . . . . _; Situation assessment
Vehicle control
Figure 1 Structure of an object-oriented obstacle recognition system Each object process receives image data (pixels) from the video section of the vision system and continuously generates a description of its assigned object. All object descriptions are evaluated by the situation assessment module and combined into a situation description. Depending on the perceived situation, vehicle control commands or messages for the human operator may be issued. Step by step the information contained in the pixels is transformed into more and more abstract forms. Starting with a large number of pixels a chain of data fusion processes eventually leads to a description of an object or a situation. At each step specific knowledge in the form of appropriate models is introduced into the process (Table 1). At the first step, the feature extraction, groups of pixels are combined into features, such as edges or corners. Knowledge about characteristics of the camera and of the features being searched is used in this process. Feature extraction may be supported by the next higher level in the processing hierarchy, the 2-D object level. Here, knowledge about the visual appearance of specific physical objects is used to generate hypotheses regarding the nature of the object being seen and expectations as to which of its features should be visible, and where in the image they should show up. Feature extraction is thus not a form of schematic image processing, but part of a model- and knowledge-based hypothesis verification or falsification process; its task is to provide answers to very specific questions as to the presence and location of certain expected features in the image. A very robust image interpretation may be realized by such an interaction between the feature extraction and the 2-D object levels. It is possible to add additional 3-D or 4-D (spatio-temporal) levels if a description of the 3-D shape of physical objects, or of their motions, is desired.
76
Table 1 Levels of Modeling and Data Fusion Model Level
Data Fusion Performed at that Level
groups of pixels feature
-, elementary features complex features
2-D object
groups of features 2-D objects
4-D object
2-D objects, time, sensor data 4-D objects
environment and situation
robot state, external objects -- situation
3. THE FALSE-ALARM PROBLEM
For reasons of safety an obstacle detection system must be perfectly reliable in the sense that it never fails to detect any obstacle that is actually present. Moreover, in order to allow a timely reaction, obstacles must be detected while they are still far away. These requirements make it practically unavoidable to generate false alarms occasionally, caused, for instance, by shadows, dirt, markings or patches on the road surface. Although false alarms are not directly dangerous they must be processed and, thus, constitute an unnecessary load for the vision system. To minimize this load, false alarms should be identified and discarded as quickly as possible. Distinguishing between actual obstacles and shadows is easy in many cases, but sometimes it can be extremely difficult. We use a set of heuristics to reject most shadows within the detector module as quickly as possible. Therefore, all alarms reported by the detection module are classified by matching the visible features that have been detected in the image against generic 2-D models of obstacles. If a fairly good match is possible, the obstacle candidate is accepted as an actual obstacle. This decision is final. If, on the other hand, the detected features do not match the expectations derived from the model, the obstacle candidate is considered a possible false alarm. A final rejection, however, does not occur at this point. Rather, the detector is requested to report the same obstacle candidate again if it can be detected in one of the subsequent images. Due to the motion of the vehicle the subsequent images will differ from the previous ones in respect of perspective, specular reflections, occlusions, etc., and, accordingly provide an independent chance to recognize an actual obstacle if there is one. A final rejection thus occurs only if an obstacle candidate has been classified many times and has not been accepted even once.
77 If the time required for one cycle of detection and rejection is short, many independent images may be analyzed while approaching an obstacle candidate. The probability of rejecting an actual obstacle is then very small. On the other hand, it is possible that once in a while false alarms are accepted as an obstacle. They must be rejected during tracking.
4. GENERIC 2-D OBJECT M O D E L S 4.1 Effects of Distance on the Visual Appearance
As Figure 2 shows, the visual appearance of an object depends strongly on the distance from which it is seen. Moreover, for a given camera and object the area of the object's image is inversely proportional to the square of the distance. Images of nearby objects are clear with large contrast and many visible details. Many features may, therefore, be detected in such an image. A lateral motion of the object causes a significant motion of its image. Images of distant objects have a low contrast and do not display much internal structure. Their contours tend to be somewhat blurred. Because of their small size and low contrast hardly any internal features can be detected. Rotation of the camera is the dominating cause for motions of objects in the image. In an intermediate range of distances the contours of objects tend to be clearly visible, but the object images do not have much internal structure. Motions of the camera and of the objects have similar effects on the motions of the image. Because of these differences specific models are used to model the appearance of physical objects, depending on their distance. The models include a number of free parameters in order to let them match objects of different shapes. Therefore, they are called generic object models.
4.2 The 2-D Model for the Nearby-range
Most of the obstacles that exist on freeways display a nearly rectangular contour with vertical and horizontal sides when seen from an approaching vehicle. Therefore, parts of rectangular contours are searched in the image.
Figure 2 Different 2-D object models (top) take into account the significant differences in the appearance of nearby (left) and far away (right) objects.
Characteristic points on the contour that may be used for an object description are the corners or the centers of the sides of the rectangle. The upper part of an obstacle is often seen before a structured background, such as trees, mountains, or other vehicles; this makes it difficult to locate the upper part of the contour in the image. Usually numer-
78 ous horizontal edges are visible, and it is not easy to determine which one is the upper edge of an obstacle. The lower part of the contour is usually seen before the surface of the road and may, therefore, be detected and tracked more easily. Recognizing only the left, right, and lower edge is sufficient for tracking the object in the image and for measuring its distance. Therefore, only these edges are looked for in the image. Nevertheless, the large number of features that are often visible in the images of nearby objects makes it difficult to single out those belonging to the contour of the object. A statistical approach was chosen to solve this problem. The image of a typical object contains a number of homogeneous areas, delimited by approximately vertical edges. Many of these homogeneous areas begin, or end, near the le~ and right contours of the object. In an area of interest that encloses the image of the object candidate, the le~ and right ends of homogeneous areas, and their centers, are counted in each image row. Local maxima of the distribution of these features indicate a possible location of the object contour and its center line. Combining these yields strong expectations where the contours of the object should be found in the image. Vertical edge detectors based on controlled correlation [Kuhnert 1986] are then applied in the vicinity of the expected contours to find the exact locations of the contours. Generally, the lower edge of an object is distinctly visible. In case of vehicles the lower edge of the shadow underneath the vehicle is taken as the lower edge of the object. It may usually be detected reliably and robustly because the shadow appears much darker than the pavement. 4.3 The 2-D Model for Distant objects
Images of distant objects are typically small and of low contrast (Figure 2). Only few features may be extracted; therefore, a statistical approach is not feasible for recognizing such objects. The left, right, and lower edges are detected directly by applying suitable edge detectors. Due to the large image motions induced by unavoidable camera motions, the tracking of distant objects would not be robust if it were based only upon a few local edge elements. Using additional features, such as corners and symmetry, improves the robustness. In contrast to the near range where, due to the high spatial resolution of the image, object contours rarely contain any sharp corners, corners may be detected robustly in the far range. Specific corner masks are used for detecting the lower left and right corners of objects by the method of controlled correlation. The images of many distant objects, including the rear-views of vehicles, have a vertical symmetry axis. Symmetry is, therefore, included as an additional feature in the object model. In the near range symmetry would be difficult to detect because of the many visible details, and finding a symmetry axis would be time-consuming due to the large areas in the image that must be processed. In the far range, where object images are small and do not contain many features, symmetry detection is fast and robust.
5. DETECTING VERTICAL SYMMETRY AXES Numerous methods for detecting and exploiting symmetry have been proposed. If object contours have been determined and converted into B-splines, symmetries may be extracted according to [Saint-Marc, Medioni 1990]. A method for detecting symmetries of polyhedrical objects has been described by [Jiang, Bunke 1990]. Shape recognition based on symmetry was introduced by [Yuen 1990]. [Friedberg 1986] determined symmetry by computing a moment matrix. A known
79 symmetrical object may be detected by convolving its image with a mirror image of a model; atter detection it may be localized by convolving it with its own mirror image.
Figure 3 Pairs of pixels, centered around a pixel, x, are evaluated. For each pair the difference of the grey-levels is computed. The absolute values of the differences are added up. Their sum is the symmetry function at the center pixel x.
All these methods presuppose a good segmentation of the image (e.g. into object and background) or have been tested only on synthetic images or line drawings. Moreover, they require computing times of tens of seconds.
Methods that are suitable for real-time and real-world applications have also been proposed. [Zielke et al. 1992] determine for each pixel a symmetry measure, based on the even and the odd part of the symmetry function. A symmetry-based statistical approach has been used by [Ki~hnle 1991] to determine the center of the image of a car as seen from the rear. Histograms of symmetrical contour pixels, of symmetrical grey-level pixels, and of the lengths of horizontal edges are computed; local maxima in all three histograms indicate candidates for a vertical symmetry axis [Marola 1989]. In the sequel a method is described that detects individual symmetry points and then determines image columns containing a maximum number of symmetry points. For finding a symmetry point the even part of the symmetry function is determined. To accomplish this the similarity of greylevels of pairs of pixels, located symmetrically to the left and to the right of a point in question, is studied (Figure 3). The absolute values of the differences of the grey-levels of each pair of pixels are added up and assigned to the pixel in question. A symmetry function is obtained by repeating this procedure for all pixels within an area of interest. Local minima within a row of the symmetry function indicate symmetry points; in Figure 4 the symmetry points have been marked. Figure 5 shows the corresponding symmetry function, computed over a 40 pixels by 40 pixels area of interest around the vehicle. When symmetry points are searched, the width of the environment upon which the symmetry function is based should be somewhat larger than the width of the object. However, if the
Figure 5 Figure 4 Symmetry points determined from 12-pixel wide environments
The symmetry function in an area of interest covering the car in Figure 4; each graph corresponds to one image row.
80 environment is chosen too large, many background pixels are included in the computation. A strongly asymmetrical background may then severely distort the symmetry function.
6. EXPERIMENTAL RESULTS 6.1 Test Environment
A real-time vision system BVV 3 [Graefe 1990] was used for testing the obstacle recognition system. Each of the 3 sub-processes, road recognition, obstacle detection, and obstacle tracking, was implemented on a separate parallel processor within the B VV 3. The obstacle recognition system was designed to cooperate closely with other modules of a driver support system, for instance modules for modeling the dynamics of the recognized obstacles. Such a cooperation has been tested earlier and described by [Dickmanns, Christians 1989] and [Regensburger, Graefe 1990]. The tests reported in the sequel were, however, performed without any additional modules. The results may, therefore, be directly attributed to the modules described here. The situation assessment module (cf. Figure 1) was replaced by a message passing system that allowed communication between the recognition modules, but did not perform any additional processing. 6.2 Recognition Distance
The ability to recognize obstacles from a great distance is of critical importance for driving at high speed. Experiments have shown that cars (moving and resting) can be tracked in distances of about 200 m; under cooperative conditions, e.g. high contrast and little camera motion, cars could be tracked in distances of over 300 m. As Figure 6 shows, a car in a distance of 300 m appears rather small in the camera image. The obstacle tracking module does not include any distance measurement. The distance from the obstacle was, therefore, measured by external means. Visible markers were placed on the pavement in known intervals. As this was impossible on a public road, the runway of a closed airfield was used for the experiments to measure the maximum recognition and trackingrange,
Figure 6 Passenger car seen from a distance of 300 m through the camera of the vision system, with overlaid markings showing the detected contour and symmetry axis
6.3 Robustness
Although the obstacle recognition system was designed for freeways, due to the model-based tracking module it proved to be so robust that it was also tested on ordinary highways and furthermore in heavy city traffic. There, other cars, as well as the own car, changed lanes frequently causing time-varying occlusions, and together with structured backgrounds and markings on the pavement generated a high degree of visual complexity. Even in such difficult scenes (Figure 7) vehicles could be tracked reliably over extended periods of time.
81 The 2-D object models employed by the recognition system assume a nearly rectangular object contour and they have been optimized especially for cars and trucks. Nevertheless, it could be shown that various other objects that may occasionally be seen on roads, such as boxes, barrels and, in some cases, persons, may also be classified and tracked. However, tracking such objects that do not match the generic models well is less reliable, especially under conditions of poor lighting or when they are far away and their images are small. Increasing the repertoire of different object models in the system should further improve the system performance in this respect.
6.3 Speed Figure 7 The computing time required by a module of Tracking a vehicle in heavy city traffic a real-time vision system is always of prime importance. The cycle time of the object classifier and tracker was, therefore, measured in a number of experiments. One parallel processor of the robot vision system BVV 3 using an Intel 386/20-MHz CPU, augmented by a special co-processor for fast feature extraction [Graefe, Fleder 1991], was used as a hardware platform in the experiments. The basic cycle time of the vision system is the image field period of the camera's video signal, i.e. 20 ms. Thus, all cycle times are integer multiples of 20 ms. Tracking a distant object requires always less than 20 ms of computing time; the cycle time is in this case 20 ms. The time required for tracking a nearby object depends on the size of its image (see Table 2). On freeways the cycle time is usually between 40 and 60 ms. Only when the distance to the preceding vehicle is exceptionally short, e.g. in stop-and-go traffic, a cycle time of 80 ms may be observed. In such situations it would be preferable anyway to evaluate the image of a wideangle camera instead of the telescopic image used, because the lower edge of the vehicle may not even be visible if a vehicle in such a short distance is imaged by the telescopic camera.
Table 2 Cycle times for tracking nearby-objects Object width
Cycle time
< 45 pixels
40 ms
45 - 90 pixels
60 ms
90 - 120 pixels
80 ms
82 6.4 Reliability of Classification
Parameters such as scene complexity or image quality that are essential for judging the performance of the tracking module are difficult to define quantitatively. Therefore, long image sequences covering a wide variety of situations were evaluated. A 20-min run on a freeway may be used as an example. It included 60,000 images processed in real time without any preselection. Goal of the classification of obstacle candidates is the discrimination between false alarms and actual obstacles. It should be remembered that primarily it must be avoided to ever mistakenly reject any actual obstacle as a false alarm. (The other type of misclassification, accepting a shadow as an obstacle, is much less dangerous.) Table 3 shows the results obtained during the test. All objects that were reported by the detector were correctly classified, and only a very small fraction of the system resources were wasted tracking false alarms. Table 3 Results of object classification during a 20-min run on a freeway
Number of actual objects misclassified as false alarms Percentage of false alarms that were correctly rejected
0
immediately
67%
within a few seconds of tracking
33%
Number of final misclassifications
0
Fraction of time spent tracking false alarms
1.3%
6.5 Quality of Tracking
Because the true motions of the objects tracked in natural scenes were not measured, it is difficult to determine the quality of the tracking, or the accuracy of the localization of object features, quantitatively. Figures 8 and 9 are intended to give at least a qualitative impression of the quality of tracking. The ranges of applicability of the two models, for the near range and for the far range, overlap to a large extent. This allows a direct comparison between the two models by using an object in the overlap range. In each of the Figures, 8 and 9, the left graphs show the image coordinates of the two sides of a car driving in front of the vehicle with the camera. The right graphs show the width of the car's image in pixels. The scene was recorded on video tape while driving in a car, and then evaluated in real time in the laboratory, using the methods for the near range (Figure 8) and for the far range (Figure 9) on the same analog data. Both methods yield nearly identical results. The method for the far range yields slightly smoother curves (in both cases raw, unfiltered data are shown). This is probably due to the robust localization of the lower corners. Altogether both curves indicate that the unfiltered measured data are consistent and free from significant outliers, in spite of the image deterioration caused by the video tape recorder. This may be taken as evidence for a robust tracking.
83
image column 240
/
width of object (pixels)
"'-~ ,,,~_
"~--- I " ~
60
200
-- _ ---"---
\ 40
160 , 120
20 0
100
200
300 400 program cycles
0
100
200 300 400 program cycles
Figure 8
Results of tracking a car in an intermediate distance range using the model for the far range (1 program cycle is equivalent to 20 ms)
image column
width of object (pixels)
240
200
F'-
6(1 ~ ' ~
9, - , - ~
n r -
-
4()
160 /
"~1.. " ' , v ~ ~
120 0
100
200 program cycles
2C 0
100
200 program cycles
Figure 9
Results of tracking a car in an intermediate distance range using the model for the near range (1 program cycle is, on average, approximately equivalent to 40 ms)
7. SUMMARY As an essential part of a driver support system for road vehicles a module for the robust classification and tracking of objects on roads has been realized. Obstacles are recognized in real time and from great distances (200 to 300 m) as required when driving at high speed on freeways. The obstacle recognition module was originally designed for the relatively simple scenes on freeways, but it has proven sufficiently robust to allow the tracking of moving vehicles even in heavy city traffic. The module is based on generic 2-D models optimized for the rear views of cars and trucks. However, the genetic models are sufficiently general to also allow the tracking and classification of various other objects that may occasionally be encountered on roads. For a truly reliable recognition of arbitrary objects the repertoire of internal object models should be expanded.
84
ACKNOWLEDGEMENT Part of the work reported has been performed with support from the Ministry of Research and Technology (BMFT) and from the Daimler-Benz AG within the EUREKA project PROMETHEUS.
REFERENCES Alizon, J.; Gallice, J.; Trassoudaine, L.; Treuillet, S. (1990): Multi-sensory data fusion for obstacle detection and tracking on motorway. In Monique Thonnat (Ed.): Proceedings of the Workshop on Vision (Prometheus, Pro-Art). Sophia Antipolis, pp 179-187.
Carlsson, S.; Eklundh, J.-O. (1990): Object detection using model based prediction and motion parallax. In Monique Thonnat (Ed.): Proc. of the Workshop on Vision (Prometheus, Pro-Art). Sophia Antipolis, pp 139-145. Davies, E.R. (1990): Machine Vision: Theory, Algorithms, Practicalities. London: Academic Press, 1990. Dickmanns, E.D.; Christians, Th. (1989): Relative 3-D State Estimation for Autonomous Visual Guidance of Road Vehicles. Proceedings, Intelligent Autonomous Systems. Amsterdam. Efenberger, W.; Ta, Q.-H.; Tsinas L.; Graefe V. (1992): Automatic Recognition of Vehicles Approaching from Behind. Proceedings of the IEEE Intelligent Vehicles '92 Symposium. Detroit, pp 57-62 Enkelmann, W. (1990): Obstacle detection by evaluation of optical flow fields. In Monique Thonnat (Ed.): Proceedings of the Workshop on Vision (Prometheus, Pro-Art), Sophia Antipolis, pp 146-166. Friedberg, S.A. (1986): Finding Axes of Skewed Symmetry. Computer Vision, Graphics, and Image Processing, 34, Nr. 2. San Diego: Academic Press, pp 138-155. Graefe, V. (1989): Dynamic Vision Systems for Autonomous Mobile Robots. Proceedings, IEEE/RSJ International Workshop on Intelligent Robots and Systems, IROS '89. Tsukuba, pp 12-23. Graefe, V. (1990): The BW-Family of Robot Vision Systems. In O. Kaynak (ed.): Proceedings of the IEEE Workshop on Intelligent Motion Control, Istanbul, pp IP55-IP65. Graefe, V. (1992): Vision for Autonomous Mobile Robots. Proceedings, IEEE Workshop on Advanced Motion Control. Nagoya, pp 57-64. Graefe, V. (1993): Vision for Intelligent Road Vehicles. Proceedings, IEEE Symposium on Intelligent Vehicles. Tokyo, pp 135-140. Graefe, V.; Fleder, K. (1991): A Powerful and Flexible Co-processor for Feature Extraction in a Robot Vision System. International Conference on Industrial Electronics, Control, Instrumentation and Automation (IECON '91). Kobe, pp 2019-2024. Graefe, V.; Jacobs, U. (1991): Detection of Passing Vehicles by a Robot Car Driver. IEEE/RSJ International Workshop on Intelligent Robots and Systems, IROS '91. Osaka, pp 391-396. Hartmann, G.; Mertsching, B. (1992): A Hierarchical Vision System. Proceedings, IEEE Intelligent Vehicles '92 Symposium. Detroit, pp 18-23.
85 Jiang, X.Y.; Bunke, H. (1990): Detektion von Symmetrien polyedrischer Objekte. In GroBkopf (Hrsg.): Mustererkennung 1990, 12. DAGM-Symposium, Oberkochen-Aalen, InformatikFachberichte 254. Berlin: Springer, pp 225-231.
Korn, A. (1989): Zur Erkennung yon Bildstmkturen durch Analyse der Richtungen des Grauwertgradienten. In Burkhardt et al. (Hrsg.): Mustererkennung 1989, 11. DAGM-Symposium, Hamburg, Informatik-Fachberichte 219. Berlin: Springer, pp 507-511. Kiihnle, A. (1991): Symmetry-based recognition of vehicle rears. Pattern Recognition Letters, Volume 12, Number 4. North-Holland, pp 249-258. Kuhnert, K.-D. (1986): A Model-driven Image Analysis System for Vehicle Guidance in Real Time. Proceedings of the Second International Electronic Image Week. CESTA, Nice, pp 216-221. Marola, G. (1989)" Using Symmetry for Detecting and Locating Objects in a Picture. Computer Vision, Graphics, and Image Processing, Vol. 46, Nr. 2, San Diego: Academic Press, pp 179-195. Masaki, I. (1992): Vision-based Vehicle Guidance. I. Masaki, editor. New York: Springer 1992. Mori, It.; Nakai, M.; Chen, H. (1989): A Mobile Robot Strategy: Stereotyped Motion by Sign Pattern. Preprint of the 5th International Symposium of Robotics Research. Tokyo, pp 225-236. Raboisson, S.; Even, G. (1990): Road Extraction and Obstacle Detection in Highway Environment. 2nd Prometheus Workshop on Collision Avoidance (CED 3). Coventry, pp 74-83. Regensburger, U.; Graefe, V. (1990): Object Classification for Obstacle Avoidance. Proceedings, SPIE Symposium on Advances in Intelligent Systems. Boston, pp 112-119. Saint-Marc, Ptt.; Medioni, G. (1990): B-Spline Contour Representation and Symmetry Detection. In G. Goos, J. Hartmanis (Ed.): Computer Vision- ECCV '90, Lecture Notes in Computer Science, Vol. 427. Berlin: Springer, pp 604-606. Seitz, P.; Lang, G.K.; Giiliard, B.; Pandazis, J.C. (1991): The robust recognition of traffic signs from a moving car. In Radig (Ed.): Mustererkennung 1991, 13. DAGM-Symposium, Informatik-Fachberichte 290, Mtinchen, Springer, pp 287-294. Schmid, M.; Thomanek, F. (1992): Real-time Detection and Recognition of Vehicles for an Autonomous Guidance and Control System. Pattern Recognition and Image Analysis, Vol. 3, pp 337-38O. Solder, U. (1992): Echtzeitfa,hige Entdeckung von Objekten in der weiten Vorausschau eines StraBenfahrzeugs. Dissertation, Fakultfit fur Luft- und Raumfahrttechnik der Universit~it der Bundeswehr Munchen. Solder, U.; Graefe, V. (1993): Visual Detection of Distant Objects. IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS '93. Yokohama, pp 1042-1049. Thomanek, F.; Dickmanns, D. (1992): Obstacle Detection, Tracking and State Estimation for Autonomous Road Vehicle Guidance. Proc. 1992 IEEE/RSJ International Conference on Intelligent Robots and Systems IROS '92. Raleigh, pp 1399-1406. Tsinas, L., Graefe, V. (1992): Automatic Recognition of Lanes for Highway Driving. IFAC Conference on Motion Control for Intelligent Automation. Perugia, pp 295-300.
86 Wershofen, K. P. (1992): Real-time Road Scene Classification Based on a Multiple-lane Tracker. Proceedings, International Conference on Industrial Electronics, Control, Instrumentation and Automation (IECON '93). San Diego, pp 746-751. Young, E.; Tribe, R.; Conlong, R. (1992): Improved Obstacle Detection by Sensor Fusion. IEE Colloquium on "Prometheus and Drive", Savoy Place.
Young, G.-S.; Hong T.-H.; Herman, M.; Yang, J.C.S. (1992): Obstacle Detection for a Vehicle Using Optical Flow. Proc. IEEE Intelligent Vehicles '92 Symposium. Detroit, pp 24-29. Yuen, Sh.-Y.K. (1990): Shape from Contour Using Symmetries. In G. Goos und J. Hartmanis (Ed.): Computer Vision- ECCV '90, Lecture Notes in Computer Science, Vol. 427. Berlin: Springer. pp 437-453. Zieike, T.; Brauckmann, M.; v. Seelen, W. (1992): Intensity and Edge-Based Symmetry Detection Applied to Car-Following In G. Sandini (Ed.): Computer Vision- ECCV '92, Lecture Notes in Computer Science, Vol. 558. Berlin: Springer, pp 865-873.
Intelligent Robots and Systems V. Graefe (Editor) 1995 Elsevier Science B.V.
87
Visual Collision A v o i d a n c e b y S e g m e n t a t i o n Ian Horswill* *MIT Artificial Intelligence Laboratory, Cambridge, MA 02139 USA Visual collision avoidance involves two difficult subproblems: obstacle recognition and depth measurement. We present a class of algorithms that use particularly simple methods for each subproblem and derive a set of sufficient conditions for their proper functioning based on a set of idealizations. We then discuss and compare different implementations of the approach and discuss their performance. Finally, we experimentally validate the idealizations.
1
Introduction
One of the major difficulties of visual obstacle avoidance is that obstacles can seemingly have arbitrary appearance. They can be tall or fat, plain or textured, rigid or flexible. Different algorithms can be seen as embodying different assumptions about the appearance of obstacles. Moravec's Stanford Cart [11] can be seen as assuming that obstacles (1)project significantly from the ground plane and (2) are densely covered with visual corners. Kriegman, Triendl and Binford's Mobi system [9] can be seen as assuming that obstacles (1) have at least a certain height and (2) have vertical edges around their perimeters. The stereo algorithm of Storjohann et al. [13] is similar, but does not require the presence of vertical edges, only some sort of texture. The motion-based systems of Coombs and Roberts [2] and of Sandini et al. [ 12] also make these assumptions. Each system is compatible with a particular class of obstacles and incompatible with another. For example, none of these systems would know to avoid an oil slick, a patch of ice, or a delicate cable lying on the floor, because none of these objects would project sufficiently from the floor to be geometrically distinguishable from the floor. The other major difficulty with visual collision avoidance is depth recovery. Both the Stanford Cart and Mobi use stereo triangulation to determine position. Triangulation involves computationally expensive correspondence calculations and generally requires the use of well-calibrated cameras. In this paper, we will discuss a class of algorithms based on assumptions about the appearance of the background. Simply put, the method is as follows: take an image, use some criterion to discard pixels that look like the floor, and avoid driving toward the remaining pixels. Our current implementations are designed to operate in environments with textureless carpeted floors. In such environments, the presence of an edge is a cue to the presence of an obstacle, so the robot need only steer to avoid image edges. The algorithm uses 2D image position to determine obstacle position: higher edges are assumed to be farther away. The algorithm has several advantages: 9 All computations can be performed on 2D rasters. 9 The algorithm requires very little computation. 9 The algorithm is very conservative, preferring false positives to false negatives. We have implemented the algorithm on five different robots here at MIT. The algorithm has also been implemented and tested in other labs [5][ 14]. The algorithm is quite robust and has seen hundreds of hours of testing in many different environments.
88
Figure 1: Polly (left), the first generation vision-based robot, Frankie (center), the second generation, and Elvis (right), the third generation. The next section describes the edge-based algorithm. Section 3 shows that the algortihm's behavior is equivalent to the behavior of a system with a perfect depth map, provided that a set of assumptions about the environment and the robot dynamics are valid. Section 4 gives the details of several implementations done here at MIT. Section 5 discusses their performance and failure modes and concludes that the assumptions about environment and vehicle dynamics are accurate enough for the purposes of the analysis. Finally, section 6 provides conclusions.
2
Algorithm
We assume a forward-looking camera. Let I(t) be the image at time t and let E(.) be some edge detection operator. We define the bottom projection ("bprojection") of a binary image J to be a vector b ( J ) , indexed by the x (horizontal) coordinate, whose x'th element is the height of the lowest marked pixel in the x'th column of J:
b~(J) = min{y : J ( x , y ) =
1}
It will be shown below that under the right conditions, b ( E ( I ) ) is a radial depth map: a mapping from direction to the amount of freespace in that direction. Given the radial depth map b(E(I(t)))), we define the left, right, and center freespaces to be the distances to the closest objects to the left, right and center:
l(t)
=
r(t)
-
c(t)
=
min
b~(E(I(t)))
min
b~(E(I(t))) b~(E(l(t)))
~<xe
x~>~c
min
(1)
where xc is the x coordinate of the center column of the image, and w is a width parameter which in our implementations has been 1/4 the width of the image. The exact choices of the ranges over which the minima are taken is not important. Finally, we define a simple reactive control system for a holonomic base:
dO dt (t)
v(t)
-
co ( l ( t )
r(t))
(2)
=
c. (c(t) - dm,~)
(3)
-
89 where (dO/dt)(t) is the rotational velocity of the robot, v(t) is its translational velocity, d,~n is the closest that the robot should ever come to an obstacle, and co, c~ are user-defined gains. The algorithm turns in the direction of greater freespace at a rate proportional to the difference in freespace on the two sides. It drives forward if the distance to the closest object ahead is greater than d~,~, otherwise it drives backward. Of course, real systems will implement velocity caps (maximum velocities), which introduce min and max terms to the definitions above. We have left these out for simplicity.
3
Analysis
The discussion above leaves open a number of questions: in what environments will this algorithm work? What types of edge detectors can be used for N? Do the gains matter? In this section, we will examine these questions by comparing the algorithm to one which uses the same velocity equations (eq. 2), but which computes the distances l(t), T(t) and c(t) using a perfect oracle. 1 Space precludes a formal presentation in full detail. The interested reader is referred to the proofs of lemma 3 and theorem 2 in [7]. We can represent the perfect-information system schematically as follows: image
oracle
l, r, c
velocity eqs
-d-~' v
An image is fed to an oracle that somehow determines for each direction the distance to the closest object in that direction. The l, r and c values are computed from the resulting depth map and these values are passed to the velocity equations (eq. 2), which control the motors. Suppose that we were to warp the oracle's distance estimates by some strictly increasing function f: q
oracle ~ - ~
/, v, c H velocity eqs b
Because f is strictly increasing, this transformation will preserve the signs of both dO/dr and v. If we model the rotation and translation subsystems as first-order, zero delay systems, and if we can treat them as approximately separable (see section 5), then each subsystem will always converge to the same angle/distance, regardless of our choice of f. Only the rate of convergence will be affected. To say it more abstractly, if we treat the robot-environment combination as a dynamical system (whose state space is the configuration space), then the dynamical system will have the same attractors, repellors, and fixed points, regardless of the choice of f. Thus, the qualitative behavior of the system is invariant with respect to the introduction of a strictly increasing function. The practical import of this is that that we do not need to use a calibrated depth m e a s u r e ~ a n y monotonic measure will do, so long as we adjust d ~ n accordingly. Furthermore, the system should be robust in the presence of variations in sensor parameters. An oracle is a theoretical device generally used to prove that some function is not Turing computable. It is ,assumed to correctly compute the value of some function whether or not that function is formally computable. Oracles ,arenot intended to be physically realizable. We use the term here simply to mean a system that somehow correctly computes the right answer.
90 Now suppose we have an oracle that can identify obstacle pixels in the image: its input is an image, and its output is a binary image whose pixels are 1 iff the corresponding pixels in the input were projected from an obstacle. Call this the "figure/ground" image. It has been known at least since Euclid (see Cutting [3]) that for points projected from a ground plane, image plane height is a strictly increasing function of distance. This result can be extended to show ([7], theorem 2) that in the right environments, the bottom projection of the figure/ground image is equal to the radial depth map, modulo a strictly increasing function. Therefore we can substitute the figure/ground oracle for the distance oracle:
q F/Goracle H bproj H l,r,c H vel. eqs ground-planeconstraint
without altering the behavior of the system, provided that the (GPC) is true of the robot's environment. The GPC requires that all obstacles rest on the ground plane and that their projections be contained within the interiors of their regions of contact with the ground plane (see [7]). To complete the transformation of the distance oracle algorithm into the section 2 algorithm, we need only replace the figure/ground oracle with an edge detector. We would like to justify this substitution and to know what constraints there are on the choice of edge detector. Suppose the surface markings of the ground plane (e.g. the carpet) consist of a high frequency component and a constant (DC) component. Let ta be the lowest spatial frequency in the high frequency component. If the image plane is parallel to the ground plane, looking down at the ground plane from a distance d, then the to component will appear on the image plane with frequency tat
__
tad
Y where f is the lens focal length. Increasing d or changing the camera orientation can only increase this frequency. If we apply a low-pass filter with cutoff w t to the image, only the DC component will be preserved. Since an edge detector is effectively a band-pass filter, it should not respond to regions of the image projected from the ground plane, provided that it's high cutoff is less than w t. Thus we can substitute the edge detector for the F/G oracle:
q edgedet. H bprojH l, r,c~ vel.eqs b backgroundtextureconstraint
provided that the (BTC) holds: that the environment is uniformly illuminated, that the ground plane's surface markings have no components less than w other than DC, and that the scene is viewed from a distance of at least d. 2 The derivation of our algorithm from the oracle algorithm tells us that we can expect them to behave equivalently provided that the GPC and BTC hold of the environment, and that the robot base can be approximated as two non-interacting first-order systems. We will return to these
2This substitution step is not strictly valid, since it assumes that the edge detector fires for all the object pixels. In real life, the edge detector will only fire for the outline of the object and part of its interior. To be strictly valid, the substitution step should replace the F/G oracle with an edge detector combined with a fill algorithm (the fill would mark the object pixels missed by the edge detector). However, it is easy to show that the bprojection is invariant w.r.t, the fill operation, so it can be safely omitted.
91
Robot Polly Gopher Frankie Wilt Elvis
Speed lrn/s 0.2m/s lm/s lm/s 2m/s
Processor TMS320C30 MC68332 MC68332 MC68332 TMS320C31
Frames/s 15 5-10 5-10 5-10 15
Cost $11,000 $700 $7OO $7OO $1,000
Power <40W <3W <3W <3W 12W
Weight <15kg 2kg 2kg 2kg 6kg
Table 1: MIT implementations of the algorithm. Cost, weight and power refer only to the vision system, not to the base or power supply. The frame rates of the 68332-based systems are variable because of varying CCD integration times. assumptions shortly. The derivation also formalizes the computational value of the individual assumptions: the ground plane constraint allows a particular formal simplification of the depth estimation problem, and the BTC allows a particular formal simplification of the figure/ground problem.
4 Implementations To date, we are aware of several implementations of the algorithm that are presently in use. Most have been done at MIT (see table 1), although we know of two others that have been implemented by outside labs [5][ 14].
Polly The first implementation was part of a vision-based mobile robot called Polly (Horswill [6][7]). Polly used vision to seek out visitors and give them primitive tours of our laboratory. Polly processed images at 15Hz using an on-board, off-the-shelf DSP card and frame grabber mounted on an RWI B 12 holonomic base. The complete robot was built for less than $20K US. Polly uses several different algorithms in parallel. The motor outputs of the collision collision avoidance algorithm discussed here are summed with the outputs of a simplified version of Bellutta's vanishing-point algorithm [ 1] for corridor following. All the MIT implementations, except for Elvis and Gopher, run at roughly lm/s. Higher-level navigation was performed by separate processes that attempt to recognize landmarks and choose paths. The details of these other algorithms are outside the scope of this paper. See [7] for further information.
Frankie and Wilt Frankie and Wilt are newer implementations based on new-generation hardware. Polly's major reliability problem was that its off-the-shelf hardware was not intended to run on batteries, nor was it intended to be knocked around and shaken on board a mobile robot. Frankie and Wilt, by contrast, use only a small 68020-class microcontroller board designed to run on batteries. The board is much more robust, both mechanically and electrically, than Polly's processor. Image acquisition is performed by directly interfacing a CCD image sensor to the processor bus by way of an A/D converter (see Gavin and Yamamoto [4], Horswill and Yamamoto [8]). The CCD is mounted on a pan/tilt head built from Futaba model airplane servo motors. The complete vision system is less than $700 US (parts cost). The collision avoidance algorithm was modified to take advantage of the pan/tilt head; Instead of using the center distance to control
92 the rate of translation, the center distance is used to tilt the head up or down so as to keep the nearest obstacle centered in the image. The translate velocity is then based on the tilt angle of the camera. This increases the effective field of view of the system and helps compensate for various mechanical deficiencies (see below). The Polly, Frankie and Wilt implementations work by subsampling the image to 64 x 48 pixels, and then smoothing the result with a 3 x 3 filter. Edge detection is done by computing the sum of the absolute differences of each pixel with its upper and right neighbors. If the sum is greater than a threshold (15 grey levels out of a possible 510 on Polly, 22 grey levels for Frankie and Wilt which have more camera noise). The Polly system is described in detail in [7]. The important parts of the code for the Frankie system are given in section 6. Elvis
The Frankie and Wilt implementations were more robust and were dramatically cheaper, however, their 1-2MIP processors severely limited the processing they could perform. Elvis, our third-generation robot is based on much higher-performance (50MFLOP) hardware that is still relatively inexpensive ($1,000 parts cost). Because Elvis can use a higher frame rate and has a lower center of gravity, it can run dramatically faster than the other implementations. We have reliably achieved speeds of up to 2m/s. The Elvis vision hardware, designed by Chris Barnhart, consists of a TMS320C31 floatingpoint DSP with 128KW of ram, a memory-mapped NTSC color frame grabber, and a 24 bit memory-mapped video display for debugging. We hope to use the color capabilities of the machine to simplify place recognition.
5
Evaluation
The algorithm has been thoroughly tested, having seen many hundreds of hours of service. Polly has given over 100 tours and has patrolled the lab for periods of up to two hours before exhausting its motor batteries. Wilt, while more limited in functionality, is light enough and robust enough to permit much greater testing~ It is reliable (and safe) enough that researchers regularly bring their children to play with it. The researchers often demo the system themselves, even though they did not design it and do not know how it works. Frankie and Wilt have also been demo'ed off site by researchers unfamiliar with their workings. The algorithm is dramatically more reliable than the sonar-based systems running in the lab, which are very sensitive to obstacle geometry. In all, the various implementations of the algorithm have been tested in a few dozen different environments: 9 Various floors of our lab (see below). 9 Various class rooms and auditoriums of MIT. 9 The AAAI-93 convention center, exhibition halls, and hotel. 9 The Brown University Computer Science Department. 9 West Point Military Academy 9 The New Hampshire offices of DEC 9 A local elementary school.
93
5.1
Environmental approximations
The analysis of section 3 showed that the behavior of our algorithm is equivalent to the behavior of a system with a perfect distance oracle, provided that the environment satisfied certain properties: that all obstacles rest on a ground plane and have a certain type of shape (the GPC), and that the ground plane was textureless and uniformly lit (the BTC). These are idealizations; real environments contain shadows, real carpets have stains. The usefulness of the algorithm is determined by the range of environments which satisfy these properties well enough to allow the system to work. The ground-plane constraint has turned out to be mostly non-problematic, even though many objects, such as chairs and tables, violate its shape restrictions. It is particularly unproblematic for Frankie because it tilts its head downward as it approaches an obstacle. In the limit, when the head is pointed straight down, the equivalence of the bprojection to a depth map can be shown to be independent of object shape. Thus the active head actually makes the system somewhat more robust. Frankie's biggest problem is that its head happens to be exactly at desktop level. As Frankie approaches a desk and looks downward to keep the desk legs in view, the desktop goes out of view and Frankie hits itself in the head. The background-texture constraint is more complicated. All floor surfaces in our building satisfy its surface marking requirements, but the basement and top floor both have shiny tile floors that reflect images of the overhead lights like mirrors. The robot treats these reflections as obstacles and tries to avoid them. Depending on room geometry, the robot may successfully navigate or may back away from the lights until it collides with something in the rear (Frankie can't see backward). Another problem occurs when a room has two different carpets. Even though each carpet satisfies the BTC, the boundary between them looks like an obstacle to the robot. Polly uses a separate algorithm to identify specific carpet boundaries and override the edge detector, but we hope to handle the situation more intelligently on Frankie. Shadows have not been a problem in our environment, although scenes which are illuminated both by sunlight and by room light often have illumination gradients strong enough to trigger the edge detector. Such scenes also cause the automatic gain control of the camera to fail. Although such experiments are purely anecdotal, we have also run the robots in a number of off-site environments. Frankie has been run successfully at a number of sites on campus: classrooms, the main auditorium, and the so-called "infinite corridor." One problem in auditorium environments is that the overhead lights are often point-sources which generate strong shadows. These can sometimes confuse the robot. Polly has been run successfully in those parts of the Brown University computer science building that were well enough lit for the video camera to operate. Frankie has also been taken off-site by other researchers to give demos. A smaller robot using an early version of the Frankie vision hardware was run on-site at last year's AAAI. The robot suffered many hardware problems and did not perform at all well in the office environment that had been carefully engineered for the sonar-based robots of the AAAI robot contest. However, it was the only robot that could function effectively in the real lounge area, the sessions, or the conference hotel.
94
5.2 Dynamicapproximations Our analysis assumed that the translation and rotation systems of the robot are independent and that they are both first order dynamic systems. In real life, the two systems are coupled: rotations in response to the l and d distances change the c distance, while translations in response to c can change the I and r. This coupling raises the possibility of oscillation. Fortunately, we have not experienced this problem in practice because the robot is almost always either blocked or in rotational equilibriummit tends either to translate or rotate, but rarely both. Thus the coupling between rotation and translation rarely shows itself. While most modern robot bases have good velocity control mechanisms, they can only generate finite accelerations, so the first-order approximation is also only partly accurate. Another limitation of our robots is that their centers of gravity are high enough for them to tip over when the robots stop suddenly. Deceleration limits cause an interesting problem: if the robot fails to detect a faint obstacle until it is nearby, the robot may not have time to stop fully before the obstacle goes out of view. Since the algorithm is purely reactive, it forgets the out of view obstacle and accelerates into it. Any reactive control system will have this problem, whether it is based on a first-order model or not, so the problem does not really lie with our analysis. On Frankie, the problem is greatly reduced because it can compensate with its pan/tilt head. The low mass head is not subject to the same acceleration restrictions as the body, so Frankie can always keep the object in view. Frankie is also drastically lighter and has a lower center of gravity, so its dynamic problems are less to begin with. Elvis, being still lighter and lower to the ground, can safely run at up to 2m/s and yet stop without falling.
6
Conclusions
We have presented a simple collision avoidance algorithm and an analysis of its behavior based on a set of assumptions regarding the environment and the robot's dynamics. We also presented various implementations of the algorithm and evaluated their performance and the empirical validity of the assumptions used in the analysis. Experiments show that a surprisingly wide range of environments satisfy the constraints closely enough to allow the algorithm to work effectively and that the real robot dynamics generally match the approximations used in the analysis. Thus the environment and dynamic models, while imperfect, are sufficient for the analysis. The algorithm is simple, conservative, extremely efficient, and quite reliable within our local environment. In our environment, it is far more reliable than the sonar-based methods. The algorithm belongs to a general class of algorithms that exploit the fact that it is often easier to recognize the background than to recognize an obstacle. Depending on the context, a robot might use any number of cues to distinguish the background from the obstacles. The algorithm presented here uses an edge detector, and therefore relies on an assumption about the surface markings of the background. One can easily imagine other algorithms using other cues (e.g. color or scale) in other contexts. To change the cue, one need only change the final substitution in the analysis given in section 3. For example, one could substitute a color-based identification system (e.g. Swain [ 15]) for the edge detector:
q
H
H ',',oH ve' b
95 if the BTC didn't hold of the environment but the floor could be relied on to have a predictable color. In theory, one could even use an adaptive system that dynamically switched between an array of cues, depending on local context, but this is more speculative. The reader may object that this algorithm is just a particular hack for a particular environment. In a sense, this is true. However, it is a hack which has been demonstrated to operate in a larger number of environments than any other vision-based robot of which we are aware. In addition, nearly any vision system will make some set of assumptions about its environment (see Marr [ 10] for the canonical discussion of this point). A stereo or motion system, for example, requires the presence of texture not only on the obstacles, but on the floors and walls also, at least if it is to make accurate depth readings there. This property simply isn't true of our laboratory. We suggest that the issue is not so much whether we should specialize our systems to tasks and environmentsmsince nearly all systems are specialized--but what specializations are most useful.
Acknowledgments Support for this research was provided in part by the Advanced Research Projects Agency under Office of Naval Research contract N00014-91-J-4038 and in part by the NASA Jet Propulsion Laboratory under contract 959333. Additional support was provided by Matsushita Electric Industrial Co., Ltd.
Appendix: code excerpts from Frankie Below is most of the important code from the Frankie system. Some variable names have been changed to be more consistent with the text. Device drivers and obvious functions, such as a b s have been omitted. The code is written in C and makes extensive use of pointer arithmetic to speed up the inner loops of the vision code. Note that only 1D iterations are performed over the images, rather than 2D iterations. This makes the code somewhat faster and simpler, but does produce erroneous results at the edges of the image. The rest of the code is written to ignore the values it finds there.
#define image_width 64 #define image_height 48 #define image size (image_width*image height)
* ipointer is the type of a pointer into an image. * pointer() casts an image into a pointer. */ typedef unsigned char image [image_height] [image_width ] ; typedef unsigned char *ipointer; #define pointer(a) ((ipointer) (a))
96
/* I t e r a t i o n m a c r o s */ # d e f i n e c o u n t d o w n ( v a t , count) \ for (var : (count)-l; v a r > = 0 ; var--) # d e f i n e d o t i m e s ( v a r , count) \ for (var = 0; v a r < ( c o u n t ) ; var++) /* P o i n t e r a r i t h m e t i c for i m a g e p o i n t e r s */ #define shift_left(i,pix) ((i)-(pix)) #define shift right(i,pix) ((i)+(pix)) #define shift_down(i,pix) ((i)+image_width*(pix)) #define shift_up(i,pix) ((i)-image_width*(pix)) /* T h i s is a 3x3 s e p a r a b l e l o w - p a s s void smooth(image in, i m a g e out)
filter
*/
{ image scratch; i p o i n t e r left, int c o u n t e r ;
middle,
right,
up,
down,
i,
i = pointer(in) ; s = p o i n t e r (scratch) ; o = s h i f t _ r i g h t (s, i) ; left = i ; middle = shift right(i,l) ; right = shift right(i,2) ; countdown(counter, image_size-2) *o++ = ( * l e f t + + + * r i g h t + + + ( * m i d d l e + + >> 2;
o,
<
o = s h i f t _ d o w n ( p o i n t e r (out) , i) ; up = s; middle = shift down(s,l) ; down - shift down(s,2) ; countdown(counter, image size-(image_width*2)) *o++ : ( ( i n t ) * u p + + + ( i n t ) * d o w n + + + ( ( i n t ) * m i d d l e + + << I)) >> 2;
void
edge_detect
(image in, i m a g e out, int e d g e t h r e s h o l d )
{ i p o i n t e r up, int c o u n t e r ,
left, center, c, dx, dy;
i ,o;
s;
97
center : i : pointer(in) ; o : p o i n t e r (out) ; up : s h i f t _ u p (i, i) ; l e f t = s h i f t l e f t ( i , i) ; countdown(counter, i m a g e size) c = *center++; d x = (int) ( * l e f t + + ) -c; dy = (int) (*up++) -c; *o++((abs(dx) + abs(dy))> ?255-0;
#define
thresh
edge_threshold)
22
/* Do all t h e v i s i o n to and center distances v o i d do v i s i o n ( )
compute */
left,
right,
{ grab frame(framebuf) ; s m o o t h ( f r a m e b u f , bufl) 9 e d g e _ d e t e c t (bufl, b u f 2 , t h r e s h ) ; b p r o j e c t i o n (buf2, d e p t h _ m a p ) ; /* c o m p u t e l, r, a n d c d i s t a n c e s */ left=region_min ( d e p t h _ m a p , 4,25) ; r i g h t = r e g i o n _ r a i n ( d e p t h _ m a p , 35,25) ; center=region_min ( d e p t h _ m a p , 20,24) ;
int
pan,
/*
tilt-
T i l t t h e c a m e r a so as obstacle */ void cameracontroller()
to
center
the
nearest
{ int int
vertical_error = center - desired_height; control_signal = min(max(-(vertical_error>>l),-l),5);
/* t i l t t h e c a m e r a u p / d o w n to c a n c e l o u t control_signal *,I tilt camera relative( control signal) ;
98
/* R o t a t e the b a s e to b a l a n c e the free s p a c e on the left a n d right. D r i v e f o r w a r d / b a c k to k e e p the n e a r e s t o b s t a c l e at a c o n s t a n t d i s t a n c e */ void
base
controller()
{ int int int if
rotational_error = right-left; translational_error = center - desired_height; tilt_error = 20-current camera_tilt();
(current c a m e r a _ t i l t ( ) = = b o t t o m _ l i m i t ) /* s p e e d up turns w h e n v e r y c l o s e to o b s t a c l e this m a k e s it r e s p o n d f a s t e r w h e n it r e a c h e s a d e a d end */ rotational error = (rotational_error>0) ?
4000- (-4000); set_rotate_velocity
(rotational_error*200)
/* If too c l o s e to object, m o v e b a c k full this p r o b a b l y isn't n e e d e d n o w that we m o v e the h e a d */ if (center< d e s i r e d h e i g h t -i0) set_translate_velocity(-1000); else set_translate velocity(tilt error*100);
/* D r i v e r main()
loop
*/
{ w h i l e (I) { dovi s ion ( ) ; c a m e r a _ c o n t r o l l e r () ; base controller() ;
}
; speed can
99
References [ 1] R Bellutta, G. Collini, A. Verri, and V. Torre. Navigation by tracking vanishing points. In AAM Spring Symposium on Robot Navigation, pages 6-10, Stanford University, March 1989. AAAI. [2] David Coombs and Karen Roberts. "bee-bot": using peripheral optical flow to avoid obstacles. In Proc. of the SPIE Conf. on Intelligent Robots and Computer Vision XI: Algorithms, Techniques, and Active Vision, (Boston, MA, November 15-20, 1992), 1992. [3] James E. Cutting. Perception with an Eye for Motion. MIT Press, 1986. [4] Andrew S. Gavin and Masaki Yamamoto. A fast, cheap, and easy system for outside vision on mars. In Intelligent Robots and Computer Vision XI, Cambridge, MA, September 1993. SPIE. [5] Takashi Gomi and Koh ichi Ide. Vision based navigation for an office messenger robot. In this volume, 1995. [6] Ian Horswill. Polly: A vision-based artificial agent. In Proceedings of the Eleventh National Conference on Artificial Intelligence, pages 824-829. AAAI, MIT Press, 1993. [7] Ian Horswill. Specialization of perceptual processes. PhD thesis, Massachusetts Institute of Technology, Cambridge, May 1993.
[8] Ian Horswill and Masaki Yamamoto. A $1000 active stereo vision system. In Submitted to CVPR-94, 1994. [9] David J. Kriegman, Ernst Triendl, and Tomas O. Binford. A mobile robot: Sensing, planning and locomotion. In 1987 IEEE lnternation Conference on Robotics and Automation, pages 402--408. IEEE, March 87. [ 10] David Marr. Vision. W. H. Freeman and Co., 1982. [ 11] Hans R Moravec. The stanford cart and cmu rover. Technical report, Robotics Institute, Carnegie-Mellon University, February 1983. [ 12] G. Sandini, E Gandolfo, E. Grosso, and M. Tistarelli. Vision during action. In Yiannis Aloimonos, editor, Active Perception, chapter 4, pages 151-190. Lawrence Erlbaum Associates, Inc., 1993. [ 13] K. Storjohann, T. Zeilke, H. A. Mallot, and W. von Seelen. Visual obstacle detection for automatically guided vehicles. In Proceedings of the IEEE International Conference on Robotics and Automation, pages 761-766, May 1990. [ 14] Michael Swain. Personal communication. [ 15] Michael J. Swain. Color indexing. Technical Report 390, University of Rochester Computer Science Department, November 1990.
This Page Intentionally Left Blank
Intelligent Robots and Systems V. Graefe (Editor) 9 1995 Elsevier Science B.V. All rights reserved.
101
Using Perceptions to Plan Incremental Adaptations A.J. Hendriks and D.M. Lyons Philips Laboratories, Philips Electronics North America Corporation, 345 Scarborough Road, BriarcliffManor, NY 10510, USA. [email protected], [email protected], com
Abstract
A robot system operating in an environment in which there is uncertainty and change needs to combine the ability to react with the ability to plan ahead. In a previous paper we proposed a solution to the problems of integrating planning and reaction: cast planning as adaptation of a reactive system. The planner, asynchronously tuning the reactor, decides on the appropriate parts of the reactor to be modified based on perceptions: information gathered in the reactor for the express purpose to inform the planner. In this paper, we show the benefits of using perception to adapt the reactor where it needs to be updated most and present our first experimental results from the planner-reactor architecture. 1.
INTRODUCTION
The importance of integrating deliberative ("planning") capabilities and reactive capabilities when building robust, 'real-world' robot systems is becoming widely accepted [1, 6, 13]. Purely deliberative approaches (e.g., [4]) are good at determining globally good courses of action for a robot, selecting an optimal assembly sequence for example. They are not very robust, however, in the face of changing or uncertain conditions in the robot's environment. Typically, a deliberative system expends a lot of effort deriving a good course of action for the robot, only to find on starting that the environment has changed and rendered the actions unsuitable [13]. Purely reactive approaches (e.g., [2, 7]) operate very robustly in some uncertain and dynamic e n v i r o n m e n t s - namely those for which the reactions have been manually p r e - p r o g r a m m e d - but lack any ability to plan ahead. However, many application domains such as mobile robots [6], autonomous explorers [1], emergency response planning [13] as well as our domain, robotic kitting, demand the integration of deliberative and reactive capabilities. In [11] we suggested a straightforward approach to this integration: allow the deliberative component to incrementally adapt the reactive component to suit
102 current goals and operating conditions. In subsequent papers, we looked at the theoretical issues of incremental adaptation [9] and of safely making changes to an operating reactive system [10]. Here we present our first performance results, using our approach to construct a robot kitting work cell. Some alternative approaches to this integration have been suggested. Schoppers[14] amongst others has proposed an 'off-line' generator for reactive systems; Xiaodong and Bekey [16] also suggest something similar for an assembly cell equipped with a reactive scheduler. This approach works well only when the deliberative component can be separated 'off-line'. We address the problems that arise when both reactive and deliberative components need to be 'on-line'. Connell's SSS [3] addresses this integration for mobile robots. In SSS, though, the deliberative component is restricted to enabling/disabling behaviors, while we need to be able also to generate new behaviors. Bresina & Drummond's ERE [1] comes closest to our approach. A key difference is that our domain, kitting, is a repetitive activity. Our concept of the improvement of a reactive system exploits this repetition, whereas ERE was designed for once-off activities. We begin with a summary of the planner-reactor architecture we developed to implement of our approach. Section 3 describes the kitting robot domain. Sections 4 and 5 provide more details of our implementation as a prelude to our experimental results in Section 6.
Adaptations
PLANNER
Actions
REACTOR
WORLD
J Perceptions
Sensing
Figure 1: Planner-Reactor-World System
2.
THE P L A N N E R - R E A C T O R A R C H I T E C T U R E
Our definition of planner is a system that continually modifies a concurrent and separate reactive system (the reactor) so that its behavior becomes more goaldirected. Figure 1 illustrates this architecture. The reactor contains a network of reactions: hardwired compositions of sensor and motor processes. In our framework, these are sensory processes composed with effector (or action) processes in such a fashion that should the sensory process detect its triggering condition, then it will in turn trigger the effector process. The reactor has two key properties:
103 1.
2.
It can produce action at any time. Unlike a plan executor, a reactor can act independently of the planner; it is always actively inspecting the world, and will act should one of its reactions be triggered. A reactor should produce useful behavior even without a planner~ It is a real-time system. The central objective of the reactor is to produce the appropriate action at the appropriate time. To make this possible, all the 'knowledge' in a reactor is encoded into reactions and cross-connections between them, and annotated with time constraints.
The reactor encapsulates what would typically be called the plan in more traditional approaches. By making it a completely separate system, we remove the delays involved when a general deliberation subsystem is always interposed between sensing and action. The reactor is also responsible for the default behavior of the system in the following sense: It is initialized with a number of generally useful behaviors already in place, for example, distinguishing objects from background, locally avoiding obstacles, and scanning its sensors in the environment. Unlike a plan executor or a hierarchical system of planner and reactive system, our reactor acts asynchronously and independently of the planner. It is always actively inspecting the world, and will act should one of its reactions be triggered. A reactor should produce timely, useful behavior even without a planner. Rather than viewing the planner as a higher-level system that loads plans into an executor, we see the planner as an equal-level system that continually tunes the reactor to produce appropriate behavior. Interaction between the planner and reactor is entirely asynchronous. The planner continually determines if the reactor's responses to the current environment would indeed conform to the given objectives. If not, then the planner makes an incremental change to the reactor configuration to suit the reactor's behavior better to the objectives. As indicated in figure 1, there are two routes of interaction between planner and reactor. 1. 2.
Adaptations These are structural alteration commands issued by the planner causing changes in the reactor structure. Perceptions These are sensory data collected by the reactor to be sent to the planner.
An adaptation is essentially an instruction to delete part of the reactor structure or to add in some extra structure. Each individual adaptation should be small in effect and scope; large changes in reactor behavior only come about as the result of iterated adaptation. Large adaptations would be equivalent to downloading 'plans' to the reactor; something we need to avoid. A planner can only adapt a reactor in a well-defined way, as is explained shortly. This allows a formal analysis of the adaptation mechanism. To distinguish between reactor and planner sensory data, we refer to any data collected by the reactor as sensory data, but sensory data collected to be sent to the planner will be referred to as perceptual data. In the remainder of this section, we briefly reiterate the details of the planner-reactor approach as presented in our previous work[9, 10]. We begin by introducing the formal language ~U we use to describe the reactor and its incremental adaptation.
104
2.1. The T5 Model The ~5 model [8] was developed to r e p r e s e n t and analyze the kind of prog r a m s involved in sensory-based robotics. This language allows a user to construct programs by 'gluing' together processes in various ways. This style of language is called a process-algebra language. These languages have the additional a d v a n t a g e t h a t they facilitate the m a t h e m a t i c a l analysis of programs. A description of a process, or n e t w o r k of processes, is called a schema. For example, L o c a t e m (X) denotes a process t h a t is an instance of the schema L o c a t e with one i n p u t p a r a m e t e r m and one result x. Networks are built by composing processes together using several kinds of process composition operators. This allows processes to be ordered in various ways, including concurrent, conditional and iterative orderings. At the bottom of this hierarchy, every n e t w o r k m u s t be composed from a set of atomic, pre-defined processes. The composition operations are the following:
1. 2. 3. 4. 5.
sequential concurrent conditional negation disabling
A; B,
do A t h e n B.
(A I B), A: B,
do A and B concurrently.
~A,
do A but fail if A succeeds and vice-versa.
A#B,
do B only if A succeeds. do A and B concurrently until one terminates.
We use two r e c u r r e n t operators:
6. 7.
synchronous recurrent A: ; B =A: (B; {A: ; B) ), while A succeeds do B. asynchronous recurrent A: : B =A: (B I (A: 9B) ), while A succeeds s p a w n off B.
It is i m p o r t a n t to represent how process networks evolve over time, as processes dynamically t e r m i n a t e or are created. This is captured by the evolves operator: We say t h a t process A evolves into process B u n d e r condition ~ if A can possibly become equal to B w h e n condition a occurs; we write this as A -~ B. A process must evolve to one of the set of processes to which it can possibly evolve.
2.2. I n c r e m e n t a l R e a c t o r A d a p t a t i o n We define the reactor R as a set of concurrent reactions, and a well-defined interface t h r o u g h which the p l a n n e r can modify the reactor structure:
REACTOR
=
(EI#pI I ......I En#pn
I (ADD(k): ~ (Ek#p k) )
(1 )
w h e r e there are three m a i n items: 1.
pi is a reaction or reactor segment.
2.
E i is a g u a r d process by which the p l a n n e r can remove pi from the reactor.
3.
ADD is the interface t h r o u g h which the p l a n n e r (by the definition of : :) can add new guarded reactions.
A reactor does not contain/execute plans in the same sense t h a t a classical plan executor does, i.e., as a d a t a structure. The reactions are represented procedurally, so the reactor is more like a compiled controller. Nonetheless, certain
105
p a r t s of the reactor s t r u c t u r e can be identified as corresponding to the instructions necessary for a specific t a s k or subtask; we call these parts reactor segments. It is possible to write down how the reactor is affected by the changes described in items 2 and 3 of eq. 1 above using the T5 evolves operator: ( R
Dk
[ Ek#p k )
--> Ak -->
R
where Dk is where A k is
R ( a
(2)
I Ek#p k )
STOP, and
gk --+ ADD(k> --->
STOP.
Saying t h a t a process P successfully t e r m i n a t e s (under some condition) is the s a m e as saying t h a t P evolves to STOP (under t h a t condition). So in eq. 2 above, Dk can be used to delete reactions, and A k can be used to add reactions. Using the evolves operation, we can express the process of reactor a d a p t a t i o n as follows: R
~
(3)
R'
w h e r e a' is the a d a p t e d reactor and a is some combination of A k and Dk conditions. In [10] we describe how to m a k e these a d a p t a t i o n s in a safe m a n n e r despite the fact t h a t the p l a n n e r does not know w h e n or which reaction m i g h t be firing. Ideal React.or. The u l t i m a t e objective of the p l a n n e r is the construction of a reactor t h a t achieves its goals not only in the current environment, but because change is possible, in all e n v i r o n m e n t s t h a t m i g h t occur, as indicated by the e n v i r o n m e n t model. A reactor t h a t fulfills this criterion is called an ideal reactor, and is similar in behavior to Schoppers' universal plans[14]. It is unrealistic to suppose t h a t a p l a n n e r will always be able to generate the ideal reactor in one step, and in [9] we introduced the more restricted m-ideal reactor am. In p l a n n e r terms, m is a set of assumptions u n d e r which the reactor R~ has been constructed. We propose the following i n c r e m e n t a l reactor construction strategy: m is initially chosen to allow the p l a n n e r to quickly produce a goal achieving reactor; t h e n co is g r a d u a l l y relaxed over time. The evolution in eq. 3 can be used to r e p r e s e n t this s t r a t e g y as follows: R m1
for
~
e(col) >
Rm2
e(co2)
a2 --+ >
a--+ n
... ....
>
R*
(4)
e(O)
where the ai is an a d a p t a t i o n c o m m a n d g e n e r a t e d by the p l a n n e r and e0 is an evaluation function. The a s s u m p t i o n relaxation priorities are governed by the ordering on o~ dictated by e0. The conditions t h a t trigger the p l a n n e r to produce an improved reactor are described in [9] and the constraints on convergence to a* overviewed.
106 3.
THE KITTING ROBOT PROBLEM
Assembly components are fed from stores to a kitting station on a conveyor belt. The kitting station consists of one or more kitting robots. Each kitting robot must take parts off the belt, place them in the appropriate slots of a kitting tray, and then place the tray onto the belt. The trays are stored in a stack in the robot's work space. The product we are kitting is a small DC servomotor manufactured by Philips. It has three parts: a cap, a motor and a body, all of which have a number of variants. The planner-reactor approach provides a way to incrementally construct reactive systems with improving performance. At the heart of this iterative mechanism is the concept of characterizing the environment by a set of assumptions and methods to detect whether they hold or not. Kitting Assumptions. The set of assumptions we have developed for the kitting environment is described below in order of likelihood of not holding: the ordering we have chosen for the planner to relax them (i.e., the e0 function in eq. 4). The planner uses these assumptions to help it to quickly build a reasonable reactor. Later, as it has time, it relaxes assumptions and fashions improved versions of the reactor. When the environment goes through regimes where different subsets of assumptions hold, the planner can capitalize on this, especially when new factory goals have been received and resultant changes in behavior are necessary. Some of the assumptions, such as that of the quality and substitutability of parts, must eventually be relaxed to get to a robust work cell. Some other assumptions, such as that of cooperation in kit assembly and tray disturbance, describe contingencies that make a more versatile work cell, but are not necessary for a robust system. Still other assumptions pertain to unusual operating conditions, such as the assumptions of no downstream disturbances. The kitting robot problem uses nine assumptions in total. In our experiments, however, only the following five were relaxed: 1. 2. 3. 4. 5.
Assumption of Parts Quality (AQ): All the parts coming into the kitting work cell are of good quality and do not need to be tested. Assumption of non-substitutability of parts (AS): Each part has only one type. Assumption of no parts motion (AM): The kit parts do not move around once delivered on the belt to the work cell. Assumption of no downstream disturbance (ADD): Downstream automation is always ready to receive finished kits. Assumption of parts ordering (AO): The robot cannot modify the preplanned parts ordering.
In addition to the iterative relaxation of assumptions based on likelihood, the planner can also be forced to relax assumptions. Whenever the planner builds a reactor that depends upon a particular assumption holding, it needs to embed a monitor process that can detect whether the assumption actually holds in practice or not. The monitor process triggers a 'clean up' action if the assumption ever fails, and notifies the planner that it needs to relax this assumption next.
107 4.
SITUATIONS
Reactor situations are a m e c h a n i s m to group related reactions together in a hierarchical fashion. This m o d u l a r i t y is i m p o r t a n t because it allows (human) designers to more easily u n d e r s t a n d the reactor and diagnose any problems t h a t m a y occur; it provides the p l a n n e r with a unit around which to define safe changes to the reactor s t r u c t u r e [10] and it provides a way to give computational resources to those reactions t h a t currently need it, while 'suspending' others. Intuitively: a situation being active expresses the a p p r o p r i a t e n e s s for the reactor to enable the set of reactions associated with t h a t situation. Situations can be nested hierarchically, and m a n y situations m a y be active at one time so t h a t the execution of their reactions will be interleaved. We introduce a small set of basic processes with which to build reactor situations in T5 (table 1). For example, an instance of a situation is asserted by the execution of the A S S E R T s p l , ~ 2 .... process, w h e r e p l, p 2 .... are the values for the p a r a m e t e r s associated wit~ the situation. The SIT s ( k , p l , . . . ) process, w h e n executed, suspends itself until an instance of situation s is asserted. It t h e n t e r m i n a t e s and passes on the details of the situation instance as its results.
* ASSERTS,p Assert situation s and initialize the situation p a r a m e t e r s p,. . . . . . This t e r m i n a t e s w h e n the situation is t e r m i n a t e d and stops or aborts depending on w h e t h e r FAIL or SUCCEED was used. * FAILS, k T e r m i n a t e s instance k of situation s with fail status. * SUCCEEDs, k T e r m i n a t e s instance k of situation s with success status. * SIT s ( k , p l , p 2 , . . . ) T e r m i n a t e s if an instance of situation s is currently asserted. k is the instance n u m b e r and p l , p2, ... are the current values of the situation parameters. Table 1: Situation basic processes We use the S I T s process to ensure t h a t a reaction associated with a situation is only 'enabled' w h e n an instance of t h a t situation is asserted. For example, let p1, . . . . , p n be the reactions for situation s, t h e n we would r e p r e s e n t this in the reactor as follows:
PS
=
SITs:;pI I SITs:;P2 I ""I SITs:;Pn
(5)
The ':;' operation ensures t h a t as long as the situation is active, t h e n the reactions are continually re-enabled. Note t h a t once enabled, a reaction cannot be disabled until it has t e r m i n a t e d . The asserting and t e r m i n a t i o n of instances of situation s h a p p e n as the side effects of ASSERTs and FAIL s or SUCCEEDs processes in reactions in other situations.
108
5.
THE PLANNER
The planner needs to adapt the reactor to suit new goals and/or changing operating conditions. New goals may be issued by factory management. Through feedback from perceptions t h a t the planner has inserted in the reactor the planner is made aware of the current operating conditions, e.g. the current set of assumptions t h a t hold in the environment. Based on these two sources of information, the planner reasons to arrive at an incremental set of adaptations, such t h a t the reactor's behavior becomes ever more goalworthy. This section outlines the principles of operation of the planner with emphasis on its interactions with the reactor, a detailed description of the planner is given in [9].
5.1. The P l a n n e r Algorithm At every planner iteration t, based on the environment model TM~ and current goals Gt, the planner generates what we call an expectation T_t:an abstract description of the changes it expects to make to the reactor to achieve Gt. At every iteration, the combination of the reactor model and the expectation is exactly the co-ideal reactor (where co is the current set of assumptions the planner is working with).The process of incremental improvement of the reactor can now be seen as incremental reduction of the planner's expectation by some ATt and corresponding increment of the reactor ARt, until when Tt is null, the current reactor is exactly the ideal reactor. Additionally, when assumptions are used to construct the reactor increment, the planner will insert assumption monitors AIt into the reactor. Similarly, if the planner needs specific information for its own planning purposes, it can insert additional perception processes APt, processes t h a t collect information in the reactor and report back to the planner. Goals G
T,
r I I I I I Z~
t
I
i i i
current reactor
Rt
AIr ~
perception monitors
SI
t
I I I I
Pt
=1
Wt
PS i
PLANNER
',
Fit
REACTOR
.i
Figure 2: Planner-Reactor Interactions
WORLD
12
109 The input/output interactions between planner and reactor are as shown in Figure 2. The planner incrementally produces changes in the reactor structure ARt plus the associated assumption monitors AI t and additional perception processes APt. It receives back information from existing perception process vP t and any signals from assumption monitors that have failed FI t .
Figure 3: The Planner Architecture
5.2. The p l a n n e r architecture. The block diagram of the planner's internals is shown in figure 3. Based on the environment model TM and the goals G,, the planner accumulates a current Problem-Solving Context (PSC). Within that PSC, the planner generates an abstract plan, the expectation Tt. The current set of assumptions ~ s ~ is used to filter the environment model to generate a simpler model in the PSC. The current expectation Tt is analyzed to determine how much of it can be reduced A~. This incremental expectation is decomposed (in the planning sense) in the context of this filtered world, to generate an adaptation of the reactor ARt and the set of assumption monitoring processes to go along with it AIt. A set of perception requests can also be generated at this point APt; these requests will return data that clarify uncertainty in T M t and allow further decomposition of ~. The assumption monitor processes may signal that the assumptions they protect have been violated; this results in ~ 5 ~ being decremented by that assumption, and it may thus result in being revised and probably increased again.
110
5.3. Expectation reduction. To reduce this expectation, the planner reasons within a Problem Solving Context consisting of the relevant parts of the environment model TM, the action repertoire A and the assumptions o~ The outcome will be a reactor adaptation ARt, the reactions necessary to implement the expectation reduction ATt. The manageable size of AT_t is constrained by the following factors: 1.
The response time of the planner. This can come from the design specification of the planner, or from the temporal constraints on goals and actions in the plan. 2. The effects of uncertain knowledge on the plan. The planner may need to issue perception requests before more of the expectation can be reduced. 3. Resource use and mappings. Initially, the planner reduces an expectation by looking for an abstract plan that achieves the given goals. This abstract plan, representing the structure of a more detailed plan, is maintained through insertion of situations into the reactor. The more detailed actions corresponding to the expansion of an abstract action are encoded as reactions in the situation, representing the abstract action. Any reactor segment sent to the reactor must be directly executable. However it need not necessarily be concrete, i.e., a reaction of a given situation may only contain an STUB process, a process that acts as a 'placeholder' and, when executed, only sends a perception back to the planner stating that this particular situation has become active. We call this perception signal a stub-trigger. In this way, the planner can adapt the reactor without needing to fully refine its plan first. Before inserting any adaptation into the reactor however, the planner proceeds to detail the first step in the abstract plan, such that for this first step a concrete plan is in place (i.e., no STUB) when the adaptation is sent. This avoids an immediate stub-trigger signal from the reactor for this segment. Instead it allows the reactor to start operating immediately, while the planner can proceed in parallel to flesh out other abstract parts of the plan. For the kitting robot domain, an initial abstract plan would be a sequential composition of the situations AcquireTray, FillTray, and RemoveTray. Only AcquireTray needs to be concrete in order for the planner to adapt the reactor. The other situations can have S'I~'B reflexes initially. The planner can refine these while the reactor acquires a tray. The s ~ m processes in the reactor provide the planner with much needed focus-of-attention information, i.e. the stub trigger signal tell the planner which part of the reactor configuration is in immediate need of refinement. Hence the planner can concentrate on refining the relevant parts of the reactor first, as indicated by the environment by a stub-trigger. The following process descriptions show a reactor segment generated by the planner for this initial adaptation. Each reaction in the segment below is in an ifthen-else form ( ( - C o n d : I f T r u e ) : E l s e ) . The REASSERT process continually reasserts failing situations. P0 is necessary to assert the topmost situation repeatedly. The process MOVeobj,loc repositions a grasped object obj to a location loc.
111 S T O P : ; ASSERTKitting
PO
=
P1
=
SITKitting(k >: ; (- ( (REASSERTAcquireTray (xl> : REASSERTFilITray, xl : REASSERTFinishTray ,xl ) " SUCCEEDKitting, k) : FAILKitting, k)
P2
=
SITAcquireTray(k >: ; ( - ( ( REASSERTFindPart (xl> : MOVExl ' trayarea : (UPDATEAcquireTray, k, xl; SUCCEEDAcquireTray, k ) ) : FAILAcquireTray, k )
P3
=
SITFindpart(k> : ; ( " ( ( Findtray(xl> : UPDATEFindPart, k, xl ; SUCCEEDFindPart, k ) ) : FAILFindPart, k )
P4
=
SITFillTrayt(k, x l } : ; (STUBFillTray: SUSPEND2000; FAILFiUTray, k)
P5
=
SITFinishTrayt(k , xl>. ; (STUBFinishTray : SUSPEND2000; FAILFinishTray, k)
Barring any perceptions received from the reactor, the planner continues this process, incrementally fleshing out abstract segments. At every iteration, the expectation is reduced by fleshing out a situation. The situation hierarchy is restricted to be an acyclic digraph, i.e., abstract actions used higher in the plan hierarchy cannot be used to construct a concrete reaction. If all the reactor segments are made concrete and all the STUB processes removed. Then, the planner has achieved an co-ideal reactor, and can proceed to relax the next assumption.
5.4. Perceptions Perceptions may change the course of operation of the planner. They provide key information for the planner to decide what part of the reactor to elaborate upon next. The planner specifies which perceptions it wants to receive, i.e. which sensory data the reactor needs to collect for the planner's purpose. Reactions are ideally suited for situations where the data yielded by the embedded sensing is of no interest to the planner, because such data has only local effect within the reflex. Unknown or uncertain data that can have an effect on future reactor segments, however, should be collected differently. For example, in the case of a constant, but unknown, fact, it is inefficient to manufacture a reaction with a conditional on this fact, since we know a priori that all but one branch of that conditional is wasted. Factoring in resource use constraints means that some problems would become impossible when implemented with conditionals. There simply won't be enough resources available to cover all the a priori options. In addition, posting perception processes is useful for longer term information that is important to the planner, e.g. in the kitting domain, the fact of whether downstream automation is always ready to receive finished kits. In these cases, the planner produces perception requests: these are actions that report information back to the planner. The planner uses this information to refine its knowledge about the world, and thus remove the troublesome uncertainty. In contrast, insertion of communicating networks in the reactor is clearly most appropriate for transitory information, e.g., the position of a cap on the belt, or for determining which of the situations, sit(t), of a task, t, is currently present.
112 This fosters a fast response on the part of the reactor, it allows the reactor to deal with contingencies, and to take advantage of opportunities such as fortuitously occurring goal conditions. Apart from informational perceptions, the planner relies on four specific perceptions to set its priorities for the planning process. These perceptions are the following: 1.
2.
3.
4.
Guard-kills. The planner is informed when an ~. process is killed. ~. processes ('guards') form the basis by which an adaptation can remove old structure and install new structure in the reactor. This perception indicates to the planner the completion of an adaptation. Stub trigger. The execution of a sTtm process in the reactor signals the planner that the reactor has started to execute a segment for which no concrete plan yet exists. Situation failure. This signals the planner that a reactor segment failed (usually through an invalidated assumption), and needs to be modified as soon as possible. Assumption failure. The planner installs dedicated processes (A]:t) in the reactor to monitor assumptions. If one of the assumptions is invalidated, its monitor will signal the planner.
Guard-kills govern the adaptation cycle and are necessary for the planner to preserve the safety of the adaptation process. If a situation is still being adapted, the planner cannot send out a next adaptation for that same situation since the safe adaptation mechanism is only safe for one adaptation per situation at the time. Thus the planner holds back the next adaptation instructions for that situation until the guard-kill perception is received. The other three types influence the planner's priority in planning adaptations. The planner uses these to decide which part of the expectation to reduce first. In effect, the environment (through the reactor) tells the planner what parts of the reactor should be considered for adaptation. The priory ordering for planning is the following: Failed situations > Forced assumption relaxation > Active stubs > Quiescent stubs.
In absence of any perceptions received i.e in a fully cooperating environment, the normal procedure is to build an initial reactor first using all assumptions. This initial reactor is built in increments with stub refinement proceeding along temporal ordering of the reactor segments. Subsequently, assumptions are relaxed to widen the scope of the reactor.
113 5.5. F o r c e d R e l a x a t i o n .
Any of the assumption monitors or stub triggers i n the initial reactor segments can potentially signal the planner and divert its attention from the a priori established ordering of assumptions relaxation. Failure of a situation to successfully complete its reactions is also cause for a perception. These three sources of perceptual input have different effects on the planner. Stub trigger. On receiving a stub trigger signal, the planner redirects its focus of attention for refining the abstract plan to the portion of the plan containing the stub trigger. The planner inserts the appropriate refinement into the reactor. The reactor then can start executing that segment immediately, since the situation is still active. Assumption failure. An assumption failure perception has a larger effect than simply refocusing attention. The assumption failure signal causes the planner to negate the assumption in its Problem Solving Context and begin to rebuild the effected portions of its plan. Situation failure. If a situation relying on that assumption failed, that particular situation is given highest priority for adaptation, even though other assumptions may have priority in the assumption ranking. This means the planner may have to turn its focus back to parts of the plan it had previously considered finished. Apart from its cause, a forced relaxation or refinement results in the same reactor segment and adaptation sequence as a normal relaxation or refinement. 6.
EXPERIMENTAL RESULTS
The kitting example described section 3 was implemented and several experimental runs conducted on a PUMA-560 based kitting work cell. Trace statistics were gathered on each run on, e.g., the times when assumption failures were detected, when adaptations were made, the number of assumptions in use in the reactor, and so on. In each run, the planner utilized all the assumptions described in section 3. However, only five of the assumptions were actually relaxed in these trials. The purpose of these experimental runs was to begin to explore the behavior of a planner-reactor system in practice. Results from the runs are presented in the following subsections. Initial Reactor Construction. Figure 4 contains trace statistics for the start-up phase on a typical experimental run of the kitting work cell. Graph (a) shows the number of assumptions in use at any time in the work cell by the planner and by the reactor. Graph (b) shows the number of adaptations issued (by the planner) and the number of discrete changes to the reactor structure (adaptations applied). Initially there is an empty reactor in operation. After 26 seconds (in this example) the planner is in a position to start to update the reactor. (This time include planning plus communication time.) The planner begins by sending adaptations to the reactor (adaptations issued). Each adaptation involves one or more changes to the reactor structure. Thus, eventually, the adaptations applied will outnumber the adaptations issued.
114 The constraints involved in safe adaptation can cause a time lag in implementing changes. Active situations cannot be i n t e r r u p t e d - that would leave the reactor in an undefined s t a t e - instead the adaptation waits for the situation to end before applying the changes; the theory behind this is presented in [10]. Thus the adaptations applied trace always lags the adaptations issued trace. The lag time varies, depending on the activity of the reactor. This is what causes the reactor assumptions trace in Figure 4(a) to lag the planner assumption trace.
Figure 4: Start-up phase statistics for one run of the Planner-Reactor based Kitting Robot The first reactor was in place roughly 28 s after start-up. Even though this reactor was not complete, the kitting robot could start kitting; its kitting actions were overlapped in time with the further refinement of its kitting 'program'. This reactor was elaborated over the following 20 s until by 40 s after start-up the first complete reactor was in place. This reactor employed all nine assumptions. Up to this point, there was little delay between the planner reducing an assumption and the reactor following suit. Assumption Relaxation. Then the planner began to relax assumptions to incrementally improve the reactor. It took roughly 5 s to relax the first assumption (the length of the 'plateau' in the planner trace in Figure 4(a)). It then began to issue the adaptations to the reactor. Typically one or more adaptations is necessary to relax any one assumption. Each adaptation will give rise to one or more structural changes in the reactor, again subject to the delays of the safe adaptation constraint. In the case of this first assumption relaxation, its took the reactor roughly 60 s before it was able to implement the last structural change to relax the assumption (the 'plateau' on the reactor trace in Figure 4(a)). It took the planner roughly 30 s to relax the second assumption; but the change was implemented in the reactor almost immediately after the first relaxation. These lags and overlaps in operation underscore the highly asynchronous and independent operation of planner and reactor.
115 In these experiments, we restricted to planner to only relax two assumptions voluntarily. Having done this, the planner will only relax other assumptions when the environment forces their failure. This could occur within the start-up phase, of course, but for convenience of presentation we have separated the two here.
Figure 5: Forced relaxation statistics Environment Forced Relaxation. Figure 5 shows trace statistics for the ongoing operational phase in a typical experimental run of the kitting work cell. Again, graph (a) displays the assumptions in use by the planner and reactor, and graph (b) displays the trace of adaptations issued and applied. Additionally, Figure 5 (b) now also shows the trace of assumption failure perceptions. These are the perceptions that force the planner to relax an assumption. The two forced assumption relaxations in Figure 5 are the no-motion (AM) and downstream disturbance (ADD) assumptions. The failure perception for AM is generated when the robot fails to acquire a part it had identified in an initial visual image of the work space. The failure perception for ADD is manually invoked, and would normally be an input signal from downstream automation. At 152 s the AM assumption failure was signaled (Fig. 5(b)). The planner responded very quickly and began to issue adaptations by 155 s, continuing for about 10 s. There was little delay in the safe adaptation procedure and the structural changes to the reactor lag the adaptations issued by only about ls. This is also evident from the assumptions trace in (b). Part of the changes in this adaptation was to remove the assumption failure monitor for this assumption; thus, partway into the adaptation (161 s) the signals from the assumption monitor cease. The total forced relaxation response time ~ the time from failure to final change in the reactor m is roughly 11 s. The ADD failure occurred at 215 s and proceeded in a similar fashion. Reinstating Assumptions. In our theoretical analysis [9, 10] we did n o t address the problem of reasserting previously failed assumptions. Nonetheless, this is convenient in practice to model operating regimes, and therefore is part of our objectives. In the experimental run, at time 357 s the ADD assumption was reinstated
116 (Figure 5b)). On the relaxation of such reinstateable assumptions, the planner adds a reactor monitor that will signal when the assumption holds again. (It doesn't show up as an assumption failure, since it isn't). As the trace statistics show, our implementation does indeed handle the reintroduction of assumptions. However, further work is necessary to extend our convergence results to include this case. Ordering of Assumption Relaxation. The graphs of assumptions versus time do not indicate which assumptions are relaxed at each step. Figure 6 shows the interval for which each assumptions is in use in both planner (solid line) and reactor (broken line). The acronyms on the Y-axis refer to assumptions. Four assumptions m ADT (tray disturbance), ACP (cooperation), ACT (competition), and AF (filled trays) m are not relaxed in this experiment and hence hold throughout. The assumptions of quality (AQ) and substitutability (AS) are introduced first. Note that the reactor does not drop these assumptions for some time after the planner has dropped them. This is also shown in Figure 4 but there it wasn't evident which assumptions were involved. The downstream disturbance assumption (ADD) is introduced, then relaxed and then reintroduced, as shown in Figure 5.
Figure 6: Reactor assumption usage over time
Figure 7: The response to immediate failure of assumptions
Immediate Assumption Failure. The assumption relaxation in the experiment shown in graphs 4 and 5 is artificial: The planner-directed relaxation of assumptions is carefully separated in time from the environment-directed relaxation.
117 This is unlikely in practice, both planner and environment will be simultaneously directing relaxation. The assumptions graph in Figure 7 was generated in simulation (the previous graphs have been from real robot experiments) by assuming that whenever the reactor tests an assumption, the assumption failso The first difference with the previous experiment, (Figure 4) is that the planner never gets to use all the assumptions. Failure signals begin as soon as the first reactor update is made, even before the entire first ideal reactor is constructed. The planner quickly relaxes the AQ, AS and AO assumptions. It then finishes the reactor, introducing the ADD assumption which also immediately fails (the 'blip' at 44 s) and is replaced. 7.
DISCUSSION
In summary, we have reviewed our motivation and approach to casting planning as adaptation of a reactive system and have presented the first performance results of the approach, a first for integrated systems of this kind. These results show the feasibility of our approach. The incremental reactor construction and assumption relaxation strategy minimizes planning delays. The reactor can start kitting while the planner still is refining later parts of the task. The asynchronous communication between planner and reactor allows the latter to be real-time and not bogged down by the planner. The planner stays informed through its implantation of perceptions in the reactor. Information from these perceptions gets shuttled back to the planner, which employs it to decide on the appropriate parts of the reactor to modify next. Thus, the planner is able to support the reactor's direct needs, despite the asynchronous link between the two. REFERENCES
1. J. Bresina & M. Drummond, Integrating Planning and Reaction, in: AAAI Spring Workshop on Planning in Uncertain, Unpredictable or Changing Environments, Stanford, CA, J. Hendler (ed.), Systems Research Center, Univ. of Maryland (1990). 2. R. Brooks, A Robust Layered Control System for a Mobile Robot, IEEE Journal Robotics & Automation, 2 (1986). 3. J. Connell, SSS: A hybrid architecture applied to robot navigation. IEEE Int'l Conf. on Robotics & Automation (1992). 4. R.E. Fikes and N.J. Nilsson. Strips: A new approach to the application of theorem proving to problem solving. Artificial Intelligence 2, (1971), 189-208 5. B.R. Fox and K.G. Kempf, Opportunistic scheduling for robotic assembly. IEEE Int'l Conf. Robotics & Automation, St.Louis MO (1985), 880-889. 6. Th. Fraichard & C. Laugier, On-line Reactive Planning for Non-Holonomic Mobile Robot in a Dynamic Environment, IEEE Int'l. Conf. on Robotics & Automation, CA (1991). 7. E. Gat, Alfa: A language for programming reactive robot control systems, IEEE Int'l Conf. on Robotics & Automation, Sacramento, CA (1991), 1116-1121. 8. D.M. Lyons, Representing and analyzing action plans as networks of concurrent processes. IEEE Trans. Rob. & Aut}, 9 (3), (1993).
118 9. D.M. Lyons & A.J. Hendriks, Planning for Reactive Robot Behavior, IEEE Int'l Conf. on Robotics & Automation, Nice, France (1992). 10.D.M. Lyons & A.J. Hendriks, Safely Adapting a Hierarchical Reactive System, SPIE Intelligent Robotics & Computer Vision Boston, MA, (1993). 11. D.M. Lyons, A.J. Hendriks & S. Mehta, Achieving Robustness by Casting Planning as Adaptation of a Reactive System, IEEE Int'l Conf. on Robotics & Automation, Sacramento, CA (1991), 198-203. 12. A.J. Hendriks and D.M. Lyons, A Methodology for Creating and Adapting Reactive Systems, IEEE Int'l conference on Tools for AI, San Jose, CA (1991), 220227. 13.D. McDermott, Robot Planning, Tech. Report YALEU/CSD/RR #861, Yale University, (1991). 14.M. Schoppers, Representation and Automatic Synthesis of Reaction Plans, Ph.D. Thesis, Tech. Rep. uiucdcs-r-89-1546, CS dept., University of Illinois at Urbana-Champaign (1989). 15. C.J. Sellers & S.Y. Nof, Performance Analysis of robotic kitting systems, Rob. & Comp. Integ. Manuf. 6 (1989). 16.X. Xiadong and G.A. Bekey, Sroma: An adaptive scheduler for robotic assembly systems, IEEE Int'l Conf. on Robotics & Automation, Philadelphia, PA (1988).
Intelligent Robots and Systems V. Graefe (Editor) 9 1995 Elsevier Science B.V. All rights reserved.
119
Robot task programming by human demonstration: Mapping human grasps to manipulator grasps Sing Bing Kang and Katsushi Ikeuchi
School of Computer Science Carnegie Mellon University Pittsburgh, PA 15213
Abstract To alleviate the problem of overwhelming complexity in grasp synthesis and path planning associated with robot task planning, we adopt the approach of teaching the robot by demonstrating in front of it. A system with this programming technique is able to temporally segment a task into separate and meaningful parts for further individual analysis and recognize the human grasp employed in the task. With such derived information, this system would then map the human grasp to that of the given manipulator, plan its trajectory, and proceed to execute the task. This paper describes how grasp mapping can be accomplished in our system. The mapping process essentially comprises three steps. The first step is local functional mapping, in which grasps of functionally equivalent fingers are established. This is followed by gross physical mapping which produces a kinematically feasible manipulator grasp. Finally, by carrying out local grasp adjustment using some task-related criterion, we arrive at a locally optimal manipulator grasp. We describe these steps in detail in this paper and show results of example grasp mappings.
1. Introduction There is always a need for a portable, safe, easy, fast, and robust means of programming robots to perform tasks. Current conventional methods of programming robots such as programming using a robot language and direct manipulation of the robot through the desired trajectory (i.e., manual lead-through), while useful, fall short on some of these characteristics. Completely automatic task planning (with the inputs being just the high-level specifications of the task goals) is difficult to achieve in practice. This is attributed to the high combinatorial complexity of grasp synthesis and path planning in the presence of obstacles. We seek to mitigate this problem by using cues extracted directly from human performance of the task of interest. The
120 approach of programming a robot by allowing it to observe the human demonstrating the task is also known as Assembly Plan from Observation (APO) [9]. A major component in task planning is planning the manipulator grasp. In the following section, we briefly discuss some of the issues involved in grasp planning and reiterate the justification for our approach.
2. Grasp Planning Grasp planning involves determining the hand placement relative to the object of interest as well as the posture of the hand assumed in grasping the object, both in a manner consistent to the requirements of the task. It is computationally intensive due to both the large search space and the incorporation of geometric and task-oriented constraints. Approaches to grasp planning can be categorized as geometric and mechanical [ 19]. The geometric approach considers only the spatial aspects of the objects and arm/hand system, while the mechanical approach incorporates the arm/hand kinematics, system dynamics, and their task-related constraints. The large search space for an optimal grasp can be reduced in size or dimensionality by using certain heuristics or structural constraints. For a 2-finger gripper, for example, the surface element pair selection can be guided by a series of constraints, such as face parallelism, permissible face separation, and non-zero overlap between projected faces [24]. By imposing fixed contact locations or fixed types of grasp, Pollard computes a 6D projection of the 15 DOF configuration space for the Salisbury hand [20]. This reduced space represents the space of wrist poses for geometrically feasible grasps, and is used to search for a globally optimal grasp. In addition, a popular strategy for reducing the complexity of planning the dextrous hand grasp is by using a small set of predetermined grasps or grasping behaviours (e.g., [2], [7] and [21 ]). We adopt the approach to make grasp planning more tractable by using cues extracted from human execution of the grasp. The approach first involves the temporal segmentation of the recorded task executed by the human which locates the instant when the grasp occurs [ 11]. The subsequent steps are recognizing the type of grasp [12][13] and recovering the approximate grasp contact locations from observed data. This is followed by determining the initial approximate grasp which satisfies kinematic and geometric constraints, and subsequently performing local optimization of the grasp using the approximate grasp as the starting condition and a taskoriented objective function. Our grasp mapping approach is described in detail in the following sections.
3. Mapping grasps 3.1. General approach At the highest level of description, planning the manipulator grasp based upon the observed human grasp comprises the following steps:
121
1. Local functional mapping This is done by observing the type of grasp used by the human, and the functions and grouping of the palm and individual fingers. The latter is determined using the real-finger-to-virtual finger mapping described in [ 13]. The mapping of the manipulator end-effector to the virtual fingers is dependent on the effective cohesive index and the cohesive indices of the individual virtual fingers.
2. Adjustable or gross physical mapping This operation is carried out using the outcome of the local functional mapping and the initial location of the manipulator (near the pose of the human hand while carrying out the task). It results in an approximate form of the manipulator grasp of the object which is geometrically feasible.
3. Fine-tuning or local adjustment of grasp The grasp determined from steps 1 and 2 may not exhibit static stability, or that it may not be locally optimal according to a chosen task-oriented criterion. By analyzing the criterion and iteratively perturbing the initial grasp configuration, we arrive at a locally optimal grasp. In the case of the power grasp which involve high kinematic constraint, this step is skipped. 3.2. A n a l y t i c m e a s u r e s of a g r a s p Cutkosky and Howe [5] summarizes a number of analytic measures that have been used by various researchers in their analyses of grasps, several of which are for the purpose of grasp synthesis. We can group these grasp analytic measures into different categories according to both the depth of knowledge of the grasp and hand mechanism, and the type of analysis (Fig. 1). These analytic grasp measures are instrumental in determining the manipulator grasp. The overall strategy of mapping grasps is depicted in Fig. 2. The human grasp can be described in a hierarchical fashion [ 12]. With a priori knowledge of the human hand and data derived from observation, certain analytic grasp measures can be extracted (such as connectivity and mobility). Armed with these information, together with a priori kinematic, structural and geometric knowledge of the manipulator, the grasp translator would then attempt to produce a manipulator grasp with compatible analytic grasp measures. In our work reported in this paper, as a starting point, we employ the manipulability measure as the criterion used in generating the locally optimal manipulator grasp.
122
Analytic Grasp Measures Degree-of-freedom Measures connectivity mobility
Static Measures
Increasing depth of knowledge required of grasp and hand mechanism
force closure form closure internal forces resistance to slipping manipulability grasp isotropy
Structural and Servo Parameter Measure$ compliance stability Fig. 1 Categories of analytic grasp measures
f
Human grasp
~
]g l ras7 ulat~
observation
| | ofHierarchicalgrasp ~ description
a priori
knowledge of human ' hand ~.,
i/
~.~ ~ ~ ~" '~ ~ ~ [ "~ ~
.
description of grasp Analytic grasp measure________~s
I a priori
knowledge f manipulator
~
J
Fig. 2 Grasp mapping strategy
The functions of the hierarchical description of the human grasp are local functional mapping and gross physical mapping. Meanwhile, the functions of the analytic grasp measures are: 1. Verification of appropriateness of gross physical mapping; and 2. Local fine-tuning or perturbation of grasp to a locally optimal posture.
4. Local functional mapping This first step in mapping grasps is common to all types of grasps, i.e., power and precision grasps. The local functional mapping associates the manipulator fingers to those of the hand,
123 and is generally different for different grasps. To guide the local functional mapping, we first assign manipulator fingers as either a primary finger, secondary finger, or a palm.
4.1. Assigning manipulator fingers Prior to mapping the the manipulator fingers to those of the human in the observed grasp, the manipulator finger has to be assigned as one of the groups of fingers:
1. Primary finger/s A finger in this group would correspond to the human thumb, which has at least the same number of degrees of freedom as the other fingers. It serves not only to secure the hold of the object against the palm and/or other fingers, but also to manipulate the object.
2. Secondary finger/s This group would correspond to the fingers of the hand other than the thumb. This group of fingers would serve to secure the hold of the object against the palm and/or primary finger/s. A special case of a "finger" which is also assigned is the palm; it is passive with no local degree of freedom (i.e., discounting the global 6 DOF of the manipulator). Manipulators are designed with specific functions or level of dexterity in handling objects. The repertoire of grasping abilities may be as limited and simple as just clamping the object between its jaws as a means of securing a hold on it (as in the parallel-jaw gripper). The manipulator may be anthropomorphic to some degree and be able to mimic certain grasps of the human hand; examples of such hands are the Utah/MIT hand [10], Salisbury hand [22], Okada hand [18], Minnesota hand [15], and the Southampton hand [4]. It is usually simple to label a priori each of the manipulator fingers as either a primary or secondary finger. Once the primary and secondary fingers have been identified, the hand-to-manipulator finger mapping can proceed with the aid of another type of mapping. This mapping relates real fingers to groups of functionally equivalent fingers called virtual fingers [1][13]. A consequence of this real-finger-to-virtual finger mapping is the cohesive index [ 13], which is associated with the degree to which the real fingers in a virtual finger act in a similar manner (by comparing the object normals at the contact points). A cohesive index of a virtual finger is maximum at unity, and would indicate that the fingers within that virtual finger act exactly alike. 4.2. U s i n g t h e
cohesive index to guide the mapping of fingers
An indication of how the cohesive index is used to guide the virtual finger mapping between those of the human hand and the manipulator is illustrated in Fig. 3. (VF n refers to the nth virtual finger with C n being its associated cohesive index.) The palms, primary fingers, and secondary fingers are mapped appropriately as seen in Fig. 3. The cohesive index serves as a check - the virtual finger corresponding to the group of secondary fingers normally has a cohesive index less than that of the collection of primary fingers. The virtual finger composition determined from the real-finger-to-virtual finger mapping [ 13] take precedence, however.
124
VF o = {0} (palm) .~ VF 1={1} VF 2 = {2, 3, 4, 5}
C o = 1.0 C 1=1.0 C 2 < 1.0
5-fingered cylindrical power grasp VF o' = {0 } (palm) VF 1' = {1} VF 2' = {2}
VF0" = {0 } (palm) VFI" = {1} VF2" = {2, 3}
VF0'" = {0 } (palm) VFI'" = {1} VF2"' = {2, 3, 4}
(Decompose VF 1 and VF 2 this way because C 2 < C 1)
Two-fingered parallel-jaw gripper
Three-fingered Salisbury hand
Four-fingered Utah-MIT hand
Fig. 3 Mapping grasps according to cohesive indices of virtual fingers Once the functional finger assignments have been accomplished, the more detailed manipulator grasp planning can then proceed.
5. Approach in generating grasp and pregrasp trajectory The diagram representing the entire manipulator grasp and trajectory planning approach is shown in Fig. 4. Prior to the more detailed manipulator grasp planning, the manipulator assumes a fixed pose and posture relative to the human hand throughout the task execution; local planning is carried out within the vicinity of this pose and posture. Note that by virtue of the different form and function of the power and fingertip precision grasps, they are planned in a different manner elucidated in subsequent sections. Once the local planning is done, starting with the pose and posture of the manipulator during the grasping stage, the pregrasp trajectory is then planned backwards, with the goals of a smooth and collision-free path. This path would closely follow that assumed by the human hand trajectory.
125 Depending on the type of grasp, the approach in manipulator grasp planning differs. In the subsequent description of such mapping techniques, we concentrate on the most widely used grasps, namely the power grasp and the fingertip precision grasp. Identify human grasp Generate grasp hierarchy
$ Functional finger group assignment for manipulator xV
Follow approximate hand trajectory
Power grasp < t~
Fingertip precision grasp ~,,i~
Gross grasp posture followed by fine-tuning of grasp posture ~V
Regenerate the pregrasp trajectory
Fig. 4 Scheme for generating the manipulator grasp and pregrasp trajectory
6. Generating the power grasp The primary purpose of the power grasp, in which the hand generally envelops the object, is to ensure a secure hold of the held object at the expense of dexterity. Hence, we can dispense with the third step of local grasp adjustment.
6.1. General approach The approach taken to generate the power or "enveloping" grasp follows that taken in "reactive" planning (similar to that described in [2]). Note that this approach is taken in generating the grasp, and is not used in the actual execution of the grasp itself. The approach is characterized with the following phases:
126
1. Aligning the dextrous hand with the approach vector assumed by the human hand just prior to grasping (approach alignment phase). 2. Translating the hand (preserving its orientation) towards the object until the palm touches the object (approach phase). 3. Adjust the hand to "fit" the object to the palm (adjustment phase). 4. Flex the fingers to envelope the object (wrapping phase). Central to the alignment motions of the dextrous hand is the pivot point. It is predefined to be a point on the surface of the palm between the fingers. Its frame is oriented such that its z-direction points away from the palm and the x-direction points from the thumb (or a primary finger/ s) to the rest of the fingers. 6.2. A p p r o a c h a l i g n m e n t of t h e d e x t r o u s h a n d In order to minimize the problem of collision with the object during the pregrasp and post grasp phases, we first align the pivot frame of the dextrous hand to the approach vector assumed by the human hand just prior to grasping the object. This is known as the approach alignment phase. The approach vector is determined as the translation component of the differential transformation AT, where AT = Tg Tg_k-1, where Tg is the transformation of the hand at the grasp frame and Tg_k is the transformation of the hand k frames prior to the grasp frame (k is chosen to be 2). Once the rotational transformation (about the pivot point) has been determined, the dextrous hand is rotated in steps, with the intermediate collision detection with the object. If collision is detected, the dextrous hand is translated or "repulsed" away (preserving its orientation) from the surface of object that makes contact with the hand. The normals of the surface or surfaces in question determine the direction of the repulsion vector. 6.3. A d j u s t i n g t h e h a n d a n d e n v e l o p i n g t h e o b j e c t Once the desired orientation of the hand is assumed, planning proceeds with the following behaviors:
1. Pure translation toward the object (approach phase) The translational motion is dictated by attractive forces by the object on the pivot point on the dextrous hand. This behavior is active prior to any collision between the object and the manipulator palm.
127
~ Object n j--
~
Fig. 5 Determining direction of translation From Fig. 5, p is the pivot point on the manipulator palm, and qj is the jth sampled object surface point, rj is the vector from point qj to p, i.e., rj = qj - p. fij is the object normal at qj. The motion at each stage is M:
d M = 8=-=, Idl
d = -
E llj. rj>o
fD.r. J J
where 8 is the step size in 3D space. r is the weighting factor dependent on Irjl; the function used is o~j = Irj1-2.
Translation and rotation about the pivot point (adjustment phase) Once collision between the object and the manipulator palm is detected, this behavior becomes active. The motion associated with this behavior is defined by both the point or points of contact and the entire object. The points of contact determine the motion which tend to push away the manipulator (repulsive forces) while the entire object tends to pull the manipulator (via the pivot point) towards the object. This has the property of trying to align the manipulator relative to the object prior to grasping. It should be noted that while these behaviors are active, the posture of the manipulator is that of full extension of the fingers (for parallel jaw gripper, maximally distanced fingers). Note: The circumflex (^) over the symbol indicate its normalized version. The rotation about the pivot point is ~, where Q-
E Qj qjCP
and P is the palm, Qj is the quaternion associated with the rotation due to the jth contact point and is given by Qj =
2 Ecos ~-Oj, mjsin-Oj
0j = 8/Irjl and mj = -rj x nj.
128
3. Flexion of fingers around the object (wrapping phase) Once the manipulator has been adequately aligned with respect to the object, the manipulator would then proceed to wrap its fingers around it. This is done by flexing successively distal joints (starting with the most proximal joint) until contact is made between the associated link and the object (Fig. 6 (a) and (b)).
Fig. 6 Results of mapping a cylindrical power grasp for Utah/MIT hand (a) and Salisbury hand (b) This approach is a simple one, and is relatively independent of the manipulator (once the manipulator models have been built). This approach is made possible by the cues given in the human execution of the same task.
7. Generating the fingertip precision grasp 7.1. G e n e r a l a p p r o a c h The approach taken to plan the fingertip precision grasp is different than that for the power grasp, primarily because this precision grasp does not envelope the object. As with the power grasp, there are basically two stages:
1. Initial grasp pose and posture determination phase Let the (N+ 1) contact points on the object be denoted Pc0, Pc 1 . . . . PcN" The initial grasp pose is determined by calculating the pose of the contact frame F c and orienting the pivot frame Fpivot centered at the pivot point on the hand by a certain fixed transformation relative to F c (Fig. 7).
129
Pcl Pco
9
d .....~ P c N Fpivot
Fig. 7 Orienting the pivot frame with respect to the contact frame Subsequently, the pose of the hand is searched at this neighhorhood until a kinematically feasible grasp is found. The contact frame F c is centered about point c, defined to be
1
N
1
1/
N
c -- ~ Z 2 (Pco +pc/) = ~ Pco + ~ Pc/ i=l i=1
/
The orientation of F c is determined as follow (note that the caret ^ on a vector term indicates its normalized version):
N N l = ~ (Pc/-Pco) - Z P c / - N P c o i=1 i=1 If the best fit plane H to the contact points is r 90 = a, where r is any point on H, and 0 its unit normal, then
h = I 0,
l
if
-0,
(P X1) 9(PeN--Pc0) >0 otherwise
and~h = ~ x ~ .
2. Local grasp pose and posture adjustment phase This is done by locally optimizing a task-oriented criterion with respect to the pose and posture of the hand. The local adjustment is done at two levels; the two levels are associated with the optimization with respect to the pose (outer level) and with respect to the joint angles (inner level). Note that the inner level optimization require iterations only if redundancy of finger DOFs exists. For the Utah/MIT hand, each finger has a redundant DOF since there are 4 DOFs per finger with only the 3D position of each fingertip being fixed. There is no redundancy of DOF for the three fingers of the Salisbury hand.
130 Let Aj = (tj, rj) be the vector representing the hand pose, Ojk be the vector of joint angles representing the hand posture, and C0 be the task-oriented criterion to be optimized. Then the optimization can be symbolically represented as:
max fmax C (Aj, Ojk) ) = max C (Aj, Oj, opt)
A.J Lo.
)
A.j
The outer optimization level uses the Nelder-Mead simplex algorithm (see, for example, [ 17]), while the inner optimization is accomplished using Yoshikawa's scheme for optimizing manipulator postures with redundant DOFs [25]. For a given initial hand pose Aj, the locally optimal hand posture Oj, opt is determined using the following pseudocode:
Repeat until either the joint angle limitations are exceeded or the magnitude of joint angle change vector 6)8t is below a threshold {
= ( I - J+J) ~k since ic = O, the contact points being fixed at each iteration.
%k <---%k + l~jkSt
} where J is the hand Jacobian and J+ is the pseudoinverse of the Jacobian, i.e.,
j+ = jT ( j j T ) - I
8. Estimating the dextrous hand-object contact points Approach is as follows:
1. Determine the rough positions of the human hand-object contact points. Fit these contact points with a plane H: p 9h = d.
2. Cull off planes and edges as potential grasping places by imposing the constraint that the associated object normal should not be too close in direction to h. At each point on edge i (edge point) and point on face j (face point), let the associated normals be nei and hf respectively. Then, disregard points on edge i if Ihe 9h I > COS0e and points on face j if lhfj. h > cos0f. Since we favor faces over edges, Oei> Of, with 0 e = 75 ~ and Of = 45 ~
131 3. For each virtual finger with more than 1 real finger, fit a line to the human hand contact points, and space the hypothesized dextrous hand-object contact points equally along this line. 4. Find the object surface points nearest to the hypothesized dextrous hand-object points; for each face point, impose a margin ~mfrom the nearest edge (~m = 1.Ocm) if possible; other wise arrange its position in the center between opposite edges. An example of a fingertip precision grasp mapping is shown in Fig. 8.
Fig. 8 Results of mapping a fingertip precision grasp for Utah/MIT hand (a) and Salisbury hand (b)
8.1. The optimization criterion The precision grasp may be generated using a variety of task-oriented measures such as the manipulability measure [25], task compatability index [3], sum of force minimization [ 16], measures based on the task ellipsoid [14], among many others. We illustrate the notion of generating the precision grasp by local perturbation using Chiu's task comparability index [3]. The velocity and force transmission characteristics can be represented geometrically as ellipsoids defined as a function of the manipulator Jacobian. Let the kinematic transformation from joint space to task space be given by x = x (| where | = [01, 02 ..... 0n] T and x = [x 1, x 2, .... Xm]T are the joint and task coordinate vectors. The Jacobian links the time differentials: :e = J ( | where J(| is the mxn manipulator Jacobian matrix, whose elements are Ji,j = ~gxi/~90j. The unit sphere in joint space 91n defined by II| -< 1 is mapped into an ellipsoid in ~ m defined by
This ellipsoid is variously called the manipulability ellipsoid [25] and the velocity ellipsoid [3]. Analogous to the velocity ellipsoid is the force ellipsoid defined by fT ( j (O) jT ( O ) ) f < 1
132 w h e r e f is the force vector in task space and x is the joint torque vector. The force transmission ratio c~ and velocity transmission ratio ~ along direction u in task space can be determined to be 1
1
cz = [u T ( J J T ) u ] 2 and
13 = [u T(JJT)-~u]
respectively. Chiu defines the
compatibility index as
C : Z i
i iw~-t-"k'Zwj~j2 2 j
= Z W i [U Ti ( j j T ) Ui] ~=1 +
i
2
~_wj [u~ (jjT)
-1 uj]
:]:1
j
where the positive exponent of ~i or 13jis used for directions in which the magnitude is of interest, and the negative exponent is used for the directions in which accuracy is of interest. The w's are weighting factors that indicate the relative magnitude or accuracy requirements in the respective task directions. In our application, the finer force control directions are chosen to be along the object normals at the points of contact, while the finer velocity control directions are along the tangent plane defined by two spanning tangents.
9. System implementation The arm/hand system used to verify the grasp mapping technique is shown in Fig. 9. The arm used is the PUMA 560 arm while the dextrous hand used is the Utah/MIT hand. Both the arm and hand is controlled via Chimera, which is a real-time operating system for sensor-based control developed at Carnegie Mellon University [23]. The joint angles of the arm and the hand are fed from the grasp simulator (the display of which is shown in Fig. 10) to the arm/ hand system via Chimera. The two types of grasp performed by the Utah/MIT hand are shown in Fig. 11.
133
Fig. 9 PUMA 560 and Utah/MIT hand system
Fig. 10 Simulator display
Fig. 11 Cylindrical power grasp (left) and fingertip precision grasp (right) by the Utah/ MIT hand
134
10. Concluding remarks Our approach in programming a robot to perform a task is to have the system observe a human executing that task, identify and plan what needs to be done to accomplish the task, and finally execute the task. A very important component of planning the execution of the task is the generation of the manipulator grasp. In this paper, we have described our method in mapping the observed human grasp to that of the manipulator. While this method may not produce a globally optimal manipulator grasp (by some task-related measure), it greatly simplifies grasp planning. In addition, it illustrates our philosophy of easing the programming burden to mostly demonstrating the task; it reduces the complexity of programming and planning the task by taking advantage of the cues derived from observing a human.
Acknowledgments Many thanks to Richard Voyles for helping us set up the PUMA arm and Utah/MIT hand system. This research was supported in part by the Avionics Laboratory, Wright Research and Development Center, Aeronautical Systems Division (AFSC), U.S. Air Force, Wright-Patterson AFB, Ohio 45433-6543 under Contract F33615-90-C-1465, ARPA Order No. 7597, and in part by NSF under Contract CDA-9121797.
References [1]
M.A. Arbib, T. Iberall, and D.M. Lyons, "Coordinated control programs for movements of the hand," Hand Function and the Neocortex, eds. Goodwin, A.W., and DarianSmith, I., Springer-Verlag, 1985, pp. 111-129.
[2]
D.L. Brock and J.K. Salisbury, "Implementation of behavioral control on a robot hand/ arm system", 2nd lnt'l Symposium on Experimental Robotics, 1991, pp. 1-19.
[31
S.L. Chiu, "Task compatibility of manipulator postures," The lnt'l Journal of Robotics Research, vol. 7, no. 5, 1988, pp. 13-21.
[4]
R.M. Crowder, "An anthropomorphic robotic end effector," Robotics and Autonomous Systems, vol. 7, no. 4, 1991, pp. 253-268.
[5]
M.R. Cutkosky and R.D. Howe, "Human grasp choice and robotic grasp analysis," Dextrous Robot Hands, S.T. Venkataraman, and T. Iberall (eds.), Springer-Verlag, 1990, pp. 5-31.
[6]
C. Ferrari and J. Canny, "Planning optimal grasps," Proc. Int'l Conf. on Robotics and Automation, 1992, pp. 2290-2295.
135
[7]
E. Grosso and G. Vercelli, "Grasping strategies for reconstructed unknown 3D objects," Proc. IEEE/RSJ Workshop on Intelligent Robots and Systems, Osaka, Japan, 1991, pp. 70-75.
[8]
K.H. Hunt, Kinematic Geometry of Mechanisms, Oxford University Press, 1978.
[9]
K. Ikeuchi and T. Suehiro, "Towards an Assembly Plan from Observation, Part I: Assembly task recognition using face-contact relations (polyhedral objects)," Proc. Int'l Conf. on Robotics and Automation, 1992, pp. 2171-2177.
[10]
S.C. Jacobson, J.E. Wood, D.EKnutti, and K.B. Biggers, "The Utah/MIT dextrous hand: Work in progress," Int'l Journal of Robotics Research, vol. 3, no. 4, 1984, pp. 2150.
[11]
S.B. Kang and K. Ikeuchi, "Determination of motion breakpoints in a task sequence from human hand motion," to appear in Proc. IEEE Int'l Conf. on Robotics and Automation, San Diego, May, 1994.
[12]
S.B. Kang and K. Ikeuchi, "A grasp abstraction hierarchy for recognition of grasping tasks from observation," Proc. IEEE/RSJ Int'l Conf. on Intelligent Robots and Systems, July, 1993, pp. 194-201.
[13]
S.B. Kang and K. Ikeuchi, "Toward automatic robot instruction from perception - Recognizing a grasp from observation," IEEE Trans. on Robotics and Automation, vol. 9, no. 4, 1993, pp. 432-443.
[14]
Z. Li and S. Sastry, "Task-oriented optimal grasping by multifingered robot hands," IEEE Journal on Robotics and Automation, vol. 4, no. 1, 1988, pp. 32-44.
[15]
D. Lian, S. Peterson, and M. Donath, "A three-fingered, articulated, robotic hand," Proc. 13th Int'l Symp. on Industrial Robots, vol. 2, 1983, pp. 18/91-18/101.
[16]
X. Markenskoff and C.H. Papadimitriou, "Optimum grip of a polygon," The Int'l Journal of Robotics Research, vol. 8, n. 2, 1989, pp. 17-29.
[17]
J.H. Mathews, Numerical Methods for Computer Science, Engineering, and Mathematics, Prentice-Hall, 1987.
[18]
T. Okada, "Object-handling system for manual industry," IEEE Trans. on Systems, Man, and Cybernetics, vol. SMC-9, no. 2, 1979, pp. 79-89.
[19]
J. Pertin-Troccaz, "Grasping: A state of the art," The Robotics Review 1, O. Khatib, J.J. Craig, and T. Lozano-P6rez (eds.), MIT Press, Cambridge, MA, 1989, pp. 71-98.
[20]
N. Pollard, "Planning grasps for a robot hand in the presence of obstacles," Proc. Int'l Conf. on Robotics and Automation, 1993, pp. 723-728.
[21]
K. Rao, G. Medioni, H. Liu, and G.A. Bekey, "Robot hand-eye coordination: Shape description and grasping," Proc. Int'l Conf. on Robotics and Automation, 1988, pp. 407411.
136 [22]
J.K. Salisbury, Kinematic and Force Analysis of Articulated Hands, Ph.D. dissertation, Stanford University, Stanford, CA, 1982.
[231
D.B. Stewart and P.K. Khosla, Chimera 3.1, The Real-Time Programming Operating System for Reconfigurable Sensor-Based Control Systems, Carnegie Mellon University,
[24]
1993. J.D. Wolter, R.A. Volz, and A.C. Woo, "Automatic generation of gripping positions for assembly," Robot Grippers, D.T. Pham and W.B. Heginbotham (eds.), Springer-Verlag,
[25]
1986, pp. 55-74. T. Yoshikawa, "Manipulability of robotic mechanisms," The Int'l Journal of Robotics
Research, vol. 4. no. 2, 1985, pp. 3-9.
Intelligent Robots and Systems V. Graefe (Editor) 1995 Elsevier Science B.V.
137
Robot Learning By Nonparametric Regression Stefan Schaal and Christopher G. Atkeson Department of Brain and Cognitive Sciences & The Artificial Intelligence Laboratory Massachusetts Institute of Technology, 545 Technology Square, Cambridge, MA 02139"
Abstract: We present an approach to robot learning grounded on a nonparametric regression technique, locally weighted regression. The model of the task to be performed is represented by infinitely many local linear models, i.e., the (hyper-) tangent planes at every point in input space at which a prediction is to be made, i.e., at every query point. Such a model, however, is only generated at the time of prediction and is not retained. This is in contrast to other methods using a finite set of linear models to accomplish a piecewise linear model. Architectural parameters of our approach, such as distance metrics, are also a function of the current query point instead of being global. Statistical tests are presented for when a local model is good enough such that it can be reliably used to build a local controller. These statistical measures also direct the exploration of the robot. We explicitly deal with the case where prediction accuracy requirements exist during exploration: By gradually shifting a center of exploration and controlling the speed of the shift with local prediction accuracy, a goal-directed exploration of state space takes place along the fringes of the current data support until the task goal is achieved. We illustrate this approach by describing how it has been used to enable a robot to learn a challenging juggling task: Within 40 to 100 trials the robot accomplished the task goal starting out with no initial experiences. 1. INTRODUCTION Learning control means improving a motor skill by repeatedly practicing a task. We are exploring systems that learn by explicitly remembering their experiences. These systems will be useful in situations where the form or structure of the task to be performed is not known in advance. Previous work tested such methods by implementing learning for one-shot or static tasks, such as throwing a ball at a target (Aboaf et. al., 1988), and also repetitive or dynamic tasks, such as bouncing a ball on a paddle (Aboaf et. al., 1989) and hitting a stick back and forth (a form of juggling known as devil sticking) (Van Zyl, 1991). This experimental work has highlighted the importance of making sure the control paradigm used is robust to uncertainty, that the representational scheme is able to compute what is known about the task, and how well it is known, and that there is some process that generates exploration, so that modNew address of both authors: Georgia Institute of Technology, College of Computing, 801 Atlantic Drive, Atlanta, GA 30332, USA
138 els and controllers based on insufficient data are improved. In this paper we discuss the current stage of our non para metric learning methods by describing some of their recent extensions with the help of an implementation of learning with the devil sticking robot. In the given context, we are using a form of indirect learning, where a model is learned and then control actions are chosen based on the model, rather than direct learning, where appropriate control actions are learned directly (Barto et. al., 1991); the demonstrated framework, however, is equally well suited to direct learning, too. Our starting point for modeling is that we do not know the structure or form of the system to be con trolled. However, we assume we do know what constitutes a state of the system, and that we measure the complete state. Future work will discuss how to approach tasks in which complete measurements of the state are not available, or what constitutes a state is not even known. We adopted a recently developed statistical technique from nonparametric regression analysis, locally weighted regression (LWR (Cleveland, 1979; Farmer & Sidorowich, 1987; Atkeson, 1992), to learn models the system we are trying to control (Section 2). The LWR approach allows to efficiently estimate local linear models for different points in the state space. This approach is extended here to give information about the reliability of the predictions and local linearizations generated by locally weighted regression. Thus, as will be demonstrated in Section 3, a robot can monitor its own skill level and guide its exploratory behavior, and it can ensure that even in high dimensional state space good learning results are accomplished. 2. L O C A L L Y W E I G H T E D REGRESSION The point of view acquired here that the goal of a learning system for robots is to be able to build internal models of tasks during execution of those tasks. These models are multidimensional functions that are approximated from sampled data (the previous experiences or attempts to perform the task). The learned models are used in a variety of ways to successfully execute the task. Models should incorporate the latest information. As they will be continuously updated with a stream of new training data, updating a model should take a short period of time. There are also time constraints on how long it can take to use a model to make a prediction. Because we are interested in control methods that make use of local linearizations of the plant model, a representation is needed that can quickly compute a local linear model of the represented transformation. Moreover, negative interference from learning new knowledge on previously stored information should be as minimal as possible. As the most generic approximator that satisfies many of these criteria, we explore a version of nonparametric learning called locally weighted regression (LWR). LWR is a memory-based learning (MBL) system: it is trained just by storing the training data in a memory. This allows to achieve real-time learning and avoids interference between new and old data by retaining and using all the data to answer each query. Nonparametric learning sy stems like LWR approximate complex functions by using simple local models, as does a Taylor series. Examples of types of local models include nearest neighbor, weighted average, and locally weighted regression. Each of these local models combines points near to the input point of a desired prediction, the query point, to estimate the appropriate output. Locally weighted regression uses a regression procedure to form the local model, and is thus more expensive than nearest neighbor and weighted average memory-based learning procedures. However, local linear models have been shown to have a favorable balance between bias and variance
139 (Cleveland et al., 1979). They also provide the first derivative at the query point which will be shown below to be very use ful for control tasks. For each query LWR forms a new local model. The rate at which local models can be formed and evaluated limits the rate at which queries can be answered. At the end of this paper, it will be described how locally weighted regression was implemented for a real-time application. Unweighted regression finds the solution to the equations: y=X.13 (2.1a) by minimizing j=l m )2 -~E(Yi--Yi (2. lb) i=l
where X is an m.(n+ 1) matrix consisting of m data points, each represented by its n input dimensions and a "1" in the last column, y is a vector of the corresponding outputs for each data point, ~ is the (n+l)-vector of regression parameters which are to be determined, and J is the sum of squared errors over all given data points (Yi is the predicted result after fitting the parameters [3). The well-known result for [3 yields = ( x T x ) -1 XTy (2.2) and a prediction of the outcome of a query point xq becomes: (2.3)
yq = xT~
However, this gives distant points equal influence with nearby points on the ultimate answer to the query. To weight similar points more, locally weighted regression first calculates the distance of each of the stored data points (rows in the X matrix) to the query point Xq:
d~
n = E(Xq r=l
-- Xir) 2
(2.4)
The weight for each stored data point is a function of that distance:
f(d~) (2.5) Each i-th row in X and y is multiplied by the corresponding weight wi, and the regression problem is solved as be fore. In matrix notation, this introduces the diagonal m , m weight matrix Win J: j= 1 m .~Z(wi(Y i _ ~i))2 = 11 z W ( y - ~)l2 (2.6) W i -"
i=1
and the local weighted regression parameters become: 13= [(wx)T WX] -1 (wx)T Wy
(2.7)
A simple weighting function just raises the distance to a negative power, which determines how local the regression will be (the rate of drop-off of the weights with distance). 1
wi=(df) k
(2.8)
This type of weighting function goes to infinity as the query point approaches a stored data point which forces the locally weighted regression to exactly match that stored point. If
140 the data is noisy, exact interpolation is not desirable, and a weighting scheme with limited magnitude is desired. One such scheme, which we use in what follows, is a Gaussian kernel: w i = exp - 2k 2 )
(2.9)
As in (2.8), the parameter k scales the size of the kernel to determine how local the regression will be. A LWR lookup has three stages: forming the weights (Eqns. (2.4), (2.5)), forming the regression matrix WX, and solving for the regression parameters as given in (2.7). A more thorough introduction of LWR can be found in Atkeson (1992) or Schaal and Atkeson (1994). We have implemented the local weighted regression procedure on a 33MHz Intel i860 microprocessor. The peak computation rate of this processor is 66 MFlops. We have achieved effective computation rates of 20 MFlops on a learning problem with 10 input dimensions and 5 output dimensions, using a linear local model. This leads to a lookup time of approximately 15 milliseconds on a database of 1000 points.
2.1. Tuning The Fit Parameters LWR uses a few parameters which require tuning to optimize the quality of fit. Of most importance is the parameter k in (2.9) and, in multiple input systems, the distance metric m which determines the weighting of the input dimensions with respect to each other. The distance metric enters (2.4) and modifies this equation to: d 2 =~[(Xq--Xir)mr]
2
(2.10)
r=l
In the past we have used off-line global cross validation to estimate reasonable values for these fit parameters. Since we are using a local model that is linear in the unknown parameters, derivatives can be computed of the cross validation error e i = ~ - Yi with respect to the fit parameters k and m and minimize the sum of the squared cross validation error using a Levenberg-Marquardt (nonlinear least squares) procedure (MINPACK, NL2SOL). However, it is clear that these parameters should depend on the location of the query point. In this section we describe new procedures that estimate local values of the fit parameters optimized for the site of the current query point. We want to demonstrate the differences between local and global fitting in an example where we only focus on the kernel width k of a Gaussian weighting function (2.9). In Figure la, a noisy data set of the function 3 3 y = x - s i n 3 (2trx)cos(2tcx )exp(x 4 ) was fitted by locally weighted regression with a globally optimized constant k. In the left half of the plot, the regression starts to fit noise because k had to be rather small to fit the high frequency regions on the right half of the plot. The prediction intervals, which will be introduced below, demonstrate high uncertainty in several places. To avoid such undesirable behavior, a local optimization criterion is needed. Standard linear regression analysis provides a series of well-defined statistical tools to assess the quality of fits, such as coefficients of determination, t-tests, F-test, the PRESS-statistic, Mallow's Cp-test, confidence intervals, prediction intervals, and many more (e.g., Myers 1990). These tools can be adapted to locally weighted regression. We do not want to discuss all possible available statistics here but rather focus on two that have proved to be quite helpful.
141
Figure 1: Optimizing the locally weighted regression fit using different measures: (a) global cross validation, (b) local cross validation Cross validation has a relative in linear regression analysis called the PRESS residual error. The PRESS statistic performs leave-one-out cross validation, however, without the need to recalculate the regression parameters for every excluded point. This is computationally very efficient. Schaal and Atkeson (1994) show how the PRESS residual can be expressed as a mean squared cross validation error M S E c.... . In Figure l b, the same data as in Figure l a was fitted by adjusting k to minimize M S E c. . . . a t e a c h q u e r y p o i n t . The outcome is much smoother than that of global cross validation, and also the prediction intervals are narrower. It should be noted that the extrapolation properties on both sides of the graph are quite appropriate (compared to the known underlying function), in comparison to Figure 1a.
142 Prediction intervals I i are expected bounds of the prediction error at a query point X i . Their derivation can be found in most text books on regression analysis (e.g., Myers 1990), and an adaptation for LWR is straightforward (Schaal and Atkeson, 1994). Besides using the intervals to assess the confidence in the fit at a certain point, they provide another optimization measure. Using prediction intervals as optimization criterion for k, a plot similar to Figure l b is obtained. The curves in Figure 2a,b are generated by using this optimization criterion.
2.2. Assessing The Quality of the Local Model Both the local cross validation assess the quality of the local fit:
a:t
=
4MSEcr~
or
Q:t =
error
17-17'
C
MSEcros s and the prediction interval I i may serve to
(2.11)
C
The factor c makes Qat dimensionless and normalizes it with respect to some user defined quantity. In our applications, we usually preferred Q:t based on the prediction intervals, which is the more conservative assessment.
2.3. Dealing with Outliers Linear regression analysis is not robust with respect to outliers. This also holds for locally weighted regression, although the influence of outliers will not be noticed unless the outliers lie close enough to a query point. In Figure 2a we added three outliers to the test data of Figure 1 to demonstrate this effect. Again, we would like to localize our criterion for outlier removal. The PRESS statistic can be modified to serve as an outlier detector in LWR. For this, we need the standardized individual PRESS residual ei,cros s (Schaal and Atkeson, 1994). This measure has zero mean and unit variance. If, for a given data point x i, it deviates from zero more than a certain threshold, the point can be called an outlier. A conservative threshold would be 1.96, discarding all points lying outside the 95% area of the normal distribution. In our applications, we used 2.57 cutting off all data outside the 99% area of the normal distilbution. As can be seen in Figure 2b, the effects of the outliers is reduced.
2.4. Data Compression Despite that the original version of LWR is purely memory-based and does not attempt any kind of data compression, it is straightforward to add this property. The "infinity" of local linear models can be converted into a finite set of piecewise linear models. An efficient way to do this is by means of a recursive tessellation of the inputs space, another nonparametric technique well-known from decision tree induction (Breiman et al., 1984). K-d tree partitioning is the most common of these methods (Friedman et al., 1977). K-d trees partition the input space into hyper-rectangles by recursively splitting the partitions along one input dimension. There are many different ways when and where to split, and we will sketch only what was used in our algorithms.
143
Figure 2: Influence of outliers on LWR: (a) no outlier removal, (b) with outlier removal For building piecewise linear models, our method starts with one partition which includes all the data. Local linear models are fit to the center of the partition and all its vertices, by LWR. If the local linear model at the center is capable of predicting the outputs of the LWR lookups at the vertices within a certain error margin, the partition is not split anymore, and, if the prediction intervals of the lookup at the center are narrow enough, the partition will be marked valid. The outputs of all query points falling into this partition can be predicted with the linear model at the center. If the linear model of the center of the partition fails to predict the vertices well enough, the partition is split at the center along the dimension where the greatest mismatch between predicted value and real value had been encountered. Then the process repeats with the newly
144 created partitions. The algorithm terminates when no more splitting needs to be done. Figure 3 shows the result of this method for the function introduced in Figure 1. As Figure 3 illustrates, the piecewise linear fit of the data represents the given curve very well. At locations where the data has high frequencies, more partitions are allocated than at the more linear regions. By changing the error margin when to accept a partition and its local linear model, the piecewise fit could become coarser or finer. One advantage of this method over other piecewise linear partitioning techniques is that the partitioning of the input space is only done in order to find candidate centers for the linear models. For the local linear fits, all data are always included in the LWR lookup. Thus, there is no hard but rather a soft partitioning of the input space since each local linear models also includes data from the neighboring partitions. Due to space limitations, this data compression method cannot be discussed in detail. It can be applied in conjunction with all the methods mentioned so far.
1.5
.-.I1 >,,
0.5
-
+
0
-0.5 0
''''1''''1''''1''''1''''1''''1''''1''''1''''1'''' 0.1 0.2 0.3 0.4 0.5
0.6
0.7
0.8
0.9
1
XH>
Figure 3: Piecewise linear fit of the data from Figure 1; the black dots denote the boundaries between two neighboring partitions.
3. R E A L - T I M E LEARNING OF R O B O T J U G G L I N G We have constructed a system for experiments in real-time motor learning (van Zyl 1991). The task is a juggling task known as "devil sticking". A center stick is batted back and forth between two handsticks ( Figure 4a). Figure 4b shows a sketch of our devil sticking robot. The juggling robot uses its top two joints to perform planar devil sticking. Hand sticks are mounted on the robot with springs and dampers. This implements a passive catch. The center stick does not bounce when it hits the hand stick, and therefore requires an active throwing motion by the robot. To simplify the problem the center stick is constrained by a boom to move on the surface of a sphere. For small movements the center stick movements are ap-
145 proximately planar. The boom also provides a way to measure the current state of the center stick. Ultimately we want to be able to perform unconstrained three dimensional devil sticking. The task state is the predicted location of the center stick when it hits the hand stick held in a nominal position. Standard ballistics equations for the flight of the center stick are used into a task state. The dynamics of to map flight trajectory measurements throwing the devilstick are parameterized by 5 state and 5 action variables, resulting in a 10/5-dimensional input/output model for each hand:
(x(t),y(t),O(t))
x=(p,O,k,p,O)
(3.1)
The task command is given by a displacement of the hand stick from the nominal position 0,, and a throw velocity vector (v~, vy).
(Xh,Yh), a center stick an gular velocity threshold to trigger the start of a throwing motion
U:'(Xh,Yh,Ot,Vx,Vy)
(3.2)
Figure 4: (a) an illustration of devil sticking, (b) sketch of our devil sticking robot: the flow of force from each motor into the robot is indicated by different shadings of the robot links, and a position change due to an application of motor 1 or motor 2, respectively, is indicated in the small sketches
146 Every time the robot catches and throws the devil stick it generates an experience of the form: (Xk,Uk,Xk+l)
(3.3)
where x k is the current state, u k is the action performed by the robot, and xk+1 is the state of the center stick that results. Initially we explored learning an inverse model of the task, using nonlinear "deadbeat" control to eliminate all error on each hit. Each hand had its own inverse model of the form: uk = )~-1(Xk,Xk+l)
(3.4)
Before each hit the system looked up a command with the expected impact state of the devilstick and the desired state:
=?
Inverse model learning was successfully used to train the system to perform the devil sticking task. Juggling runs up to 100 hits were achieved. The system incorporated new data in real time, and used databases of several hundred hits. Lookups took less than 15 milliseconds, and therefore several lookups could be performed before the end of the flight of the center stick. Later queries incorporated more measurements of the flight of the center stick and therefore more accurate predictions of the state x k of the task. However, the system required substantial structure in the initial training to achieve this performance. The system was started with a default command that was appropriate for open loop performance of the task. Each control parameter was varied systematically to explore the space near the default command. A global linear model was made of this initial data, and a linear controller based on this model was used to generate an initial training set for the memory-based system (approximately 100 hits). Learning with no initial data was not possible. We also experimented with learning based on both inverse and forward models. After a command is generated by the inverse model, it can be evaluated using a memory-based forward model with the same data: xk+, = )~(Xk,fik)
(3.6)
Because it produces a local linear model, the LWR procedure generates estimates of the derivatives of the forward model with respect to the commands as part of the estimated parameter vector [3. These derivatives can be used to find a correction to the command vector that reduces errors in the predicted outcome based on the forward model: - ~ aa = x k + l - x d
(3.7)
The pseudo-inverse of the matrix ~ / oM is used to solve the above equation for Aft k in order to handle situations in which the matrix is singular or a different number of commands and states exists (which does not apply for devil sticking). The process of command refinement can be repeated until the forward model no longer produces accurate predictions of the outcome. This will happen when the query to the forward model requires significant extrapolation from the current database. We investigated this method for incremental learning of devil sticking in simulations. The outcome, however, did not meet expectations: without sufficient initial data around the setpoint, the algorithm did not work. Two main reasons can be held responsible:
147 i)
ii)
Similar to the pure inverse model approach, the inverse-forward model acts as a one-step deadbeat controller. One-step deadbeat control applies unreasonably high commands to correct for deviations from the setpoint. In the presence of errors in the model, this is detrimental since it magnifies the model errors. Additionally, the workspace bounds and command bounds of our devil sticking robot limit the size of the commands. Due to the nonlinearities in the dynamics of the robot, the 10-dimensional input space of the forward model suffers from the first symptoms of Bellman's "curse of dimensionality". Error reduction as described in (3.7) only works if sufficient data exists at the query sites. The inevitable model errors will make the robot explore randomly, leading to dispersed data. This gives little chance for model improvements. Imagine we had to place data in a (hyper-)cube of normalized edge length 0.1. In a 3-dimensional input space there are 103 such cubes which leaves some probability to finally arrive at the goal. In a 10-dimensional state space, there are 101~such cubes - a prohibitive number for random exploration. Thus, two ingredients had to be added to the devil sticking controller:
a)
b)
Control must start as soon as possible with the primary goal to increase the data density in the current region of the state-action space, and the secondary goal to arrive at the desired goal state. Control actions must avoid deadbeat properties and must be planned to go to the goal in multiple steps.
Both requirements are fulfilled by a simple exploration method, the shifting setpoint algorithm (SSA) (Schaal and Atkeson, 1994). The SSA attempts to decompose the control problem into two separate control tasks on different time scales. At the fast time scale, it generates controls as a nonlinear regulator by trying to keep the controlled system at the setpoints. On a slower time scale, the setpoints are shifted to accomplish a desired goal. The SSA tries to explore the world by going to the fringes of its data support in the direction of the goal. It extends its knowledge in the fringes until statistically sufficient data has been collected to make a further step towards the goal. In this way the SSA builds a narrow tube of data support in which it knows how to control. This data can be used by more sophisticated control algorithms for planning or further exploration. Applied to devil sticking, the SSA proceeds as follows: (1) Regardless of the poor juggling quality of the robot (i.e., at most two or three hits per trial), the SSA makes the robot repeat these initial actions with small random perturbations until a cloud of data is collected somewhere in state-action space of each hand. An abstract illustration for this is given in Figure 5-top-left---> 5-top-right. (2) Each point in the data cloud of each hand is used as a candidate for a setpoint of the corresponding hand by trying to predict its output from its input with LWR. The point achieving the narrowest local confidence interval becomes the setpoint (Xs,Us) of the hand and an LQ controller is calculated for its local linear model: x s = A x s + B u s+c, or Xs,ou t
:
X s --C
:
Ax s + Bu s
(3.8)
148 As in (3.7), the linear model is estimated by LWR. By means of these controllers, the amount of data around the setpoints can quickly and rather accurately be increased until the quality of the local models exceeds a certain statistical threshold.
Figure 5: Abstract illustration how the SSA algorithm collects data in space: top-left: sparse data after the first few hits; top-right: high local data density due to local control in this region; bottom-left: increased data density on the way to the goals due to shifting the setpoints; bottom-right: ridge of data density after the goal was reached
149 (3) At this point, the setpoints are gradually shifted towards the goal setpoints until the data support of the local models falls below a statistical value. The shifting algorithm executes as follows: 1) calculate the error of the predicted output state: e r r s , o u t = Xdesire d - - X S , o u t 2) solve the equation: &S,out
O3Us
m u s ._
BAus = a err s out
for Au s , e.g., by singular value decomposition (Press et al, 1989); o~ e [0,1] determines how much of the error should be compensated for in one step. 3) update u s" u s = u s + Au s and calculate new predicted output setpoint state Xs,ou , with a LWR lookup. 4) assess the fit for the updated setpoint with the help of prediction intervals. If the quality is above a certain threshold, continue with 1), otherwise terminate shifting. Shifting occurs for both x s and Xs,ou ,. After shifting, the kernel k (cf. Eqn. (2.9) is optimized by minimizing the local cross validation error M S E c.... . (In Figure 5, the goal setpoints are given explicitly, but they can also develop automatically from the require ment to throw the devilstick increasingly close to a place, in which the other hand has data, i.e., X desired,lefi : X s,right and X desired,right : X S,lefi .) (4) The SSA repeats itself by collecting data in the new regions of the workspace until the setpoints can be shifted again (Fig. 5-bottom-left). The procedure terminates by reaching the goal, leaving a (hyper-) ridge of data in space (Figure 5-bottom-right). The LQ controllers play a crucial role for devil sticking. Although we statistically exploit data rather thoroughly, it is nevertheless hard to build good local linear models in the high dimensional forward models, particularly at the beginning of learning. LQ control has useful robustness even if the underlying linear models are imprecise.
Figure 6: Learning curves of devil sticking for three runs.
150 We tested the SSA in a noise corrupted simulation and on the real robot. Learning curves of the real robot are given in Figure 6. The learning curves are typical for the given problem. It takes roughly 40 trials before the setpoint of each hand has moved close enough to the other hand' s setpoint. For the simulation a break-through occurs and the robot rarely loses the devilstick after that. The real robot (Figure 6) takes more trials to achieve longer juggling runs. In our first experiments, its performance was not very consistent. Applying LWR and a LQ controller in the above described form turned out to undersample certain directions in control space. Regression analyses need full rank data matrices to achieve good results. By undersampling one direction, the regression matrix becomes close to singular. To avoid this problem some small amount of uniformly distributed noise was added to the controls. The performance of three runs is shown in Figure 6 (note the scale of the ordinate). On average, humans need roughly a week of 1 hour practicing a day before they learn to juggle the devil stick. With respect to this, the robot demonstrated rather nice learning performance, but the controller is still local and could not correct large perturbations, and due to the boom the task is slightly simplified. Future work will attempt to further improve learning rate and robustness. 4. DISCUSSION In this paper we adopted a nonparametric approach to leaming control. By means of locally weighted regression we built models of the world first, and exploited the models subsequently with statistical methods and algorithms from optimal control to design controllers. Despite the computational complexity of these methods, we demonstrated the usefulness of our algorithms in a real-time implementation of robot learning. Using models for control according to the certainty equivalence principle is nothing new and has been supported by many researchers in the last years (e.g., An et al., 1988; Sutton, 1990; Jordan, 1992; Moore, 1991). Using memory-based or non parametric models, however, has only recently received increasing interest. One of the advantages of memory-based modeling lies in the least commitment strategy which is associated with it. Since all data is kept in memory, a lookup can be optimized with respect to the few open architec tural parameters. Parametric approaches do not have this ability if they discard their training data; if they retained all the training data they essentially become memory-based. As we demonstrated in our LWR approach to nonparametric modeling, several established statistical methods may be adopted to assess the quality of a model. These statistics form the backbone of the SSA exploration algorithm. So far we have only examined some of the most obvious statistical tools which directly relate to regression analysis. Many other methods may be suitable, too, particularly from Bayesian statistics. Training a memory-based model is computationally inexpensive, as the data is simply stored in a memory. Training a nonlinear parametric model typically requires an iterative search for the appropriate parameters. Examples of iterative search are the various gradient descent techniques used to train neural network models (Hertz et al., 1991). Lookup or evaluating a memory based model is computationally expensive, as described in this paper. Lookup for a nonlinear parametric model is often relatively inexpensive. If there is a situation in which a fixed set of training data is available, and there will be many queries to the model after the training data is processed, then it makes sense to use a nonlinear parametric model. However,
151 if there is a continuous stream of new training data intermixed with queries, as there typically is in many motor learning problems, it may be less expensive to train and query a nonparametric memory-based model then it is to train and query a nonlinear parametric model. A question that often arises with memory-based models is the effect of memory limitations. Memory use can be minimized based on several methods. One approach is to only store "surprises". The system would try to predict the outputs of a data point before trying to store it. If the prediction is good, it is not necessary to store the point. Another approach is to forget data points. Points can be forgotten or removed from the database based on age, proximity to queries, or other criteria. Because memory-based learning retains the original training data, forgetting can be explicitly controlled. Additionally, we also presented a method for data compression. Data compression on basis of k-d tree partitioning will be useful when sufficient data was collected in a region such that there is enough confidence to build a local parametric model. The piecewise local models which are generated by our method can be used in form of a lookup table which quickly provides predictions and derivatives at a query point. Smoothing across partition can be done by interpolation and will guarantee continuous first derivatives. The LWR-k-d-tree can also be built on-line and in combination with the regular LWR learning. If the k-d tree does not provide a valid partition for a query, the system falls back to the normal LWR mode, otherwise the result is generated by the LWR-k-d-tree. That computational complexity does not necessarily limit real time applications was demonstrated with our successful devil sticking robot. We are able to do lookups for memorybased local models in less than 15 milliseconds for a thousand data points modeling a 10 to 5 mapping, and we are able to build on-line LQ controllers in another 5ms. The initial shortcomings of our deadbeat inverse or inverse-forward model controllers are not due to the LWR learning algorithm but rather to the inherent problems of this kind of control. As has been pointed out by Jordan (1992), inverse models are not goal-directed and perform data sampling in action and not state space. They do not establish a connection between a certain sensation and a certain action but rather a connection between two sensations. Hence, they do not learn from bad actions. A forward model overcomes these problem. Pure forward model controllers, however, are still deadbeat controllers which try to cancel the plant dynamics in one step. This results in large commands if the system deviates only moderately from its desired goal and conflicts with the workspace bounds and command bounds of our robot. Additionally, modeling errors are strongly amplified by deadbeat control. Accurate data sampiing, as it is necessary in high dimensional spaces, becomes thus rather difficult. Due to the statistical properties of locally weighted regression, a simple exploration algorithm like the shifting setpoint algorithm is powerful enough to accomplish the desired task. Deadbeat control was replaced by LQ control which naturally blends into the LWR framework. By no means was the SSA algorithm intended to replace high-level controllers. Indeed, it remains to be explored in how far the chaining of individual LQ controllers is actually robust, and whether an approach from trajectory optimization (Dyer and McReynolds, 1970) would not be more appropriate. In favor of the SSA algorithm stands its easy implementation for real-time systems. Two crucial prerequisites entered our explanations on robot learning. First, we assumed we know the input/output representations of the task, and second, we were able to generate a goal state for the SSA exploration. A good choice of a representation is crucial in order to be able to accomplish the goal at all (Schaal and Atkeson, 1993, Schaal and Sternad, 1993), and
152 we have very limited insight so far how to automate this part of the learning process. Of equal importance is a good choice of a goal state. In devil sticking, the goal state developed out of the necessity that the left and right hand have to cooperate. The initial action, however, which was given by the experimenter, clearly determined in which ballpark the juggling pattern would lie. Certain patterns of devil sticking are easier than others (Schaal et al., 1992), and we picked an initial action of which we knew that it was in the right ballpark. One part of our future work will address these issues in more detail in that we search for good initial actions and strategies to approach a task. 5. CONCLUSIONS The paper demonstrated that a real robot can indeed learn a non-trivial task. As pointed out above, by taking input/output representations and good learning goals as given, a large portion of the task was already solved in advance. Solving the remaining problems became practicable mainly because of the characteristics of the LWR learning method. The local linear models that this algorithm generated at every query point allowed us to make use of optimal control techniques which added useful robustness to the controllers. Since LWR is memory-based, the local linear models could be optimized with respect to statistical uncertainty measures. These measures also served as the basis of the SSA exploration algorithm. Such statistical tools are not generally available in learning. LWR is particularly suited to exploit statistics since it originates from a statistical method, and we could thus easily assure compatibility of the statistics and the learning algorithm. As a last point, LWR does not suffer from problems of interference when being trained on new data. Interference means a degradation of performance in one part of the model when training the model with data relevant for different parts. Such an effect could happen during SSA shifting if a parametric learning method were applied. But since lookups with LWR are affected only by a small neighborhood of the query point, interference problems are greatly reduced. Our future work will focus on extending LWR model-based control to multistage problems in the optimal control domain. Devil sticking should not only be stable within the validity of the local linear controllers but rather exhibit global stability. This requires nonlinear optimal control and planning techniques which we are currently exploring. Future work must furthermore address how we could approach tasks in which complete measurements of the states are not available, or what constitutes a state is not even known. ACKNOWLEDGMENTS Support was provided by the Air Force Office of Scientific Research, by the Siemens Corporation, the German Scholarship Foundation and the Alexander von Humboldt Foundation to Stefan Schaal, and a National Science Foundation Presidential Young Investigator Award to Christopher G. Atkeson. We thank Gideon Stein for implementing the first version of LWR on a DSP board, and Gerrie van Zyl for building the devil sticking robot and implementing the first version of learning of devil sticking.
153
REFERENCES Aboaf, E. W., Atkeson, C. G., & Reinkensmeyer, D. J. (1988). "Task-level robot learning." In: Proceedings of the IEEE International Conference on Robotics and Automation, April 24-29, Philadelphia, Pennsylvania. Aboaf, E. W., Drucker, S. M., & Atkeson, C. G. (1989). "Task-level robot leafing: Juggling a tennis ball more accurately." In: Proceedings of lEEE Interational Conference on Robotics and Automation, May 14-19, Scottsdale, Arizona. An, C. H., Atkeson, C. (3., & Hollerbach, J. M. (1988). Model-based control of a robot manipulator. Cambridge, MA: MIT Press. Atkeson, C. G. (1992). "Memory-based approaches to approximating continuous functions." In: Casdagli, M., & Eubank, S. (Eds.), Nonlinear Modeling and Forecasting, pp.503-521. Redwood City, CA: Addison Wesley. Barto, A. G., Bradtke, S. J., & Singh, S. P. (1991). "Real-time learning and control using asynchronous dynamic programming." Technical Report 91-57, University of Massachusetts at Amherst, Amherst: Univ. of Massachusetts. Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1984). Classification and regression trees. Belmont, CA: Wadsworth International Group. Cleveland, W. S. (1979). "Robust locally weighted regression and smoothing scatterplots." Journal of the American Statistical Association, 74, pp.829-836. Dyer, P., & McReynolds, S. R. (1970). The computation and theory of opitmal control. New York: Academic Press. Farmer, J. D., & Sidorowich (1987). "Predicting chaotic time series." Phys. Rev. Lett., 59 (8), pp.845-848. Friedman, J. H., Bentley, J. L., & Finkel, R. A. (1977). "An algorithm for finding best matches in logarithmic expected time." A CM Transactions on Mathematical Software, 3 (3), pp.209-226. Hertz, J., Krogh, A., & Palmer, R. G. (1991). Introduction to the theory of neural computation. Redwood City, CA: Addison Wesley. Jordan, I. M. (1992). "Supervised learning with a distal teacher." In: Cognitive Science, 16, 3, pp.307-354. Mallows, C. L. (1966). "Choosing a subset regression." unpublished paper presented at the annual meeting of the American Statistical Association, Los Angles,. Moore, A. (199 l a). "Fast, robust adaptive control by learning only forward models." In: Moody, J. E., Hanson, S. J., & and Lippmann, R. P. (Eds.), Advances in Neural Information Processing Systems 4. San Mateo, CA: Morgan Kaufmann. Myers, R. H. (1990). Classical and modern regression with applications. Boston, MA: PWSKENT. Press, W. P., Flannery, B. P., Teukolsky, S. A., & Vetterling, W. T. (1989). Numerical recipes in C - The art of scientific computing. Cambridge, MA: Press Syndiacate University of Cambridge.
154 Schaal, S., Atkeson, C. G., & Botros, S. (1992b). "What should be learned?." In: Proceedings of Seventh Yale Workshop on Adaptive and Learning Systems, p.199-204, New Haven, CT. Schaal, S., & Atkeson, C. G. (1993b). "Open loop stable control strategies for robot juggling." In: IEEE International Conference on Robotics and Automation, 3, pp.913-918, Georgia, Atlanta. Schaal, S., & Stemad, D. (1993c). "Learning passive motor control strategies with genetic algorithms." In: Nadel, L., & Stein, D. (Eds.), 1992 Lectures in complex systems. Redwood City, CA: Addison-Wesley. Schaal, S., & Atkeson, C. G. (1994a). "Robot juggling: An implementation of memory-based learning." Control Systems Magazine, 14, 1, pp.57-71. Sutton, R. S. (1990). "Integrated architectures for learning, planning, and reacting based on approximating dynamic programming." In: Proceedings of the International Machine Learning Conference. van Zyl, G. (1991). "Design and control of a robotic platform for machine learning." MS Thesis, Cambridge, MA: Massachusetts Institute of Technology, Dept. of Mech. Engineering. Wahba, G., & Wold, S. (1975). "A completely automatic french curve: Fitting spline functions by cross-validation." Communications in Statistics, 4 (1),.
Intelligent Robots and Systems V. Graefe (Editor) 9 1995 Elsevier Science B.V. All rights reserved.
155
Behavioral control in mobile-robot navigation using a fuzzy decision making approach H.R. Beom a, K.C. Koh a and H.S. Chob aResearcher, Research & Development Laboratory, LG Industrial Systems, Co., Ltd. 533 Hogae-dong, Dongan-gu, Anyang, 430-080, Korea bprofessor, Center for Robotics and Automation, Department of Precision Engineering and Mechatronics, Korea Advanced Institute of Science and Technology 373-1 Kusong-dong, Yusong-gu, Taejon, 305-701, Korea
This paper proposes a sensor-based mobile robot navigation which utilizes a multi-criterion decision-making method in uncertain environments. In order to acquire information about the environment around a mobile robot, ultrasonic sensors mounted on the front of the mobile robot are used. The multi-crtefion decision-maker uses fuzzy sets to select a reactive and purposive behavior in accordance with a global mission from a higher-level planner and the situation around the robot. The reactive and purposive behaviors correspond to avoidance and goal-seeking behaviors, respectively. The two behaviors are designed to produce the prescribed responses to currently perceived sensory information using fuzzy logic and reinforcement learning. A decision function to appropriately select one of two behaviors is formulated by use of a fuzzy set. The optimal alternative of two behaviors is defined as the one achieving the higher degree of membership in the fuzzy decision function. The fuzzy decisionmaker provides the mobile-robot navigator with a degree of flexibility which cannot be found in conventional decision-making methods. The effectiveness of the proposed method is verified by a series of simulations. 1. INTRODUCTION Path planning is one of the most vital tasks in navigation of autonomous mobile robots and is usually divided into two categories. One is global path planning based on a priori complete information about the environment and the other is local path planning based on sensory information in a cluttered environment. Local path planning utilizes the information provided by sensors such as an ultrasonic sensor, vision, laser rangefinder, proximity sensor or touch sensor. Brooks [1] applied the force-field concept to the obstacle avoidance problem for a mobile robot equipped with ultrasonic sensors whose readings are used to 'compute the resultant repulsive force. Borenstein and Koren [2] proposed the vector-field histogram method for a fast-running mobile robot equipped with ultrasonic sensors. However, with the above methods, it is difficult to find the force coefficients that influence the velocity and the direction of the mobile robot in a cluttered environment. As the altematives, fuzzy logic and a neural network have been employed in obstacle avoidance for mobile-robot navigation [3]. The neural network informs the mobile robot of its
156 situation at the present instant. According to the situation, fuzzy rules are brought into play to cause the mobile robot to act. However, it is difficult to construct and tune the rule bases. Thus, in the earlier study [4], a navigation method using avoidance behavior and goal-seeking behavior was proposed. Each behavior was designed to make the mobile robot act in accordance with currently perceived sensory information by using the fuzzy logic and reinforcement learning. Recently, advanced behavior-based systems have been proposed to give a purposive capability to the mobile robot. Arkin [5] proposed a motor schema-based architecture describing the interaction between perception and action of the mobile robot. The behaviorbased approach decomposes the system into individual modules, each of which is responsible for one behavior to be performed by the mobile-robot system. Whenever the mobile robot navigates in a cluttered environment towards the goal position, the two behaviors, avoidance behavior and goal-seeking behavior, always conflict. Hence, in order for the mobile robot to arrive at the goal position without colliding with the obstacles, a method to effectively fuse the behaviors is required. Saffiotti et al. [6] have proposed a behavior-blending method using fuzzy logic, and Salichs et al. [7] have proposed a method using reinforcement leaming. In this study, a new way of combining two behaviors using a multi-criterion decisionmaking method is proposed. A general decision-making problem can be expressed in terms of the simultaneous satisfaction of conflicting criteria. In conventional decision-making problems, a single scalar criterion, the weighted sum of the deviation from an ideal goal value, is constructed and optimized to identify non-dominated solutions. One drawback of such a formulation is that the optimum solution tends to be sensitive to numerical weights. The fuzzy decision-making approach models objective functions and constraints in the form of fuzzy goals and then transforms the problem into a conventional optimization using fuzzy weights. The decision function is a combination of criteria obtained by some aggregation operations. Two well-known aggregation operations are intersection and union. In multi-criterion decisionmaking using fuzzy sets, decision alternatives are usually obtained by constructing a decision function. In this study, the fuzzy decision function is constructed by using the sensory information about the environment around the mobile robot and global mission from a higherlevel planner. 2. MOBILE-ROBOT NAVIGATION ARCHITECTURE The proposed navigation method is used for a wheeled mobile robot named LCAR which has been developed in the Laboratory for Control Systems and Automation, KAIST. As shown in Fig. 1, the robot has four wheels: two drive wheels fixed at the sides of the mobile robot and two castors attached at the front and rear of the robot. The robot moves and changes its heading angle by the rotation of the two drive wheels. Four DC servo motors are used to drive the two wheels and one camera mounting device. LCAR is equipped with a camera, a laser lightstripe, ultrasonic sensors and a DSP board. The mobile-robot navigation architecture employed in this study is shown in Fig. 2. The architecture consists of the following modules: higher-level planner, perception module, primitive behaviors, fuzzy decision-maker, and lower-level controller. The higher-level planner, giving the goal and sub-goal positions, consists of a global path planner and a mission planner. The primitive behaviors produce a prescribed robot action to the currently perceived sensory information. According to the intervention of the fuzzy decision-maker, the output of only one
157 behavior transfers to the lower-level controller, called the dead reckoner. The lower-level controller controls the robot, taking into account the limitation of its driving angle and maximum speed. The primitive behaviors and fuzzy decision-maker will be explained in detail later.
Figure 1. The wheeled mobile robot, LCAR
Figure 2. The mobile-robot navigation architecture
2.1. Perception system using ultrasonic sensors Environments of mobile-robot navigation can be classified as static or dynamic. The dynamic environment is defined as follows: obstacles can move or change their shapes, and some obstacles may not initially be registered on the global map. In this study, eighteen ultrasonic sensors mounted in the front face of the mobile robot are used to understand environmental situations, for example, what types of obstacles exist in front of the mobile robot or whether the robot can proceed without colliding with obstacles. These sensors act both as transmitters and receivers. Fig. 3 shows the arrangement of ultrasonic sensors mounted in the front face of the mobile robot with sensors being marked as dots in the figure. The distances ej ( j - 1,2 ..... 18), from the origin of the robot frame {R} to obstacles detected by jth sensor Sj, can be defined as ej - 8j + R. Here, R is the radius of the robot and the 6j is thejth sensor reading shown as a dashed line. In this figure, ~pj denotes the orientation of thejth sensor with respect to the flame {R}. Thus, the orientations of ultrasonic sensors can be expressed by the relationship: ~oj - (j - 9.5)~, (j = 1,2,...,18).
(1)
158 Here, the angle ~ denotes the angle between two adjacent sensors and is fixed at 11.25~. The number, 9.5, is required to express the orientation angles of the sensors installed at the fight and the let't hand sides from x-axis by negative and positive values, respectively.
Figure 3. The arrangement of ultrasonic sensors and sensor suites It is assumed that the y-axis of the coordinate frame {R} is aligned with the direction of motion of the mobile robot. As shown in Fig. 3, in order to avoid the increase of the dimension of input space, sensor suites are introduced. A sensor suite consists of a group of neighboring sensors. The first and the filth sensor suites, ssa and s s 5, a r e composed of three neighboring sensors, while the other sensor suites are composed of four neighboring sensors. The distance d~ ( i - 1,2 ..... 5) measured by the ith sensor suite from the center of the mobile robot to the obstacle, and the direction #i of the obstacle with respect to the frame {R} are defined, respectively, by R + min(6j IJ : 4 i - 4, 4 i - 3, 4 i - 2, 4 i - 1) /f i : 2,3,4 at, =
+min(6j Ij = 2'-~, 2 '-' +1,2 '-I +2)
7r.
~i -- "~-(1 --
3)
9
/ f i = 1,5
(2)
(3)
The ultrasonic sensors are controlled by an 8031 microcontroller communicating with a PC/AT computer. Their control boards consist of a transmitter module to fire the pulses, a receiver/gain-control module to receive the attenuated echo pulses, and a switching module to arbitrarily select the activation of the sensor according to the sensing mode. In the simulation study, it is assumed that the sensors can ideally detect the obstacles irrespective of inclination angle. 2.2. Primitive behaviors The primitive behaviors may be classified as follows: goal-seeking behavior, wall-following
159 behavior, keep-away behavior, free-space explorer, and emergency stop. The primitive behavior can be characterized by a temporal sequence of appropriate values for its heading velocity v and incremental steering angle A0 which cause the robot to exhibit the prescribed response to sensory information. Thus, the output vector of a primitive behavior n(t) is defined as u ( t ) = (v(t),AO) r
(4)
where T denotes the transpose and t denotes the time step of the robot controller. From now on, the index t is omitted for notational simplicity. The wall-following behavior follows the boundary of stationary obstacles maintaining a prespecified distance from the center of the mobile robot. This behavior considers the minimum radius of curvature of the mobile robot. The free-space explorer provides the mobile robot with movement in a direction with the greatest distance between the robot and obstacle boundaries. The keep-away behavior basically makes the mobile robot avoid obstacles in its proximity, and produces an output vector corresponding to movement in the direction that ensures no collision with obstacles. This behavior takes information from measurements provided by the front sensors. Since the above behaviors produce mobile-robot actions based on the currently perceived sensory information, we will unify them as an avoidance behavior producing the same action as the individual behaviors do. The goal-seeking behavior produces an output which directs the robot towards a specific goal defined by a point ( x , y ) in the world coordinate frame {W}. This behavior receives the goal command from the higher-level planner and produces the mobile-robot action that causes the robot to move smoothly towards the goal position. This takes into account the minimum radius of curvature of the mobile robot. Therefore, it corresponds to purposive behavior. In this study, we will design the navigator using the avoidance behavior and the goal-seeking behavior. The fuzzy decision-maker decides the next behavior based on the fuzzy goals. Let {u~,u2,...,n,} be a set of outputs of behaviors and {q~,q2,...,q.} be an output of the fuzzy decision-maker. If the ith behavior is chosen by the fuzzy decision-maker, then q~(t) is equal to 1 and the others are zero. Thus, the mobile-robot motion command transferred to the lowerlevel controller at time step t is u(t) - ~ q~(t)u,(t) = q,(t)ul(t ) + q2 (t)u2 (t) + "'" + q~(t)u,(t) + ... + q , ( t ) u , ( t ) = u, (t).
(5)
i
2.3. Design of two behaviors Our goal is to design a navigator operating in an uncertain environment. In the previous study [4], we designed two behaviors using fuzzy logic and reinforcement learning. In this study we will deal with how to choose one of two behaviors using fuzzy decision-making Design of the two behaviors proceeds in the following sequence: (A) fuzzification of the input/output variables, 03) rule-base construction through reinforcement learning, (C) reasoning process, and (D) defuzzifieation of output variables. The fuzzy decoder partitions the input space formed by the information from internal and external sensors into subspaces. The partitioned input spaces are represented by the rules. Each behavior is composed of fuzzy
160 decoder, decision making, rule base, reinforcement-learning mechanism, and defuzzifier. The goal of navigation in an uncertain environment is to make the mobile robot arrive at a destination point without colliding with obstacles. In order to accomplish the navigation effectively, we must control the mobile-robot motion by considering the obstacle position Xoi =(Xo~,Yoi), the robot's position X = (x,y), and its heading angle 0 with respect to the world coordinate frame {W} shown in Fig. 4.
Figure 4. The coordinate frames and control variables The motion of the mobile robot is characterized by its heading velocity v and incremental steering angle A0. Thus, we choose the input variables for avoidance behavior, 4 -IlXo,- xll 1,2 ..... 5), and those for goal-seeking behavior, heading-angle difference and distance to goal z=
IIx,- xll
output v
u
for two behaviors is expressed by the
velocity v and the incremental steering angle A0. Here, the subscript i denotes the number of the sensor suite detecting the obstacles in front of the robot and I1~ denotes the Euclidean norm. The variable d~ is calculated by Eq. (2). The angle g is that between the heading direction of the mobile robot and the direction of the goal position, z is the distance from the current position X-(x,y) to the goal position Xg- (xg,yg) with respect to the frame {W}. All the variables are defined in Fig. 4. Using these variables, a rule base is constructed for each behavior to infer two actions. Avoidance behavior and goal-seeking behavior have 324 rules and 56 rules, respectively. The output fuzzy sets are expressed by the linguistic values with membership functions having the triangular shaped functions. Initially, all the center values of the membership functions of the output fuzzy sets, representing the linear velocity and the incremental steering angle, are set to constants. When the control-action signals are applied to the mobile robot at time step t the robot moves by the amount of the movement vector re(t). As a result of the series of movement vectors {m(0),..., m(t - 3), m(t - 2), m(t - 1), re(t)}, shown in Fig. 5, the mobile robot may collide with the obstacle or move away from the goal position. When the robot navigates
161 through a section of the environment similar to the previously experienced one, the rules contributed to generate the control action must be corrected in order to avoid collision. The rules influencing the mobile robot's failures are the rules used at the previous time steps t, t - 1, t - 2 . . . . . Thus, such rules must be changed into the correct rules. This task is accomplished by two neuronlike adaptive elements [8] consisting of an associative search element and an associative critic element. By use of these dements, the center values of the membership functions of output fuzzy sets approach the correct values as the learning step progresses.
Figure 5. The movement vectors associated with colliding with an obstacle or moving away from the goal position
3. BEHAVIOR SELECTION Whenever the mobile robot navigates in an uncertain environment towards the goal position, two behaviors, avoidance and goal-seeking, always conflict. Therefore, for the complete navigator, it is necessary for the fuzzy decision-maker to efficiently select one of the behaviors.
3.1. Multi-criterion decision-making using fuzzy sets In 1970, Bellman and Zadeh considered the classical model of a decision and suggested a model for decision making in a fuzzy environment [9]. They considered a situation of decision making under certainty, where the objective function and constraints were characterized by their membership functions. Since we want to satisfy the objective function and the constraints simultaneously, the decision in a fuzzy environment is defined by analogy to that in a crisp environment as selecting the one among the alternatives which satisfies the objective function and the constraints. Thus, the decision in the fuzzy environment can be considered as the intersection of fuzzy objective function and fuzzy constraints. In this case there is no difference between objective function and constraints. Usually it is impossible for a single criterion to determine an optimal solution to a decision problem and to judge the suitability of the solution. Thus, a multi-criterion decision-making has been proposed and it has led to numerous evaluation schemes. The multi-criterion decisionmaking using fuzzy sets can be divided into two major areas: multi-objective decision-making
162 and multi-attribute decision-making [10]. The former concentrates on continuous decision space. The latter focuses on the problem with discrete decision spaces. The problem for selection of appropriate behavior corresponds to multi-attribute decision-making. Let u = ( u 1, u2,..., u,) be a set of alternatives resulting from primitive behaviors and a set of fuzzy goals (~ = ((~1,(~,..., (~m), by which the suitability of a behavior is judged. The jth fuzzy goal (~s is characterized by its membership function/aoj(u). In the following, the tilde sign (-~) representing the fuzzy sets will be dropped for notational simplicity. However, there are some cases where some goals and constraints are of greater importance than others. In such cases, the fuzzy decision function D might be expressed as the goals with weighting coefficients reflecting the relative importance of the constituent terms. The importance of goal Gs is expressed by the exponent a s. Thus, the fuzzy decision function is defined as the intersection of all the fuzzy goals (6)
D = G~' N GZ~ N ... N G:, ..
The rationale behind using weights as exponents to express the importance of a goal can be found in [10]. The greater the importance of a goal, the larger is the exponent of its fuzzy set, if the fuzzy sets are normalized. The condition aj > 1 means more importance, while a / < 1 means less importance. The problem is to determine one of the alternatives u i with the highest degree of suitability with respect to all relevant goals Gj, j = 1,2,...,m. The fuzzy set decision D in discrete space as the intersection of the Gj(u~) becomes
o- [{o,. min o
}.,- ....
1
(7)
The relative importance, a/, of the goals is established in the form of matrix A by pairwise comparison as follows:
A
=
I a,/b, a,/a~ ... a,/a. a2(a~ .
.
(8)
By applying Saaty's eigenvector method [11] to matrix A, consistent weights ws for the membership function of the j th goal are determined. As an alternative to the above, a fuzzy set decision D may be calculated by
o: [{.i, 1 , 2 , . . I,. =, 1,2,...,o;j: o ] ,
(9)
163 m
where
~ wj = 1.0. This method is useful when the number of fuzzy goals is small. The J optimal alternative is defined as the one achieving the highest degree of membership in D.
3.2. Fuzzy goals in the navigation problem In order for the mobile robot to arrive at its goal position without colliding with obstacles, an appropriate behavior must be selected according to the situation around the robot. When the obstacles are located around the mobile robot as shown in Fig. 4, the robot must reduce its speed in order not to collide with them. If the maximum acceleration am~x is applied to the mobile robot with an arbitrary initial velocity, the minimum time required to avoid the obstacle is calculated by trn~ - v c o s t p J a ~
x .
(10)
If a constant acceleration less than the maximum value is applied to the mobile robot, the time required to avoid the obstacle is calculated by to -
2(ej
-
R)/(veos~j).
(11)
Therefore, when the distance from the center of the mobile robot to the obstacle and the velocity component towards the obstacle are given, the marginal avoidance time tma r shown in Fig. 6 can be defined as the difference between the above two required times.
Figure 6. The marginal avoidance time
Figure 7. The minimum time to transfer the mobile robot to the goal position
Since the marginal avoidance time denotes the degree of collision, it is used in this study to represent the repulsive potential. Repulsive potential E r e ~ is produced by the inversion of the marginal avoidance time, which is a function of the mobile robot's velocity and the distances to the obstacles measured by the ultrasonic sensor array. Thus, the repulsive potential is defined by
164 su x"
Krar~xv COS~j 2am~ (ej - R ) - v2 cos2 %
(12)
where K r is a scale factor set to 0.0003 and SN is the number of ultrasonic sensors. Repulsive potential increases as the mobile robot goes closer to the obstacles, and approaches zero as its velocity in the direction of an obstacle approaches zero. When the mobile-robot configuration is as given in Fig. 4, the attractive potential is defined in order to transfer the robot with initial velocity v to the goal position. This potential is calculated using the minimum time tg as follows:
:
c~
+4a xl[X-
x ll-v ~
(13)
ar~x
Attractive potential depends on the distance to the goal position, the heading angle difference, and the initial velocity. As can be seen from Eq. (13) and Fig. 7, attractive potential increases as the mobile robot moves away from the goal position. The situation around the robot can be represented by the use of two potentials, Erep and Ea. When two behaviors are assumed to apply at the next step, the control actions derived from each behavior are calculated. Then, the changes of potential, Crep(Ui ) "-" AEre p (Ui) and cau(u~) = AEau(U~) (i = 1 (avoidance), 2 (goal-seeking)) resulting from the actions of the two behaviors are calculated. In order that the mobile robot can move through the environment to the goal position without colliding with obstacles, the changes of the weighted sum of the two potentials must be negative. However, it is very difficult to find the weighting coefficients in a cluttered environment. Furthermore, the uncertainties in the sensor measurements can be introduced into the potentials. Hence, in this study, one behavior to be used at the next action step is selected by the fuzzy decision-maker. To perform this, three fuzzy goals are defined by Gl: the change in repulsive potential Crev (Ui) shouM be smaller than q and G 2: the change m attractive potential Can(Ui) shouM be smaller than c2 and G 3:the repulsive potential Erev (u~) should be smaller than E 1 9
Here, the connective "and" denotes the intersection of fuzzy goals. The membership functions of the fuzzy goal are defined by 1.0 ~s 1(Crep (Ui )) -
~s (Catt(Ui)):
1.0 1.0 + M 1(Crep(Ui) + C1) Nl
i f Crep(Ui) ~_ C1
otherwise
g c=,(u,)_< c~ 1.0 1.0 otherwise 1.0 + Mz (c~u (u, ) + c2 ) N~
(14)
(15)
165
if E,,,(,,,)__ E, otherwise ~as (Er~v(ui))= 1.O+Ms(Er,v(ui)+E,)Ns 1.0 1.0
Here,
(16)
Nj, j = 1, 2, 3 are even integers and the Mj, j = 1,2, 3 are positive constants. The q, c2
and E1 are determined through a series of simulations. As can be seen from the above equations, all the fuzzy sets are normalized. 4. SIMULATION AND RESULTS 4.1. Simulation conditions As an illustration of the foregoing process, a series of simulations were carried out using an arbitrarily constructed environment with sixteen obstacles. It was assumed that the radius of the mobile robot is 0.2m and the extent of the environment is 10m • 10m. The maximum speed of the robot was assumed to be 35cm/s. The measurement range of the sensor was assumed to be eight times the radius of the mobile robot. In order to acquire the information about the environment, a sensor simulator giving the range data of the ultrasonic sensors was used. For the sensor simulator, it was assumed that the ultrasonic sensors can ideally detect the nearest obstacles within their beams. The correct rule bases for each behavior were those constructed in the previous study [4] and they were employed here for the simulations to select a behavior using the fuzzy decision-maker. In the simulations, the parameters Nj, j = 1,2,3 determining the shape of the membership
functions of the fuzzy goals, were all set to 2. The constant values, q, c~ and E 1, were determined through a series of simulations and were set to -0.01, -0.02 and 1.0 respectively. 4.2. Simulation of the selection of behaviors By use of Eqs. (2) and (3), the eighteen distance values measured by the sensors are grouped into the five distance values belonging to each sensor suite. The behaviors produce the prescribed responses to these five distance values. Let us assume that avoidance and goalseeking behaviors are both used at the next action step. Then, we can obtain two movement vectors corresponding to the outputs of the two behaviors. Using these two vectors and the mobile robot's current position and orientation, the potentials are calculated by Eqs. (12) and (13) and their changes are calculated. Using these values, the membership degrees representing the degree of fuzzy goal satisfaction are determined by Eqs. (14) through (16). Then, the minimum value of the membership degrees is chosen. This value denotes the degree of behavior satisfaction with respect to fuzzy goals. Thus, the fuzzy decision set D represented by Eq. (7) is obtained. The optimal alternative, the behavior to be used at next step, is determined by the behavior with the highest degree of membership in fuzzy decision set D. To test the effectiveness of the method using the fuzzy decision-maker, various start positions and initial heading angles were employed. In all simulations, the goal position is fixed and assumed to be located at the upper comer of the right-hand side within the environment as shown in Fig. 8. The weights wj (j = 1,2 ..... m) reflecting the relative importance of the fuzzy goals must be
determined. It was assumed that all the goals are of equal importance. Since a 1, % and a 3 in Eq. (8) can be equally set to 1.0, we can obtain the eigenvalue set as {0.0, 0.0, 3.0}. After
166 calculating three eigenvectors we can determine one eigenvector with the elements of the same sign as w = {w~,w2,w3} r = {1.0,1.0,1.0} r among the three eigenvectors.
Figure 8. The results of execution of navigation Fig. 8(a) shows the result of execution of navigation when M 1, M 2 and M 3 are all set to 0.2. The mobile robot can arrive at the goal position by passing through the environment without colliding with any obstacles. Fig. 8(b) shows the result of execution of navigation under the same conditions as above, except that M 2 = 0.05. As can be seen from Eqs. (14) through (16), the parameter M~ plays the role of modifier. The modifier is an operation which
167 changes the meaning of a fuzzy set. The case with the small _M2 corresponds to the case when w2 < 1.0. Therefore, this corresponds to the case when the fuzzy goal (7,_ associated with the change in attractive potential is less important than the others. As can be seen from this figure, the smaller the value of M 2, the larger should be the number of time steps for arrival at the goal position. Fig. 8(c) shows the result of execution of navigation when a different start position and initial heading angle are given. Fig. 8(d) shows the result of execution of the case when a local minimum occurs. As can be seen from these figures, even if the mobile robot is faced with situation where a local minimum occurs, the navigator with fuzzy decision maker shows a feasibility to escape from the local minimum. We believe that the proposed navigator with the fuzzy decision-maker enables the mobile robot to navigate through the environment more smoothly if more than three fuzzy goals are used. 5. CONCLUSIONS A behavior selection method based on fuzzy decision-making has been presented for mobile-robot navigation in complex environments. The mobile robot's situation at any instant is classified by calculating the repulsive and attractive potentials using both the currently perceived sensory information and the goal position provided by the higher-level planner. The decision function to select the behavior suitable in the vicinity of the robot is formulated as a fuzzy set using the fuzzy goals. Then, optimal selection of behaviors is determined by achieving the highest degree of membership in a fuzzy decision set. The fuzzy decision-making provides a flexibility which cannot be found in conventional decision-making. Furthermore, it is easy to formulate the relevant problems since there is no difference between goals and constraints in the fuzzy environment. In this study, even if three fuzzy goals are used, the mobile robot can move towards the goal position through the environment without colliding with obstacles. It has been shown that the proposed method enables the robot to navigate through complex environments where a local minimum occurs. REFERENCES
1. R.A. Brooks, "A robust layered control system for a mobile robot", IEEE J. of Robotics and Automation, vol. RA-2, no. 1, pp. 14-23, 1986. 2. J. Borenstein and Y. Koren, "Real-time obstacle avoidance for fast mobile robot", IEEE Trans. Syst. Man Cybern., vol. 19, no. 5, pp. 1179-1187, Sept./Oct. 1989. 3. H.R. Beom and H.S. Cho, "A sensor-based obstacle avoidance controller for a mobile robot using fuzzy logic and neural network", in Proc. Int. Conf. Intelligent Robots and Systems, pp. 1470-1475, 1992. 4. H.R. Beom and H.S. Cho, "A sensor-based navigation for a mobile robot using fuzzy logic and reinforcement learning", IEEE Trans. Syst. Man Cybern., vol. 25, no. 3, Mar. 1995. 5. R.C. Arkin, "Motor schema-based mobile robot navigation", International Journal of Robotics Research, vol. 8, no.4, pp. 92-112, Aug. 1989. 6. A. Saffiotti, E.H. Ruspini and K. Konolige, "Blending reactivity and goal-directedness in a fuzzy controller", in Proc. IEEE Int. Conf. Fuzzy Systems, pp. 134-139, 1993. 7. M.A. Salichs, E.A. Puente, D. Gachet and J.R. Pimentel, "Learning behavioral control by reinforcement for an autonomous mobile robot", in Proc. IECONV93, pp. 1436-1441, 1993.
168 8. A.G. Barto and R.S. Sutton, "Neuronlike adaptive elements that can solve difficult learning control problems", IEEE Trans. Syst. Man Cybem., vol. SMC-13, no. 5, pp. 834-846, Sept./Oct. 1983. 9. R.E. Bellman and L.A. Zadeh "Decision making in a fuzzy environment", Management Science, vol. 17, no.4, pp. 141-164, Dec. 1970. 10. H.J. Zimmermann, Fuzzy set theory and its application. Kluwer-NijhoffPublishing, 1985. 11. T.L. Saaty, "Exploring the interface between hierarchies, multiple objectives and fuzzy sets", FSS 1, pp.57-68, 1978.
Intelligent Robots and Systems V. Graefe (Editor) 9 1995 Elsevier Science B.V. All rights reserved.
169
Learning Emergent Tasks for an Autonomous Mobile Robot D. Gachet, M.A. Salichs and L. Moreno
Dpto. Ingenier[a, Universidad Carlos III de Madrid, Butarque 15, 28911, Leganfis, Madrid, Spain e-mail: gachet~ing.uc3m.es We present an implementation of a reinforcement learning algorithm trough the use of a special neural network topology, the AHC (Adaptive Heuristic Critic). The AHC is used as a fusion supervisor of primitive behaviors in order to execute more complex robot behaviors, for example go to goal, surveillance or follow a path. The fusion supervisor is part of an architecture for the execution of mobile robot tasks which are composed of several primitive behaviors which act in a simultaneous or concurrent fashion. The architecture allows for learning to take place at the execution level, it incorporates the experience gained in executing primitive behaviors as well as the overall task. The implementation of this autonomous learning approach has been tested within OPMOR, a simulation environment for mobile robots and with our mobile platform Robuter. Both, simulated and actual results are presented. The performance of the AHC neural network is adequate. Portions of this work has been implemented within the EEC ESPRIT 2483 PANORAMA Project. 1. I n t r o d u c t i o n The process of guiding autonomous mobile robots in unstructured, dynamic, and complex environments is difficult. A multitude of problems arise such as noisy measurements, lack of appropriate sensors, difficulty in achieving real-time sensor processing, and constructing an appropriate model of the environment based on sensory information. For example, when working with ultrasonic range sensors there are inaccuracies in the values measured due to noise or to the physics of the ultrasound waves. Thus obtaining an accurate state of the environment is difficult. To deal with the aforementioned problems, researchers are advocating the development and use of intelligent systems not only in mobile robots but also in diverse areas such as manufacturing, control, vision and communications. Intelligent systems are difficult to define and indeed there are several definitions in the literature. There are also several theories of intelligent systems [1-4] and intelligent machines [7-10]. Regardless of a precise definition of an intelligent system, most definitions include learning as one of its main attributes. In addition to learning, the intelligent systems that we advocate follow a behavioral
170
methodology where all robot tasks are performed in terms of a set of elementary tasks referred to as primitive behaviors. The fundamental idea of behavioral control is to view a robot mission or task, also referred to as an emergent behavior as the temporal execution of a set of primitive behaviors. One of the most challenging problems in behavioral control involves the learning the appropriate mixture of primitive behaviors in order to execute complex tasks. For this purpose, in our research group several approaches have been investigated [5-7]. Our experience indicates that perhaps the most important learning requirement for autonomous systems is that of autonomous learning, in other words, the ability to learn from self-exploration or observation without the need of examples or a teacher. Although there is a great deal of research on learning in a general context [1-3], research on autonomous learning is just beginning. The neural network paradigm of reinforcement learning appears promising for autonomous learning. Accordingly, the goals of this paper are to discuss the reinforcement learning with a neural network AHC topology in the context of a behavioral control methodology for mobile robots, and to discuss needs for future research. 2. T h e A F R E B Control Architecture Several researchers have already argued the importance of looking at a mobile robot as a set of primitive behaviors. Primitive behaviors are also important components of reactive control which is a recently emerged paradigm for guiding robots in unstructured and dynamic environments [10,11]. Mobile robots must continuously interact with its environment and this is the essential characteristic of reactive programs. By reactive, we mean that all decisions are based on the currently perceived sensory information. An e m e r g e n t behavior can be defined as a simple or complex task or a task which is made up of more elementary (primitive) behaviors. Whereas primitive behaviors correspond to robot actions, emergent behaviors correspond to mission, goals, or subgoals. We have developed a behavioral control architecture called AFREB Adaptive Fusion of Reactive Behaviors for experimenting with a wide range of reactive control methodologies [5],[12]. The architecture is depicted in Figure 1 and basically consists of the following modu-
les:behavioral analysis, fusion supervisor, behavior primitives, and executor. The function of behavior primitives and fusion supervisor will be explained later, the function of the executor module is the calculation of the actual robot commands while enforcing control limitations due to non-holonomic constraints (e.g., limitation of driving angle of the robot, maximum speed and so on) and exhibiting robot reflexive behavior (e.g., emergency stop). We have used the behavioral architecture in several simulation and actual experiments with our mobile robot Robuter. All simulation work has been performed with OPMOR [13].
2.1. Behavior Primitives The set of primitive behaviors active at any given time depends upon the occurrence of specific events in the environment (e.g., the detection of an obstacle, time for the robot
171
UPPER LEVELS
BEHAVIORAL - Task decomposition ANALYSIS - Chek for inconsistences I - Check for constrains - Enable behaviors
( R o b o t Model~
I
, os,o
I P
L
E
SUPERVISOR Evaluate modified tasks
~1
al
9
n
C1
c2
- Produce robot ,,C,,,n commands for individual behaviors
R
C E P T
~r
PRIMITIVE BEHAVIORS
ains IlJ
i
I
- Final robot commands ] I Enforce control limitations I EXECUTOR i range, rates (vel.) [ i Safety monitoring I Emergency stop ]
i I
I
O N
~r W: Dynamic Environment
Figure 1. The A F R E B Control Architecture
--
172 to recharge itself, the issuance of an interactive command, etc). The primitive behaviors (i.e., actions) which have been implemented are: cl :
goal attraction
c2 :
perimeter following (contour left)
c3 :
perimeter following (contour right)
c4 :
free space
c5 :
keep away (from objects)
C6
follow a path
"
A primitive behavior can be characterized by a temporal sequence of appropriate values for linear velocity v(i) and curvature k(i) which cause the robot to exhibit the pre-specified response to sensorial information. Thus we define the output of a primitive behavior c(i) as the vector:
c(i) = (v(i),k(i)) T
(1)
where the variable i denotes the ith cycle of the robot controller. In what follows we will drop the index i for notational simplicity. The goal attraction primitive produces an output which directs the robot towards a specific goal which is defined by a point (x, y) in a global reference frame. The perimeter following behavior follows the perimeter of a fix (i.e., static) obstacle maintaining a pre specified distance away from it. This behavior take into account the minimum radius of curvature R~i~ = v/w~ax of the robuter in order to avoid uneven corners. The movement can be performed left or right. The free space behavior will cause movements in a direction so that the robot frees itself from objects. This behavior takes the information from measurements provided by the frontal sensors and choose the direction of the longer adjacent measures in order to ensure a safe course in that direction. The outputs of this behavior is the direction of motion at a constant speed. The keep away behavior basically avoids obstacles in the proximity of the robot, choosing directions that ensure no collision with nearby obstacles. The follow path behavior drives the robot closest to a pre- defined path. The path is composed by several cartesian coordinates (xi, yi) in the robot's environment.
2.2. D y n a m i c Fusion of Primitive Behaviors The fundamental assumption of the fusion supervisor module is that more complex behaviors, also called emergent behaviors can be obtained in terms of primitive behaviors, in the following way: If we denote Cl, c2,..., cn the output of each primitive behavior, then the output of an emergent behavior (i.e., mission, task, or subtask) is: N
= E i=1
(2)
173 Where the ai coefficients, with 0 _< ai _< 1 are found by an appropriate combination of measurement information provided by the perception system. Thus, the main function of an intelligent controller is the learning of the weights ai so that the performance of the robot for the execution of the tasks is adequate. There are two possibilities for learning the coefficients ai: on line or off line. For autonomous systems the on line learning method is the most interesting. Whether or not learning is present it is to be noted that the coefficients ai are dynamically adjusted by the controller when the robot is in motion. For the system described above, the learning problem dealt with in this paper can be formulated as follows" how can the system learn the appropriate values for the ai coefficients (i.e., mixture of primitive behaviors) in order to execute a given task in a on-line, real-time fashion without the intervention of a trainer?. In the context of the AFREB architecture, the learning process is performed within the fusion supervisor module. For demonstrating the power of autonomous learning approach, we have implemented the following emergent behaviors (i.e., tasks or missions)" M~
~
M2
~
surveillance of the environment (exploration). motion between two points (i.e., towards a goal) while avoiding unexpected obstacles.
M3 " follow a predefined path while avoiding unexpected obstacles. In general, a mission is executed while taking into account the dynamic nature of the environment as sensed by the perception system of the robot. It is important to note that in the case of navigation or follow a path there is a principal primitive behavior (go to goal or follow a path) and the others can be seen as auxiliary behaviors in order to ensure a safe motion in a dynamic and unstructured environment. 3. T h e A d a p t i v e H e u r i s t i c Critic as Fusion S u p e r v i s o r In this section, we summarize a fusion supervisor module which learns the values of the relative gains ai by the use of an external reinforcement signal [14] which rewards or punishes the actions of the mobile robot. The objective is the successful completion of missions M1 to M3,( i.e., surveillance of the environment, navigation and follow a path). A learning system that is built based on this technique is not provided with any sets of instances or examples of a concept. In addition, the learning system may have to deal with a number of observations that represent several concepts rather than just concentrating on a single concept at a time. In the learning approach based on exploration or observation the system generates both, a set of control rules and appropriate examples autonomously. The Neural Network used is called Adaptive Heuristic Critic (AHC). The architecture of the AHC topology is depicted in Figure 2, where r(t) represents the external reinforcement signal to be used in the learning algorithm. Our implementation of the AHC neural network consists of a classifier system which has five or six inputs, depending on the mission, that correspond to a zone arrangement of frontal and lateral ultrasonic sensors, the output of this module is a situation that
174
Figure 2. The A H C Neural Network
consists in a vector of 32 binary elements with only one of them with value 1 at any time (64 elements in the case of navigation and follow a path missions). The AHC Network has an output layer with 4 processing elements for mission M1 or 5 processing elements for missions M2 and M3. This is called the Associative Search Element (ASE), and computes the values of a~ (the relative gains for primitive behaviors) in the following way : k ---,grb
ai = f( ~
Ik. Wki + cr(t))
(3)
k=O
Where Wki is a matrix of adaptive weights between the input and output layers, a(t) is a random value in the interval [-1,1] with a uniform distribution, rn is the number of inputs. The function f is defined by:
f(x)
_ f
1
if x > 0
- 1 otherwise
The weight matrix W is modified according to the following law :
Wki(t + 1) = Wki(t) + a . bi(t) . eki(t)
(4)
Where a is the learning rate, In our case a = 3, eki(t) is the eligibility of the weight Wki for reward or punishment. This is an important concept because by it we can reward or punish not only the latest action but the predescesors. The eligibility is modified b y :
ek~(t + 1) = 5. eke(t) + (1 - 5). I k ' a i
(5)
175 With 5 is a value in the range [0,1] and in our case 0.3. The internal reinforcement signal bi(t) is given by the other layer ACE (Adaptive Critic Element), this layer computes the prediction P~(t) of b~(t) at time t : k~'m,
=
(6) k=0
Where Vki are the weight connections between the classifier and ACE layer. The internal reinforcement signal is computed by : bi(t) = r ( t ) + q,. P i ( t ) - P i ( t - 1)
(7)
Where 7 is a positive constant in the range [0,1], r(t) is the external reinforcement signal and the weights Vki are updated b y :
(8) Where/3 is a positive constant and Xk is the eligibility for this layer and :
Xk(t +
1) = s
(1 - s
Ik(t)
(9)
With ~ between 0 and 1, in our case 0.4. The four or five output nodes correspond to the relative contributions of the last four primitive behaviors (i.e., a2 through as) for the surveillance mission, al through as for the navigation and a2 to a6 for the follow path mission. 4. S i m u l a t i o n E x p e r i m e n t s 4.1. S u r v e i l l a n c e R o b o t For the surveillance mission, the external reinforcement r(t) provides a graded performance value describing how well the system performed (-1 = robot collision and 0 -- no collision). Roughly the system works as follows: At the beginning, the weights of the AHC network were initialized with random values and the robot executes random movements with r(t) evaluated continuously and used for changing the internal networks weights. When collisions occur the robot is returned to the position it was 30 steps before the collision took place and the process is resumed from there. The learning process involves the association of actions with the external reinforcement (i.e., whether collisions are present) for each situation. As a result of the learning process, the system learns how to move across the environment without collisions. Figure 3 depicts the initial collisions of the robot in an initial simulation experiment whereas Figures 4 and 5 depict the path of the robot in subsequent simulation experiments where the weights of the NN were stored at the end of each experiment and used in the following. It can be seen that the number of collisions decrease as the robot goes through more learning cycles. After a sufficient number of collisions, the robot navigates with virtually no collisions in the simulator including unknown environments as in the case of Figure 5.
176
Figure 3. Initial Path of the Surveillance robot
Figure 4. Intermediate path of the Surveillance robot
177
Figure 5. Final path of the Surveillance robot in an unknown environment
4.2. G o t o G o a l In the case of execution of mission M2 the external reinforcement signal r(t) take a negative value-1 in the following cases:
9 When the robot is too close to an obstacle (imminent collision) 9 When the frontal sensors do not detect obstacles in the goal direction and the current heading of the robot is too far from the correct direction. In all other cases the reinforcement signal is 0. In this case there is an additional input in the AHC network as compared to the surveillance mission, this input is a binary signal indicating whether obstacles are present in the goal direction. The execution loop is the same as described above for the surveillance robot. We can see in Figure 6, the first environment used for simulating the execution of the AHC network. In this case the weights of the network were initialized with random values, the networks start by giving random values to outputs al to a5 in the first cycles because the system does not have any knowledge of the environment. In this first cycles the robot accepts many punishments until the networks decide to utilize the contour followin 9 and free areas behaviors in order to avoid collisions, some cycles later the network uses the 9oal attraction. When the robot advances in the appropriate direction and no collisions occurs, it accepts rewards. Figure 7 depicts the path of the robot in a subsequent simulation environment where the weights of the NN were stored at the end of each experiment and used in the following.
178
Figure 6. Go to goal, initial path
Figure 7. Go to goal, final path
179 4.3. Follow P a t h In the case of execution of this mission, the path is defined a priori as a set of points (x~,y~), the external reinforcement signal r(t) take a negative value -1 in the following situations: 9 When the robot is too close to an obstacle. 9 When the frontal sensors do not detect obstacles and the current heading of the robot is too far from the direction of the path's next point. In all other cases the reinforcement signal is 0.
Figure 8. Follow a path, first environment
We can see in Figures 8 and 9, the first and the last environment used for simulating the execution of this mission. The control cycle is the same described above for the missions M1 and /1//2. In the follow path mission, the primitive behaviors used are c2 to c6 5. E x p e r i m e n t s w i t h t h e R o b u t e r The final step is to execute the control software on the actual robot, Robuter, which is a mobile platform with two differential wheels, and equipped with a belt of 24 ultrasonic sensors as shown in Figure 10. Perhaps the main advantage of our software development environment is that the control software developed under the simulator is virtually the same as that used to control the actual vehicle.
180
Figure 9. Follow a path, second environment
Figure 10. The Robuter vehicle
181
In most cases the only changes made to the control software in the simulator before it controls the actual robot are of two categories: lowering the speed of the robot, and adjusting some parameters (e.g., the distance away from contours when following it). Other changes involved additional sensor processing (e.g., a filter) to deal with highly noisy sensors, sensor failures, multiple reflections, and no-reflections. The filter used with experiments with the actual robot works as follows: if a range measurement is below 30 cm (the lower limitation of the ultrasonic sensor) it is discarded and replaced with the mean of two valid measurements at each side of the sensor in question. This filter has proven practical to solve many problems dealing with bad range measurements. The physical laboratory environment used correspond to a small room of 6.5 m x 4.5 m where we usually put boxes as obstacles. The size of the room poses a number of practical problems as it does not leave enough space to maneuver. Thus the room is an excellent test bed for maneuvering in tight spots. By contrast, we did not have similar maneuvering problems in the simulation environment. As noted, by reducing the velocity of the robot and adjusting some parameters of some primitive behaviors we were able to run all our simulation control algorithms with the real vehicle in the small room.
Figure 11. Motion of the Robuter executing the Surveillance mission
Figures 11 and 12 depicts the movement of the Robuter in the room where we have placed four boxes at different locations and orientations. Figure 11 shows the execution of the mission M1 i.e. surveillance, and Figure 12 depicts the movement of the robot execution mission M2, in this case the goal point is located in the upper right corner of the lab.
182
Figure 12. Path of the vehicle with the Go to goal mission
As noted above, there are only two differences in the control software running with the robot as that running with the simulator: the actual robot speed is smaller and the number of back steps after a collision is reduced from 30 to 10.
6. D i s c u s s i o n
A major problem with the use of reactive behavioral control is the correct fusion of the primitive behaviors in order to execute complex tasks. In order to solve this problem, it is clear that the autonomous learning is an interesting paradigm, because it liberates us from explicitly programming or training the system via examples. The implementation of the autonomous learning paradigm presented here, demonstrates the feasibility of this approach by the execution of some missions for an autonomous mobile robot, the observation of the overall behavior of the robot confirms the idea of that the self-learning methodology trough the use of the behavioral control is possible. Some drawbacks are present, for example the incomplete coverage of situations due to the limitations in the number of inputs for the classifier system. The characteristics of the environment are very important in the learning process of the AHC, the choice of the best set of situations is a non trivial problem, because we must take care of not training the network with contradictory situations. In other words the reinforcement signal must always be coherent, and the design of a correct external reinforcement signal is perhaps the most important step in this approach.
183 7. Conclusions and F u t u r e W o r k s The successful completion of three missions through the application of autonomous learning has been presented in this paper. The use of a special neural network topology AHC as a fusion supervisor of primitive behaviors within a control architecture AFREB has been demonstrated. We have built a system that learns by itself how to execute the missions, surveillance, go to goal and follow a path, the simulated and real experiments demonstrate that the autonomous learning through reinforcement is a viable technology for motion control of non- holonomic robots operating in complex environments. In future publications we will present the implementation of others task with this methodology for example the execution of a complex missions composed of several sub-missions. We are also attempting to solve the dimculties which arise due to the poor resolution of the inputs with the use of a Kohonen Neural Network.
Acknowledgments We wish to thank Prof. J.R. Pimentel for his efforts in the Panorama project. The first author wishes to thank the Direccidn General de Investigacidn Cientifica y T~cnica (DGICYT) of the Spanish Ministry of Education and Science for research funding.This research has been founded by the Commission of the European Communities ( Project ESPRIT 2483 PANORAMA ) and the Comisidn Interministerial de Ciencia y Tecnolog{a CICYT (Projects ROB90-159 and ROB91-64). REFERENCES
1. W. Fritz, R.G. Martinez, J. Banque, A. Rama, R.E. Adobbati, and M. Sarno, The Autonomous Intelligent System, Robotics and Autonomous Systems, Vol. 5, pp. 109125, 1989. 2. P.F.M.J. Verschure, B.J.A. Krose, and R. Pfeifer, Distributed Adaptive Control: The Self Organization of Adaptive Behavior, Robotics and Autonomous Systems, Vol. 9, pp. 181-196, 1992. 3. A. Famili, Integrating Learning and Decision-Making in Intelligent Manufacturing Systems, Intelligent and Robotics Systems, Vol.3, pp. 117-130, 1990. 4. W. Fritz, World View and Learning Systems, Robotics and Autonomous Systems, Vol. 10, pp. 1-7, 1992. 5. D. Gachet, M.A. Salichs, J.R. Pimentel, L. Moreno, A. de la Escalera, A Software Architecture for Behavioral Control Strategies of Autonomous Systems, Proc. IECON'92, pp. 1002-1007, San Diego CA, Nov. 1992. 6. E.A Puente, D. Gachet, J.R. Pimentel, L. Moreno, and M.A. Salichs, A Neural Network Supervisor for Behavioral Primitives of Autonomous Systems, Proc. IECON'92, San Diego, CA, Nov. 1992. 7. D. Gachet, J.R. Pimentel, L. Moreno, M.A. Salichs, and V. Fernandez, Neural Network Control Approaches for Behavioral Control of Autonomous Systems, 1st IFAC Int. Workshop on Intelligent Autonomous Vehicles, Southampton, UK, pp. 330-334, April 1993. 8. G.Pang, A Framework for Intelligent Control, Intelligent and Robotics Systems, Vol. 4, pp. 109-127, 1991.
184
10. 11. 12.
13.
14.
W. Davis, A. Jones, and A. Saleh, Generic Architecture for Intelligent Control Systems, Computer-Integrated Manufacturing Systems, Vol. 5, No. 2, pp. 105-113, 1992. R. A. Brooks, A Robust Layered Control System for a Mobile Robot, IEEE Journal on Robotics and Automation, RA-2, pp 14-24, Apr. 1986. R.C. Arkin, Motor Schema-Based Mobile Robot Navigation, The International Journal of Robotics Research, Vol. 8, No. 4, pp. 92-112, Aug. 1989. M.A. Salichs, E.A. Puente, L. Moreno and J.R. Pimente, A Software Development Environment for Autonomous Mobile Robots, Chapter 8 of Recent Trends in Mobile Robots. Ed. Y.F. Zheng. World Scientific. 1993. J.R. Pimentel, E.A. Puente, D. Gachet, and J.M Pelaez, OPMOR: Optimization of Motion Control Algorithms for Mobile Robots, Proc. IECON'92, pp. 853-861, San Diego CA, Nov. 1992. J. del R. Millan, and C. Torras, Learning to Avoid Obstacles Through Reinforcement: Noise-tolerance, Generalization, and Dynamic Capabilities, Proc. IROS'92, Raleigh, N.C., pp. 1801-1807, Jul. 1992.
Intelligent Robots and Systems V. Graefe (Editor) 1995 Elsevier Science B.V.
185
Efficient reinforcement learning of navigation strategies in an autonomous robot Jos~ del R. Millgm a and Carme Torras b a Institute for Systems Engineering and Informatics, European Commission, Joint Research Centre, TP 361, 21020 Ispra (VA), Italy bInstitut de Cibern~tica (CSIC-UPC), Diagonal, 647, 08028 Barcelona, Spain
This paper describes a reinforcement connectionist learning architecture that allows an autonomous mobile robot to adapt to an unknown indoor environment. As a result, the robot learns efficient reactive navigation strategies. The robot learns on top of built-in reflexes. Also, the neural controller is a modular network which is built automatically. In this way, the robot is operational from the very start and improves its performance rapidly and incrementally as it safely explores the environment. These appealing features make this learning robot's architecture very well suited to real-world applications. The paper also reports experimental results obtained with a real mobile robot that demonstrate the feasibility of this approach.
1.
INTRODUCTION
Efficient navigation is critical for autonomous robots operating in hostile environments, which are usually unknown the first time robots face them. This paper deals with the problem of controlling an autonomous mobile robot so that it reaches efficiently a goal location in an unknown indoor environment. Instances of the problem where the goal is not inside the perception range of the robot all the time are also considered. An usual approach to this problem is that of reactive systems (e.g., [1]). However, basic reactive systems stiffer from two shortcomings. First, they are difficult to program. Second and most important, pure reactive controllers may generate inefficient trajectories since they select the next action as a function of the current sensor readings and the robot's perception is limited. To address this second shortcoming, some approaches combine planning and reaction (e.g., [2, 3]). In contrast to classical planning that acts on a perfect (or sufficiently good) model of the environment, these approaches only require a coarse global map of the environment made out of landmarks. Then, planning takes place at an abstract level and all the low level details are handled by the reactive component as the robot actually moves. In the case of robots operating in initially unknown environments, this
186
global map can be built from sensory data gathered either while travelling to the goal (e.g., [21) or in a previous exploration phase (e.g., [3]). Global maps are a valuable aid for navigation which must be used when available. When not, we claim that the addition of learning capabilities to reactive systems is sufficient to allow a robot to generate efficient trajectories after a limited experience. This paper presents experimental results that support this claim: a real autonomous mobile robot equipped with low-resolution sensors learns efficient goal-oriented obstacleavoidance reactive strategies in a few trials. Moreover, a learning approach like ours even overcomes the first shortcoming of reactive systems. As some researchers have recently shown, the robot programming cost is considerably reduced by letting the robot learn automatically the appropriate navigation strategies (e.g., [4, 5, 6, 7, 8]). This paper describes the testing of a reinforcement connectionist architecture on a real autonomous mobile robot. A reinforcement-learning robot learns by doing and does not require a teacher who proposes correct actions for all possible situations the robot may find itself in. Instead, the robot simply tries different actions for every situation it encounters and selects the most useful ones as measured by a reinforcement or performance feedback signal. Reinforcement connectionist learning [9, 10, 11] brings four benefits to autonomous robots. First, this kind of learning robot can improve its performance continuously and can adapt itself to new environments. Second, the connectionist network does not need to represent explicitly all possible situation-action rules as it shows good generalization capabilities. Third, connectionist networks have been shown to deal well with noisy input data, a capability which is essential for any robot working upon information close to the raw sensory data. Fourth, connectionist learning rules are well suited to on-line and real-time learning. In addition to these benefits, the architecture described in this paper also overcomes three critical limitations of basic reinforcement connectionist learning that prevent its application to autonomous robots operating in the real world. The first and most important limitation is that reinforcement learning might require an extremely long time. The main reason is that it is hard to determine rapidly promising parts of the action space where to search for suitable reactions. The second limitation has to do with the robot's behavior dttring learning. Practical learning robots should be operational at any moment and, most critically, they should avoid catastrophic failures such as collisions. Finally, the third limitation concerns the inability of "monolithic" connectionist networks ~i.e., networks where knowledge is distributed over all the weights-- to support incremental learning. In this kind of standard networks, learning a new rule (or tuning an existing one) could degrade the knowledge already acquired for other situations. An important assumption of our approach is that the robot receives a reinforcement signal after performing every action. Although this assumption is hard to satisfy in many reinforcement learning tasks, it is not in the case of goal-directed tasks since the robot can easily evaluate its progress towards the goal at any moment. A second assumption of our approach is that the goal location is known. In particular, the goal location is specified in relative cartesian coordinates with respect to the starting location.
187
Figure 1. The Nomad 200 mobile robot.
2.
EXPERIMENTAL
SETUP
The system composed by the robot and the reinforcement connectionist controller has been called T E S E O . The physical robot is a wheeled cylindrical mobile platform of the Nomad 200 family (see Figure 1). It has three independent motors. The first motor moves the three wheels of the robot together. The second one steers the wheels together. The third motor rotates the turret of the robot. The robot has 16 infrared sensors and 16 sonar sensors, from which distances to the nearest obstacles can be estimated, and 20 tactile sensors detect collisions. The infrared and sonar sensors are evenly placed around the perimeter of the turret. Finally, the robot has a dead-reckoning system that keeps track of the robot's position and orientation. The connectionist controller maps the cLtrrently perceived situation into a spatially continuous action. Then, the controller waits until the robot has finished to perform the corresponding motor command before computing the associated reinforcement signal and the next action. We will describe next what the input, output and reinforcement signals are. The input to the connectionist network consists of a vector of 40 components, all of them real numbers in the interval [0, 1]. The first 32 components correspond to the infrared and sonar sensor readings. In this case, a value close to zero means that the corresponding sensor is detecting a very near obstacle. The remaining 8 components are
188 derived from a virtual sensor that provides the distance between the current and goal robot locations. This sensor is based on the dead-reckoning system. The 8 components correspond to a coarse codification of an inverse exponential function of the virtual sensor reading. The main reason for using this codification scheme is that, since it achieves a sort of interpolation, it offers three theoretical advantages, namely a greater robustness, a greater generalization ability and faster learning. The output of the connectionist network consists of a single component that controls directly the steering motor and indirectly the translation and rotation motors. This component is a real number in the interval [-180,180] and determines the direction of travel with respect to the vector connecting the current and goal robot locations. Once the robot has steered the commanded degrees, it translates a fixed distance (25 cm.) and, at the same time, it rotates its turret in order to maintain the front infrared and sonar sensors oriented toward the goal. In this way, a relative codification of both sensor readings and motor commands with respect to the goal direction is always maintained. This codification scheme is directly responsible for TESEO's generalization capabilities. The reinforcement signal is a real number in the interval [-3, 0] which measures the cost of doing a particular action in a given situation. The cost of an action is directly derived from the task definition, which is to reach the goal along trajectories that are sufficiently short and, at the same time, have a wide clearance to the obstacles. Thus actions incur a cost which depends on both the step clearance and the step length. Concerning the step clearance, the robot is constantly updating its sensor readings while moving. Thus the step clearance is the shortest distance measured by any of the sensors while performing the action. Finally, TESEO is equipped with a low-level asynchronous emergency routine to prevent collisions. The robot stops and retracts whenever it detects an obstacle in front of it which is closer than a safety distance.
3.
EXTENSIONS
TO BASIC REINFORCEMENT
LEARNING
TESEO's aim is to learn to perform those actions that optimize the total reinforcement received along the trajectory to the goal. As mentioned in the Introduction, TESEO has been designed to overcome 3 limitations of basic reinforcement learning: lack of incremental improvement, slow convergence, and failure to be operational from the very beginning. The following three aspects of TESEO's architecture address these limitations. First, the connectionist controller is a modular network, each module codifying a set of similar reaction rules. That is, these rules map similar sensory inputs into similar actions and, in addition, they have similar long-term consequences. Modularity guarantees that improvements in the response to a given situation will not negatively alter other unrelated reactions. Second, instead of learning from scratch, TESEO utilizes a fixed set of basic reflexes every time its connectionist controller fails to generalize correctly its previous experience to the current situation. The connectionist controller associates the selected reflex with the perceived situation in one step. This new reaction rule is tuned subsequently through reinforcement learning. Basic reflexes correspond to previous elemental knowledge about
189
Figure 2. Network architecture.
the task and are codified as simple reactive behaviors [1]. Each reflex selects one of the 16 directions corresponding to the current orientations of the infrared and sonar sensors. We have chosen this fixed set of directions because they are the most informative for the robot in terms of obstacle detection. It is worth noting that, except in simple cases, these reflexes alone do not generate efficient trajectories; they just provide acceptable starting points for the reinforcement learning algorithm to search appropriate actions. Integrating learning and reaction in this way allows T E S E O t o focus on promising parts of the action space immediately and to be operational from the very beginning. Third, T E S E O explores the action space by concentrating the search around the best actions currently known. The width of the search is determined by a counter-based scheme associated to the modules (see Section 4.2). This exploration technique allows T E S E O to avoid experiencing irrelevant actions and to minimize the risk of collisions.
4.
NETWORK
ARCHITECTURE
The connectionist controller is a modular two-layer network (see Figure 2). The first layer consists of units with localized receptive fields, which we call exemplars, because each of them represents a point of the input space and covers a limited area around this point. The second layer is made of one single stochastic linear unit, the output unit. There exists a full connectivity between the exemplars and the output unit. 4.1.
Exemplars The activation level of an exemplar is a real value in the interval [0, 1], it being 0 if
190 the perceived situation is outside its receptive field, and 1 if the situation corresponds to the point where the exemplar is centered. The j t h module consists of the exemplars c~,..., e~ and their related links. All modules respond to each perceived situation, but only the module owning the exemplar with the maximum response propagates the activities of its exemplars to the output unit. If no exemplar "matches" the perceived situation --i.e., if the input does not fall in the receptive field of any exemplar--, then the basic reflexes are triggered and the current situation becomes a new exemplar. Section 5.1 provides more details on this resource-allocating procedure. The modules are not predefined, but are created dynamically as T E S E O explores its environment. Every module j keeps track of four adaptive values. First, the width of the receptive fields of all the exemplars in this module, dj. Second, the expected total future reinforcement, bj, that the robot will receive if it uses this module for computing the next action. Third, a counter that records how many times this module has been used without improving the robot's performance, Q. Fourth, the prototypical action the robot should normally take whenever the perceived situation is classified into this module, paj. There are as many prototypical actions as reflexes. Thus the first one is the direction of the front infrared and sonar sensors --i.e., a deviation of 0 degrees from the vector connecting the cm'rent and goal robot locations--, the second one is 22.5 degrees, and so on. Sections 5.1 and 6.2 explain how paj is initially determined and how it evolves, respectively. After reacting, the cvaluator computes the reinforcement signal, z, as specified in Section 2. Then, if the action was computed through the module j, the difference between z and bj is used for learning (see Section 5.4). Only the weights of the links associated to the winning module j are modified. 9
4.2.
Output Unit The output of the connectionist controller is a prototypical action pa, normally the prototypical action of the winning module, plus a certain variation s that depends on the location of the perceived situation in the input subspace dominated by that module (see Figure 3). In order to find a suitable action for each situation through reinforcement learning, T E S E O needs to explore the action space. However, this exploration is not conducted upon the whole action space, but only around the best actions currently known. Since each action is the sum of two components, pa and s, the exploration mechanism works on each of them separately. This exploration mechanism depends on cj, the counter associated to the winning module. On the one hand, the exploration mechanism selects pa from the prototypical actions associated to all the modules that classify the perceived situation. That is, if the situation is located in the receptive field of one of the exemplars of the module m, then pare is a candidate. Assuming j to be the winning module, the selection goes as follows. If cj is not divisible by, say, 3 then paj is chosen. Otherwise, pa is taken to be the prototypical action associated to the module m with the best expected total future reinforcement, bin. The basic idea behind this exploration mechanism is that the winning module could well benefit from the knowledge of neighboring modules. On the other hand, the deviation s from pa is computed through a stochastic process in the interval [-45 ~ 45~ Thus, TESEO will only explore actions between pa and its four
191
Figure 3. The output unit.
neighboring prototypical actions (two to the left and two to the right). The computation of s is done in three steps. The first step is to determine the value of the stochastic process' parameters. The mean # is a weighted sum of the activation levels of the exemplars e l , . . . , e{ of the winning module: n
#=
(1)
k k, k=l
where w~ is the weight associated to the link between e~ and the output unit, and a~ is the activation level of e~. The variance a is proportional to cj. This follows from the idea that the most often the module j is used without improving T E S E O ' s performance, the higher a must be. In the second step, the unit calculates its activation level 1 which is a normally distributed random variable: 1 = N(p, a).
(2)
In the third step, the unit computes s: s=
{
5.
FOUR LEARNING
5.1.
45~ - 5, l,
if I > 45, if 1 < - 4 5 , otherwise.
(3)
MECHANISMS
Network Growth The first learning mechanism makes the controller network grow as a function of the inputs received. Initially, there exist neither exemplars nor, consequently, modules and the resource-allocating procedure creates them as they are needed.
192 As mentioned in Section 4.1, if no exemplar "matches" the perceived situation, then the basic reflexes are triggered and the current situation becomes a new exemplar. That is, both represent the same point of the input space. The weight of the link from this exemplar to the output unit is initially set to zero and evolves subsequently through reinforcement learning. The new exemplar is added to one of the existing modules if its receptive field overlaps the receptive fields of the module's exemplars and the selected reflex is the same as the module's prototypical action. The first condition assures that every module will cover a connected input subspace. If any of the two conditions above is not satisfied, then the new exemplar is added to a new module. This module consists initially of the exemplar and its associated connections. Concerning the four parameters associated to this new module f, they are initially set to the following values: d I equals 0.4, c I equals 0, pa I is the selected reflex, and bI is estimated on the basis of the distance from the next location to the goal and the distance from the next location to the perceived obstacles in between the robot and the goal. T u n i n g Exemplars The second learning mechanism moves the position of the exemplars e ~ , . . . , e~ of the winning module j in order to better cover the input subspace dominated by that module. That is, the coordinates of the k th exemplar, v~, are updated proportionally to how well they matches the input coordinates x: 5.2.
9
v~(t +
I) =
v~(t) + e * a~(t) 9 [x(t) - v~(t)],
(4)
where e is the learning rate. In the experiments reported below, the value of e is 0.1.
5.3.
Improving Reinforcement Estimates
The third learning mechanism is related to the update of the future reinforcement estimates bj, and it is based on temporal difference (TO) methods [12]. Every value bj is an estimate of the total future reinforcement TESEO will obtain if it performs the best currently known actions that take it from its current location (whose associated observed situation is classified into the jth module) to the goal. Consequently, the value bj of the module j should, after learning, be equal to the sum of the cost z of reaching the best next module i plus the value bil:
bj =
max (z) + bi.
Actions
(5)
In order to iteratively update the values of bj, so that finally (5) holds for all of them, we have used the simplest TD method, i.e. TD(0) (see [12] for details). If the situation perceived at time t is classified by module j and, after performing the computed action, the next situation belongs to module i and the reinforcement signal - - o r cost-- is z(t + 1), then:
bj(t + 1) = bj(t) + U * [z(t + 1) + bi(t) - bj(t)].
(6)
1 As described in Section 2, z takes negative values. So, minimizing future cost corresponds to maximizing future reinforcement.
193 The intensity of the modification is controlled by r], which takes the value 0.75 when TESEO behaves better than expected, and 0.075 otherwise. The rationale for modifying less intensively bj when z(t + 1) + b~(t) - bj(t) < 0 is that the error in the estimation is probably due to the selection of an action different from the best currently known one for the module j.
5.4.
Weight U p d a t e
The fourth learning mechanism concerns weight modification and uses the classical associative search (AS) [9, 11]. AS uses the estimation given by TD to update the situation-action mapping, which is codified into the connectionist controller:
w~(t + 1) = wJk(t) + a
*
[z(t + 1) + bi(t)
-
bj(t)]
*
r
(7)
where a is the learning rate, and r is the eligibility factor. The eligibility factor of a given weight measures how influential that weight was in choosing the action. In our experiments, r is computed in such a manner that the learning rule corresponds to a gradient ascent mechanism on the expected reinforcement
[11]: Oln____N , l(t) r (t) = Owjk (t) = a~ (t)
(8)
where N is the normal distribution function in (2). The weights w~ are modified more intensively in case of reward--i.e., when T E S E O behaves better than expected-- than in case of penalty. These two values of a are 0.2 and 0.02, respectively. The aim here is that T E S E O maintains the best situation-action rules known so far, while exploring other reaction rules.
6.
FOUR LEARNING OPPORTUNITIES
Let us present now the four occasions in which learning takes place. The first arises during the classification phase, the next two happen after reacting, and the last one takes place when reaching the goal.
6.1.
Unexperienced Situation
If the perceived situation is not classified into one of the existent modules, then the basic reflexes get control of the robot, and the resource-allocating procedure creates a new exemplar which is added either to one of the existing modules or to a new module.
6.2.
Performing within Expectations
If the perceived situation is classified into the module j and z(t+ 1)+b~(t)-bj(t) >_ kz, where kz is a negative constant, then (i) the exemplars of that module are tuned to make them closer to the situation, (ii) the weights associated to the connections between the exemplars and the output unit are modified using the AS reinforcement learning rule, (iii) bj is updated through TD(0), and (iv) dj, cj, and paj are adapted.
194 The adaptive parameters are updated differently in case of reward than in case of penalty. In case of reward, dj is increased by 0.1, cj is initialized to 0, and if the output of the action unit, pa + s, is closer to a prototypical action other than paj, then paj becomes this new prototypical action. In case of penalty, cj is increased by 1 and dj is decreased by 0.1 if it is still greater than a threshold kd.
6.3.
Performing rather Badly If the perceived situation is classified into the module j and z(t+ 1)+b~(t)-bj(t) < kz, then the topology of the network is slightly altered and dj is decreased by 0.1 if it is still greater than a threshold kd. If the total future reinforcement computed after reacting, z + bi, is considerably worse than the expected one, bj, this means that the situation was incorrectly classified and needs to be classified into a different module. The resource-allocating procedure creates a new exemplar, e~, that has the same coordinates as the perceived situation, but it does not add it to any module. The next time this situation will be faced, e~ will be the closest exemplar. Consequently, no module will classify the situation and the basic reflexes will get control of the robot. Then, the resource-allocating procedure will add e~ either to one of the existing modules or to a new module as described in Section 5.1. 6.4.
Reaching the Goal Finally, whenever the goal is reached, the value bj of every winning module j along the path to the goal is also updated in reverse chronological order. Thus, TESEO needs to store the pairs (j(t),z(t + 1)) along the current path. This supplementary update only accelerates the convergence of the bj's, but does not change their steady values [13].
7.
E X P E R I M E N T A L RESULTS
TESEO's performance has been tested on a corridor with offices at both sides. The task is to generate a short but safe trajectory from inside an office to a point at the end of the corridor. TESEO achieves the target location every time and it never gets lost or trapped into malicious local maxima. The first time it tries to reach the goal it relies almost all the time on the basic reflexes. Essentially, the basic reflexes make TESEO travel in the direction that (i) is the closest to the current one, (ii) is the safest, and (iii) brings TESEO toward the goal. As illustrated in Figure 4, in the first trial, TESEO enters into a dead-end section of the office (but it does not get trapped into it) and even it collides against the door frame because its sensors were not able to detect it. Collisions happened because the frame of the door is relatively thin and the incident angles of the rays drawn from the sensors were too large resulting in specular reflections. Thus this simple task offers TESEO the opportunity to learn three skills. The first one is to smooth out certain sections of the trajectory generated by the basic reflexes. The second skill is to be able to avoid dead-ends or, in general, not follow wrong walls. The third and most critical is to avoid obstacles that its sensors cannot detect. TESEO learns these three skills very rapidly. It reaches the goal efficiently and without colliding after travelling 10 times from the starting location to the desired goal
195
Figure 4. The environment and first trajectory generated for a starting location within the office. Note that T E S E O has some problems in going through the doorway.
(Figure 5). The total length of the first trajectory is approximately 13 meters while the length of the trajectory generated after TESEO has learned the suitable sequence of reactions is about 10 meters. This experiment was run several times, obtaining similar results. Concerning the acquisition of the third skill, TESEO associates safe actions to malicious situations --made out of a coarse codification of the distance to the goal and of uncorrect estimated distances to the obstacles-- immediately after colliding with unperceived obstacles. Moreover, due to its noise tolerance and generalization capabilities (see below), TESEO only hits a few times with obstacles that its sensors cannot detect before learning to avoid them. In addition, the learned navigation strategies have the following featL~es. First, the trajectories are quite smooth even if the basic reflexes have not been programmed in this way (it is rather difficult to do it!!) and the reinforcement signal only penalizes long and unsafe paths. Second, the acquired reactions are robust to noisy sensory data. Since sensors are not perfect, TESEO does not perceive the same situations along similar trajectories. However, it is still able to generate suitable actions using only its acquired reactions. Figure 6 illustrates instances of the reaction rules learned. For every location considered (little circles) the move to be taken is depicted. Figure 6 shows that TESEO generates solution paths from any starting location inside the room. This simulation experiment indicates that TESEO exhibits good generalization abilities, since it can handle many more situations than those previously perceived. Once TESEO has learned efficient navigation strategies, occasional obstacles are put
196
Figure 5. Trajectory generated after travelling 10 times to the goal.
Figure 6. Generalization abilities: Situation-action rules applied for a sample of locations within the office and the first part of the corridor.
197 in the way to the goal. TESEO moves around the obstacles and then it returns to the original, efficient trajectory. In other experiments, the goal is changed. T E S E O learns also to navigate to this new goal in a few trials and it is still able to reach the first goal as efficiently as before.
8.
RELATED
WORK
The robot learning architectme described in this paper shows some resemblances with several previous works. In the case of goal-directed tasks, it is possible to use the future reinforcement associated with situation-action pairs to define a vector field over the environment. Then, the robot will normally perform the action that follows the gradient of the vector field. Seen in this way, our approach is reminiscent of potential field approaches (e.g., [14, 15]). However, our approach differs from them in three fundamental aspects. First, any potential field algorithm relies on predetermining a potential function that will allow the robot to reach the goal efficiently. Contrarily, a reinforcement approach like ours is adaptive in that it starts with an initial and inefficient vector field generated from rough estimates of the future reinforcements and adapts the field to generate efficient trajectories as it converges to the correct estimates through TD methods. Second, the kind of vector fields that can arise in our case are much more varied than those defined by a weighted sum of the potential fields originated by the different obstacles. Third, most of the potential field approaches require a global model of the environment while our approach only needs local information obtained from the robot's sensors. There exist also potential field approaches which only need local workspace models built out of sensory information (e.g., [14]), but they cannot produce efficient trajectories. The benefits of letting the robot learn automatically the appropriate reaction rules have recently been emphasized by several authors. [4] combines reinforcement connectionist learning and teaching to reduce the learning time. In this framework, a human teacher shows the robot several instances of reactive sequences that achieve the task. Then, the robot learns new reaction rules from these examples. The taught reaction rules help reinforcement learning by biasing the search for suitable actions toward promising parts of the action space. In our approach, the basic reflexes play the same guidance role, requiring only a programmer instead of a teacher. [5] use Kohonen maps [16] to split the sensory input space into clusters, and then associate an appropriate action to every cluster through reinforcement learning. Their architecture maps all the situations of a given cluster to a single action, and classifies situations solely by the similarity of their representations. Ore" architecture classifies situations, first, based on the similarity of their input representations. But, then, it also incorporates taskspecific information for classifying based on the similarity of reinforcements received. In this manner, the input space is split into consistent clusters since similar future reinforcement corresponds to similar suitable actions for similar situations. [7] integrate inductive neural network learning and explanation-based learning. The domain theory is previously learned by a set of neural networks, one network for each discrete action the robot can perform. Our basic reflexes also represent prior knowledge about the task; however, they are much more elemental and are used in a different way. Our approach is also related to the Dyna integrated architectures introduced by
198 [17] and further extended by [4, 18, 19], among others. Roughly, Dyna uses planning to speed up the acquisition of reaction rules through reinforcement learning. Dyna also learns a global model of the task on which it plans.
9.
SUMMARY
AND FUTURE
WORK
We have described a reinforcement learning architecture that makes an autonomous mobile robot rapidly learn efficient navigation strategies in an unknown indoor environment. Our robot T E S E O not only is operational from the very beginning and improves its performance with experience, but also learns to avoid collisions even when its sensors cannot detect the obstacles. This is a definite advantage over non-learning reactive robots. T E S E O also exhibits incremental learning, high tolerance to noisy sensory data, and good generalization abilities. All these features make our robot learning architecture very well suited to real-world applications. However, the current implementation of our approach suffers from one main limitation, namely it requires a reliable odometry system that keeps track of the robot's relative position with respect to the goal. Currently, odomtery is totally based on deadreckoning. In all the experiments we have carried out so far, dead-reckoning has proven sufficient to reach the goal. As long as its estimation of the position of the robot does not differ greatly from the actual one, the connectionist controller is still able to produce correct actions. But, dead-reckoning will probably be insufficient in more complicated missions requiring long travels and many turns. Current work focusses on reliable odometry. The goal is designated by a modulated light beacon and the robot is equipped with sensors especially designed to detect that light. Whenever T E S E O detects the goal it uses the beaconing system; otherwise, it relies on dead-reckoning.
ACKNOWLEDGMENTS We thank Aristide Varfis for helpful discussions. JRM is also grateful to people of Nomadic Technologies for their excellent support with the physical robot. The work reported in this paper was conducted in the Institute for Systems Engineering and Informatics, Joint Research Centre of the European Commission. This research has been partially supported by the ESPRIT Basic Research Action number 7274.
REFERENCES
[1] R.A. Brooks, "A robust layered control system for a mobile robot," IEEE Journal of Robotics and Automation, vol. 2, pp. 14-23, 1986. [2] D.P. Miller and M.G. Slack, "Global symbolic maps from local navigation," Proc. of the 9th National Conf. on Artificial Intelligence, pp. 750-755, 1991. [3] J.H. Connell, "SSS: A hybrid architecture applied to robot navigation," IEEE Int. Conf. on Robotics and Automation, pp. 2719-2724, 1992.
199
[4] [5] [6]
[7]
[s] [9]
[10] Ill] [12]
[13] [14] [15]
[16] [17] [is] [19]
L.-J. Lin, "Programming robots using reinforcement learning and teaching," Proc. of the 9th National Conf. on Artificial Intelligence, pp. 781-786, 1991. K. Berns, R. Dillmann, and U. Zachmann, "Reinforcement-learning for the control of an autonomous mobile robot," IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, pp. 1808-1815, 1992. J. del R. Millhn and C. Torras, "A reinforcement connectionist approach to robot path finding in non-maze-like environments," Machine Learning, vol. 8, pp. 363395, 1992. T.M. Mitchell and S.B. Thrun, "Explanation-based neural networks learning for robot control," in C.L. Giles, S.J. Hanson, and J.D. Cowan (eds.), Advances in Neural Information Processing Systems 5, pp. 287-294. San Mateo, CA: Morgan Kaufmann, 1993. J. del R. Mills "Reinforcement learning of goal-directed obstacle-avoidance reaction strategies in an autonomous mobile robot," Robotics and Autonomous Systems, in press. A.G. Barto, R.S. Sutton, and C.W. Anderson, "Neuronlike elements that can solve difficult learning control problems," IEEE Trans. on Systems, Man, and Cybernetics, vol. 13, pp. 835-846, 1983. A.G. Barto, R.S. Sutton, and C.J. Watkins, "Learning and sequential decision making," Tech. Report 89-95, Dept. of Computer and Information Science, University of Massachusetts, Amherst, 1989. R.J. Williams, "Simple statistical gI"adient-following algorithms for connectionist reinforcement learning," Machine Learning, vol. 8, pp. 229-256, 1992. R.S. Sutton, "Learning to predict by the methods of temporal differences," Machine Learning, vol. 3, pp. 9-44, 1988. S. Mahadevan and J. Connell, "Automatic programming of behavior-based robots using reinforcement learning," Artificial Intelligence, vol. 55, pp. 311-365, 1992. J. Borenstein and Y. Koren, "Real-time obstacle avoidance for fast mobile robots," IEEE Trans. on Systems, Man, and Cybernetics, vol. 19, pp. 1179-1187, 1989. J. Barraquand and J.C. Latombe, "Robot motion planning: A distributed representation approach," The International Journal of Robotics Research, vol. 10, pp. 628-649, 1991. T. Kohonen, Self-Organization and Associative Memory. Second Edition. Berlin: Springer-Verlag, 1988. R.S. Sutton, "Integrated architectures for learning, planning, and reacting based on approximating dynamic programming," Proc. 7th Int. Conf. on Machine Learning, pp. 216-224, 1990. L.-J. Lin, "Self-improving reactive agents based on reinforcement learning, planning and teaching", Machine Learning, vol. 8, pp. 293-321, 1992. J. Peng and R.J. Williams, "Efficient learning and planning within the Dyna framework," Adaptive Behavior, vol. 1, pp. 437-454, 1993.
This Page Intentionally Left Blank
Intelligent Robots and Systems V. Graefe (Editor) 9 1995 Elsevier Science B.V. All rights reserved.
201
A Lifelong Learning Perspective for Mobile Robot Control Sebastian Thrun Universit~t Bonn Institut ffir Informatik III R5merstr. 164, 53117 Bonn, Germany E-mail: [email protected], [email protected] Designing robots that learn by themselves to perform complex real-world tasks is a still-open challenge for the field of Robotics and Artificial Intelligence. In this paper we present the robot learning problem as a lifelong problem, in which a robot faces a collection of tasks over its entire lifetime. Such a scenario provides the opportunity to gather general-purpose knowledge that transfers across tasks. We illustrate a particular learning mechanism, explanation-based neural network learning, that transfers knowledge between related tasks via neural network action models. The learning approach is illustrated using a mobile robot, equipped with visual, ultrasonic and laser sensors. In less than 10 minutes operation time, the robot is able to learn to navigate to a marked target object in a natural office environment. 1. I N T R O D U C T I O N Throughout the last decades, the field of robotics has produced a large variety of approaches for the control of complex physical manipulators. Despite significant progress in virtually all aspects of robotics science, most of today's robots are specialized to perform a narrow set of tasks in a very particular kind of environment. Most robots employ specialized controllers that have been carefully designed by hand, using extensive knowledge of the robot, its environment and its task. If one is interested in building autonomous multi-purpose robots, such approaches face some serious bottlenecks. 9 K n o w l e d g e b o t t l e n e c k . Designing a robot controller requires prior knowledge about the robot, its environment and the tasks it is to perform. Some of the knowledge is usually easy to obtain (like the kinematic properties of the robot), but other knowledge might be very hard to obtain (like certain aspects of the robot dynamics, or the characteristics of the robot's sensors). Moreover, certain knowledge (like the particular configuration of the environment, or the particular task one wants a robot to do) might not be accessible at all at the design-time of the robot. 9 E n g i n e e r i n g b o t t l e n e c k . Making domain knowledge computer-accessible, i.e., hand-coding explicit models of robot hardware, sensors and environments, has often been found to require tremendous amounts of programming time. As robotic
202 hardware becomes increasingly more complex, and robots are to become more reactive in more complex and less predictable environments, the task of hand-coding a robot controller will become more and more a cost-dominating factor in the design of robots. 9 T r a c t a b i l i t y b o t t l e n e c k . Even if the robot, its environment and its goals can be modeled in sufficient detail, generating control for a general-purpose device has often been found to be of enormous computational complexity (e.g., [18]). Moreover, the computational complexity often increases drastically with the complexity of the mechanical device. Machine learning aims to overcome these limitations, by enabling a robot to collect its knowledge on-the-fly, through real-world experimentation. If a robot is placed in a novel, unknown environment, or faced with a novel task for which no a priori solution is known, a robot that learns can collect new experiences, acquire new skills, and eventually perform new tasks all by itself. For example, in [5] a robot manipulator is described which learns to insert a peg in a hole without prior knowledge regarding the manipulator or the hole. Maes and Brooks [8] successfully applied learning techniques to coordinating leg motion for an insect-like robot. Their approach, too, operates in the absence of a model of the dynamics of the system. Learning techniques have frequently come to bear in situations where the physical world is extremely hard to model by hand (e.g., the characteristics of noisy sensors). For example, Pomerleau describes a computer system that learns to steer a vehicle driving at 55mph on public highways, based on input sensor data from a video camera [15]. Learning techniques have also successfully been applied to speed-up robot control, by observing the statistical regularities of "typical" situations (like typical robot and environment configurations), and compiling more compact controllers for frequently encountered situations. For example, Mitchell [11] describes an approach in which a mobile robot becomes increasingly reactive, by using observations to compile fast rules out of a database of domain knowledge. However, there is a fundamental shortcoming in most of to date's rigorous learning approaches. Most of the robot control learning approaches focus on learning to achieve single, isolated performance tasks. If one is interested in learning with a minimum amount of initial knowledge, as is often the case in approaches to robot learning, such approaches have a critical limiting factor: the number of training examples required for successful generalization. The more complex the task at hand and the lesser is known about the problem beforehand, the more training data is necessary to achieve the task. In many robotics domains, the collection of training data is an expensive undertaking due to the slowness of robotics hardware. Hence, it does not surprise that the time required for realworld experimentation has frequently been found to be the limiting factor that prevents rigorous machine learning techniques from being truly successful in robotics. The task of learning from scratch can be significantly simplified by considering robots that face whole collections of control learning problems over their entire lifetime. In such a lifelong robot learning scenario [27], learning tasks are related in that they all play in the same environment, and that they involve the the same robot hardware. Lifelong learning scenarios open the opportunity for the transfer of knowledge across tasks. Complex tasks, which might require huge amounts of training data when faced in isolation, can conceivably
203 be achieved much faster if a robot manages to exploit previously learned knowledge. For example, a lifelong learning robot might acquire general-purpose knowledge about itself and its environment, or acquire generally useful skills that can be applied in the context of multiple tasks. Such functions, once learned, can be applied to speed up learning in new tasks. Hence, a lifelong learning perspective, as proposed in this paper, promises to reduce the number of training examples required for successful learning, and hence to scale machine learning technology to more complex robot scenarios. In this paper we will present one particular candidate approach to the lifelong learning problem: The explanation-based neural network learning algorithm (EBNN) [13,26,27]. EBNN uses a hybrid learning strategy to generalize training data. On the one hand, it allows to learn functions inductively from scratch, just like neural network Backpropagation [17]. On the other hand, EBNN also allows to learn task-independent domain knowledge, which applies to multiple tasks. More specifically, in the robot control learning scenarios studied in this paper, domain knowledge is represented by neural network action models, which model the effect of the robot's actions. These models are independent of the particular control learning task at hand, and they are acquired over the lifetime of the robot. When learning control, they are used to analyze observations, in order to generalize better to unseen situations. Since in this paper we are interested in learning robot control, we will describe EBNN in the context of Q-Learning [28]. Q-Learning is a popular method for learning to select actions from delayed and sparse reward. To illustrate the appropriateness of the EBNN learning mechanism for robotics problems, we will describe experimental results obtained in an indoor mobile robot navigation domain. 2. L E A R N I N G
CONTROL
Q-Learning [28] is a reinforcement learning technique that has frequently been applied to robot control. The goal of Q-Learning is to learn a policy for generating whole action sequences, which maximizes an externally given reward function. In general, the reward may be delayed and/or sparse, adding a significant degree of complexity to the learning problem. For example, in the robot navigation experiments reported here, reward is only received upon reaching a particular goal position, or upon total failure (i.e., loosing a target object out of sight, as described in Sect. 4 ). The goal of learning, thus, is to construct a reactive controller that carries the robot to its goal position in as few steps as possible. Q-Learning works as follows. Let 5' be the set of all possible robot percepts. For simplicity, let us assume that at any (discrete) point in time, the agent can observe the complete state of the world 1, denoted by sES. Based on this observation, the learner then picks an action a (from a set of actions A). As a result, the world state changes. In addition, the learner receives a scalar reward value, denoted by r(s, a) (or rt, if we refer to the expected reward at a certain time t), which measures its current performance. Throughout this paper we will assume that reward will exclusively be received upon reaching a designated goal location, or upon total failure, respectively. 1This restrictive assumption is frequently referred to as the Markov assumption. See [2,7,10,22] for approaches to reinforcement learning in partially observable worlds.
204
Q
(goal/failure)
valuenetworkQ~I N~
valuenetw~
N~
valuenetw~
N~
Figure 1. Episode and Q-networks in Q-Learning.
Formally, in Q-Learning one seeks to find an action policy 7r:s ~a, i.e., a mapping from environment states s to actions a, which, when applied for action selection, maximize the
cumulative discounted future reward oo
R
=
(1) t'-tO
Here 7 (with 0 < 7 5 1 ) is a discount factor that favors rewards reaped sooner in time. Notice that actions may impact reward many time steps later. Hence, in order to learn to pick reward-maximizing actions, one has to solve a temporal credit assignment problem. The key of Q-Learning is to learn a value function for picking actions. A value function, denoted by Q:S• ~ , maps robot percepts sqS and actions aCA to scalar "utility" values. In the ideal case, Q(s, a) is, after learning, the maximum cumulative, discounted reward one can expect upon executing action a in state s (cf. Eq. (1)). Hence, Q ranks actions according to their reward: The larger the expected cumulative reward for picking action a at state s, the larger its value Q(s, a). Q, once learned, allows to generate optimal actions by greedily picking the action which maximizes Q for the current state s: 7r(s)
=
argmax Q(s, ~t) aeA
Initially, all values Q(s, a) are set to zero. During learning values are refined incrementally, using the following recursive update procedure. Suppose the robot just executed a whole action sequence which, starting at some initial state Sl, led to a final reward r(sT, aT). Such an episode is shown in Fig. 1. For all time steps t within this episode, Q(st, at) is then updated through mixture of the values of subsequent observation-action pairs, up to the final value at the end of the episode. The exact update equation, combined with a modified temporal differencing rule [24], is: +100 if at final action, robot reached goal - 1 0 0 if at final action, robot failed
~target(st, at)
(2)
7" [(l-A). max Q(St+l,a) a action l
+)~. ~target(st+l,at+l) ]
otherwise
205 actionmodelMal
actionmodelMa2
(goal/failure)
valuenetworkQal ~
valuenetworkQa2 ~
valuenetworkQa3
Figure 2. E B N N . General-purpose action models are used to derive slope information for training examples.
Here A (0_A
NEURAL NETWORK
LEARNING
Q-Learning learns individual policies independently, ignoring the opportunity for the transfer of knowledge across different tasks. An important aspect of successful approaches to the lifelong learning problem is the ability to transfer knowledge to related tasks. In this section we will describe the explanation-based neural network learning algorithm (EBNN), which uses task-independent knowledge to bias generalization when learning robot control. Notice that the EBNN algorithm is a general learning algorithm, albeit the fact that EBNN is described in the context of Q-Learning. A more detailed description can be found in [26,27]. In order to transfer knowledge, EBNN learns general-purpose predictive action models. Action models, denoted by M:S• ~A, model the effect of the robots actions. In EBNN, these models are represented using artificial neural networks, which are trained with observed state transitions using the Backpropagation algorithm [17] (or EBNN itself). Once trained, they are used to analyze observed episodes using the following three-step procedure: 1. Explain. An explanation is a post-facto prediction of an observed episode. More specifically, when explaining an episode, action models are employed to predict each state st from the previously observed state action tuple (st-l, at-l), as illustrated in
206
Figure 3. F i t t i n g values and slopes in E B N N : Let f be the target function for which three examples (zl,f(zl)), (z2, f(z2)), and (z3, f(za)) are known. Based on these points the learner might generate the hypothesis g. If the slopes (tangents) are also known, the learner can do much better: h.
Fig. 2. 2. Analyze. The explanation is now analyzed in order to extract the slopes of the target Q function. More specifically, consider the update rule (2). The slope of the target output, Qtarget(st, at), with respect to its input st is governed by: 0r(M (s, at)) ]
~8
V~,Qt~s~t(st, at) =
7" [(l-A)
[st
if at final action
oe(s,at+l)los ~,+, +
~.Vst.F1 ~target(st+x,at+x)] " OM(s, at)
(3)
st otherwise
If both Q and M are correct, Eq. (3) can be viewed as the derivative of Eq. (2) with respect to s with fixed actions. The auxiliary derivatives OM(s,a,) Os and OQ(s,a,§ are obtained by computing the first derivative of the neural network funcOs tion which, since neural networks are differentiable functions, is straightforward to obtain. Hence, for each target value of the form ((st, at), Qt~sct(st, at)) obtained via Eq. (3), EBNN finds the slope of the target function Q at (st, at), denoted by ~Tst~target(3t, at). This slope vector informs the learner about the sensitivity of Q with respect to infinitesimal changes in st. Clearly, action models M play an integral part in the extraction of slopes. 3. Learn. Once the slopes of the target function are known, one can generalize better by incorporating them into the training procedure, as depicted in Fig. 3. In EBNN both the target values Qtarget(st, at) and the target slopes VstQtarget(,.qt, at) are used to refine the weights of the target networks. Hence, a combined error function E
=
~ E~a~+,l.E~op~ patterns
207 is minimized, which takes both value error, Evmue, and slope error, Eslope, weighted by a gain parameter r/into account. Target values are fitted using the Backpropagation algorithm [17]. Target slopes are fitted using the Tangent-Prop algorithm, which is an analogue of Backpropagation for fitting slopes [20]. Clearly, slopes extracted by EBNN can be wrong. This is because they are computed using artificial neural networks, which themselves are constructed from training examples. Consequently, target slopes can mislead the generalization. EBNN provides a simple but effective mechanism to recover from malicious slopes. Whenever a model is used for predicting the next state, EBNN measures its root mean square prediction error, denoted by St, which is obtained by comparing the real state with the model prediction. This error indicates the correctness of the action model, and hence the accuracy of the slopes. Consequently, it is used to weight the slope information when refining the weights of the target network. More specifically, the weighting factor of the tth slope when refining the weights, denoted by r/t, is governed by T/t =
I-[
1-
Here 5t is the RMS error normalization factor such network, the learning rate which is called LOB*, has learning from weak action
of the model M at (st, at), and 5m~ is an appropriate that O<_St<_5~x. When refining the weights of the Qfor the tth slope is multiplied with r/t. This mechanism, empirically been found to be important for successfully models [26].
Notice that EBNN employs a neural network version of explanation-based learning [4,12]. To summarize, EBNN employs action models to analyze training episodes, and to derive slopes that are used for generalizing these episodes. These slopes generalize training instances in input space, since they indicate how small changes will affect the target function, Q. They are extracted from general-purpose action models, which is acquired and used over the entire lifetime of the robot. Hence, in a lifelong learning context, EBNN transfers knowledge by virtue of its action models. 4. R E S U L T S I N A R O B O T N A V I G A T I O N
DOMAIN
In this section we will present empirical results obtained in a mobile robot navigation domain. Xavier 2, the robot at hand, is shown in Fig. 4. It is equipped with a ring of 24 sonar sensors, a laser light-stripper (range finder), and a color camera mounted on a pan/tilt unit. Sonar sensors return approximate echo distances, along with noise. The laser light-stripper measures distances more accurately, but its perceptual field is limited to a small range in front of the robot. The task of the robot was to learn to navigate to a specifically marked target location in a natural laboratory environment. In some experiments, the location of the marker was fixed throughout the course of learning, in others it was moved across the laboratory and only kept fixed for the duration of a single training episode. Sometimes parts of the environment were blocked by obstacles. The marker was 2Xavier has been built by and is property of Carnegie Mellon University, Pittsburgh, USA.
208
Figure 4. The CMU Xavier robot.
detected using a visual object tracking routine that recognized and tracked the marker in real-time, making heavily use of the robot's pan/tilt unit. Every 3 seconds the robot could chose one of seven possible actions, ranging from sharp turns to straight motion. In order to avoid collisions, the robot employed a pre-coded obstacle avoidance routine. Whenever the projected path of the robot was blocked by an obstacle, the robot decelerated and, if necessary, changed its motion direction (regardless of the commanded action). Xavier was operated continuously in real-time. Each learning episode corresponded to a sequence of actions which started at a random initial position and terminated either when the robot lost sight of the target object, for which it was penalized, or when it halted in front of the marker, in which case it was rewarded. Penalty/reward was only given at the end of an episode, and Q-Learning with EBNN was employed to determine optimal action policies. In our implementation, percepts were mapped into a 46-dimensional perceptual space, comprising 24 logarithmically scaled sonar distance measurements, 10 locally averaged laser distance scans, and an array of 12 camera values that indicated the horizontal angle of the marker position relative to the robot. Hence, each action model mapped 46 sensor values to 46 sensor predictions (15 hidden units), plus a prediction for the immediate reward. The models were learned beforehand using the Backpropagation training procedure, employing a cross-validation scheme to prevent over-fitting the data. Initially, we used a training corpus of approximately 800 randomly generated training examples, which was gradually increased through the course of this research to 3,000 examples, taken from some 700 episodes. These training examples were distributed roughly equally among the
209 seven action model networks. In all our experiments the action models were trained first, prior to learning Q, and frozen during learning control. When training the Q networks (46 input, eight hidden and one output unit), we explicitly memorized all training data, and used a replay technique similar to "experience replay" described in [7]. This procedure memorizes all past experiences explicitly. After each learning episode, it re-estimates the target values of the Q function by recursively replaying past episodes, as if they had just been observed. Experience replay makes extensive use of the training instances, hence allowing for more accurate evaluations of the minimum requirements on data volume. In all our experiments the update parameter 3, was set to 0.9, and ~ was set to 0.7, which was empirically found to work best. We performed a total of seven complete learning experiments, each consisting of 25-40 episodes. In order to evaluate the importance of the action models for rapid learning in a lifelong learning setting, we ran two sets of experiments: one in which no prior knowledge was available, and one where Xavier had access to the pre-trained action models. One of the experiments is summarized in Fig. 6. In this figure, we visualized several learning episodes viewed from a bird eye's view, using a sonar map techniques described in [27]. In all cases Xavier learned to navigate to a static target location in less than 19 episodes (with action models) and 24 episodes (without action models). Each episode was between two and eleven actions in length. It consistently learned to navigate to arbitrary target locations (which was required in five out of seven experiments) always in less than 25 (35, respectively) episodes. The reader should notice the small number of training examples required to learn this task. Although the robot faced a high-dimensional sensation space, it always managed to learn the task in less than 10 minutes of robot operation time, and, on average, less than 20 training examples per Q-network. Of course, the training time does not include the time for collecting the training data of the action models. Almost all training examples for the action models, however, were obtained as a side effect when experimenting with the robot. When testing our approach, we also confronted Xavier with situations which were not part of its training experience, as shown in Fig. 7. In one case, we kept the location of the marker fixed and moved it only in the testing phase. In a second experiment, we blocked the robot's path by large obstacles, even though it had not experienced obstacles during training. It was here that the presence of appropriate action models was most important. While without prior knowledge the robot consistently failed to approach the marker under these new conditions, it reliably (>90%) managed this task when it was trained with the help of the action model networks. Obviously, this is because in EBNN the action models provides a knowledgeable bias for generalization to unseen situations.
210
Figure 5. Prediction and slopes. A neural network action model predicts sensations and penalty/reward for the next time step. The large matrix displays the output-input slopes of the network. White boxes refer to positive and black boxes to negative values. Box sizes indicate absolute magnitudes. Notice the bulk of positive gradients along the main diagonal, and the cross-dependencies between different sensors.
211
Figure 6. L e a r n i n g navigation. Traces of three early and three late episodes are shown. Each diagram shows a two-dimensional occupancy maps of the world, which have been constructed based on sonar information. Bright regions indicate free-space and dark regions indicate the presence of obstacles. Note that the location of the target object (marked by a cross) is held constant in this experiment.
Figure 7. Testing navigation. After training, the location of the target object was moved. In some experiments, the path of the robot was also blocked by obstacles. Unlike plain inductive neural network learning, EBNN almost always manages these cases successfully.
212 5. C O N C L U S I O N In this paper we have described an approach to the lifelong robot learning problem, based on Q-Learning and EBNN. We have empirically illustrated rapid learning capabilities in a mobile robot navigation domain, demonstrating the appropriateness of QLearning and EBNN for robot navigation with unstructured, high-dimensional perceptual spaces. The reader should notice that others have studies the transfer of knowledge. For example, in the context of reinforcement learning, models have been successfully employed for the transfer of knowledge via planning [9,25] or replay [7]. Others proposed hierarchical approaches, in which the building blocks, once learned, can be applied to multiple tasks [3,7,21]. A third way for the transfer of knowledge is concerned with the construction of better internal representations, which improve generalization across multiple tasks [1,16,19,23]. While this list is clearly incomplete, it nevertheless illustrates the importance for transferring knowledge for scaling machine learning to more complex domains [27]. From a lifelong learning perspective, much of the work presented in this paper is preliminary. While we have not yet studied robot control in the context of multiple tasks in practice, in experiments described here and elsewhere [13,26,27] we consistently found that EBNN outperforms pure inductive neural network learning, which does not employ background knowledge and hence learns from scratch. In a related paper we have illustrated superior generalization due to EBNN in a robot perception task [14]. Learning mechanisms that allows for the effective knowledge transfer, like EBNN, is a necessary prerequisite for successful approaches to the lifelong robot learning problem. ACKNOWLEDGMENT The author thanks Tom Mitchell, Joseph O'Sullivan and the CMU robot learning group for their invaluable feedback. He also thanks CMU for granting him access to the Xavier robot, without which this research would not have been possible. This research is sponsored in part by the National Science Foundation under award IRI9313367 to Tom Mitchell. Views and conclusions contained in this document are those of the author and should not be interpreted as necessarily representing official policies or endorsements, either expressed or implied, of NSF. REFERENCES
1. Richard Caruana. Multitask learning: A knowledge-based of source of inductive bias. In Paul E. Utgoff, editor, Proceedings of the Tenth International Conference on Machine Learning, pages 41-48, San Mateo, CA, 1993. Morgan Kaufmann. 2. Lonnie Chrisman. Reinforcement learning with perceptual aliasing: The perceptual distinction approach. In Proceedings of 1992 AAAI Conference, Menlo Park, CA, July 1992. AAAI Press / The MIT Press. 3. Peter Dayan and Geoffrey E. Hinton. Feudal reinforcement learning. In J. E. Moody, S. J. Hanson, and R. P. Lippmann, editors, Advances in Neural Information Processing Systems 5, San Mateo, CA, 1993. Morgan Kaufmann.
213 4. Gerald DeJong and Raymond Mooney. Explanation-based learning: An alternative view. Machine Learning, 1(2):145-176, 1986. 5. Vijaykumar Gullapalli, Judy A. Franklin, and Hamid Benbrahim. Acquiring robot skills via reinforcement learning. IEEE Control Systems, 272(1708):13-24, February 1994. Tommi Jaakkola, Michael I. Jordan, and Satinder P. Singh. On the convergence of stochastic iterative dynamic programming algorithms. Technical Report 9307, Department of Brain and Cognitive Sciences, Massachusetts Institut of Technology, July 1993. Long-Ji Lin. Self-supervised Learning by Reinforcement and Artificial Neural Networks. PhD thesis, Carnegie Mellon University, School of Computer Science, Pittsburgh, PA, 1992. Pattie Maes and Rodney A. Brooks. Learning to coordinate behaviors. In Proceedings Eighth National Conference on Artificial Intelligence, pages 796-802, Cambridge, MA, 1990. AAAI, The MIT Press. Sridhar Mahadevan. Enhancing transfer in reinforcement learning by building stochastic models of robot actions. In Proceedings of the Seventh International Conference on Machine Learning, pages 290-299. Morgan Kaufmann, 1992. 10. R. Andrew McCallum. Overcoming incomplete perception with utile distinction memory. In Paul E. Utgoff, editor, Proceedings of the Tenth International Conference on Machine Learning, pages 190-196, San Mateo, CA, 1993. Morgan Kaufmann. 11. Tom M. Mitchell. Becoming increasingly reactive. In Proceedings of 1990 AAAI Conference, Menlo Park, CA, August 1990. AAAI, AAAI Press / The MIT Press. 12. Tom M. Mitchell, Rich Keller, and Smadar Kedar-Cabelli. Explanation-based generalization: A unifying view. Machine Learning, 1(1):47-80, 1986. 13. Tom M. Mitchell and Sebastian Thrun. Explanation-based neural network learning for robot control. In S. J. Hanson, J. Cowan, and C. L. Giles, editors, Advances in Neural Information Processing Systems 5, pages 287-294, San Mateo, CA, 1993. Morgan Kaufmann. 14. Joseph O'Sullivan, Tom M. Mitchell, and Sebastian Thrun. Explanation-based neural network learning from mobile robot perception. In Katsushi Ikeuchi and Manuela Veloso, editors, Symbolic Visual Learning. Oxford University Press, to appear. 15. Dean A. Pomerleau. ALVINN: an autonomous land vehicle in a neural network. Technical Report CMU-CS-89-107, Computer Science Dept. Carnegie Mellon University, Pittsburgh PA, 1989. 16. Lori Y. Pratt. Discriminability-based transfer between neural networks. In J. E. Moody, S. J. Hanson, and R. P. Lippmann, editors, Advances in Neural Information Processing Systems 5, San Mateo, CA, 1993. Morgan Kaufmann. 17. David E. Rumelhart, Geoffrey E. Hinton, and Ronald J. Williams. Learning internal representations by error propagation. In D. E. Rumelhart and J. L. McClelland, editors, Parallel Distributed Processing. Vol. I + II. MIT Press, 1986.
214 18. Jacob T. Schwartz, Micha Scharir, and John Hopcroft. Planning, Geometry and Complexity of Robot Motion. Ablex Publishing Corporation, Norwood, N J, 1987. 19. Noel E. Sharkey and Amanda J.C. Sharkey. Adaptive generalization and the transfer of knowledge. In Proceedings of the Second Irish Neural Networks Conference, Belfast, 1992. 20. Patrice Simard, Bernard Victorri, Yann LeCun, and John Denker. Tangent prop a formalism for specifying selected invariances in an adaptive network. In J. E. Moody, S. J. Hanson, and R. P. Lippmann, editors, Advances in Neural Information Processing Systems ~, pages 895-903, San Mateo, CA, 1992. Morgan Kaufmann. -
21. Satinder P. Singh. Transfer of learning by composing solutions for elemental sequential tasks. Machine Learning, 8, 1992. 22. Satinder P. Singh, Tommi Jaakkola, and Micheal I. Jordan. Learning without stateestimation in partially observable markovian decision processes. In Proceedings of the Eleventh Machine Learning Conference, 1994. Steven C. Suddarth and Y. L. Kergosien. Rule-injection hints as a means of improving 23. network performance and learning time. In Proceedings of the EURASIP Workshop on Neural Networks, Sesimbra, Portugal, Feb 1990. EURASIP. 24. Richard S. Sutton. Learning to predict by the methods of temporal differences. Machine Learning, 3, 1988. 25. Richard S. Sutton. Integrated architectures for learning, planning, and reacting based on approximating dynamic programming. In Proceedings of the Seventh International Conference on Machine Learning, June 1990, pages 216-224, San Mateo, CA, 1990. Morgan Kaufmann. 26. Sebastian Thrun and Tom M. Mitchell. Integrating inductive neural network learning and explanation-based learning. In Proceedings of IJCAI-93, Chamberry, France, July 1993. IJCAI, Inc. 27. Sebastian Thrun and Tom M. Mitchell. Lifelong robot learning. Robotics and Autonomous Systems, to appear. Also appeared as Technical Report IAI-TR-93-7, University of Bonn, Dept. of Computer Science III, 1993. 28. Christopher J. C. H. Watkins. Learning from Delayed Rewards. PhD thesis, King's College, Cambridge, England, 1989.
Intelligent Robots and Systems V. Graefe (Editor) 9 1995 Elsevier Science B.V. All rights reserved.
215
A Multilevel Learning Approach to Mobile Robot Path Planning F. Wallner, M. Kaiser, H. Friedrich, and R. Dillmann Institute for Real-Time Computer Systems and Robotics Prof. Dr.-Ing. U. Rembold and Prof. Dr.-Ing. R. Dillmann Department of Computer Science University of Karlsruhe, D-76128 Karlsruhe, Germany E-Mail: wallner ,kaiser,friedric, dillm~nn@ira, uka. do
The problem of adapting mobile robot navigation to changes in the environment is usually approached by modifying an internal world model. Descriptions on different levels of abstraction provide the information necessary for navigation and therefore influence the robot's behaviour. The effect of such indirect adaptation is limited. The approach presented in this paper describes a new technique for direct integration of navigation experience in path planning. Thus, not only the world knowledge, but also the planning behaviour is improved over time. Experiments are carried out on a robot which is controlled by a layered architecture. It is integrated in a multi-robot control environment which is described. The focus of the article is towards improving the higher navigation levels. The main idea being presented is the realization of adaptive behaviour not only on the level of reflexes, but also with respect to the planning capabilities of the robot. The application of learning techniques allows to continuously improve the estimation of plan costs and therefore the inherent strategy of the topological planner. It is illustrated that a combined learning of world description and navigation allows fast and sophisticated reaction to new environmental conditions. 1. I n t r o d u c t i o n In most applications, a mobile robot operates in a world which is not exactly known and changing over time. A robot which is not able to adapt itself to a dynamic environment will operate inefficiently or even fail after a while. Two approaches are investigated how to modify the robot's behaviour when the environment has changed. The first Call be called indirect adaptation. Changes in the environment, such as newly detected obstacles, are perceived via the robot's sensors and integrated in a world model [1]. As planning is based on world knowledge, the robot will behave according to the new situation. However, such indirect adaptation offers no possibility to integrate navigation experience made by the robot when driving along a route. A second way of adaptive behaviour can usually be found at the level of reflexes. Especially learning of collision avoidance strategies has already been investigated by different researchers such as [4]. In addition, adaptation on a higher level is an appealing field of research. Besides its dependency on knowledge about obstacles, the feasibility of a path
216 depends on additional features that are difficult to extract from a world model, such as the time since a route has been last visited, the number of successful (or less successful) visits, the obstacle occurrence and externally given constraints like maximum throughput or m a x i m u m safety.
Figure 1. The mobile system PRIAMOS
This paper describes a learning mobile robot system that improves high level path planning through navigation experience. In combination with sensor based map improvement, this high-level adaptation enables the robot to operate efficiently even in changing environments. Experiments are carried out on the mobile robot PRIAMOS 1. PRIAMOS is a mobile system with three degrees of freedom, i.e. motion in longitudinal and transverse directions and rotation around the centre of the vehicle. A set of ultrasonic range sensors is mounted on a circle around the centre of the vehicle. As a highly sophisticated sensor, an active stereo-vision head is also available. There are two onboard computer systems, a real-time image processing system which provides a vector description of stereo-images at videorate and a multi-processor system for the control of the vehicle. The robot is connected to a stationary control computer via a radio link. 2. T h e control a r c h i t e c t u r e of P R I A M O S For experiments in the field of mobile robotics, the control environment MARS 2 has been developed. The structure of this environment is shown in figure 2. MARS has a strictly modular concept. Higher level modules are running as independent UNIX processes and are connected via sockets. Lower level modules with stronger real1PRI = Permutation of the name of our Institute, AMOS = Autonomous MObile System 2Mobile Autonomous Robot control System
217
MMI We rl"d Specification
Missionspecification
J
Control and Surveillance
Vehicle control ..... a r c h i t e c t u r e ,,
11 II
! i .........
b Sensor/ emulator
t5 Robot/ emulator
Figure 2. The control environment MARS
time constraints are executed on the robot's internal OS/9 based computer systems. Central components are a man-machine-interface for monitoring and control and a database. The number of robots the operator wants to control can freely be specified. Due to the modular concept, a separate controller for each robot is initiahzed. For all physical components, a software emulation is available which allows off-line program development and testing in simulation. Local data structures that are updated automatically only when changes occur reduce the load on the database. The control architecture of each single robot is structured horizontally and vertically. The vertical structure consists of different layers of abstraction. Each layer contains a navigation and a perception module that are connected through a world model. This world model is constructed by the perception module and integrates sensor information and (on higher levels) information stored in the database. Figure 3 shows the levels of abstraction of the world representation. On the highest level, the world model has a topological structure. Path segments are described as edges E and intersections as nodes N in a graph. This allows simple path planning over longer distances without the need to take geometric details into account. In the vicinity of the robot, obstacle geometry is needed for precise path planning. Therefore, a local dynamic model of the perceivable environment which also integrates recent sensor information is used for geometric path planning. A problem of layered control architectures is the difficulty to react quickly to obstacles that are not known a-priori, but newly detected. The information used for planning on higher levels must be filtered and integrated over time due to sensor noise. During the time between an observation, the appearance of the corresponding object in the world
218
Figure 3. World model hierarchy
model, and the planning of a new path, the robot would have to stop its motion to insure collision-freeness. However, as path modifications are often of a local nature, a simple collision avoidance algorithm using recent sensor data can increase the average operating speed of the robot. On this reflexive level, the world knowledge consists of raw sensor data only. In some obstacle configurations, local reactions are wrong. A path that seems to be the best can be blocked by some obstacle that has not yet been observed. A similar problem can occur if an obstacle exists in the geometric map but not in the topological one. The topological planner will produce a path which the geometrical one can not follow. Section 3.3 shows the integration of both layers and how to deal with these problems. 3. T h e h i e r a r c h i c a l p l a n n i n g a p p r o a c h Based on the world model, on each level a navigation module decomposes the task commanded by the next higher level into a set of more simple ones, such that a location the robot should reach is finally transformed into a set of incremental position commands that are passed to the robot motion controller. The reflexive level provides the basic motion capabilities that are used by all higher levels, such as the topological and the geometrical planning level described below. A collision avoidance strategy modifies a given path on the base of recent sensor data [6]. Additionally, a set of reflexive behaviors exists that are employed by higher level planners or directly by an operator in situations requiring the robot to move through a door, to enter a narrow passage, or to dock at a recharging or loading-station.
3.1. The topological p l a n n e r The idea of introducing topological graphs (see figure 5) as a more abstract way to represent free spaces and existing passages in the environment is very appealing (see also [3]). The graph representation allows easy access to the existing information, and
219 algorithms known from graph theory are directly applicable. However, an efficient use of topological maps in a real world requires that the following aspects are taken into account: The topological map itself is subject to change due to changes in the environment. It must be updated as soon as changes in the geometrical map occur. As soon as the robot's current position and its goal are not directly located on the topological map, selecting good or even optimal points to enter and leave the graph is not straightforward. In a dynamic environment it is possible that two motions, starting at the same point and aiming at the same goal, require the use of different topological paths. The cost of moving along a certain edge of the topological map cannot completely be calculated a-priori, since it does not only depend on static features of the environment (such as the length of the corresponding path), but also on the "difficulty" to traverse the edge (i.e., on the complexity of the required reflexive behaviour), and on global constraints imposed on the whole transport subsystem. The basic strategy of topological path planning is the following: temporary edges from the start position to the entry node and from the exit node to the goal position are generated, the two distance are obtained by a geometrical planning step. A branch and bound search algorithm is used to find a sequence of edges satisfying a given set of criteria (e.g. length of a path) with the least overall costs. Section 4.1 shows how to find good points to enter respectively leave the graph, given arbitrary start and goal positions. During these considerations, the topological map will be assumed to be static. This is a reasonable assumption since changes in the topological map that affect the current plan will force the robot to perform a new planning step, which will possibly result in a completely new path and can therefore be regarded as a completely new mission. However, temporal changes such as a passage blocked by a human, a vehicle, or a large obstacle, are taken into account. They are handled on the level of edge cost function estimation (see section 4.2). 3.2. T h e g e o m e t r i c a l p l a n n e r Geometrical planning is restricted to a local area which is slightly larger than the perception range of the robot's sensors, as planning is performed in the local geometric model. The central part of the geometric model is obtained by a fusion process of actual sensor data and a-priori knowledge. Planning in areas further away is mainly a static procedure and can be performed on layers of higher abstraction. However, due to the local nature of the geometrical model, the next graph node might not be located within the area of geometric planning. In such cases, special care has to be taken in order to plan a path which finally leads to the correct topological node. Path planning is based on the idea of potential distance fields [6]. The algorithm described below guarantees to find the shortest collision free path. Its main advantage is its computational efficiency. The holonomic driving principle of PRIAMOS offers the advantage that no additional constraints have to be taken into account. Planning is done in five steps.
220
9
If a topological path node is located outside the local model, a goal at the border of the local model in the direction of the next topological path node is computed.
9 Obstacles are enlarged by half of the robot length so that further planning can consider the robot as a point. This approximation can not be applied in narrow passages (e.g. doorways). Here the reflexive collision avoidance is replaced by appropriate reflexive control strategies. 9 A square grid that covers the planning region and which is centered around the current position of the robot is generated. Each cell of the grid contains binary information indicating if this cell is free or not. 9 The next step is path planning itself. The cell of the goal position is initialized with a start-value ('1'). The 4 neighbours of a cell are recursively (beginning with the goal-cell) given a value: celLvalue + I if they are free. This procedure is repeated until the cell which covers the robot position is reached. Starting there, a path is found by successively moving to the lowest valued neighbour cell (8 neighbours are considered in this step). 9 Finally, a polygon is defined by the list of cells. corner-points of this polygon.
A path is represented by the
The complete obstacle transformation and planning procedure is illustrated in figure 4.
Figure 4. A geometric planning example. Shown is a ground plan of an office environment as well as the result of a planning cycle. (#: obstacles; $$: robot position; filled squares indicate the path)
The geometric planning module provides two different modes of operation. In MOTION mode, a trajectory towards a given goal is planned within the extension of the current local model. In NO_MOTION mode, two positions are specified. In this mode, the planner obtains a geometric description of a specified area from the database and executes a planning step in this static model. The NO_MOTION mode is necessary to support the
221 topological planner in finding the optimal nodes for entering and leaving the topological graph by planning paths from the vehicle's start and goal position to the preselected entry and exit nodes. In case of successful planning the length of the generated path is returned to the topological planner.
3.3. Integration of navigation layers Even if the hierarchical organization of the planning and navigation process is necessary to handle the complexity involved in these tasks, it also causes a number of probles. In some obstacle configurations, local reactions are wrong. A path that seems to be the best can be blocked by some obstacle that has not yet been observed. A similar problem can occur if an obstacle exists in the geometrical, but not in the topological model. The higher level planner will produce a path which cannot be followed by the lower levels. To overcome such inconsistencies, higher level planners have to be continuously re-activated. Such replanning starts at the current robot position and can therefore be used to verify if the current path is still the best. The local nature of the geometrical planner implies constraints on the nature of the topological graph. If the distance between topological nodes is bigger than the area covered by the local model and obstacles exist in between, geometrical planning can fail in planning a path between these segments. The flow of information between the different layers is strictly hierarchical. A higher level generates a path and sends it to the next lower planning layer. It is possible that a high level planner or the task decomposition module itself assigns a low level reflexive behaviour to a path segment. In such cases layers which are located inbetween are not activated. The path execution is controlled by higher levels which are activated automatically. The only information which is returned by lower levels is the state of the execution, i.e., if a certain task could be completed or if it failed (e.g., because no path to the target position existed). 4. Multilevel adaptive b e h a v i o u r The apphcation of mobile robots in future factories, and the subsequent definition of more complex evaluation criteria regarding the robot's behaviour (such as maximum throughput of the whole transport subsystem) require adaptive behaviour also on higher cognitive (planning) layers. In this section, adaptive behaviour on the two layers of topological and geometrical planning is described. 4.1. E x p e r i e n c e - b a s e d selection of t o p o l o g i c a l n o d e s The efficient application of topological maps in a real world requires to cope with the problems of a changing environment and arbitrary start and goal positions. While the next section describes a way to overcome the former, a possible solution to the problem of selecting good entry and exit points into and out of the topological graph is given now. The task to be solved is to determine good or even optimal entry and exit points into respectively out of the topological graph. This task is difficult to solve because of several problems (see figure 5): 9 The edge and node closest to the start and end point do not necessarily represent optimal or even a good entry and exit points.
222
.................................................. : .....- i
,yg)
o .
!i 9
9
-
-!r.......... ......... ! ~ i
..................
...
i
.~.~
9 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
r . . . . . . . . . . .
t,
x(~,y 0
Figure 5. Start point (Zo, y,), goal position (zg, yg) in the topological map T of PRIAMOS' operating environment.
9 Obstacles might exist between the start/exit point and the theoretically optimal entry/exit nodes/edges. If --*G denotes a geometrically planned path, and --~T a topologically planned one, the
problem can be stated as follows: Given: The position (z,,V,) of the start and (zg,Vg) of the goal point, the topological map T = (N, E) and the geometrical map G. D e t e r m i n e : The entry node N,n and the exit node N,z such that the path P = {(zo,y,) ~ G .Are, ~ T N,, ~ G (zg, yg)} causes minimal costs. Initially, entry and exit nodes are selected by applying an heuristic H*. n nodes nearest to the start/end position are selected. On the base of the world model, the geometrical planner determines the costs to reach each of these nodes. Afterwards, en <_ n possible entry and ez _< n possible exit points exist. The topological planner determines the cost of the resulting en x ez paths (which might partially include each other), and the best (i.e., most cost effective) combination of path and entry/exit point is selected. Since this heuristics cannot guarantee that movement starts with a "bad" node resulting in a much more costly topological path, the generated path must be checked in a global context, using experience that has been obtained from already driven paths. The applied method is based on the idea of incrementally building a decision structure, providing a good guess of useful entry and exit points. Given the start position (z,, y,), the goal position (zg, yg), and a decision structure D, the method works as follows" 1. Apply the heuristic H* to obtain a possible path P with costs c from ( z , , y , ) to 2. If D is empty, use path P and continue with 8. 3. If D is not empty, try to classify the example B = ((x,,y,), (zg, yg)) by application of C L A S S I F Y - W l T H - D S ( D , B ) in order to obtain ( ,N, , BN ~ ) .B If B cannot be classified, use P and continue with 8.
223 4. Calculate the costs ~ of N~n -~T N~. 5. If c < ~, calculate the costs c. and % for moving from the current start and goal point to the entry and exit node N~n and N ~ , respectively. Else use P and continue with 8. 6. If c < ~ + cs + co, use P and continue with 8. 7. Generate the path P = {(zo,y.)--*a N~S,, ~ T N ~ ~ c (zg, yg)} and use it. 8. After the goal has been reached, build an example B = ((z.,yo), (zg, yg), Nen,/Vex). Let c be the actual costs for moving from (z.,y.) to (zg,yg) (and vice versa). 9. Insert B with costs c into the decision structure D using
BUILD-DS(D,B,c).
Summarizing, the selection of "good" nodes to enter and leave the topological map is based on the idea that typical paths connecting regions of the workspace can be identified.
Figure 6. Part of a decision structure D.
These regions are represented as nodes inside the decision structure. As shown in figure 6, these nodes as well as the connections between them are generated during operation, such that the decision structure D is built incrementally. The applied method is similar to incremental clustering [2]. Let S = ((x,,y,),(xg,ya)) be the example to be classified. Let z ( N ) , y ( N ) be the center of the cluster represented by a node N of D and d(N) be the radius of this cluster. Then, classification using D works as follows: C L A S S I F Y - W I T H - D S ( D , B): 1. Find the set
SN. of nodes of D such that + V/(y(N)- y,)' < d(N).
a= 2. If SN, is empty, return failure.
3. Let Nm be the node with the minimum d.
SNg of nodes connected to Nm such that d = V/(x(N)- zg)2+ V/(Y(N) ya) 2 < d(N)for N C Sgg.
4. Find the set
224 5. If SN, is empty, let SN~ = SN~ - N~. Goto 2. 6. Find the node N a E SN, such that the newly calculated costs c of the connection between N,,, and Ng are minimal. 7. Return N ~ := Nm, N B := Ng. After a path has actually been used by the robot, its costs are exactly known to the system. Therefore, the combined information (path, costs) is used to incrementally built the decision structure. During the generation of the structure, the geometrical distance between two positions serves as a lower-bound estimate of the costs to move from one position to the other. With B = (((z,, y,), (%, ya), N~, N~) and c being the real costs of the path from (zo,y,) to (za,yg), the building of D works as follows:
BUILD-DS(D,B,c): 1. Find the set SN. of nodes of D such that
dl = V / ( z ( N ) - z,) 2 + V / ( Y ( N ) - y,)2 < d(N). 2. If SNo is empty, goto 12. 3. Let N1 be the node with the minimum di. 4. Find the set Slvg of nodes connected to Nx such that d2 -
r
2+
V / ( y ( Y ) - ya) 2 < d(N) for Y e SNo. 5. If SNg is empty, let /V = N1 and goto 13. 6. Let N2 be the node with the minimum d2. 7. Let ~ be the newly calculated costs for the path from N1 to N2, N~,, be the entry node and/Vex be the exit node associated to that path. 8. I f ~ < d x + d 2 + c , (a) if dx > d2 expand the region represented by N2 with d(N2) = d(N2) + d2 and continue with 12. (b) else expand the region represented by N~ with d(N~) = d(N1)+dx, let N - N1 and continue with 13. 9. Else use the geometrical planner to calculate the costs cl to move from (zs,ys) to Ne,, and the costs c2 to move from Ne= to (zg,yg). 10. If ~ > ci + c2 + c, move and expand the region represented by NI: Let ~ = c + cl + c2, N.,, = N:B., N.: - N~B: x ( N 1 ) - :(N,)+:. y ( N ~ ) - u(N,)+u, z ( N 2 ) - :(N:)+:, '
2
'
2
'
2
'
y(N2) = ~(N~)+y,, d(N~) - d(Nx) + dl and d(N2) - d(N2) + d2. 11. Else (a) if Cl > c2 expand the region represented by N2 with d(N2)= d(N2)+ d2 and continue with 12. (b) else expand the region represented by N1 with d(N1) = d(N~)+ d~, let/V = Nx and continue with 13.
225 12. Insert a new node representing the current start region Add a node N1 to D.
(b)
Let (z(N1),y(N1)) = (zo,yo) be the center of the cluster represented by the node and d(N1) = dmi, be its width.
(c)
Find the set SNo of nodes of D such that d = y/(ZiN) ' Zg)~+ V/(Y(N) - yg)2 < d(N) for N e Sn,. (a) If SNg is empty, let N = N1 and continue with 13. (e) Let N2 6 Sn, be the node with the minimum d.
(f) (g)
Initiahze the connection between N1 and N2 with the data obtained from B. exit BUILD-DS.
13. With N representing the corresponding start region, insert a new node representing the current goal region: (a) Add a node N2 to D. _
(b) Initialize the connection between N and N2 with the data obtained from B. Summarized, the method works by building a structure representing (possibly overlapping) regions of different size and virtual connections, estabhshed by already driven paths between them, thus improving the selection of good entry/exit points over time. The method features a minimum overhead regarding the apphcation of the geometrical planner, since geometrical planning is during learning only necessary in the case of very close start- and goal positions. It should be noted that the costs of known paths are not stored in the decision structure, because the estimation of edge costs is improved continuously (see below). 4 . 2 . I n c r e m e n t a l i m p r o v e m e n t of edge c o s t e s t i m a t i o n
Given the topological map, a start node, a goal node, and the costs of moving along the individual interconnections (edges) between each of the nodes in the map, an optimal path from the start to the goal can be found by several means. Provided that a lower bound estimation of the individual edge's costs exists, algorithms such as A* produce optimal solutions. However, they also require to keep track of the real costs that emerge and possibly to climb up the search tree if the current branch does not yield the desired solution. After the goal node has been reached, the best path between the start and the goal is known. Transferred to the given apphcation domain, this behaviour would result in the robot moving back and forth along paths. This solution is of course not satisfactory, since the aim must be to obtain a near optimal path before the robot starts moving. Therefore, the lower-bound condition for the edgecost estimation (as, for instance, given by the length of the corresponding passage) is not sufficient. In fact, the edge costs must be known as accurately as possible in order to produce a good path. Moreover, in a dynamic environment the costs of traversing an individual edge can also be changing. Therefore, the edge cost function C(e) must meet the following requirements:
226 9 The a-priori estimation of C must incorporate not only static components, but also information about changes in the environment. 9 After the real costs of traversing an edge are known, the edge's cost function must be updated. These requirements yield the following definition of the cost function C: Ca_priori(e) =
g(l,,p,) + h(hist(e)),
where 9 a monotonically increasing function of the length l, of the edge and the complexity p, of the required behaviour, hist(e) is the accumulated information about the costs to move along the edge e and h is a function mapping this complex information to a numerical value. After the robot has moved along the planned path, the real costs of the individual edges are known: Ca.posteriori(e) =
f(t(e),t,t(e)) + g(l,,pe) + f(o(e),o,t(e)),
where f is a monotonically increasing function of the fraction between the measured time required to move along an edge and the estimated time t,,t(e), and f is a monotonically increasing function of the fraction between the measured traversing difficulty o(e) and the estimated one o~,t(e). Summarizing, the following terms must therefore be determined:
t(e)
9 The parameter te,t(e), which is first given by the length of the edge and the nominal speed of the robot, but which will change according to the measured time t(e) required to traverse edge e. t(e) 9 The function f, which is realized a s Ote t,,,(e). t(,) ). 9 The function f, which is realized as oT,t,o,(, -
9 The parameter p,, which can be calculated for each edge from the complexity of the commanded movement, which might, for instance, depend on certain security requirements. 9 The parameter le, which is a static attribute of each edge and can be calculated from the geometrical model of the environment. 9 The function g which combines the length of the edge and the commanded movement to a numerical value. This function is static as are the physical characteristics of the robot and the length of the edges. 9 The information about the edge e over time hist(e) 9 The history analyzer function h. It is obvious that the information that is most difficult to handle concerns the behaviour of a particular edge in the past. While the exploration strategy described in section 4.2.2 accounts for a changing traversability of the edge (i.e., if the edge can be used at all), dealing with continuous valued, possibly periodically changing parameters (such as the density of obstacles) requires much more sophisticated strategies.
227 4.2.1. P r e d i c t i o n of edge costs Analyzing the history of an edge in terms of its costs over time aims at value of the function h. Therefore, the problem can be stated as follows:
predicting the
Given: The history of the edge in terms of the costs C ( e ) t , . . . , C(e)l~J (with le[ being the number of times the edge has been used) to traverse it. D e t e r m i n e : ] t ( C ( e ) t , . . . , C(e)lel) such that ]t(C(e)t,..., C(e)lel) ~ h(hist(e)). Given the definition of Ca_priori(e)and Ca_posteriori(e), it can be found that h(hist(e)) and therefore h should approximate H = f(t(e),t~,t(e))+ ](o(e),o+ot(e)). As H can be calculated directly after the edge has been passed, each passing of an edge yields an example from which the general shape of the function tt can possibly be derived. The task to be solved is therefore to improve the prediction of the value of it by means of learning from examples of h(hist(e)). An appropriate learning technique to deal with such problems is temporal difference learning [5]. 4.2.2. A s t r a t e g y for e x p l o r a t i o n As mentioned in section 4.1, temporal changes such as a passage blocked by a human, a vehicle, or a large workpiece, must be taken into account. In fact, allowing the robot to try again to use a passage that has been found to be blocked before, requires an exploration strategy. Principally, this strategy works by assigning a temporally valid blocked attribute to edges that could not be traversed. This attribute is only valid for a certain time, with the time depending on the experiences made with the corresponding edge, i.e., how often the edge has been found to be traversable respectively blocked. In detail, the exploration strategy works as follows: 1. Initialization (a) Assign a blocked counter to each edge. (b) Assign a blocked h i s t o r y counter to each edge. (c) Set the blocked counter to zero. (d) Set the blocked h i s t o r y counter to one. 2. During planning: (a) If the blocked counter of an edge is zero, the edge can be traversed, (b) else decrease the blocked counter by one. 3. During execution" (a) If an edge is found to be traversable, set its blocked h i s t o r y counter to mar.(-ti blocked h i s t o r y counter, 1). (b) If an edge is found to be blocked, set its blocked counter equal to the blocked h i s t o r y counter, double the blocked h i s t o r y counter. This strategy realizes a dynamic adaptation of the frequency of retries the robot performs on edges that have been found to be blocked.
228 5. Conclusion and f u r t h e r work The work described in this paper deals with an important aspect of the realization of mobile systems that are adaptive with respect to their environment. The main idea that has been followed is to realize adaptive behaviour not only on the level of reflexes, but also with respect to the planning capabilities of the robot. This approach is closely related to the hierarchical control architecture of the mobile robot PRIAMOS, which has been used for experiments. By means of continuously improving the estimation of edge costs and the evaluation criteria of the topological planner, changes in the environment are not only reflected in the explicit world model, but also in the robot's reasoning system. The main activities at the moment are concerned with the practical evaluation of the theoretically achieved results. Additionally, some more theoretical work has to be done concerning the decision structure. Especially algorithms for shrinking, deleting, splitting and merging of regions seem to be necessary. It is not clear if the different evaluation functions cover all situations that occur in a real factory. However, assuming that the robot's working environment is not too cluttered (which should be the case e.g. in an automated factory), both the planning and the adaptive components can be expected to work flawlessly. Acknowledgement This work has partially been supported by the ESPRIT Project 7274 "B-Learn II". It has been performed at the Institute for Real-Time Computer Systems & Robotics, Prof. Dr.-Ing. U. Rembold and Prof. Dr.-Ing. R. Dillmann, Department of Computer Science, University of Karlsruhe, 76128 Karlsruhe, Germany. The authors would like to thank the anonymous reviewers for their helpful comments. REFERENCES 1. A. Elfes. A sensor-based mapping and navigation system. In Proceedings of the IEEE International Conference on Robotics and Automation, 1986. 2. J . H . Gennari, P. Langley, and D. Fisher. Models of incremental concept formation. Artificial Intelligence, 40:11- 61, 1989. 3. P. Kampmann and G. Schmidt. Topological structured geometrie-database and global motion planning for the autonomous mobile robot macrobe. Robotersysteme, 5, 1989. 4. J. del R. Mills and C. Torras. Efficient reinforcement learning of navigation strategies in an autonomous robot. In Proceedings of the IEEE//RSJ Conference on Intelligent Robots and Systems, 1994. 5. R.S. Sutton. Learning to predict by methods of temporal difference. Machine Learning, 3:9 - 44, 1988. 6. F. Wallner, T. C. Lfith, and F. Langinieux. Fast local path planning for a mobile robot. In Proc. of the 2nd Int. Conf. on Automation, Robotics and Computer Vision, Singapore, 1992.
Intelligent Robots and Systems V. Graefe (Editor) 9 1995 Elsevier Science B.V. All rights reserved.
A self-organizing representation
229
of s e n s o r s p a c e for
mobile robot navigation Ben J. A. KrSse and Marc Eecen a aFaculty of Mathematics and Computer Science, University of Amsterdam Kruislaan 403, 1098 SJ Amsterdam, the Netherlands k r o s e @ f w i , uva. nl
The paper describes a sensor-based navigation scheme for a mobile robot system which makes use of a global representation of the environment by means of a self-organizing map or Kohonen network. In contrast to existing methods for self-organizing environment representation, this discrete map is not represented in the world domain or in the configuration space of the vehicle, but in the sensor domain. The map is built by exploration. A conventional path planning technique now gives a path from the current state to a desired state in the sensor domain, which can be followed using sensor-based control. Collisions with obstacles are detected and used in the path planning. Results from a simulation show that the learned representation gives correct paths from an arbitrary starting point to an arbitrary end point. 1. I N T R O D U C T I O N Goal-directed, collision-free robot navigation can be a result of planned actions, where a path planner determines off-line the optimal path from the current configuration to the desired configuration. Alternatively, it can be induced by a reactive behaviour, where sensor data is used directly to generate actions. If a planning method is used for navigation, global information about the environment is needed. Since such global information is not present in the data of sensors on the robot system, an internal representation of the environment must be used. Such a global model is usually represented in a configuration space spanned by the degrees of freedom of the robot system. For a mobile robot these are its position and its orientation. The environment is usually represented as a (geometric) model indicating the free and occupied areas. Given knowledge about desired and current configuration, a large number of methods exist for planning in configuration space or world space [4]. The construction of such an environment model can be done on the basis of a-priori knowledge, but the maps can also be learned by exploration. Self-organizing networks have been presented as maps in the world domain or configuration space [6],[7]. The role of sensor measurements in such a navigation scheme is to estimate the actual state of the robot system in the configuration space. Controlling the system consists of sending control commands to the actuators such that the error between actual state and desired state in the configuration space is minimized.
230 Reactive systems on the other hand consider the state of the robot not in configuration space but in sensor space. A state of the robot is now defined as a constellation of sensory data or features extracted from these data. It has been shown that the difference between the actual and desired state in sensor space can be used to control the robot, either using an explicit model of the system [11] or a learned, implicit model in the form of a neural network [9]. However, since the sensor information is inherently local, reactive systems suffer the from the existence of local minima. Methods have been described for escaping from those minima but do not result in optimal paths. The only guarantee for an optimal path is the use of global knowledge. For reactive systems this should be a representation in the sensor space, so that paths in sensor space can be planned which can be followed by a reactive system. In this paper we introduce the sensor space as the state space of the robot and describe some of the possibilities and problems for representing environment information. A selforganizing model is described which represents the environment in the sensor space. The model is used for path planning and path following for a non-holonomic mobile robot. 2. S E N S O R - B A S E D
CONTROL
AND GLOBAL
REPRESENTATION
Although animals pick up objects and navigate through complex environments without precise numerical three-dimensional position information, most robot systems still use an internal representation of the world in a carthesian domain. However, for certain robot tasks, robot motion can be successfully controlled directly by using sensor data or features derived from these data instead of computing positional information from these data. For robot manipulators, "visual servoing" methods have been developed [2],[11] which use a direct mapping from image features to control commands. These methods form the basis for our sensor space representation and navigation scheme for mobile robots. A mobile robot is equipped with a variety of sensors, which may be range sensors, visual sensors which detect marker positions in the image, beacon detectors, and so on. Let the state be described by the vector f - (fl, ..., fro) C 9r in which fi is a sensor output or the value of a feature derived from sensor data. The robot is controlled by sending it displacements Ax. For sensor-based navigation, the vehicle has to follow a trajectory f(t) in sensor space 9t-. Similar to visual servoing methods, an error in the sensor domain Af(t) can be mapped into the necessary motion A x via the m x n "Feature Sensitivity Matrix" JR: Ax = JFtAf.
(1)
in which J F t is the generalized inverse of JF, and can be derived from a model of the system or can be represented in a neural network which is trained from a learning set. For goal-directed navigation in a cluttered environment, the problem is to find a trajectory from a current state f to the goal state fgoaz. Similar as for carthesian-based navigation schemes, a procedure which navigates the vehicle along the vector f(t) - fgoaZ will be stuck in local minima. For this, we need a global representation of the sensor space, as well as methods to plan paths in this space. Our first question is how global environment knowledge looks like when expressed in the sensor space ~-. When the robot operates within a given environment R, all possible measurements f will lie in a fixed sub-space 7~ of the m-dimensional sensor space F , since not all sensor
231
Figure 1. For a fixed orientation of the vehicle all sensor data f will be represented as a 2-dimensional manifold in the n-dimensional sensor space. In a) this manifold is drawn for a 3-dimensional sensor space for the sensor configuration sketched in b). The environment is a square room.
vectors will occur in R. Furthermore, we know that this subspace is 3-dimensional, since the number of degrees of freedom to generate those data is only 3: position and orientation of the vehicle. In order to study the complexity of such a subspace we have computed and visualized this subspace for some simple situations. For example, the sensor space of a vehicle equipped with 3 range sensors is 3-dimensional. For a fixed orientation of the vehicle all sensor data f will be represented as a 2-dimensional manifold in this sensor space. Figure 1 shows the complex nature of the data. A trajectory of the vehicle in the room will result in a trajectory on the manifold. The remainder of the paper is concerned with how to construct this subspace which is basically finding a non-linear projection for the mapping of sensor data onto this subspace. The model is used for pathplanning and navigation.
3. R E P R E S E N T A T I O N MAIN
OF T H E E N V I R O N M E N T
IN THE SENSOR
DO-
Because it is assumed that the environment is not known (and even if it is known it is not a trivial task to find an analytical expression of the subspace) a learning m e t h o d has to be found which builds the internal representation, and is able to represent the nonlinear mapping from 9c to 7Z. In this paper we used a self-organizing vector quantization scheme in the form of a Kohonen network. A similar approach was presented by [3]. The Kohonen algorithm learns from examples a mapping from the input space jc to a lattice ~4 of N formal neurons rl, ...rN. Since we know that the intrinsic dirnensionality of the sensor data is three, a three-dimensional lattice is used. The map Cw : 9r ~ j t assigns to each sensor vector f C ~ an element Cw(f) E A, and is defined by the "weights"
232
l
16-d range data
compass r3
Figure 2. Different positions may result in identical sensor states. Therefor a compass was added. The value of the compass determines the winning "layer" while the value of the range sensor determines the winning neuron in a layer.
w = (wl, w2, ...wN), wi E ~-. The function Cw(f) is specified by the condition IWcw(f)- f l -
min Iwr - fl rcA
(2)
i.e., an sensor vector f is mapped onto that neuron r for which I W r - f] becomes minimal. During learning the weights of all neurons r in the network are adapted according to the Kohonen rule nWr = ~A(r, k ) ( f -
Wr),
(3)
where the learning parameter 7/is multiplied with A(r, k), a decreasing function of distance of neuron r to the winner k [8]. Learning also decreases with time to obtain a stable network.
4. E X P E R I M E N T S
ON REPRESENTATION
Our mobile robot is equipped with 16 range sensors, each of which gives the distance to the nearest object within a beamwidth of 2 degrees. The sensors are fixed on the vehicle at orientations uniformly distributed between 0 and 2~r. Each measurement can be considered as a vector f in a 16-dimensional sensor space. It is obvious that if the vehicle is not equipped with only range sensors but also with other external sensors such as beacon detectors, ultrasonic sensors, etc, the dimensionality of the Kohonen network can still remain three. A problem with using range data only is that (in particular for symmetric environments) range data at a particular location can be identical to range data at another location but for a different orientation of the vehicle. We equipped the vehicle with a compass in order to distinguish between these situations. With data from the 16-dimensional range data and the compass, a 10 x 10 x 10 network was trained. The 17dimensional sensor vector was not fed directly into the network but instead a hierarchical arrangement was used, similar to [5]. According to the output ~o of the compass a layer in the "z" direction of the network was selected. Within this layer a winning neuron k was selected as the neuron having a weight vector closest to the 16-dimensional range input vector f (see Figure 2). Weights of all neurons r in the network were adapted according to the Kohonen rule (Eq.3). Periodicity in the orientation ~ was taken into account by
233
Figure 3. Regions in the work space where a neuron is winner and positions where the distance to the winning neuron is m i n i m a l (vertices of network). This is an easy room.
a w r a p - a r o u n d of the network: neurons of the u p p e r and lower layer of the network are considered neighbours. For two environments we tested the algorithm. The first environment was a square room of 20 x 20 meters, with only two small obstacles in the lower-left and lower-right corner. The second test was done in a square r o o m with a n u m b e r of large obstacles. The latter r o o m was also used as test environment for the p a t h planning. Training samples are generated by positioning the vehicle at 40.000 r a n d o m positions and orientations in the r o o m and d e t e r m i n i n g the range data. In this way the 3-dimensional network in the sensor space is built. For visualization we d e t e r m i n e d which neurons were active for which configurations of the vehicle. This was done by systematically displacing the vehicle by small steps in the working space for a fixed orientation and r e p e a t i n g it for other orientations. For each position the label of the winning neuron was stored as well as the distance to this neuron.
234
Figure 4. Positions where the distance to the winning neuron is minimal (vertices of network). This is a difficult room.
Results are depicted in Figure 3. The boundaries between the regions in which a neuron is winner are plotted as well as the positions at which the distance to the neuron is minimal (vertices of the network). For the simple room depicted in Figure 3, the neurons form a nice square lattice in the room. However, for the more complex room we have used in our experiments, the results are somewhat different, as shown by Figure 4. Noticeable in the first place is the fact that some connections are missing. The reason for this is that only connections are drawn between nearest neighbours in the lattice. Apparently not all neurons will become winner during the test. This may be a result of the fact that because of the obstacles there will be many discontinuities in the sensor space. The Kohonen learning procedure may put neurons at the discontinuity. Secondly, the figure shows that very distant positions in the world space may activate neighbouring neurons in the network.
235 5. P L A N N I N G
AND NAVIGATION
It has been shown (e.g., [10]) that discrete representations of the working space, in the form of Kohonen or similar networks can be used for path planning using conventional search algorithms. Also, if the neighbour relations of neurons are considered as excitatory synaptic connections, the dynamics of such a topologically organized neural net can be used for path planning and obstacle avoidance [1]. In our application, where a path in sensor space has to be planned for a mobile robot this approach can not be followed for a number of reasons. 1. For non-holonomic systems such as our mobile robot, two neighbouring ('connected') neurons in the sensor space are not necessarily connected in the sense that an elementary action can bring the system from one state to the other. 2. If planning is done within the Kohonen network, the metric which will be used for path planning will be the metric of the lattice of neurons. The Kohonen learning rule places the nodes according to the probability density function of the inputs: regions in input space with high probability will have a large density of neurons. This means that the length of the path will depend on the distribution of learning samples. For this reason we decided not to use the neighbour relations of the Kohonen network for path planning. Instead we considered the navigation as a Markovian decision problem with the neurons of the Kohonen network as a discrete set of sensor states. During exploration of the environment, the sensor values f and corresponding actions u were stored. For each f the winning neuron ri is calculated before and after the action u, and an estimation of the state transition probability pij(u) is made. pij(u) is defined as the probability that, given an initial state (neuron) i, the vehicle is found in state (neuron) j after action u. In this way, neighbourhood relations between sensory states are imposed by the elementary actions. For navigation a greedy policy is used with respect to a cost-function f* which assigns to each neuron i the (expected) cost f* for driving toward the goal:
u -- argmin [c(u) + 9/ ~j pij(u) f~] ,
(4)
with c(u) the immediate costs of the action. The problem is to find the cost function f* for the Markovian decision problem. For this a dynamic programming approach is chosen. The cost function can be approximated via an iterative process if the direct cost c(u) of an action u and the transition probabilities are known:
fi(k + l) - min [c(u) + 3'~PiJ(u)fj(k) 3
,
(5)
in which fi(k) is the approximated costs for neuron i at time step k. This procedure converges to the optimal cost function f* if the goal state is initialized to zero and 7 < 1.
236
During exploration, collisions with obstacles are stored as well, so for each state i we can make an estimate of pi,cot(U), defined as the probability that given the state (neuron) i the vehicle will collide with an obstacle after action u. Note that the collision is seen as a discrete state rcoz, which also has a value for the cost function f*ot. In our model the cost for the collision neuron is always fcoz(k) = 1 + maxi fi(k), so the system will prefer actions which do not result in a collision.
6. E X P E R I M E N T S
ON PLANNING
Experiments were carried out in simulation. In our experiments, the set of possible actions b / w a s restricted to left, straight, right, indicating that the vehicle rotates along - 3 0 , 0 respectively 30 degrees, followed by a linear displacement of .75 m. The workspace of the robot is shown in Figure 4, which also shows a layer from the Kohonen network. S t a t e transition probabilities pij(u) were estimated by performing 100.000 random actions at random positions. The immediate costs c(u) of an action are equal to one for each action. We tested the planning algorithm by selecting an a r b i t r a r y goal state in the sensor domain. The vehicle was put at an arbitrary position and the cost function given in Eq 5 was iteratively determined. As soon as the current neuron (the neuron which is active at the current position) has neighbours with lower cost, a steepest descent m e t h o d can be applied: the action is selected for which the expected future costs are minimal. Note that this does not necessarily mean that the function f(k) at time step k has converged to the optimal solution f*(k). We therefore tested how many iterations were needed for a correct path. Results are shown in Figures 5 and 6. The lower part of Figure 5 shows the path of the vehicle if it starts moving after 10, 30, 40 or 200 iterations of the dynamic programming algorithm. The upper part shows the number of elementary actions of the vehicle which are needed to reach the goal state. If the number of iterations is less than 50 the vehicle does not reach the goal but collides with an obstacle. Figure 6 shows a more dimcult situation: the vehicle has to make a turn in a narrow alley. At least 50 iterations are needed before the cost function is sufficiently good for collision free navigation. Remarkable is that the path length increases after 300 iterations (while a good, short path was found after 50 iterations). The reason for this is our choice for the costs for the collision "neuron", which is always larger than the costs of any other neuron and therefore contributes (too) much to an expected cost assessment. The longer path is apparently a result of the trade-off between minimizing the probability on a collision and minimizing the expected number of actions needed to reach the goal.
7. D I S C U S S I O N Results show that an internal representation of the sensor space, acquired by an exploration of the environment, can be used for path planning and navigation. The sensor space of the robot system consists all possible constellations of sensory data which can occur in a given static environment of the system. In this paper a Kohonen network is used for the nonlinear mapping from the high-dimensional sensor space to the 3-dimensional network. The use of such a projection also enables us to take into account the uncertainty
237
Figure 5. The upper part shows the number of actions needed to reach the goal. A black disk indicates termination of navigation because of a collision. The lower part of the figure shows some typical paths of the vehicle if it starts moving after 10, 30, 40 or 200 iterations of the dynamic programming algorithm.
238
Figure 6. As in the previous figure. The lower part of the figure shows some typical paths of the vehicle if it starts moving after 10, 50, 200 or 300 iterations of the dynamic programming algorithm.
239 of the sensor data. If the data of one of the sensors is more uncertain than the data from another sensor, we can take this into account by lowering the contribution fi of this sensor to the distance measure which is used to determine the winning neuron. Some shortcomings of the approach have shown up which will have to be studied in the near future. One of the major drawbacks of the approach presented in this paper is that many learning samples are needed: the Kohonen map already needs about several thousands of samples, but to estimate the transition probabilities we need even more samples. In a more efficient approach both the Kohonen map and the transition probabilities could be learned at the same time, while a more realistic learning procedure the learning samples will not be generated by random placement of the vehicle but by an exploratory navigation.
Acknowledgements The authors thank Leo Dorst for discussions on the shape of the sensor data subspace and his help in plotting the space for a simple situation. The work was carried out partly within the project "Active Perception and Cognition", supported by the Japanese RWC programme.
REFERENCES 1. Glasius, R., Komoda, A., Gielen, S. Biologically Inspired Neural Networks for Trajectory Formation and Obstacle Avoidance. in: S. Gielen and B. Kappen (eds) ICANN'93, Proceedings of the International Conference on Artificial Neural Networks. Springer-Verlag, (1993) 281-284. 2. Won Jang and Zeugnam Bien, Feature-based Visual Servoing of an Eye-In-Hand Robot with Improved Tracking Performance. Proceedings of the 1991 IEEE Int. Conf. on Robotics and Automation, (1991), 2254-2260. 3. Kurtz, A. Building maps for path-planning and navigation using learning classification of external sensor data. in: I. Aleksander and J. Taylor (eds): Artificial Neural Networks 2, North-Holland/Elsevier Science Publishers, Amsterdam, (1992) 587-591. 4. Latombe, J.C. Robot motion planning. (1991) Kluwer Academic Publishers, Boston. 5. Martinetz T, Schulten K. Hierarchical neural nets for learning control of a robot's arm and gripper. IJCNN-90 Conf. Proc., San Diego 1990, III, (1990) 747-752. 6. Morasso, P., Vercelli, G. and Zaccaria, R. Hybrid systems for robot planning, in: I. Aleksander and J. Taylor (eds): Artificial Neural Networks 2, North-Holland/Elsevier Science Publishers, Amsterdam, (1992) 691-697. 7. Najand, S., Zhen-Ping Lo, Bavarian, B. Application of Self-organizing neural networks for mobile robot environment learning, in: G. Bekey and K. Goldberg, Neural networks in robotics, Kluwer Academic Publishers, Dordrecht, (1992) 85-96. 8. Ritter, H, Martinetz T, Schulten K. Topology conserving maps for learning visuomotor coordination. Neural Networks 2, (1989) 159-168. 9. P.P. van der Smagt and B.J.A. KrSse. A real-time learning neural robot controller. In: Kohonen, M/ikisara, Simula and Kangas (eds): Artificial Neural Networks. Elseviers Science Publishers B.V. (1991) 351-356.
240 10. J.M. Vleugels, J.N. Kok and M.H. Overmars A self-organizing neural network for robot motion planning, in: S. Gielen and B. Kappen (eds) ICANN'93, Proceedings of the International Conference on Artificial Neural Networks. Springer-Verlag, (1993) 281-284. 11. Weiss, L.E., A.C. Sanderson and C.P. Neuman. Dynamic sensor-based control of robots with visual feedback. IEEE Journal on Robotics and Automation, RA-3, (1987) 404-417.
Intelligent Robots and Systems V. Graefe (Editor) 1995 Elsevier Science B.V.
241
Path Planning and Guidance Techniques for an Autonomous Mobile Cleaning Robot Christian Hofner and G/inther Schmidt ~* ~Department for Automatic Control Engineering (LSR), Technische Universit~t Mfinchen, D-80290 M/inchen, Germany P h o n e : (+49 89) 2105-3416, F A X : (+49 89) 2105-8340 E-mail: [email protected] K e y w o r d s : Cleaning path planner, self-localization, sensor application planner, tracking error compensation, obstacle avoidance. Application of freely navigating mobile robots to service tasks such as floor cleaning or inspection requires specific techniques for path planning and vehicle guidance. This article describes in its first part a rule-based geometric path planner. In addition to a 2D environmental map the planner takes into account robot kinematics and geometry as well as technological requirements. The result of path planning is a list of motion commands which leads during path execution to a most complete coverage of a user-specified cleaning area. In its second part the article discusses problems of vehicle guidance including initialization of the robot's start location, cleaning path execution, accurate path tracking, vehicle localization as well as path replanning in case of obstacles. The article concludes with a discussion of implementation issues and results of a floor coverage experiment with the full-scale mobile robot MACROBE. 1. I N T R O D U C T I O N Many challenging research problems arise in the application of freely navigating mobile service robots to floor cleaning or related tasks in extended public areas such as corridors, halls or platforms [11]. Although most of todays cleaning and sweeping machines are still guided by human operators there exist already several semi-autonomous cleaning robots for commercial usage [12]. These systems assist a human operator in cleaning of rectangular or polygonal shaped areas. Obstacles cause stops, until either the operator or the obstacle clears the blocked path. A few publications [4],I2] discuss application of a mobile robot to the cleaning task without making use of explicit a-priori knowledge about geometry and complexity of the cleaning area. In spite of these developments there exist many open problems in intelligent path planning and vehicle guidance for cleaning robots due to advanced requirements as listed in Table 1. *The work reported in this article was supported by the Deutsche Forschungsgemeinschaft DFG as part of an interdisciplinary research project on "Information Processing Techniques in Autonomous Mobile Robots (SFB 331)". The authors would like to thank colleagues and students of the laboratory for their valuable contributions to this work.
242 Table 1 Advanced requirements for an autonomous floor coverage robot Task-related requirements Identification of the cleaning area Path planning considering technological and economical aspects Accurate long distance path tracking
Safety/operational requirements Man-robot-interface for operator assistance Collision and tumbling avoidance Detection of obstacles with the result of velocity adaptation and path replanning
Detecting, memorizing and cleaning temporarily occupied parts of the cleaning area
In [8], [9] we have introduced basic ideas of our planning and guidance approach for autonomous floor-covering operations in extended indoor environments. In Section 2 of this article we describe in more detail the rule-based planning system which automatically generates the motion command sequence for an appropriate cleaning path according to robot geometry and kinematic restrictions. Results from automatic planning are illustrated for robots with various degrees of agility in typical cleaning environments. In Section 3 we discuss a vehicle guidance scheme comprising regular robot localization, i.e. estimation of position and heading, as well as appropriate compensation of drift errors during path execution. Unlike obstacle avoidance approaches known from point-to-point operations we propose in Section 4 a new approach for detection and handling of obstacle situations during cleaning motion. In Section 5 we discuss the implementation of the total control structure as well as results from a floor coverage experiment evaluating the proposed path planning and tracking techniques. 2. P L A N N I N G
OF A C L E A N I N G P A T H
Reasonable guidance of a cleaning robot in real indoor environments requires an appropriately planned cleaning path. A few publications have already introduced grid-based [13] or rule-based [4] planners for complete floor coverage. However, a flexible path planner must pay attention to the technological feasibility of a cleaning path in real indoor environments and to problems arising during path execution. Thus robot geometry and agility as well as appropriate adaptation of the cleaning path to the specific environment must be considered. In this Section we describe a semi-automatic approach for generation of a robot motion program which ensures most complete coverage of a specified area.
2.1. Geometry and Agility of the Robot and its Cleaning Units To define relevant design parameters of mobile robots for floor coverage operations we refer to the features of typical cleaning machines as illustrated in Figure 1. Most of these conventional systems are equipped with two or more cleaning units [6]. The front unit comprises brushes soaked in detergent. They scrub the dirt from the floor. Dirt is then removed by a vacuum suction unit at the rear end of the vehicle. For path planning we will always assume rectangular shaped vehicles with car-like non-
243
Figure 1. Manual cleaning machine
holonomic kinematics but varying degrees of agility. Agility is characterized by the robot's minimum turning radius train. Furthermore we will assume the existence of two cleaning units mounted at the front and rear end of the vehicle being as wide as the robot itself, i.e. dd~an - b. 2.2. R e p r e s e n t a t i o n of t h e C l e a n i n g A r e a With respect to the environment we assume the existence of a 2D-map of a-priori known walls, pillars, staircases or other fixed objects. These items are represented by closed 2D-object polygons as illustrated in Figures 4b) and 5b). Planning starts with operator-based identification of the borders of the cleaning area in a visualized 2 D - m a p which is part of the planning system. Both, the borders and all object polygons inside the cleaning area are represented by contours. Each contour c is described by cartesian end points SPc and EPc with respect to an inertial reference system E, see Figure 3. 2.3. S e l e c t i o n of a Start L o c a t i o n To determine the alignment of motion patterns to be planned, the operator selects in the visualized map an appropriate start position and a reference contour. Typically a point at a corner and a neighbouring wall are chosen for this purpose. Based on this information the robot's start location S as illustrated in Figures 4b) and 5b) is automatically calculated. S is defined by an initial path frame/9o = (x0, y0, r representing vehicle position (x0, y0) and heading r with respect to E.
244 2.4. Automatic Planning of a C l e a n i n g P a t h For satisfactory floor coverage a cleaning path consists typically of overlapping parallel tracks. Two basic changing maneuvers are used for connecting neighbouring tracks. 9 U - T u r n defined by desired turning radius rturn and a turning angle of 180 ~ 9
S i d e - s h i f t defined by a concatenation of two circular arcs with different sign of
curvature U-turn maneuvers are preferred for path planning, because no change in motion direction is required and operating time as well as floor coverage redundancy can be reduced. Because of vehicle swing out during U-turns, safety distances between objects and the vehicle have to be considered [8]. On the other hand Side-shift maneuvers are preferably employed in narrow areas or in case of a lateral approach to a wall. Once all parameters of the robot and its environment are defined the planning procedure a u t o m a t i c a l l y constructs a technologically feasible cleaning path based on basic path planning templates.
2.5. Basic Path Planning Templates Five templates proposed for path planning are defined in Table 2 and Figure 2. Table 2 Basic path planning templates Template TM UT SS UT-Stretched UT-Squashed
Purpose Move Move Move Move Move
straight straight straight straight straight
forward/backward forward and change track using a backward and change track using forward and change track using a forward and change track using a
simple U-turn a Side-shift stretched U-turn squashed U-turn
Application of a certain template depends on the robot's agility (rmi,). For this reason, path planning distinguishes two types of robots. Vmin -< rt~,rn rturn < r m i n <~ 2 " r t ~ n
" Type I robot I with 9 Type II robot f
vtu,.n - dclean -- doverlap
(1)
2
rt=rn is the radius required for a U-turn to connect neighbouring tracks. Each track has an efficient cleaning width ddean including an overlapping width d.... tap. Most complete floor coverage is achieved for both Type I and Type II robots, if the templates as illustrated in Figure 2 are used for path construction. The determination of template parameters is discussed next.
245
Figure 2. Templates for path construction With each template a geometric motion corridor (rectangular, circular or combined) is defined. It is used for map-based collision analysis between the robot and known contours. The result is a list of discrete subgoals Pi = (zi,yi,r defining a collision-free section of the robot's motion path. The procedure of subgoal calculation is discussed in more detail for the UT-template as shown in Figure 3.
Figure 3. Collision analysis for UT-template
246 Table 3 Algorithm for subgoal calculation in case of a UT-template
(1)
Define a rectangular motion corridor RC with width dclean from current robot location Pi-1. Transform contours with respect to Pi-1. Calculate critical collision point Ccrit - (Zcrit,Ycrit) between contours c and RC with respect to Pi-1. For a circular motion corridor CC with radius rcc (1) calculate its center point CP for Cc~it intersecting CC. The lateral displacement of CP with respect to Pi-1 is given by rturn. Consider contours to be part of CC. Determine critical contour by calculating point Ccoll which is part of this contour while having the smallest radial distance to CP. Calculate the vertical displacement Az of CP for CC touching the critical contour. If Ax > v e t then return FAILURE and stop else calculate Pi relative to Pi-1 " Pi - (xi,yi,r and return SUCCESS. In case of SUCCESS calculate subgoals Pi+l - (rtur,,, -+-rtu.,, +90 ~ and Pi+2 = (rtu~,,,-+-rt,,~,,, • ~ to complete the collision-free U-turn description. r~ depends on the vehicle's geometric parameters and rt=~,,
Assuming that the robot is currently located at Pi-1, the last subgoal from application of a preceding template, collision analysis is performed by the algorithm given in Table 3. From the subgoals for the UT-template illustrated in Figure 3 the planner generates the following list of motion commands for a mobile robot with a cartesian motion controller [10]. M OV E_REL ATIVE (zi, MOVE_RELATIVE(zi+I, MOVE_RELATIVE(zi+2,
yi,
r
Yi+I,
r r
Yi+2,
FORWARD, FORWARD, FORWARD,
i) i+l) i+2)
2.6. Automatic Selection of Basic Path Planning Templates A complete cleaning path as illustrated in Figures 4b) and 5b) is constructed by successive selection of basic path templates according to a logical sequence of rules. The rule sequence for a Type I robot is illustrated in Figure 4a) and can be easily verified by the planning example given in Figure 4b). From the operator-defined start location S = P0 inside the selected cleaning area the appropriate shift direction is determined. For the first section of the path in Figure 4b) the planner selects a UT-template with shift direction to the right. If the UT-template algorithm returns SUCCESS, the robot will be located at P3. From P3 the new shift direction is determined and again a UT-template is selected. The concatenation of feasible UT-templates leads automatically to a Snake-trail pattern with overlapping tracks, see Figure 4b). If a U-turn cannot be performed due to environmental restrictions such as a side wall, a SS-template is called upon. The radius
247 of the two arcs is automatically calculated for a collision free track change. Successful application of SS-templates leads to a Side-shift pattern, see Figure 4b). Application of a SS-template fails, if a lateral border of the cleaning area is detected inside the specified motion corridor. In this case floor coverage is assumed to be completed and path planning is terminated.
Figure 4. Construction of a complete cleaning path for Type I robot : (a) Outline of template selection procedure (b) Path planning example
Due to their size, weight and velocity Type II robots usually operate in extended cleaning areas. To achieve reasonable floor coverage with this type of robot the set of path templates needs to be extended by the so-called stretched UT-template and squashed UT-template as illustrated in Figure 2. Application of this set of templates requires a more sophisticated selection procedure, given by the rule sequence for Type II robots. In Figure 5a) only the major selection cycle is illustrated which concatenates feasible path templates to an interlaced Snake-trail pattern. This is equivalent to the UT-template cycle in Figure 4a). It is obvious that additional template selections are required for completing the cleaning path, if application of a template returns FAILURE. Application of interlaced Snake-trail patterns to cleaning areas with complex structured boundaries leads to unsatisfactory floor coverage. To cope with this drawback, the planning procedure tests before and after each feasible U-turn, whether an additional forward/backward track motion can cover additional floor by using a TM-teraplate. Examples are shown in Figure 5b).
248
Figure 5. Construction of a complete cleaning path for Type II robot: (a) Major cycle of template selection procedure (b) Path planning example 2.7. Evaluation of Floor Coverage The ultimative objective of autonomous floor cleaning is complete coverage of a specified area at minimum operating time. However, technological restrictions and navigation in real indoor environments make it rather difficult to specify an "optimal" cleaning path. Experience has shown, that reasonable floor coverage is achieved by a cleaning path composed of appropriate Snake-trail and Side-shift patterns. To evaluate floor coverage for a planned cleaning path the following performance measures are defined m
top [ m i n ]__ ~ s , ~ i=1
qtoua
[%]=
Vi
100.
cov
q~
+
(2)
taccj j=l
Acovered A~pr162 Aco~,e,-~d
(3) (4)
top
[%]-
loo.
Apath
A~o~
_
_
~)
(5)
top represents the operating time for floor coverage and is calculated according to the velocity profile of the robot along its planned path. This profile considers path sections si with constant velocity vi, as well as (de-)acceleration phases tacc~ due to kinematic restrictions and changes in motion direction.
249 qtotal represents the ratio of the actually covered area A . . . . . . d and the specified cleaning area Asp~. Due to complexity of motion patterns and the modelled environment, qtot~t is estimated by segmentation of the visualized cleaning area with methods known from image-processing. Cleaning performance pco~ is a common term in cleaning technology and represents the floor space in square meters which can be covered within one hour. qred considers the ratio of Acovered and the area covered according to the planned cleaning path Apath. This measure indicates redundancy caused by overlapping tracks, interlaced Snake-trail and Side-shift patterns. Ap~th is determined as follows .
circles m
n
i=1
j=l
(6) tracks with Ai
=
dd~an . li
A,
-
n (~'jec,~an -- h ) + (R '~ + h') ~ t ~ .
R
=
dd~n rtu,-n+-2
h,f
:
distance from robot's turning axis to its rear end, resp. front end
dd~
:
efficient cleaning width
l,
:
length of a single track
~j, rt ....
:
turning angle, turning radius
-k
+ T " (f" -
<'~~176
Table 4 Results from floor coverage evaluation Geometric data Path length Average velocity Specified floor Covered floor Coverage along the path
1 ~ A.,,e~
A ....... d Avao~
[m] [mm] 8 [rn2] [m 2] [m 2]
Performance value Operating time Total coverage Coverage redundancy Cleaning performance
top qtolat q,-ed pco,,
[min] [%] [%1 [-~]
Figure 4b)
Figure 5b)
236.6 430 124.6 113.1 122.8
144.6 460 124.6 101.1 158.3
Figure 4b)
Figure 5b)
9.5 90.8 8.5 715.7
6.4 81.0 56.6 953.0
250 The described path planning system automatically generates performance values for a planned cleaning path. Table 4 summarizes evaluation results for the examples of Figures 4b) and 5b). Analysis of the quality values indicates, that achieved floor coverage and cleaning performance depend on the specific environment and the appropriately selected robot technology (geometry, agility etc.). The proposed quality measures strongly support further improvement of path planning techniques with respect to the ultimative objectives of cleaning operations. 3. V E H I C L E G U I D A N C E There are two important issues to be considered for successful nominal execution of a planned cleaning path : 9 The specified start location S of the cleaning path from an arbitrary robot location in the vicinity of S must be found, an operation which we call initialization. 9 Path errors must be compensated in order to achieve satisfactory floor coverage over long distances. Both operations are supported by a combined sensor and map based self-localization procedure. 3.1. I n i t i a l i z a t i o n of R o b o t M o t i o n Cleaning motion starts at the selected start location S of the planned cleaning path, e.g. a room corner as illustrated in Figures 4b) and 5b). Since a human operator cannot be expected to position the robot precisely at S, but somewhere in its vicinity, selflocalization is performed and the robot maneuvers to S. The maneuver is planned and executed by a local path planning and execution module which is part of the robot control system as described in [1]. Self-localization performed during initialization and path execution is briefly discussed in the following Section. 3.2. S e l f - l o c a l i z a t i o n For estimating robot position and heading in the environment, natural landmarks detected by sensors are matched with corresponding contours in the 2D-map. For landmark detection ultrasonic sensors with a maximum range of 1.5 m [3] are used. A minimum configuration of eight sensors mounted around a rectangular shaped robot is illustrated in Figure 6a). Taking into account typical problems arising with ultrasonic range sensors, a reliable absolute localization technique as reported in [7] was adopted for vehicle guidance. The main processing steps for estimating robot location L = (ZL, YL,r by means of ultrasonic range measurements and dead-reckoning data are summarized in Table 5. Position and heading of the robot cannot be completely determined if either the robot moves along or towards a single contour or if faulty sensor measurements have to be eliminated. In this case missing components of L are replaced by the corresponding values of the dead-reckoning system. This avoids undefined situations when path tracking errors must be compensated.
251 Table 5 Main steps of location estimation Self-localization starts with a rough estimate of the robot's current location characterized by an uncertainty area E, see Figure 6a). 9 For robot initialization, E is parameterized by the path planner around the selected start location S. 9 During path execution, E is parameterized around the current robot location 0 = (ZDn, YDR,CDR), given by the dead-reckoning system. .
.
W h e n approaching a landmark a correspondence tree as shown in Figure 6b) is generated. Each branch represents the set of map contours detectable by relevant ultrasonic sensors. Different branches are generated by variation of robot location in E. By use of appropriate models of the sensors' field of view, the number of possible branches can be kept small. Based on map and current sensor data a possible robot location and a quality measure is calculated for each branch using least-squares-fitting, see [7]. The robot's true location L = (XL,YL,~ZL) is selected from the set of possible locations by use of the quality measures.
Figure 6. Self-localization at natural landmarks: (a) Ultrasonic range sensor configuration (b) Correspondence tree
252
3.3. Automatic Planning of Sensor Application Automatic planning of sensor application, i.e. selection of appropriate localization points, is supported by the subgoals already determined during path planning. Since subgoals are typically located in the vicinity of landmarks such as walls or pillars, they are easily detectable by ultrasonic sensors. Localization points are selected from subgoals according to the following criteria: Landmarks in the vicinity of a subgoal must be detectable by at least two sensors. This is a prerequisite for deriving a good estimate of current heading which is a most sensitive parameter for accurate path tracking during cleaning. For checking this condition the set of intersections between the modelled sensors' field of view and the respective contours of the 2D-map is tested for the robot positioned at the subgoal, see Figure 6a). Path length between two localization points should not exceed a threshold value given by the errors of the dead-reckoning system and the specified overlap dowrtap of neighbouring tracks. If a subgoal meets the above requirements, the corresponding motion command is marked by a so-called localization flag. If such a command is passed to the cartesian motion controller during path execution, vehicle guidance initiates self-localization when the robot approaches the subgoal. Localization points as selected by automatic sensor application planning are given in Figure 9b).
3.4. Compensation of Path Tracking E r r o r s For compensation of tracking errors during path execution, the following data are available 9 current subgoal Pi = (xi, yi, r 9 current location L = (xL,YL, eL) estimated by self-localization, see Section 3.2 9 current location O = (XDR, YDR, ~,3DR) provided by the dead-reckoning system As all data refer to the same reference system E, transformation vectors -~L between L and P as well as -~ORbetween O and P are calculated. The robot's errors in heading and position are then given by e = ]-~L- ~-DRI" Error compensation is activated according to the following criterion: 3
Compensation
YES
NO
if
V ISt,- 5DR,I > ~ , i--1
(7)
otherwise
The threshold ~max is known from experiments. Errors are compensated during robot motion by appropriate modification of subgoal values generated by the path planner. If localization fails locally, compensation is not initiated and the robot continues its path to the current subgoal. A more detailed description of this procedure is given in [81.
253 4. O B S T A C L E A V O I D A N C E Obstacle avoidance strategies are constrained by requirements of satisfactory floor coverage. This means among others, that temporary obstacles should not cause immediate path modification, while permanent obstacles should be passed according to task-related requirements. 4.1. O b s t a c l e D e t e c t i o n A structured-light CCD-sensor as described in [5] is used for obstacle detection. It provides 2D line segments in a 3m * 3m field of view in front of the vehicle. According to Figures 7a) and 7b) a line segment represents an obstacle, if it intersects the surveillance corridor (SC) of the robot moving to its current subgoal. Detected objects, which will never be reached by the robot along its planned path, are not considered as obstacles.
Figure 7. Obstacle avoidance: (a) Obstacle detection (b) Replanned cleaning path after map update
For a blocked path the robot estimates collision distance d. If d < dcrit, a softstop is performed. The robot slows down smoothly, but does not stop in case of temporary obstacles, while a safe stop occurs in front of permanent obstacles. If the path is not cleared within a given time, path replanning is initiated.
254
4.2. Path Replanning The path planner as described in Section 2.4 is used online to construct a new cleaning path. Passing an obstacle on this path complies with the requirements of good floor coverage. For path replanning the map of the cleaning area is updated by the line segments representing an obstacle. Current robot location is considered as the new start location S*. A simulation result of path replanning and resulting floor coverage with a Type II robot is shown in Figure 7b). 5. R E S U L T S
5.1. Implementation Issues MACROBE, which must be classified as a Type II mobile robot, is used for experimental validation of the proposed planning and guidance techniques. The overall control structure is shown in Figure 8. Two cascaded loops of cyclic perception, planning and execution ensure both accurate long distance path tracking and adaptive cleaning motion in case of obstacles. The gray shaded blocks have been implemented on a notebook computer.
Figure 8. Overall control structure for floor coverage robot
255
5.2. Floor Coverage Experiment The following experiment was performed in a FMS pilot factory. A view into the cleaning environment is shown in Figure 9a), the corresponding map and results from planning are presented in Figure 9b).
Figure 9. Floor coverage experiment" (a) Cleaning environment (b) Path planning results
Using a joystick, the operator positions the robot in the vicinity of S. Next the robot initializes its location and maneuvers to S. The result of initial self-locahzation in Table 6 demonstrates, that the operator had positioned the robot with deviations up to lm in motion direction and up to 0.5m in lateral direction with respect to S. During path execution the tracking error is determined, whenever the vehicle approaches one of the planned localization points LP 1 to LP 4, see Figure 9b). If the error threshold Cma. = (Az, Ay, A r (50 mm, 50 mm, 0.5 ~ is exceeded, compensation is initiated. Numbers given in Table fi demonstrate, that path tracking errors are kept small and overlapping of neighbouring tracks is guaranteed. The '*' indicates, that at LP3 and LP4 the robot's lateral displacement and lateral error cannot be determined by self-localization. Processing time for location estimation is in the order of 300 msec on a 48fi/33-MHz computer. This includes generating the correspondence tree as well as calculating the
256 Table 6 Initialization and path tracking errors Task Initialization Tracking Tracking Tracking Tracking
(LP (LP (LP (LP
1) 2) 3) 4)
Lateral error [mm]
Longitudinal error [mm]
Orientation error [o]
Compensation
-963
+510
+1.1
yes
+2 +45 * *
* +2 +10 -10
-0.4 -0.5 +0.9 -2.7
no yes yes yes
current location. Experiments demonstrate, that vehicle guidance satisfies real time requirements for velocities up to 500 ram. This is a velocity typical for conventional cleaning machines. 8eC
Figure 10. Achieved floor coverage in the experiment
Figure 10 illustrates floor coverage achieved in the experiment according to recorded data based on MACROBE's dead-reckoning system and the localization module. This and further experiments [9] have demonstrated that the presented planning and guidance approach meets major requirements as given in Table 1.
257 6. C O N C L U S I O N S This paper proposed an automatic path planner which generates a technologically feasible cleaning path and the corresponding list of cartesian motion commands. Execution of this path leads to most complete floor coverage both in narrow and wide cleaning areas. Stable path execution is supported by regular self-localization at preplanned points in the vicinity of natural landmarks. Obstacle detection and avoidance is achieved by use of a new structured-light CCD-sensor and online application of the proposed path planner. Experiments in real-world environments have demonstrated that the implemented sensor and control approach can improve capabilities of current cleaning machines at reasonable additional cost. Further work will focus on intelligent behaviour of the robot in case of obstacles. For example, memorizing of temporarily inaccessible subareas can help to cover remaining uncleaned floor after completion of the nominal cleaning task. REFERENCES 1. Bott, W. et al: A utomatische Planung und A usfiihrung lokaler FahrmanSver fiir Roboterfahrzeuge; VDI-Bericht, Nr. 1094, Diisseldorf, Germany, 1993, pp. 763-772 2. Burhanpurkar, V.P. : Design of Commercial Autonomous Service Robots; Publication of Cyberworks Inc., Ontario, Canada, 1993 3. FrShlich, C. et ah Multisensor System for an Autonomous Robot Vehicle; Information Processing in Autonomous Mobile Robots; Proc. of the International Workshop, Mfinchen, 1991, Springer Verlag, Heidelberg, pp. 61-76 4. Furuhashi, H. et al: Development of an Autonomous Cleaning Robot with an External Power Cable.; Proc. of the International Conference on Motion and Vibration Control, Yokohama, Japan, 1992, pp. 499-504 5. Gilg, A.: Laser-Lichtschnittsensor mit homogener AuflSsung dutch asphSrischen Spiegel; Internal report, Dept. for Automatic Control Engineering, TU Mfinchen, Miinchen, Germany, 1993 6. HAKO-Company: Information brochures; Bad Oldesloe, Germany, 1994 7. Hanebeck, U.: Kurziibersicht zur Lokalisierung mit Hilfe yon UltraschaUsensorik; Internal report, Dept. for Automatic Control Engineering, TU Mfinchen, Mfinchen, Germany, 1993 8. Hofner, C.; Schmidt G.: Fl~ichendeckende Kursplanung und Fiihrung autonomer Reinigungsfahrzeuge; Proc. of the 9th Workshop on AMS, Mfinchen, Germany, 1993, pp. 27-38 9. Hofner, C.; Schmidt G.: Path Planning And Guidance Techniques For An Autonomous Mobile Cleaning Robot; Proc. of the International Conference on Intelligent Robots and Systems (IROS), Mfinchen, Germany, 1994, Vol. 1, pp. 610-617 10. Horn, J. : Cartesian Motion Control of a Mobile Robot; Proc. of the Intelligent Vehicles 94 Symposium, Paris, France, 1994, pp. 443-448 11. Schraft, R.D. et ah Service robots : The Appropriate Level of Automation and the Role of Users/Operators in the Task Execution; Proc. of the International Conference on Systems Man and Cybernetics, France, 1993, Vol. 4, pp. 163-169 12. Villmer, F.-J.: Einsatzfelder und Applikationen yon Dienstleistungsautomaten in der Reinigung; Proc. of the IPA-Technologie-Forum, Innovative Technologien ffir Dienstleistungen, Stuttgart, Germany, 1994, pp. 81-120 13. Zelinsky, A. et al: Planning Paths of Complete Coverage of an Unstructured Environment by a Mobile Robot; Proc. of the ICAR 93,1993, pp. 533-538
This Page Intentionally Left Blank
Intelligent Robots and Systems V. Graefe (Editor) 1995 Elsevier Science B.V.
259
Grasp Strategies for a Dextrous Robotic Hand Kurt Woelfl and Friedrich Pfeiffer Lehrstuhl B fiir Mechanik Technische Universitgt Miinchen 80290 Munich, Germany
Abstract The grasp strategies presented in this paper were developed at the Lehrstuhl B [iir Mechanik for the T U M Hand, a dextrous hydraulic hand with four fingers. They cm'er three main aspects of grasping: the calculation of the grasp points and fi.nger forces, the positioning of the object wit hil~ the ha.nd, and the regrasping of the object within the hand. A decompostion of the finger forces into their normal and tangential, rather than internal and manipulation components it used together with a descriptio~ of the object as a generalized ellipsoid. Once the grasp points and the finger forces are known, the probiem of correctiy placing the object within the hand can be considered. In order to maximize the range of possible applications for the T U M Hand, a. stra.teoiv is presented which allows the regrasping of an object. During regrasping, a new grasp .it established on the object while it remains firmly hehl in the hand.
Introduction Since the development of the Stanford/JPL Hand [10] and the U t a h / M I T Ha,nd [1], the dextrous manipulation of objects has emerged as a new field in robotics. \\:ork done in this area can be devided into three groups: the development, of robotic hands, the control of robotic hands, and the design of grasp strategies. In recent years, grasp planning for dextrous hands has become a new area of research for robotic applications. If dextrous hands are to find industrial applications, grasp strategies are needed to automate t he grasping process. These strategies must not, only calcul~te the finger forces necessary to manipulate the object [9], but also locate the fingers on tlle object in such a way that a st al)le grasp can be achieved [8]. With a few exccptions [6], the work on grasp planning llas focused on one aspect or the other. In this paper, a grasp strategy is demonstrated which accomplishes both tasks. Given the desired external forces on the object and the object geometry, llle strategy calculates the grasp points and the finger forces necessary to achieve the desired external wrench on the object. Also critical to automatic grasping processes is the positioning of the hand in such a, manner that the object can l~e efficiently grasped. Both the position and the orientation
260 of the hand relative to the object play an important role in reaclling the desired grasp. This paper describes a method in whicl~ both the orientation of the objc'ct within the hand and the position of the ol)ject relative to the hand are determined, so that all the fingers can reach the desired grasp points. During complex manipulalions, it is often necessary to change the grasp of the hand on the object. A method is presented which allows a free finger to grasp tlle object at a new point and thus freeing one of the other fingers from its contact with the object. During the entire regrasping process, the object remains firmly held within the hand.
Description 2.1
Object
of the Object
as a Generalized
and the Finger
Force
Ellipsoid
Throughout the work presented in this paper, the object's surface is described as a generalized ellipsoid:
clpy iz cztp +
a.~,
ay
-
az
1, Pi
> 0
}
(1)
and a z are the half-axis lenghts. With powers Pi - 2 , (i - x , y , z ) t h e equation describes an ellipsoid, whose center is located at ( c x , Cy, Cz). It is always assumed that the origin of the body-fixed coordinate system is located at the center of the object, so that c x - c y - c z - O. h lcreasing the powers Pi of the generalized ellipsoid makes the object more rectangular.
ax, ay
K
Figure 1" The Geometrical Parameters Using Equation 1, each point on the ol)ject's surface can be described as a function of the two angles C and ~ (see Figure 1). The cartesian coordinates are then described by:
y -
z -
r(C,~)sinCcos
(2)
261 Substituting these equations into equation (1) leads to
~(r ' ~) cos r cos ~ ]p~ ax
+
r(r ' ~) sin r cos ~ Ipy ay I
+
~(r ~)si~ ~ Ip~
'--
-t
(3)
When the two angles r and ( are known, the distance r(r ~) to the point can be found by solving this equation numerically. The cartesian coordinates of the point can then be found through Equations (2). The base-vectors which are normal (n) and tangent (ell , el2 ) to the surface are then described by:
(~r ell
=
~5--7" =
~ ( r ~~) ~in r ~o~ ~ + ~(r ~) cos r cos
~r
~(r ~~)cos r Cos ~ - ~(r ~) cos r sin (~r
e~2 =
5~-
5r(r ~) sin r cos ~ - r(r ~) sill r Sill
(4)
&(r n
=
6tlet2
Every generalized ellipsoid can thus be described as a function of the two paralnters and 4. With these two angles, the cartesian coordinates of the point and the base-vectors normal and tangent to the surface at this point can be found. In order to demonstrate lhe effect of increasing the power of the generalized ellipsoid, Figure 2 shows an ellipsoid with powers Pi = 2, Pi = 4 and Pi = 10. The transformation of the ellipsoid to a rectangular solid can be clearly seen.
2.2
Decomposition
of the Finger Forces
In most of the past research performed on grasping an object with a multi-fingered hand, the conta.ct finger forces have been divided into two components, the manipulation force and the internal force. The manipulation force, introduced by Yoshikawa and Nagai [12], is defined as the force wllicl~ generates the required external force on the object. The internal force was originally defined by Salisbury [10], as the component of the finger force which exerts no net resllltant force on the object. Although variations of these definitions have been proposed [13], most researchers decompose the finger" forces into these two components (for" examples see [3] [4]). This paper proposes the llse of a. novel decomposition of the finger forces into its normal and tangential directions, a met llod supported by [2]. Because the finger forces rely on the contact between tlle finger and the object, this decomposition not only allows physical insight into the roles of the normal and tangential (frictional) forces but also simplifies the mathematical description of such terms as the pressing of the finger against the object, or the slipping o[ l lie finger oll the object's surface.
262
Figure 2: Generalized Ellipsoid with Increasing Powers
3
Grasp C a l c u l a t i o n
One major aspect of grasp planning involves the determination of the grasp points and the calculation of the finger forces necessary to achieve the desired external wrench on the object. A method is presented which, given the desired external wrenches on the object. and the object geometry, calculates not only the necessary finger forces, but also the points where the fingers should grasp the object. If the grasp points are already known, a very similar method described in [11] can be used. In this paper, it is assumed that each finger acts as a point contact with fl'iction and thus cannot exert moments. It is also assumed that the object geometry (as described in Section 2.1) and the desired external wrench which the hand should exert on the object are known. In the example presented, a. three-fingered grasp is assumed, so that the fourth finger of the TUM Hand remains available for possible regrasping.
263
3.0.1
The Optimization Problem
The grasp strategy presented here which calculates the finger forces and the corresponding grasp points can be described as an optimization problem with specific constraints. This problem is summarized in Table 1.
O p t i m i z a t i o n Criterion
n
2
G - ~ n _ 1 ~ j _ l(ifil _ ifj12)
2
~ mi.,~
Constraints Force Equilibrium Moment Equilibrium Contact
E n - l(fni + f t i ) - Fe - 0
En-
1 ri(fni + ft i) - M e - 0 fn i n i < 0
Friction Cone
IQil 2 - ~21f7~i12 < 0
Stability
I E n = l n i l <_ S
Separation
Iri- r j l - e m i n >-- 0
/~1--""IX~I Rectangular Solids
i# j
>
0
>
0
al
~-IX• a2
Table 1" Optimization of the Finger Forces and Grasp Points
Optimization Criterion 9Similar to [9], the optimization criterion is so chosen that all the fingers exert the same magnitude of force. This criterion is used in order to ensule an optimal distribution so that no one finger assumes too large a share of the external load. By minimizing the difference in tlle magnitude of finger forces, the optimization criterion divides the external wrench evenly over all the fingers and subdues the tendency for one of the fingers to become overloaded. Force a n d M o m e n t E q u i l i b r i a 9 In order to assure that the fingers exert tile, desired external wrench, the force and moment equilibria are used as constraints for the above optimization. Here fn i and fti describe the normal and tangential co~nponents respectively of the the force for the i-th finger. Fr and Me are the desired external force and moment that the fingers should exert on the object, r i describes the vector from t]le center of gravity of the object to the grasp point of the i-th finger, and n i describes the vector normal to the object's surface at that i-th grasp point and always points outwards. These conditions ensure that the finger forces exert the desired force and t noment on the object. Although these equations describe static equilibrium, the dynamics of the object can be taken into account. (lurillg the path planning stage, and thus included in Fe an,:l Me.
Contact w i t h F r i c t i o n 9The contact, of the fingers on the object plays a.n obviously important role when grasping and manipulating a.n object. Contact wit.h friction adds
264 two constraints to the optimization problem: force direction and the friction cone. The decomposition of the finger force into it's normal and tangential components f~Lcilitates the understanding of the role of friction during grasping. Tile directional condition is an obvious one. It assures that the fingers remain in contact with the obj('.ct. T'he nornlal component of the finger force must always point inwards, towards the object. The second condition ensures that the finger forces remain within the friction cone so that the fingers do not slip on the object. The tangential component of the finger force should not exceed the m a x i m u m transferable tangential force, determined by #lfT~,il , where t I is the static coefficient of friciton. G r a s p S t a b i l i t y 9 The previous conditions ensure that the fingers exert the desired external wrench on the object., and that they do not slip on the object while doing so, but do not in any way control the stability of the grasp. Although there have been many different technical definitions of grasp stability [3], the definitions used most frequently are that of force closure and form closure (also known as complete restraint). Force closure arises when disturbance forces on the object act to maintain the contact. A grasp has form closure when it is able to resist any arbitrary disturbances. For example, if the wrench suddenly changes magnitude or direction, the grasp must be able to compensate for these changes so that the object does not fall out of the hand. A grasp with complete restraint is thus one which can exert any external force on the object. A fully stable grasp can then be considered as a grasp with form closure, and a stable grasp as one which is capable of resisting certain, but not all disturbances (force closure). In order to take this into account, a stability condition was incorporated into the optimization problem so that the grasp point can be chosen with varying degrees of stability. If one is very certain that the forces or moments will not change direction greatly, a less stable grasp may very well be the better grasp. A grasp can be chosen with force closure here, so that the small dist.urbances would tend to maintain the contact. The fingers can be located more efficiently on the object with a larger share of the finger forces being used to exert the desired wrench, and a smaller share being used t.o maintain the ability to compensate for non-exist.ant disturbances. The vectors normal to the object's surface at the grasp points provide a good insight into the stability of the grip: the smaller the sum of the vectors, the more stable the grasp. The grasp is less stable in the direction opposite the resulting sum, which lneans that it is less capable of resisting disturbances in t llat direction. However, the grasp can still be used in many force closure situalions. A sum of zero implies equal stability in all directions. Assuming the three points do not. lie on a line, a grasp with a sutn equal to zero has forl~l closure. The internal force with all fingers pressing eqaully strong corresponds exactly to the normal forces at tlle fingertips, wllich means that any tangential force necessary to overcome disturbances, can be brougllt back into the friction cone 1)3, increasing t.he internal forces. This stability condition can be described as follows: ..
I ~ nil <- S i-1
(5)
where S is the desired stability measure. For example, a sphere is grasped most stably with three fingers when the sum of the normal vectors is zero. With such a grasp all
265
external wrenches can be acllieved by adjusting the normal finger forces as necessary to keep the finger forces within the friction cone. As shown in Figure 3, when grasping a cube with three fingers however, it is impossible to grasp it in such a way that the sum is zero. In this case S - 1 describes the most stable grasp. With S - 1 it. is possible to have a grasp which can exert all external forces, but it does have a direction in which it is more difficult to resist disturbances, namely that in the direction opposite the sum of the vectors. On the other hand, disturbances which act in the direction of the sum tend to maintain contact.
Top View
Figure 3: Achievable S for a Sphere and a Cube
Side View
Figure 4: Achievable S for a Pyrami
This effect can be most easily seen on a pyramid (see Figure 4). It is impossible to grasp a p y r a m i d with three fingers in such a way that S = 0. In this example the sum of the vectors points towards the untouched face. If a disturbance force were to act in this direction, it would push the object towards the fingers and thus maintain the grasp. If a force were to act in the opposite direction however, it would be at best very difficult and under normal frictional conditions impossible to compensate for the disturbance. S e p a r a t i o n 9 This condition guarantees that a m i n i m u m separation e,,~i,~ is mainrained between the grasp points, so that the fingers do not come too close to one another. R e c t a n g u l a r S o l i d s 9 For rectangular solids, the conditions insure that the grasp points are not located too near the edges, where a small position error could lead to very large errors in the grasp. O111v the inner ~ fraction of the face can be used. Unknown Parameters 9 fn i, fti and r i are the unknown parameters which must be optimized. However, wilh a m a t h e m a t i c a l description of the object's surface, these vectors can be described as f,,nctions of the scalars fni, k li, k2i, (7,i, a.nd ~i"
f,,: -
f~i+f~i
= f n i n i + f t l i e t l i + ft2iet2 i ri - f((,i,#i) (Section 2.1) n i , e t l i , e t 2 i - f((,i,(i) (Section 2.1) Thus we have 5 , n parameters to optimize, where n is the n u m b e r of fingers grasping the object. The optimization is carried out using a, 'Successive Quadratic P r o g r a m m i n g ' procedure with a, core routine from the I~ISL Software Library.
266
3.0.2
Results
As an example, a sphere (radius 1 crn) is held against the forces of gravity (the hand should exert a net force of 1 N in the z direction and no m o m e n t s ) . A coefficient of friction # - 0.5 is assumed. In order to d e m o n s t r a t e the effects of the stability measure S on the optimization, two cases were performed, one with S - 0, and one with S _< 3. Tables 2 and 3 show the results of the optimizations. In each table, the first block of d a t a shows the values of the unknown p a r a m e t e r s delivered by the opt.imization. The second block shows the m a g n i t u d e of the calculated finger forces and the coefficient of fi'iction necessary to hold the object with the calculated grasp. The third block shows the grasp points and the forces and m o m e n t s (,hat each finger exerts on the object in Cartesian coordinates. As can be seen in both tables, the optimization provides finger forces which exert the desired wrench e x t r e m e l y well. Additionally, the tables show that no one finger assumes too large a share of the total force, nor does any finger slip (#n~ < #). 1] Finger 1 Finger 2 Finger 3 Sum 2.095 0.000 -2.09~ (red) -0.01J - 0 . 0 1 0 -0.011 .In (N) -1.01~ - 1 . 0 1 8 -1.01~ ftl (N) 0.000 0.000 0.000 ft2 (N) 0.323 0.323 0.323 Ifl (N) 1.0681.0681.068 ~nec [I 0.317 I 0.317 I 0.317 I [ z (m) -0.010 0.020 -0.010 y(m) 0.017 0.000 -0.017 (rad)
z (~)
o.ooo
o.ooo
o.ooo
fx (N) fy (N) fz (N) mx (Nm) my (Nm) mz (Nm)
0.507 -0.878 0.333 0.006 0.003 0.000
-1.014 0.000 0.333 0.000 -0.006 0.000
0..507 0.878 0.333 -0.006 0.003 0.000
~ (tad) ~ (red) fn (N) ftl (N) ft2 (N)
Sum
Ifl (N)
#nec x (m)
y(m)
-0.008 0.000
z (~)
0.000 Ix (N) 0.000 fy (N) 1.000 fz (N) 0.000 mx (Nm) 0.000 my (Nm) 0.000 mz (Nm)
Table 2: Sphere held against gravity (S - 0)
[I Finger 1 Finger 2 Finger 3 0.000 2.095 -2.094 -2.56( - 2 . 5 6 0 -2.560 -0.36~ - 0 . 3 6 8 -0.368 0.000 0.000 0.000 -0.15~ - 0 . 1 5 7 -0.157
0.004 -0.007
0.004 0.007
-o.oo6
-o.oo6
-o.oo6
0.220 0.000 0.333 0.000 0.002 0,000
-0.110 0.191 0.333 -0.001 -0.001 0,000
-0.110 -0.191 0.333 0.001 -0.001 0,000
0.000 0.000 1.000
0.000 0.000 0,000
Table 3: Sphere held against gravity (S _< 3)
In order to b e t t e r visualize the effects of the stability measure, the results of the optimizations are shown in Figures 5 and 6. It is i m p o r t a n t to r e m e m b e r that the m e t h o d not only calculates the finger forces, but also the points where the object should be grasped. As can been seen, tile grasp in Figure 6 is less stable than that in Figure 5. The less stable grasp relies on the fact that the gravitational force holds the object in the hand and thus a larger percentage of the finger force can be used to generate the external force. The grasp points shifted towards the b o t t o m of the sphere, which allows the finger forces to point more in the direction of the desired external force. This makes the grasp more effecient, but if a disturbance would suddenly push the object upwards (opposite the direction of the sum of the normals), the grasp would be ill-conditioned to c o m p e n s a t e for this change and the object would most likely fall out of the hand. The grasp in Figure 5 on the other hand can hold the sphere in the presence of adverse disturbances and thus is fully stable.
267
Figure 5" Sphere held against gravity
Figure 6:
(s-o)
(s<3)
Sphere held against gravity
Depending on the knowledge of the environment and possible disturbances, an appropriate stability measure S can be chosen to best accomplish a desired task.
4
Hand
Placement
In order to automate the grasping process, a strategy which can orient and locate the hand in such a manner that. all three fingers can reach their designated grasp points is needed. The object has six degrees of freedom relative to the hand wlfich have to be limited in such a way that the grasp points are reachable. The method presented in this paper has four parts: the definition of the grasp-triangle, a rough hand orientation, the finger assignment, and, finally, an optimization of the hand orientation and distance to the object.
4.1
Definition of the Grasp-triangle
The three grasp points, which can be obtained from the method described in Section 3, always form a triangle (assuming a relatively stable grasp so that the points do not form a line) and thus define a plane. The points Q and R are defined as the points which form the shortest side of the triangle, while E is the remaining point. The center of the grasp G is defined as the point halfway between point E and the midpoint of the segment QR. The grasp coordinate system can then be defined as follows" x: points towards point E, lies in the grasp plane z: normal to the grasp plane, found by the cross product between x and the vector from E to (2. y: lies in the grasp plain, found by the cross-product between z and x. The grasp-triangle and the corresponding grasp coordinates are thus fully defined.
4.2
Rough Hand Orientation
A rough hand orientation then follows, in which different degrees of freedom of the object relative to the hand are eliminated. The palm of the hand is oriented, so that it is
268 parallel to the grasp plane and its center lies on the z-axis of the grasp coordinate systen:. Whether the hand should be located on the positive (case 1) or on the negative (case 2) z-axis depends on the desired wrench. The TUM hydraulic hand. similar to the hun:an hand, has the property that, when grasping an object, the fingers close towards the palm. The fingers thus usually exert, forces towards and not away from the pahn when grasping. The robotic hand should thus be located such that the external force, whicl: the fingers should exert on the object, points towards the palm. With this rough hand orientatiol/, four of the six degrees of freedom (two translational, and two rotational) are eliminate,l. Only two are left" the distance o, between the object and the hand, and the orientation :3 of the grasp plane with respect to the plane of the palm (both are parallel, but the object. can be rotated relative to the hand about the z-axis). Figure 7 shows the rough orientation of the hand and the grasp plane. The grasp plane formed by the points QRE is parallel to the palm of the hand with a separation c~. The grasp plane can be rotated about its z-axis where/3 represents the angle between the x axis of the palm and the y axis of the grasp plane.
Figure 7: Rougll Orientation of the Hand
4.3
Finger Assignment
After the rough hand orientation, each finger is designated a grasp point. The current TUM-Hand has four fingers, where three fingers are located opposite a.nother which acts as a thumb. During the assignment of the fingers, this geometry is used to find a finger for each grasp point. The two points Q and R, which form the shortest side of tl:e grasp-triangle, are assigned to the two outer fingers shown in Figure 7. Of course, ttte assignment of each finger depends on whether the hand is on the positive (Case 1) or t.he negative (Case 2) z-axis. Tlw last point, point E is then assigned to the thumb. Tl~e fourth finger remains free for regrasping.
4.4
Optimization of the Hand Orientation and Distance
Now that each finger has been assigned a. grasp point, the optimization of the hm:d orientation /3 and distance (~. can proceed. In this optimization, the hand orientation
269 and distance to the object are varied and optimized under consideration of two criteria. The first and most important criterion is obviously that all three fingers can reach their assigned grasp points without penetrating the surface of the object. If any finger cannot reach its designated point, the hand orientation and distance combination is considered invalid. Every valid combination is assigned a penalty value, depending on the finger angles and the direction of the object's velocity. The combination with the smallest penalty is then the optimal combination. The penalty function r is described by the following equation
- ~ lciD ~
(6)
i where the ki represents the penalty factor for the i-th finger angle, and the Di is a function of the i-th finger segment angle "/i and its velocity as shown below.
{ ~i-T1 i Di-
" :'/i > 0
7u i - T i l ("Yu i 17i - -~
" ;/i < 0 --
(7)
71 i )1 " "5~i - 0
7u i and 71 i are the maximum and minimum angle that the i-th finger segment can achieve. This function ensures that each finger has the maximum range of movement. If an angle is increasing, for instance, the optimum starting position is the smallest angle, so that the finger has a larger range motion before the maximum angle is obtained. If there is no motion, the optimum angle is chosen to be in the middle of the allowable range. For example, for an object trajectory in which a finger must move from right to left, the optimal combination is that combination in which the finger starts farthest to the left, thus allowing greater finger motion before the finger reaches its kinematic limits.
4.5
Results
The hand placement method can be coupled with the grasp calculation strategies described earlier. First, the grasp calculation strategy is used to determine the finger forces and the points at which the finger grasp the object. The calculated finger location can then be used as input for the hand placement method, in order to orient and locate the hand properly. The resulting system is one in which an object of defined geometry can be grasped 1)37 the TUM-Hand in such a way that the finger forces and grasp points achieve the desired external wrench, and that all three fingers reach the grasp points with a maximum range of motion for the desired trajectory. In order to demonstrate l llis procedllre, a. cylinder with a radius of 1 cm and a height of 1 cm is grasped and turned about its longitudinal axis in the abscence of gravity. It is assumed that the cylinder represents a mating part which needs to be screwed onto another piece. Thus, the hand must grasp the cylinder and exert, a moment of 0.1 N'm about the z-axis. A friction coefficient t l - 0.80 was assumed for this example. Table 4 displays the results of the grasp point and finger forces optimization. Using this data, the hand orientation was then carried through and found an optimal separation c ~ - 82 m'n~, and orientation ,3 - -45.0 ~ Figure 8 shows the cylinder being grasped by t.he TUM
270 Hand. The maximization of the range of motion can be clearly seen. The manipulation represents a turning of the cvlinder in a clockwise motion. As can be seen, the fingers grasp the object in such a way that a large clockwise motion is possible, before the fingers reach their limits.
~(rad) ~(mm) I~(N) ftl(N) ft2(N)
Finger 1 0.000 0.000 4.444 3.333 0.000
Finger 2 Finger 3 2 095 -2.095 0 000 0.000 - 4 . 4 4 4 -4.444 3 333 3.333 0 000 0.000
Sum
Ifl (N)
Pnec f~(N) fy(N) fz(N) mx(Nm) my(Nm) mz(Nm)
9
.75
-4.445 3.332
-( 1.663 O6
oooo
0.000 0.000 0.033
0.000 0.000 0.033
.
5.109 2.183
0.000 0.000
oooo
oooo
0.000 0.000 0.033
0.000 0.000 0.100
Figure 8: Cylinder being turned in the abscence of gravity (S = 0)
Table 4: Zylinder being turned in the abscence of gravity (S = 0)
5
Regrasping the Object
In order to perform more complicated manipulations, it is often necessary to change the grasp of the hand on the object. The method which is most often used by humans is the regrasping of the object within the hand. Typical examples are the turning of a pen or a ball within the hand. During the regrasping phase of the manipulation, a free finger makes contact with the object at a new point and thus establishes a new grip. One of the other fingers is now fi'ee to release the object and can in turn be used to establish yet another grip on the object. \Vith this regrasping, the grip of the hand on the object can be changed without having to release tlle object. The object remains firmly held in the hand during the entire manipulation. Of course it is necessary to remember that three fingers are needed to form a stal)le grasp. Thus in order to perform a manipulation with regrasping, a hand with at least four fingers is required. The fourth finger of the TUM Hand is located directly opposite the thumb between the other two fingers. Although the regrasping of an object within the hand plays such an important role in human manipulations, it has not received much attention in robotic manipulations. A short formulation of the regrasping problem is presented in [5]. However methods which propose a corresponding solution to the problem are lacking. In order to make robotic regrasping possible, a strategy is presenled here, which not only calculates the finger forces necessary to generate the desired external wrench, but also finds a suitable grasp point at which the new finger should grasp the object.
271
5.1
The
Optimization
Problem
During the observation of human regrasping, it becomes clear that the regrasping represents the change from one grip to another, in which the existing contact points of the fingers remain unchanged while a new contact point is established by the free finger. Only after this contact is established can another finger release its grasp on the object. The calculation of the finger forces and the new contact point can be described as an optimization problem with (3n) + 2 unknowns" the 3~7 finger force components .fl~i, f t l i , ft2i and the two pa.rameters for the position of the new grasp point ~'u and Cu,. The new finger forces have to sa.stisfy the equilibrium, contact, and friction conditions so that the fingers exert the desired external force on the object without slipping. I~ addition the grasp point for the new finger has to satisfy the reachability, penetration and collision avoidance conditions. As in the previous optimization problems, the grasp is considered optimum when the difference between the magnitude of the finger forces reaches a minimum. Table 5 summarizes the optimization problem.
Optimization Necessary
Criterion
G : E n_
1 E j _n
l(Ifil 2 _ Ifjl2) ~
~ ,nt.i n.
Conditions
Force Equilibrium Momemt Equilibrium Contact Friction Cone Grasp Point
z? _ ~' -1
l(fni + fti)
Fe - 0
Fi (fni + fti ) - Me - 0
fn i . n i < 0 Ifti[ 2 - #21fr~,.i12 < 0 Reachability Penetration Collision Avoidance
Rectangular Solid
E1--''IDI al
~
~-IX~l a2
> o
0
Tal)le 5: Optimization with Regrasping The first four conditions and the edge avoidance condition for rectangular solids are the same as those discussed in Section 3. The last three conditions are geometrical ones, which means that not only t lie desired external wrench and the object geometry have to be known, but also the the elltire geometry of the hand. This includes the geometry of the fingers, the arrangement of the fingers on the palm and the position and orient.ation of the object within the hand. Because this geometry is different for each hand, only general principles which can be applied to every hand will be discussed here. (For a description
272 of the TUM Hand which is modelled in the example please see [7]). I"or the following discussion, it is assumed that. the geometrical values listed in Table 6 are known.
Hand geometry
Object Orientation
HrHOi
Position of the i-th finger on the pahn
R HOi
Orientation of the i-th finger on the palm
IIrHK
Position of the object in the hand syst.enl
RHK
Orientation of the object relative to the hand
Table 6: General geometrical values R e a c h a b i l i t y 9 If the regrasping is t.o be successful, the free finger lnust be abh: to reach the new grasp point. This reachability must be continually checked during tlle search for a suitable grasp point r K u for the free finger u. \Vith the position the grasp point, the position of the fingertip whic]~ would be necessary to establish contact at tlLis point can be calculated as follows:
ourOsu
-
R-1 H O u ( R H K i(ri(u + HrIII(
HrHOu)
(S)
With the inverse finger kinematics it is then possible to determine if the finger can reacll this grasp point. If the point cannot be reached it is unsuitable for regrasping. P e n e t r a t i o n 9 The free finger cannot penetrate into the object in order to reach t]ae new contact point. With the position of the fingertip calculated in equation (8) and the finger angles obtained through the inverse finger kinematics, the position of critical poil~ts along the finger ourOcrit can be calculated in the object's coordinate system as follows:
KrKcrit - R ~ K ( HrHOu - I i r H i ( + RHO u OurOcrit)
(9)
Now it can be checked if any of the critical points lie within the object's surface. If a point lies within the surface, the point is unsuitable for regrasping, because the finger has to penetrate the object in order to reacll it. C o l l i s i o n A v o i d a n c e 9 If t.he finger can reach the grasp point withotlt penetrating the object, it still remains to 1)e checked if a collision danger exists, because a successful regrasping is only possible if t lie fingers do not disturb one another. The position of the new grasp point has to be chosen in suc]l a way that it does not lead to collisions. First the separation between the new grasp point and all existing grasp points is checked, il~ order to be sure that small errors in the finger positions do not lead to collisions. If the minimum separation is g~laranteed, the finger cross-over has to be checked. For tl~is purpose a method which uses the projection of the fingers on the palm was developed. If the projection of the new finger crosses over, or comes too close to the projection of another finger on the palm, the danger of a collision is very high a.nd the gra.sp point is deemed unsuitable. Using the following equation, the position of the each grasp point found in equation (8) can be transformed into the hand coordinate system:
HrOsi - RHO i oiros i
(1 O)
273 By considering the projection of this vector onto the plane of the pahn, the collision danger can be determined. If lhe fingers must cross one another, the chance of collision is too high, and the point is considered unsuitable for regrasping. S o l u t i o n 9 Because this optimization problem is of the same form as the grasp optilnization problem, the same "Successive Quadratic Programming" inethod can be used to solve it. However, in this case, the success of the optimization depends strongly on the initial point. In order to a cl~ieve a good convergence, a suitable starting point, lnust be found before the optimization. The reacl~able point which achieves the most stable grasp (defined after the stability measure from section 3.0.1) is used as the starting point for the optimization. As before, when grasping a. rectangular object, the gra.sp points must remain within within the allowed axislength ~, so that they do not come too close to the edge. This optimization delivers not only the finger forces for the new" grasp, but also a suitable new grasping point.
5.2
Results
In order to demonstrate the effectivity of this method, an example is presented in which the hand holds an ellispoid (a:r = 13m~)~, a 9 - 16ram and az = 25ram) against gravity and rotates it about the verlical axis. The hand has to exert an external vertical force of 1.4 N in the z-direction and a moment of-0.03 N m about this axis. A coeflizient of friction of tt = 0.55 is assumed. The orientation and position of the object within tl~e hand are also known. The h)llrih finger should grasp the object and replace the second finger. Table 7 shows the resulis of the optimization of the finger forces before the optimization while Table 8 shows the res~llts of lhe regrapsing optimization. The first data. group in the tables shows the the rallies for the unknowns which the ol)timization delivers. During the optimization of the fillger forces 1)efore the regrasping the geometric parameters for the grasp points (4i and (i) are given. During the regrasping optimization these two parameters for the fourth finger are unknown and must be determined by the optimization. The finger forces for the fingers wllich grasp the object are also determined by ~he optimization. The other data groups show values which are calculated fronl the optimized values. As both tables show, the fourth finger is not used before the regrasping. During the regrasping phase, the follrth finger is used to replace the second finger's grasp on the object. The hand uses a new grasp point (and a new finger) to exert the external force on the object. The optimization computes finger forces which accurately achieve the desired external force. One can also see that no finger assumes too large a share of the total load, and that all the finger forces remain within the friction cone (g~ec < It). During the regrasping, the load is nol distrilmted as well before. Because the free finger has to be able to reach the new grasl) poini wiillout penetrating the object or disturbing the other fingers, the number of 1)ossil)le grasp points is reduced. This reduces the number of possible finger force coml)illalions wllicll in turn leads to a less idea.1 distribution of the total load. Figure 9 shows the reslllls for l llis example. The grasp used before the regrasping is shown at the top and the grasp afier lhc regrasping at the bottoln. The point during the
274
regrasping at which all four fingers briefly contact the object is shown in the middle. This figure shows clearly t h a t the reachability, p e n e t r a t i o n , and collision avoidance criterion are all fulfilled. T h e new grasp point is so chosen t h a t the fourth finger replaces the second finger w i t h o u t p e n e t r a t i n g into the object or d i s t u r b i n g the other fingers.
(tad) (tad) fn (N) ftl (N) ft2 (N) ,fl(N)
#nec x (m)
y(m) z (m) fx (N)
fy (N) fz (N) mx (Nm) my (Nm) mz (Nrn)
II Finger 1 I Finger 2 I Finger 3 I Finger 4 I Sum I -2.094 0.000 2.094 0.000 0.000 0.000 -2.677 -2.858 -2.950 -1.245 -0.694 -0.121 0.443 0.513 0.443 [I 2.985 2.985 2.985 __ [1 0.494 0 302 0.156 -0.007 0.013 -0.007 -0.013 0.000 0.013 0.000 0.000 0.000 0.825 -2.858 2.033 0.000 2.835 -0.694 -2.141 0.000 0.4,13 0.514 0.443 1.400 -0.006 0.000 0.006 0.000 0.003 -0.007 0.004 0.000 -0.011 -0.009 -0.010 -0.030
Table 7: Finger Force O p t i m i z a t i o n : Before R e g r a s p i n g
1[ Finger 1 I Finger 2 I Finger 3 I Finger 4 I Sum ] -2.094 2.094 -0.593 (tad) 0.000 0.000 0.628 fn (N) -4.367 -4.632 -4.497 fti (N) -0.918 0.958 -1.603 ft2 (N) 1.957 1.172 -0.977 If, (N)11 ,4.873 __ 4.873 4.837 .nec 0.495 0.327 0 417 x (m) -0.007 -0.007 0.011 y(m) -0.013 0.013 -0.007 z (m) 0.000 0.000 0.009 :Ix (N) 2.184 2.328 -4.512 0.000 fy (N) 3.891 -4.117 0.226 0.000 fz (N) 1.955 1.170 -1.725 1.400 mx (Nm) -0.025 0.015 0.010 0.000 my (Nrn) 0.015 0.009 -0.024 0.000 mz (Nm.) -0.001 0.001 -0.030 -0.030 (tad)
II
Tal)le 8: O p t i m i z a t i o n with R e g r a s p i n g
275
Figure 9: R egrasping an Ellipsoid
6
Conclusion
This paper provided a. brief look into (llree aspects of tile high-level grasp stra.tegi(::s developed at the Leh'rst, l~l B fl}'r Mechaoik which represent the first steps towards the goal of complete grasp automation" calclllation of the finger forces and grasp points, the placing of the object wit.l~in (lie hand, and the regrasping of the object withirl the hall,:l. If an object is be manil~lllated effectively, both the finger position and the finger forces necessary to exert the desired force oll the object must be deternained. The strategy presented not only calculates the finger forces necessary to manipulate the object, but also provide points of variable stability wllere the fingers should grasp the object. Example grasps were shown and discllssed. Once the finger forces and grasp points are known, the hand must be so positioned and oriented so that the fingers can reach their assigned grasp points. A method was presented which places the ol>.iect so wi(llin the hand that all grasp points are reachable and the fingers have the maximuna range of movement to complete the desired task. Many manipulations require that the hand change its grasp on the object. A naet,hod
276 was presented which plans the regrasping of the object within the hand. A free finger grasps the object at a new point and thus establishes a new grip. Another finger can break its contact with the object is now free to perform further regrasping. During the entire regrasping process, the object is held firmly within the hand.
Acknowledgement Funding for this work was provided by the German Ministry for Research a.nd Technology (BMFT) through the SEKON Project (Grant" 01 IN 104 D/7).
References [1] Biggers K.B., S.C. Jacobsen and G.E. Gerpheide. "Low-Level Control of the Utah/MIT Dextrous Hand". Proc. of IEEE Int. Co@ o1~ Robotics and Automation, April 1987. [2] Chen Y.-C. and I.D. Walker. "An Analysis of Squeezing and Twisting for Multifingered Grasping". Proc. of IEEE [,t. Conj'. on Robotics and Automation, Atlanta, Georgia, May 1993. [3] Kerr J. and B. Roth. "Analysis of Multifingered Hands". Int. Jotlrnal of Robotics Research, 4(4):3-17, Winter 1986. [4] Kerr J. and B. Roth. "Special Grasping Configurations with Dextrous Itands". Pl'oc. of I E E E Int. Conf. o,n Robotics and Automation, April 1986. [5] Li Z., J.F. Canny and S.S. Sastry. "On Motion Planning for Dexterous Maniptllation, Part I: The Prol)lem Formulalion". Proc. of IEEE Int. Conf. o l~ Robotics a l~d Automation, May 1989. [6] Li Z., P. Hsu and S.S. Sastry. "Grasping and Coordina.ted Manipulation by a Multifingered Robot Hand". l,t. Jovr, al of Robotics Research, 8(4)'33--50, 1989. [7] Menzel R., K. \Voelfl and F. Pfeiffer.
"The Development ot'a ttydraulic tland".
Proc. of the 2nd Co o.[cre,ce o, ilhch.atronics and Robotics, pages 225-238, Duis-
burg/Moers, Germany. September 1993. [8] Omata T. "Fingertip positions of a multifingered Hand". Proc. of 1LgEE Int. Co@ on Robotics and Automation, May 1990. [9] Park Y.C. G raspiT~.g and 3la,ipulotio., of an Object usi~g a M ult~ifingcrcd Robot Hand. PhD thesis, Universily of New Xlexico, May 1990. [10] Salisbury J.K. I(iT~ematic a'od Force Analysis of Articulated Ha'rids. PhD thesis, Stanford University, Nlay 1982.
277 [11] Woelfl K. and E. Pfeiffer. "Grasp Strategies for a Dextrous Robotic Hand". P roc. of I E E E / R S J Int. Co,,f. 0o h~tellige,t Robots and Sgstems, Munich, Germany. September 1994. [12] Yoshikawa T. and K. Nagai. "Manipulating and Grasping Forces in Manipulation by Multi-Fingered Hands". Proc. of Intl. Conference o lz Robotics a l~,d A'tttomation,, pages 1998-2004, Raleigll, March 1987. [13] Yoshikawa T. and K. Nagai. "Manipulating and grasping forces in manipulation by multifingered robot hands". I E E E Journal of Robotics a l~d A~ttomatiolz, 7(1):67-77, February 1991.
This Page Intentionally Left Blank
Intelligent Robots and Systems
V. Graefe (Editor) 9 1995 Elsevier Science B.V. All rights reserved.
279
Motion Scheme for Dextrous Manipulation in the Intelligent Cooperative Manipulation System--ICMS M. Buss ~ and H. Hashimoto b ~Department of Automatic Control Engineering, Technical University of Munich, Theresienstrasse 90, D-80333 Munich, Germany, E-mail: [email protected] bInstitute of Industrial Science, University of Tokyo, 7-22-1 Roppongi, Minato-ku, Tokyo 106, Japan, E-mail: H.Hashimoto~ieee.org This paper proposes a motion scheme for dextrous manipulation for the Intelligent Cooperative Manipulation System--ICMS. Key idea of this motion scheme is the hand manipulation skill model used for skill acquisition, transfer and manipulation assistance in the ICMS. The proposed skill mapping maps from desired object task trajectories to finger contact point locations and can therewith describe complex manipulation tasks, where contact states of the fingers may change during the task. Characteristics of this skill mapping are discussed and a scale transformation increases applicablity for a change in object scale. Based on the motion scheme for dextrous manipulation, a skill example of pen rotation by four fingers is acquired from the human operator, and the acquired skill is transferred to a four-fingered dextrous robotic hand with a different kinematic structure and size than the human hand. Replacing the complex input device attached to the human hand by a much simpler position and orientation sensor is the idea of the manipulation assistant also shown in this paper. The human supervisory operator inputs desired object position and orientation and the ICMS assists in that the references for the dextrous hand are generated by the proposed motion scheme. 1. I N T R O D U C T I O N The authors have been proposing the Intelligent Assisting System--IAS [1,2], and as a first step towards the IAS the Intelligent Cooperative Manipulation System--ICMS for human machine cooperative manipulation. The ICMS uses a 10 degrees-of-freedom force feedback sensor glove device connected to a virtual reality including a dynamic force simulator. The human operator can interact with objects in this artificial environment and data describing the manipulation task can be measured. This paper briefly shows the system structure of the ICMS and discusses hand manipulation skill modeling for acquisition and transfer in the motion scheme for dextrous manipulation. Dextrous manipulation using robotic hands is well studied. Salisbury has described some of the basic issues of interest [3], Kerr and Roth [4], Yoshikawa et al. [5,6] have proposed schemes to solve for internal grasping forces suspect to friction and other constraints, Li et al. [7] and Yoshikawa et al. [8] have developed control strategies. Other
280 problems of sensing and hardware realization of dextrous robotic hands are issues of ongoing research, see e.g. [9]. Preshaping of the hand to prepare for manipulation task execution can be observed in humans reaching for objects to be grasped. Power, precision, force closure and other ways of grasp prehension were observed [10-12]. One important and less understood issue is how to actually generate desired trajectories for dextrous manipulation tasks. Some results using primitive based hierarchical control strategies were shown by Speeter [13] and Stansfield [14]. This paper develops the idea of a manipulation skill mapping to achieve finger contact point locations for given object task trajectories. It is noted that this can be also used during the preshaping stage but the ideas in this paper focus mainly on time-dependent or task-dependent grasp transitions during task execution after an initial grasp has been accomplished. In Section 2 the Intelligent Cooperative Manipulation System-ICMS is briefly reviewed. Section 3 develops a motion scheme for dextrous manipulation, defines the manipulation skill mapping and shows the process from desired object task trajectory to contact wrench intensity trajectories. Applicability of the proposed skill mapping is discussed for object scale changes and a manipulation example of pen rotation is shown. Section 4 discusses how manipulation skill can be acquired, which then is transformed to an executing dextrous robotic hand in Section 5. Section 6 shows grasping assistance as an example of human operator assistance by the ICMS. 2. I N T E L L I G E N T
COOPERATIVE MANIPULATION SYSTEM--ICMS
This section briefly describes the ICMS consisting of a sensor glove device for force feedback to the human operator, and a virtual reality including a Dynamic Force Simulator (DFS) which calculates the feedback forces to the operator (see [15] for details about the DFS). Most important characteristics of this system are the 3 information and power (IP) flows also illustrated in Figure 1. I P Flow l m S k i l l Acquisition: The human operator wears a sensor glove device enabling interaction with objects in the virtual world while getting appropriate force feedback. Visual feedback information from the virtual world graphic animation module is utilized by the operator for task goal verification and overall environment perception. Measurements during manipulation by the human operator yield data for acquisition into the manipulation skill data base of the ICMS. I P Flow 2 - - S k i l l Transfer: The acquired skill now available in the skill data base is used for more sophisticated control of the executing dextrous robotic hand. I P Flow 3 - - A s s i s t i n g M a n i p u l a t i o n " The ICMS anticipates human motion and if appropriate skill is available in the data base, assists the human operator in executing the task. In this paper an example of grasping assistance is shown to demonstrate cooperative work by human operator and ICMS. The following section proposes a motion scheme for dextrous manipulation also modeling hand manipulation skill. Using this motion scheme the three IP flows of skill acquisition, transfer and assisting manipulation can be realized which is then shown in subsequent sections.
281
Figure 1. Intelligent Cooperative Manipulation System--ICMS.
3. M O T I O N
SCHEME
FOR DEXTROUS
MANIPULATION
Motion of an object in the environment can be described by its position and orientation vector as a function of time. This is common practice for many problems in the field of robotics. In case this motion is subject to physical laws like gravitational acceleration and moments of inertia differential equations of motion can be derived easily. Complexity of these equations increases for constrained environment conditions opposing or inducing object motion due to geometric or other constraints. One central objective is to make objects follow reference trajectories of position and orientation given by a global task planner or human teleoperator under varying environment conditions. The terms task and goal planning describe the operation of deriving such object reference trajectories, which is a rather unconscious undertaking for the human but a major effort of research in robotic systems. For redundant manipulators and dextrous robotic hands this problem has no simple straightforward solution since there is an infinite number of solutions due to redundancy. This chapter deals with a motion scheme for dextrous manipulation useful to derive a set of task trajectories for dextrous hands from desired object motion assumed to be given by a global task planner or in case of the ICMS by the human operator. After a task definition given in the next section, a model for the performed manipulation task is shown and its characteristics are discussed. 3.1. Task D e f i n i t i o n Figure 2 shows a hierarchical model for task planning and its execution. First there generally is an intention and motivation of what task to execute next, then the objects off the task objectives are located in the environment and a task trajectory is planned. Herei
282 various environment conditions can change the resulting task trajectory such as e.g. a requirement of avoiding obstacles in the direct way to the goal position of the task object or other relations between several objects.
Figure 2. Hierarchy of task and path planning during manipulation.
To realize a specific task in the environment is considered equivalent to moving the task objects along a desired path. The manipulation task can then be defined by this object trajectory in space and time. Definition 3.1 (Manipulation Task) A manipulation task of an object is described by its task trajectory, the position and orientation vector p(t) C IR6. Often this task trajectory is assumed to be given by a task and goal planner in the context of dextrous manipulation. For the ICMS it can also be given directly by the human operator interacting with the system. To show that Definition 3.1 is actually describing much more than only object position and orientation the following well known
283 relations to the velocity and force domain are given. The object velocity can be written in terms of p as v 0.)
- tI/(p) ib
(1)
where ~(p) C IR6x6 is an appropriate transformation matrix. Differentiation of (1) with respect to time yields linear and angular acceleration of the object
dJ
]-
+
(p)p
(2)
The position and orientation trajectory p(t) can be related to the force domain f(t) using the Newton-Euler equations of the object as
0
,7
iJ+
wx,Jw
-f-f~t+
0
'
(3)
where rh E IR3x3 is a diagonal matrix with the object mass m in the diagonal elements, J E IR3x3 is the object inertia tensor, f~t E IR6 is the generalized external force acting on the object and g E IR3 is the gravitational acceleration vector. It is noted here, that geometric and other environment constraints can be included in the external force f~t.
3.2. Definition of M o t i o n S k i l l - - M S K In the following motion skill is defined according to related results from cognitive science, see e.g. [16-19]. Other engineering approaches to human skill modeling or robot programming by human demonstration can be found in [20-23]. Definition 3.2 (Motion Skill--MSIQ Motion skill (MSIQ is defined as the ability to derive motor actuation commands from perceived environment states assuming a task goal known from intention and task planning. This definition of motion skill is based on and closely related to the skill-based human behavior proposed by Rasmussen [16], in that for most skilled sensory-motor tasks, the body acts as a multivariable continuous control system synchronizing movements with environment states, see Figure 3. In summary from several works of cognitive science one can conclude, that skill-based performance is the result of processing (physical) signals perceived as time-space signals, continuous quantitative indicators from sensors to control the body actuators. These signals have no "meaning" or significance except as direct physical time-space data. At this stage there is no symbolic processing like e.g. symbolic environment state perception or symbolic goal planning, which is used and processed in the rule-based behavior level [16].
3.3. Hand M a n i p u l a t i o n Skill M o d e l - - H M S K In this section hand manipulation skill (HMSK) is defined as the ability to decide finger-tip positions l(t) for grasping from object task trajectories p(t). It basically makes no difference for this model where the data comes from, i.e. finger-tip locations could be
284
Intention
Task goal
Environment ~
Perception / - ~
SKILLMOTIONMSK ,'~'----~ Action
Figure 3. Motion skill (MSK) model as sensory-motor behavior. calculated analytically as the solution of e.g. manipulability optimization or be learned from a human operator in case of the ICMS. As a more specific form of motion skill (MSK), the following model of hand manipulation skill (HMSK) for dextrous hands is defined. A s s u m p t i o n 3.1 The desired object trajectory pd(t) @ ]R 6 as of definition 3.1 is given by a manipulation task planner or a human teleoperator. A s s u m p t i o n 3.2 Object shape and center of mass are known. Definition 3.3 (Skill M a p p i n g - - S M ) pd(t )
SM
l(t),
The skill mapping S M is defined as
(4)
where pd(t) e IR 6 is the object reference trajectory, l(t) - [ l~ ... 1T ]T e IR 3n are the contact point positions of the n fingers used for manipulation, and li C IR 3 is a vector .from the object center of mass to the i th contact point.
Figure 4 shows the skill mapping and other essential mappings involved in dextrous manipulation tasks. The skill mapping 5'M gives the relation between the desired object trajectory pd(t) and finger tip contact point locations, which can be used to directly compute the grip transform W ( t ) assuming the object shape as known. Once the grip transform has been found by S M the second problem is to transfer this into reference trajectories for joint angles and torques in manipulating hand joint space. Solving inverse hand kinematics from finger tip trajectories yields trajectories in hand joint space realizing the desired finger-tip positions, however, the problem of appropriate internal grasping force trajectories remains. Optimization for appropriate internal forces (IFO) can be solved as a nonlinear or linear programming problem and a variety of approaches to solve the IFO can be found in the literature [24-26,4,27-33].
285 pa(t) object r trajectory
Object shape______~ class ~1 SM finger locations
]
l(t)
SM: Skill Mapping HIK: Hand Inverse Kinematics
Object shape ] grip transform Object shape ~ mass, friction ~ and contact model
W(t) +
I
IFO
].. HIK 0 ~(0 fingerjoint reference IFO:Internal Force Optimization
contact force ca(t), f~xJt(t), fifft (t) internal reference
Figure 4. Manipulation skill mapping SM. R e m a r k 3.1 Note that the skill mapping S M does not depend on the kinematic structure of the manipulating hand and therewith is quite general, but naturally the resulting contact point locations li need to be within the operable workspace of the corresponding finger i. R e m a r k 3.2 This skill mapping S M maps .from, the 6-dimensional object position and orientation space to the 3n-dimensional contact point space. The mapping is nontrivial, difficult to solve analytically and can include time variant contact conditions and also timely past and/or future steps during manipulation. 3.4. Skill M a p p i n g A p p l i c a b i l i t y The skill mapping of Definition 3.3 assumes exact knowledge of the object shape. This assumption can be relaxed by transforming the desired contact point 1 resulting from S M for differently scaled objects. Therewith the same skill mapping S M can be used for larger and smaller objects within the same shape class, which increases the applicability and robustness of the modeled manipulation skill. Definition 3.4 (Scale Transformation) To transfer the acquired contact point locations 1 to new locations l' for a differently scaled object a scaling matrix E E IR3• is defined as
l!i - E~,:,
(5)
where i = 1, ..., n designates finger i of n manipulating fingers.
R e m a r k 3.3 To increase the robustness during execution of the modeled manipulation skill this scale transform might be included in the control algorithm through an adaptive mechanism, which will be a topic of.future research.
286 3.5. Characteristics of SM S M looked upon as a static mapping from object position and orientation p solely defines how the object should be grasped in its workspace. It is considered compact, because
1. it is independent on hand kinematics (Remark 3.1), 2. of the scale transformation for object size variance (Definition 3.4), 3. it does not depend on a specific task trajectory in time or task execution speed, i.e. once S M is learned it is applicable for arbitrary task trajectories (see grasping assistance in Section 6, where an arbitrary trajectory is given by a human operator). 4. rolling or sliding contacts are implicitly modeled. Comparing finger-tip positions between two time-instants can be used to select constraints for the programming problem to solve for internal forces resulting in sliding or rolling contact, and further can be used to select an appropriate mode in the executing controller.
5. SM also can model preshaping strategies, i.e. before contact is achieved with the object finger-tip contact locations (virtual contact) describe the preshape. 6. it defines also impedance. Finger-tip positions depend on object position and orientation, and are realized by the executing controller. Reference to the controller are also the contact wrench intensities as the result of internal force optimization (IFO). Object impedance defined in the object reference coordinate system depends therewith on SM, IFO and dextrous hand controller gains [34]. A more general SM depends also on time 1 = SM(p, t) and has the following additional characteristics: 1. it may change the type of grasp drastically depending on execution speed. This can be understood by a simple example of a skill mapping SM(p, p) depending on task trajectory and speed intervals as
9
SMI(p)
SM(p, I5) =
SM2(p)
for p E [P0, Pl) for /5 E [/51,ib2)
(6)
SMn(p) for /i C [ibn-1,/i~) Nevertheless, the static S M also works for differing execution speeds, but does not change the grasp state and type drastically. 3.6. E x a m p l e This example shows how the grip transformation matrix W can be derived from finger contact point locations l = SM(p).
287 Consider the case of a pen shown in Figure 5 and a soft finger contact model with the following unit wrenches in the contact frame C = CI: CwT1,1 --
[ 1 0 0 0 0 0 ]
CwT
[010000] [001000] [000001]
CwT
This yields the grip transform W as these unit wrenches transformed into the coordinate frame B fixed at the center of mass of the object
W
=
BWl, 1 BWl, 2 BWl, 3 BWl,4 ] --
1 0 0 0 r --ll
0 1 0 -r 0 0
0 0 1 ll 0 0
0 0 0 0 0 1
Figure 5. Example of contact force for a pen-like object.
4. H M S K
ACQUISITION
Hand manipulation skill (HMSK) can be acquired according to Definition 3.3 by learning the skill mapping parameters. There are a number of ways this can be done: 1. Learning S M from a human operator: Using a sensory device attached to the hand of a human operator giving exact knowledge about finger contact locations and object position and orientation, data describing the skill mapping can be measured [1,35]. 2. Using a telerobotic system to implicitly get the required data for the slave robot the manipulation skill is to be implemented on later. Recently, methods for learning S M , in particular an artificial neural network and memory-based learning approach have been investigated [36].
288 Contact point locations of finger 1 (m)
Contact point locations of finger 2 (m)
0.04
oo3t
~176
I _o.o,I /
v
-~176
t
\
-0"04 r
o
so
loo
Phi (deg)
1so
200
-00%
so
16o
Phi (deg)
1;0
\
200
Figure 6. Approximated finger contact point locations as acquired from the human operator wearing a VPL data glove depending on object rotational angle ~b of the object. Joint angles of finger 2 (rad)
Joint angles of finger 1 (rad) 2 1.5 1 0.5 0 -0.5 -1 -1.5 -2
\ / 0
s'0
~oo
Phi (deg)
v
1;0
2o0
%
~o
16o Phi (deg)
1;o
2oo
Figure 7. Finger joint angles of the dextrous 24 degree-of-freedom robotic hand used for execution of pen rotation manipulation skill.
4.1. Acquisition Example For a simple manipulation example of rotating a pen using 4 fingers, experimental data from a human operator wearing a VPL dataglove were acquired. The measured data were filtered, interpolated, and then approximated by a 5 element fourier series for the /x-component of the contact point and linear approximation for the ly- and/z-components, which yields a rather compact representation of the acquired manipulation skill. Figure 6 shows contact point locations for two of the four fingers depending on the rotational angle 6 of the pen. The task was carried out with approximately pa = [ rx ry r~ 0 0 ~ ]T (as exactly the human operator can handle this requirement; errors for the two zero orientation angles where neglected), where r=, ry, rz is the location of the pen center of mass and [ 0 0 ~ ]T are the RPY-angles giving the orientation of the pen (radius rp - 0.004 m, length lp - 0.15 m).
289
Figure 8. Illustration of the manipulation task execution by the 24 degree-of-freedom robotic hand (r denotes the rotational angle of the object).
5. M S K T R A N S F E R
AND EXECUTION
Once the skill mapping S M has been acquired as described in Section 4 solving inverse kinematic equations of the robotic hand structure on which the manipulation task is to be executed yields trajectories for the robotic hand in hand joint space. Further the contact wrench intensities c(t), fint(t) as the solution of the internal grasping force programming problem (IFO) need to be realized by the control law for the robotic hand. Such control algorithms are proposed in the literature [2,37,8]. 5.1. Transfer E x a m p l e The acquired manipulation task as shown in the example in section 4.1 can be transferred for execution by a robotic hand. Assuming a kinematic structure of a 4-fingered 24 degrees-of-freedom hand as developed by Namima et al. [38], which is quite larger in scale than the human hand, the acquired manipulation skill must be transformed to this
290 larger scale. The pen to be handled by the robotic hand is assumed to have a radius of rp - 0.02 m and a length of lp - 0.5 m. To transfer the acquired contact point locations the scale transformation of definition 3.4 is used, where !
!
!
!
i
lp --,rP _rp ) - d i a g ( = diag( ~ , rp rp
3.33, 5, 5 ) ,
(7)
assuming the x-axis of the pen center of mass coordinate frame is in the direction of the length. If a soft finger contact model with 4 degrees-of-freedom is assumed between finger-tip and object the grip transformation matrix W follows immediately as 0
1
0
0 ...
-sign(lj,y) 0
0 0
0 1
0 ... 0 ...
0
0
lj,,,
0 ...
0 -sign(lj,y)lj,x
0 -lj,y
-lj,x 0
0 ... 1 ...
W-
'
(8)
where sign(x) = - 1 for x < 0, sign(x) = 1 for x > 0, lj,x, lj,y are the x- and y-components of lj, respectively, and j are the fingers is contact with the object. Then the programming problem (IFO) is solved yielding wrench and internal force intensities subject to friction and other constraints. The resulting joint angle trajectories in hand joint space depending on object rotational angle r are shown in Figure 7. Figure 8 visualizes how the manipulation skill of pen rotation acquired from the human is executed.
Object orientation reference (deg) 180 160 140
object posmon/ orientation
Z
.
Polhemus + 3D_Tracker~
~...._~_.1..-.--
x "--
.
p d(k) ---. y
contact state, grip transform
__~ S M ~---~ l(k) W(k) Skill
Mapping
120 100 80 60 40
~o/ t~
u(
1
2
3
4
5
6
time (s)
Figure 9. Grasping assistance using SM. Figure 10. Rotational angle of the object commanded by the human operator.
291 6. G R A S P I N G A S S I S T A N C E It is possible to use the acquired hand manipulation skill mapping SM for grasping assistance. Figure 9 shows how the object task trajectory is given by a human operator using a 3D tracking device (here a Polhemus tracker). Applying S'M yields contact point location references and finally after solving for internal grasping forces hand joint torque references. Figure 10 and Figure 11 show an experimental result of grasping assistance.
0.1[
Contact point locations of finger 2 (m)
Contact point locations of finger 1 (m)
0.15
0.25 0.2 0.15
0.05
0.1
0
0.05 0
-0.05 -0.05
-~
~
i
3
~
;
;
7
-ol
1
Joint angles of finger 1 (rad) 2
2
3
4
5
6
7
6
7
time (s) Joint angles of finger 2 (rad)
time (s) 4
1.5 1 0.5 0 -0.5
-2 time (s)
0
1
2
3 4 time (s)
5
Figure 11. Finger contact point locations and joint angles during grasping assistance.
7. C O N C L U S I O N S This paper discussed the Intelligent Cooperative Manipulation System--ICMS and the underlying motion scheme for dextrous manipulation. Using the proposed skill mapping as a hand manipulation skill model is our approach to skill acquisition from human operators, skill transfer to an executing dextrous robotic hand and manipulation assistance is realized as human-machine cooperation in the ICMS. Specifically the following issues were discussed:
292 9 Within the motion scheme for dextrous manipulation a discussion about motion skill in general and specifically for hand manipulation skill was given. 9 The manipulation skill mapping is the proposed model for hand manipulation skill, mapping between desired object task trajectory and finger contact points locations for changing contact states during manipulation. Some special characteristics of the skill mapping SM were discussed, in that SM is independent on hand kinematics, a scale transformation makes it adaptive to a change in object size, rolling or sliding contacts and impedance imposed on the object are implicitly modeled, SM can also model preshaping strategies, and a time dependent S M may change the grasp strategy drastically depending on execution speed. 9 Acquisition of hand manipulation skill mappings from human operators was discussed and shown in an example of pen rotation by four fingers. 9 The acquired manipulation skill example of pen rotation using four fingers was transferred to a 24 degree-of-freedom dextrous robotic hand of larger scale than the human hand and a larger scale object. 9 Grasping assistance was shown as an example of human--ICMS cooperative work, where the complex input device attached to the human hand during skill acquisition can be replaced by a simpler position and orientation command input device, and actual grasping and control of the dextrous hand is achieved by the proposed motion scheme. Future work will include robustness issues for manipulation skill transfer to cope with object model errors and disturbances from the environment. REFERENCES
1. M. Buss and H. Hashimoto, "Skill acquisition and transfer system as approach to the intelligent assisting system--ias," in Proceedings of the 2nd IEEE Conference on Control Applications, (Vancouver, British Columbia, Canada), pp. 451-456, 1993. 2. M. Buss and H. Hashimoto, "Human manipulative skill analysis and acquisition for the intelligent assisting system," in Proceedings 12th IFA C World Congress, International Federation of Automatic Control, vol. 9, (Sydney, Australia), pp. 321-326, 1993. 3. M. Mason and J. Salisbury, Robot Hands and the Mechanics of Manipulation. Cambridge, Massachusetts: MIT Press, 1985. 4. J. Kerr and B. Roth, "Analysis of multifingered hands," International Journal of Robotics Research, vol. 4, pp. 3-17, Winter 1986. 5. Y. Nakamura, K. Nagai, and T. Yoshikawa, "Mechanics of coordinative manipulation by multiple robotic mechanisms," in Proceedings of the IEEE International Conference on Robotics and Automation, pp. 991-998, 1987. 6. T. Yoshikawa, "Dynamic hybrid position/force control of robot manipulators- description of hand constraints and calculation of joint driving force," IEEE Journal of Robotics and Automation, vol. 3, pp. 386-392, October 1987. 7. Z. Li and S. Sastry, "Task-oriented optimal grasping by multifingered robot hands," IEEE Journal of Robotics and Automation, vol. 4, pp. 32-44, February 1988.
293
10.
11.
12.
13.
14.
15.
16.
17.
18.
19. 20. 21.
22.
23.
T. Yoshikawa and X. Zheng, "Coordinated dynamic hybrid position/force control for multiple robot manipulators handling one constrained object," International Journal of Robotics Research, vol. 12, pp. 219-230, June 1993. S. Horikoshi, H. Hashimoto, and F. Harashima, "Identification of object parameters with robot hand," in Proceedings of the International Conference on Industrial Electronics, Control, and Instrumentation IECON, (Hawaii), pp. 45-50, 1993. M. Cutkosky, "On grasp choice, grasp models, and the design of hands for manufacturing tasks," IEEE Transactions on Robotics and Automation, vol. 5, pp. 269-279, June 1989. T. Iberall, "The nature of human prehension: Three dextrous hands in one," in Proceedings of the IEEE International Conference on Robotics and Automation, pp. 396401, 1987. T. Nguyen and H. Stephanou, "A computational model of prehensility and its applications to dextrous manipulation," in Proceedings of the IEEE International Conference on Robotics and Automation, (Sacramento, California), pp. 878-883, 1991. T. Specter, "Primitive based control of the utah/mit dextrous hand," in Proceedings of the IEEE International Conference on Robotics and Automation, (Sacramento, California), pp. 866-877, 1991. S. Stansfield, "Primitives, features and exploratory features: Building a robot tactile perception system," in Proceedings of the IEEE International Conference on Robotics and Automation, pp. 1274-1279, 1986. H. Hashimoto, M. Buss, Y. Kunii, and F. Harashima, "Intelligent cooperative manipulation system using dynamic force simulator," in Proceedings of the IEEE International Conference on Robotics and Automation, (San Diego, California), pp. 25982603, 1994. J. Rasmussen, "Skills, rules, and knowledge; signals, signs, and symbols, and other distinctions in human performance models," IEEE Transactions on Systems, Man and Cybernetics, vol. 13, pp. 257-266, May/June 1983. R. Redding, "Taking cognitive task analysis into the field: Bridging the gap from research to application," in Proceedings of the Human Factors Society 3~th Annual Meeting, pp. 1304-1308, 1990. E. Roth and D. Woods, "Analyizing the cognitive demands of problem-solving environments: An approach to cognitive task analysis," in Proceedings of the Human Factors Society 3~th Annual Meeting, pp. 1314-1317, 1990. R. Zucco, C. Tausky, and G. Sutton, "Skill as proficiency: Introducing the process of task mastery," , . S. Arimoto, "Learning for skill refinement," in Proceedings of the International Workshop on Intelligent Robots and Systems IROS, pp. 10-13, 1991. N. Delson and H. West, "Robot programming by human demonstration: Subtask compliance controller identification," in Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems IROS, pp. 33-41, 1993. B. Yang and H. Asada, "Skill acquisition for robot grinding: Learning of high-level feedback laws from teaching data," in Proceedings of the 1990 JAPAN-U.S.A Symposium on Flexible Automation, pp. 649-655, 1990. J. Yang, Y. Xu, and C. Chen, "Hidden markov model approach to skill learning and
294
24. 25.
26.
27.
28.
29.
30. 31.
32.
33.
34. 35.
36.
37.
38.
its application in telerobotics," in Proceedings of the IEEE International Conference on Robotics and Automation, (Atlanta, Georgia), pp. 396-402, 1993. A. Bicchi, "Optimization of robotic grasping forces," in Proceedings of the American Control Conference, 1992. F.-T. Cheng and D. Orin, "Efficient algorithm for optimal force distribution--the compact-dual lp method," IEEE Transactions on Robotics and Automation, vol. 6, pp. 178-187, April 1990. J. Jameson and L. Leifer, "Automatic grasping: An optimization approach," IEEE Transactions on Systems, Man and Cybernetics, vol. 17, pp. 806-814, September/October 1987. C. Klein and S. Kittivatcharapong, "Optimal force distribution for the legs of a walking machine with friction cone constraints," IEEE Transactions on Robotics and Automation, vol. 6, pp. 73-85, February 1990. V. Kumar and K. WMdron, "Sub-optimal algorithms for force distribution in multifingered grippers," in Proceedings of the IEEE International Conference on Robotics and Automation, pp. 252-257, 1987. Y. Nakamura, K. Nagai, and T. Yoshikawa, "Dynamics and stability in coordination of multiple robotic mechanisms," International Journal of Robotics Research, vol. 8, pp. 44-61, April 1989. V. Nguyen, "Constructing stable grasps in 3d," in Proceedings of the IEEE International Conference on Robotics and Automation, pp. 234-239, 1987. D. Orin and S. Oh, "Control of force distribution in robotics mechanisms containing closed kinematic chains," Transactions of the ASME: Journal of Dynamic Systems, Measurement and Control, vol. 102, pp. 134-141, June 1981. P. Sinha and J. Abel, "A contact stress model for multifingered grasps of rough objects," IEEE Transactions on Robotics and Automation, vol. 8, pp. 7-22, February 1992. T. Yoshikawa and K. Nagai, "Evaluation and determination of grasping forces for multi-fingered hands," in Proceedings of the IEEE International Conference on Robotics and Automation, (Philadelphia, Pennsylvania), pp. 245-251, 1988. M. Buss, Study on Intelligent Cooperative Manipulation. PhD thesis, University of Tokyo, Tokyo, June 1994. K. Shimoga, "A survey of perceptual feedback issues in dexterous telemanipulation: Part i," in Proceedings of the IEEE Virtual Reality Annual International Symposium VRAIS, pp. 263-270, 1993. M. Buss and H. Hashimoto, "Hand manipulation skill learning for the intelligent cooperative manipulation system," in Proceedings of the Intelligent Autonomous Systems Conference IAS-4, 1995. Z. Li, P. Hsu, and S. Sastry, "Dynamic coordination of a multiple robotic system with point contact," in Proceedings of the American Control Conference, pp. 505-510, 1988. T. Namima and H. Hashimoto, "Dextrous manipulation using a 24 dof robotic hand," in Proceedings of the IEEE//RSJ/GI International Conference on Intelligent Robots and Systems IROS, 1994.
Intelligent Robots and Systems V. Graefe (Editor) 9 1995 Elsevier Science B.V. All rights reserved.
295
Designing a Behavior of a Mobile Robot Equipped with a Manipulator to Open and Pass through a Door Keiji Nagatani and Shin'ichi Yuta ~ Intelligent Robot Laboratory Inst. of Information Science and Electronics, University of Tsukuba, Tsukuba 305, Japan One approach of how to pass through a doorway by a mobile robot is discussed. In this task-oriented approach, proposed task is to realize a total behavior with opening a door and passing through a door-way by a mobile robot equipped with a manipulator. In this paper, we describe a design of such behavior by a robot, and show a simulation of robot motion series for this task. The key issue for realizing such behavior is cooperation between robot's function sub-systems, such as the locomotion control system, the manipulator control system and the sensor systems. 1. I n t r o d u c t i o n Research of mobile manipulators has received much attention. These approaches treat the redundancy of mobile manipulators [1] [2] [3], and analyze the dynamics of mobile manipulators [4] [5]. Some approaches propose a control concept or an algorithm for a simple task, such as tracing surfaces[6]. However, few approaches have dealt with a complex task for the mobile manipulator directly. In this paper, we present a task-oriented approach for researching mobile manipulators on a specific mission. Our approach consists of the following steps. 1. Defining a task for a robot to perform 2. Designing about an algorithm and an architecture to realize the task. 3. Implementing the algorithms and architecture on a real robot system for experimental verification of our design 4. If problems existing in the implementation, repeat steps 2 and 3 until a satisfactory solution is found. We define the task of this research as realizing a behavior to open the door and to pass through a door-way using an autonomous mobile robot equipped with a manipulator. Considering the architecture of the robot, it is most reasonable to realize a distributed control system. In the distributed architecture which is proposed in references [7] [8]. The robot's task we propose here can be realized by cooperation between the manipulation module, locomotion module and sensor modules.
296 Section 2 presents a detailed definition of the task the robot will perform. In section 3, a design based on behaviors to analyze the task is presented. A sinmlation result using the proposed design is shown in section 4. In section 5, an implementation and experimental result of stage 4 on the real robot is reported. Future work is discussed in section 6. 2. D e f i n i t i o n of t h e t a s k The problem task is opening a door and passing through a door-way using a wheeled mobile robot equipped with a manipulator. To realize this task, the robot finds the door knob, then grasps and rotates the knob with an end-effector mounted on the manipulator, pushes the door and passes through the door-way using both its locomotion system and the manipulator. Such a task depends on the size of mobile robot, the size of manipulator, the ability of the robot hardware and the condition of the environment. First, we define the assumptions and constraints concerning the environment and the robot. 2.1. E n v i r o n m e n t The target environment is indoors, such as an office building. The surfaces of the floor are sufficiently smooth so that a wheeled mobile robot can move easily. Rooms are connected with corridors by open-sided doors which are usually closed. Every door has a rotating door knob which is centered about 5 centimeter from the opening side at a height of about 100 centimeter. When the knob is rotated, a prop in the door is released and the door can be opened by pushing or pulling it. Every door is motionless while no external force is applied to the door. The size of the door is, about 200 • 90 • 3 centimeter. Doors of this form and size are standard in Japanese offices. The above described environment exists in our laboratory at University of Tsukuba (Figure 1).
Figure 1. Target Environment
297 2.2. R o b o t Our laboratory has developed several small sized wheeled mobile robots, called "YAMABICO"[9] for experiments. One of these robots, which is called "YAMABICO typeTEN", is rather large and is equipped with a light manipulator. Figure 3 shows a photograph of "YAMABICO type-TEN". In this research project, we use this autonomous mobile robot to realize the proposed task. The configuration of the YAMABICO type-TEN robot system are now discussed.
9 A light weight 6 degree of freedom manipulator is mounted on the mobile robot. 9 A force sensor is located at the end of the manipulator arm. 9 An end-effector to grasp the door knob is mounted on top of the force sensor. 9 A vision sensor is mounted on the naanipulator to confirm the position of the knob of the door. 9 Twelve ultra-sonic sensors are mounted around the mobile robot for navigation and obstacle detection. 9 The mobile robot uses a locomotion system with two driving wheels mounted on a central axis [10]. 9 The locomotion system uses dead-reckoning to estimate its position based on odometry. 9 The schematic details of the manipulator and its degree of freedom (DOF) are shown in Figure 2. 9 The position of the sensors are illustrated in Figure 3.
285.... / ~ ...................
6"".,
2
/'""
1
/ / / / / / / / / /
Figure 2. Schematic Diagram of the Manipulator
65ram
298
Figure 3. YAMABICO type-TEN
3. Designing a B e h a v i o r to Pass T h r o u g h a D o o r - w a y To realize the proposed task of robot passage through a door-way, a number of technical considerations are necessary. The cooperation between function controllers of the robot must be considered. In this section, we analyze the complete behavior to pass through a door-way, particularly the cooperation between the manipulation and the locomotion. First, we divide the task into 9 stages from the point of view of designing cooperative actions between controllers. The stages are shown in Figure 4. Second, we developed a simulator to analyze the robot's behavior. In this simulator, the cooperation between the locomotion controller and manipulation controller is simulated. The parameters of the initial position, paths and speed of opening door can be changed in the simulator to determine the best parameters empirically. Details of the simulator are presented in Section 4. Nine stages of detailed behavior to realize the proposed task is discussed in following. In order to generalize the environment, width of the robot is expressed as R~, length of the robot is expressed Rd, width of the door is expressed D~. These parameters are shown in Figure 5. Where, the manipulator mounted on the robot has the similar length with door width, and such parameters need to satisfy the condition as D w > R~.
299 START
Stage 1
9. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . { Position the robot : in front of the Door ~. . . . . . . . . . . . . . . . . . . . . -~ . . . . . . . . . . . . . . . . . . . . . .
"~ i,
Obstacle
on
the
Path'?
,
l
Stage 2
............................................. Find and Grasp the Door Knob ~9 . . . . . . . . . . . . . . . . . . . . -~ . . . . . . . . . . . . . . . . . . . . .
Stage 3
............................................ Rotate the Knob and Push the Door ~9 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
~
Can't
see
Door
Knob?
l ;
Open ......................
the Door "~ . . . . . . . . . . . . . . . . . . . . .
.............................................
Stage 5
R e l e a s e
~9 . . . . . . . . . . . . . . . . . . . .
the Knob -~ . . . . . . . . . . . . . . . . . . . . .
Door
Locked'?
,
,. . . . . . . . . . . . . . . . . . . . . . . . . . . ."~ ................
Stage 4
The
i J
Obstacle Back
at of
the the
Obstacle
on
the
Door
?
] i J
l Stage 6
~ ........................................... Move Behind the Door ~,. . . . . . . . . . . . . . . . . . . . -~ . . . . . . . . . . . . . . . . . . . . . .
Stage 7
. . . . . . . . . . . . . . . . . . . . . . . . . . . .T. . . . . . . . . . . . . . Find and Grasp the Door Knob ...................... ~ .....................
~
Path?
I
--,I l .'
f
............................................
Stage 8
Close 9~ ....................
the
Door
-~- .....................
1 ,
f
Another (Obstacle
Task
avoidance
etc.)
............................................
Stage 9
Release ~9 . . . . . . . . . . . . . . . . . . . .
the Knob "~ . . . . . . . . . . . . . . . . . . . . . . I END
Figure 4. Procedure to pass through a door-way
S t a g e 1 --Position the Robot in Front of the Door
The mobile robot is supposed to have an environment map and navigates itself to the front of the door by the navigation method reported in reference [13]. The stop position is d ~ i g ~ d to be (R_~2-t- r/21) from the front of the door and (R_~ + rn2) from the right side of the door. This position is shown in Figure 5. Where, TIZ 1 and rn2 are the margins for safe movement of the robot. S t a g e 2 - - F i n d and Grasp the Door Knob
If the robot knows its position and knob's position exactly, the robot can grasp the knob by the end-effector own only by position control of the manipulator. However in reality, errors exist between the estimated position and the actual position of the mobile robot. Therefore, the robot must confirm the exact position of the knob by
300
Figure 5. Position the Robot in front of the Door
using the vision which is mounted on the manipulator. If the robot can't find the knob of the door, the robot abandons the task. The next step in the task is to move the end-effector toward the knob and grasp it with the manipulator. This behavior is controlled by a kind of visual servo based on the vision sensor. S t a g e 3 - - R o t a t e the Knob and Push the Door The robot rotates the knob, and pushes the door with a small force. If the force sensor mounted on manipulator senses excess force, the robot recognizes that the door is locked and abandons the task. S t a g e 4 - - O p e n the Door The total length of the manipulator is limited. Therefore the mobile robot must move forward while the manipulator pushes the door. The manipulation controller nmst cooperate with locomotion controller. Designing this behavior is realized in the following way. The locomotion controller moves the robot along a straight line. The line is (_5_~+ rn2) from the right side of the door, shown in Figure 6. At the same time, the manipulation controller controls the manipulator to push the door. The desired position of the mobile robot depends on the angle of the door. As the door angle increases, this results larger forces acting on the end-effector. To open the door
301 with a fixed angular velocity, the robot's velocity must be varied proportional to the angle of the door. As the door angle increases, the robot's velocity must be decreased. The robot's position is calculated as follows.
P~176
D~176
The desired posture of the manipulator is also calculated from the knob's position which also depends on the angle of the door. While a robot pushes the door with mobile base moving forward, position and orientation errors of the mobile base may generate large forces in the end-effector. To absorb the force, a compliant control at the top of the end-effector is needed by using the force sensor. It is discussed in section 5.3. In this stage, the robot also monitors the force for the direction of the motion of door with the force sensor. If excessive force is sensed, the robot concludes that there are obstacles behind the door and abandons the task. According to above formula, the door is opened 90 degrees when the robot arrives at stop position.
Figure 6. Open the Door
302 S t a g e 5 --Release the Knob The end-effector releases the knob, and the manipulation controller returns the manipulator to its initial position. In this stage, only the manipulation controller is active. S t a g e 6 --Move Behind the Door In order to close the door, the robot moves behind it by navigating itself [13]. The robot's path and the exact parameters are shown in Figure 7. At the stop position, the robot uses its ultra-sonic range sensor to correct its position using the wall and the opened door.
Figure 7. MoveBehind the Door S t a g e 7 - - F i n d and Grasp the K n o b The robot finds and grasps the knob by using vision sensor as same as stage 2. S t a g e 8 --Close the Door In this phase, the robot pushes the door to close while moving. This behavior is similar as the Stage4, but the motion of the robot is different. The straight line is {D..,- (-~ + m2)} from the wall, as shown in Figure 8. The desired position of the robot is calculated, P~176
- D w * sin ( 90 - D~176Angle "
303 When the robot stops at the stop position, the door may already be closed. Confirmation of the closure of the door is done by sensing the force acting on the knob.
Figure 8. Close the Door
Stage 9 --Release the Knob The end-effector releases the knob, and the manipulation controller returns the manipulator to its initial position. In this stage, the robot completes the task, and continue to navigate itself to the destination of the robot. The activity of controllers is shown in Figure 9. In this figure, the locomotion controller needs to cooperate with manipulation controller in Stage 4 and Stage 8, and the vision sensor unit needs to cooperate with manipulation controller in Stage 2 and Stage 7.
304
Locomotion Control
Manipulator Control
Vision Sensor
Cooperation
9 9 9
9
Vision and Manipulator
0
Stage 1 Stage 2 Stage 3
0
Stage 4
Locomotion and Manipulator
O
Stage 5
0
Stage 6 Stage 7
O 9
0
Stage 8
Vision and Manipulator
O
Locomotion ,and Manipulator
O
Stage 9
Figure 9. Activity and Cooperation between Controllers
4. S i m u l a t i o n We implemented a program to simulate the behavior of the mobile robot equipped with a manipulator. In this simulator, the model of the robot and environment are built into the simulator. The cooperation between a locomotion controller and a manipulation controller can be studied. Calculating method of the arm position is performed using differential motion control [12]. To find the best parameters for perforlning the behavior, the follow parameters can be changed and the simulation can be repeated. 9 The initial position of the robot 9 The trajectory and the speed of the robot 9 The desired angular speed of the door during opening and closing According to the assumption of the robot, the parameters of detailed behavior is, D w 90cm Rw - 70cm Rd -- 50cm. Under these parameters, we used simulator repeatedly, margin parameter is fixed as follows empirically. ml
-
50cm,
m2
-
5crrz,
m3
-
15c'm
Sn example of the simulation result is shown in Figure 10. In this simulation, we did not consider the dynamics of the robot and accumulated error of the locomotion. We should consider these factors in real environment.
305
I
(3)
(2)
(I)
!
I
(4)
(5)
I
I
(9)
oo)
.__----=.---
i
I
I
(6)
(7)
(8)
@ ---03
8
l _ I
(11)
(12)
(13)
(14)
(15) 1
S_
k__ I
I
I
I
I
(16)
(17)
(18)
(19)
(20)
I
I
(21)
(22)
9 (23)
F i g u r e 10. An Exa, l-nple of S i m u l a t i o n R e s u l t
@ -\ (24)
t_..a
(25)
306 5. I m p l e m e n t a t i o n Next step of our task-oriented approach is to implement the proposed algorithms and architecture on a real robot system. First, we implemented stage 4 behavior on real robot system. One key of the behavior is how to realize a cooperation between locomotion control and manipulation control, and the other key is how to realize a compliant control of the manipulator. In this section, we report a control architecture of robot-system, outlines how to realize a cooperation behavior and compliant control, and experimental result of stage 4.
5.1. A r c h i t e c t u r e of R o b o t - S y s t e m Our group has proposed a distributed control architecture for autonomous robot systems [7][8]. To realize the behavior which is proposed in Section 3, we basically use it. Each function of the robot is modularized in this distributed architecture, and we refer to each distributed function as a module. The architecture form two level hierarchy. The top of the structure is master module which supervises the total control of the robot system. Other modules are connected by bus and each module communicates with master module by using dual-port memory. In this implementation, we allow for direct communication between the locomotion module and the manipulation module. This is because, fast communication is necessary in stages 4 and 8 of the robot's task. The schematic diagram of proposed architecture is shown in Figure 11. Now, master module, manipulation module, locomotion module and force sensor module are working to realize a behavior of stage 4.
. ......................................................................................
B U S [. . . . . . . . . . . . . . . . . . . . .
..... 9- p
I ....
.....
I
I
~2~I::~~ .........
[
.....
/ .....
_[~'- ...... .r..... " ' ~ 1 M,.,o~ I ............
(Locomotion "~ Board
i
~.aoara/i,~
il Data
(~~
.oco
o, on
o0ue
i
an
ua, o0
[.......
sensor
)
.......... Mo~u[~............ :"................................,
Ultra-Sonic Transducer and Receiver
~ Motors for Manipulator
I
)
! Motors for Locomotion
I ....
l,,'V~a S o ~
i(V~onsenso~!
i '.. aoar~ l - i l
Data( ~ ~
I
.."...... I M~,,,~ I........: ."...... 1 M~.~
(Manipulati~
--
o0, e ....
.........................................
'" .........................................
:i [
] { i
i
[
9 Module . 9. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Figure 11. Architecture of the Robot
i :
Force Sensor
J ;
Module
"- . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
307
5.2. Cooperation between Manipulator and Locomotion According to the design of stage 4, we realized cooperation behavior between manipulator and locomotion. All parameters are same as parameters of simulation. To synchronize a behavior of locomotion and manipulation, master module totMize both modules. It sends reference position to locomotion module, and also sends reference joint angle to the manipulation module. Master module always check differences between current and reference position of the robot vehicle and joint angles of the manipulator. When the both differences are enough small, the master module changes references of position and joint angle in small amount to step forward.
5.3. Compliant Control for Manipulator When a robot pushes the door while moving, position and orientation errors of the robot may result in large forces in the end-effector. Therefore, we have to control the end-effector compliantly along the plane perpendicular to axis of the knob using the force sensor (Figure 12).
Figure 12. Compliant Control
The compliant control is realized as follows. The force sensor detects a force on the vartical plane. In propotion to the amount of the force, the reference position of the endeffector is shifted to detected force direction. It is calculated in every 20 mili-seconds. This mechanism absorbs low frequency component of the position error. The high frequency component of the error is covered by soft rubbers attached inside of the end-effector. 5.4. E x p e r i m e n t a l R e s u l t s We implelnented proposed stage 4 behavior on the target robot "YAMABICO-type TEN", and made door opening experiment. In the experiment, the robot can open the door while the robot-base moving forward slowly. This behavior is shown by continuous pictures in Figure 13.
308
Figure 13. An Experiment of Door Opening Behavior
6. C o n c l u s i o n and F u t u r e W o r k s In this paper, we proposed a design of a behavior of the mobile robot equipped with a manipulator to open a door and to pass through a door-way. Such a behavior is realized by a cooperation between the locomotion controller module and manipulation controller module. We verified this cooperation of it by simulation. Finally, we showed the implementation of stage 4 on our mobile robot "YAMABICO type-TEN" and experimental result. To realize a total behavior of robot's passing through a door-way, we must solve a number of technical problems. An important topic is how to confirm the position of the knob and how to grasp it with the end-effector (stage 2), and our next step is to realize such grasping behavior. To confirm the exact position of the knob, we will use a vision sensor. After realizing the stage 2 behavior, we will make a total behavior of robot's passing through a door-way. In a future, we will make a long distance navigation for an autonomous mobile robot equipped with a manipulator including a door opening behavior.
309
Acknowledgement T h e a,uthors express their g r a t i t u d e to the members of Intelligent Robot L a b o r a t o r y of University of T s u k u b a .
REFERENCES 1. Wayne F.Carriker, Pradeep K.Khosla, Bruce H.Krogh : "Path Planning for Mobile Manipulators for Multiple Task Execution", in IEEE Transactions on Robotics and Automation, VOL.7, No.3, JUNE 1991. 2. W.Miksch, D.Schroeder :"Performance-Functional Based Controller Design for Mobile Manipulator", in Proc. of IEEE Int. Conf. on Robotics and Automa, tion, pp227-232, 1992. 3. Homayoun Seraji : "An On-line Approach to Coordinated Mobility and Manipulation", in Proc. of IEEE Int. Conf. on Robotics and Automation, pp28-33, 1993. 4. S.Dubowsky, E.E.Vance : "Planning Mobile Manipulator Motions Considering Vehicle Dynamic Stability Constraints", in Proc. of IEEE Int. Conf. on Robotics and Automation, pp 1271-1276, 1989. 5. Norbert A.M.Hootsmans, Steven Dubowsky : "Large Motion Control of Mobile Manipulators Including Vehicle Suspension Characteristics", in Proc. of IEEE Int. Conf. on Robotics and Automation, pp2336-2341, 1991. 6. Yoshio Yamamoto, Xiaoping Yun : "Control of Mobile Manipulators Following a Moving Surface", in Proc. of IEEE Int. Conf. on Robotics and Automation, ppl-6, 1993. 7. Yutaka Kanayama, Shin'ichi Yuta : "Computer Architecture for Intelligent Robots", in Journal of Robotic Systems, 2(3), pp237-251, 1985. 8. S.Yuta and J.Iijima : "State Information Panel for Inter-Processor Communication in an Autonomous Mobile Robot Controller", in Proc. of IEEE Int. Workshop on Intelligent Robots and Systems, 1990. 9. S.Yuta., S.Suzuki, S.Iida : "Implementation of a. small size experimental self-contained autonomous robot", in Proc. of Second International Symposium On Experimental Robotics, 1991. 10. S.Iida and S.Yuta : "Vehicle Command System and Trajectory Control for Autonomous Mobile Robots", in Proc. of IEEE Int. Workshop on Intelligent Robots and Systems, pp212217,1991. 11. K.Nakazawa, M.Shimizu, S.Yuta, M.Nakajima : "Measurement of 3-D Shape and Position by Fiber Grating Vision Sensor Installed on a Manipulator", in Proc. of IEEE Int. Workshop on Intelligent Robots and Systems, pp611-616,1988. 12. Rechard P.paul : "Robot Manipulators", the MIT Press, 1981. la. K.Nagatani and S.Yuta : "Path and Sensing Point Planning for Mobile Robot Navigation to Minimize the Risk of Collision", in Proc. of I E E E / R S J Int. Conference on Intelligent Robots and Systems, pp2198-2203, 1993.
This Page Intentionally Left Blank
Intelligent Robots and Systems V. Graefe (Editor) 9 1995 Elsevier Science B.V. All rights reserved.
311
The D e v e l o p m e n t of R e - u s a b l e S o f t w a r e S y s t e m s for Intelligent A u t o n o m o u s R o b o t s in Industrial and Space Applications E. Freund and J. Rol3mann Institute of Robotics Research, University of Dortmund, Otto-Hahn-Str. 8, 44227 Dortmund, Germany, Email: rossmann @damon.irf.uni-dortmund.de ABSTRACT
The aim of the development of modern robot control systems at the Institute of Robotics Research (IRF, Institut fiir Roboterforschung) is to provide higher flexibility and autonomy for robot systems, together with consistent and user-transparent concepts for installation, programming and operation of robots in their working-environment. An important feature of a flexible future-oriented robot controller is the capability to incorporate information from different classes of sensors and to have standardized communication interfaces with other factory-automation components. To ease the robot systems' operation an intuitive graphical user-interface is provided. Furthermore, implicit programming and action planning methods allow robot programming on a much higher level of abstraction than today's procedural programming languages. For industrial applications, this implies easier and quicker adaptation of robot workcells to new tasks and a significant increase of the robots' flexibility to allow the feasibility of small quantity orders. This paper gives a basic description of "the principle of the hierarchical coordinator", the basic system concept for the development of complex control systems at the IRF. This concept has been developed and employed for the design and implementation of intelligent, autonomous robot systems, which are described in detail. The applications described range from the design of a flexible assembly workcell to a multirobotsystem for autonomous experiment-servicing in a space laboratory module. 1. INTRODUCTION The development of intelligent and autonomous robotic systems with a high degree of flexibility, good operational capabilities and the potential for efficient integration into an industrial CIM-environment is still a difficult task which requires both, technical and management skills. In the recent years the transfer of know-how from the area of space robotics to industrial applications has been emphasized at the Institute of Robotics Research because robotic system capabilities for space applications have already reached a considerable standard with regard to system autonomy, safety and local intelligence that is appropriate to be applied to industrial applications. The development of robot control software at the IRF has always aimed at the realization of components which take into account industrial standards and which can be integrated into comprehensive automation concepts. Particular care in the design of precise interfaces has paid off, not only when adapting various software components to control systems in an industrial environment, but also has permitted a consistent integration of modules in the design and development of new products.
312 2. COORDINATION OF THE DEVELOPMENT OF I N T E L L I G E N T AUTONOMOUS ROBOTS Fig. 1 gives an overview of the interdependence between the research activities at the IRF dedicated to the design of intelligent autonomous robot and automation systems. This overview is structured according to the activities of the four research groups at the IRF: "Autonomously Guided Vehicles and Sensor Technology", "Multirobot Systems and Space Robotics", "Automation Systems" and "Communication Structures and Intelligent Systems". In Fig. 1, the arrows symbolize the transfer of results into new projects and products. A strictly modular realization of functions with the capability for transfer and further development requires a great deal of inter-project management, especially regarding the specification of module interfaces. Experience has shown, that this additional expense in the first stage of a new development is made up with increasing efficiency and lower costs for the resulting product.
Fig. 1: Coordination of research projects at the IRF Fig. 1 shows that developments in the projects ROTEX (Robot Technology Experiment) and CIROS (Control of Intelligent Robots in Space) [3] can be regarded as basic research works at the IRF, whose functionality contributed significantly to the results of the projects evolving.
2.1 Space Robotics as a Motor for the Development of Intelligent Robot Control Systems Essential parts of the path control software of the ROTEX-robot its capabilities were demonstrated in the second German space mission D 2 - were developed at the IRF. One of the major challenges of this development was the cooperation with several project partners and the demand to meet the quality standards for flight software. Fig. 2 displays a simplified scheme of the ROTEX path control structure. The functionality for microgravity path planning deserves special attention due to its capability to plan robot motions which minimize the forces and torques exerted on the space shuttle. Robot motions are planned such, that the amplitude of disturbances remains limited in a specified frequency band. This guarantees that payload experiments requiring zero gravity can be carried out successfully. The other path control modules such as collision detection and -avoidance are
313 not specifically designed for space applications and can directly be used in terrestrial robot controllers.
Fig. 2: Simplified structure of the ROTEX path control While the ROTEX software was developed concentrating especially on the precise project timing and the quality standards for flight software, research beyond the scope of ROTEX was carried out developing a robot control system suitable to control a multirobot system for space applications. In the C I R O S (Control of Intelligent Robots in Space) [3] project, this multirobot control structure was built, based on the concept of the hierarchical coordinator [ 1][9]. Using the methodology of the hierarchical coordinator, a robot control system with a significant increase in autonomy and the operational capabilities for space laboratory servicing was designed. The implemented control system IRCS (Intelligent Robot Control System) and is well suited to perform service tasks in a space laboratory environment such as the CIROS testbed shown below.
Fig. 3: The CIROS multirobot testbed
314 The CIROS testbed is equipped with two robots with redundant kinematics. Each has 6 revolute and one translational joint. The layout of the laboratory is similar to that of the German Spacelab and it was built in a modular manner, so that it is possible to incorporate the latest experiments and to adapt the environment to reality as precisely as necessary. In four double and two single racks, switches and other operating elements of the experiments were reproduced and arranged in order to be performing realistic operational sequences. A tool exchange capability and a torque/force-sensor was mounted at each robot's wrist to allow the robots to operate drawers and flaps for experiment processing. Example tasks comprise the exchange of printed circuit boards and inserting of a sample into a heater. The redundant two-armed robot configuration with the torque/force-sensors at the robots' wrists permits fully coordinated operation as well as synchronized or independent action of the two robots. Furthermore, the robots are equipped with hand cameras and the whole laboratory can be supervised by a scene camera. A vision system performs the imageprocessing in order to provide the planning levels with necessary information, i.e. about the exact location of objects to be gripped. The control desk in the foreground contains devices for system control and supervision.
Fig. 4." Structure of the multirobot control system IRCS
315 Fig. 4 shows the structure of the CIROS multirobot control IRCS (Intelligent Robot Control System), that contains, superimposed on the level of the individual robot controllers, the additional levels for online-collision-avoidance, multirobot-coordination and automatic action planning [3][ 10][11]. The capabilities of this multirobot system include the ability to operate both robots independently or in a coordinated manner. Cooperative operation enables the operator to handle big, heavy or sensitive loads with high precision, while independent robot operation allows a parallel execution of tasks. Both, cooperative and parallel operation are presented in the following sections as coordinated operation [2]. A task oriented programming interface facilitates the programming of the multirobot system. The automatic action planning component [4] can interpret action descriptions which are formulated on a high level of abstraction. With the aid of the central world model that contains a mathematical description of the environment, tasks formulated in a "user-friendly" manner like "close drawer 1 in rack 4" or, as shown in fig. 5, "put sample 1 into heater slot 1" are decomposed into so-called elementary actions. These elementary actions are selected from a predetermined set of actions comprising the activation of endeffectors, locking or unlocking of the gripper exchange system and the path planning functionality for independently working or cooperating robots.
Fig. 5: Automatic action planning for multirobot systems During task execution, the robots are supervised by the level of online collision avoidance [13][12][14], which predicts potential collision dangers between robots and between robots and their environment. In case of collision danger, it actively avoids a collision by changing the pre-programmed path of the endangered robots in real-time, considering the static and dynamic limits of the robots and task specific constraints. Fig. 6 shows an example of such an automatically generated avoidance trajectory. As a part of its task, the upper robot starts its motion on the right side of the CIROS-environment, moving to a drawer in the left side of the cell. The lower robot is not busy during this motion and should therefore give way without disturbing the upper robot's motion. Snapshots of the resulting paths are shown in fig. 6. It
316 should be emphasized here, that the lower robot's motion only results from the need to avoid a collision with the upper robot and to return to its starting position. With the help of the online collision avoidance level, the upper robot was able to follow its trajectory exactly in the way it was planned by the path planning module in the level of coordinated operation (fig. 4).
Fig. 6: Online-collision-avoidance in the CIROS-demonstrator The description of the functionality of the CIROS multirobot control system gives an idea of the complexity of a system including capabilities for automatic action planning, coordinated operation and online collision avoidance.
Fig. 7: Structure of the multirobot system using the hierarchical coordinator
317 The entire control architecture was designed using the concept of the hierarchical coordinator which provides the system theory to describe the interfaces and the assignment of functionality to the different hierarchical levels in a systematic manner. Fig. 7 shows the architecture of the multirobot system from the system-theoretical point of view of the hierarchical coordinator as introduced in [9].
2.2 Modern Virtual-Reality Based Man-Machine-Interfaces for Intuitive Operation The use of methods for automatic action planning facilitates the programming of a multirobot system like CIROS very much, but still requires a great deal of knowledge about the workspace environment and symbolic descriptors for individual components. The project VITAL (Virtual Reality for Telerobotics) pursues the goal to develop an easy to use, intuitive man-machine-interface for single robots and multirobot systems that uses today's Virtual Reality technology. Various tasks that are executed by robots in a space laboratory environment in order to conduct experiments, are carried out by researchers, who do not have robot specific knowledge. In the VITAL project, these researchers can now perform their experiments in a graphically simulated virtual environment, employing a data glove for intuitive interaction with the control system. Eye-phones, small displays integrated into a helmet, give a stereoscopic view of the environment and the experiments. Any operation executed with the data glove, which is an animated human hand in virtual reality, is translated into robot actions with the aid of the action planning component and executed by the robots in the physical environment. Fig. 8 shows a view into the Virtual-Reality representation of the CIROS environment, showing the animated hand, a drawer and the printed circuit boards of a computer system, that the operator is manipulating. Please note that the robots are not shown in the virtual environment, the operator's hand is the only active part.
Fig. 8: A view of the CIROS environment in the virtual reality representation (Photo: CAE)
318 A straight forward approach to control a robotic system via Virtual Reality (VR) is to directly track the data-glove with the robot's tool-center-point (TCP); this mode of operation will further be called "telemanipulation or hand-tracking mode". The n theoretical ~ advantage of this approach is its flexibility. With the robot directly tracking the operators hand, the robot should ~ in theory ~ be able to do the same things as the human operators. In practice, this mode of operation cannot really be recommend for most of the tasks that involve contact between a robot and its environment. If e.g. an assembly task should be commanded by a virtual reality system in hand-tracking-mode, the following problems are encountered: 9 time delays between the display of a robots movement in the VR and its physical movements are critical for the stability of the handling process, because, similar to standard telemanipulation approaches, the user is still "in a real-time control loop" 9 the graphical model has to be very precise 9 the measurement of the position and orientation of the data-glove has to be very precise 9 measures have to be taken to reduce "trembling" of the operators hand 9 online-collision avoidance features are necessary to grant safe operation 9 a versatile sensor-control is necessary to compensate for unwanted tensions when objects are inserted into tight fittings The problems mentioned above make very difficult the control of the robot by just tracking the operators hand, so that a new method to control the robotic system [ 15] was developed that helps to facilitate the user's job to control the robots. The aim was to hide from the user the typical robot-related difficulties like singularity-treatment, collision avoidance, sensor-based guidance of the robot and the problem to coordinate different robots. By making use of modern control techniques that can provide these features, the necessary expertise to e.g. conduct an experiment in a space laboratory environment like CIROS is thus shared between the user with the necessary knowledge about the experiment and the robot-control with the necessary "knowledge" about how to control the robots. So another mode of operation of the VR-system was developed: the task deduction mode, which makes use of the capabilities already developed for the IRCS. The solution was to enhance the VR-system in the way that while the user is working, the different subtasks that are carried out by the User are automatically recognized and task descriptions for the IRCS are deduced. These task descriptions are then sent to the action planning c o m p o n e n t of the IRCS and are carried out safely by the IRCS with its inherent capabilities. This approach has several advantages over the direct hand-tracking: 9 the user is no longer in the "real-time control loop". Complete subtasks are recognized and carried out as a whole without the necessity for immediate feedback to the user 9 the subtasks are carried out safely using the sensor-control and -supervision capabilities of the IRCS 9 for physical assembly tasks, the accuracy of the environment model can be compensated for by automatic sensor-supported strategies 9 the accuracy of the data-glove tracking-device is not as important as for the direct tracking mode. The allowable tolerances when the user is gripping an object or inserting a peg in a hole can be adjusted in the VR-software 9 if a heavy or fragile object is moved in the VR, the planning component is automatically capable of using the two CIROS robots in a coordinated manner to do the transport in reality 9 different users working at different VR-systems can work on different tasks that are sent to the planning component of the IRCS, which then can decide, depending in the available
319 resources, an adequate sequence of the tasks to be done. Thus one robotic system can serve e.g. multiple experimentators in a space laboratory environment. if the robot control is versatile enough, there is no longer a need to even show a robot in the virtual environment displayed to the user; so the user more and more gets the impression of carrying out a task "himself", which is the highest level of intuitivity, that can be reached. if the planning component is versatile enough, it cannot only control the robots, but also other kinds of automated devices. The action planning component in the CIROS environment "knows" that to open the leftmost one of the three drawers, it doesn't need to employ a robot. This drawer is equipped with a motor, so that it just has to control the motor to open this drawer. Robot-automated and hard-automated tasks are thus controlled under one unified framework. The only deficiency of this mode of operation is, that the IRCS must principally know the different types of tasks carried out by the user. Therefore, for special applications that are hard to describe as parameterized tasks m e.g. the shaking of a material sample m or for applications that were not foreseen by the developers of the action planning component, the hand-tracking mode is still available in the realized VR-system. Experience showed, that 95% of the tasks carried out in the CIROS environment can successfully be commanded in task-
deduction-mode. In August 1994, as a part of a cooperation between the Institute for Robotics and Intelligent Systems of the University of Southern California (USC) and the Institute of Robotics Research in Germany, the PC-based version of the VR-system, that builds on the PC-based robot simulation system COSIMIR, described in chapter 3.2, successfully performed the control of the CIROS testbed in Germany from Southern California via INTERNET. In a research project at USC, the same VR-system is used to simulate, supervise and control a sixlegged walking machine. The user here can "enter the same room" with the simulated walking machine, he can place different obstacles in the way of the walking machine and see how the different implemented algorithms behave in the simulated world before they are downloaded into the physical walking machine for testing purposes. This allows several team-members to work with the machine at the same time, running several instances of the VR-system and thus reduces the need to work with the physical machine.
3. SCALEABLE ROBOT CONTROL ARCHITECTURES: HIGH-PERFORMANCE AND L O W - C O S T NO C O N T R A D I C T I O N In the research field "robot control and simulation" at the IRF, experiences and results from the projects ROTEX and CIROS were directly applied to products for industrial applications. As shown in fig. 1, the product development was conducted in the projects FRC (Flexible Robot Control), PCROB (PC-based Robot Control) and C O S I M I R (Cell Oriented Simulation of Industrial Robots).
3.1 Robot Control Development As only an Intel 8086 based computer had space qualification when the ROTEX software was developed, the control software had to b e coded very efficiently. Exactly these efficient routines make up the kernel of the PC-based robot control PCROB (PC-based Robot Control).
320 At first, PCROB was planned as a low cost robot control with an easy to learn user-interface and dedicated to small robots. PCROB runs on a PC, a low-cost hardware platform, in order to provide a low cost robot control for didactic purposes and as a starting point for automation and robotics in smaller companies. In parallel, the VME-based robot control FRC (Flexible Robot Control) was implemented, providing all the features of a modern robot control. In the FRC, a sensor-interface, a network-interface and a programming environment for IRL, VAL-II and IRDATA were provided. In the past two years, due to the increasing capabilities and speed of PC-hardware, the development of both robot control could be harmonized, such that more and more features could be realized for the PCROB as well. So today high performance and low cost are no longer a contradiction because PCROB now provides the same functionality as the FRC including a teach pendant, the programming environments for IRL, VAL-II, B APS and IRDATA, a sensor interface and a network interface as shown in fig. 9.
Fig 9: The structure of the PC-based robot control PCROB
3.2 Robot Simulation Systems Based on the results of the P C R O B and CIROS projects the robot simulation system C O S I M I R (Cell Oriented Simulation of Industrial Robots) was developed. It runs on either a PC with MS-Windows, Windows NT or on a Silicon Graphics workstation. COSIMIR consists of three major components: the control and display component, the robot controller (PCROB) and the environment model (fig. 10).
321
Fig. 10: Structure of the COSIMIR simulation system In addition to providing a graphical user interface, the control and display component supervises the environment model and robot controller components. During each simulation cycle, new state values for the robot and simulated mechanisms in the workcell (i.e. revolving table, conveyor belts, parts feeders, etc.) are requested from the robot controller (PCROB) and then transferred to the environment model. An object manager maintains the environment model of the simulated workcell calculating the current 3D polyhedral model of the workcell taking into consideration the interaction between the individual mechanisms. E.g. an object on a conveyor belt or an object placed on a revolving table, must be moved together with a moving belt or table. The object manager was adopted from CIROS project. In the CIROS project, the object management task is principally the same, however the volume model of the workcell here is not used for the model visualization but rather makes up the basis of calculation for online collision detection and avoidance. In the CIROS collision avoidance, corresponding to its task in CIROS, this component is named the collision avoidance specific world model [13]. The reason that the CIROS collision avoidance can be incorporated in COSIMIR is one of the important characteristics of the CIROS collision avoidance, it is extremely flexible because the model information is not hard coded in the source code but rather formulated by a textual description language known as GEMOL. The model description is read during the initialization phase and is converted during run time to the necessary realtime data structures. COSIMIR optimally uses this flexibility because the user just loads the GEMOL-description of another work cell to work with it. To facilitate the work cell definition, i.e. the programming of a GEMOL work cell description, the program COSIMOD,
322 delivered with COSIMIR, offers a graphical interactive environment to develop the work cell descriptions; thus alleviating the user from learning GEMOL. Besides the flexible modeling of the environment, the integration of the industrial controller PCROB is a strong argument for COSIMIR. The robot programs generated by COSIMIR will be capable of running in the actual robot work cell without changes. This is because the PCROB kernel that controls the simulated robot in COSIMIR also controls the actual robot allowing the problems such as joint limits, static and dynamic limits and reachability in the neighborhood of singularities to be found during the simulation execution. The integration of the tested object manager and PCROB components shortens the integration time of COSIMIR. With the implementation of an advanced communication concept, COSIMIR will always be integrated with the latest version of PCROB; thus all advances of PCROB will be utilized in COSIMIR. COSIMIR is available in English and German and since December 1993 is distributed world wide by an industrial partner.
3.3 Robot Controllers for Flexible Manufacturing Scaleable, task adaptable robot controllers form an important base of modern factory automation. These controllers are not developed with only a specific industrial robot in mind but rather with standardized interfaces, network connections and are programmable with a standard robot programming language for applicability with a wide range of industrial robots. In addition such systems must not only be of high performance to make flexible computer controlled manufacturing possible, they must also be cost effective in the integration into factory's automation and control as well as provide means to be integrated into the factory's communication infrastructure. In the project SENROB at the IRF, several new communication techniques have been developed for flexible manufacturing cells. Fig. 11 shows a work cell and its various components that are integrated together to form a manufacturing cell.
Fig. 11: The flexible manufacturing cell
323 In fig. 11, a SCARA type robot and a 6 revolute axis type robot coordinately work on a process plan determined by the cell controller with the goal of minimizing the cycle time. The cell controller selects the appropriate process plan, after which the workpiece carrier is moved onto the transport system and is identified by the barcode reader. The cell controller is realized similar to the C I R O S action planning component, such that it represents a superimposed control level for the different components of a work cell. The components of the work cell receive their commands via the component drivers which provide standardized interfaces of the different components for the cell controller. Fig. 12 shows the overall communication structure of the flexible manufacturing cell which is an important aspect of the practical realization for application of industrial robots. The communication functions are implemented using standardized busses such as the InterBus-S or ProfiBus and the simple control functions are implemented via a SPS that is connected to the bus system.
Fig. 12:
The communication structure related to the flexible manufacturing cell
Due to the standardized communication mechanisms incorporated, the cell controller, as shown in fig. 12, can also request logistic services e.g. from an autonomously guided vehicle, which delivers new production materials. This is only one example illustrating the advantages
324
of the IRF's aim to connect all automation components in a consistent manner via networks and to provide the necessary interfaces and control structures to request special services as needed.
3.4 Autonomously guided vehicles For the material flow control in the frame of flexible manufacturing today, many autonomously guided vehicles (AGV) are available for use. Experience from the European Community's project PROMETHEUS demonstrated that the problems of the control and coordination of AGV transport systems are tightly related with the control of a multirobot systems. The AGV controller contains a route planning system, a level for multiple vehicle coordination and a level for collision detection and avoidance. It was obvious that the theoretical basis for the realization of the vehicle controller could be the principle of hierarchical coordinator [9], the same principle that served as an architectural design guide for the CIROS multirobot system. Using the standardized communication structure that was shown in fig. 12, the cell controller, submits a transport request. By means of either the sensory or model based provided path information, the vehicle's trajectory is generated by the control computer for all the vehicles within the control computer's movement zone while considering the physical constraints such as the speed and lateral and longitudinal accelerations. The partial steps of the trajectory determination are for example: 9 Detection of conflict situations 9 Generation of alternative maneuvers 9 Calculation and selection of optimal maneuver combinations The calculated desired paths are then transmitted to the vehicle, in our case by means of a communication link based on infrared light transmission.
Fig. 13:
The IRF's autonomously guided vehicle CAESAR
325 In addition to the control of the vehicle the exact determination of the position of the AGV is an extremely difficult problem. Because of the possible inertial and the odometric systems with unacceptable accuracy, a new landmark navigation system was developed on the basis of a 2D laser scanner [6] in the project CAESAR. A special feature of this method is that no artificial landmarks or induction wires are needed and that the process is very robust against visibility obstructions. For the application with the AGV using this navigation process, for example in a manufacturing plant, no structural steps are required such as the laying of induction wires. To initialize the environment model for the navigation system, only o n e vehicle has to be driven over the predetermined paths o n c e being controlled by hand. This defines the environment model that will then be transmitted to all the other vehicles which will then be immediately ready for action. Fig. 13 shows one of the three autonomously guided vehicles at the IRF equipped with this new navigation system. Above the laser, there is a hemisphere visible that contains the infrared transceiver for communication with the control computer.
4. C O N C L U S I O N In the scope of this paper the principle functionalities necessary for the realization of an intelligent autonomous robot system were discussed. Based on the discussion of the CIROS multirobot controller it was illustrated which functionalities are needed and how they can be implemented to ensure optimal cooperation. In addition, it was pointed out how the development of various sub-components are coordinated and how their requirements are defined, such that the necessary subsystem interfaces are suitable. Done properly, this allows the re-use of various components to economically generate a whole family of compatible automation products ranging from scaleable controllers to robot simulators and virtual-reality man machine interfaces. Reacting to the market requirements, new products can be quickly and effectively realized such as the presented PC-based robot controller PCROB and the PCbased robot simulation system COSIMIR. The control structure underlying the theoretical system principles of the hierarchical coordinator was briefly illustrated and the most important functions were presented; examples of these are the coordinated operation of robots and the collision avoidance. Besides the mentioned direct spin-offs for the space robotics, the experience from the development of the intelligent autonomous robot systems for the CIROS and ROTEX projects could be adopted for the development of control structures for an autonomously guided vehicle that serves as a flexible transport system in a factory environment. All the examples presented here demonstrate that in the components' development careful attention is paid to the maintaining of the interface definitions and when possible they use industrial standards for interfaces and communication protocols. The described realizations are characterized by high modularity and precise interfaces that guarantee that a desired automation system can be developed piecemeal and can be easily and clearly adopted to new requirements and technologies.
REFERENCES Freund, E., Fast Nonlinear Control with arbitrary Pole-Placement for Industrial Robots, The International Journal of Robotics Research, Vol. 1, No. 1, 1982.
326 2. .
.
Kaever, P., Kinematische und Taktile Koordination in Mehrrobotersystemen, Ph.D.Thesis, Institute of Robotics Research, University of Dortmund, February 1993. Kaever, P.; Kernebeck, U.; Pesara, J.; Rol3mann, J., Abschlul3bericht zum Projekt CIROS, Institute of Robotics Research, Dortmund, Germany, April 1991. Kernebeck, U., Action Planning for Multiple Robots in Space, The Fifth International Conference on Industrial Engineering Applications of Artificial Intelligence and Expert Systems, Paderborn, Germany, 1992. Freund, E.; Buxbaum, H.-J., Control of Robot-based Flexible Manufacturing Work Cells, Proc. Int. Conf. on Advanced Mechatronics, Tokyo, Japan, 1993, pp. 348-353. Freund, E.; Dierks, F.; Rogmann, J., Untersuchungen zum Arbeitsschutz bei Mobilen Robotern und Mehrrobotersystemen, Schriftenreihe der Bundesanstalt ffir Arbeitsschutz und Unfallforschung, Wirtschaftsverlag NW, ISBN 3-89429-297-0, 1993. Freund, E.; Hypki, A.; Uthoff, J.; van der Valk, U., COSIMIR und PCROB: Integration von Zellensimulation und Robotertsteuerung auf PCs, Fachtagung Intelligente Steuerung und Regelung von Robotern, Langen, Germany, November 1993. Freund, E.; Judaschke, U., Automated Vehicle Guidance as a Component of a Traffic Control System, Proceedings of the 2nd Prometheus Workshop, Stockholm, Sweden, October 1989, pp. 204-213. Freund, E.; Rogmann, J., A Generic Hierarchical Control Structure for a Large Class of Automation Problems: Theory and Applications, Proceedings of the International Workshop on Advanced Robotics, Beijing, China, August 1991.
10. Freund, E.; Rol3mann, J., Autonomous Operation of Multi-Robot-Systems in Factory-ofthe-Future Scenarios, Proceedings of the 6th International Conference on CAD/CAM, Robotics and Factories of the Future, Vol. 1, S. 269-274, London, Great Britain, August 1991. 11. Freund, E.; Rol3mann, J., Control of Multi-Robot-Systems for Autonomous Space Laboratory Servicing, Proceedings of the International Symposium on Artificial Intelligence, Robotics and Automation in Space, i-SAIRAS, Vol. 1, S. 443-453, Toulouse, France, September 1992. 12. Rol3mann, J., Echzeitf/ihige, kollisionsvermeidende Bahnplanung ftir Einzel- und Mehrrobotersysteme, 27. Colloquium on Control Techniques, Boppard, Germany, February 1992. 13. Rol3mann, J., Echtzeitf~ihige kollisionsvermeidende Bahnplanung far Mehrrobotersysteme, Ph.D.-Thesis, Institute of Robotics Research, University of Dortmund, Germany, February 1993. 14. Rol3mann, J: "Online Collision Avoidance for Multi-Robot-Systems, a new Security Methodology", Proceedings of the 6.th Topical Meeting on Robotics and Remote Systems, Monterey, CA, Februar 1995. 15. Rol3mann, J., Virtual Reality as a Control and Supervision Tool for Autonomous Systems, Proc. of the 4th International Conference on Intelligent Autonomous Systems, Karlsruhe, Germany, March 1995.
Intelligent Robots and Systems
V. Graefe (Editor) 9 1995 Elsevier Science B.V. All rights reserved.
327
Toward Integrated Operator Interface for Advanced Teleoperation under Time-Delay Antal K. Bejczy, Paolo Fiorini, Won Soo Kim, and Paul S. Schenker ~* ~Jet Propulsion Laboratory California Institute of Technology, Pasadena, CA, USA This paper briefly describes an Advanced Teleoperator (ATOP) system and its control station where a variety of computer-based operator interface devices and techniques are integrated into a functional setting, accomodating a primary operator and secondary operators. The overall sensing and computer aided system setting intends to include all perceptive components that are necessary to perform sensitive, dual-arm manipulation efficiently, focussed at non-repetitive and unexpected tasks. Computer graphics is a key operator interface component in the control station where new types of manual interface devices also are employed. The results of some generic and application task experiments are summarized, including the performance of a simulated remote satellite servicing task, carried out under four to eight seconds communication time delay, using satellite TV and Internet computer communication links. In conclusion, the paper highlights the lessons learned so far. 1. I N T R O D U C T I O N In general, teleoperation implies continuous human operator involvement in the control of remote manipulators. Typically, the human control is a manual one, and the basic information feedback is through visual images. Continuous human operator involvement in teleoperation has both advantages and disadvantages. The disadvantages become quite dramatic when there is an observable, two-way communication time delay between the operator and the remotely controlled equipment. Modern development trends in teleoperators control technology are aimed at amplifying the advantages and alleviating the disadvantages of the human element in teleoperation through the development and use of various non-visual sensing, intelligent or task-level computer controls, computer graphics or virtual reality displays, and new computer-based human-machine interface devices and techniques in the intormation and control channels between the operator and the remotely controlled manipulators. These development trends are typically summarized under the popular titles of telepresence and supervisory control technologies. In this paper, those two titles are lumped under the term advanced teleoperation. Several notes should be added here to the objective description of telepresence and supervisory control technologies. First, none of them eliminates the human operator from the control operation, but both change the operator's function assignments and employ *The research described in this paper has been carried out at the Jet Propulsion Laboratory, California Institute of Technology, under contract with the National Aeronautics and Space Administration.
328 human capabilities in new ways. Second, both technologies promise the performance of more complex tasks with better results. But, in doing so, both technologies also make a close reference to human capabilities of operators who will use evolving new devices and techniques in the control station. Third, both telepresence and supervisory control make reference to evolving capabilities or features of other technologies like sensing, high performance computer graphics, computerized electro-mechanical devices, algorithms-based flexible automation, expert systems for planning and error recovery, and so on. Thus, the progress in both technologies are tied to rich multidisciplinary activities. Fourth, both technologies need the evaluation and validation of their results relative to application environments. This paper is focused at the description and some practical evaluation of an integrated operator interface system for advanced teleoperation, developed at the Jet Propulsion Laboratory (JPL), during the past six to seven years, and exercised recently for realistic remote control experiments between JPL in California and the Goddard Space Flight Center (GSFC) in Maryland, 4000 Km away from JPL, using satellite TV and computer communication links between JPL and GSFC. First we describe the JPL Advanced Teleoperator (ATOP) system and its control station where a variety of operator interface devices and techniques are integrated into a functional setting, accomodating a primary operator and secondary operators. Then, we will summarize the results of some generic and application task experiments. In the third part of the paper, the JPL-GSFC simulated remote satellite servicing task, under communication time delay, will briefly be described. Throughout the paper we will emphasize the designs and use of the operator interface elements and their functional integration. In the concluding part of the paper, we will highlight the lessons learned so far. 2. A T O P A N D I T S C O N T R O L S T A T I O N The basic underlying idea of the JPL ATOP system setting is to provide a dual arm robot system together with the necessary operator interfaces in order to extend the twohanded manipulation capabilities of a human operator to remote places. The system setting intends to include all perceptive components that are necessary to perform sensitive remote manipulation efficiently, including non-repetitive and unexpected tasks. The general goal is to elevate teleoperation to a new level of task performance capabilities through enhanced visual and non-visual sensing, computer-aided remote control, and computer-aided human-machine interface devices and techniques. The overall system is divided into two major parts: the remote (robot) work site and the local (control station) site, with electronic data and TV communication between the two sites. The remote site is a workcell. It comprises: (i) two redundant 8-d.o.f. AAI arms in a fixed base setting, each covering a hemispheric work volume, and each equipped with the latest JPL-developed Model (3 smart hands which contain 3D force-moment sensors at the hands' base and grasp force sensing at the base of the hand claws, (ii) a JPL-developed control electronics and distributed computing system for the two arms and smart hands, and (iii) a computer controllable multi-TV gantry robot system with controllable illumination. This gantry robot currently accomodates three color TV cameras, one on the ceiling plane, one on the rear plane, and one on the right side plane. Each camera can be
329
Figure 1. The JPL ATOP Dual Arm Workcell with Gantry TV Frame
position controlled in two translational d.o.f, in the respective plane, and in two orientation directions (pan and tilt) relative to the respective moving base. Zoom, focus and iris of each TV camera can also be computer controlled. A stereo TV camera system is also available which can be mounted on any of the two side camera bases. The total size of the rectangular remote work site is" about 5 m width, about 4 m depth, and about 2.5 m hight. See Figure 1 for ATOP remote workcell. The control station site organization follows the idea of accomodating the human operator in all levels of human-machine interaction, and in all forms of human-machine interfaces. Presently, it comprises: (i) two general purpose Force-Reflecting Hand Controllers (FRHC), (ii) three TV monitor, (iii) a TV camera/monitor switchboard, (iv) manual input device for TV control, and (v) three graphics displays" one is connected to the primary graphics workstation (IRIS 4D/310 VGX) which is used for preview/predictive displays and for various graphical user interfaces (GUI's) in four-quadrant format; the second is connected to an IRIS 4D/70 GT workstation and is solely used for sensor data display; the third one is connected to a SUN workstation (SparcStation 10) and is used as a control configuration editor (CCE), which is an operator interface to the teleoperator control software based oll X-window environment. See Figure 2 for ATOP local control station.
330
Figure 2. The JPL ATOP Control Station
2.1. H a n d C o n t r o l l e r s
The human arm and hand are functionally both powerful mechanical tools and delicate sensory organs through which information is received from and transmitted to the world. Therefore, the human arm-hand system (thereafter simply called hand here) is a key communication medium in teleoperator control. With hand actions, complex position, rate or force commands can be formulated and very physically written to the controller of a remote robot arm system in all workspace directions. At the same time, the human hand also can receive force, torque, and touch information from the remote robot armhand system. Furthermore, the human fingers offer additional capabilities to convey new commands to a remote robot controller from a suitable hand controller. Hand controller technology is, therefore, an important technology in the development of advanced teleoperation. Its importance is particularly underlined when one considers computer control which connects the hand controller to the remote arm system. The direct and continuous (scaled or unscaled) relation of operator hand motion to the remote robot arm's motion behavior in real time through a hand controller is in sharp contrast to the computer keyboard type commands which, by their very nature, are symbolic, abstract and discrete (non-continuous), and require the specification of some set of parameters within the context of a desired motion. In contrast to the standard force-reflecting, replica master-slave systems, a new form
331 of bilateral, force reflecting manual control of robot arms has been implemented at the JPL ATOP project. The hand controller is a backdrivable six d.o.f, isotonic joystick. It is dissimilar to the controlled robot arm both kinemathically and dynamically. But, through computer transformations, it can control robot arm motion in six task space coordinates (in three position and three orientation coordinates). Forces and moments sensed at the base of the robot hand can back-drive the hand controller through proper computer transformations so that the operator feels the forces and moments acting at the robot hand while he controls the position and orientation of it. This hand controller can read the position and orientation of the hand grip within a 30 cm cube in all orientations, and can apply arbitrary force and moment vectors up to 20 N and 1.0 Nm, respectively, at the hand grip. The computer-based control system supports four modes of manual control: position, rate, force-reflecting, and compliant control in task space (Cartesian space) coordinates. The operator, through an on-screen menu, can designate the control mode for each task space axis independently. Position control mode servos the slave position and orientation to match the master's. In force-reflecting mode, the hand controller is back-driven based on force-moment data generated by the robot and sensor during the robot hand's interaction with objects and environment. The indexing function allows slave excursions larger or smaller than the 30 cm cube hand controller work volume. Rate control mode sets slave endpoint velocity in task space based on the displacement of the hand controller. This is implemented through a software spring in the control computer of the hand controller. Through this software spring, the operator has a sensation of the commanded rate, and the software spring also provides a zero-referenced restoring force. Rate mode is useful for tasks requiring large translations. Compliant control mode is implemented through a low-pass software filter in the hybrid position-force loop. This permits the operator to control a springy oi" less stiff robot. Active compliance with damping can be varied by changing the filter parameters in the software menu. Setting the spring parameter to zero in the low pass filter will reduce it to a pure damper which results in a high stiffness hybrid position-force control loop. The overall control organization permits a spectrum of operations between full manual, shared manual and automatic, and full automatic (called traded) control, and the control can be operated with variable active compliance referenced to force-moment sensor data. More on the hand controller and on the overall ATOP control system can be found in [1] through [6].
332
Figure 3. DATAHAND Switch Modules Integrated with FRHC Hand-Grip
333
1. Each module contains five switches. 2. Switches can give tactile and audio feectloack. 3. Switches require low strike force.
4. Switches surround finger creating differential feedback regarding key that has been struck.
Figure 4. Five Key-Equivalent Switches at a DATAHAND Fingertip Switch Module
The present FRHC has a single hand grip equipped with a deadman switch and with three function switches. In order to better utilize the operator's finger input capabilities, an exploratory project recently evaluated a design concept that would place computer keyboard features attached to the hand grip of the FRHC. To accomplish this, three DATAHAND TM [7] switch modules were integrated with the hand grip as shown in Figure 3. Each switch module at a finger tip contains five switches as indicated in Figure 4. Thus, the three switch modules at the FRHC hand grip contain fifteen function keys which directly can communicate with a computer terminal. This eliminates the need that the operator moves his/her hand from the FRHC hand grip to a separate keyboard to input messages and commands to the computer. A recent test and evaluation, using a mock-up system and ten test subjects, indicated the viability of the finger-tip switch modules as part of a new hand grip unit for the FRHC as a practical step towards a more integrated operator interface device to the ATOP system. More on this concept and evaluation can be found in [8]. 2.2. C o m p u t e r G r a p h i c s Task visualization is a key problem in teleoperation, since most of the operator's control decisions are based on visual or visually conveyed information. For this reason, computer graphics plays an increasingly important role in advanced teleoperation. This role includes: (i) planning actions, (ii) previewing motions, (iii) predicting motions in real time under communication time delay, (iv) to help operator training, (v) to enable visual perception of non-visible events like forces and moments, and (vi) to serve as a flexible operator interface to the computerized control system. The capability of task planning aided by computer graphics offers flexibility, visual
334
Figure 5. Schematic Layout of the TCE Interface
quality and a quantitative design base to the planning process. The capability of graphically previewing motions enhances the quality of teleoperation by reducing trial-and-error strategies in the hardware control and by increasing the operator's confidence in control decision making during task execution. Predicting consequences of motion commands in real time under communication time delay permits longer action segmentations as opposed to the move-and-wait control strategy normally employed when no predictive display is available, increases operation safety, and reduces total operation time. Operator training through a computer graphics display system is a convenient tool for familiarizing the operator with the teleoperated system without turning the hardware system on. Visualization of non-visible effects (like control forces) enables visual perception of different non-visual sensor data, and helps management of system redundancy by providing some suitable geometric image of a multi-dimensional system state. Last, but not least, computer graphics as a flexible operator interface to the control systems replaces complex switchboard and analog display hardware in a control station. The actual utility of computer graphics in teleoperation to a higher degree depends on the fidelity of graphics models that represent the teleoperated system, the task and the task environment. The JPL ATOP effort in the past few years was focused at the
335
Figure 6. Schematic Layout of the Hierarchical Data Interface
development of high-fidelity calibration of graphics images to actual TV images of task scenes. This development has four major ingredients. First, creation of high-fidelity 3-D graphics models of robot arms and of objects of interest for robot arm tasks. Second, high-fidelity calibration of the 3-D graphics models relative to given TV camera 2-D image frames which cover the sight of both the robot arm and the objects of interest. Third, high-fidelity overlay of the calibrated graphics models over the actual robot arm and object images in a given TV camera image frame on a monitor screen. Fourth, highfidelity motion control of robot arm graphics image by using the same control software that drives the real robot. The high fidelity fused virtual and actual reality image displays became very useful tools for planning, previewing and predicting robot arm motions without commanding and moving the robot hardware. The operator can generate visual effects of robot motion by commanding and controlling the motion of the robot's graphics image superimposed over TV pictures of the live scene. Thus, the operator can see the consequences of motion commands in real time, before sending the commands to the remotely located robot. The calibrated virtual reality display system can also provide high-fidelity synthetic or artificial TV camera views to the operator. These synthetic views can make critical motion events
336 visible that otherwise are hidden from the operator in a given TV camera view or for which no TV camera view is available. More on the graphics system in the ATOP control station can be found in [9] through [13]. The first development of a graphic system as an advanced operator interface was aimed at parameter acquisition, and was handled and called as a Teleoperation Configuration Editor (TCE) [14]. This interface used the concepts of Windows, Icons, Menus, and Pointing Device to allow the operator to interact, select and update single parameters as well as groups of parameters. TCE utilizes the direct manipulation concept, with the central idea to have visible objects such buttons, sliders, icons, that can be manipulated directly, i.e. moved and selected using the mouse, to perform any operation. A graphic interface of this type has several advantages over a traditional panel of physical buttons, switches and knobs: the layout can be easily modified and its implementation cycle, i.e. design and validation, is significantly shorter than hardware changes. The TCE (Figure 5) was developed to incorporate all the configuration parameters of an early single arm version of the ATOP system. It was organized in a single menu divided in several areas dedicated to the parameters of a specific function. Dependencies among different graphical objects are embedded in the interface so that, when an object is activated, the TCE checks for parameters congruency. A significant feature of this implementation is the capability of storing and retrieving sets of parameters via macro buttons. When a macro command is invoked, it saves the current system configuration and stores it in a function button which can later restore it. The peg-in-hole task, for instance, requires mostly translational motions but when holes have a tight clearance, compliance is necessary. An appropriate macro configuration is one that enables x,y and z axes, with position control in the approach direction and automatic compliance on the other two axes. This configuration can be assigned to a macro button and then recalled during a task containing a peg-in-hole segment. The continuing work on a graphic system as an advanced operator interface is aimed at the data presentation structure of the interface problem, and, for that purpose, uses a hierarchical architecture [13]. This hierarchical data interface looks like a menu tree with only the last menu of the chain (the leaf) displaying data. All the ancestors of the leaf are visible to clearly indicate the nature of the data displayed. The content of the leaf includes data or pictures and quickly conveys the various choices available to the operator. A schematic figure of this layout is shown in Figure 6. Parameters have been organized in four large groups that follow the sequence of steps in a teleoperation protocol. These groups are: (i) Layout, (ii) Configuration, (iii) Tools, (iv) Execution. Each group is further subdivided into specific functions. The Layout menu tree contains the parameters defining the physical task structure, such as relative position of the robots and of the FRHC, servo rates etc. The Configuration menu tree contains the parameters necessary to define task phases, such as control mode and control gains. The Tools tree contains parameters and commands for the off line support to the operator, such as planning, redundancy resolution and software development. Finally, the Execution tree contains commands and parameters necessary while teleoperating the manipulators, such as data acquisition, monitoring of robots, hand controllers and smart hands, retrieval of stored configurations and camera commands. This hierarchical data interface helps solving the problem of displaying the large amount
337 of data needed during a teleoperation task, but it does not address the issue of data entry by the operator. Traditional interfaces require several operators, using a different input device for each controlled function. A single-operator using this interface needs to use different input devices: joysticks to move the manipulators, buttons for camera and video control, keyboard and mouse for parameters entry. Operating these devices is time consuming and distracts the operator from the task. An alternative approach would be using the FRHC as the only input device for the operator. This scheme is similar to a virtual reality interface, where operators can access commands and data by simply moving their arms and hands. In our implementation, the operator would use the FRHC as pointing device on the interface menus and would use the FRHC trigger to click the selected button. The discrimination between commands for the robot and those for the data interface could be done by the d e a d m a n switch on the FRHC handle. When not pressed, this switch inhibits the generation of motion commands for the robots, but FRHC motion data are still available and are used to move the cursor. This scheme will be tested to validate its effect on operator performance. This scheme also can be combined with the DATAHAND TMswitch module integrated with the FRHC hand grip, discussed previously in this paper. 3. C O N T R O L
EXPERIMENTS
To evaluate advanced teleoperation capabilities, two types of experiments were designed and conducted: experiments with generic tasks and experiments with application tasks. Generic tasks are idealized, simplified tasks and serve the purpose of evaluating some specific advanced teleoperation features. Application tasks are simulating some real-world uses of teleoperation. 3.1. G e n e r i c T a s k E x p e r i m e n t s In these experiments, described in detail in [15], four tasks were used: attach and detach velcro; peg insertion and extraction; manipulating three electrical connector; manipulating a bayonet connector. Each task was broken down to subtasks. The test operators were chosen from a population with some technical background but not with an in-depth knowledge of robotics and teleoperation. Each test subject received 2 to 4 hours of training on the control station equipment. The practice of individuals consisted of four to eight 30-minute sessions. As pointed out in [15], performance variation among the nine subjects was surprisingly slight. Their backgrounds were similar (engineering students or recent graduates) except for one who was a physical education major with training in gymnastics and coaching. This subject showed the best overall performance by each of the measures. This apparent correlation between performance and prior background might suggest that potential operators be grouped into classes based on interest and aptitudes. The generic task experiments were focused at the evaluation of kinesthetic force feedback versus no force feedback, using the specific force feedback implementation techniques of the JPL ATOP project. The evaluation of the experimental data supports the idea that multiple measures of performance must be used to characterize human performance in sensing and computer aided teleoperation. For instance, in most cases kinesthetic force feedback significantly reduced task completion time. In some specific cases, however, it
338 did not, but it did sharply reduce extraneous forces. More on the results in [15,16].
3.2. Application Task Experiments These experiments were grouped around a simulated satellite repair task. The particular repair task is the duplication of the Solar Maximum Satellite Repair (SMSR) mission, which was performed by two astronauts in Earth orbit in the Space Shuttle Bay in 1984. Thus, it offers a realistic performance reference data base. This repair is a very challenging task, since this satellite was not designed for repair. Very specific auxiliary subtasks must be performed (e.g. a hinge attachment) in order to accomplish the basic repair which, in our simulation, is the replacement of the Main Electric Box (MEB) of the satellite. The total repair, as performed by two astronauts in Earth orbit, lasted for about three hours, and comprised the following set of subtasks: thermal blanket removal, hinge attachment for MEB opening, opening of the MEB, removal of electrical connectors, replacement of MEB, securing parts and cables, replug of electrical connectors, closing of MEB, reinstating thermM blanket. It is noted that the two astronauts were trained for this repair on the ground for about a year. The SMSR repair simulation by ATOP capabilities was organized so that each repair scenario had its own technical justification and performance evaluation objective. For instance, in the first subtask-scenario, performance experiments, alternative control modes, alternative visual settings, operator skills versus training, and evaluation measures themselves were evaluated [17,18]. The first subtask-scenario performance experiments involved thermal blanket cutting and unscrewing MEB bolts. That is, both subtasks implied the use of tools. Several important observations were made during the above-mentioned subtask-scenario performance experiments. The two most important ones are: (i) the remote control problem in any teleoperation mode and using any advanced component or technique is at least in 50% a visual perception problem to the operator, influenced greatly by view angle, illumination and contrasts in color or in shading; (ii) the training or, more specifically, the training cycle has a dramatic effect upon operator performance. It was found that the first cycle should be regarded as a familiarization with the system and with the task. For a novice operator, this familiarization cycle should be repeated at least twice. The real training for performance evaluation can only start after completion of a familiarization cycle. The familiarization can be considered as completed when the trainee understands the system I/O details, the system response to commands, and the task sequence details. During the second cycle of training, performance measurements should be made so that the operator understands the content of measures against which the performance will be evaluated. Note, that it is necessary to separate each cycle and repetitions within cycles by several days. Once a personal skill has been formed by the operator as a consequence of the second training cycle, the real performance evaluation experiments can start. A useful criteria for determining the sufficient level of training can be, for instance, that of computing the ratio of standard deviation of completion time to mean completion time (that is, computing the coefficient of variation). If the coefficient of variation of the last five trials of a subtask performance is less than 20%, than sufficient level of training can be declared. In the above-quoted subtask-scenario experiments, the real training, on the average, required one week per subject. More details on application task experiments can
339
Figure 7. Partially Recurrent Neural Network for Peg-in-Hole Task Segmentation
be found in [17,18]. The practical meaning of training is, in essence, to help the operator develop a mental model of the system and of the task. During task execution, the operator acts through the aid of this mental model. It is, therefore, critical that the operator understands very well the response characteristics of the sensing and computer-aided ATOP system which has a variety of selectable control modes, adjustable control gains and scale factors. The procedure of operator training and the expected behaviour of a skilled operator following an activity protocol offers the idea of providing the operator with performance feedback messages on the operator interface graphics, derived from a stored model of the task execution. A key element for such advanced performance feedback tool to the operator is a program that can follow the evolution of a teloperated task by segmenting the sensory data stream into appropriate phases. A task segmentation program of this type has been implemented by means of a Neural Network architecture [19] and it is able to identify the segments of a peg-in-hole task. With this architecture, the temporal sequence of sensory data generated by the wrist sensor on the manipulators are turned into spatial patterns and a window of sensor observations is
340 20 16
-12 -16 -20
I
I
2
I
I
4
I
I
6
I
I
8
I
I
10
I
I
12
I
I
14
I
I
16
I
I
18
I
I
20
I
I
22
I
I
24
IIII
26
III
28
30
IIII
32
34
36
Time (seconds)
Figure 8. Segmentation in the Real-Time Experiment for a Peg-in-Hole Task
associated to the current task phase. A Partially Recurrent network algorithm [20] was employed in the computation. Partially Recurrent Networks represent well the temporal evolution of a task, since they include in the input layer a set of nodes connected to the output units, to create a context memory. These units represents the task phase already executed - the previous state. A set of fixed weight connections have been established among the output and context layer units (see Figure 7). The Partially Recurrent network gave good results both in training and in simulation and it has been interfaced to the ATOP telemanipulator system for real-time tests. The ATOP configuration is significantly different from the one used for the training data and, therefore, these tests also verified the robustness of the approach to hardware variations. Figure 8 is the output of an experiment of the peg-in-hole task. Three curves are plotted in this figure: the X axis force signal input to the network, the real time output of the network (dotted line) and the off-line classification of the network (solid line). The dotted line shows the actual output of the classifier and the solid line is the the output of the off-line segmentation of the same data. The values of the segments in the two lines are the indices of the peg-in-hole phases, as described in [21]. On the solid line, phase transitions are synchronous with the corresponding data, since the data rate is determined by the processing speed of the network. The dotted line, instead, shows a lag between its phase transitions and the solid line ones, due to the low speed of the on-line segmentation. In the real time segmentation, the delay between corresponding transitions increases as a
341 function of the time elapsed from the beginning of the experiment, since samples arrive to the network at a much higher rate than their propagation speed through the network. Several experiments of the peg-in-hole task have been carried out and the results have been encouraging with a percentage of correct segmentations approximately equal to 65%. Approximately at time 29 s the network misclassified the end of the extract phase (index n. 8) that should have occurred at time 34 s. This can be explained by considering that the network was trained on the data of a single experiment and, therefore, it was very sensitive to changes in the duration and/or amplitude of the force signal. Furthermore, the tests also showed that the temporal dependency built into the network was not as strong as it should have been necessary. This weak temporal structure resulted in phases being classified in the wrong sequence. In other experiments, the network showed the capability of recovering after a classification error and the unexpected feature of classifying tasks whose phases sequence was significantly different from the expected one. During peg extraction, for example, the operator decided to regrasp the peg and the network correctly recognized the new sequence of phases by inserting an insertion, corresponding to the regrasp, within the extraction phase. 4. A T I M E - D E L A Y
CONTROL
EXPERIMENT
The benefits of integrated operator interface to sensing and computer control aided and computer graphics supported advanced teleoperation system become most convincing when the operation has to be performed under communication time delay. The technical meaning of integrated operator interface for such cases signifies two major features of the overall ATOP architecture of remote control and control station: (i) the operator, through high fidelity overlay of computer graphics images of work scenes (virtual reality) over TV camera images of the same work scenes (actual reality), can, with high visual fidelity, preview and predict the outcome of command and control actions in real time; (ii) the operator can, with high cognitive confidence, delegate some commands and control authority to the sensor-based closed loop remote control based on visual preverification of the expected action domain of that control loop. 4.1. C a l i b r a t i o n M e t h o d A high-fidelity overlay of graphics and TV images of work scenes requires a high fidelity TV camera calibration and object localization relative to the displayed TV camera view. Theoretically, this can be accomplished in several ways. For the purpose of simplicity and operator-controllable reliability, an operator-interactive camera calibration and object localization technique has been developed, using the robot arm itself as a calibration fixture, and a non-linear least-squares algorithm combined with a linear one as a new approach to compute accurate calibration and localization parameters. The current method uses a point-to-point mapping procedure, and the computation of camera parameters is based on the ideal pinhole model of image formation by the camera. In the camera calibration procedure, the operator first enters the correspondence information between the 3-D graphics model points and the 2-D camera image points of the robot arm to the computer. This is performed by repeatedly clicking with a mouse a graphics model point and its corresponding TV image point for each corresponding pair of points on a monitor screen which, in a four-quadrant window arrangement, shows both the
342
Figure 9. Graphics User Interface for Calibrating Virtual (Graphics) Images to TV Images
graphics model and the actual TV camera image. (See Figure 9). To improve calibration accuracy, several poses of the manipulator within the same TV camera view can be used to enter corresponding graphics model and TV images points to the computer. Then the computer computes the camera calibration parameters. Because of the ideal pinhole model assumption, the computed output is a single linear 4 by 3 calibration matrix for a linear perspective projection. Object localization is performed after camera calibration, by entering corresponding object model and TV image points to the computer for different TV camera views of the object. Again, the computational output is a single linear 4 by 3 calibration matrix for a linear perspective projection. The actual camera calibration and object localization computations are carried out by a combination of linear and non-linear least-squares algorithms. The linear algorithm, in general, does not guarantee the orthonormality of the rotation matrix, providing only an approximate solution. The non-linear algorithm provides the least-squares solution that satisfies the orthonormality of the rotation matrix, but requires a good initial guess for a convergent solution without entering into a very time-consuming random search. When a reasonable approximate solution is known, one can start with the non-linear algorithm directly. When an approximate solution is not known, the linear algorithm can be used to find one, and then one can proceed with the non-linear algorithm. More on the calibration and object localization technique can be
343 found in [22,23]. After completion of camera calibration and object localization, the graphics models of both robot arm and object of interest can be overlaid with high fidelity on the corresponding actual images of a given TV camera view. The overlays can be in wire-frame or solid-shaded polygonal rendering with varying levels of transparency, providing different visual effects to the operator for different task details. In the wire-frame format, the hidden lines can be removed or retained by the operator, dependent on the information needs in a given task. The calibrated virtual reality display system can also provide synthetic TV camera views to the operator. Synthetic TV camera views can make critical motion events or robot arm position and alignment states visible which otherwise would be hidden from the operator in a given TV camera view or for which no real TV camera views are available. The calibrated virtual reality method also enables the placement of calibrated 3-D target symbols over the live TV video helping the real-time visual guidance/control and real-time visual verification by the operator.
4.2. P e r f o r m a n c e R e s u l t s The performance capabilities of the high-fidelity graphics overlay preview/predictive display technique were demonstrated on a large laboratory scale in May 1993. A simulated life-size satellite servicing task was set up at GSFC and controlled 4000 Km away from the JPL ATOP control station. Three fixed camera setting were used at the GSFC worksite, and TV images were sent to the JPL control station over the NASA-Select Satellite TV channels at video rate. Command and control data from JPL to GSFC and status and sensor data from GSFC to JPL were sent through the Internet computer communication network. The roundtrip command/information time delay varied between four to eight seconds between the GSFC worksite and the JPL control station. The task involved the exchange of a satellite module. This required inserting a 45 cm long power screwdriver, attached to the robot arm, through a 45 cm long hole to reach the module's latching mechanism at the module's backplane, unlatching the module from the satellite, connecting the module rigidly to the robot arm, and removing the module from the satellite. The placement of a new module back to the satellite's frame followed the reverse sequence of actions. Four camera views were calibrated for this experiment, entering 15 to 20 correspondence points in total from 3 to 4 arm poses for each view. The calibration and object localization errors at the critical tool insertion task amounted to about 0.2 cm each, well within the allowed insertion error tolerance. This 0.2 cm error is referenced to the zoom-in view (fovy=8 ~ from the overhead (front view) camera which was about 1 m away from the tool tip. For this zoom-in view, the average error on the image plane was typically 1.2 to 1.6 % (3.2 to 3.4 % maximum error); a 1.4 % average error is equivalent to 0.2 cm displacement error on the plane 1 m in front of the camera. The idea with the high-fidelity graphics overlay image over a real TV image is that the operator can interact with it visually in real time on a monitor within one perceptive frame when generating motion commands manually or by a computer algorithm. Thus, this method compensates in real time for the operator's visual absence from reality due to the time-delayed image. Typically, the geometric dimensions of a monitor and geometric
344 dimensions of the real work scene shown on the monitor are quite different. For instance, an 8-inch long trajectory on a monitor can correspond to a 24-inch long trajectory in the actual work space, that is, three times longer than the apparent trajectory on the monitor screen. Therefore, to preserve fidelity between previewed graphics arm image and actual arm motions, all previewed actions on the monitor were scaled down very closely to the expected real motion rate of the arm hardware. The manually generated trajectories were also previewed before sending the motion commands to the GSFC control system in order to verify that all motion data were properly recorded. Preview displays contribute to operational safety. In order to eliminate the problem associated with the varying time delay in data transfer, the robot motion trajectory command is not executed at the GSFC control system until all the data blocks for the trajectory are received. An element of fidelity between graphics arm image and actual arm motion was given by the requirement that the motion of the graphics image of the arm on the monitor screen be controlled by the same software that controls the motion of the actual arm hardware. This required to implement the GSFC control software in the JPL graphics computer. A few seconds after the motion commands were transmitted to GSFC from JPL, the JPL operator could view the motion of the real arm on the same screen where the graphics arm image motion was previewed. If everything went well, the image of the real arm followed the same trajectory on the screen that the previewed graphics arm image motion previously described, and the real arm image motion on the screen stopped at the same position where the graphics arm image motion stopped earlier. After completion of robot arm motion, the graphics images on the screen were updated with the actual final robot joint angle values. This update eliminates accumulation of motion execution errors from the graphics image of robot arm, and retains graphics robot arm position fidelity on the screen even after the completion of a force sensor referenced compliance control action. The actual contact events (moving the tool within the hole and moving the module out from or in to tile satellite's frame) were automatically controlled by an appropriate compliance control algorithm referenced to data from a force-moment sensor at the end of the robot arm.
345
Figure 10. A. Predictive/Preview Display of End Point Motion. B. Status of Predicted End Point after Motion Execution, from a Different Camera View, for the Same Motion Shown Above.
346 The experiments have been performed successfully, showing the practical utility of highfidelity predictive-preview display techniques, combined with sensor-referenced automatic compliance control, for a demanding telerobotic servicing task under communication time delay. More on these experiments and on the related error analysis can be found in [22,23]. Figure 10 illustrates a few typical overlay views. A few notes are in place here, regarding the use of calibrated graphics overlays for time-delayed remote control. (i) There is a wealth of computation activities that the operator has to exercise. This requires very careful design considerations for an easy and user friendly operator interface to this computation activity. (ii) The selection of the matching graphics and TV image points by the operator has an impact on the calibration results. First, the operator has to select significant points. This requires some rule-based knowledge about what is a significant point in a given view. Second, the operator has to use good visual acuity to click the selected significant points by the mouse. 5. C O N C L U S I O N S The following general conclusions emerged so far from the development and experimental evaluation of the JPL ATOP: 1. The sensing, computer and graphics aided advanced teleoperation system truly provides new and improved technical features. In order to transform these features into new and improved task performance capabilities, the operators of the system have to be transformed from naive to skilled operators. This transformation is primarily an undertaking of education and training. 2. To carry out an actual task requires that the operator follows a clear procedure or protocol which has to be worked out off-line, tested, modified and finalized. It is this procedure or protocol following habit that finally will help develop the experience and skill of an operator. 3. The final skill of an operator can be tested and graded by the ability of successfully improvising to recover from unexpected errors in order to complete a task. 4. The variety of I/O activities in the ATOP control station requires workload distribution between two operators. The primary operator controls the sensing and computer aided robot arm system, while the secondary operator controls the TV camera and monitor system and assures protocol following. Thus, the coordinated training of two cooperating operators is essential to successfully use the ATOP system for performing realistic tasks. It is yet not know what a single operator could do and how. To configure and integrate the current ATOP control station for successful use by a single operator is a challenging R&D work. 5. The problem of ATOP system development is not so much the improvements of technical components and subsystems. Though, they also present challenges. The final challenge is, however, to integrate the improved technical features with the natural capabilities of the operator through appropriate human-machine interface devices and techniques to produce an improved overall system performance capability in which the operator is part of the system in some new way.
347 REFERENCES
[1] A.K. Bejczy and J.K. Salisbury. Controlling remote manipulators through kinesthetic coupling. Computers in Mechanical Engineering, 1(1), July 1983. also in Kinesthetic Coupling Between Operator and Remote Manipulator in Proc. ASME Int'l Computer Technology Conference, Vol. 1, San Francisco, CA, August 12-15, 1980, pp. 197-211. [2] A.K. Bejczy, Z. Szakaly, and W.K. Kim. A laboratory breadboard system for dual arm teleoperation. In Third Annual Workshop on Space Operations, Automation and Robotics, JSC, Huston, TX, July 25-27, 1989. [3] Z. Szakaly, W.K. Kim, and A.K. Bejczy. Force-reflective teleoperated system with shared and compliant control capabilites. In NASA Conference on Space Telerobotics, JPL Publication 89-7, Vol. IV, Jet Propulsion Laboratory, Pasadena, CA, January 31, 1989. [4] A.K. Bejczy, Z. Szakaly, and T. Ohm. Impact of end effector technology on telemanipulation performance. In Third Annual Workshop on Space Operations, Automation and Robotics, number NASA Conf. Publication 3059, JSC, Huston, TX, July 25-27, 1989. [5] A.K. Bejczy and Z. Szakaly. Performance capabilities of a JPL dual-arm advanced teleoperation system. In Space Operations, Applications, and Research Symposyum (SOAR'90), Albuquerque, NM, June 26, 1990. [6] A.K. Bejczy and Z. Szakaly. An 8-d.o.f. dual arm system for advanced teleoperation performa.nce experiments. In Space Operations, Applications, and Research Symposyum (SOAR '91), Houston, TX, July 1991. see also, Lee, S. and Bejczy, A.K., Redundant arm kinematic control based on parametrization, in Proc. IEEE Int'l Conf. on Robotics and Automation, Sacramento, CA, April 1991. [7] L.W. Knight and D. Retter. Datahandtm: Design, potential performance, and improvements in the computer keyboard and mouse. In National Human Factors Society Conference, Denver, CO, November 1989. [8] L.W. Knight. Single environment: Experimental hand-grip controller for ATOP. Jet Propulsion Laboratory, DOE Summer Faculty Fellow (SFF) Reports, July 31, 1992 and July 30, 1993. [9] A.K. Bejczy, W.S. Kim, and S. Venema. The phantom robot: Predictive display for teleoperation with time delay. In IEEE International Conference on Robotics and Automation, Cincinnati, OH, May 1990. [10] A.K. Bejczy and W.S. Kim. Predictive displays and shared compliance control for time delayed telemanipulation. In IEEE Int'l Workshop on Intelligent Robots and Systems (IROS'90), Tsuchiura, Japan, July 1990.
348 [11] W.S. Kim and A.K Bejczy. Graphics displays for operator aid in telemanipulation. In IEEE International Conference on Systems, Man and Cybernetics, Charlottesville, VA, October 1991. [12] W.S. Kim. Graphical operator interface for space telerobotics. In IEEE International Conference on Robotics and Automation, Atlanta, GA, May 1993. [13] P. Fiorini, A.K. Bejczy, and P. Schenker. Integrated interface for advanced teleoperation. IEEE Control Systems Magazine, 13(5), October 1983. [14] P. Lee et al. Telerobot configuration editor. In IEEE International Conference on Systems, Man and Cybernetics, Los Angeles, CA, November 1990. [15] B. Hannaford et al. Performance evaluation of a six-axis generalized force-reflecting teleoperator. Publication 89-18, Jet Propulsion Laboratory, June 15, 1989. [16] B. Hannaford et al. Performance evaluation of a six-axis generalized force-reflecting teleoperator. IEEE Transaction on Systems, Man and Cybernetics, 21(3), May/June 1991. [17] H. Das et al. Performance experiments with alternative advanced teleoperated control modes for a simulated solar max satellite repair. In Space Operations, Automation and Robotics Symposium (SOAR'91), JSC, Huston, TX, July 9-11, 1991. [18] H. Das et al. Performance with alternative control modes in teleoperation. PRESENCE: Teleoperators and Virtual Environments, 1(2), Spring 1993. MIT Press Publ. [19] P. Fiorini et al. Neural networks for segmentation of teleoperation tasks. PRESENCE: Teleoperators and Virtual Environments, 2(1), Winter 1993. MIT Press Publ. [20] J. Hertz, A. Krogh, and R.G. Palmer. Introduction to the Theory of Neural Computation. Addison-Wesley, Redwood City, CA, 1991. [21] B. Hannaford and P. Lee. Hidden markov model analysis of force-torque information in telemanipulation. International Journal of Robotics Research, 10(5), October 1991. [22] W.S. Kim and A.K. Bejczy. Demonstration of a high-fidelity predictive/preview display technique for telerobotics servicing in space. IEEE Transaction on Robotics and Automation, October 1993. Special Issue on Space Telerobotics. [23] W.S. Kim. Virtual reality calibration for telerobotic servicing. In IEEE International Conference on Robotics and Automation, San Diego, CA, May 1994. (Submitted).
Intelligent Robots and Systems V. Graefe (Editor)
9 1995 Elsevier Science B.V. All rights reserved.
349
A c t i v e U n d e r s t a n d i n g of H u m a n Intention by a R o b o t t h r o u g h M o n i t o r i n g of H u m a n Behavior Tomomasa SATO", Yoshifumi NISHIDA b, Junri ICHIKAWA ~, Yotaro HATAMURA ~ and Hiroshi MIZOGUCHI a "Research Center for Advanced Science and Technology, The University of Tokyo 4-6-1 Komaba, Meguro-ku, Tokyo, 153 Japan bFaculty of Engineering, The University of Tokyo 7-3-1 Hongo, Bunkyo-ku, Tokyo, 113 Japan ':Hitachi Inc. Understanding human intention is an essential function if a robot is to help adequately support human beings. It helps smooth communication between the human and the robot. Human behavior is an expressive media of communication. This paper proposes a new flmction, "active understanding of human intentions" by a robot through monitoring human behavior. The proposed flmction is unique because it utilizes multiple communication channels in parallel, i.e., human intentions are understood not only through conscious behavior but also through unconscious behavior. The paper also proposes a new robot architecture to realize the proposed function. The key architecture features are: 1) A robot possesses multiple sensors which surround the human. 2) Information is processed by dual loops-- a loop for information exchange between the human and the robot and a loop for human intention understanding. As an example of a robot with the human intention understanding functions, the authors constructed a micro-teleoperation robot. It can automatically understand an operator's intention through such unconscious human behavior as an operator's hand touching a desk. The understood result is utilized to change the control mode of a master-slave manipulation system from fine motion to rough motion and vice versa. The experimental results prove that the 15roposed function effectively simplifies operation system, making the system operator friendly. 1. I n t r o d u c t i o n Our aging society requires robots which can adequately support human beings. The support should be what the human wants, i.e., it should be in accord with the human's intentions. The authors refer to such a service robot as "a human-supporting robot." Robots which assist senior persons and robots which offer medical helping hands are typical examples of human-supporting robots. The human-supporting robot should be able to discern a human's intentions through comnmnication with the human, because it is impossible for the robot to adequately
350 support the human without knowing what the human wants. Understanding hmnan intention is a key function to realize human-supporting robots. The conventional robots have comnmnicated through input devices such as teachingboxes, keyboards, or switches to obtain information from the human. A common feature of such input devices is that the devices can obtain information only when the human consciously generates a behavior to operate these input devices. Although these devices have the merit that the information which the human doesn't intend to input can be suppressed consciously, they also have such the demerit that they only accept the human's conscious behavior. The demerit comes from the passive utilization of human behavior as a communication media. This imposes a burden on the human. Specifically, human operators must convert all their intentions into conscious behaviors to express their intention. Let us consider an example of writing a letter with a pencil. We usually move the pencil with one side of our hand touching a desk because we desire fine positioning. Moving the pencil is usually a conscious behavior. In contrast, touching the desk with the hand is usually an unconscious behavior. If a robot can automatically perceive such types of unconscious behavior, it can determine whether the human intends to move his or her hand in a fine motion control mode or a rough motion control mode. The example of the touching behavior indicates that unconscious behavior includes instructive information to convey a human's intention to the robot. Conventional robots have not utilized such behavior. Utilization of unconscious behavior requires the robot to actively perceive human behavior. Therefore, if a robot can more actively perceive a human's behavior as a conmmnication media, i.e., if the robot can perceive both conscious and unconscious behavior, the robot can understand human intention without forcing the human to teach all his or her desires consciously. While the conventional robot is a passive one which waits for input behavior as a command, a robot having the proposed function is an active one which understands human intention through monitoring human behaviors. This paper will demonstrate a method by which a robot actively recognizes expression of human intention through human behavior. This paper is organized as follows: The next section shows a model of communication between humans. The authors clarify the mechanism for understanding a partner's intention by the model. In addition, a robot function for active understanding of human intention is proposed. An architecture to realize the proposed function is described in section 3. In section 4, the authors present a microteleoperation robot as an example of the robot with the proposed function. The system effectiveness is proven experimentally. The conclusion is given in section 5. 2. U n d e r s t a n d i n g of h u m a n i n t e n t i o n In this section, we firstly analyze the function and mechanism of understanding of human intention through communication between human beings, based on the analysis, we present the robot's function for actively understanding human intention. 2.1. U n d e r s t a n d i n g of h u m a n i n t e n t i o n by h u m a n s Humans understand each other's intention by interpreting information received through communication. Therefore, a function for information exchange with the other person and a function for interpreting the received information are essential for understanding human
351 intention. We propose a model of communication between humans as shown in Fig. 1. We will use this model to analyze the mechanism of understanding of human intention by a human. Human understands the partner's intentions through communication via two information processing loops as shown in Fig. 1.
Figure 1. Loops in communication between humans
An information-exchange loop refers to a loop on which humans exchange information with each other. Information exchange is realized via language and behavior. An intention-understanding loop refers to a loop on which a human understands another human's intentions. Intention is understood based on knowledge concerning the partner (see models in Fig. 1) . Figure 2 shows the channels which convey information. The information flow from a speaker to a listener is illustrated in the diagram. The speaker's behavior is divided into two classes. The first class is the behavior generated when the speaker is conscious of the listener, the second is the behavior generated when the speaker is unconscious of the listener. The listener can obtain about 35% of the information by means of language and about 65% of it through nonverbal communication [1][2]. Following are the characteristics of communication channels which convey information in the information-exchange loop. Char. 1 H i e r a r c h y There exists a hierarchical structure among channels of communication between humans. Layer (1) L a n g u a g e c h a n n e l
352
Figure 2. Human communication channels conveying information
353 Language is the highest level and most flexible communication channel. A speaker is conscious of the listener's presence when the speaker takes expression. Layer (2) G e s t u r e c h a n n e l ( c h a n n e l of conscious b e h a v i o r w i t h c o n s c i o u s n e s s of a listener) Pointing at something by one's index finger is a typical example of a gesture. This layer generally provides a substitutional comnmnication channel for language. A speaker is trying to transmit information to a listener. Even information that can't be conveyed via language can often be communicated easily via gesture. Layer (3) C h a n n e l of conscious b e h a v i o r w i t h o u t c o n s c i o u s n e s s of a l i s t e n e r The listener can know that the speaker is hot by noting that the speaker fans his or her face with a fan. This is a typical example of conscious behavior without consciousness of the listener. This behavior is not assumed because the operator wants to send information consciously to the listener but because the operator wants to cool the speaker's face, i.e., the speaker behaves for his of her self. This channel differs from the previous two channels in term of consciousness of speakers. Since the speaker's behavior is not taken to convey information to the listener, the listener should intentionally look at and obtain information to communicate with the speaker. Layer (4) C h a n n e l of u n c o n s c i o u s b e h a v i o r Turning over in sleep or cat-napping are typical examples of unconscious behavior. A speaker doesn't send information to a listener intentionally, and the speaker isn't conscious of the behavior. The listener can understand the speaker's intention in the same way as (3). Layer (5) C h a n n e l of i n v o l u n t a r y b e h a v i o r or s y m p t o m s Behavior or symptoms in vital activity which have nothing to do with the consciousness. Breathing, sneezing, and sweating are typical examples of such behavior. A listener can understand a speaker's intention in the same way as the last two channels. Although behavior isn't always conscious behavior or unconscious behavior, i.e., conscious behavior can often change to unconscious behavior and vice versa, human behavior can always be categorized to a certain category if we take one moment into account. Char. 2 D i r e c t i v i t y There is directivity in comnmnication channels. The channels of layer (1) and (2) have directivity since the information is transmitted from the speaker toward the listener. However, the channels of layers (3) to (5) have no directivity since the information is not transmitted from the speaker toward the listener. The listener, therefore, has to actively watch a speaker's behavior to obtain information through layers (3) to (5). Human beings utilize not only directive communication channels but also indirective comnmnication channels through active recognition.
354 The following are characteristics for utilizing information which flows in the dual loops. Char. 1 Parallel utilization of multiple channels Human beings exchange information through multiple communication channels described above depending on the situation in parallel and interpret the received information in parallel. More specifically, human beings utilize multiple channels in the information-exchange loop and in the intention-understanding loop. Char. 2 Interpretation based on tacit rules Rules play an important role in interpreting the received information. For example, English language rules in channel (1) enable humans to interpret the spoken language. Communication in channels (2) through (5) is based on tacit rules which we know instinctively or as common sense. The inner square filled with dots of Fig. 2 represents common rules which make communication possible in a given culture. Considering the above characteristics, we can sunnnarize features of human intention understanding by human beings as follows. Human beings use the hierarchical channels fully by transmitting and receiving information through both the directive and the indirective channels in parallel. Humans can thus receive more information than transmitted consciously by the speaker, because humans can utilize unconscious behavior to receive information as well. Human beings understand the each other's intention by interpreting the received information based on the tacit rules. These features maximize the ability of human beings to understand each other's intention. 2.2. A c t i v e u n d e r s t a n d i n g of h u m a n intention by robot Based on the above analysis, we will propose essential functions of a robot which can efficiently understand a human's intention through communicating with the human to understand human intention, a robot must be able to exchange information with the human and to interpret information to achieve the same understanding as in communication between humans. Let us consider the problems of a conventional robot in terms of information exchange and information interpretation method. Figure 3 illustrates communication loops between a human and a conventional robot. Figure 4 shows communication channels between a human and a conventional robot in the loops. The problems of conventional robots can be summarized as follows. 1. Limited exchange of information Conventional robots receive only information from channels (1) and (2). For example, when an operator inputs commands from a keyboard, the operator uses only (1). In the information-exchange loop between a conventional robot and a human, unlike the information-exchange loop between humans, the robot imposes a burden on the human. Specifically, the human must consciously input all the information from sensors by converting the information of channels (3) and (4) into that of channels (1) or (2).
355
Figure 3. Loops in conmmnication between a human and a conventional robot
Such limited utilization of the channels occurs because the robot doesn't have sensors to receive information conveyed through the channels (3) to (5). 2. L i m i t e d i n t e r p r e t a t i o n of i n f o r m a t i o n Conventional robots interpret only information conveyed by channels (1) and (2). Therefore, the information conveyed by the channels (3) to (5) can't be processed even if received. In other words, conventional robots don't have the necessary function to interpret information from the channels (3) to (5). This is another reason humans have had to convert information of channels (3) and (4) into that of channel (1) or (2). A robot needs interpretation rules to interpret received information. The conventional robot establishes new artificial rules for interpretation which must be remembered by a human. Robot language is a typical example of such artificial rules. Figure 4 illustrates that a human and a robot interpret information by the artificial rules. The reason a beginner differs from a skilled worker is that the beginner is not accustomed to the artificial rules while the skilled worker is. The artificial rules established between a conventional robot and a human differ from the tacit rules in a human society. Therefore, the hulnan must remember complex operation methods. In the following, we will describe how to solve such problems. To solve the first problem, i.e., to overcome the limited receipt of information from a human, a robot should be able to receive information conveyed through all human comnmnication channels in parallel
356
Figure 4. C o m m u n i c a t i o n channels b e t w e e n a h u m a n and a conventional robot
357 channels (1) to (5). We refer to this function Function 1. To solve the second problem, i.e., to interpret received information, a robot should have a function to process all the information conveyed through the channels (1) to (5) in parallel. We refer to this function Function 2. The robot also requires a function to interpret the received information, not based on the artificial rules which must be learned by the human, but also based on tacit rules which are naturally known by the human. We refer to this function Function 3. (In this paper, "efficient" means there is a little the human must teach explicitly and consciously in order to convey his or her intention.) We propose a new function of "active understanding of human intention" which integrates Function 1 to 3 stated above. The function enables a robot to efficiently understand human intention through communication with the human. The main features of this function are described below. 1. A c t i v e n e s s We use the term "activeness" to mean that a robot with the proposed functions can receive information which a human doesn't transmit consciously. The robot thus receives information even through indirective channels. 2. U t i l i z a t i o n o f h u m a n b e h a v i o r as c o m m u n i c a t i o n m e d i a While conventional robots utilize only channels (1) and (2), the robot with the proposed function can utilize not only the language channel (channel (1)), but also the human behavior channels (channels of (2) to (5)) as communication media. The function for actively understanding human intention provides the advantages described below. I. E a s y c o m m u n i c a t i o n w i t h a r o b o t A human can smoothly conmmnicate with a robot because the robot can receive information which is not transmitted consciously. The human therefore need not convert the information of channels (3) and (4) to information of channels (1) and
(2). II. E a s y o p e r a t i o n for b e g i n n e r s A beginner can easily operate the system provided with the proposed function, because the system understands his or her intentions on the basis of tacit h u m a n rules. As a result, the beginner need not remember new artificial rules. As a practical example of robots with the "active understanding of human intention" function, let us explain a teleoperational robot by which the operator performs teleoperations while watching a CRT monitor. Depending upon the channels the robot utilizes, the robot can offer the various support below. 9 Utilization of channels (1) and (2) When the operator points his index finger at something on the CRT monitor and says to the robot, "bring me that bolt over there," the robot can understand the operator's intention from the hand gesture as well as the spoken language. The robot can then bring the desired object to the speaker.
358 9 Utilization of channel (3) When the operator leans toward the CRT monitor to manipulate something more precisely, the robot can display a zoomed image of the object by perceiving the behavior. 9 Utilization of channel (4) The robot can prevent the operator from doing something dangerous by watching him or her cat-napping. 9 Utilization of channel (5) The robot can keep the room at a comfortable temperature by monitoring perspiration on the operator's forehead. 3. Robot architecture to realize active understanding of h u m a n i n t e n t i o n This section proposes an information processing architecture for the robot with the function of active understanding of human intention. 3.1. I n f o r m a t i o n exchange loop and intention understanding loop We propose the information processing architecture shown in Fig. 5. The essence of the architecture is existence of two information processing loops, the information exchange loop and the intention understanding loop.
3.1.1. Information exchange loop: The robot exchanges information with the human via the information exchange loop. As shown in Fig. 6, information exchange for communication is realized by multiple communication channels ((1) to (5)). To utilize hierarchical human communication channels in parallel in the information exchange loop, it is necessary to equip the robot with appropriate input output devices such as sensors, actuators, and displays corresponding to each of the communication channels. 3.1.2. I n t e n t i o n understanding loop" The robot understands human intention by processing information in the intention understanding loop. The following processing is repeated continuously: acquisition of sensor data, understanding of behavior, understanding of intention, and selection of sensing mode. For each channel, there exist multiple intention understanding loops. All the loops run in parallel to realize hierarchical understanding of human intention. Information processing of each loop is based on the model. The models are described below.
Behavior model Using the behavior model, the robot interprets sensor data of human behavior and tries to understand the meaning of human behavior, i.e., it converts sensor data to the meaning of behavior.
359
Figure 5. Conceptual diagram of the information processing architecture
360
Figure 6. Parallel utilization of hierarchical communication channels
361
Intention model The intention model refers to the model which converts input behavior to human intention. It also contains future human intentions which can be predicted from the current intention in order to control sensing modes. Sensing model The sensing model is utilized to determine the sensing modes based on the predicted human intention. Sensing modes includes the kind of sensors to use in the future, the configuration of these sensors, and sensor data handling. Determination of the next sensing mode enables efficient sensor data processing by attention focusing. In addition to these models for active understanding of human intention, the human supporting robot should have a support model. This is the model by which the human supporting robot retrieves the support description based on the understood intention. Figure. 7 shows a detailed flow chart of active understanding of human intention.
Figure 7. Flow chart of active understanding of human intention
362 3.2. M u l t i p l e sensors c o n f i g u r a t i o n s u r r o u n d i n g a h u m a n We propose a new configuration (Fig. 8) for human supporting robots.
Figure 8. Multiple sensors surrounding a human
The configuration is such that the robot's multiple sensors surround the human. Surrounding the human by nmltiple sensors enables the robot to receive information from any desired directions. The information from multiple directions contains various aspects of human behavior. Consequently, the configuration makes the robot sensitive to the human behavior. The robot with the proposed configuration can communicate with the human differently from communication between humans because artificial robot sensors have the following advantages: 1) Some sensors can input data much more accurately than human sensors. 2) Some sensors can obtain data which cannot be obtained by human sensors, e.g. an infrared camera. 3) The robot can select a number and configuration of such sensors freely, e.g. six spherically arranged eyes using six cameras.
363
4. Application to Microteleoperation In this section, we present a microteleoperation robot as an example of a robot with the proposed function. 4.1. Microteleoperation A microteleoperation system is a master-slave manipulation system in which a slave robot handles very small objects according to an operator's movement. The size of handled object is on the order of micrometers[3]. In the system, a pen-shaped master manipulator, referred to a "pen-shaped master," is utilized as a master manipulator [4]. The slave robot executes tasks under a microscope. A CRT display monitor attached beneath a desk surface with its face up serves as the system human interface device. It displays the visual information from the slave robot microscope. The operator can perform teleoperation with the hand-held pen-shaped master while monitoring the working environment displayed on the monitor. Microteleoperation requires the slave robot to execute both fine and rough motions. Consider the task of micromachine assembly as an example. Before the assembly starts, the micromachine components should be conveyed into the microworking environment from the macro world. Therefore, the robot should have rough motion capability to move the object the long distance between the micro world and the macro world as quickly as possible. During the assembly, the robot should move the object with fine motion to align the object precisely. After the assembly, the assembled machine has to be moved from the working environment back to the macro world. The robot is again required to move the object a long distance by rough motion. In microteleoperation, it is necessary to change from the fine motion control mode to the rough motion control mode and vice versa frequently depending upon the working situation to execute tasks smoothly. In case of the conventional robot, the operator is forced to consciously change these modes. If a system can change these two modes automatically in accordance with the operator's intention by understanding the intention from an operator's unconscious behavior, i.e., by utilizing channels (3) and (4), the operator can perform microteleoperation smoothly because the operator is not required to consciously switch modes. 4.2. A p p l i c a t i o n of the active u n d e r s t a n d i n g of h u m a n intention A intention model, a support model, and a behavior model for the task of repairing a micro pattern are presented below. In this example, the operator makes the slave robot shave micro patterns painted on a sheet by maneuvering the pen-shaped master in his or her hand. (A) Intention model Here we will explain the relation between the operator's intention and the operator's behavior, in another word, the relation between the operator's intentions and motions of the operator's hand. Figure 9 shows the typical situations of the operator's hand. In (a), the operator's hand is floating in the air. In this situation, the operator intends to move operator's hand rapidly. In (b), the operator's hand is touching the desk. In this situation, the operator intends to execute the task precisely. These are the relations between the operator's behavior and their intentions. Such human intentions can be judged from
364
Figure 9. Typical situations of operator's hand
whether the operator's hand touches the desk or not. The intention model to judge the operator's intentions from his or her behavior is given as follows: (a) If the hand is not contacting the desk, the operator wants the slave robot to move quickly. (b) If the hand is touching the desk, the operator wants the slave robot to move precisely. (B) S u p p o r t m o d e l The slave robot supports the operator in accordance with the understood intention. When the operator wants to move the slave manipulator quickly, the system should set the slave robot control mode to rough motion control. When the operator wants to execute the task precisely, the system should set the fine motion control mode. The following support model realizes these activities: (a) If the operator wants to execute the task quickly, the slave robot should be in rough motion control mode. (b) If the operator wants to execute the task precisely, the slave robot should be in fine motion control mode. (C) B e h a v i o r m o d e l A touch-sensor panel is a suitable input device for obtaining the touch information between the hand and the desk. The following behavior model is constructed to recognize the operator's behavior from the output of the touch-sensor panel When there are no signals from the touch-sensor panel, the operator's hand is not contacting the desk as shown in (a) of Fig. 9. When there are signals from the touch-sensor panel, the operator's hand is touching the
365 desk as shown in (b) of Fig. 9. A flow chart of the intention-understanding loop in the microteleoperation robot can be summarized as Fig. 10
Figure 10. Flow chart of the intention understanding loop in the micro-teleoperation robot
4.3. E x p e r i m e n t a l S y s t e m Figure 11 shows the experimental micro-teleoperation system. The system consists of a human interface section, a slave robot and computers and controllers. 4.3.1. H u m a n i n t e r f a c e s e c t i o n Figure 12 shows a photograph of the human interface section. In this section, the information-exchange loop is formed between an operator and the system. The human interface section consists of a pen-shaped master manipulator, CCD cameras, a CRT display monitor, and a touch-sensor panel. The pen-shaped master is equipped with two
366
Figure 11. System configuration
367 LEDs to measure its three-dimensional (3D) position. Two CCD cameras are located above the CRT display monitor to obtain the image of two LEDs. The touch-sensor panel is attached to the screen of the CRT display monitor. The touch-sensor panel generates the output data indicating whether the surface is touched or not as well as the 2D touched position. The operator can perform the teleoperation with the pen-shaped master in the operator's hand while watching images of micro objects on the CRT display monitor. Note that the CRT display monitor screen is utilized as the desk surface so the operator can easily view the image as shown in Fig. 11. With the above input devices, the system can receive both conscious behavior of moving the pen-shaped master and unconscious behavior of touching the desk in parallel. In the system, the touch-sensor panel and CCD cameras are "the robot's multi-sensors surrounding a human" stated in the previous section. 4.3.2. Slave r o b o t Figure 13 shows a photograph of the slave robot. The slave robot consists of a slave manipulator, a work table, a stage, and a microscope. The slave manipulator has a handling tool (a knife) at its hand tip. The work table is equipped with a stage having three translational degrees of freedom. The nficroscope is located above the work table in a slave robot. 4.3.3. C o m p u t e r s a n d c o n t r o l l e r s The system computers consist of a host computer (Sun Sparc IPX), a real-time image processor (MaxVideo200) and transputers (T805). Transputers control the motion of the work table and the stage of the robot. The host computer integrates the system. The intention-understanding loop is implemented in the host computer.
Figure 12. Photo of hmnan interface
Figure 13. Photo of slave robot
368
4.3.4. System architecture The system realizes active understanding of human intention by the proposed architecture which consists of the information-exchange loop, the intention-understanding loop, and multiple sensors surrounding a human as shown in Fig. 14. Intention understanding loop (1) in Fig. 14 represents a loop in which the system understands the operator's desired direction. Intention understanding loop (2) represents a loop in which the system understands desired control mode. The two intention understanding loops work in parallel.
4.4. Experiment The following experiment was conducted. The operator shaves micropatterns on a sheet using the slave robot. The sheet is placed on the work table. Figure 15 shows a photograph of examples of patterns. The diameter of the dotted pattern is about 120 pro. Two identical sheets are prepared for the experiments. In the first experiment, the system is manually switched between fine motion control mode and rough motion control mode. In the second experiment, the proposed robot switches the system automatically. We measured the time to complete the task for six subjects. The subjects are taught how to manually change modes switch and how to use the touch-sensor panel for automatic changing in advance. Figures 16 and 17 show repairing times for each pattern in experiment (1) and experiment (2). These results indicate there is reproducibility in data because the same subject has almost fixed time for repairing, although there are differences among subjects. In order to clarify the difference between the two experiments, we compare repairing time by averaging the times for 6 subjects. Figure 18 shows the result. The results indicate that times of the second experiment (automatic control mode switching) are shorter than those of the first, though there are individual differences. The differences can be explained as follows. The operator need not consciously change modes because the system automatically changes them based on the operator's unconscious behavior. Consequently, the operator need pay attention only to the microteleoperation itself. The operator doesn't need to remember how to change modes because the modes are changed based on the operator's unconscious behavior and that unconscious behavior is based on tacit human rules.
369
r
Intention understanding / loop(2) Channel Obtain data from touch panel Understand behavior
vior Judge touching or floating of hand from sensor data.
Sensors surrounding operator
"F
Judge intention from situation of hand.
Understand intention
Touch panel CCD cameras
Select support mode
g
Select rough or fine motion control mode.
Conscious Channel uator and display Obtain data from CCD cameras Monitor
I'
Understand behavior
Support
Information exchange 100p
-~-Behavior model-~ C a l c u l a t e 3D position of pen-shaped master.
Move stage according to 3D positoin.
Intention under loop(l)
Micro-teleoperation robot Figure 14. System architecture
J
370
Figure 15. Example of dotted patterns on a Figure 16. Repairing time for each pattern sheet in experiment(1)
Figure 17. Repairing time for each pattern Figure 18. Comparison of repairing time for in experiment(2) experiment (1)and experiment (2)(average time of subjects)
371 5. C o n c l u s i o n This paper proposed a new function of "active understanding of human intention by a robot through monitoring of human behavior," which should be used by robots in order to efficiently understand human intention. Robots with this function can provide a human appropriate support based on understood intentions. The unique features of the proposed function are as follows: 1) The robot sinmltaneously receives information through nmltiple hierarchical human comnmnication channels of human behavior. 2) The robot understands human intention by interpreting the received information on the basis of tacit human rules. The authors also proposed a robot architecture to realize the function. The key features of the architecture are: 1) The robot is configured such that the robot's nmltiple sensors surround the human. 2) The robot has dual loops, an intention-understanding loop for understanding human intention and an information-exchange loop for exchanging information between the human and the robot. As an example of a robot having the proposed function and architecture, the authors constructed a micro-teleoperation robot. The robot switches between fine motion control mode and rough motion control mode, which is required frequently in micro-master-slave manipulation. The switching is performed automatically by interpreting the touch-sensor information which indicates whether the operator's hand is touching the desk or not. The hand touching the desk is a typical example of unconscious behavior of the operator. The experimental results proved that the function is useful for providing support based on the operator's intention and consequently in making the system friendly to the operator. Finally, the authors describe an example of a robot with the proposed active behavior understanding function for medical application. A patient is taken into an intensive care unit (ICU) when the patient is seriously ill or injured so that the state of the patient can be carefully monitored. Monitoring is performed by obtaining reliable data from many sensors directly attached to the patient. However, such monitoring imposes mental and physical burdens on the patient. A patient who is not so seriously ill or injured should be taken into a simple I CU which doesn't impose burdens on the patient. We are planning to construct "a robotic sickroom" as a simple type of intensive care unit (ICU). The robotic sickroom monitors the patient with the function proposed in this paper and provides the appropriate support. For example, it monitors a patient's turning over in sleep to prevent bedsores. It monitors unusual behaviors 1 and breathing motion to keep the patient's situation from becoming dangerous. The behaviors of turning over in sleep, unusual behavior, and breathing are other examples of unconscious human behavior. Understanding a patient's intention by recognizing such unconscious behavior enables the robotic sickroom to offer the functions of an ICU without imposing a burden on the patients. Future research should be devoted to modeling and processing such human behavior. The authors would like to express the sincere appreciation to Yoshinobu Kotani for his suggestions.
1Extracting a needle of an intravenous medication is a typical example. It arises f r o l n a side effect of a medicine or depression of brain.
372 REFERENCES
1. M. F. Vargas, "Louder than words-An introduction to nonverbal conmmnication-" Iowa State Unversity Press, 1987 2. K. Tachibana, "The structure of Communication", pp45-54, NTT-shuppan, 1993 3. T. Sato, K. Koyano, M. Nakao, Y. Hatamura, "Novel manipulation for micro object handling as interface between micro and human world," Proc. of IEEE/RSJ International Conference on Intelligent Robots and Systems, Vol3, pp1674-1681, July 1993 4. T. Sato, J. Ichikawa, M. Mituishi, "Micro teleoperation system concentrating visual and force information at operator's hand," Proc. of International Symposium on Ezperimental Robotics, pp118-124, Oct 1993
Intelligent Robots and Systems V. Graefe (Editor) 9 1995 Elsevier Science B.V. All rights reserved.
373
Human/machine sharing control for telerobotic systems* Tzyh-Jong Tarn a, Ning Xi a, Chuanfan Guo a, and Antal K. Bejczy b a Department of Systems Science and Mathematics, Washington University, St. Louis, MO 63130, U.S.A. bJet Propulsion Laboratory, California Institute of Technology, Pasadena, CA 91109, U.S.A Telerobotic systems usually consist of autonomous operations and master/slave operations. Based on our newly developed event-based planning and control theory, a new action planning and control scheme for the telerobotic system is presented. It provides a unified model to integrate a human operator control command (supervisory command) with motion planning and control for automatic operation. As a result, supervisory command and autonomous motion planning and control are naturally combined together such that the autonomous motion plan can be adjusted and modified by a human operator during its execution. This scheme lays down a foundation for planning and control of a general robotic system involving human operators. The scheme is implemented and tested on a PUMA 560 dual-arm system. A Spaceball is used by the human operator as a command generator to input the supervisory command. The experimental results of both single arm teleoperation and dual-arm coordinated teleoperation are presented. 1. I N T R O D U C T I O N Applications of the teleoperation of robotic systems range from material handling and automation in hazardous environments to servicing tasks in space. A telerobotic system can be commanded by manually generated motion commands, preplanned motion commands or a combination of both commands. Generally, manually generated commands are obtained from command generators manipulated by human operators in real-time. But preplanned commands are given before the motion execution. The preplanned commands are sometimes called autonomous commands or autonomous functions. Since autonomous functions may not work perfectly in unexpected situations, intervention by the human operator is often needed. The focus of this paper is to present a new approach to the planning and control of telerobotic operations, such that the telerobotic system has the ability to cope with some unexpected events. In order to interface a human operator with a robotic system, many planning and control schemes for telerobotic systems have been proposed. Among them, *Research Partially supported under NSF Grant IRI-9106317, CAD-9404949 and by Sandia National Laboratories Contract No. AC-3752-C.
374 the master/slave type of telerobotic system was intensively studied in [5, 6][8]- [12]. In addition, different kinds of semiautonomous control of the telerobotic system were presented in [8]. The teleoperation of a redundant manipulator is discussed in [9], and the importance of coordination between the human operator and remote robot is addressed in [10]. Obstacle avoidance by using semiautonomous control approach is presented in [11, 12]. In existing planning and control schemes for teleoperations, autonomous functions and manual operations are separated into two different operation modes. Based on some logic conditions, the operation modes can be switched between each other. However, if some unexpected event, such as occurrence of obstacles etc., happens in the autonomous operation, the system has to be switched to the manual mode. After proper manual operation, complete replanning of the autonomous operation is required to switch the system back into the autonomous mode, since the autonomous motion plan is usually described as a function of time, and the switches do not happen at some predefined time. This significantly limits the capability of the telerobotic system to deal with unexpected events. In addition, due to the uncertainty of the switching time caused by unexpected events, it could be extremely difficult to achieve stable and smooth switching between the different operation modes. In order to overcome the above disadvantages, a new event-based planning and control scheme is introduced for autonomous operations. It can be easily adjusted or modified by a human operator, through a command generator, during the operation. As a result, the autonomous and manual operations are naturally combined, an problems caused by mode switching are avoided. The proposed method is applied to single arm, as well as to dual-arm teleoperations. Finally, experimental results for single arm action and for dual-arm material handling are presented. The ultimate goal of this paper is to develop a new planning and control scheme for a telerobotic system to achieve an intelligent and robust performance. 2. E V E N T - B A S E D
PLANNING
AND CONTROL
THEORY
The basic idea of the event-based planning and control theory [3] is to introduce the concept of an action reference parameter. This parameter is directly related to the system output measurement relative to the task. Instead of being parameterized by time, the plan/desired input to the system is parameterized by an action reference parameter. As a result, for any given instant of time, the desired input is a function of the system output. This creates a mechanism for adjusting the base plan based on the output measurement, thus converting the planning to a closed loop real-time process. That is, the plan (the desired action) is an "abstract" entity, like a "desired goal". It only becomes a "real" plan when measurements are made. The event-based planning and control scheme is compared with the traditional time-based planning and control scheme in Figure I. In the event-based planning and control scheme, the function of the Action Reference block of Figure Ib is to compute the action reference parameter s on-line based on system output measurement. The planner then gives the desired input to the system according to the on-line computed action reference parameter s. In addition, the action reference parameter is calculated at the same rate as the feedback control. In other words, the
375
planning process is adjusted at a very high rate, which enables the planner to handle unexpected or uncertain discrete and continuous events together with the control execution.
Yd(t)
[Time-based[
y(t)
e(t)]
...~ K~t ~ ' ~-~/I
a. Traditional Planning and Control Scheme
yd(s) Event-based] "Y.'~ e(~[ Planner ~ C o n t r o l l e r
s1 !
" ._|
|
y(t) ~
Robot
Motion
Reference
L [--..
b. Event-Based Planning and Control Scheme
Figure I. Comparison
of Traditional and Event-Based
Planning and Control Schemes
As an example, the event-based planning and control scheme has been successfully applied to a robot arm motion in which the arm was stopped by an unexpected obstacle [3]. Since the desired input of the system is directly related to the output through an action reference, the error remained constant while the robot arm stopped. Once the obstacle is removed, the robot arm automatically continues its motion, according to the original plan, in a stable manner without replanning. In this case, if the conventional time-based motion plan were implemented, the system error would continue to increase, since the planner would give the desired input according to the original fixed plan. As a result, the system would become unstable. In order to apply the event-based planning and control theory to the telerobotic system, the most important thing is to integrate the human operator command with the action reference parameter [7]. As a result, the action plan will be based on sensing information in an autonomous operation, or in a manual operation on human operator command. This provides a unified planning and control scheme for operations which are commanded by both a human operator and an autonomous motion plan. 3. C O N T R O L
STRATEGY
FOR SINGLE ARM TELEROBOTIC
SYSTEM
A single arm telerobotic system mainly consists of a robot arm and a command generator. It can perform simple teleoperations such as master/slave operation, as well as complicated teleoperations such as object avoidance, which combines both autonomous functions and master/slave commands. "Simple teleoperation" means that the commands
376 of teleoperation are either from a human operator or from some autonomous motion plan. "Complicated teleoperation", on the other hand, is a combination of human operator commands and of autonomous motion plan.
3.1 Robot
dynamic
The dynamic
model
model for a robot arm can be written as
D(q)~ + C(q, (1) + G(q) = T
(1)
The task space is defined as Y c R 6, Y=
(x,y,z,O,A,T)
T,
where (x, y, z) is the position of the tip in the Cartesian space, and (O, A, T) is an orientation representation (Orientation, Altitude, Tool angles) [4]. The relation between Y and joint angle q is Y = h(q)
(2)
Therefore we have = Jh(q)(1
(3)
where Jh(q) is the Jacobian matrix. The robot dynamics in task level can be described as follows D ( q ) J h l ( q ) ( Y - Jh(q)(1) + C(q, (t) + G(q) = 7
(4)
The following nonlinear feedback [3] T = n ( q ) J [ l [ Y d + K,,(Y d - 9 ) + K p ( Y d - Y) + Jh(q)q] + C(q, (1) + G(q)
(5)
is applied to obtain a totally linearized and decoupled error dynamics, (?d _ ? ) + K~(?d _ ? ) + K~(yd _ y ) = 0
(6)
where yd, ~zd, ~d are the desired position, velocity and acceleration, respectively, and Kv, Kp are the velocity and position feedback gains which can be designed to stabilize the system. 3.2 C o m m a n d g e n e r a t o r Master robots which are kinematically similar to the telerobots (slave robots) are often used as command generators [II]. Another kind of command generator is called Position Hand Controller [5, 6]. In this paper, the Spaceball (Figure 2), is used as a command generator. Both Position Hand Controller and Spaceball are not kinematically similar to the telerobots. The Spaceball can generate operation commands in 6 directions. From a
377 manipulation point of view, the end-effector of a robot should follow Spaceball commands generated by a human operator.
Figure 2. Command Generator: The Spaceball Scaled sampled data from the Spaceball are used as the desired velocity of end-effector motion in Cartesian space, which consist of translation velocity and angular velocity. The Spaceball commands can be used as the desired velocity of the end-effector since only the relative motion of the end-effector is important. The desired position and acceleration of the end-effector can easily be obtained by integration or differentiation of the desired velocity. Assume the Spaceball output after scaling is Vx ~3y
~
=
~z
(7)
COx COy COz
Notice that Vsp is described as a function of the motion reference s [3]. A human operator gives a proper command according to his/her observed robot position and orientation. In this sense, the operator plays the same role as a sensor in the event-based planning and control scheme [3]. The desired velocity of the end-effector can be obtained as follows ~/'id -- J1Vs p
(8)
where J1 is the Jacobian matrix from Cartesian space to task space and is described by
[4] J1 = [/a•
[ 0
where
0
K~r
V~p
(9)
378
1] CA -So 0 Co 0
SoSA COSA
KE(~OAT---
CA --Co _So CA
CA
which represents the relationship between the OAT derivatives and angular velocities. The desired position change of the end-effector can be determined by: =
and the desired acceleration is
]/';dp=
jiVs p -.Ji- ,.]19sp
For a simple human operator control, i.e., a teleoperation without an autonomous motion plan, AY~%, ]?~d, and ~;d will provide the robot arm with all the necessary motion commands. 3.3 S e m i a u t o n o m o u s c o n t r o l for t e l e r o b o t i c s y s t e m If the robot arm is moving along a preplanned trajectory and some obstacle occurs in the given path, a scheme must be designed so that the human operator can temporarily stop the robot motion to wait for the clearance of the obstacle, or, by superposing an additional command through the Spaceball, change the motion to avoid the obstacle. Let the autonomous function of robot motion be described by yd(s), Yd(s), Yd(s), and the supervisory command from Spaceball is Ayd(s), yd(s), Y~d(s), where s is the motion reference parameter. To obtain a semiautonomous function which combines the attonomous function and the supervisory command to achieve a complicated teleoperation, the following scenarios have to be considered: (1) After the robot motion is either stopped or slowed down with the Spaceball, the robot motion defined by an autonomous motion plan can resume after the obstacle is removed and the Spaceball is released. (2) Switch between autonomous and master/slave operations. (3) Semiautonomous control, superposing the Spaceball commands to an autonomous motion plan to achieve complicated teleoperations. Notice that case (1) is a special situation of case (3). Case (2) is currently a common operation in telerobotic systems. Based on event-based planning and control schemes, semiautonomous control for the above cases can be easily implemented. Let the command motion direction from the autonomous motion plan be ]~d, and the command motion direction from the Spaceball be Y~p. "d If the autonomous motion plan needs some modification by a human operator command to ensure proper robot motion, the following two conditions may be considered: (1) The projection of ]2~%on the positive direction of ]?d cannot make the total velocity of robot motion in this direction exceed the maximum velocity limit of the robot arm.
379 (2) The projection of ])~d on the negative direction of yd must be modified so that the motion reference s keeps nondecreasing to ensure the stability of the motion [3].
~Pd~k. d ~
lisp2
Sf
So
(a) cos ~ >= 0 yd
d
Ysp2
sf
,
y;d So
(b) cos ot < O
Figure 3 Semiautonomous Commands for Single Arm Teleoperation Let c~ be the angle between ]~d and ]~d in Figure 3, and
~/d (10)
,pl
Ii?dll
~d
" d __ y d
sp2 ~ Ysp
(11)
spi
The semiautonomous command is defined as follows
{ ?..., !cos >0
"d
Yip,~ =
"d
"d
<
]zip, IY;pll I _ I17~11, ~os~ < 0 ?d,~. _ ?d II?~ I > II?d[L, COS ~ < 0
The desired position change of the end-effector will be A Y 4 , s = Ji Vsp,sAt
and the desired acceleration will be "'d ~/'sp,s
__
J1 Vsp,s Jr- Jl Vsp,s
(12)
380
Notice that the desired position command from the Spaceball is actually a relative position increment command superposed on the autonomous function yd. The total desired input is then/kY~dp,~ + yd, Ys d "t" ~/'d and ~2"sp,s-Jc ~.zd.
Spac .'Ball
_. e, er
gobo, I , Controller ~ - Motion Reference ]--i. Human Operator ,
I~--. I-"
Figure 4. Telerobotic System with Semiautonomous
Control
Remark:
In this case, if the autonomous commands are functions of time, it cannot be so easy to combine the Spaceball commands with the autonomous commands because the Spaceball commands are naturally functions of event: the given path of robot motion. Figure 4 shows the structure of single arm telerobotic system with autonomous control.
4. C O N T R O L
STRATEGY
FOR
DUAL-ARM
TELEROBOTIC
SYSTEM
The dual arm telerobotic system consists of two robot arms and a command generator. In this system, under the command of a human operator, two robot arms can manipulate an object in 6 dimensional task space with automatic coordination. 4.1 R o b o t d y n a m i c m o d e l s The dynamic model for each robot can be written as
Di(qi)qi + Ci(qi, qi) + Gi(qi) + J~(qi)fi = "ri, i = 1,2
(13)
where the term J~(q~)f~ represents the interaction force between robots and the object, and Jh~ is the Jacobian matrix and is defined as
381
= Jh~(qi)(ti,
i = 1,2.
where Yi = hi(qi), i = 1, 2, are task space outputs of each robot. The robot dynamics in task space can be described as
Di(qi)j~X(qi)(Yi - Jh (qi)oi) + C~(q~, Oi) + Gi(qi) + j T (qi)fi
7-i
(14)
The following nonlinear feedback with the consideration of force/torque compensation [7] is applied for each robot
Ti = Di(qi) J -h, l [Yid nt- I(v ~. ( t d -- Yi ) -Jc-I(p ~. ( Yi d -- Yi ) +Jh~(qi)oi] + Ci(qi, 0~) + Gi(qi) + j Thi( q i ) f i
(15)
Then a totally linearized and decoupled error dynamics can be obtained for each robot,
(Yi a - ~ ) + Kiv(Yi a - Yi) + Kw(Yi a - Yi) = 0
(16)
where Kiv, Kip can be designed to stabilize the system.
4.2 D u a l - A r m coordinated teleoperation In a dual-arm telerobotic system, it is extremely important to achieve coordinated control. Based on the event-based planning and control scheme [3], the automatic coordination in a teleoperation could be easily achieved.
, v
?,t ......
iR~ I V///•
I I,,i- K.
R~176 "--
////Z
Figure 5. Coordinate Frame Systems of Dual Arm Robot System In Figure 5 , Kw is the world coordinate frame, and robot 1 and robot 2 hold an object by squeezing. It is assumed that the two robots manipulate an object with coordination and there is no relative motion between each robot end-effector and the object. Therefore the following constraints must be satisfied
382 (i) Position constraints 0 pod = pd + Rid (pob _ po),
i=1,2
where pod = (xdb, yo%, Zod)T is the desired position of the object in the world coordinate frame; P~ = (X~ Yob,~Zobo)T is the initial position of the object in the world coordinate frame; pd, p0 and pd , p0 are the desired position and initial position of robot 1 and robot 2 in the world coordinate frame; R1d and R~ are the desired rotation matrix of robot 1 and robot 2 in the world coordinate frame. (ii) Orientation constraints
( R d ) - l R d - (RO)-lRO and
(.Rdb)-l.R d ---(.ROb)-lR o where Rdb and ROb are the desired rotation matrix and the initial rotation matrix of the object in the world coordinate frame; R ~ and R2~ are the initial rotation matrix of robot 1 and robot 2 in the world coordinate frame. Without loss of generality, it can be assumed that the initial rotation matrix of the object is equal to that of robot I, i.e. ROb -- R ~ Then the desired orientation of the object and the desired orientation of robot 1 should remain the same all the times, i.e. R d = Rod. The initial position of the object can be assumed to be the center of the straight line from Y1~ to Y2~ i.e. pOob- 7l(p0 + p0) The sampled data from the Spaceball after scaling is used as the desired velocity of the object motion in Cartesian space. Assume the Spaceball output after scaling is Vsp. The desired velocity of the object is
?g =
Jl%
The desired position increment of the object is = J1y
At
and the desired acceleration is
Using the constraints, the desired position, velocity and acceleration for each robot can be obtained. The goal is to achieve the best possible coordination. In the event-based motion plan, the desired inputs are described by functions of motion reference s, which should closely relate to the goal of control. In other words, the problem is to determine s such that the position constraints are satisfied and also minimize the following coordination criterion
J = klllPld(s)- Pill 2 + k~.llP~(S ) -p~.ll ~
(17)
383 where kl and k2 are weighing coefficients which depend on the importance of coordination requirements in different directions and are usually determined by the task itself. The closed form solution can be obtained by solving
OJ =0 Os subject to constraints (i)[7]. 5. E X P E R I M E N T A L
RESULTS
The above planning and control schemes for telerobotic systems were experimentally implemented and tested on two PUMA 560 6-DOF robot arms. Each robot arm is equipped with FSA-3254 6-Axis force/torque sensor. The planning and control algorithms were implemented in a SGI 4D/340 workstation. A graphical user interface has been developed. It can display sensory information and provide a means for the human operator to adjust the system parameters.
.-~ 1 ooolE /
4
'.
g s0o~.--.-..---..--..'.----.---=---.i --.- x - - - -
~, ...........
!. . . . . . . . . . . . . .
i .... Y - - -
i
i
z ....
.
,,_ _
===========================
a.
0
500
.....
Y--" z ....
:
i
i
:
5
10
2- i
.
i~
.
r, ~ ! ~ ' : ''! ' :
.
'
..
-2 0
15
0
5
time (sec)
,000
10
15
time (sec)
,
] ,oo01
....
g 50oFz.---.-..=.-.-.'...=-.-..---.-~ .. x - - ' ]
"
500 ! ............. 0
i ..........
':i
5
.... .....
10 time (sec)
15
0
9
0
5
10
15
time (sec)
Figure 6. Single Arm Obstacle Avoidance Figure 6 shows the experimental results of single-arm-teleoperation. At the beginning, the arm was commanded by an autonomous motion plan to move along a given straight line. While an unexpected obstacle was moving toward the given path of the arm, an operator used the Spaceball to superpose an additional command to the autonomous motion plan. The arm stopped the motion, and waited until the obstacle passed the given path. Then, after the operator released the Spaceball, the robot arm continued its original autonomous motion without any replanning. It can be seen that the robot arm undergoes
384 smooth transfer between "simple teleoperation" and "complicated teleoperation" to cope with the unexpected event 9 For both modes of operation, highly accurate control has been achieved 9
E~"1.5f
.
,
,.
........ ', .......... i ........
-
L~
1"]"
i
~0.5[..... 8 o[
i
.
i ........
"
:
"~" 1ooo1,,
E
o
i
i
i
5
10
15
20
.
,
,
!
!
i
I
! Y--"
soo[ . . . . . . . . . .
13-
:
0
,
/
i. x "
;..i ......
!
!
..-..-.- ..'-,_.~..::.~._.._.._._ .... !
I.......... ! z
8-soo ~
'~"
0
time (sec)
..~ . . . . . . . . . .
,
5
!
i ........
,
10 time (sec)
15
I
l
l
9 9 9
. . .
9 9 9
o
.... ,
20
A
~.2,
',
,
,
"
I
"- 1[I .......... ~ i..........i....... i...... t_ ~. 1 ....i-T iO E
,
9
.
~w"
0
.
.
.
.
.
.
.
5
:
.
10
15
.
.
.
.
.
.
9
i x~
o o
0
. 9
y--.
9 9
~-50 20
!
0
|
5
.
9. . ,
.
!." .... ! "'..
: .
l..,
. . . .
.2 ........ ! .:
"....'.." o
9
9
i
i
10
15
20
time (see)
time (sec) Figure 7. Dual-Arm
.
- !-: ......
Teleoperation
In Figure 7, a carton weighing 0.45kg, was manipulated by dual arm teleoperation system as shown in Figure 5. The motion command was generated by a human operator through the Spaceball. Dual arm coordination was automatically maintained 9 The coordination error was defined as e~oo~d
--
IlPl(S)
-- m(~)ll
-- IIP~(S) -- P~(S)II
which is the difference between the actual and the desired distances of two contact surfaces. The experimental results show that high accurate coordination has been achieved in the teleoperation. 6. C O N C L U S I O N A new action planning and control scheme for the telerobotic system has been proposed in this paper. Based on the event-based planning and control theory, a unified model has been developed to integrate the human operator command with the autonomous motion plan. This makes it possible for human operator to adjust or modify the autonomous motion plan on-line to cope with unexpected events. It significantly improves the robustness of the teleoperation. In addition, the method provides a scheme to automatically coordinate the dual-arm in a teleoperation. This significantly reduces the human operator's required operating skill level. The experimental results presented in this paper verify
385
the theoretical results and clearly demonstrate their advantages. More importantly, the proposed methods provide a mechanism for integrating various sensor information with human operator commands. The methods could easily be extended to a general theory for modeling, analyzing and designing a human/machine interaction system. This will be an important step in the advancement of the development of an intelligent machine. REFERENCES
I. J.Y.S. Luh, Y.F. Zheng, Constrained Relations Between Two Coordinated Industrial Robots for Motion Control, The International Journal of Robotics Research, Vol.6, No.3, Fall 1987. 2. T.J. Tam, A.K. Bejczy and Ning Xi, Hybrid Position/Force Control of Redundant Robot with Consideration of Motor Dynamics, Proc. of the Thirtieth Annual Allerton Conf. on Communication, Control and Computing, Urbana, 1992. 3. T.J. Tam, A.K. Bejczy and Ning Xi, Motion Planning in Phase Space for Intelligent Robot Arm Control, Proc. of IROS'92, Raleigh, 1992. 4. T.J. Tam, A.K. Bejczy, G.T. Marth and A.K. Ramadorai, Kinematic Characterization of the PUMA 560 Manipulator, Robotics Laboratory Report SSM-RL-92-15, Washington University, Dec. 1991. 5. A.K. Bejczy and Z.F. Szakaly, A Harmonic Motion Generator for Telerobotic Applications, Proceedings of the 1991 IEEE International Conference on Robotics and Automation, Sacramento, California, April, 1991, pp.2032-2039. 6. Z.F. Szakaly and A.K. Bejczy, Performance Capabilities of a JPL Dual-arm Advanced Teleoperation System, (Preprint) SOAR'90 Workshop, Albuquerque, NM, June 26-28, 1990. 7. Ning Xi, T.J. Tam and A.K. Bejczy, Event-Based Planning and Control for MultiRobot Coordination, Proceedings of the 1993 IEEE conference on Robotics and Automation, Atlanta, Georgia, pp.251-258, May 2-6, 1993. 8. Yasuyoshi Yokokohji, Akira Ogawa, Hitoshi Hasunuma, and Tsuneo Yoshihawa, Operation Modes for Cooperating with Autonomous Functions in Intelligent Teleoperation Systems, Proceedings of the IEEE International Conference on Robotics and Automation, Atlanta, Georgia, May 2-6, 1993, Vol.3, pp.510-515. 9. Paul G. Backes, Mark K. Long, and Robert D. Steele,The Modular Telerobot Task Execution System for Space Telerobotics, Proceedings of the IEEE International Conference on Robotics and Automation, Atlanta, Georgia, May 2-6, 1993, Vol.3, pp.524-530. I0. T. Sato and S. Hirai,MEISTER: A Model Enhanced Intelligent and Skillful Teleoperation Robot System, Robotics Research- The Fourth International Symposium-, The MIT Press, pp.155-162, 1988.
386
II. P.T. Boissiere and R.W. Harrigan,Telerobotic Operation of Conventional Robot Manipulators, Proceedings of the 1988 IEEE International Conference on Robotics and Automation, pp.576-583. 12. V. Lumelsky and E. Cheung,Towards Safe Real-time Robot Teleoperation: Automatic Whole-sensitive Arm Collision Avoidance Frees the Operation for Global Control, Proceedings of the 1991 IEEE International Conference on Robotics and Automation, pp. 797-802.
Intelligent Robots and Systems V. Graefe (Editor) 1995 Elsevier Science B.V.
387
T a s k - d i r e c t e d p r o g r a m m i n g o f s e n s o r - b a s e d robots B. Brunner, K. Arbter and G. Hirzinger German Aerospace Research Establishment (DLR), Oberpfaffenhofen, Institute of Robotics and System Dynamics, P.O. Box 1116, D-82230 Wessling, Germany
Abstract
We propose the so-called TeleSensorProgramming concept that uses sensory perception to achieve local autonomy in robotic manipulation. Sensor-based robot tasks are used to define elemental moves within a high level programming environment. This approach is applicable in both, the real robot's world and the simulated one. Beside the graphical off-line programming concept, the range of application lies especially in the field of teleoperation with large time delays. A shared autonomy concept is proposed that distributes intelligence between man and machine. The feasibility of graphically simulating the robot within its environment is extended by emulating different sensor functions like distance, force-torque, and vision sensors to achieve a correct copy of the real system behaviour as far as possible. These simulation features are embedded in a task-oriented high level robot programming approach. This implicit programming paradigm is supported by a sophisticated graphical man-machineinterface. Programming by demonstration is performed by introducing an analytical controller generation approach. Sensor fusion aspects with respect to autonomous sensor-controlled task execution are discussed as well as the interaction between the real and the simulated system. 1
Introduction
Programming a robot off-line has been restricted to showing a sequence of cartesian positions or joint angle configurations. But mechanical inaccuracies at the manipulator level and differences between real and simulated world lead to difficult problems in the execution of off-line defined robot activities. Local adaptations on path generation level have to be done via teach-in commands by a human operator, especially in the context of manipulation tasks, such as gripping or handling objects. Even small deviations in position, orientation, and shape of all the objects to be handled are not allowed, because the task execution will fail in an uncertain environment. Graphical simulation of robot tasks and downloading the generated commands to the real robot is limited to the joint or cartesian motion level. This approach is only useful if geometrical consistency of the real environment and the simulated one can be guaranteed. This demand cannot be met with available programming systems.
388 The most important requirement to achieve autonomy is the ability to successfully execute a given task by intelligent sensor data processing. Actual sensory data are matched with a predefined reference pattern to receive information for the robot controller to achieve the desired end position. Local deviations from the desired state are detected by the sensors and handled by the robot system in an autonomous way. Therefore, sensor-controlled movements have to be used to achieve local autonomy at the machine level. High level planning facilities for task scheduling as well as intelligent error handling mechanisms are required for full autonomy, but state-of-the-art techniques are insufficient to provide the adequate tools. Presently, full autonomy is unattainable.
2 The TeleSensorProgramming approach Therefore, we favour a shared autonomy concept that distributes intelligence between man and machine [ 1]. Presuming that sufficient information about the actual environment is available from the sensor systems, partial tasks can be executed independently at the machine level. Specifications and decisions at a high task planning level have to be performed by human operators. Local sensory feedback loops are executed by the robot system, while global task level jobs have to be specified interactively by a human operator. Coarse planning activities have to be done at a task-oriented level by human intelligence, fine path planning at manipulator level takes place at a sensor-based control level [2]. human operator
low frequency
planning
task-directed
robot system
high frequency
execution
sensor-based
Figure 1 Shared autonomy This shared autonomy approach is the basis of our programming paradigm which is determined TeleSensorProgramming (TSP), resulting in robot programming at a task-directed level. The teach-in of a robot system occurs not at the joint or cartesian manipulator level but at a high language level, i.e. the operator plans activities at a level which can be processed by the robotic system independently from human intervention. Hence, TSP means teaching by showing typical situations at a task-directed level including nominal sensory patterns with the aid of sensory refinement in the real, or a completely simulated, world. All the taught sensor-based actions can be activated in analogous situations. Figure 2 shows the structure of our TSP concept consisting of two parallel working control cascades. One of them is the real system control loop containing internal control loops for local autonomy. The other one is a simulation environment which is structurally equivalent to the real system, with few exceptions. Most important is a signal delay which may occur in the real (remote) system, e.g. in space applications, which is not implemented in the simulation. This makes the simulation predictive with respect to the real system. Another exception is the display of internal variables in the simulation part, which cannot be observed (measured) in the real system. This gives the operator or task planner more insight to what happens or may happen on the system according to his commands. Communication between the two loops
389 runs via a common model data base which delivers a priori knowledge for the execution at the remote level and a posteriori knowledge for the model update of the simulated world.
i!
Graphics and Animation
'1
virtual feedback
real feedback
l
i
local autonomyloop
I
I
,-.-,o0~, I
..... t ...... : .......
task planning and error handling
]i~
path planning 1 and parameter determination
|i
I r.o~,~
.... " ...... ".... t f :
l
(Ground)
:
Simulation
"-!
!
I
,',',o~,
....... : l I
world model data base sensor fusionalgorithms
I! .... "
J
t a posterioriknowledge update of world model
a prioriknowledge usageof world model
,obo,
images Plots Graphics
Real System i 9 9 ; ......................................................................................
;
Figure 2 The TeleSensorProgramming concept For such an intelligent programming system different tools are necessary to implement the required functionality. First, a sophisticated simulation system has to be provided to emulate the real robot system. This includes the simulation of sensory perception within the real environment. Beside this sensor simulation, the shared autonomy concept has to provide an efficient operator interface to set up task descriptions, to configure the task control parameters, to decide what kind of sensors and control algorithms should be used, and to debug an entire job execution phase. Two applications in the field of robotics are evident to the proposed TSP approach. First, the graphical off-line programming concept is extended by the processing of simulated sensory data and by the introduction of local autonomy on the task description level. This means that not only the joint and cartesian information is gathered by graphically guiding the robot during the off-line programming task, but also the simulated sensory informations are stored off-line as nominal patterns for subtask execution on a local feedback loop. The fine motion control loop for handling uncertainties takes place independently of any human intervention both at the simulation side and the real one. A main advantage of this off-line programming scheme is the feasibility to verify all the robot actions before real execution is performed. This includes the perception and processing of sensory data and the activation of the sensor-controlled elemental moves.
390 Second, the field of telerobotics with large time delays, especially in space and subsea applications, is a good area for this sensor-based, task-directed programming approach [1]. Direct visual feedback in the control loop, where a time delay of a few seconds is inherent, is not feasible for the human operator to handle the robot movements in a suitable way. Predictive simulation is the appropriate means for an operator to telemanipulate the remote system on-line [3]. Similar approaches are known, which use force reflecting hand-controllers to feed back force sensor signals for shared and teleoperated control modes [4]. Furthermore, an interactive supervisory user interface makes it possible to set up the environmental and control parameters. But in our approach, the sensor-based elemental move concept (see section 3), feasible for all kind of sensors and execution tasks, is preferred. The operator has to only guide the robot through the task space in a coarse way and to activate specific sensor control phases. Sending these basic commands to the remote system enables the real robot to execute the desired task with its own local autonomy after the delay time has elapsed. The main feature of our telerobotic concept is to replace the time-delayed visual feedback by predictive stereo graphics with sensor simulation, providing a supervisory control technique, that will allow to shift more and more autonomy and intelligence to the robot system. The main focus of this paper lies in the task-directed programming approach which facilitates the robot programming work by an intuitive task description level. Predefined complex sensor-based tasks can be described at a task-oriented level which allows for usage in a varying context. To describe robot activities at a high language level we introduce the elemental move concept. This concept allows us to define subtasks that can be executed by the robot system in an autonomous way. To program a complex robot task it is only necessary to define a sequence of elemental moves at an intuitive planning level. An analytical approach is proposed to define sensor-based elemental moves without an extensive controller design. Therefore, we call our method programming by demonstration, which represents an easy variant of teaching by showing. 3
The elemental move concept
In the past, many attempts have been started to describe robot assembly plans at a high language level. Command language approaches [5] or planning tools such as Petri nets [6] or assembly graphs [7] have been proposed. They work well in a structured, well-defined environment for a specific application field like block world assembly [8] or compliant motion tasks [9]. Our task-directed programming approach is driven by an elemental move concept that enables us to program robot tasks in an intuitive implicit manner. A complex robot task is composed of such elemental moves (EMs) that can be divided into three categories. First, pose-controlled EMs that are described by the goal pose (position and orientation) in the cartesian space; second, the sensor-controlled EMs that have to be defined via nominal sensory patterns, and, third, shared-controlled EMs. Each class of EMs has a template of pre- and postconditions that describe the prerequisites to activate or stop the execution of an EM instance. An EM instance can be regarded as a node of a state machine. The
391 transition between two nodes is defined by the matching of post- and preconditions of the corresponding EM nodes. 3.1 Pose-controlled elemental moves
Pose-controlled EMs are defined via the goal pose in the cartesian space. Therefore, the precondition template for the pose-controlled EM class is empty, the postcondition is met if the tool frame of the robot has reached the desired pose within the predefined limits. The path generation considers the inverse kinematic problem that is solved by an iterative approach. Currently, an on-line collision avoidance algorithm is implemented to find a collision-free path from the current start to the defined end pose [10] . 3.2 Sensor-controlled elemental m o v e s
Sensor-controlled EMs are defined via the nominal sensory pattern in the reference pose relative to the environment. Therefore, the precondition template for the sensor-controlled EM class includes the availability and correctness of the sensory data, which is necessary to apply the control algorithm for achieving the desired goal state. This class of EMs represents full autonomy, in the sense that the sensor-based control mechanism leads to a proper system behaviour. An EM is fully sensor-controlled, if all 6 degrees of freedoms (DOFs) in the cartesian space are controlled by sensor values. With the aid of sensor-based path refinement it is possible to act in an environment under the constraints of the sensor measurements. This class of the sensor-controlled EMs also gives us the possibility to handle similar tasks in analogous situations. E.g. APPROACH movements, that will be able to align the robot gripper with the object to be picked up, can be defined as one sensor-controlled EM for a desired distance and/or vision sensory pattern. Similar objects at different poses can be gripped by the same EM. 3.3 Shared-controlled elemental moves
The shared-controlled EM class is a mixture of the previous two. For each instance of shared-controlled EM the constraint space configuration has to be defined. The constraint space configuration description specifies which degree of freedom in the cartesian 3D-space is pose- or sensor-controlled. For instance a contour-following task in the xy-plane can be described by the pose-controlled DOFs Ztrans, Xrot, Yrot, Zrot and the sensor-controlled Xtrans and Ytrans. Based on Mason's C-frame concept [11 ] sensor-controlled subspaces are defined using nominal sensory patterns to control the selected DOFs. The free DOFs are controlled by the path planning algorithm or, in the case of teleoperation, by the human operator using a 6-DOF input device. The techniques used for projecting gross move commands into the pose and sensor-controlled subspaces have been discussed in a number of previous papers, e.g. [12].
4 High-level task-directed programming Task-directed robot programming has been discussed so far in the field of generalized planning [13] and assembly plan generation as an AI paradigm [14],[15]. Especially for the
392 telerobotic applications interactive planning facilities are of crucial importance. To integrate such planning tools into the framework of sensor-based task execution, only theoretical work has been done so far [ 16]. We believe that the realization of task-directed robot programming by the elemental move approach leads to an intuitive high level programming concept. Complex robot tasks are described by an action graph of elemental moves. The nodes of this graph are represented by the EM controllers. The transitions between the corresponding nodes are defined by matching the successor preconditions with the predecessor postconditions. A plausibility check has to be initiated to look for the congruence between the end conditions of the subtask and the start conditions of the following one, so that the consistency of the subtask sequence is ensured within the entire robot task. The feasibility of an EM, i.e. the question whether the preconditions can be met, depends on availability of the required sensor values as well as on determination of the constraint space configuration. To implement a real task-directed programming tool we propose a hierarchical declaration of the EM framework. At the lowest level we define the pose-, sensor-, and shared-controlled EMs as so-called atomic elemental moves that cannot be devided into several control items. Each node in the action graph represents one instance of an atomic EM. A sequence of connected nodes as a part of the action graph can be joined to a so-called composed elemental move, which can be further used as a logical EM at a high task description level. We have to consider that the uniqueness of the graph transitions are guaranteed by the alignment of the pre- and postconditions of successive EMs. Furthermore, a composed EM is put together by a non-empty sequence of atomic and/or composed EMs in a recursive way. One remark to the definition of the atomic EMs: In contrast to other approaches, e.g. [5] or [4], we don't have to predefine all the elemental operations which can be used to execute any task. We can interactively define and configure an atomic EM via programming by demonstration, which is the topic of the next section.
5
Programming by demonstration
Visual programming tools for the direct transfer of human manipulation skills to the robot have been developed, but they are constrained to location gathering by using a marker-based vision system [17] or trajectory generation from observing object features [18]. Compliance controller identification by human demonstration [19] is restricted to straight line motion in well-defined constraint configurations. Methods proposed during the last years [20],[21],[22] for programming by demonstration utilize the real robot system to collect the required demonstration data. The same system is used for further execution, so that this class of direct demonstration approaches can be regarded as simple playback methods. 5.1 Remote demonstration
We have extended this direct demonstration method to the separation of demonstration and execution system that gives us the ability to define task level robot actions off-line or to overcome the problems attended upon time-delayed telerobotic applications. Before the execution of the robot task takes place in the real system, we demonstrate the task by
393 using sophisticated simulation tools including the emulation of the interactions between the manipulator and the environment concerning collision information of contact sensors as well as non-contact sensor devices. Currently we have implemented the functionality of laser distance sensors, a feature-based simulation of stereo cameras, and the behaviour of a stiff force-torque sensor in contact with the environment [23]. We emphasize that the usage of simulated sensory behaviour is only feasible, if the virtual system is properly calibrated. Especially in the emulation of computer vision the knowledge of internal and external camera parameters is indispensable for finding the right mapping between the 3D world model and the 2D camera planes. 5.2 Definition of atomic elemental moves It should be noted that the following concepts for teaching atomic elemental moves are applicable to both, first to the simulation with regard to graphical off-line programming or telerobotics, second to the real robot (including sensor equipment) without any simulation environment. Applying programming by demonstration to simple pose-controlled EMs is straightforward. The actual cartesian pose of the robot's tool frame is stored with current manipulator data like velocity or acceleration. By activating such a pose-controlled EM the robot moves from any pose to the desired one regarding the inverse kinematic problems as well as the collision detection. To move the robot a 6-DOF input device is used, which allows the operator to easily generate such target points within the robot's workspace. To define the various instances of the sensor- or shared-controlled EM class, we propose an intuitive programming by demonstration method. The key issue is to find an appropriate controller to perform the desired EM correctly. 5.3 Automatic controller generation For the handling of the sensor-controlled EMs we outline a method to estimate the motion parameters by the known sensor values to achieve the predefined goal state of the robot's pose (cartesian position in translation and orientation or joint angle position). In other words, we have to find a control sequence that transduces the robot's end effector from a pose xo into the nominal pose x* which is described by its corresponding sensor values y*. x* is assumed to be unknown or uncertain. The motion parameters relative to the sensed environment are expressed by the vector increment Axk, the actual sensor values by the vector Yk. The nominal sensor values y* generally are non-linear functions of the actual interaction between the robot, the sensor and the environment in the robot's nominal pose x*. y*-f(x*)
(1)
In the execution phase we want to find a controller sequence that is able to reach the goal state dependent on the nominal sensor value vector, which the system was taught for. Starting from a pose xo we calculate stepwise the motion parameters Axk in order of the actual and nominal sensor values, i.e. we apply a linear controller of the form
Axk = ~ C ( y k - y*),
(2)
394 where C is a constant N'M-Matrix, which maps the displacement in the M-dimensional sensor space to the N-dimensional control space, and where a is a scalar damping factor, which determines the dynamic behaviour of the closed control loop. The optimal controller coefficients, in the sense of least square estimate, are expressed by the pseudoinverse
C = ( J T j ) - I J T,
(3)
where the elements of J are the partial derivatives of the sensor values y to the motion parameters x:
Ji j
Oyi
=
'
Oxj
(4) x*
The teach-in process is implemented in an efficient way, because the nominal sensor values y* can be easily derived from either the simulation or the real system. The controller design, i.e. the construction of J is also implemented in a simple manner by experimentally estimating the Jacobian J, i.e. moving the simulated or real robot by increments, using the difference quotients AYi (5) J i 'J ~
-s
i
x*
"
The reason why we determine the Jacobian J by experimentally estimating the partial derivatives of the sensor values to the motion parameters, is the robustness of the control approach in the local working space, i.e. if Yk ~ Y* is in a suitable bound. Doing so, we avoid the very high effort which would be necessary if we would analytically determine the partial derivatives using a large non-linear model. Furthermore, the estimates determined by using the real system are expected to be better than the analytical ones, because they are independent of (inevitable) model abstractions. To get the mapping from the actual sensory pattern to the desired robot's reference pose, a sensor calibration is not required because the training process delivers the inverse relation between the robot's movements and the corresponding sensor values. In the case of programming by virtual demonstration within the graphical simulation environment, sufficiently correct world models with respect to sensor and object descriptions must be available for the generation of the controller matrix C. The method is plain and robust. The simplicity of the programming by demonstration paradigm is fulfilled by an easy way to configure the sensor- or shared-controlled EM description. The reference sensor pattern, i.e. the goal state definition of the atomic EM is directly shown by the current sensor value configuration. The controller for achieving this desired state is determined in an automatic manner by the generation of the Jacobian as described above. The human operator has only to specify the controller input variables, i.e. the sensor combination, which should be used for the control task, and the controller output variables, i.e. the sensor-controlled DOFs, to activate the controller generation. This procedure can be viewed as a form of training by doing. Currently an intelligent sensor-planning system is in progress to find out the related sensor usage in combination with the determination of the sensor-controlled subspaces in an autonomous way [24].
395
5.4 Sensor fusion aspects The proposed sensorimotion approach allows a simple integration of different sensor values. An efficient sensorfusion algorithm is inherent implemented by assigning all the used sensor values to the elements of the measurement vector y. To get better estimation results, it is often helpful to use redundant and normalized sensor information. We have implemented this approach, among other cases, to combine stereo vision (2 cameras) data with laser range data (4 distance values) and have observed fast (exponentially) as well as wide (full sensor range) convergence. In the following the main advantages of the proposed controller generation process are shortly discussed. The controller matrix C remains constant during the control process. The training of sensor-controlled EMs, i.e. to set the linear controller coefficients in the Jacobian, is necessary only once for a specific sensor/DOF combination in the desired reference state. The number of training steps for the determination of the controller is equal to the number of DOFs to be controlled. Any constraint space configuration can be selected to integrate the shared-control concept into the control generation process. Sensor failures can easily be handled by introducing a binary valued diagonal weight matrix W extending the controller design to (
C = jTWj
)-1
JTw
with
Wi, j - -
{
E
O" i ~ j
{0, 1} 9 i - j
'
(6,
The controller redesign only needs a failure detection, but does not need to apply a new training process for the changed sensor configuration. If j T j is regular, then JTWJ may be regular too, if the original sensor configuration contains enough redundancy. The condition number of j T w J may be used to qualify different sensor configurations. To realize such a sensorimotion coordination and sensor fusion concept, we already have applied other methods such as neural networks [25]. Our ongoing work is to compare both approaches with regard to applicability, robustness, and efficiency.
6 Experiments The TSP concept has been verified within R O T E X an advanced space RObot Technology EXperiment in May '93 at the D2 spacelab mission [12]. The experimental environment of the ROTEX ground station is used and currently extended to implement our task-directed robot programming approach. A powerful man-machine-interface is one of the key issues to perform the programming by demonstration in an efficient way. A 6-DOF control-ball is used to move the robot's tool center point in the 3D-stereographics simulation. A user-friendly graphical interface based on X and OSF/Motif has been implemented to support the operator in the definition and composition of the various elemental moves. Three types of sensors are available in the real as well as the simulated gripper environment: an array of 8 laser distance sensors with a working range from 0-3 cm, one with 0-30 cm, two 6-axis force-torque wrist sensors (a stiff strain gauge based and a more compliant optical one), and a tiny pair of cameras to provide a stereo image viewed in the gripper's z-direction.
396 The image processing system used in our laboratory is able to extract features like contours or markers in real time (application dependent). In addition to this direct object feature extraction more abstract measurements, such as the Fourier coefficients of a contour, can be delivered as an efficient tool for the determination of the pose, size, and shape of an object given by its contour line [26]. The corresponding simulation works analogously at a featurebased level with respect to the internal and external camera parameters. Especially for the acquisition of stereo vision data this model-based method offers the advantage to avoid the corresponcience problem. Corresponding object features in the different camera views can easily be detected by using the same geometric world model interface. The experimental environment consists of a powerful Silicon Graphics workstation with a multi-processor architecture to enhance the simulation and graphical visualization of the workcell environment. The graphical user interface as well as the controller task are running as independent processes on the same machine. The inter-process-communication occurs via shared memories protected by sema