Fast Motions in Biomechanics and Robotics: Optimization and Feedback Control (Lecture Notes in Control and Information Sciences)

Lecture Notes in Control and Information Sciences 340 Editors: M. Thoma, M. Morari Moritz Diehl, Katja Mombaur (Eds.)...

Author: Moritz Diehl | Katja Mombaur

143 downloads 915 Views 7MB Size Report

This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!

Report copyright / DMCA form

DOWNLOAD PDF

Lecture Notes in Control and Information Sciences 340 Editors: M. Thoma, M. Morari

Moritz Diehl, Katja Mombaur (Eds.)

Fast Motions in Biomechanics and Robotics Optimization and Feedback Control

ABC

Series Advisory Board F. Allgöwer, P. Fleming, P. Kokotovic, A.B. Kurzhanski, H. Kwakernaak, A. Rantzer, J.N. Tsitsiklis

Editors Moritz Diehl Katja Mombaur University of Heidelberg Interdisciplinary Center for Scientiﬁc Computing - INF 368 69120 Heidelberg Germany E-mail: [email protected] [email protected]

Library of Congress Control Number: 2006930404 ISSN print edition: 0170-8643 ISSN electronic edition: 1610-7411 ISBN-10 3-540-36118-9 Springer Berlin Heidelberg New York ISBN-13 978-3-540-36118-3 Springer Berlin Heidelberg New York This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, speciﬁcally the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microﬁlm or in any other way, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer. Violations are liable for prosecution under the German Copyright Law. Springer is a part of Springer Science+Business Media springer.com c Springer-Verlag Berlin Heidelberg 2006 The use of general descriptive names, registered names, trademarks, etc. in this publication does not imply, even in the absence of a speciﬁc statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. Typesetting: by the authors and techbooks using a Springer LATEX macro package Cover design: design & production GmbH, Heidelberg Printed on acid-free paper

SPIN: 11544777

89/techbooks

543210

Preface

In the past decades, much progress has been made in the ﬁeld of walking robots. The current state of technology makes it possible to create humanoid robots that nearly walk like a human being, climb stairs, or avoid small obstacles. However, the dream of a robot running as fast and as elegantly as a human is still far from becoming reality. Control of such fast motions is still a big technological issue in robotics, and the maximum running speed of contemporary robots is still much smaller than that of human track runners. The conventional control approach that most of these robots are based on does not seem to be suitable to increase the running speeds up to a biological level. In order to address this challenge, we invited an interdisciplinary community of researchers from robotics, biomechanics, control engineering and applied mathematics to come together in Heidelberg at the ﬁrst Ruperto-CarolaSymposium “Fast Motions in Biomechanics and Robotics – Optimization & Feedback Control” which was held at the International Science Forum (IWH) on September 7–9, 2005. The number of participants in this symposium was kept small in order to promote discussions and enable a fruitful exchange of ideas. This volume contains a selection of papers from this symposium. Thus, one aim of the volume is to study the control and stabilization principles of biological motions and to determine which aspects can be exploited for the control of fast walking and running robots. In addition, the applicability of recent advances in control engineering, in particular in nonlinear model predictive control, to the ﬁeld of walking robots is discussed. Another focus is on model based simulation and optimization methods that can be used for analysis and optimal design of motions and feedback control systems. We would like to thank all authors for their interesting contributions, and all reviewers for their careful reading and their helpful comments which made it possible to assure a high quality of this book. In this context we would particularly like to thank Prof. Jim Bobrow for handling the editorial process of the papers in which we were involved.

VI

Preface

Both the symposium and the work leading to this volume were ﬁnancially supported by the Hengstberger-Prize donated by Dr. Klaus Georg and Sigrid Hengstberger, for which are very grateful. We also thank the members of the IWH team for their hospitality and friendly help with the organization of the symposium, in the beautiful surroundings of IWH. Special thanks are due to Tanja Binder for doing the technical compilation of this book. We also thank Dr. Thomas Ditzinger and Heather King of Springer Verlag for their support with publishing this volume. Heidelberg June 2006

Moritz Diehl Katja Mombaur

Contents

Re-injecting the Structure in NMPC Schemes M. Alamir, F. Boyer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1

Recent Advances on the Algorithmic Optimization of Robot Motion J.E. Bobrow, F.C. Park, and A. Sideris . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 A Spring Assisted One Degree of Freedom Climbing Model J.E. Clark, D.E. Koditschek . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 Fast Direct Multiple Shooting Algorithms for Optimal Robot Control M. Diehl, H.G. Bock, H. Diedam, and P.-B. Wieber . . . . . . . . . . . . . . . . . . 65 Stability Analysis of Bipedal Walking with Control or Monitoring of the Center of Pressure D. Djoudi, C. Chevallereau . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 Multi-Locomotion Control of Biped Locomotion and Brachiation Robot T. Fukuda, M. Doi, Y. Hasegawa, and H. Kajima . . . . . . . . . . . . . . . . . . . . 121 On the Determination of the Basin of Attraction for Stationary and Periodic Movements P. Giesl, H. Wagner . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147 Task-Level Control of the Lateral Leg Spring Model of Cockroach Locomotion J. Lee, A. Lamperski, J. Schmitt, and N. Cowan . . . . . . . . . . . . . . . . . . . . . 167 Investigating the Use of Iterative Learning Control and Repetitive Control to Implement Periodic Gaits R.W. Longman, K.D. Mombaur . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189

VIII

Contents

Actuation System and Control Concept for a Running Biped T. Luksch, K. Berns, and F. Fl¨ orchinger . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219 Dynamical Synthesis of a Walking Cyclic Gait for a Biped with Point Feet S. Miossec, Y. Aoustin . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233 Performing Open-Loop Stable Flip-Flops – An Example for Stability Optimization and Robustness Analysis of Fast Periodic Motions K. Mombaur . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253 Achieving Bipedal Running with RABBIT: Six Steps Toward Inﬁnity B. Morris, E.R. Westervelt, C. Chevallereau, G. Buche, and J.W. Grizzle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 277 Velocity-Based Stability Margins for Fast Bipedal Walking J.E. Pratt, R. Tedrake . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 299 Nonlinear Model Predictive Control and Sum of Squares Techniques T. Raﬀ, C. Ebenbauer, R. Findeisen, and F. Allg¨ ower . . . . . . . . . . . . . . . . 325 Comparison of Two Measures of Dynamic Stability During Treadmill Walking M. Schablowski, H.J. Gerner . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 345 Simple Feedback Control of Cockroach Running J. Schmitt . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 361 Running and Walking with Compliant Legs A. Seyfarth, H. Geyer, R. Blickhan, S. Lipfert, J. Rummel, Y. Minekawa, and F. Iida . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 383 Self-stability in Biological Systems – Studies based on Biomechanical Models H. Wagner, P. Giesl . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 403 Holonomy and Nonholonomy in the Dynamics of Articulated Motion P.-B. Wieber . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 411 Dynamic Stability of a Simple Biped Walking System with Swing Leg Retraction M. Wisse, C.G. Atkeson, and D.K. Kloimwieder . . . . . . . . . . . . . . . . . . . . . 427

List of Contributors

Alamir, Mazen∗ Laboratoire d’Automatique de Grenoble CNRS-INPG-UJF BP 46, Domaine Universitaire 38400 Saint Martin d’H`eres – France [email protected] Allg¨ ower, Frank∗ Institute for Systems Theory and Automatic Control University of Stuttgart Pfaﬀenwaldring 9 70550 Stuttgart – Germany [email protected] Aoustin, Yannick∗ Institut de Recherche en Communications et Cybern´etique de Nantes U.M.R. 6597 Ecole Centrale de Nantes CNRS, Universit´e de Nantes 1, rue de la No¨e BP 92 101 44321 Nantes Cedex 03 – France Yannick.Aoustin@irccyn. ec-nantes.fr Atkeson, Christopher Carnegie Mellon University – USA ∗

Participant of the symposium

Berns, Karsten∗ Arbeitsgruppe Robotersysteme Fachbereich Informatik Technische Universit¨at Kaiserslautern Postfach 3049 67653 Kaiserslautern – Germany [email protected] Blickhan, Reinhard Science of Motion University of Jena Seidelstr. 20 07749 Jena – Germany Bobrow, James∗ Department of Mechanical and Aerospace Engineering Engineering Gateway 3220 University of California, Irvine Irvine, CA 92697 – USA [email protected] Bock, Hans Georg∗ Interdisciplinary Center for Scientiﬁc Computing University of Heidelberg INF 368 69120 Heidelberg – Germany [email protected]

X

List of Contributors

Boyer, Fr´ ed´ eric IRCCyN Ecole Centrale de Nantes Universit´e de Nantes UMR CNRS 6597 1, rue de la No¨e BP 92 101 44321 Nantes Cedex 03 – France Frederic.Boyer@irccyn. ec-nantes.fr Buche, Gabriel LAG Ecole Nationale d’Ing´enieurs Electriciens de Grenoble BP 46 38402 St Martin d’H`eres – France [email protected] Chevallereau, Christine∗ IRCCyN Ecole Centrale de Nantes Universit´e de Nantes UMR CNRS 6597 1, rue de la No¨e BP 92 101 44321 Nantes Cedex 03 – France Christine.Chevallereau @irccyn.ec-nantes.fr Clark, Jonathan∗ University of Pennsylvania Dept. of Electrical and Systems Engineering 200 S. 33rd Street Philadelphia, PA 19104-6314 – USA [email protected] Cowan, Noah∗ LIMBS Laboratory Johns Hopkins University 230 Latrobe Hall 3400 N. Charles Street Baltimore, MD 21218-2681 – USA [email protected]

Diedam, Holger Interdisciplinary Center for Scientiﬁc Computing University of Heidelberg INF 368 69120 Heidelberg – Germany Diehl, Moritz∗ Interdisciplinary Center for Scientiﬁc Computing University of Heidelberg INF 368 69120 Heidelberg – Germany [email protected] Djoudi, Dalila IRCCyN Ecole Centrale de Nantes Universit´e de Nantes UMR CNRS 6597 1, rue de la No¨e BP 92 101 44321 Nantes Cedex 03 – France [email protected] Doi, Masahiro Department of Micro-Nano Systems Engineering Nagoya University Furo-cho 1 Chikusa-ku Nagoya, 464-8603 – Japan [email protected] Ebenbauer, Christian Institute for Systems Theory and Automatic Control University of Stuttgart – Germany [email protected] Findeisen, Rolf Institute for Systems Theory and Automatic Control University of Stuttgart – Germany [email protected]

List of Contributors

Fl¨ orchinger, F. Technische Universit¨at Kaiserslautern – Germany [email protected] Fukuda, Toshio∗ Department of Micro-Nano Systems Engineering Nagoya University Furo-cho 1 Chikusa-ku Nagoya, 464-8603 – Japan [email protected] Gerner, Hans J¨ urgen Stiftung Orthop¨ adische Universit¨atsklinik Heidelberg Geyer, Hartmut Locomotion Laboratory University of Jena Dornburger Str. 23 07743 Jena – Germany Giesl, Peter∗ Zentrum Mathematik Technische Universit¨at M¨ unchen Boltzmannstr. 3 85747 Garching – Germany [email protected] Grizzle, Jessy Control Systems Laboratory EECS Department University of Michigan Ann Arbor Michigan 48109-2122 – USA [email protected] Hasegawa, Yasuhisa Depertment of Intelligent Interaction Technologies University of Tsukuba Tennodai 1-1-1 Tsukuba Ibaraki, 305-8573 – Japan [email protected]

XI

Iida, Funiya Locomotion Laboratory University of Jena Dornburger Str. 23 07743 Jena – Germany Kajima, Hideki Department of Micro-Nano Systems Engineering Nagoya University Furo-cho 1 Chikusa-ku Nagoya, 464-8603 – Japan Kloimwieder, Daniel Kaalim Carnegie Mellon University – USA Koditschek, Daniel University of Pennsylvania Dept. of Electrical and Systems Engineering 200 S. 33rd Street Philadelphia, PA 19104-6314 – USA [email protected] Lamperski, Andrew California Institute of Technology Pasadena, CA – USA [email protected] Lee, Jusuk LIMBS Laboratory Johns Hopkins University 216A Latrobe Hall 3400 N. Charles Street Baltimore, MD 21218-2681 – USA [email protected] Lipfert, Susanne Locomotion Laboratory University of Jena Dornburger Str. 23 07743 Jena – Germany

XII

List of Contributors

Longman, Richard∗ Department of Mechanical Engineering Columbia University Rm. 232 S.W. Mudd Building 500 West 120th Street New York, NY 10027 – USA [email protected] Luksch, Tobias∗ Arbeitsgruppe Robotersysteme Fachbereich Informatik Technische Universit¨at Kaiserslautern Postfach 3049 67653 Kaiserslautern – Germany [email protected] Minekawa, Yohei Locomotion Laboratory University of Jena Dornburger Str. 23 07743 Jena – Germany Miossec, Sylvain Joint Japanese-French Robotics Laboratory Intelligent Systems Research Institute National Institute of Advanced Science and Technology AIST Tsukuba Central 2 Umezono 1-1-1 Tsukuba 305-8568 – Japan [email protected] Mombaur, Katja∗ Interdisciplinary Center for Scientiﬁc Computing University of Heidelberg INF 368 69120 Heidelberg – Germany [email protected]

Morris, B. Control Systems Laboratory EECS Department University of Michigan Ann Arbor Michigan 48109-2122 – USA [email protected] Park, Frank School of Mechanical and Aerospace Engineering Seoul National University Kwanak-ku, Shinlim-dong San 56-1 Seoul 151-742 Korea [email protected] Pratt, Jerry∗ Florida Institute for Human and Machine Cognition 40 South Alcaniz Street Pensacola, FL 32502 – USA [email protected] Raﬀ, Tobias Institute for Systems Theory and Automatic Control University of Stuttgart – Germany [email protected] Rummel, J¨ urgen Locomotion Laboratory University of Jena Dornburger Str. 23 07743 Jena – Germany Schablowski-Trautmann, Matthias∗ Forschung Abteilung Orthop¨ adie II Schlierbacher Landstraße 200a 69118 Heidelberg – Germany Matthias.Schablowski @ok.uni-heidelberg.de

List of Contributors

Schmitt, John∗ Department of Mechanical Engineering 214 Rogers Hall Oregon State University Corvallis, OR 97331-6001 – USA [email protected] Seyfarth, Andre∗ Locomotion Laboratory University of Jena Dornburger Str. 23 07743 Jena – Germany [email protected] Sideris, Athanasios Department of Mechanical and Aerospace Engineering University of California, Irvine Irvine, CA 92697 – USA [email protected] Tedrake, Russ Massachusetts Institute of Technology Media Laboratory 32 Vassar Street Cambridge, MA 02193 – USA [email protected]

XIII

Wagner, Heiko∗ Biomechanics and Motor Control WWU M¨ unster Horstmarer Landweg 62b 48149 M¨ unster – Germany [email protected] Westervelt, Eric∗ Locomotion and Biomechanics Laboratory Department of Mechanical Engineering The Ohio State University 650 Ackerman Rd, Suite 255 Columbus, OH 43210 – USA [email protected] Wieber, Pierre-Brice∗ Projet BIPOP INRIA Rhˆone-Alpes 38334 St-Ismier Cedex – France Pierre-Brice.Wieber @inrialpes.fr Wisse, Martijn∗ Department of Mechanical Engineering Delft University of Technology Mekelweg 2 2628 CD Delft – The Netherlands [email protected]

Re-injecting the Structure in NMPC Schemes Application to the Constrained Stabilization of a Snakeboard M. Alamir1 and F. Boyer2 1

2

Laboratoire d’Automatique de Grenoble, CNRS-INPG-UJF, BP 46, Domaine Universitaire, 38400 Saint Martin d’H`eres, France [email protected] IRCCyN, 1, rue de la No¨ e BP 92 101, 44321 Nantes Cedex 3, France [email protected]

Summary. In this paper, a constrained nonlinear predictive control scheme is proposed for a class of under-actuated nonholonomic systems. The scheme is based on fast generation of steering trajectories that inherently fulﬁll the contraints while showing a “translatability” property which is generally needed to derive stability results in receding-horizon schemes. The corresponding open-loop optimization problem can be solved very eﬃciently making possible a real-time implementation on fast systems (The resulting optimization problem is roughly scalar). The whole framework is shown to hold for the well known challenging problem of a snakeboard constrained stabilization. Illustrative simulations are proposed to assess the eﬃciency of the proposed solution under saturation constraints and model uncertainties.

1 Introduction One of the most attractive features of Nonlinear Model Predictive Control (NMPC) schemes [1, 2] is their complete independence of the mathematical structure of the system’s model. Indeed, from a conceptual point of view, given any system satisfying a rather intuitive set of assumptions, one may write down a concrete state feedback algorithm that theoretically asymptotically stabilizes a target equilibrium state. Unfortunately, such generically deﬁned formulations may lead to optimization problems that cannot be solved in the available computation time when rather fast dynamics are involved. This can be formulated in a kind of “no free lunch” statement: Genericity reduces eﬃciency Therefore, to overcome the consequences of the above unavoidable truth, speciﬁc features of each system under study have to be explicitly taken into account as far as possible.

2

M. Alamir and F. Boyer

In a series of papers [3, 4, 5, 6, 7, 8, 9], it has been shown that when the constrained stabilization is the main issue, that is, when the optimality is not rigorously required, eﬃcient stabilizing NMPC schemes can be obtained as soon as open-loop “steering trajectories” can be generated by some systematic and eﬃcient algorithm. Now, the way these trajectories are generated is system dependent and the associated eﬃciency may be greatly increased if the speciﬁc structure of the system is explicitly exploited. This is what reduces the genericity to increase eﬃciency. By allowing low dimensional parametrization of these trajectories, corresponding low dimensional NMPC schemes can be deﬁned in which, the decision variable is the parameter vector of the steering trajectory. In this paper, it is shown that for a particular class of mechanical systems including the snakeboard, it is possible to use the particular structure of the system equations in order to derive eﬃcient computation of parameterized steering trajectories. Moreover, these trajectories have the nice property of being structurally compatible with the saturation constraints on the actuators. Since in addition, they have the “translatability property”, they can be used to implement a stable closed loop receding horizon feedback. The paper is organized as follows: First, the particular class of mechanical systems under study is deﬁned in Sect. 2 together with the associated assumptions. In Sect. 3, the proposed state feedback algorithm is explained and the associated convergence results are derived. The fact that the snakeboard falls into the particular class depicted in Sect. 2 is discussed in Sect. 4. Finally, illustrative simulations are proposed in Sect. 5 in order to assess the performance of the proposed solution.

2 The Class of Systems Considered We consider nonlinear systems that may be described by the following set of ODE’s r˙ = f1 (χ)g1 (η)η˙ η˙ = f2 (χ)g2 (ξ) ; g2 (0) = 0 ξ˙ = f3 (ξ, χ, χ, ˙ u1 )

(1) (2) (3)

χ ¨ = f4 (ξ, χ, χ, ˙ u2 )

(4)

where equations (1)–(2) stands for a kinematic stage while equations (3)–(4) represent the dynamic stage. r ∈ Rnr is a kind of generalized position; η ∈ Rnη is an orientation variable; ξ ∈ Rnξ is a generalized velocity while χ ∈ R stands for an internal conﬁguration variable. f1 : R → R, g1 : Rnη → Rnr ×nη . All the maps invoked in (1)–(4) are assumed to be continuously diﬀerentiable. Note that equation (1) generally describes a nonholonomic constraint, namely, a constraint on the velocities that is not derived from a position

Re-injecting the Structure in NMPC Schemes

3

related constraints. The control inputs u1 and u2 have to meet the following saturation constraints ∀t

,

ui (t) ∈ [−umax , +umax ]; i i

i ∈ {1, 2}

(5)

for some given upper bounds umax and umax . 1 2 The aim of the present paper is to derive a state feedback algorithm that steers the sub-state (r, η, ξ) of the system (1)–(4) to the origin under the saturation constraint (5). Note that since g2 (0) = 0, the origin (r, η, ξ) = 0 is an equilibrium position for the dynamics (1)–(3) provided that ξ can be maintained at 0 by convenient use of the control input u1 as it is suggested by Assumption 1 hereafter. Note also that χ is an internal conﬁguration variable whose value is irrelevant for the control objective. This is for instance the angular position of the wheels in the snakeboard example (see Sect. 4). Beside the structure depicted in equations (1)–(4), the class of systems of interest has to satisfy the following assumptions: Assumption 1 1. For all χf , there exists a feedback law u := (u1 , u2 ) = K1 (ξ, χ, χ, ˙ χf ) under which, the closed loop behavior respects the constraints (5) and such that (ξ = 0, χ˙ = 0, χ = χf ) is a globally asymptotically stable equilibrium for the closed loop dynamics deﬁned by (3)–(4) and K1 . Furthermore, the subset {ξ = 0} is invariant under the closed loop behavior. 2. For all χf = 0 and all η f , there exists a feedback law u = (u1 , u2 ) = K2 (η, ξ, χ, χ, ˙ η f , χf ) such that the closed loop behavior respects the constraints (5) and such that (η = η f , ξ = 0) is a globally asymptotically stable equilibrium for the closed loop dynamics deﬁned by (2)–(3) and K2 . Furthermore, the set {χ = χf , χ˙ = 0} is invariant under this dynamics. 3. For all η 0 , there exists a parameterized sequence (η k (p, η 0 ))k≥0 (deﬁned for some parameter vector p ∈ P where P is a compact set) that is continuous in (p, η 0 ) such that η 0 (p, η 0 ) = η 0 , limk→∞ η k (p, η 0 ) = 0 (exponentially) and the following rank condition is satisﬁed for all p ∈ P: (6) Rank A(p, η 0 ) = A1 (p, η 0 ) . . . Aj (p, η 0 ) . . . = nr

4

M. Alamir and F. Boyer

where η j (p,η 0 )

Aj (p, η 0 ) =

g1 (η)dη

(7)

η j−1 (p,η 0 )

with σmin (A(p, η 0 )) > smin > 0. Furthermore, for all p ∈ P, there exists p+ ∈ P such that η k (p+ , η 1 (p, η 0 )) = η k+1 (p, η 0 )

∀k ≥ 0

(translatability)

(8)

4. The function g2 in (2) is such that for any pair (η a , η b ), there exists a sequence (a,b) r (a,b) ; η1 = η a ; ηn(a,b) = ηb (ηj )nj=1 r such that the matrix



 M (η a , η b ) :=  

(a,b) ηn r

(a,b)

η 2

g1 (η)dη . . .

(a,b) η1



 nr ×nr g1 (η)dη  ∈R

(9)

(a,b) ηnr −1

has full rank. 5. The function f1 maps some domain D ⊂ R onto R − {0} so that an inverse map f1−1 : R − {0} → D may be deﬁned by ﬁxing some selection rule (when needed). In the following sequel, the state of the system is denoted by x, namely T x := r η ξ χ χ˙ ∈ Rn ;

n = nr + nη + nξ + 2

3 The Proposed Feedback Algorithm The basic idea of the control algorithm is to decompose the behavior of the controller into basically 2 modes 1. In the ﬁrst mode, ξ ≈ 0 and the feedback K1 is used to steer the state χ from some initial value χj−1 to some ﬁnal desired one χj . This is possible thanks to assumption 1.1. Note that under this mode the position r as well as the orientation variable η are maintained almost constant (since ξ ≈ 0 and g2 (0) = 0 by assumption 1.5). 2. In the second mode, χ ≈ χj is maintained almost constant while the feedback K2 is used to steer the variable η from some initial value η j−1 to some ﬁnal value η j . Note again that this is possible thanks to assumption 1.2. Note moreover that under this mode, ξ asymptotically goes to 0 enabling the ﬁrst mode to be ﬁred again.

Re-injecting the Structure in NMPC Schemes

5

Note that from equations (1) and (7), it comes that under constant χ ≡ χj , when the mode 2 is used with η changing from η j−1 (p, η 0 ) to η j (p, η 0 ), the corresponding variation in r is given by: ∆j r := Aj (p, η 0 ) f1 (χj ) under constant χ ≡ χj (mode 2) (10) Therefore, the condition   v0  v1    . 0 0  .  r + A(p, η )  .  = 0 ;  vj    .. .

vj = 0

for all j ;

χj = f1−1 (vj )

(11)

characterizes the family of sequences (χj )j≥0 such that when the two-modes procedure deﬁned above is applied in an open-loop way, the vector (r, η, ξ) is steered to 0. The state feedback proposed in the present paper amounts to use the openloop strategy deﬁned above in a receding horizon way. This is because the sequences (η j )j≥0 and (χj )j≥0 may become irrelevant because of unavoidable disturbances and because of the simple fact that during mode 1 [resp. mode 2], ξ = 0 [resp. χ = χj ] cannot be rigorously satisﬁed making necessary to re-compute the steering strategy. In order to properly deﬁne the proposed receding horizon formulation, the following deﬁnitions are needed: Deﬁnition 1. Given any (p, r0 , η 0 ), let q ∈ N be a suﬃciently high integer for Aj (p, η 0 ) to be negligible for j > q and deﬁne the vector vˆ(p, r0 , η 0 ) ∈ Rq to be the solution of the following Linear Programming (LP) problem: q v˜j vˆ(p, r , η ) := Arg minq − 0

0

v ˜∈R

j=1

under r0 + A1→q (p, η 0 ) v˜ = 0 and ˜ v ∞ ≤ vmax

(12)

where A1→j (p, η 0 ) is the matrix built with the j ﬁrst matrix terms of A(p, η 0 ) [see (6)] while vmax is a suﬃciently high value making the above constrained problem feasible for any initial value r0 of interest. Note that for each candidate value of p ∈ P, the LP problem (12) may be solved almost instantaneously using LP solvers. Note also that the cost function to be minimized suggests solutions that avoid vanishing components of vˆ. However, if in spite of this vˆk (p, r0 , η 0 ) = 0 for some k, intermediate values have to be introduced in the sequence (η j (p, η 0 ))j≥0 between η k (p, η 0 ) and η k+1 (p, η 0 ) in order to remove this vanishing components without altering the remaining solution, namely vˆi (p, r0 , η 0 ) for i ≥ k + 1. This is possible thanks to assumption 1.4. Indeed:

6

M. Alamir and F. Boyer

• Assume that for some k, one has vˆk = 0, this means that the positions r at the sampling instants k and k + 1 are the same. • Consider the following matrices deﬁned according to (9): 1

M (η k , η k+ 2 ) ;

1

M (η k+ 2 , η k+1 )

where

1

η k+ 2 :=

η k + η k+1 2

together with the corresponding sequences k,k+ 12 nr k+ 1 ,k+1 nr ηj , ηj 2 j=1

j=1

• Our aim is to prove that there exists a sequence of controls

vˆjk

2nr j=1

with no zero elements and such that the net variation on r vanishes. But this amounts to ﬁnd a vector v ∈ Cp := [−vmax , vmax ]2nr with vi = 0 for all i such that the vector −1 1 1 M (η k+ 2 , η k+1 ) M (η k , η k+ 2 ) v (13) A

has no zero elements. But this is clearly always possible since the matrix A is regular according to assumption 1.4. (See lemma 1 in the appendix for a brief proof of this evident fact). Repetitive application of this technique enables all vanishing (or too small) components of vˆ to be removed. Deﬁnition 2. Given any (r0 , η 0 ), the optimal parameter vector pˆ(r0 , η 0 ) is deﬁned by pˆ(r0 , η 0 ) := Arg min J(p, r0 , η 0 ) p∈P

q := r0 + A1→j (p, η 0 )ˆ v1→j (r0 , η 0 ) 2 + α · |η j (p, η 0 )|2 (14) j=0

namely, pˆ(r0 , η 0 ) minimizes a quadratic cost on the excursion of the conﬁguration vector (r, η). Note that deﬁnition 2 assumes that the admissible parameter set P is such that (14) admits a solution. This is typically guaranteed because of the continuity of the cost function J w.r.t. p and the compactness of the admissible set of parameters P. Putting together deﬁnitions 1 and 2 enables us to deﬁne for any initial conﬁguration (r0 , η 0 ) the optimal sequence given by

Re-injecting the Structure in NMPC Schemes

7

v opt (r0 , η 0 ) := vˆ(ˆ p(r0 , η 0 ), r0 , η 0 ) = v1opt (r0 , η 0 ) . . . vqopt (r0 , η 0 ) ∈ Rq (15)

and since according to the discussion that follows deﬁnition 1, one can assume without loss of generality that vjopt (r0 , η 0 ) = 0, the following may be deﬁned for a given pair (r0 , η 0 ) thanks to assumption 1.5: ηˆ(r0 , η 0 ) := η 1 (ˆ p(r0 , η 0 ), η 0 ) ; χ(r ˆ 0 , η 0 ) := f1−1 v1opt (r0 , η 0 ) (16) which is nothing but the ﬁrst part of the optimal open loop trajectory on (η, χ) mentioned above when used in a receding horizon way. Finally, in order to monitor the selection of the controller’s mode, the following functions are needed: V1 (x, χf ) = max |χ − χf |, |χ|, ˙ |ξ| ; V2 (x, η f ) := max |η − η f |, |ξ| (17) More precisely, when V1 (x, X f ) approaches 0, this means that the controller task at mode 1 is likely to be achieved. Similarly, when V2 (x, η f ) approaches 0, this means that the controller task in mode 2 is not far from being achieved. Now, it goes without saying that one cannot wait for V1 or V2 to be exactly equal to zero since this never happens rigorously. That is the reason why a ﬁnite threshold ε > 0 is used in the deﬁnition of the switching rules given hereafter. Using the above notations, the proposed state feedback algorithm can be expressed as follows Feedback algorithm parameters ε > 0 a small threshold. vmax > 0 suﬃciently large value to be used in (12) ˆ η(t)) and η f = ηˆ(r(t), η(t)) according to (16) 1) Compute χf = χ(r(t), 2) mode 1 Use the feedback K1 with χf as computed in step 1) until V1 (x, χf ) ≤ ε. 3) mode 2 Use the feedback K2 with η f and χf as computed in step 1) until V2 (x, ηf ) ≤ ε 4) go to step 1) As a matter of fact, the feedback algorithm presented above describes a Finite State Machine that is depicted in Fig. 1. Associated to the proposed feedback algorithm, the following convergence result can be proved Proposition 1. Let (tk )k≥0 be the inﬁnite sequence of instants at which the algorithm visits the updating state of Step 1 (see Fig. 1). We have the following asymptotic property    r(tk )  (18) lim  lim η(tk ) =0 ε→0 k→∞ ξ(tk )

8

M. Alamir and F. Boyer

Fig. 1. The Finite State Machine describing the proposed feedback algorithm

namely, by taking ε suﬃciently small, the sequence (r(tk ), η(tk ), ξ(tk )) may be steered as close as desired to 0. sketch of the proof The regularity assumption on the functions appearing in (1)–(4) together with the rank condition and the uniform regularity of the matrix A(p, η 0 ) used in (12) enables to prove the continuity of the optimal ˆ 0 , η 0 ) = J(ˆ p(r0 , η 0 ), r0 , η 0 ) w.r.t its arguments r0 and η 0 . value function J(r Using this property with the translatability assumption (8) on the parameterization being used enables to write a classical inequality in receding horizon analysis, namely 2 2 ˆ ˆ J(r(t k+1 ), η(tk+1 )) − J(r(tk ), η(tk )) ≤ − r(tk ) − α η(tk ) + O(ε) (19)

where the ﬁnal term regroups all the second order eﬀects due to the use of ﬁnite stay time in each mode (ε > 0) and the use of ﬁnite horizon q in the equality constraint of (12). Finally, since by deﬁnition of the exit condition of Step 3, one necessarily has |ξ(tk )| ≤ ε

(20)

the result clearly follows from (19)–(20).

4 Application to the Constrained Stabilization of a Snakeboard The snakeboard we are interested in is the mechanical system depicted in Figs. 3 and 2. It consists of two wheel-based platforms upon which a rider

Re-injecting the Structure in NMPC Schemes

9

Fig. 2. The snakeboard “without the rider”

Fig. 3. Schematic view of the snakeboard’s variables deﬁnition. Note that the front and the back steering angles are coupled to be the same

is to place each of his feet. These platforms are connected by a rigid coupler with hinges at each platform to allow rotation about the vertical axis. The snakeboard allows the rider to propel himself forward without having to make contact with the ground. This motion is obtained by using the conservation of angular momentum eﬀect together with the nonholonomic constraints deﬁned by the no-slipping condition at the wheels level. This system was ﬁrst outlined by [10] to be a particularly interesting nonholonomic system example that has no straightforward direct biological counterpart. In [10], a ﬁrst dynamical model for the system was given and used to check some standard behaviors that correspond to some given oscillatory gaits. Furthermore, controllability analysis has been proposed showing that this system is locally controllable except at some singular conﬁgurations. The complete proof of controllability has been achieved in [11]. Since then, many works have been done to construct steering trajectories and control design. In

10

M. Alamir and F. Boyer

[12], a strategy leading to the design of steering trajectories has been proposed based on small amplitude, short duration and cyclic inputs. Based on such sinusoidal inputs, a controller based on average theory has also been proposed recently [13]. Much closer to our approach that does not invoke cyclic open loop parametrization is the work proposed in [14]. Indeed, in [14] the generation of path steering trajectories is based upon switching between two vector ﬁelds in order to obtain sub-curves starting from the initial conﬁguration and ending at the desired one. The system is at rest at the switching instants. The constraints handling is obtained by means of time scaling. As long as the snakeboard is concerned, the diﬀerence between the approach proposed in [14] and the one proposed in the present paper lies in the following diﬀerences: The way the sub-curves deﬁned above are derived in [14] deeply depends on the 2D nature of the snakeboard example (intersections of circles and/or straight lines, etc. are extensively used). In our approach, this is basically done, even in the general case by solving linear systems. In that sense, the approach proposed here seems to be more easily generalizable, as long as the assumed structure of the system holds. The choice of the steering trajectories in [14] is based on minimizing the corresponding number of switches whatever is the resulting transient spacial excursion. The reason behind this is that the system has to be at rest at switching instants. Therefore, having a high number of switches may lead to a slow motion. In our case, monitoring the number of switches can be directly and explicitly obtained by the number (q) of intermediate values in the parameterized sequence: q η k (p, η 0 ) k=1

while the additional d.o.f p are used to minimize the corresponding spacial excursion in the x − y plane. The work in [14] concentrates basically on the open-loop steering strategy. It is not completely clear whether the resulting steering strategy can be used in a receding horizon manner in order to yield an eﬀective feedback in presence of uncertainties or modelling errors. In other words, the “translatability” of the open-loop trajectories proposed in [14] has to be checked. If some other feedback strategy is to be used, this is still to be designed. The model of the snakeboard used in the forthcoming developments is given by:

Re-injecting the Structure in NMPC Schemes

     x ¯˙ cos θ − sin θ 0 2l cos2 φ ξ y¯˙  =  sin θ cos θ 0  0 sin(2φ) 0 0 1 θ˙ J r ¨ ˙ · tan φ ψ + φξ ξ˙ = − 2ml2 u2 = 2Jw φ¨ Jr 2 ¨ + 2φξ ˙ cos2 φ u1 = Jr (1 − sin φ) ψ ml2

11

(21) (22) (23) (24)

Note that the third line of equation (21) writes: ξ=

θ˙ θ˙ = sin(2φ) 2 sin(φ) cos(φ)

Using this in the equations enables us to show that te snakeboard equations (21)–(24) are of the standard form (1)–(4) provided that the following correspondances are used 2l x ¯ r= ; nr = 2 ; η = θ ; nη = 1 ; χ = φ ; f1 (χ) = y¯ tan φ cos θ g1 (η) = ; f2 (χ) = sin(2φ) ; g2 (ξ) = ξ sin θ with straightforward deﬁnitions of f3 and f4 that may be obtained by re¨ Note that in the classically used models moving the auxiliary variable ψ. [10, 14, 12], the conﬁguration is given by q = (¯ x, y¯, θ, φ, ψ). Indeed, the variable ξ used in the above equations is an intermediate variable that can be removed since from (21), it comes that ξ=

x ¯˙ 2l cos θ cos2 φ

(25)

which is clearly a function of (q, q). ˙ However, in the derivation of the control law, writing the equation in the form (21)–(24) is mandatory in order to ﬁt the standard form (1)–(4). This is a classical feature: when using partially structural approach, the coordinate system plays a key role. In order to use the feedback scheme deﬁned in Sect. 3, the feedbacks K1 , K2 and the parameterized sequence η k (p, η 0 ) are successively introduced. 4.1 Deﬁnition of the Feedback Laws K1 and K2 In both modes, the deﬁnition of u2 is the same and is given by u2 := −2Jw

2

1 φ˙ + 2 (φ − φf ) tr tr

(26)

12

M. Alamir and F. Boyer

where tr is a time scaling, namely, for tr = 1, a pole placement (with identical double poles = −1) is assigned by imposing φ¨ = −2φ˙ − (φ − φf ) The parameter tr is then used to meet the saturation requirement by taking: tr =

min

τ ∈[0.01 10]

τ

such that

2Jw · | τ2 φ˙ +

1 τ 2 (φ

− φf )| ≤ umax 2

(27)

As for u1 , it is mode dependent, namely Jr ˙ cos2 φ ; u1 |mode i := Jr (1 − sin2 φ)ψ pp |mode i + 2φξ 2 2ml × i ∈ {1, 2} ˙ ψ pp (φ,φ,ξ) 10sign(φ)ξ ψ pp |mode 1 := Satψmax pp ˙ min (φ,φ,ξ) 2 µ 2ml ˙ ψ pp |mode 2 := −0.1|sin2φ|ξ + (ξ − ξr (µ)) + φξ Jr tan φ

(28) (29) (30)

where • ξr (µ) is a varying reference value for ξ that is given by ξr := −0.1µ · sign(sin 2φ)(θ − θf )

(31)

• µ is an adaptive gain computed according to µ=

max ν

ν∈[0 100]

such that

the r.h.s of (30) with (µ = ν) is in

pp ˙ ξ), ψ pp (φ, φ, ˙ ξ)] (32) (φ, φ, [ψmin max

pp ˙ ξ) and ψ pp (φ, φ, ˙ ξ) are the variable lower and upper bound on • ψmin (φ, φ, max ¨ ψ in (22) that are compatible with the saturation constraint on u1 in (24), namely

˙ cos2 φ /(1 − Jr sin2 φ) (33) τ /J − 2 φξ r ml2 τ ∈{−u1 ,+umax } 1 pp ˙ ξ) := ˙ cos2 φ /(1 − Jr sin2 φ) (34) ψmax τ /J (φ, φ, max − 2 φξ r ml2 τ ∈{−umax ,+umax } 1 1 pp ˙ ξ) := ψmin (φ, φ,

min max

In order to understand how (28) (with i = 1) asymptotically stabilizes ξ while meeting the saturation constraint, one may analyse what happens when φ converges to 0 under the action of u2 . Indeed, this asymptotically leads to an admissible region for ψ¨ that contains an open neighborhood of 0 [see equations (33)–(34)]. This with (22) in which φ˙ = 0 is injected clearly shows that implementing (28) yields an asymptotically stable behavior for ξ.

Re-injecting the Structure in NMPC Schemes

13

As for the design of u1 |mode 2 , it is based on a sliding mode like approach in which the manifold S = ξ − ξr is stabilized where ξr is the control that stabilizes θ around its desired value θf [see (31)]. Since ξr asymptotically tends to 0 with the error θ−θf , ξ does the same. Again, variable adaptive gain µ [see equation (32)] is used in order to meet the saturation constraints on u1 . To end this presentation of the feedback laws K1 and K2 , it is worth noting that the constants 0.01, 10 and 0.1 that appear in equations (27), (29), (30) and (31) are used in order to avoid very high gains near the desired targets and to obtain compatible response times in the back-stepping design approach. They might have been left as design parameters. The choice ﬁxed here aims to avoid having too many parameters to tune. 4.2 Deﬁnition of the Parameterized Sequences (η k (p, η 0 ))k≥0 Recall that for the snakeboard example, η = θ. Consider the following parameterized trajectory Θ(t, p, θ0 ) := 2p1 π + (θ0 − 2p1 π)e−λθ t + p2 e−λθ t 1 − e−λθ t ; λθ > 0 (35) where p = (p1 , p2 ) belongs to the compact subset P ⊂ R2 deﬁned by P := {−1, 0, +1} × [−pmax , +pmax ]; 2 2

pmax >0 2

(36)

Note that for all p ∈ P, Θ(0, p, θ0 ) = θ0 while limt→∞ Θ(t, p, θ0 ) = 0 (mod 2π). Note also that the use of p2 = 0 and p1 = 0 enables non constant trajectories with identical boundary conditions θ0 = θf to be generated. This is crucial to obtain “good” solutions in some singular situations like for instance the one given by x ¯0 = 0, 0 = y¯0 ≈ 0 and θ0 = 0. Indeed, without the parameter p2 , whatever small is y¯0 = 0, an entire rotation would be necessary to steer the snakeboard to the desired position. Given an initial value θ0 and some parameter vector p ∈ P, the generation of the parameterized trajectory is done using the following three steps: 1. Choose some sampling period δ > 0, take an integer q the sequence (θk (p, θ0 ))qk=0

where

1 . Generate δλθ

θk (p, θ0 ) := Θ(jδ, p, θ0 )

2. Solve the corresponding LP problem (12) to get the optimal sequence vˆ(p, x ¯0 , y¯0 , θ0 ) Remove all vanishing (or too small) components, if any, by introducing intermediate terms in the sequence θk (p, θ0 ) as explained above (In fact, this has never been necessary in our experimentation).

14

M. Alamir and F. Boyer

3. Reduce the size of the resulting sequence θk (p, θ0 ) by keeping only the last value of all sequences corresponding to the same value of vˆ(p, x ¯0 , y¯0 , θ0 ). As an example, if the sequence vˆ(p, x ¯0 , y¯0 , θ0 ) takes the following form: T vˆ(p, x ¯0 , y¯0 , θ0 ) = v1 v1 v1 v2 v2 v3 v3 v3 then the reduced sequence θk (p, θ0 ) which is ﬁnally retained is the following one 3 θ (p, θ0 ) θ5 (p, θ0 ) θ8 (p, θ0 ) this enables useless waste of time asking successive values of θk (p, θ0 ) to be successively approached with almost zero-velocity with the same value of φ. 4.3 Checking the Remaining Assumptions The rank condition (6) is obviously satisﬁed provided that the sampling period δ > 0 invoked in Sect. 4.2 is taken suﬃciently small thanks to the properties of the trigonometric functions. The same can be said about the condition expressed in assumption 1.4. The translatability property naturally follows from the properties of the exponential functions used in the deﬁnition of the parameterized trajectory (35). Finally, the conditions of assumption 1.5 are clearly satisﬁed with D =] − π/2, +π/2[.

5 Simulation Results In this section, some simulations are proposed in order to show the eﬃciency of the proposed feedback algorithm. The numerical values used in the simulations as well as a recall on where each of them appears in the preceding sections is depicted in Table 1. The proposed simulations aim to underline the following features 1. The ability to explicitly handle the saturation constraints on the control input. This may be observed by comparing Figs. 4 and 5 where the same initial conditions are simulated for two diﬀerent saturation levels, namely = umax = 8 (Fig. 4) and umax = umax = 4 (Fig. 5). umax 1 2 1 2 Table 1. Numerical values of the system’s and the controller’s parameters Parameter Appearing in Value Parameter Appearing in Value l (21) 0.5 Jr (22) 0.72 ml2 (28) 0.24 Jw (23) 0.13 λθ (35) 0.1 vmax (12) 2 ε Fig. 1 0.01 pmax (36) 10 2

Re-injecting the Structure in NMPC Schemes

15

Fig. 4. Closed loop behavior. Initial condition x ¯(0) = y¯(0) = 1, θ(0) = −π/4 and = umax = 8. This ﬁgure is to be compared with ξ(0) = 0. Saturation levels umax 1 2 Fig. 5 in order to appreciate the saturation constraints handling

Fig. 5. Closed loop behavior. Initial condition x ¯(0) = y¯(0) = 1, θ(0) = −π/4 and = umax = 4. This ﬁgure is to be compared with ξ(0) = 0. Saturation levels umax 1 2 Fig. 4 in order to appreciate the saturation constraints handling

16

M. Alamir and F. Boyer

Fig. 6. Closed loop behavior. Initial condition x ¯(0) = 0, y¯(0) = 0.05, θ(0) = 0 and ξ(0) = 0. Saturation levels umax = umax = 4. Note how the parametrization 1 2 of the trajectories avoids the need for a whole rotation even in this rather singular situation

Fig. 7. Closed loop behavior. Initial condition x ¯(0) = 0, y¯(0) = 0.1, θ(0) = π/2 and = umax =4 ξ(0) = 0. Saturation levels umax 1 2

Re-injecting the Structure in NMPC Schemes

17

Fig. 8. Closed loop behavior. Initial condition x ¯(0) = −1, y¯(0) = 1, θ(0) = 3π/4 and ξ(0) = 1. Saturation levels umax = umax = 8. This simulation is done under 1 2 nominal model without uncertainties. The result is to be compared to that of Fig. 9 where model uncertainties are introduced

2. The ability to realize rather “economic” trajectories when starting from some almost singular situations (like the ones shown on Figs. 6 and 7) avoiding whole rotations to be used. 3. Finally, Figs. 8 and 9 show the behavior of the closed loop under the nominal model (Fig. 8) and under model uncertainties (Fig. 9). The uncertainties are introduced in equations (22) and (23) as follows J r ¨ + φξ ˙ · tan φ + δ2 (1 + δ ) ψ ξ˙ = − 1 2ml2 u2 = 2Jw (1 + δ3 )φ¨ ; δ1 = −0.1 ; δ2 = 0.05 ;

(37) δ3 = −0.1

(38)

Namely, δ1 and δ3 stand for relative error on the values of the physical parameters while δ2 stands for persistant external disturbance such as wind related drift term.

18

M. Alamir and F. Boyer

Fig. 9. Closed loop behavior. Initial condition x ¯(0) = −1, y¯(0) = 1, θ(0) = 3π/4 and ξ(0) = 1. Saturation levels umax = umax = 8. The uncertainties given by (37)–(38) 1 2 are used to test the robustness of the proposed feedback algorithm

6 Conclusion & Discussion It has been shown that by exploiting the particular structure of a class of nonholonomic system, it is possible to derive an eﬃcient steering procedure that can be used in a receding-horizon scheme to yield a stabilizing state feedback. By doing so, a complex dynamic problem is transformed into a rather simple discrete problem that can be solved by linear programming tools. The solution is successfully applied to the snakeboard constrained stabilization problem. The main drawback of the proposed approach is the constraint of almost stopping motion in-between each of the two control modes being used. This constraint can be practically avoided by using a state dependent switching parameter ε, namely ε(x) that would be large when x is far from the desired state and small in its neighborhood.

A Appendix Lemma 1. Let A ∈ Rn be a regular matrix. Deﬁne a compact subset C ⊂ Rn that contains a neighborhood of the origin. Let Cp ⊂ C be the subset deﬁned by (39) Cp := v ∈ C | vi = 0 ∀i ∈ {1, . . . , n} then there always exist v ∈ Cp such that Av ∈ Cp

♠

Re-injecting the Structure in NMPC Schemes

19

Proof Let O ⊂ C be a suﬃciently small neighborhood of the origin such that A−1 O is also a neighborhood of the origin that is contained in C. The set A−1 (O ∩ Cp ) is clearly an open neighborhood of the origin that contains an element in Cp . Let v be such an element, v clearly satisﬁes the requirements of the lemma.

References [1] D. Q. Mayne, J. B. Rawlings, C. V. Rao, and P. O. M. Scokaert, “Constrained model predictive control: Stability and optimality,” Automatica, vol. 36, pp. 789–814, 2000. [2] F. Allg¨ ower, T. Badgwell, J. Qin, J. Rawlings, and S. Wright, “Nonlinear predictive control and moving horizon estimation. an introductory overview,” in Advances in control Highlights of ECC’99, P. M. Frank, Ed., 1999, pp. 81–93. [3] M. Alamir and G. Bornard, “Globally stabilising discontinuous feedback law for a class of nonlinear systems.” Eur. J. Control, vol. 2, no. 1, pp. 44–50, 1996. [4] M. Alamir and H. Khennouf, “Discontinuous receding horizon control-based stabilizing feedback for nonholonomic systems in power form,” in Proceedings of the IEEE Conference on Decision and Control, New Orleans, 1995. [5] M. Alamir and F. Boyer, “Fast Generation of Attractive Trajectories for an Under-Actuated Satellite : Application to Feedback Control Design,” Journal of Optimization in Engineering, vol. 4, pp. 215–230, 2003. [6] N. Marchand and M. Alamir, “Numerical Stabilisation of a Rigid Spacecraft with two Actuators,” Journal of Dynamic Systems, Measurement and Control, vol. 125, pp. 489–491, 2003. [7] M. Alamir and N. Marchand, “Numerical stabilization of nonlinear systems -exact theory and approximate numerical implementation,” European Journal of Control, vol. 5, pp. 87–97, 1999. [8] M. Alamir, “ New path generation based receding horizon formulation for constrained stabilisation of nonlinear systems,” Automatica, vol. 40, no. 4, pp. 647– 652, 2004. [9] M. Alamir and N. Marchand, “Constrained minimum-time oriented feedback control for the stabilization of nonhoonomic systems in chained form,” Journal of Optimization Theory and Applications, vol. 118, no. 2, pp. 229–244, 2003. [10] A. D. Lewis, J. P. Ostrowski, J. W. Burdick, and R. M. Murray, “Nonholonomic mechanics and locomotion: the snakeboard example,” in Proceedings of the 1994 IEEE International Conference on Robotics and Automation, San Diega, USA, pp. 2391–2400. [11] J. Ostrowski and J. Burdick, “Controllability tests for mechanical systems with constraints and symmerties,” Journal Applied Mathematics and Computer Science, vol. 7, no. 2, pp. 101–127, 1997, proof of local controllability. [12] J. Ostrowski, “Steering for a class of dynamic nonholonomic systems,” IEEE Transaction on Automatic Control, vol. 45, no. 8, pp. 1492–1498, 2000.

20

M. Alamir and F. Boyer

[13] P. Vela and J. W. Burdick, “Control of biomimetic locomotion via averaging theory,” in Proceedings of IEEE Intl. Conf. on Robotics and Automation, ICRA2003. [14] S. Iannitti and K. M. Lynch, “Exact minimum control switch motion planning for a snakeboard,” in Proceedings of the IEEE/RSJ Intl. Conf. on Intelligent Robots and Systems, IROS2003, Las Vegas, USA.

Recent Advances on the Algorithmic Optimization of Robot Motion J.E. Bobrow, F.C. Park, and A. Sideris Department of Mechanical and Aerospace Engineering, University of California, Irvine, Irvine, CA 92697 {jebobrow, asideris}@uci.edu Summary. An important technique for computing motions for robot systems is to conduct a numerical search for a trajectory that minimizes a physical criteria like energy, control eﬀort, jerk, or time. In this paper, we provide example solutions of these types of optimal control problems, and develop a framework to solve these problems reliably. Our approach uses an eﬃcient solver for both inverse and forward dynamics along with the sensitivity of these quantities used to compute gradients, and a reliable optimal control solver. We give an overview of our algorithms for these elements in this paper. The optimal control solver has been the primary focus of our recent work. This algorithm creates optimal motions in a numerically stable and eﬃcient manner. Similar to sequential quadratic programming for solving ﬁnite-dimensional optimization problems, our approach solves the inﬁnite-dimensional problem using a sequence of linear-quadratic optimal control subproblems. Each subproblem is solved eﬃciently and reliably using the Riccati diﬀerential equation.

1 Introduction For many biological systems, it has long been observed that motion generation can likely be the result of a minimization process. The objective function used has been characterized by a physical criteria like energy, control eﬀort, jerk, or time. Unfortunately, to date the algorithms that generate such optimal motions have been successfully used on only the simplest of robots. The need for such an algorithm is increasing dramatically since many new walking, crawling, hopping machines, rehabilitation devices, and free-ﬂying air and space systems are currently under development. All of these devices will beneﬁt from a numerically stable and eﬃcient algorithm that produces optimal movements for them. We are interested in obtaining solutions to optimal control problems for systems of the form x˙ = f (x(t), u(t)) , (1) where f : Rn × Rm → Rn ∈ C 1 (continuously diﬀerentiable) and x(0) = xo . We assume that the optimal control cost functional has the form

22

J.E. Bobrow et al.

m q

l 1= 0.5

yp

1

y

l2= 1.0 q2

Fig. 1. The left-hand system was used for Case 1, a fully actuated robot. Case 2 is the same system, but with the base joint unactuated. The center system represents Case 3, which is an application to human step rehabilitation. The right system is a hopping machine for Case 4

Minimize J(u(t)) = φ(x(tf )) + u(t)

tf L(x(t), u(t), t) dt ,

(2)

0

subject to (1) with φ : Rn → R ∈ C 1 and L : Rn × Rm × R → R ∈ C 1 . Although the Maximum Principle [3] provides the optimality conditions for the solution to (2), it is not suitable for numerical computation. Because of the importance of solving these problems, many numerical algorithms and commercial software packages have been developed to solve them since the 1960’s [1]. Most of the existing algorithms do not have adequate numerical stability properties and are too slow computationally to solve optimal control problems for current multibody systems. As a means to discuss the numerical features of algorithms, we provide example solutions in four case studies. These examples demonstrate the strength and limitations of current numerical algorithms. Figure 1 shows model systems used for four case studies in this paper. Case 1 is a minimum eﬀort control of a fully actuated robot. We have found that with care in the choice of basis functions, a parameterization of the joint motion can be used along with a static optimization method to adequately solve this problem. In addition, this approach easily allows one to handle the case when some of the initial states are free to vary, as was done in [10]. Case 2 is an underactuated robot. We have found that even with exact gradients of the dynamics, our parameterization of the motion had numerical problems from round-oﬀ errors during the simulations. Case 3 is an application to human leg step rehabilitation [21]. We experienced even more numerical problems for this problem due to the added ground constraint. Finally, Case 4 is a simpliﬁed gas actuated hopping machine. We found it diﬃcult to achieve stable convergence with existing methods for this case. The cause was numerical integration errors introduced at the sudden change in the dynamics between the stance phase and the ﬂight phase, and the fact that the times for the switch from stance

Recent Advances on the Algorithmic Optimization of Robot Motion

23

to ﬂight were not known apriori. The approach developed in this paper (see also [14] ) eﬃciently solves Case 4 , and we feel that it has great potential for application to general optimal control problems for robot systems. 1.1 First Order Necessary Conditions for the Solution of the Optimal Control Problem In order to discuss the solution to Cases 1–4, we ﬁrst brieﬂy summarize the ﬁrst order necessary conditions for the optimal control of a general nonlinear system (see [3] for more details). First deﬁne the Hamiltonian as H(x, u, λ, t) ≡ L(x, u, t) + λT f (x, u) ,

(3)

where L and f were deﬁned in (1) and (2). Then in (3), λ(t) is chosen to satisfy the costate or adjoint equations λ˙ = −Hx (xo (t), uo (t), λ(t), t) ,

(4)

where Hx and Hu (used below) denote partial derivatives of H with respect to x and u respectively, and the boundary conditions are λ(tf ) = φTx (xo (tf )). Let uo (t) be a nominal control, xo (t) and λo (t) be the corresponding solutions to (1) and (4), respectively. For general problems, the ﬁrst order necessary conditions for a local minimum of J require that H(xo (t), uo (t), λ(t), t) be minimized with respect to uo (t) subject to any constraints on it. For unconstrained controls u, the condition on H is Hu (xo (t), uo (t), λ(t), t) = 0. Note that both Hx and Hu require diﬀerentiation of the state equations (1) with respect to x and u and evaluation of these derivatives along the solution (xo (t), uo (t)). For multibody dynamic systems with more than a few degrees of freedom, the derivatives are generally not available due to the complexity of the equations of motion. However, in [15], the sensitivity algorithms based on matrix exponentials are developed speciﬁcally for this purpose. A brief introduction to that work is presented next. 1.2 Geometric Tools for Multibody Systems Analysis To represent robot systems and their dynamics, we use a set of analytical tools for multibody systems analysis based on the mathematics of Lie groups and Lie algebras [11, 13]. In the traditional formulation, a rigid motion can be represented with the Denavit-Hartenberg parameters as a 4 × 4 homogeneous transformation T (θ, d) ∈ SE(3), where θ is the rotation about the z-axis and

24

J.E. Bobrow et al.

d is the translation along it. For a prismatic joint, d varies while θ is held constant. For a revolute joint, θ varies while d is held constant. With the geometric formulation, for either type of joint the transformation has the form T (θ, d) = eAx M , where x = θ for a revolute joint or x = d for a prismatic joint, A contains the joint axis or direction, and M is a constant (M = T (0, d) for a revolute joint, M = T (θ, 0) for a prismatic joint.) This exponential mapping and its inverse have explicit formulas: exp : se(3) → SE(3) and its inverse log : SE(3) → se(3) [13]; here se(3) denotes the Lie algebra of SE(3). Although SE(3) is not a vector space, se(3) is: the log formula provides a set of canonical coordinates for representing neighborhoods of SE(3) as open sets in a vector space. The derivative of the exponential map with respect to the joint displaceAx M. In the coding of multibody dynamics algorithms, ment x is just dT dx = Ae the exponential is the lowest level primitive required for all computations. One never needs to deal with sine and cosine terms or with making a distinction for each joint type. The use of matrix exponentials to represent the link to link transformations for robot systems allows one to clarify the kinematic and dynamic equations. In the case of open chains containing prismatic or revolute joints, the forward kinematics can be written as a product of matrix exponentials [2]. Speciﬁcally, given a choice of inertial and tool reference frames, and a zero position for the mechanism, the forward kinematics can be written uniquely as T0n (q1 , . . . , qn ) = eA1 q1 · · · eAn qn where q1 , . . . , qn are joint variables, and A1 , . . . , An ∈ se(3). The kinematics of closed chains can be obtained by further adding a set of algebraic constraints. In order to determine optimal motions for the multibody systems of interest, a complete dynamic model is needed. In [13] a Lie group formulation of the dynamics has been developed, in which closed-form expressions for the inertia matrix and Coriolis terms are available. Using this representation, the forward and inverse dynamics can also be computed eﬃciently with O(n) recursive algorithms. The inverse dynamics algorithm is shown in Fig. 2. In this algorithm, Vi ∈ se(3) is the linear and angular velocity of link i, W is the applied force and moment, J is a 6 × 6 matrix of mass and inertia, Si is the joint screw, and Ad and ad are standard operators from diﬀerential geometry [11]. A useful computational feature of this algorithm is that no distinction needs to be made for revolute or prismatic joints. In [15], this algorithm was extended to forward and inverse dynamics of partially actuated systems, and to produce the derivatives needed for many optimal control solvers, as discussed in the previous section. Given the ability to compute the dynamics and derivatives of relatively complex systems, we now discuss some representative examples. The following

Recent Advances on the Algorithmic Optimization of Robot Motion •

Initialization

•

Forward recursion: for i = 1 to n do

25

V0 = V˙ 0 = Wn+1 = 0

Ti−1,i = Mi eSi qi Vi = AdT −1 (Vi−1 ) + Si q˙i i−1,i

V˙ i = Si q¨i + AdT −1 (V˙ i−1 ) + AdT −1 (Vi−1 ), Si q˙i i−1,i

•

i−1,i

Backward recursion: for i = n to 1 do Wi = Ad∗T −1 (Wi+1 ) + Ji V˙ i − ad∗Vi (Ji Vi ) i,i+1

τi =

SiT Wi

Fig. 2. The POE recursive Newton-Euler inverse dynamics algorithm

case studies demonstrate both successes and diﬃculties that we have encountered in applying the optimality conditions of Sect. 1.1 to multibody systems problems.

2 Some Representative Case Studies 2.1 Case 1: Fully Actuated Robot Consider the case of ﬁnding the minimum eﬀort control which moves the two link planar robot shown in Fig. 1 from an outstretched horizontal position to a vertical position. Assume that both joints of the arm be actuated, and let the cost function be: 1 ˙ f )|| + J = c1 ||q(tf ) − qd || + c2 ||q(t 2 2

tf ||u||2 dt ,

2

(5)

0

where q ∈ R2 are the joint angles, qd = [ π2 , 0]T , are the desired ﬁnal joint positions in radians, u ∈ R2 are the corresponding joint torques, and c1 = c2 = 100 reﬂects the desire to reach the ﬁnal vertical conﬁguration with little error. We used several approaches to solve this optimal control problem. The most straight forward approach, called the “shooting method,” is to cleverly ﬁnd the initial costate, λ(0), such that when the state and costate equations are integrated forward from t = 0 to t = tf with Hu = 0, the proper ﬁnal condition on the costate is satisﬁed (λ(tf ) = φTx (xo (tf )). Unfortunately, because the costate equations are not stable and highly sensitive to the initial conditions [3], the shooting method failed when applied to this problem. A

26

J.E. Bobrow et al.

second, more successful, approach used by several researchers [5, 4, 10, 12] in robotics is to approximate the motion of the joints using basis functions such as splines or a truncated Fourier Series. For instance, we used quintic B-spline polynomials with 12 uniformly spaced knot intervals to parameterize our solution as q = q(t, P ) with P ∈ R2 × R12 being the amplitude of the spline basis functions. For any choice of P, we can compute the required control u(t) by diﬀerentiating q(t, P ) with respect to time to obtain q˙ and q¨ and, evaluating the equations of motion M (q)¨ q + h(q, ˙ q) = u .

(6)

In order to use this “direct approach,” we guessed an initial motion that kept the second link aligned with the ﬁrst with q2 (t) = 0, and moved the ﬁrst link smoothly from q1 (0) = 0 to q1 (tf ) = π/2, with tf = 2 seconds. We then computed J(u(P )) in (5) and its gradient ∇JP using adaptive Simpson quadrature. In this case, the integrand is known explicitly throughout the integral since all the terms in (6) are known explicitly in terms of P from the joint angles q = q(t, P ). Given J(P ) and its gradient, we could easily minimize it over P using Matlab’s BFGS [9] algorithm in the function “fminunc.” Figure 3 shows the locally optimal solution found to this problem using the parameter optimization approach mentioned above. The frames are spaced at equal intervals in time, with tf = 2 seconds. At ﬁrst the robot allows gravity to take over and it swings down while folding up the second link. It then swings the ﬁrst joint into the upward posture. A small pumping motion is applied to the second link in order to move it into the vertical posture. The initial value of the eﬀort term in the cost function was 73.6 and the ﬁnal value was 9.9. The computation time for this problem was about 2 minutes on a PIII-800 PC. We have used this basic approach with our dynamics tools to solve a weight-lifting problem for a much more complex Puma 762 robot in [20], where we tripled the payload above the manufacturers speciﬁcations. Even though

Fig. 3. Final path for fully-actuated planar 2R problem

Recent Advances on the Algorithmic Optimization of Robot Motion

27

the solution to the above problem was fairly stable numerically, we still needed to choose an appropriate set of basis functions in order to approximate the numerical solution. In this paper, we develop an approach that does not use any basis functions to approximate the solution. 2.2 Case 2: Underactuated Robot The optimal control problem becomes less numerically stable and more difﬁcult to solve when the system is underactuated. For instance, suppose the motor attached to the ﬁrst joint of the above 2R robot is disconnected. The system then only has a motor at its elbow and is often called the Acrobot, which has been studied by Spong [17] and others. Consider the swing-up motion problem where the system starts from a hanging downward posture and the optimal control problem is to ﬁnd an open-loop control torque for the elbow, if one exists, that drives the system to the upward posture of the previous example. This case is much more challenging than the previous one because we can no longer use (6) to compute u(P ) because the system is underactuated. Choose the same objective function as (5), except the control u is now a scalar. One way to approach the problem is approximate the control with a set of basis functions and integrate the 4 state equations in order to evaluate (5). Any gradient-based numerical optimization will need both the value of J(P ) and its gradient ∇p J. Assuming that the state, costate, and boundary conditions are satisﬁed, the required derivative is dJ = dpi

tf Hu (xo (t), uo (t), λ(t), t)

du dt . dpi

(7)

0

This derivative is valid for any (xo (t), uo (t)) even if they are not optimal [3]. Then, in order to evaluate the objective function and its gradient for use in any gradient-based nonlinear optimization algorithm, the following steps must be performed: • Select a set of basis functions and parameters to deﬁne uo (t, P ). • Integrate the diﬀerential equations (1) of motion from 0 to tf to obtain xo (t) and J(P ). • Evaluate the costate (4) boundary conditions λ(tf ) = φTx (xo (tf )) and integrate the costate equations backwards in time from tf to 0 to obtain λ(t). • Evaluate the gradient of J using (7). We used this method to solve the Acrobot swing-up problem with one modiﬁcation. That is, instead of viewing the control u2 as the torque at joint 2, we deﬁned the motion of joint q2 with our spline functions and considered that to be the control in (1). In doing so, the actual joint torque u2 (t) can be computed algebraically from (6) assuming that (q, q) ˙ are known. The advantage of doing this is that the state reduces to 2 dimensions in this case

28

J.E. Bobrow et al.

Fig. 4. Optimal swing up motion for an Acrobot with q1 (0) = −1.0

x = [q1 , q˙1 ]T since the motion of the second joint is known from the parameterization. Note that it is not apparent what, if any, elbow motion will drive the system to the desired ﬁnal conﬁguration. Our initial guess for the elbow motion was very poor. We did not move the elbow joint at all during the motion, and let the system move like a rigid pendulum would with an initial con˙ = q2 (0) = q˙2 (0) = 0.0. Figure 4 shows the ﬁnal dition q1 (0) = −1.0, q(0) motion obtained using Matlab’s nonlinear parameter optimization with gradients computed as described above. The motion produced is similar to those proposed by Spong [17], in which the lower link pumps energy into the system and this energy causes the ﬁrst link to move into the vertical position. In addition to this example, we have used this basic approach to solve for much more complex optimal high-dive motions for a human-like diver in [16]. When we computed the above solution to the underactuated Acrobot, we did not expect numerical diﬃculties, since we had the exact gradient of the objective function and the optimization algorithm has well-established convergence properties for this case [9]. However, we did encounter some numerical problems and had to adjust some of the tolerances in the optimizer in order to achieve convergence, and the computation time, even in the best of cases (about 5 minutes on a PIII-800 PC), was much longer than in the previous example. The problem was that the round-oﬀ errors encountered during the numerical solution of (1), (4), and (7) lead to large relative errors in the gradient when the algorithm is near convergence. The algorithm developed in this paper alleviates these diﬃculties. 2.3 Case 3: Underactuated Systems with Contact Constraints-Human Step Training Example One important application of our proposed algorithm is the generation of optimal inputs for the robotic rehabilitation of paralyzed individuals [22]. In

Recent Advances on the Algorithmic Optimization of Robot Motion

29

−10 0 10 20

−100

−50

0 x (cm)

50

100

120

120

100

100

80

80 y (cm)

y (cm)

z (cm)

[21] we examined a method to control the stepping motion of a paralyzed person suspended over a treadmill (see Fig. 1) using a robot attached to the pelvis. Leg swing motion was created by moving the pelvis without contact with the legs. The problem is formulated as an optimal control problem for an underactuated articulated chain. Similar to the underactuated Acrobot, the optimal control problem is converted into a discrete parameter optimization and a gradient-based algorithm is used to solve it. To simulate a paralyzed person, a dynamic model for a branched kinematic chain was used to approximate the kinematics and dynamics of a human subject. For the dynamic analysis, four rigid bodies were used, with the head arms and torso lumped together as a single body, and the upper leg, lower leg, and foot modeled as single rigid bodies. The torso had ﬁve degrees of freedom with three translations and two rotations. Rotation about an axis through the upper leg joints was ignored. The upper leg was connected to the torso with a three degree of freedom ball joint. The lower leg was connected to the upper leg by a one dof rotation. Similarly, the foot was connected to the lower leg by a one dof rotation. Thus the entire system had ten degrees of freedom. For the swing hip, knee and ankle joints, a torque was applied to simulate the stiﬀness of passive tissue, but no torque from the muscles since the person is assumed to be paralyzed. A total of 32 B-spline parameters were used in the optimization to specify the motion of the swing hip. This problem diﬀered from the Acrobot because we had to constrain the motion of the foot to avoid contact with the ground, and the motion of the legs to avoid contact with each other. We used penalty functions to enforce these collision avoidance constraints. In the optimization results shown in Fig. 5, we found the motion of the swing hip that produced a step for the swing leg that was as close as possible

60 40

60 40

20

20

0

0

−100

−50

0 x (cm)

50

100

20 10 0−10 z (cm)

Fig. 5. Motion of the pelvis can be used to create motion for a paralyzed swing leg. The solid lines show gait which results from optimal motion of the pelvis, and the dashed lines are the gait recorded from the motion capture system

30

J.E. Bobrow et al.

to a normal human gait. The optimal control found from our algorithm lifted the swing hip to avoid collision between the swing leg and the ground. At the same time, it twisted the pelvis to pump energy into the paralyzed leg and moved the leg close to the desired ﬁnal conﬁguration, while avoiding collision between the legs. Thus we found a strategy that could achieve repetitive stepping by shifting the pelvis alone. The optimized, pelvic motion strategies are comparable to “hip-hiking” gait strategies used by people with lower limb prostheses or hemiparesis. Even though there were relatively few parameters (32) in the optimization, it was not numerically stable and took approximately 4 hours to converge, with approximately ﬁve minutes used for every major iteration of the optimization. The problems were again due to round-oﬀ errors introduced by the computation of the gradient in (7). The penalty functions for obstacle avoidance exacerbated the problem since they eﬀectively created a “stiﬀ” system of diﬀerential equations. The above results only considered the swing phase of the gait cycle. In our initial attempts to combine the stance phase with the swing phase in the optimal control solution were numerically unstable and did not converge to a solution. Based on our initial results from the simple hopping machine considered in Case 4, we believe that with our new algorithm it will be possible to combine the stance and swing phases and reliably compute an optimal motion for Case 3 in just a few minutes of computation time. This would make it possible to compute an optimal motion for each patient in a clinical setting. 2.4 Case 4: Minimum Fuel Hopping In order to explore the diﬃculties associated with the change in dynamics between the stance phase and swing phase of motion mentioned in Case 3, we considered a simple one-dimensional hopping system shown in Fig. 1. This system is driven by a pneumatic actuator, with the location of the piston relative to the mass under no external loading deﬁned as yp . After contact occurs with the ground with y ≤ yp , the upward force on the mass from actuator can be approximated by a linear spring with F = k(yp − y), where k is the spring constant. The position yp can be viewed as the unstretched spring length and it can be easily changed by pumping air into or out of either side of the cylinder. The equations of motion for the mass are m¨ y = F (y, yp ) − mg, where mg is the force due to gravity, and F (y, yp ) =

0 y>yp . k(yp −y) otherwise

Note

that in this case F (y, yp ) is not diﬀerentiable at y = yp , and gradient-based methods will have diﬃculties with this. However, the discontinuity in the derivative can easily be smoothed. For instance, let the spring compression be e = yp − y and choose an α > 0, then

Recent Advances on the Algorithmic Optimization of Robot Motion

  0 F (e) =

 

0>e 0≤e<α

k 2 2α e

ke −

31

kα 2

otherwise

is C 1 . The ﬁnal equation of motion for this system relates the air ﬂow into the cylinder, which is the control u(t), to the equilibrium position yp of the piston. Assume for the following that the equation y˙p = u approximates this relationship. When the hopping machine begins its operation, we are interested in starto at time tf . If we minimize ing from rest, and reaching a desired hop height yN J(u) =

N −1 1 tf o 2 qf in (y(N ) − yN q yp (n)2 + r u(n)2 , ) + y(N ˙ )2 + 2 2N n=0

(8)

the terms outside the summation reﬂect the desire to reach the height at time tf with zero velocity, and the terms inside the summation reﬂect the desire to minimize the gas used to achieve this. The weighting on yp is used to keep the piston motion within its bounds. We ﬁrst attempted to solve this problem by parameterizing the control u(t) with B-splines and using the basic steps used in Cases 2 and 3. Even after considerable tweaking of tolerances, the gradient-based algorithm would not converge. This drove us to develop the algorithm described in the next section.

3 Problem Formulation and Background Results We assume that the dynamic system deﬁned by (1) and the performance measure (2) have been descretized by a suitable numerical integration scheme. To simplify the notation, we use the same function and variable names for the discrete-time versions of the continuous-time variables. A more detailed discussion of this material can be found in [14]. Minimize u(n), x(n)

J = φ(x(N )) +

subject to

N −1

L(x(n), u(n), n)

(9)

n=0

x(n + 1) = f (x(n), u(n)); x(0) = x0

(10)

We further assume a quadratic performance index, namely: L(x(n), u(n), n) =

and φ(x) =

1 [x(n) − xo (n)]T Q(n)[x(n) − xo (n)] 2 + [u(n) − uo (n)]T R(n)[u(n) − uo (n)]

1 [x − xo (N )]T Q(N )[x − xo (N )] 2

(11)

(12)

32

J.E. Bobrow et al.

In (11) and (12), uo (n), xo (n), n = 1, . . . N are given control input and state oﬀset sequences. In standard optimal regulator control problem formulations, uo (n), xo (n) are usually taken to be zero with the exception perhaps of xo (N ), the desired ﬁnal value for the state. The formulation considered here addresses the more general optimal tracking control problem and is required for the linear quadratic step in the proposed algorithm presented in Sect. 3.2. 3.1 First Order Optimality Conditions We next brieﬂy review the ﬁrst order optimality conditions for the optimal control problem of (9) and (10), in a manner that brings out certain important interpretations of the adjoint dynamical equations encountered in a control theoretic approach and Lagrange Multipliers found in a pure optimization theory approach such as that mentioned in Sect. 1.1. Let us consider the cost-to-go: J(n) ≡

N −1

L(x(k), u(k), k) + φ(x(N ))

(13)

n=k

with L and φ as deﬁned in (11) and (12) respectively. We remark that J(n) is a function of x(n), and u(k), k = n, . . . , N − 1 and introduce the sensitivity of the cost to go with respect to the current state: λT (n) =

∂J(n) ∂x(n)

(14)

Since J(n) = L(x(n), u(n), n) + J(n + 1) ,

(15)

we have the recursion: λT (n) = Lx (x(n), u(n), n) + λT (n + 1)fx (x(n), u(n)) = [x(n) − xo (n)]T Q(n) + λT (n + 1)fx (x(n), u(n))

(16)

by using (11) and where Lx and fx denote the partials of L and f respectively with respect to the state variables. The previous recursion can be solved backward in time (n = N − 1, . . . , 0) given the control and state trajectories and it can be started with the ﬁnal value: λT (N ) =

∂L(N ) = [x(N ) − xo (N )]T Q(N ) ∂x(N )

(17)

derived from (12). We now compute the sensitivity of J(n) with respect to the current control u(n). Clearly from (15), ∂J(n) = Lu (x(n), u(n), n) + λT (n + 1)fu (x(n), u(n)) ∂u(n) = [u(n) − uo (n)]T R(n) + λT (n + 1)fu (x(n), u(n))

(18)

Recent Advances on the Algorithmic Optimization of Robot Motion

33

∂J(n+1) ∂x(n+1) T since ∂J(n+1) ∂u(n) = ∂x(n+1) · ∂u(n) = λ (n + 1)fu (x(n), u(n)). In (19) Lu and fu denote the partials of L and f respectively with respect to the control variables and (11) is used. ∂J = ∂J(n) Next note that ∂u(n) ∂u(n) since the ﬁrst n terms in J do not depend on u(n). We have then obtained the gradient of the cost with respect to the control variables, namely: % & ∂J(0) ∂J(1) ∂J(N − 1) ∇u J = ... . (19) ∂u(0) ∂u(1) ∂u(N − 1)

Assuming u is unconstrained, the ﬁrst order optimality conditions require that ∇u J = 0.

(20)

We remark that by considering the Hamiltonian H(x, u, λ, n) ≡ L(x, u, n) + λT f (x, u) ,

(21)

∂J we have that Hu (x(n), u(n), λ(n + 1), n) ≡ ∂u(n) , i.e. we uncover the generally known but frequently overlooked fact that the partial of the Hamiltonian with respect to the control variables u is the gradient of the cost function with respect to u. We emphasize here that in our approach for solving the optimal control problem, we take the viewpoint of the control variables u(n) being the independent variables of the problem since the dynamical equations express (recursively) the state variables in terms of the controls and thus can be eliminated from the cost function. Thus in taking the partials of J with respect to u, J is considered as a function u(n), n = 0, . . . , N − 1 alone, assuming that x(0) is given. With this perspective, the problem becomes one of unconstrained minimization, and having computed ∇u J, Steepest Descent, Quasi-Newton, and other ﬁrst derivative methods can be brought to bear to solve it. Note that we are limiting the problem to the case of unconstrained controls, but for many problems in robotics, control constraints are not the limiting factor. In addition, soft constraints on actuation bounds can often be enforced with penalty functions with good ﬁdelity since these constraints are usually not known precisely. With the control unconstrained, and with the large-scale character of the problem, methods that take advantage of the special structure of the problem become viable. The Linear Quadratic Regulator algorithm is such an approach in case of linear dynamics. We review it next and we remark that it corresponds to taking a Newton step in view of the previous discussion.

3.2 Linear Quadratic Tracking Problem We next consider the case of linear dynamics in the optimal control problem of (9) and (10). In the following, we distinguish all variables corresponding to the linear optimal control problem that may have diﬀerent values in the nonlinear

34

J.E. Bobrow et al.

optimal control problem by using an over-bar. When the cost is quadratic as in (11) we have the well-known Linear Quadratic Tracking problem. The control theoretic approach to this problem is based on solving the ﬁrst order necessary optimality conditions (also suﬃcient in this case) in an eﬃcient manner by introducing the Riccati equation. We brieﬂy elaborate on this derivation next, for completeness and also since most references assume that the oﬀset sequences xo (n) and uo (n) are zero. First, we summarize the ﬁrst order necessary optimality conditions for this problem. x ¯(n + 1) = A(n)¯ x(n) + B(n)¯ u(n) T o T ¯ λ (n) = [¯ x(n) − x ¯ (n)] Q(n) ¯ T (n + 1)A(n) +λ ¯ ∂ J(n)/∂ u ¯(n) = [¯ u(n) − u ¯o (n)]T R(n) T ¯ (n + 1)B(n) = 0 +λ

(22) (23) (24)

Note that the system dynamical equations (22) run forward in time n = 0, . . . , N − 1 with initial conditions x ¯(0) = x ¯0 given, while the adjoint dynamical equations (24) run backward in time, n = N − 1, . . . , 0 with ﬁnal ¯ T (N ) = [¯ x(N ) − x ¯o (N )]T Q(N ). From (25), we obtain conditions λ ¯ + 1) u ¯(n) = u ¯o (n) − R(n)−1 B(n)T λ(n

(25)

and by substituting in (22) and (24), we obtain the classical two-point boundary system but with additional forcing terms due to the x ¯o (n) and u ¯o (n) sequences. ¯ + 1) x ¯(n + 1) = A(n)¯ x(n) − B(n)R(n)−1 B(n)T λ(n

(26)

+B(n)¯ uo (n) ¯ T (n) = Q(n)¯ ¯ + 1) − Q(n)¯ λ x(n) + AT (n)λ(n xo (n)

(27)

The system of (26) and (27) can be solved by the sweep method [3], based on the postulated relation ¯ λ(n) = P (n)¯ x(n) + s(n)

(28)

where P (n) and s(n) are appropriate matrices that can be found as follows. For n = N , (28) holds with s(N ) = −Q(N )¯ xo (N ).

P (N ) = Q(N ),

(29)

We now substitute (28) in (26) and after some algebra we obtain x ¯(n + 1) = M (n)A(n)¯ x(n) + v(n)

(30)

where we deﬁned −1 M (n) = I + B(n)R(n)−1 B(n)T P (n + 1) o

−1

v(n) = M (n)B(n)[¯ u (n) − R(n)

T

B(n) s(n + 1)]

(31) (32)

Recent Advances on the Algorithmic Optimization of Robot Motion

35

¯ ¯ + 1) in (27) in terms of x By replacing λ(n) and λ(n ¯(n) and x ¯(n + 1) from (28), we obtain P (n)¯ x(n) + s(n) = Q(n)¯ x(n) + AT (n) [P (n + 1)¯ x(n + 1) +s(n + 1)] − Q(n)¯ xo (n),

and by expressing x ¯(n + 1) from (30) and (32) above, we get P (n)¯ x(n) + s(n) = Q(n)¯ x(n) + AT (n)P (n + 1)M (n)A(n)¯ x(n) −AT (n)P (n + 1)M (n)B(n)R(n)−1 B(n)T s(n + 1) +AT (n)P (n + 1)M (n)B(n)¯ uo (n) +AT (n)s(n + 1) − Q(n)¯ xo (n) The above equation is satisﬁed by taking P (n) = Q(n) + AT (n)P (n + 1)M (n)A(n)

(33)

s(n) = AT (n)[I − P (n + 1)M (n)B(n)R(n)−1 B(n)T ]s(n + 1) +AT (n)P (n + 1)M (n)B(n)¯ uo (n) − Q(n)¯ xo (n)

(34)

Equation (33) is the well-known Riccati diﬀerence equation and together with ¯o (n) are zero, the auxiliary equation (34), which is unnecessary if x ¯o (n) and u are solved backward in time (n = N − 1, . . . , 1), with ﬁnal values given by (29) and together with (31) and (32). The resulting values P (n) and s(n) are stored and used to solve forward in time (n = 0, . . . , N − 1), (30) and (25) for the optimal control and state trajectories. These equations are summarized in Table 1. 3.3 Formulation of the SLQ Algorithm In the proposed SLQ algorithm, the control at stage k + 1 is found by performing a one-dimensional search from the control at stage k and along a search direction that is found by solving an Linear Quadratic (LQ) optimal control problem. Speciﬁcally, let Uk = [u(0) u(1) . . . u(N − 1)] be the optimal solution candidate at step k, and Xk = [x(1) x(2) . . . x(N )] the corresponding state trajectory obtained by solving the dynamical equations (10) using Uk and with the initial conditions x(0). We next linearize the state equations (10) about the nominal trajectory of Uk and Xk . The linearized equations are x ¯(n + 1) = fx (x(n), u(n))¯ x(n) + fu (x(n), u(n))¯ u(n)

(35)

with initial conditions x ¯(0) = 0. We then minimize the cost index (9) with re¯k = [¯ spect to u ¯(n). The solution of this LQ problem gives U u(0) u ¯(1) . . . u ¯(N − 1)], the proposed search direction. Thus, the control variables at stage k + 1 of the algorithm are obtained from

36

J.E. Bobrow et al.

Table 1. Algorithm to solve the Discrete-Time Finite-Horizon Linear Quadratic Tracking optimal control problem 1. Solve backward (n = N − 1, . . . , 0) with PN ≡ QN and sN ≡ −QN x ¯oN : −1 M (n) = I + B(n)R(n)−1 B(n)T P (n + 1) P (n) = Q(n) + A(n)T P (n + 1)M (n)A(n) s(n) = A(n)T I − P (n + 1)M (n)B(n)R(n)−1 B(n)T s(n + 1) +A(n)T P (n + 1)M (n)B(n)¯ uo (n) − Q(n)¯ xo (n) 2. Solve forward (n = 0, . . . , N − 1) with x ¯(0) = x ¯0 : v(n) = M (n)B(n)[¯ uo (n) − R(n)−1 B(n)T s(n + 1)] x ¯(n + 1) = M (n)A(n)¯ x(n) + v(n) ¯ λ(n + 1) = P (n + 1)¯ x(n + 1) + s(n + 1) ¯ + 1) u ¯(n) = u ¯o (n) − R(n)−1 B(n)T λ(n

¯k Uk+1 = Uk + αk · U

(36)

where αk ∈ R+ is appropriate stepsize the selection of which is discussed later in the paper. Note again our perspective of considering the optimal control problem as an unconstrained ﬁnite-dimensional optimization problem in U . ¯k as computed above is not the steepest descent direcWe emphasize that U tion. It is the solution to a linear quadratic tracking problem for a nonlinear system that has been linearized about Uk . Note that the objective function is not linearized for this solution. Our algorithm is diﬀerent than standard Quasilinearization [3] and Neighboring Extremal [18] methods where the adjoint equations are also linearized and two-point boundary problems are solved. 3.4 Properties of the SQL Algorithm In this section, we prove two important properties of the proposed algorithm. ¯ is a descent direction. First, we show that search direction U Theorem 1 Consider the discrete-time nonlinear optimal control problem of (9) and (10), and assume a quadratic cost function as in (11) and (12) with Q(n) = QT (n) ≥ 0, Q(N ) = QT (N ) ≥ 0 and R(n) = RT (n) > 0, n = 0, 1, . . . , N − 1. Also consider a control sequence U ≡ [u(0)T . . . uT (N − 1)]T and the corresponding state trajectory X ≡ [x(1)T . . . xT (N )]T . Next, linearize system (10) about U and X and solve the following linear quadratic problem:

Recent Advances on the Algorithmic Optimization of Robot Motion

37

Minimize

1 T J¯ = [¯ x (N ) − x ¯o (N )]Q(N )[¯ x(N ) + x ¯o (N )] 2 u ¯(n), x ¯(n) N −1 1 [¯ x(n) − x ¯o (n)]T Q(n)[¯ x(n) − x ¯o (n)] 2 n=0 ' u(n) − u ¯o (n)] + [¯ u(n) − u ¯o (n)]T R(n)[¯

(37)

x ¯(n + 1) = fx (x(n), u(n))¯ x(n)

(38)

+

subj. to + fu (x(n), u(n))¯ u(n);

x ¯(0) = 0 ,

¯ ≡ [¯ where x ¯o (n) ≡ xo (n) − x(n), u ¯o (n) ≡ uo (n) − u(n). Then if U u(0)T . . . u ¯T T (N − 1)] is not zero, it is a descent direction for the cost function (9), i.e. ¯ ) < J(U ) for some α > 0. J(U + α · U Proof:

¯ is a descent direction by showing that: We establish that U ¯= ∇u J · U

N −1 n=0

∂J(n) u ¯(n) < 0 , ∂u(n)

(39)

since ∇u J in (19) is the gradient of the cost function with respect to the control variables. Now, the components of ∇u J are expressed in (19) in terms of the adjoint variables λ(n) that satisfy recursion (16) with ﬁnal values given by (17). On the other hand, x ¯(n) and u ¯(n) together with ad¯ joint variables λ(n) satisfy the ﬁrst order optimality conditions for the linear quadratic problem given in (22), (24) and (25), where A(n) = fx (x(n), u(n)) and B(n) = fu (x(n), u(n)). Let us deﬁne ˜ ¯ λ(n) = λ(n) − λ(n)

(40)

and note from (16) and (24) that ˜ + 1)T A(n); λ(N ˜ ) = Q(N )¯ ˜ T =x ¯(n)T Q(n) + λ(n x(N ) . λ(n)

(41)

Next through the indicated algebra, we can establish the following relation: ∂J(n) ·u ¯(n) = ∂u(n) ¯(n) = [u(n) − uo (n)]T R(n) + λ(n + 1)T B(n) u (using (19)) ˜ + 1)T B(n)¯ u(n) − u ¯(n)T R(n)¯ u(n) = −λ(n (using (25)) ˜ + 1)T x ˜ + 1)T A(n)¯ = −λ(n ¯(n + 1) + λ(n x(n) − u ¯(n)T R(n)¯ u(n) (using (39)) ˜ + 1)T x ˜ Tx = −λ(n ¯(n + 1) + λ(n) ¯(n) − x ¯(n)T Q(n)¯ x(n) −u ¯(n)T R(n)¯ u(n).

(using (41))

38

J.E. Bobrow et al.

Finally, summing up the above equation from n = 0 to n = N − 1 and noting ¯ ) = Q(N )¯ that x ¯(0) = 0 and from (29) that λ(N x(N ), gives: ¯ = ∇u J · U

N −1 n=0

=−

∂J(n) ·u ¯(n) ∂u(n)

N −1

(42)

[¯ x(n)T Q(n)¯ x(n) + u ¯(n)T R(n)¯ u(n)]

n=0

x(N ) < 0 −x ¯T (N )Q(N )¯

and the proof of the theorem is complete. ¯ can be found by the LQR algorithm We remark that the search direction U of Table 1 with A(n) = fx (x(n), u(n)) and B(n) = fu (x(n), u(n)). The next result shows that the proposed SLQ algorithm does in fact converge to a control locally minimizing the cost function (9). Theorem 2 Starting with an arbitrary control sequence U0 , compute recursively new controls: ¯k (43) Uk+1 = Uk + αk · U where the direction Uk is obtained as in Theorem 1 by solving the LQR problem of (37) and the linearized system (39) about the solution Uk and corresponding ¯k ] over α > 0. state trajectory Xk ; also αk is obtained by minimizing J[Uk +αU Then Uk converges (in the Euclidean norm sense) to a control that locally minimizes the cost function (9) subject to the system equations (10). Proof: See [14], or note that given the result of the previous theorem, standard convergence proofs (see [7]) apply with either an exact or an inexact linesearch such as the Armijo, Goldstein, or Wolfe search rules [9].

4 Numerical Example We conducted numerical experiments the hopping system discussed in Case 4, above. We minimized (8) with the following parameters: k/m = 100, g = 386.4, α = 0.1. We assumed that all states were initially zero, and that the initial control sequence was zero. The cost function parameters were selected as: tf = 1, q = 1000, r = 1.0. As in the last example, a simple Euler approximation was used to discretize the equations, with N = 50. As shown in Fig. 6, the algorithm produced an alternating sequence of stance phases and ﬂight phases for the hopping system and it naturally identiﬁed the times to switch between these phases. If one were to use collocation methods to solve this problem with explicit consideration of the diﬀerent dynamics in the diﬀerent phases, one would have to guess the number of switches between phases and would need to treat the times at which the switch occurs

Mass and Piston position, inches

Recent Advances on the Algorithmic Optimization of Robot Motion

39

20 15 10 Flight Phase

Stance Phase

Flight Phase

Stance Phase

5 0

−10

y (t)

y(t)

−5 0

0.1

0.2

0.3

0.4

0

0.1

0.2

0.3

0.4

0.5

p

0.6

0.7

0.8

0.9

1

0.5 0.6 Time, seconds

0.7

0.8

0.9

1

Minimum fuel control, u

40

20

0

−20

−40

Fig. 6. Maximum height hopping motion and minimum fuel control

as variables in the optimization. We note that our algorithm converged much faster when the weighting on the control r is increased; also the number of o , iterations required for convergence in this problem increases for larger yN o o ranging from 3 for yN = 1, to 166 for yN = 50. In addition, the algorithm failed to converge for α < 1 × 10−5 , which demonstrates the need for the dynamics to be continuously diﬀerentiable.

5 Conclusion We discussed the formulation and solution of several important optimal control problems for robotic systems. The most challenging case by far is an underactuated system with contact constraints. We developed an algorithm for solving such nonlinear optimal control problems for systems with quadratic performance measures and unconstrained controls. Contact constraints were accounted for with penalty functions. Each subproblem in the course of the algorithm is a linear quadratic optimal control problem that can be eﬃciently solved by Riccati diﬀerence equations. We show that each search direction generated in the linear quadratic subproblem is a descent direction, and that

40

J.E. Bobrow et al.

the algorithm is convergent. Computational experience has demonstrated that the algorithm converges quickly to the optimal solution.

References [1] J.T. Betts, “Survey of Numerical Methods for Trajectory Optimization,” Journal of Guidance, Control and Dynamics, vol. 21: (2) 193–207, 1999. [2] R.W. Brockett, “Robotic manipulators and the product of exponentials formula,” Proc. Int. Symp. Math. Theory of Networks and Systems, Beer Sheba, Israel, 1983. [3] A.E. Bryson and Y.C. Ho, Applied Optimal Control, Wiley, New York, 1995. [4] Y.-C. Chen,“Solving robot trajectory Planning problems with uniform cubic B-splines,” Optimal Control Applications and Methods, vol. 12, No. 4, pp. 247– 262, 1991. [5] A. De Luca, L. Lanari and G. Oriolo, “A Sensitivity Approach to Optimal Spline Robot Trajectories,” Automatica, vol. 27, No. 3, pp. 535–539, 1991. [6] Enright P.J, Conway B.A. Optimal ﬁnite-thrust spacecraft trajectories using collocation and nonlinear programming. Journal of Guidance, Control, and Dynamics, 1991; 14:981–985. [7] R. Fletcher, Practical Methods of Optimization, Wiley, 2nd edition, 1987. [8] Hargraves C.R., Paris S.W. Direct trajectory optimization using nonlinear programming and collocation. Journal of Guidance, Control, and Dynamics, 1987; 10:338–342. [9] D.G. Luenberger, Linear and Nonlinear Programming, Addison Wesley, 1989. [10] B. Martin and J.E. Bobrow, “Minimum Eﬀort Motions for Open Chained Manipulators with Task-Dependent End-Eﬀector Constraints,” International Journal of Robotics Research, vol. 18, No. 2, February, 1999, pp. 213–224. [11] R.W. Murray, Z. Li, and S.S. Sastry, A Mathematical Introduction to Robotic Manipulation, CRC Press, Boca Raton, Florida, 1993. [12] J.P. Ostrowski, J.P. Desai, V. Kumar, “Optimal gait selection for nonholonomic locomotion systems,” International Journal of Robotics Research, vol. 19, (no. 3), March 2000. pp. 225–237. [13] F.C. Park, J.E. Bobrow, and S.R. Ploen, “A Lie Group Formulation of Robot Dynamics,” International Journal of Robotics Research, vol. 14, No. 6, December 1995, pp. 609–618. [14] A. Sideris and J. E. Bobrow, “An Eﬃcient Sequential Linear Quadratic Algorithm for Solving Nonlinear Optimal Control Problems,” Proceeding of the 2005 IEEE Conference on Decision and Control, Portland, Oregon. [15] G.A. Sohl, and J.E. Bobrow, “A Recursive Dynamics and Sensitivity Algorithm for Branched Chains,” ASME Journal of Dynamic Systems, Measurement and Control, vol. 123, no. 3, Sept. 2001. [16] G.A. Sohl, and J.E. Bobrow, “Optimal Motions for Underactuated Manipulators,” 1999 ASME Design Technical Conferences, Las Vegas, September. [17] M. Spong, “The swing up control problem for the Acrobot,” IEEE CONTR SYST MAG, vol. 15: (1) 49–55 Feb. 1995. [18] Veeraklaew T., and Agrawal S.K., “Neighboring optimal feedback law for higher-order dynamic systems,” ASME J. Dynamic Systems, Measurement, and Control, 124 (3): 492–497 Sep. 2002.

Recent Advances on the Algorithmic Optimization of Robot Motion

41

[19] von Stryk O. Numerical solution of optimal control problems by direct collocation. Optimal Control, Bulirsch R, Miele A, Stoer J, Well KH (eds), International Series of Numerical Mathematics, vol. 111. Birkshauser Verlag; Basel, 1993; 129–143. [20] Chia-Yu E. Wang, Wojciech K. Timoszyk and James E. Bobrow, “Payload Maximization for Open Chained Manipulators: Finding Weightlifting Motions for a Puma 762 Robot,” IEEE Transactions on Robotics and Automation, April 2001, vol. 17, No. 2 pp. 218–224. [21] C.-Y.E. Wang, J.E. Bobrow, and D.J. Reinkensmeyer, “Swinging From The Hip: Use of Dynamic Motion Optimization in the Design of Robotic Gait Rehabilitation,” Proceedings of the IEEE Conference on Robotics and Automation, Seoul, Korea, May 2001. [22] I. Wickelgren, “Teaching the spinal cord to walk,” Science, vol. 279, pp. 319– 321, 1998.

A Spring Assisted One Degree of Freedom Climbing Model J.E. Clark1 and D.E. Koditschek2 1

2

University of Pennsylvania, Philadelphia, PA, USA [email protected] University of Pennsylvania, Philadelphia, PA, USA [email protected]

Summary. A dynamic model of running–the spring-loaded inverted pendulum (SLIP)–has proven eﬀective in describing the force patterns found in a wide variety of animals and in designing and constructing a number of terrestrial running robots. Climbing or vertical locomotion has, on the other hand, lacked such a simple and powerful model. Climbing robots to date have all been quasi-static in their operation. This paper introduces a one degree of freedom model of a climbing robot used to investigate the power constraints involved with climbing in a dynamic manner. Particular attention is paid to understanding how springs and body dynamics can be exploited to help relieve a limited power/weight ratio and achieve dynamic running and climbing.

1 Introduction We seek a fast and agile robot that can traverse both vertical and horizontal real world terrain. Dynamic locomotion over unstructured and natural terrain has proven to be a diﬃcult task. A large number of walking robots have been built, but only recently have running robots been developed that can move at speeds of bodylengths/second over rough terrain [6, 21, 15]. The Rhex [6] and Sprawl [15] families of dynamic machines are based on a Spring-loaded Inverted Pendulum (SLIP) model of running developed from biomechanical research [13, 20]. They have simple morphologies with only one actuator per leg, are polypedal, run mostly open-loop, and rely on springs in the legs to passively self-stabilize. On the other hand, there have only been a few legged robots that can climb vertical surfaces, and they have generally been limited to quasi-static climbing regimes. Their reliance on vacuum or magnetics to achieve the adhesive forces necessary for vertical climbing has limited them to select man-made surfaces such as glass and metal [19, 10, 23]. Recently foot-hold based [11, 12] and vectored thrust based climbers [1] have been developed, but they only

44

J.E. Clark and D.E. Koditschek

slightly extend the range of traversable surfaces and do not address the role of dynamics in climbing. The motivation for this work is the ongoing development of the RiSE robot–a bio-inspired hexapedal platform that is intended to both run and climb vertical surfaces [8]. Currently under development are micro-spine [7, 16] and dry-adhesive feet [22] to allow attachment to a wide range of natural vertical environments. The current instantiation of the robot, however, is otherwise like the remainder of climbing robots in that it moves slowly and its control is based on quasi-static assumptions. The purpose of this work is to explore how to achieve dynamic climbing and how that can be used to improve the performance of the RiSE climbing robot. To this end we discuss some of the fundamental diﬀerences between dynamic running and climbing and introduce a simple one-dimensional dynamic climbing model to investigate approaches to mitigate some of the diﬃculties in achieving dynamic climbing. 1.1 Dynamic Climbing We reserve the term dynamic for robots that manage their kinetic as well as their potential energy. For example, dynamic level ground running can be distinguished from quasi-static locomotion by the phasing of kinetic and gravitational potential energy during a stride. Generally, dynamic runners are distinguished in physical structure by their essential use of springs in the legs. These leg springs act as reservoirs that can store and return energy at whatever required rate. In a typical dynamic gait, the spring energy is collected during the initial phase of a stride (“compression”) and returned during the second phase (“decompression”) as work done against gravity needed to raise again the center of mass back close to its height lost in the initial phase. Dynamic runners can (but need not) adopt an aerial phase gait to buy time for leg recirculation, thereby aﬀording speeds that surpass the inevitable frequency limits of their leg swing actuators. In such situations, springs can recover and return the kinetic energy otherwise lost in body deceleration. Properly arranging these exchanges of kinetic and spring and gravitational energy requires control schemes designed to do more than simply track the joint reference trajectories typically used by walkers. The resulting dynamic stability confers a degree of agility and maneuverability impossible to achieve in quasi-static walking gaits. The question arises whether spring assistance can be introduced in climbing that yields analogous beneﬁts. The major diﬀerence between climbing and running is in the alignment of the gravity vector with respect to the direction of travel. We suggest that this has three primary impacts on legged climbers. The ﬁrst is that travel aligned with the gravity vector implies that any forward progression increases the gravitational potential of the robot, resulting in a net drain on the rest of the system’s energy. As a consequence the SLIP model, which relies on

Dynamic Climbing

45

the interchange of kinetic and gravitational potential energy from hopping or bouncing to regulate the energy during a stride, no longer applies. In addition to necessarily changing the way in which kinetic and spring potential energy are exchanged with gravitational potential energy, a vertical heading also implies that ground impacts are not induced by gravity. Especially for the front feet ground contact must be actively generated. This changes, and to some degree reduces, the role of springs in mitigating ground contact forces. Successful running robots have required springs to regulate the impact at ground contact, and this is not necessarily the case for climbers. A third major diﬀerence for climbing robots is the necessity of bi-lateral or “adhesive” foot constraints. The development of feet that create an inpulling force to the wall is one of the major design requirements in a climbing robot. Having to create feet that grasp the wall to deal with the inherent pitch instability serendipitously reduces the chance of tipping in the roll direction– which is a major source of instability in level ground runners. Once attached, tipping becomes less of a problem, but motions such as repositioning the feet on the ground via sliding, as is often done in turning, become more diﬃcult. In some sense these diﬀerences make climbing easier than running since severe foot impacts and lateral tipping are less likely to occur. On the other hand getting good foot attachment and regulating the system’s energy become much more diﬃcult. The problem of attachment has and continues to receive a fair amount of attention. The second problem, more eﬀectively using the system’s power resources, motivates our investigation with a simple dynamic climbing model. A dynamic robot may lend scansorial machines advantages relative to today’s quasi-static climbers analogous to the superiority of level ground runners over their quasi-static walking counterparts: simpliﬁed control; improved eﬃciency; access to and mobility through otherwise impassible terrain obstacles; and, of course, faster speeds. 1.2 Power and Speed Constraints We propose a simple one-dimensional climbing model to investigate the power requirements and constraints associated with dynamic behavior. As a target for dynamic motion we set a stride frequency of 3.5 Hz for our 3 kg robot. Speciﬁcally, this ﬁgure is associated with the natural frequency of the linear mass spring model associated with purely vertical SLIP hopping. At lower frequencies, back-of-the-envelope calculations developed in Appendix A1 suggest that spring-extension requirements for SLIP-like running (i.e., resonant bouncing in the sagittal plane over level ground) would incur impracticably long leg compression. Another method that has been used to characterize the onset of running is the Froude number, (v 2 /gl), where v is the average fore-aft running speed, l is the leg length and g is the constant of gravitational acceleration. The Froude number is a dimensionless constant that has been used in biomechanics to

46

J.E. Clark and D.E. Koditschek

predict the dynamic similarity in legged animals that is manifest over a wide range of sizes. It has been shown that many animals prefer to switch from a walk to a run at speeds where their Froude number is about 0.5 [4, 18]. RiSE climbing at the target frequency of 3.5 Hz would travel at 0.55 m/s which corresponds to a Froude number of 0.2. While it is not clear that the Froude number is as applicable to climbing as it is to terrestrial locomotion, it does give some indication of when velocity begins to signiﬁcantly aﬀect the energetics of motion. The relatively low value of our target frequency’s Froude number with respect to observed animal gait transitions suggests that the target frequency we have chosen for dynamic climbing is probably not too high. With the current trajectory-tracking, quasi-static control scheme the robot can climb with a stride frequency of 0.5 Hz. Is it theoretically possible to achieve the required 7x increase in speed without changing the motors or decreasing the robot’s mass? The current robot, weighing 3 kg and equipped with two 4.5 W rated servo motors for each of its six legs, has an input electrical power-to-mass ratio of 18:1. In order to climb vertically at our dynamic threshold (0.53 m/s with a stride length of 0.15 m) requires a mechanical output power of 16 W just to deliver the energy expended to increase the system’s gravitational potential. Experiments on Geckos running up vertical walls has shown that the mechanical power that they expend to run at speeds up to 10 bodylengths/second is about 5 W per kilogram of animal [9]–about the same ratio as for RiSE were it to run at 3.5 Hz. What is remarkable is that for the gecko the mechanical power expended when climbing is only about 10% greater than the amount of energy lost to gravitational potential. The 16 W power requirement for RiSE running at this speed represents 30% of the maximum continuous electrical input power that the robot motor’s can consume without thermal damage. In reality only a small percentage of the motors’ 54 W rating will be converted into useful mechanical work. The two major reasons for this are (1) the motors are run at maximum power for only a small segment of the stride and (2) motor ineﬃciencies (e.g. thermal losses in the windings and mechanical losses in the bearings) and system “drag” (e.g., transmission losses, generation of internal forces and negative work associated with securing footholds) signiﬁcantly diminish the mechanically useful component of the power the motors consume. In this paper we address the ﬁrst of these two problems. Speciﬁcally we consider how to use dynamic gaits and body/leg springs to better utilize the available motor power. We show that in the ideal case these approaches signiﬁcantly reduce the peak power demanded from the motors permitting a smaller gear reduction, which in turn allows a higher stride frequency. With ideal motors the changing demands for torque and speed during the leg cycle could be met by implementing a torque control law and allowing the motor to operate at diﬀerent points along the speed-torque curve. As shown in Fig. 2 for the current motors, only 20% of the torque range is available for

Dynamic Climbing

47

continuous use. This dramatically reduces the ﬂexibility of any such motor control law. Instead we investigate how to use passive springs and the body’s inertia to maximize the application of the available electrical power. The remainder of this paper is organized as follows. Section 2 describes the one-dimensional model of climbing that we use to evaluate the eﬃcacy of the proposed schemes. The simplifying assumptions and equations of motion are given. Section 3 details the numerical studies undertaken and compares the various cases considered. Section 4 reviews the results and gives some areas of future work.

2 Model Description 2.1 Assumptions The RiSE robot (see Fig. 1a) is a six limbed climber with two controlled degrees of freedom per leg. Each leg can rotate about an axis parallel to the direction of motion, lifting the foot away from the ground (wing DOF). The second actuated degree of freedom controls the rotation of the crank segment of a four bar mechanism connecting the foot to the body. The foot is attached to the follower link of the mechanism and traces a roughly elliptical path (see Fig. 3) in a plane passing through the line of action of the wing DOF. With the assumption that the wing DOF is primarily used to specify the point of touchdown and lift-oﬀ in the cycle, the motion of the legs and body can be modeled as planar. This abstraction neglects, among other things, body pitch away from the wall–which is known to be a signiﬁcant issue. The further assumptions that the robot uses an alternating tripod gait and that lateral motions of the robot are of secondary importance allow the g

F

xF

m

x

Fig. 1. (A) Picture of RiSE climbing and (B) schematic of the simple climbing model

48

J.E. Clark and D.E. Koditschek

construction of a single degree of freedom model of climbing, shown in Fig. 1b. The model consists of a point mass body with two virtual, massless legs. The extension of the foot (XF ) during stance is ﬁxed by the leg kinematics. In this very simple model, we are ignoring friction from the legs, foot slipping, etc. Although this oversimpliﬁcation of the system ignores many real and important eﬀects, it is hopefully suﬃcient to examine some basic power and stability issues and provide a basis for future examinations. 2.2 Stance Dynamics The sum of the forces in the vertical direction is given by: m¨ x = F − mg

(1)

where m is the mass of the body, F is the force generated by the motor, and g is the gravitational constant opposing the motion. The force generated by the leg actuator is based on a very simple motor model. Due to thermal concerns arising from the nearly continuous use of the motors when climbing, we assume that the motors operate within their recommended operating range, shown in Fig. 2. Although the stall torque of the motor is 28.8 mN m, the continuous operational limit (τM ax ) is only 4.98 mN m. The represents about 20% of the speed/torque curve given by: τ=

ω − ωnl −km

Velocity (rpm) Recommended operating range Short term operation

4.5 Watt @ 12V 13,900 (No-load Speed)

4.98 (Maximum Continuous Torque)

28.8 (Stall Torque)

Torque (mNm)

Fig. 2. Model and speciﬁcations for the motors used on the RiSE robot. Data from Maxon Precision Motors Inc. RE-16 motor (16 mm diameter, graphite brushes, 4.5 Watt, part number 118730) [2]

Dynamic Climbing

49

Swing (4R)

Swing R Stance (2R)

Stance

Fig. 3. Nominal RiSE foot trajectory and simpliﬁcation used in the model. The RiSE leg kinematics result in a swing phase that is almost twice as long as the stance phase. (Leg trajectory from RISE [8])

where ω is the angular velocity of the motor, ωnl is the no-load velocity limit, and km is the slope of the speed/torque curve. Thus the maximum continuously available torque is given by: ω − ωnl , τM ax τ = min −km In order to adapt to a one DOF linear model the trajectory of the four-bar traveler is approximated with a circle of radius R, as shown in Fig. 3. Linear Coordinates With the following conversions: FM ax =

τM ax R

x˙ nl = R ωnl ke =

x˙ nl R2 ωnl = = R2 km FM ax τM ax

the motor torque law becomes: F = min

x˙ − x˙ nl , FM ax −ke

where (km ) is the slope of the force/velocity curve. With the addition of a gear reduction (G), the force from the motor (F ) becomes:

50

J.E. Clark and D.E. Koditschek

F = min

x˙ − x˙Gnl , GFM ax −G2 ke

= min

G2 x˙ − Gx˙ nl , GFM ax −ke

Combining (1) and (2) yields: 2 1 G x˙ − Gx˙ nl x ¨= , GFM ax − mg min m −ke

(2)

(3)

Gait transitions occur when (XF −x) = 0, i.e. when the leg has reached the end of the vertical section of travel. Since the four-bar mechanism ﬁxes the gait trajectory, XF is ﬁxed at 2R. Whether the leg is capable of resetting within the duration of the last stance phase is a function of the swing dynamics, described below. 2.3 Massless Swing Dynamics Of course in the physical system the mass of the legs is non-zero, and the leg’s trajectory during the swing phase is a function of its dynamics. Initially, however, these dynamics are ignored and the swing phases is considered as a binary state: either the leg can return to the touchdown position in time, or it cannot. The time that it will take the leg to retract, tSwing , is bounded by no-load, x˙ nl , and max continuous velocity, x˙ τM ax , of the foot, as given below: G 2R G 2R ≥ tSwing ≥ x˙ τM ax x˙ nl

(4)

The left side of 4 represents an upper bound on the duration of the swing phase.

3 Numerical Simulation 3.1 Trajectory vs. Force Based Gaits In the current control philosophy a gait is generated by specifying a desired trajectory for the path of the feet. Typically four phases are speciﬁed: swing, attachment, stance, and detachment. Using the motor encoder readings and PD control the legs attempt to track this trajectory throughout the stride. Forces are generated when errors in the tracking occur. These generally correspond to contact with the ground during attachment, lifting the robot in the face of gravity during stance, and the inertial resistance to the rapid acceleration during swing. Figure 4 shows an idealized trajectory and the corresponding torques generated by the motors. The ﬁgure on the left is a projection of foot trajectory in the wing plane. A trace of the wing angle with respect to time is shown on the bottom right. In the plot on the right the solid horizontal

Dynamic Climbing

51

line represents the maximum continuous torque as speciﬁed by the motor’s manufacturer. The dotted line is the motor’s mean torque over a stride. The motor torque curve itself is an abstraction of reality where the large spike in the torque graph corresponds to the body acceleration of the foot during stance, and the smaller spike to the acceleration of the foot during swing. It should be noted that current gaits are designed for eﬀective attachment and detachment rather than optimizing speed or utilization of available motor power. Signiﬁcant improvements in terms of speed can (and are being) made by reﬁning the shape of the target trajectory–especially during stance and swing–such that the torque demands more closely match the abstraction shown in Fig. 4. Due to the large gear reduction employed, the ability to shape the torque trajectory is limited. If the peak load on the motors was decreased or distributed more eﬃciently throughout the stride then the gearing could be reduced and the top speed dramatically increased. 12 11

Maximum Continuous Torque

A 10

B Body Acceleration

9

Sample Motor Torque

Torque (mNm)

8

7

Foot Acceleration

Mean Torque

6

5

4

3

B

1

C D

D

A

2

C Leg ‘Wing’ Angle

0

Swing

Stance

Time (s)

Fig. 4. Sample leg trajectory projected onto the “wing” plane (left) and in terms of the wing angle (right, bottom). These are used to contextualize the idealized force pattern (right, top). The leg trajectory and mean torque plots are based on RiSE robot data [3]

52

J.E. Clark and D.E. Koditschek

Simulation Results Figure 5 shows how the theoretical minimum stride frequency for the simple model described in Sect. 2.2 varies as a function of gear ratio, G. Since each leg has a kinematically ﬁxed stride length, the stride period is inversely proportional to velocity. The dashed curved line represents the stance phase speed limit for each gear ratio, G. The sloping starred line represents the swing phase reset threshold. Point (A) shows the theoretical maximum speed with the RiSE v1.0 gear ratio. At this gear ratio any higher speeds would require the leg to complete the swing phase faster than the motor can handle. Below a certain gear ratio (point (C) in Fig. 5) the robot no longer has enough force to overcome gravity and cannot climb. Increasing the gear ratio reduces the overall available speed and above a threshold, point (B), the duration of the swing phase becomes the limiting factor.

Fig. 5. Simulation results for the base model, showing the minimum possible stride period (maximum speed) for various gear ratios (G). The shaded areas under the curves represent regions that are inadmissible as they invalidate either swing or stance phase constraints

Dynamic Climbing

53

Point (B) is therefore the theoretical upper bounds on the velocity of the robot using the current trajectory-based gait formulation. This stride frequency (and speed) will, of course, never actually be reached due to the non-instantaneous, and non-trivial, attachment and detachment requirements. Nevertheless it is clear that without some change to the energetics and actuation scheme of the robot it will never be able to reach our target dynamic threshold. Momentum If, however, non-zero attachment speeds and the momentum of the body of the robot is explicitly considered in the control scheme the robot could accelerate from one stride to another and higher “steady-state” velocities are possible. This requires either allowing the foot trajectories to change as a function of body velocity or the adoption of some sort of force-based control scheme as is done in our simple model. In either case the new maximum speed will be limited by the swing phase reset time. In this case the fastest conﬁguration corresponds to the lowest gear ratio that will actually lift the robot, as shown by point (C) in Fig. 5. A potential diﬃculty with this approach is ensuring that the foot attachment trajectory remains viable as the body velocity increases. This problem is being considered in ongoing work on foot attachment dynamics. 3.2 Spring-Assisted Climbing An alternative method for increasing speeds with a limited power budget is by the intelligent use of springs to redistribute the cyclic loading and level out the demands on the motors. By lowering the peak force requirements the drive-train gear ratio can be reduced to speed up the overall motion of the robot. This also brings the mean loading on the motors closer to the maximum continuous operation level. Since these motors get the most power at 1/2 of stall, and they are limited to 20% stall by the thermal constraints, maximum achievable power coincides with the maximum continuous operation point. In this section we consider two approaches to using springs to assist the motion of the body in climbing. The ﬁrst approach stores energy from the end of the stance phase when the body is decelerating in preparation for attachment and then releases it at the beginning of the next stance phase to help re-accelerate the body. The second approach uses the motor to store energy in a spring during the swing phase, which is then released to assist lifting the body during stance. 3.3 Stance-Stance Transfer The inspiration and physical motivation for this approach came from observing fast climbing animals such as the gecko which swing their tails and ﬂex

54

J.E. Clark and D.E. Koditschek

their backs as they run. One advantage of these motions may be that they shift the phasing of the power distribution to the beginning of a stride when it is needed most to accelerate a body after slowing for foot contact. A proper model for this behavior would include multiple bodies and degrees of freedom. Here we hypothesize that at least part of the eﬀect of these motions can be captured by the linear body spring as shown in Fig. 6. The spring is loaded during the end of the stance phase as the robot slows for attachment and then is released at the beginning of the next stride to assist with the re-acceleration of the body. The net eﬀect of this body/tail spring is to lower the peak torque spike during stance. This in turn allows us to further change the gear ratio, reducing the maximum continuous torque limit and increasing the stride frequency. With the addition of this spring the equation of motion for the body during stance becomes: 2 G x˙ − Gx˙ nl 1 , GFM ax + k(XF − x) − mg (5) min x ¨= m −ke where XF , the rest length of the spring, is located at the midpoint of stance.

Fig. 6. Schematic of model with body spring and the eﬀect of the spring on the nominal torque proﬁle. The “+” region represents when the spring is assisting the motor, and the “−” region represents when the motor is stretching the spring. (As before, the leg trajectory and mean torque plots are based on RiSE robot data [3])

Dynamic Climbing

55

The maximum stiﬀness of the virtual leg is limited by the force available from the motors (FM ax ), as given by (6) where (p) is the number of motors. k≤

G FM ax p 2R

(6)

Magnitudes ranging from 0-80% of the maximum force were evaluated. The simulation results with this spring are shown in Fig. 7. As in Fig. 5 the areas under the curved lines represent speeds for which the stance phase displacement requirement is not satisﬁed, and the starred diagonal line represents the swing phase requirement. Points (A) and (B) are the same as in Fig. 5. Point (C) represents the maximum speed with the body spring which yields a 16% improvement over trajectory reﬁnement alone, case (B). It appears that the use of such a body spring increases the maximum possible speed for a given G, but does not lower the gear ratio which is necessary to overcome gravity and lift the robot. Thus the use of a body spring to some

0.9 Body Spring k = 80% Body Spring k = 40% No Spring Swing Reset Threshold

0.8

0.7

Stride Period

Faster

A 0.6

B

0.5

0.4

0.3

0.2

C 0.1

0

20

30

40

50

60

70

80

90

100

110

120

Gear Ratio (G) Fig. 7. Simulation results for the model with the body spring. Minimum possible stride periods for various gear ratios (G) and spring constants (k)

56

J.E. Clark and D.E. Koditschek

degree duplicates the beneﬁt from implementing a stride-to-stride velocity adaptation scheme. 3.4 Swing-Stance Transfer A second approach to using springs to more eﬀectively and evenly apply the torque from the motors is to use a spring connecting the foot to the body. As shown on the left in Fig. 8, this is modeled as a spring acting in parallel to the actuator in each virtual leg. This spring is loaded during swing phase as the leg resets to a position relative to the body ready for touch down. The spring is then released at the beginning of stance to assist with the acceleration of the body. As shown on the right in Fig. 8, this adds a load to the motors during swing (when their torque output capabilities are currently underutilized) and mitigates the force requirements at the beginning of stance. The addition of this spring results in the same body equation of motion as in Sect. 3.3, but the spring is now fully loaded at the beginning of stance, and is fully unloaded at the end. The spring constant, k, is chosen in the same manner as in the previous section. Figure 9 shows the eﬀect of changing the gear ratio G on the stride period for a range of spring constants from 0–80% of the maximum spring constant for each G as given by (6). For each value of G and k, the resulting minimum

Fig. 8. Schematic of model with leg spring and the eﬀect of the spring on the nominal torque proﬁle. The “+” region represents when the spring is assisting the motor, and the “−” region represents when the motor is stretching the spring. (As before, the leg trajectory and mean torque plots are based on RiSE robot data [3])

Dynamic Climbing

57

0.9 Linear Spring Non-linear Spring No Spring Swing Reset Threshold

0.8

0.7

Stride Period

Faster

A 0.6

B C

0.5

0.4

D

0.3

0.2

0.1

0

20

30

40

50

60

70

80

90

100

110

120

Gear Ratio (G) Fig. 9. Simulation results for the model with the leg spring. Minimum possible stride periods for various gear ratios (G) and spring constants (k)

stride period is shown. As before, the line corresponding to the stride period limit for retraction of the leg during swing is indicated with a starred line. Points (A) and (B) are the same as in Fig. 5 and represent the maximum possible speed without springs. Point (C) indicates the maximum speed conﬁguration for a model with a linear spring with k = 80% of maximum. The use of a linear spring increases the maximum speed by 36% to 0.62 m/s. Softening Spring While advantageous for their simplicity, linear springs are not optimal in terms of energy storage for a limited stretching force. If the linear spring were replaced by a softening spring the spring potential energy at maximum deﬂection would increase. In the limit, with a constant force spring, the energy storage would double. A constant force spring, however, would add a large load to the beginning of the swing phase when the motors may already be saturated attempting to accelerate the leg. The leg spring analysis in the previous section

58

J.E. Clark and D.E. Koditschek

was repeated with the linear spring replaced by one with a spring equation 1 of: F = kx 2 . This signiﬁcantly increases the energy storage for the spring with minimal interference with the acceleration of the leg in swing. Point (D) in Fig. 9 represents the fastest conﬁguration that can be achieved with this softening spring, resulting in a speed increase to 0.84 m/s. Retraction Dynamics In this swing-to-stance spring approach the load during the swing phase is signiﬁcantly increased, therefore it becomes important to determine what happens when the leg dynamics are explicitly considered. In other words, at what point will the inertia of the legs and the frictional losses in the springs erode any beneﬁt from running dynamically or adding leg springs. To this end the dynamics of the swing legs were modeled with (7). 2 G x˙f − Gx˙f nl 1 x¨f = , GFM ax + k(Xbody − xf ) − bx˙f m2 g (7) min m2 −ke Where m2 is the eﬀective inertia of the robot’s leg and b is the damping term, as given by (8). √ (8) b = 2ζ mk While both adding inertia to the legs and increasing the losses in the spring detract from the gains suggested by the simulation, the model can still climb with a spring constant, k = 80% of maximum, at speeds equivalent to what the simple swing phase model predicts with leg inertias of 30% of the bodymass and a damping coeﬃcient ζ = 0.3. 3.5 Results of Numerical Study Table 1 summarizes the cases considered thus far and gives the maximum frequency, speed, and percent improvement for each case. With a forceoptimization series of trajectory reﬁnements the robot’s speed can (theoretically) be signiﬁcantly improved. Obviously real-world issues associated with mechanical losses and foot attachment/detachment will prevent the actual achievement of the 3.05 Hz theoretical speed predicted from implementing trajectory reﬁnements. The last row in the table (Combination) shows the eﬀect of combining the body and softening foot springs and allowing the body velocity to increase from stride to stride (momentum), which results in a 2.9x improvement over the trajectory reﬁnement case alone. Even if drag and attachment losses only permitted achieving 40% of the theoretical speed limit, with these changes we get near the 3.5 Hz dynamic threshold that we established previously. Of the various elements, the non-linear foot spring contributes the most. Another option to improve the speed of the robot climbing is to alter its power/weight ratio. As a point of comparison the motor specs for larger motors

Dynamic Climbing

59

Table 1. Frequency, velocity, and percent improvement for cases (B) to (G) Case (A) Trajectory Reﬁnement (B) Body Momentum (C) Body Spring (D) Foot Spring-Linear (E) Foot Spring-Softening (F) Body + Foot Springs (G) Combination

Frequency

Velocity (m/s)

% Impr.

3.05 3.66 3.55 4.15 5.59 5.78 8.85

0.46 0.55 0.53 0.62 0.84 0.87 1.33

0 20 16 36 83 89 190

from the same vendor and product line were used in the model to see how much the power needed to be increased to match the eﬀect of adding springs and dynamic gaits. In this case the 4.5 W motors for RiSE were replaced with 20 W versions. The total mass of the robot was left unchanged and the simulations repeated. This 4.4x increase in power resulted in a 2.3x increase in speed. A net increase slightly less than with springs/dynamics. In reality the use of larger motors brings with it a signiﬁcant increase in mass and complexity. The addition of these larger motors would add additional 1.5 kg to the robot’s mass, not including the necessary changes to the body, battery, and electronics design. Fundamentally, increasing the size of the motors does not substantially increase their power to weight ratio. Other motors do have higher power/weight ratios than the ones chosen for our robot, but these suﬀer from other draw backs such as controllability. Of course reducing the weight of the robot, were we able to ﬁnd a way to do it without loosing performance, would help as well. The simulations with the simple model described here suggest that gains in speed comparable to the use of springs or the addition of (magically) more powerful motors can be achieved by reducing the robot’s weight by about 50%. In the absence of further improvements to the power/weight ratio of our robots, the simulations suggest that with our current design it is not possible to reach our target speed. Simply reﬁning the gait and reducing the ineﬃciencies in the system will, by themselves, be insuﬃcient. With an appropriate use of springs and body dynamics, however, our target speed of 3.5 Hz becomes theoretically possible. The actual realization of these speeds, however, depends on how well these concepts can be incorporated with the ongoing work in improving trajectory reﬁnement and foot attachment.

4 Conclusion One of the signiﬁcant problems in achieving fast climbing is the power demand associated with delivering the required work against gravity at higher speeds. Having suﬃcient on-board power for fast locomotion on the level ground has

60

J.E. Clark and D.E. Koditschek

proven challenging, and motion against gravity, obviously, only exacerbates the problem. One approach to increasing the use of a robot’s onboard power is switch from a position-based control scheme to forced-based approach. By explicitly regulating the motors’ output rather than relying on position tracking errors, the actuators can be much more eﬀectively utilized during a stride. This adoption of this control framework also enables the robot to build up speed over a number of strides, further increasing the performance gains. Even if the switching the fundamental control scheme from a positionbased approach proves infeasible (e.g., perhaps because our limited degrees of freedom require intricately planned approaches and departures from stance to guaranteed adequate wall adhesion and limited perturbation during detachment), much of the advantage of a force-based system could be duplicated by very careful trajectory tuning and adaptation. In either case the reﬁnement of the force trajectory can bring substantial performance beneﬁts to the robot. The simple model employed here suggests that as a theoretical upper limit they could allow the robot to climb at 3Hz or at 0.46 m/s, which is near our dynamic threshold. Of course attachment, detachment and other physical constraints will necessarily reduce the actual gains. However, in this respect the remainder of the numerical results from the simpliﬁed model are encouraging. They suggest that the appropriate addition of body and leg springs could double the robot’s speed over this value. The further incorporation of a variable stride period could almost triple the speed over trajectory reﬁnement alone. This is more than the eﬀect of increasing the motor’s power 4.4x! (that is of course without acknowledging and accounting for the weight of the larger motors). While the advantages of these approaches are not entirely “free”, they do represent a signiﬁcant gain. When the various reﬁnements are combined, the model results suggest that locomotion at our dynamic threshold of 3.5 Hz (or 2 bodylengths/second) should be achievable. 4.1 Future Work In order to implement body dynamic dependent gait trajectories some sort of control system to measure body velocity and alter the leg trajectory may be necessary to ensure good attachment of the feet. More detailed foot/substrate interaction tests may provide the empirical data necessary to develop such a controller. Although we have assumed that foot contact once made will only be broken at the desired detachment point, this clearly does not reﬂect the reality. Reducing the demands on the foot attachment mechanism could be one of the major advantages of dynamic climbing. It seems possible that an appropriate use of springs and the robot’s body’s dynamics could signiﬁcantly reduce the required foot reaction forces. If force threshold limits were added to the feet the eﬀectiveness of the various schemes proposed in this paper could be evaluated in this regard.

Dynamic Climbing

61

Although not addressed here, the stability of dynamic climbers is a topic of interest. Besides the fundamental issues of insuring that the front feet stay attached to the wall, there are a number of other possible ways to consider stability. Many of these arise with the shift from a position-based control to a force-based system. The numeric simulation results suggest that when the velocity is allowed to vary from stride to stride that the simple climber tends to quickly converge to a steady period-1 gait. Have we been fortuitous in our parameter selection, or are these limit cycles almost inevitable? Are they local in nature hence hard to achieve in practice or do they have large basins (e.g., are they globally asymptotic stable)? The model we have used may be simple enough to permit a careful mathematical analysis of the system dynamics. A second interesting question that arises from decoupling multiple limbs from a trajectory-tracking control scheme is the question of synchronization. A related climbing study with an open-loop climbing model [17] indicates that legs tend to synchronize rather than staying 180 degrees out of phase. Is this also true for this model, and if so what sort of controller needs to be established to maintain a regular alternating gait? Looking further ahead, we wonder if with the addition of a lateral degree of freedom to the model we can begin to duplicate the motions and ground reaction forces seen in dynamic climbing animals such as geckos and cockroaches. We believe that enabling a robot with the ability to both dynamicly run and climb is an compelling goal. The achievement of both with an (inherently) constrained power/weight ratio is a diﬃcult task. The creative use of springs and system dynamics to modify the climbing motion of the robot may enable the construction of such robots. While we are not there yet, we at least have some simple models that suggest that it may be possible.

Acknowledgments Thanks to RiSE team for many stimulating conversations, speciﬁcally: Ron Fearing, Martin Buehler for their thoughts on motors and power in climbing, Jonathan Karpick for ideas about how to use dynamics in climbing, Dan Goldman for discussions about how biological systems climb, and Haldun Komsuoglu for insights about modeling simple climbers. We also appreciate Clark Haynes and Al Rizzi’s help with the robot trajectory and power consumption data. The development of the RiSE platform was supported by DARPA under grant number N66001-05-C-8025. Jonathan Clark is supported by the IC Postdoctoral Fellow Program under grant number HM158204-1-2030.

62

J.E. Clark and D.E. Koditschek

Appendix A1: Rise Robot Constants

Table 2. RiSE speciﬁc model parameter values Variable TStall TM ax ωnl R G p m

Value 0.0288 Nm 0.00498 Nm 13, 900 rpm 0.0762 m 126 6 3 kg

Description Stall torque Maximum continuous torque No-load speed Radius of foot trajectory Base Gear ration Number of motors per tripod Mass of the robot

Appendix A2: Minimum Leg Frequency In order to achieve resonant hopping as described by the spring-loaded inverted pendulum (SLIP) model, the motor activation frequency–whether used primarily for recirculating vertical leg springs as in RHex [6] or for powering vertical leg strokes in phase with passive recirculating springs as in Sprawl [14]–should match the natural frequency of the body’s oscillation. For a SLIPtype hopper the natural frequency, ωn , during stance is a function of the body mass, M , and stiﬀness of the legs, k, that varies in a rather subtle manner with the particular steady state gait for even the simplest two degree of freedom models [5]. Empirically, we ﬁnd this function is eﬀectively approximated by that characterizing a one degree of freedom spring-mass system: ( k (9) ωn = M The lower limit on the spring stiﬀness is constrained by its maximum displacement, ∆x, which in turn is ﬁxed by the leg kinematics. Although the force-extension proﬁle of a spring can vary signiﬁcantly depending upon whether it is “hardening” or “softening”, it will suﬃce for our present order-ofmagnitude analysis to consider the simplest relationship of constant stiﬀness arising from a Hooke’s law spring. For this model, a lower bound on the excursion of the leg spring corresponds to when the force on the spring is equal to gravity, giving: Mg (10) k= ∆x Combining equations (9) and (10) gives:

Dynamic Climbing

( ωn =

k = M

(

Mg = ∆x M

(

63

g ∆x

For RiSE with a kinematically achievable ∆x = 0.02 m: ωn = 22 rad/s = 3.5 cycles/s If large airborne phases are allowed the body oscillation frequency would become slower than the body spring-mass frequency, ωn . Any gains from this, however, would be set oﬀ by the increased required deﬂection of the spring, ∆x, during stance.

References [1] [2] [3] [4]

[5]

[6]

[7]

[8]

[9] [10]

[11]

[12]

http://www.vortexhc.com/vmrp.html. http://www.maxonmotorusa.com. Personal Communication from Al Rizzi and Clark Haynes. R. McN. Alexander. Mechanics and scaling of terrestrial locomotion. In T. J. Pedley, editor, Scale Eﬀects in Animal Locomotion, pp. 93–110. Academic Press, 1977. R. Altendorfer, D. E. Koditschek, and P. Holmes. Stability analysis of legged locomotion models by symmetry-factored return mpas. International Jouranl of Robotics Research, 23(11):979–1000, 2004. R. Altendorfer, N. Moore, H. Komsuoglu, M. Buehler, Jr. Brown H. B, D. McMordie, U. Saranli, R. Full, and D. E. Koditschek. Rhex: A biologically inspired hexapod runner. Autonomous Robots, 11(3):207–213, 2001. A. T. Asbeck, S. Kim, W. R. Provancher, M. R. Cutkosky, and M. Lanzetta. Scaling hard vertical surfaces with compliant microspine arrays. In Proceedings, Robotics Science and Systems Conf., MIT, Cambridge, MA, June 8–10, 2005. K. Autumn, M. Buehler, M. Cutkosky, R. Fearing, R. J. Full, D. Goldman, R. Groﬀ, W. Provancher, A. A. Rizzi, U. Saranli, A. Saunders, and D. E. Koditschek. Robotics in scansorial environments. In Grant R. Gerhart, Charles M. Shoemaker, and Douglars W. Gage, editors, Unmanned Ground vehicle Technology VII, vol. 5804, pp. 291–302. SPIE, 2005. K. Autumn, S. T. Hsieh, D. M. Dudek, J. Chen, Chitaphan. C., and R. J. Full. Dynamics of geckos running vertically. 2005. in press. C. Balaguer, A. Gimenez, J. M. Pastor, V. M. Padron, and C. Abderrahim. A climbing autonomous robot for inspection applications in 3d complex environments. Robotica, 18:287–297, 2000. D. Bevly, S. Dubowsky, and C. Mavroidis. A simpliﬁed cartesian-computed torque controller for highly geared systems and its application to an experimental climbing robot. Transactions of the ASME. Journal of Dynamic Systems, Measurement and Control, 122(1):27–32, 2000. T. Bretl, S. Rock, and J. C. Latombe. Motion planning for a three-limbed climbing robot in vertical natural terrain. In Proceedings of the IEEE International Conference on Robotics and Automation, vol. 205, pp. 2946–2953. Piscataway, NJ, USA: IEEE, 2003.

64

J.E. Clark and D.E. Koditschek

[13] G. A. Cavagna, N. C. Heglund, and Taylor C. R. Mechanical work in terrestrial locomotion: Two basic mechanisms for minimizing energy expenditure. American Journal of Physiology, 233, 1977. [14] J. G. Cham, S. A. Bailey, J. E. Clark, R. J. Full, and M. R. Cutkosky. Fast and robust: Hexapedal robots via shape deposition manufacturing. International Journal of Robotics Research, 21(10), 2002. [15] J. G. Cham, J. Karpick, J. E. Clark, and M. R. Cutkosky. Stride period adaptation for a biomimetic running hexapod. In International Symposium of Robotics Research, Lorne Victoria, Australia, 2001. [16] S. Kim, A Asbeck, M. R. Cutkosky, and W. R. Provancher. Spinybotii: Climbing hard walls with compliant microspines. In Proceedings, IEEE – ICAR, Seattle, WA, July 17–20, 2005. [17] Haldun Komsuoglu. Toward a formal framework for open-loop stabilization of rhythmic tasks. PhD thesis, University of Michigan, 2004. [18] R. Kram, A. Domingo, and D. P. Ferris. Eﬀect of reduced gravity on the preferred walk-run transition speed. The Journal of Experimental Biology, 200:821–826, 1991. [19] G. La Rosa, M. Messina, G. Muscato, and R. Sinatra. A low-cost lightweight climbing robot for the inspection of vertical surfaces. Mechatronics, 12(1):71– 96, 2002. [20] T. A MCMahon and G. C. Cheng. Mechanics of running. how does stiﬀness couple with speed? Journal of Biomechanics, 23(1):65–78, 1990. [21] R. D. Quinn, G. M. Nelson, R. J. Bachmann, D. A. Kingsley, J. Oﬃ, and R. E. Ritzmann. Insect designs for improved robot mobility. In Dillmann Berns, editor, Proceedings of CLAWAR 2001, Karlsruhe, Germany, pp. 69–76, 2001. [22] M. Sitti and R. S. Fearing. Synthetic gecko foot-hair micro/nano-structures for future wall-climbing robots. In Proceedings of the IEEE International Conference on Robotics and Automation, vol. 1, pp. 1164–1170. Piscataway, NJ, USA: IEEE, 2003. [23] Z. L. Xu and P. S. Ma. A wall-climbing robot for labelling scale of oil tank’s volume. Robotica, 20:209–212, 2002.

Fast Direct Multiple Shooting Algorithms for Optimal Robot Control M. Diehl1 , H.G. Bock1 , H. Diedam1 , and P.-B. Wieber2 1

2

Interdisciplinary Center for Scientiﬁc Computing, University of Heidelberg, Im Neuenheimer Feld 368, D-69120 Heidelberg, Germany [email protected] INRIA Rhˆ one-Alpes, Projet BIPOP, 38334 St Ismier Cedex, France

Summary. In this overview paper, we ﬁrst survey numerical approaches to solve nonlinear optimal control problems, and second, we present our most recent algorithmic developments for real-time optimization in nonlinear model predictive control. In the survey part, we discuss three direct optimal control approaches in detail: (i) single shooting, (ii) collocation, and (iii) multiple shooting, and we specify why we believe the direct multiple shooting method to be the method of choice for nonlinear optimal control problems in robotics. We couple it with an eﬃcient robot model generator and show the performance of the algorithm at the example of a ﬁve link robot arm. In the real-time optimization part, we outline the idea of nonlinear model predictive control and the real-time challenge it poses to numerical optimization. As one solution approach, we discuss the real-time iteration scheme.

1 Introduction In this paper, we treat the numerical solution of optimal control problems. We consider the following simpliﬁed optimal control problem in ordinary differential equations (ODE). T minimize x( · ), u( · ), T

L(x(t), u(t)) dt + E (x(T ))

(1)

0

subject to x(0) − x0 = 0, x(t)−f ˙ (x(t), u(t))= 0, h(x(t), u(t)) ≥ 0, r (x(T )) = 0

t ∈ [0, T ], t ∈ [0, T ],

(ﬁxed initial value) (ODE model) (path constraints) (terminal constraints).

The problem is visualized in Fig. 1. We may or may not leave the horizon length T free for optimization. As an example we may think of a robot that

66

M. Diehl et al.

Fig. 1. Simpliﬁed Optimal Control Problem

shall move in minimal time from its current state to some desired terminal position, and must respect limits on torques and joint angles. We point out that the above formulation is by far not the most general, but that we try to avoid unneccessary notational overhead by omitting e.g. diﬀerential algebraic equations (DAE), multi-phase motions, or coupled multipoint constraints, which are, however, all treatable by the direct optimal control methods to be presented in this paper. 1.1 Approaches to Optimal Control Generally speaking, there are three basic approaches to address optimal control problems, (a) dynamic programming, (b) indirect, and (c) direct approaches, cf. the top row of Fig. 2. (a) Dynamic Programming [5, 6] uses the principle of optimality of subarcs to compute recursively a feedback control for all times t and all x0 . In the continuous time case, as here, this leads to the Hamilton-Jacobi-Bellman (HJB) equation, a partial diﬀerential equation (PDE) in state space. Methods to numerically compute solution approximations exist, e.g. [34] but the approach severely suﬀers from Bellman’s “curse of dimensionality” and is restricted to small state dimensions. (b) Indirect Methods use the necessary conditions of optimality of the inﬁnite problem to derive a boundary value problem (BVP) in ordinary diﬀerential equations (ODE), as e.g. described in [13]. This BVP must numerically be solved, and the approach is often sketched as “ﬁrst optimize, then discretize”. The class of indirect methods encompasses also the well known calculus of variations and the Euler-Lagrange diﬀerential equations, and the Pontryagin Maximum Principle [40]. The numerical solution of the BVP is mostly performed by shooting techniques or by collocation. The two major drawbacks are that the underlying diﬀerential equations are often diﬃcult to solve due to strong nonlinearity and instability, and that changes in the control structure, i.e. the sequence of arcs where diﬀerent constraints are active, are diﬃcult to handle: they usually require a completely new problem setup. Moreover, on so called singular arcs, higher index DAE arise which necessitate specialized solution techniques.

Fast Optimal Robot Control

67

(c) Direct methods transform the original inﬁnite optimal control problem into a ﬁnite dimensional nonlinear programming problem (NLP). This NLP is then solved by variants of state-of-the-art numerical optimization methods, and the approach is therefore often sketched as “ﬁrst discretize, then optimize”. One of the most important advantages of direct compared to indirect methods is that they can easily treat inequality constraints, like the inequality path constraints in the formulation above. This is because structural changes in the active constraints during the optimization procedure are treated by well developed NLP methods that can deal with inequality constraints and active set changes. All direct methods are based on a ﬁnite dimensional parameterization of the control trajectory, but diﬀer in the way the state trajectory is handled, cf. the bottom row of Fig. 2. For solution of constrained optimal control problems in real world applications, direct methods are nowadays by far the most widespread and successfully used techniques, and we will focus on them in the ﬁrst part of this paper. 1.2 Nonlinear Model Predictive Control The optimization based feedback control technique “Nonlinear Model Predictive Control (NMPC)” has attracted much attention in recent years [1, 36],

Fig. 2. Overview of numerical methods for optimal control

68

M. Diehl et al.

in particular in the proceess industries. Its idea is, simply speaking, to use an open-loop optimal control formulation to generate a feedback control for a closed-loop system. The current system state is continuously observed, and NMPC solves repeatedly an optimal control problem of the form (1), each time with the most current state observation as initial value x0 . Assuming that the optimal control trajectory can be computed in negligible time, we can apply the ﬁrst bit of our optimal plan to the real world system, for some short duration δ. Then, the state is observed again, a new optimization problem is solved, the control again applied to the real system, and so on. In this way, feedback is generated that can reject unforeseen disturbances and errors due to model-plant-mismatch. Among the advantages of NMPC when compared to other feedback control techniques are the ﬂexibility provided in formulating the control objective, the capability to directly handle equality and inequality constraints, and the possibility to treat unforeseen disturbances fast. Most important, NMPC allows to make use of reliable nonlinear process models x˙ = f (x, u) so that the control performance can proﬁt from this important knowledge, which is particularly important for transient, or periodic processes. It is this last point that makes it particularly appealing for use in robotics. One essential problem, however, is the high on-line computational load that is often associated with NMPC, since at each sampling instant a nonlinear optimal control problem of the form (1) must be solved. The algorithm must predict and optimize again and again, in a high frequency, while the real process advances in time. Therefore, the question of fast real-time optimization has been intensively investigated [4, 28, 51, 44, 9]. We refer to Binder et al. [10] for an overview of existing methods. One reason why most applications of NMPC have so far been in the process industries [42] is that there, time scales are typically in the range of minutes so that the real-time requirements are less severe than in mechanics. However, we believe that it is only a matter of time until NMPC becomes an important feedback technique in robotics, too. The second scope of this paper is therefore to present some of our latest ideas regarding the fast real-time optimization for NMPC, which are based on direct optimal control methods. 1.3 Paper Outline The paper is organized as follows. In the next section we will describe three popular direct optimal control methods, single shooting, collocation, and multiple shooting. We will argue why we believe the last method to be the method of choice for nonlinear optimal control problems in robotics, and in Sect. 3 we will present its coupling to an eﬃcient robot model generator and show its application to the time optimal point to point maneuver of a ﬁve link robot arm. In Sect. 4 we will discuss nonlinear model predictive control (NMPC) and show how the challenge of fast online optimization can be addressed by

Fast Optimal Robot Control

69

the so called real-time iteration scheme, in order to make NMPC of fast robot motions possible. Finally, in Sect. 5, we conclude the paper with a short summary and an outlook.

2 Direct Optimal Control Methods Direct methods reformulate the inﬁnite optimal control problem (1) into a ﬁnite dimensional nonlinear programming problem (NLP) of the form min a(w) w

subject to

b(w) = 0,

c(w) ≥ 0 ,

(2)

with a ﬁnite dimensional vector w representing the optimization degrees of freedom, and with diﬀerentiable functions a (scalar), b, and c (both vector valued). As said above, all direct methods start by a parameterization of the control trajectory, but they diﬀer in the way how the state trajectory is handled. Generally, they can be divided into sequential and simultaneous approaches. In sequential approaches, the state trajectory x(t) is regarded as an implicit function of the controls u(t) (and of the initial value x0 ), e.g. by a forward simulation with the help of an ODE solver in direct single shooting [31, 45]. Thus, simulation and optimization iterations proceed sequentially, one after the other, and the NLP has only the discretized control as optimization degrees of freedom. In contrast to this, simultaneous approaches keep a parameterization of the state trajectory as optimization variables within the NLP, and add suitable equality constraints representing the ODE model. Thus, simulation and optimization proceed simultaneously, and only at the solution of the NLP do the states actually represent a valid ODE solution corresponding to the control trajectory. The two most popular variants of the simultaneous approach are direct collocation [8] and direct multiple shooting [12]. We will present in detail the mentioned three direct approaches. As all direct methods make use of advanced NLP solvers, we also very brieﬂy sketch one of the most widespread NLP solution methods, Sequential Quadratic Programming (SQP), which is also at the core of the real-time iteration scheme to be presented in the second part. A Tutorial Example For illustration of the diﬀerent behaviour of sequential and simultaenous approaches, we will use the following tutorial example with only one state and one control dimension. The ODE x˙ = f (x, u) is slightly unstable and nonlinear. 3 x(t)2 + u(t)2 dt minimize x( · ), u( · ) 0

70

M. Diehl et al.

Fig. 3. Solution of the tutorial example

subject to (initial value) x(0) = x0 , x˙ = (1 + x)x + u, t ∈ [0, 3], (ODE model)    0 1 − x(t) 1 + x(t) 0    t ∈ [0, 3], (bounds)   ≥ , 1 − u(t) 0 0 1 + u(t) x(3) = 0.

(zero terminal constraint).

We remark that due to the bounds |u| ≤ 1, we have uncontrollable growth for any x ≥ 0.618 because then (1 + x)x ≥ 1. We set the inital value to x0 = 0.05. For the control discretization we will choose N = 30 control intervals of equal length. The solution of this problem is shown in Fig. 3. 2.1 Sequential Quadratic Programming (SQP) To solve any NLP of the form (2), we will work within an iterative Sequential Quadratic Programming (SQP), or Newton-type framework. We omit all details here, and refer to excellent numerical optimization textbooks instead, e.g. [39]. We need to introduce, however, the Lagrangian function L(w, λ, µ) = a(w) − λT b(w) − µT c(w) , with so called Lagrange multipliers λ and µ, that plays a preeminent role in optimization. The necessary conditions for a point w∗ to be a local optimum of the NLP (2) are that there exist multipliers λ∗ and µ∗ , such that ∇w L(w∗ , λ∗ , µ∗ ) = 0 ,

(3)

∗

b(w ) = 0 , c(w∗ ) ≥ 0, µ∗ ≥ 0,

c(w∗ )T µ∗ = 0 .

(4) (5)

Fast Optimal Robot Control

71

In order to approximately ﬁnd such a triple (w∗ , λ∗ , µ∗ ) we proceed iteratively. Starting with an initial guess (w0 , λ0 , µ0 ), a standard full step SQP iteration for the NLP is wk+1 = wk + ∆wk , λk+1 =

λQP k ,

µk+1 =

(6) µQP k

,

(7)

QP where (∆wk , λQP k , µk ) is the solution of a quadratic program (QP). In the classical Newton-type or SQP approaches, this QP has the form

min ∆w ∈ Rnw subject to

1 ∆wT Ak ∆w + ∇w a(wk )T ∆w +2 b(wk ) + ∇w b(wk )T ∆w = 0

(8)

c(wk ) + ∇w c(wk )T ∆w ≥ 0

where Ak is an approximation of the Hessian of the Lagrangian, Ak ≈ ∇2w L(wk , λk , µk ) , and ∇w b(wk )T and ∇w c(wk )T are the constraint Jacobians. Depending on the quality of the Hessian approximation we may expect linear, super-linear or even quadratic convergence. Practical SQP methods diﬀer e.g. in the type of globalisation strategy, in the type of QP solver used, or in the way the Hessian is approximated – e.g. by BFGS updates or by a Gauss-Newton Hessian. This last approach is favourable for least squares problems, as e.g in tracking or estimation problems. When the objective is given as a(w) = r(w) 22 , the Gauss-Newton Hessian is given by Ak = 2∇w r(wk )∇w r(wk )T . It is a good approximation of the exact Hessian ∇2w L(wk , λk , µk ) if the residual r(w) 22 is small or if the problem is only mildly nonlinear. 2.2 Direct Single Shooting The single shooting approach starts by discretizing the controls. We might for example choose grid points on the unit interval, 0 = τ0 < τ1 < . . . < τN = 1, and then rescale these gridpoints to the possibly variable time horizon of the optimal control problem, [0, T ], by deﬁning ti = T τi for i = 0, 1, . . . , N . On this grid we discretize the controls u(t), for example piecewise constant, u(t) = qi for t ∈ [ti , ti+1 ], so that u(t) only depends on the the ﬁnitely many control parameters q = (q0 , q1 , . . . , qN −1 , T ) and can be denoted by u(t; q). If the problem has a ﬁxed horizon length T , the last component of q disappears as it is no optimization variable. Using a numerical simulation routine for solving the initial value problem x(0) = x0 ,

x(t) ˙ = f (x(t), u(t; q)),

t ∈ [0, T ] ,

we can now regard the states x(t) on [0, T ] as dependent variables, cf. Fig. 4. We denote them by x(t; q). The question which simulation routine should be

72

M. Diehl et al.

Fig. 4. Illustration of single shooting

chosen is crucial to the success of any shooting method and depends on the type of ODE model. It is essential to use an ODE solver that also delivers sensitivities, as they are needed within the optimization. We also discretize the path constraints to avoid a semi-inﬁnite problem, for example by requiring h(x(t), u(t)) ≥ 0 only at the grid points ti , but we point out that also a ﬁner grid could be chosen without any problem. Thus, we obtain the following ﬁnite dimensional nonlinear programming problem (NLP): T minimize q

L(x(t; q), u(t; q)) dt + E (x(T ; q))

(9)

0

subject to h(x(ti ; q), u(ti ; q)) ≥ 0, r (x(T ; q)) = 0.

i = 0, . . . , N, (discretized path constraints) (terminal constraints)

This problem is solved by a ﬁnite dimensional optimization solver, e.g. Sequential Quadratic Programming (SQP), as described above. The behaviour of single shooting (with full step SQP and Gauss-Newton Hessian) applied to the tutorial example is illustrated in Fig. 5. The initialization – at the zero control trajectory, u(t) = 0 – and the ﬁrst iteration are shown. Note that the state path and terminal constraints are not yet satisﬁed in the ﬁrst iteration, due to their strong nonlinearity. The solution (up to an accuracy of 10−5 ) is obtained after seven iterations. The strong points of single shooting are (i) that it can use fully adaptive, error controlled state-of-the-art ODE or DAE solvers, (ii) that it has only few optimization degrees of freedom even for large ODE or DAE systems, and (iii) that only initial guesses for the control degrees of freedom are needed. The weak points are (i) that we cannot use knowledge of the state trajectory x in the initialization (e.g. in tracking problems), (ii) that the ODE solution x(t; q) can depend very nonlinearly on q, as in the example, and (iii) that unstable systems are diﬃcult to treat. However, due to its simplicity, the single shooting approach is very often used in engineering applications e.g. in the commercial package gOPT [41].

Fast Optimal Robot Control

73

Fig. 5. Single shooting applied to the tutorial example: Initialization and ﬁrst iteration

2.3 Collocation We only very brieﬂy sketch here the idea of the second direct approach, collocation. We start by discretizing both, the controls and the states on a ﬁne grid. Typically, the controls are chosen to be piecewise constant, with values qi on each interval [ti , ti+1 ]. The value of the states at the grid points will be denoted by si ≈ x(ti ). In order to avoid notational overhead, we will in the remainder of this section assume that the length of the time horizon, T , is constant, but point out that the generalization to variable horizon problems by the above mentioned time transformation is straightforward. In collocation, the inﬁnite ODE x(t) ˙ − f (x(t), u(t)) = 0,

t ∈ [0, T ]

is replaced by ﬁnitely many equality constraints ci (qi , si , si , si+1 ) = 0,

i = 0, . . . , N − 1 ,

where the additional variables si might represent the state trajectory on intermediate “collocation points” within the interval [ti , ti+1 ]. By a suitable choice

74

M. Diehl et al.

of the location of these points a high approximation order can be achieved, and typically they are chosen to be the zeros of orthogonal polynomials. But we sketch here only a simpliﬁed tutorial case, where no intermediate variables si are present, to give a ﬂavour of the idea of collocation. Here, the additional equalities are given by si + si+1 si+1 − si , qi . −f ci (qi , si , si+1 ) := ti+1 − ti 2 Then, we will also approximate the integrals on the collocation intervals, e.g. by li (qi , si , si+1 ) := L

ti+1 si + si+1 , qi (ti+1 − ti ) ≈ L(x(t), u(t))dt 2 ti

After discretization we obtain a large scale, but sparse NLP: N −1

minimize s, q

li (qi , si , si+1 ) + E (sN )

i=0

subject to s0 − x0 = 0, ci (qi , si , si+1 ) = 0, h(si , qi ) ≥ 0, r (sN ) = 0.

(ﬁxed initial value) i = 0, . . . , N − 1, i = 0, . . . , N,

(discretized ODE model) (discretized path constraints) (terminal constraints)

This problem is then solved e.g. by a reduced SQP method for sparse problems [8, 48], or by an interior-point method [7]. Eﬃcient NLP methods typically do not keep the iterates feasible, so the discretized ODE model equations are only satisﬁed at the NLP solution, i.e., simulation and optimization proceed simultaneously. The advantages of collocation methods are (i) that a very sparse NLP is obtained (ii) that we can use knowledge of the state trajectory x in the initialization (iii) that it shows fast local convergence (iv) that it can treat unstable systems well, and (v) that it can easily cope with state and terminal constraints. Its major disadvantage is that adaptive discretization error control needs regridding and thus changes the NLP dimensions. Therefore, applications of collocation do often not address the question of proper discretization error control. Nevertheless, it is successfully used for many practical optimal control problems [3, 50, 47, 14, 54].

Fast Optimal Robot Control

75

Fig. 6. Illustration of multiple shooting

2.4 Direct Multiple Shooting The direct multiple shooting method (that is due to Bock and Plitt [12]) tries to combine the advantages of a simultaneous method like collocation with the major advantage of single shooting, namely the possibility to use adaptive, error controlled ODE solvers. In direct multiple shooting, we proceed as follows. First, we again discretize the controls piecewise on a coarse grid u(t) = qi

for t ∈ [ti , ti+1 ] ,

where the intervals can be as large as in single shooting. But second, we solve the ODE on each interval [ti , ti+1 ] independently, starting with an artiﬁcial initial value si : x˙ i (t) = f (xi (t), qi ), t ∈ [ti , ti+1 ] , xi (ti ) = si . By numerical solution of these initial value problems, we obtain trajectory pieces xi (t; si , qi ), where the extra arguments after the semicolon are introduced to denote the dependence on the interval’s initial values and controls. Simultaneously with the decoupled ODE solution, we also numerically compute the integrals ti+1

li (si , qi ) :=

L(xi (ti ; si , qi ), qi )dt . ti

In order to constrain the artiﬁcial degrees of freedom si to physically meaningful values, we impose continuity conditions si+1 = xi (ti+1 ; si , qi ). Thus, we arrive at the following NLP formulation that is completely equivalent to the single shooting NLP, but contains the extra variables si , and has a block sparse structure.

76

M. Diehl et al. N −1

minimize s, q

li (si , qi ) + E (sN )

(10)

i=0

subject to s0 − x0 = 0, si+1 − xi (ti+1 ; si , qi ) = 0, h(si , qi ) ≥ 0, r (sN ) = 0.

(initial value) i = 0, . . . , N − 1, (continuity) i = 0, . . . , N,

(discretized path constraints) (terminal constraints)

If we summarize all variables as w := (s0 , q0 , s1 , q1 , . . . , sN ) we obtain an NLP in the form (2). The block sparse Jacobian ∇b(wk )T contains the linearized dynamic model equations, and the Hessian ∇2w L(wk , λk , µk ) is block diagonal, which can both be exploited in the tailored SQP solution procedure [12]. Because direct multiple shooting only delivers a valid (numerical) ODE solution when also the optimization iterations terminate, it is usually considered a simultaneous method, as collocation. But sometimes it is also called a hybrid method, as it combines features from both, a pure sequential, and a pure simultaneous method. Its advantages are mostly the same as for collocation, namely that knowledge of the state trajectory can be used in the initialization, and that it robustly handles unstable systems and path state and terminal constraints. The performance of direct multiple shooting – and of any other simultaneous method – is for the tutorial example illustrated in Fig. 7. The ﬁgure shows ﬁrst the initialization by a forward simulation, using zero controls. This is one particularly intuitive, but by far not the best possibility for initialization of a simultaneous method: it is important to note that the state trajectory is by no means constrained to match the controls, but can be chosen point for point if desired. In this example, the forward simulation is at least reset to the nearest bound whenever the state bounds are violated at the end of an interval, in order to avoid simulating the system in areas where we know it will never be at the optimal solution. This leads to the discontinuous state trajectory shown in the top row of Fig. 7. The result of the ﬁrst iteration is shown in the bottom row, and it can be seen that it is already much closer to the solution than single shooting, cf. Fig. 5. The solution, cf. Fig. 3, is obtained after two more iterations. It is interesting to note that the terminal constraint is already satisﬁed in the ﬁrst iteration, due to its linearity. The nonlinear eﬀects of the continuity conditions are distributed over the whole horizon, which is seen in the discontinuities. This is in contrast to single shooting, where the nonlinearity of the system is accumulated until the end of the horizon, and the terminal constraint becomes much more nonlinear than necessary. Any simultaneous method, e.g. collocation, would show the same favourable performance as direct multiple shooting here. As said above, in contrast to collocation, direct multiple shooting can combine adaptivity with ﬁxed NLP dimensions, by the use of adaptive ODE/DAE

Fast Optimal Robot Control

77

Fig. 7. Multiple shooting applied to the tutorial example: Initialization and ﬁrst iteration

solvers. Within each SQP iteration, the ODE solution is often the most costly part, that is easy to parallelize. Compared to collocation the NLP is of smaller dimension but less sparse. This loss of sparsity, together with the cost of the underlying ODE solution leads to theoretically higher costs per SQP iteration than in collocation. On the other hand, the possibility to use eﬃcient state-of-the-art ODE/DAE solvers and their inbuilt adaptivity makes direct multiple shooting a strong competitor to direct collocation in terms of CPU time per iteration. From a practical point of view it oﬀers the advantage that the user does not have to decide on the grid of the ODE discretization, but only on the control grid. Direct multiple shooting was used to solve practical oﬄine optimal control problems e.g. in [24, 33], and it is also used for the calculations in this paper. It is also widely used in online optimization and NMPC applications e.g. in [44, 43, 52, 53, 18, 25].

3 Time Optimal Control of a Five Link Robot Arm We consider the time optimal point to point motion of a robot arm with ﬁve degrees of freedom. Figure 8 shows the robot and its possible movements. To

78

M. Diehl et al.

Fig. 8. Robot appearance with simpliﬁed last link and manipulator

provide a better visualization the last link and the manipulator in the images are shorter and simpliﬁed compared to the assumed model parameters. The robot is modelled as a kinematic chain of rigid bodies, i.e., the robot is assumed to just consist of joints and links between them. The robot has a rotational base joint with two degrees of freedom, followed by two links with rotary joints, and ﬁnally one rotational joint at the “hand” of the arm. Each of the ﬁve joints contains a motor to apply a torque ui (t). The geometric description of the robot uses the notation of Denavit and Hartenberg [16]. To provide the data for the dynamic calculation each link is associated with an inertia tensor, the mass and the position of the center of mass. This approach leads to a set of ﬁve generalized coordinates (q1 (t), . . . , q5 (t)) each representing a rotation in the corresponding joint. We have chosen parameters that correspond to a small toy robot arm, and which are listed in Table 1 using the conventional Denavit-Hartenberg notation. The corresponding equations of motion can then be generated automatically by a script from the HuMAnS Toolbox [29]. 3.1 Fast Computations of the Dynamics of Robots The dynamics of a robot is most usually presented in its Lagrangian form Table 1. Dynamic data of the example robot, and Denavit-Hartenberg parameters Joint i 1 2 3 4 5

Mass mi c.o.m. ri 0.1 (0, 0, 0)T 0.02 (0.06, 0, 0)T 0.1 (0.06, 0, 0)T 0.03 (0, −0.04, 0)T 0.06 (0, 0, 0.1)T

Inertia tensor Ii diag(23, 23, 20) · 10−6 diag(7, 118, 113) · 10−6 diag(20, 616, 602) · 10−6 diag(−51, −7, −46) · 10−6 diag(650, 640, 26) · 10−6

αi 0 − π2 0 0 π 2

ai 0 0 0.12 0.12 0

θi q1 (t) − π2 +q2 (t) π +q3 (t) 2 π +q4 (t) 2 q5 (t)

di 0 0 0 0 0

Fast Optimal Robot Control

79

M (q(t)) q¨(t) + N (q(t), q(t)) ˙ = u(t) , which gives a compact description of all the nonlinear phenomena and can be manipulated easily in various ways. Since the mass matrix M (q(t)) is Symmetric Deﬁnite Positive, it is invertible and the acceleration of the system can be related with the controls u(t) either in the way

or in the way

u(t) = M (q(t)) q¨(t) + N (q(t), q(t)) ˙

(11)

q¨(t) = M (q(t))−1 u(t) − N (q(t), q(t)) ˙ ,

(12)

corresponding respectively to the inverse and direct dynamics of the system. Very helpful from the point of view of analytical manipulations [56], this way of describing the dynamics of a robot is far from being eﬃcient from the point of view of numerical computations, neither in the form (11) nor (12). Especially the presence of a matrix-vector multiplication of O(N 2 ) complexity in both (11) and(12), and of a matrix inversion of O(N 3 ) complexity in (12) can be avoided: recursive algorithms for computing both (11) and (12) with only an O(N ) complexity are well known today. The ﬁrst algorithm that has been investigated historically for the fast computation of the dynamics of robots is the Recursive Newton-Euler Algorithm that allows computing directly the controls related to given accelerations exactly as in (11). Extensions have been devised also for cases when not all of the acceleration vector q¨ is known, in the case of underactuated systems such as robots executing aerial maneuvers [49]. This recursive algorithm is the fastest way to compute the complete dynamics of a robotic system and should be preferred therefore as long as one is not strictly bound to using the direct dynamics (12). This is the case for collocation methods but unfortunately not for shooting methods. The Recursive Newton-Euler Algorithm has been adapted then in the form of the Composite Rigid Body Algorithm in order to compute quickly the mass matrix that needs to be inverted in the direct dynamics (12), but we still have to face then a matrix inversion which can be highly ineﬃcient for “large” systems. The computation of this mass matrix and its inversion can be necessary though for systems with unilateral contacts, when some internal forces are deﬁned through implicit laws [55]. The Articulated Body Algorithm has been designed then to propose a recursive method of O(N ) complexity for computing directly the accelerations related to given torques as in (12) but without resorting to a matrix inversion. Even though generating a slightly higher overhead, this algorithm has been proved to be more eﬃcient than the Composite Rigid Body Algorithm for robots with as few as 6 degrees of freedom [26]. Moreover, avoiding the matrix inversion allows producing a less noisy numerical result, what can greatly enhance the eﬃciency of any adaptive ODE solver to which it is connected [2]. For these reasons, this recursive algorithm should be preferred as soon as one

80

M. Diehl et al.

needs to compute the direct dynamics (12), what is the case for shooting methods. Now, one important detail when designing fast methods to compute numerically the dynamics of a robot is to generate oﬄine the computer code corresponding to the previous algorithms. Doing so, not only is it possible to get rid of constants such as 0 and 1 with all their consequences on subsequent computations, but it is also possible to get rid of whole series of computations which may appear to be completely unnecessary, depending on the speciﬁc structure of the robot. Such an oﬄine optimization leads to computations which can be as much as twice faster than the strict execution of the same original algorithms. The HuMAnS toolbox [29], used to compute the dynamics of the robot for the numerical experiment in the next section, proposes only the Composite Rigid Body Algorithm, so far, so even faster computations should be expected when using an Articulated Body Algorithm. Still, this toolbox produces faster computations than other generally available robotics toolboxes thanks to its oﬄine optimization of the generated computer code (a feature also present in the SYMORO software [30]). 3.2 Optimization Problem Formulation In order to solve the problem to minimize a point to point motion of the robot arm, we consider the following example maneuver: the robot shall pick up an object at the ground and put it as fast as possible into a shelf, requiring a base rotation of ninety degrees. We formulate an optimal control problem of the form (1), with the following deﬁnitions: x(t) = (q1 (t), . . . , q5 (t), q˙1 (t), . . . , q˙5 (t))T u(t) = (u1 (t), . . . , u5 (t))T L(x(t), u(t)) = 1 E(x(T )) = 0 f (x(t), u(t)) =

(q˙1 (t), . . . , q˙5 (t))T M (x(t))−1 · (u(t) − N (x(t)))

x0 = (−0.78, 0.78, 0, 0.78, 0, 0, 0, 0, 0, 0)T r(x(T )) = x(T ) − (0.78, 0, −0.78, 0.78, 0, 0, 0, 0, 0, 0)T   xmax − x   x − xmin       umax − u  h(x(t), u(t)) =    u − umin      (1, 0, 0, 1) · T50 (x(t)) · (0, 0, l, 1)T − 0.05  (0, 0, 1, 1) · T50 (x(t)) · (0, 0, l, 1)T + 0.15

Fast Optimal Robot Control

81

The controls u(t) are the torques acting in the joints. The cost functional ,T L(x, u)dt + E(x(T )) is the overall maneuver time, T . Within the dynamic 0 model x˙ = f (x, u), the matrix M (x(t)) is the mass matrix which is calculated in each evaluation of f (x(t), u(t)) and inverted using a cholesky algorithm. The vector N (x(t)) describes the combined centrifugal, Coriolis and gravitational force. The initial and terminal constraints x(0) = x0 and r(x(T )) = 0 describe the desired point to point maneuver. As the states and controls have lower and upper bounds, and as the robot hand shall avoid hitting the the ground as well as its own base, we add the path constraints h(x, u) ≥ 0. Here, the matrix T50 (x(t) describes the transformation that leads from the local end eﬀector position (0, 0, l, 1)T in the last frame to the absolute coordinates in the base frame. 3.3 Numerical Solution by Direct Multiple Shooting We have coupled the automatic robot model generator HuMAnS [29] with an eﬃcient implementation of the direct multiple shooting method, the optimal control package MUSCOD-II [32, 33]. This coupling allows us to use the highly optimized C-code delivered by HuMAnS within the model equations x˙ = f (x, u) required by MUSCOD-II in an automated fashion. In the following computations, we choose an error controlled Runge-Kutta-Fehlberg integrator of order four/ﬁve. We use 30 multiple shooting nodes with piecewise constant controls. Within the SQP method, a BFGS Hessian update and watchdog line search globalisation is used. For initialization, the diﬀerential states on the multiple shooting nodes are interpolated linearly between desired initial and terminal state, as shown in Fig. 9. The maneuver time for initialization was set to 0.3 seconds, and the controls to zero. Starting with this infeasible initialization, the overall optimization with MUSCOD-II took about 130 SQP iterations, altogether requiring about 20 CPU seconds on a standard LINUX machine with a 3 GHz Pentium IV processor. The solution is shown in Fig. 10. The calculated time optimal robot movement of 0.15 seconds duration is illustrated in Fig. 11 with screenshots from an OpenGL visualization.

4 Nonlinear Model Predictive Control As mentioned in the introduction, Nonlinear Model Predictive Control (NMPC) is a feedback control technique based on the online solution of openloop optimal control problems of the form (1). The optimization is repeated again and again, at intervals of length δ, each sampling time tk = kδ for the most currently observed system state x ¯(tk ), which serves as initial value ¯(tk ) in (1). We have introduced the bar to distinguish the observed x0 := x system states x ¯(t) from the predicted states x(t) within the optimal control problem. Note that the time tk from now on is the physical time, and no

82

M. Diehl et al.

Fig. 9. Initalization of the optimization problem by linear interpolation

longer the time at a discretization point within the optimal control problem, as in Sect. 2. We stress that for autonomous systems, as treated in this paper, the NMPC optimization problems diﬀer by the varying initial values only, and that the time coordinate used within the optimal control problem (1) can be assumed to always start with t = 0 even though this does not reﬂect the physical time. From now on, we will denote the time coordinate within the optimal control problem with τ in this section to avoid confusion. To be speciﬁc, we denote the optimal solution of the optimal control prob¯(tk )), τ ∈ [0, T ∗ (¯ x(tk ))], to express its parametric depenlem (1) by u∗ (τ ; x

Fast Optimal Robot Control

83

Fig. 10. Solution of the optimization problem, obtained after 130 SQP iterations and 20 CPU seconds

dence on the initial value x ¯(tk ). The feedback control implemented during the following sampling interval, i.e. for t ∈ [tk , tk+1 ], is simply given by x(tk )) := u∗ (0; x ¯(tk )).1 Thus, NMPC is a sampled data feedback conu∗0 (¯ 1

Sometimes, instead of the optimal initial control value u∗ (0; x ¯(tk )), the whole ﬁrst ¯(tk )), τ ∈ [0, δ], is applied to the real control interval of length δ, i.e., u∗ (τ ; x process. This is more appealing in theory, and stability proofs are based on such an NMPC formulation. When a control discretization with interval lengths not smaller than the sampling time is used, however, both formulations coincide.

84

M. Diehl et al.

Fig. 11. Visualization of the time optimal point to point motion from Fig. 10

trol technique. It is closely related to optimal feedback control which would x(t)) for all t, which can apply the continuous, non-sampled feedback law u∗0 (¯ be called the limit of NMPC for inﬁnitely small sampling times δ. Note that the optimal predicted maneuver time T ∗ (xk ) would typically be shrinking for an optimal point to motion. In this case we speak of shrinking horizon NMPC [10]. If a large disturbance occurs, the horizon might also be enlarged as the future plan is changed. In the other case, when the horizon length is ﬁxed to T = Tp , where the constant Tp is the prediction horizon length, we speak of moving, or receding horizon control (RHC) [37]. The moving horizon approach is applicable to continuous processes and so widely employed that the term NMPC is often used as synonymous to RHC. When a given trajectory shall be tracked, this is often expressed by the choice of the cost function in form of an integrated least squares deviation on a ﬁxed prediction horizon. In fast robot motions, however, we believe that a variable time horizon for point to point maneuvers will be a crucial ingredient to successful NMPC implementations. A shrinking horizon NMPC approach for robot point to point motions that avoids that T ∗ (x) shrinks below a certain positive threshold was presented by Zhao et al. [57]. For setpoint tracking problems, extensive literature exists on the stability of the closed loop system. Given suitable choices of the objective functional deﬁned via L and E and a terminal constraint of the form r(x(T )) = 0 or r(x(T )) ≥ 0, stability of the nominal NMPC dynamics can be proven even for strongly nonlinear systems [37, 15, 38, 35]. One important precondition for successful NMPC applications, however, is the availability of reliable and eﬃcient numerical optimal control algorithms. Given an eﬃcient oﬄine optimization algorithm – e.g. one of the three SQP based direct methods described in Sect. 2 – we might be tempted to restart

Fast Optimal Robot Control

85

it again and again for each new problem and to solve each problem until a prespeciﬁed convergence criterion is satisﬁed. If we are lucky, the computation time is negligible; if we are not, we have to enter the ﬁeld of real-time optimization. The Online Dilemma Assuming that the computational time for one SQP iteration is more or less constant, we have to address the following dilemma: If we want to obtain a suﬃciently exact solution for a given initial value x ¯(tk ), we have to perform several SQP iterations until a prespeciﬁed convergence criterion is satisﬁed. We can suppose that for achieving this we have to perform n iterations, and that each iteration takes a time . This means that we obtain the optimal x(tk )) only at a time tk +n, i.e., with a considerable delay. feedback control u∗0 (¯ However at time tk + n the system state has already moved to some system ¯(tk ), and u∗0 (x(tk )) is not the exact NMPC feedback, state x ¯(tk + n) = x ∗ x(tk + nδ)). In the best case the system state has not changed much in u0 (¯ the meantime and it is a good approximation of the exact NMPC feedback. Also, one might think of predicting the most probable system state x ¯(tk + n) and starting to work on this problem already at time tk . The question of which controls have to be applied in the meantime is still unsolved: a possible choice would be to use previously optimized controls in an open-loop manner. Note that with this approach we can realize an NMPC recalculation rate with intervals of length δ = n, under the assumption that each problem needs at most n iterations and that each SQP iteration requires at most a CPU time of . Note also that feedback to a disturbance comes with a delay δd of one full sampling time. Summarizing, we would have δd = δ = n. 4.1 Real-Time Iteration Scheme We will now present a speciﬁc answer to the online dilemma, the real-time iteration scheme [17, 20]. The approach is based on two observations. • Due to the online dilemma, we will never be able to compute the exact NMPC feedback control u∗0 (¯ x(tk )) without delay. Therefore, it might be better to compute only an approximation u ˜0 (¯ x(tk )) of u∗0 (¯ x(tk )), if this approximation can be computed much faster. • Second, we can divide the computation time of each cycle into a a short feedback phase (FP) and a possibly much longer preparation phase (PP). While the feedback phase is only used to evaluate the approximation x(tk )), the following preparation phase is used to prepare the next feedu ˜0 (¯ back, i.e., to compute u ˜0 (¯ x(tk+1 )) as much as possible without knowledge of x ¯(tk+1 ). This division of the computation time within each sampling interval allows to achieve delays δd that are smaller than the sampling interval δ, see Fig. 12.

86

M. Diehl et al.

Fig. 12. Division of the computation time in the real-time iteration scheme; real system state and control trajectory, for sampling time δ and feedback delay δd δ

The crucial question is, of course, which approximation u ˜0 (¯ x(tk )) should be x(tk )). used, and how it can be made similar to the exact NMPC feedback u∗0 (¯ In its current realization, the real-time iteration scheme is based on the direct multiple shooting method. The online optimization task is to solve a sequence of nonlinear programming problems of the form (10), but with ¯(tk ) = 0. Similar to the NLP notation (2), varying initial value constraint s0 − x in the online context we have to solve, as fast as possible, an NLP P (¯ x(tk )) :

min a(w) w

subject to

bx¯(tk ) (w) = 0,

c(w) ≥ 0 ,

(13)

where the index takes account of the fact that the ﬁrst equality constraint s0 − ¯(tk ), and where w = x ¯(tk ) = 0 from bx¯(tk ) (w) = 0 depends on the initial value x x(tk )) of (s0 , q0 , s1 , q1 , . . . , sN ). Ideally, we would like to have the solution w∗ (¯ each problem P (¯ x(tk )) as quick as possible, and to take the NMPC feedback x(tk )), i.e., to set u∗0 (¯ x(tk )) := q0∗ (¯ x(tk )). law to be the ﬁrst control within w∗ (¯ ∗ ¯(tk ) is The exact solution manifold w ( · ) in dependence of the initial value x sketched as the solid line in Fig. 13 – nondiﬀerentiable points on this manifold are due to active set changes in the NLP. The exact solution, however, is not computable in ﬁnite time. Initial Value Embedding The major idea underlying the real-time iteration scheme is to initialize each new problem P (¯ x(tk )) with the most current solution guess from the last problem, i.e. with the solution of P (¯ x(tk−1 )). In a simultaneous method like direct multiple shooting, it is no problem that the initial value constraint ¯(tk ) = 0 is violated. On the contrary, because this constraint is linear, it s0 − x can be shown that the ﬁrst SQP iteration after this “initial value embedding” is a ﬁrst order predictor for the correct new solution, even in the presence of

Fast Optimal Robot Control

w

87

1st iteration 2nd iteration

solution of P (¯ x(tk−1 )) solution of P (¯ x(tk )) x ¯(tk−1 )

x¯(tk )

Fig. 13. Exact solution manifold (solid line) and tangential predictor after initial value embedding (dashed line), when initialized with the solution of P (¯ x(tk−1 )). The ﬁrst iteration delivers already a good predictor for the exact solution of P (¯ x(tk ))

active set changes [17]. This observation is visualized in Fig. 13, where the predictor delivered by the ﬁrst SQP iteration is depicted as dashed line. In the real-time iteration scheme, we use the result of the ﬁrst SQP iterax(tk )). This would already reduce the tion directly for the approximation u ˜0 (¯ feedback delay δd to the time of one SQP iteration, . Afterwards, we would need to solve the old problem to convergence in order to prepare the next feedback. In Fig. 13 also the second iterate and solution for problem P (¯ x(tk )) are sketched. But two more considerations make the algorithm even faster. • First, the computations for the ﬁrst iteration can be largely performed before the initial value x ¯(tk ) is known. Therefore, we can reduce the delay time further, if we perform all these computations before time tk , and at ˜0 (¯ x(tk )) to the time tk we can quickly compute the feedback response u current state. Thus, the feedback delay δd becomes even smaller than the cost of one SQP iteration, δd . • Second, taking into account that we already use an approximate solution of the optimal control problem we can ask if it is really necessary to iterate the SQP until convergence requiring a time n for n SQP iterations. Instead, we will considerably reduce the preparation time by performing just one iteration per sampling interval. This allows shorter sampling intervals that only have the duration of one single SQP iteration, i.e., δ = . A positive side-eﬀect is that this shorter recalculation time most probably leads to ¯(tk+1 ), so that smaller diﬀerences in subsequent initial states x ¯(tk ) and x the initial value embedding delivers better predictors. These two ideas are the basis of the real-time iteration scheme. It allows to realize feedback delays δd that are much shorter than a sampling time, and sampling times δ that are only as long as a single SQP iteration, i.e. we have δd δ = . Compared with the conventional approach with δd = δ = n, the

88

M. Diehl et al. w

3rd iteration 2nd iteration 1st iteration

0th iteration

x ¯(tk )

x ¯(tk+1 )

x ¯(tk+2 )

x ¯(tk+3 )

Fig. 14. Sketch of the real-time iterations that stay close to the exact solution manifold (solid line)

focus is now shifted from a sequence of optimization problems to a sequence of SQP iterates: we may regard the SQP procedure iterating uninterrupted, with the only particularity that the initial value x¯(tk ) is modiﬁed during the iterations. The generation of the feedback controls can then be regarded as a by-product of the SQP iterations. Due to the initial value embedding property, it can be expected that the iterates remain close to the exact solution manifold for each new problem. In Fig. 14 four consecutive real-time iterates are sketched, where the dashed lines show the respective tangential predictors. Applications The real-time iteration scheme has successfully been used in both simulated and experimental NMPC applications, among them the experimental NMPC of a high purity distillation column [23] described by a 200 state DAE model with sampling times δ of 20 seconds, or simulated NMPC of a combustion engine described by 5 ODE, with sampling times of 10 milliseconds [27]. Depending on the application, the feedback delay δd was between 0.5 and 5 percent of the sampling time. Within the studies, the approximation errors of the real-time iteration scheme compared to exact NMPC are often negligible. The scheme’s theoretical contraction properties have been investigated in [21] for the variant described in this paper, and in [22, 19] for other variants. Recently, several novel variants of the real-time iteration scheme have been proposed that can either work with inexact jacobians within the SQP procedure [11], or that only evaluate the jacobians on a subspace [46]. These variants oﬀer advantages for large scale systems where they promise to allow sampling times that are one or two orders of magnitude smaller than in the standard implementation. A numerical application of the standard real-time iteration scheme to the time optimal point to point motion of a robot arm described by 8 ODE was presented in [57], with CPU times still in the order of 100 milliseconds per sampling time. We expect that the development of

Fast Optimal Robot Control

89

real-time iteration variants that are specially tailored to robotics applications will make NMPC of fast robot motions possible within the next ﬁve years.

5 Summary and Outlook In this tutorial paper, we have tried to give a (personal) overview over the most widely used methods for numerical optimal control, and to assess the possibility of real-time optimization of fast robot motions. We discussed in detail direct optimal control methods that are based on a problem discretization and on the subsequent use of a nonlinear programming algorithm like sequential quadratic programming (SQP). We compared three direct methods, (i) direct single shooting as a sequential approach, together with (ii) direct collocation and (iii) direct multiple shooting as simultaneous approaches. At hand of a tutorial example we have illustrated the better nonlinear convergence properties of the simultaneous over the sequential approaches that can be observed in many other applications, too. The direct multiple shooting method allows to use state-of-the-art ODE/DAE integrators with inbuilt adaptivity and error control which often shows to be an advantage in practice. At the example of the time optimal motion of a robot arm we have demonstrated the ability of direct multiple shooting to cope even with strongly nonlinear two point boundary value optimization problems. Using the coupling of an eﬃcient tool for generation of optimized robot model equations, HuMAnS, and a state-ofthe-art implementation of the direct multiple shooting method, MUSCOD-II, computation times for a ﬁve link robot are in the order of 200 milliseconds per SQP iteration. Finally, we discussed the possibility to generate optimization based feedback by the technique of nonlinear model predictive control (NMPC), and pointed out the necessity of fast online optimization. We have presented the real-time iteration scheme – that is based on direct multiple shooting and SQP – as a particularly promising approach to achieve this aim. The scheme uses an initial value embedding for the transition from one optimization problem to the next, and performs exactly one SQP-type iteration per optimization problem to allow short sampling times. Furthermore, each iteration is divided into a preparation and a much shorter feedback phase, to allow an even shorter feedback delay. Based on the ongoing development of the presented approaches, we expect NMPC – that performs an online optimization of nonlinear ﬁrst principle robot models within a few milliseconds – to become a viable technique for control of fast robot motions within the following ﬁve years.

References [1] F. Allg¨ ower, T.A. Badgwell, J.S. Qin, J.B. Rawlings, and S.J. Wright. Nonlinear predictive control and moving horizon estimation – An introductory overview.

90

[2]

[3]

[4]

[5] [6] [7]

[8]

[9]

[10]

[11]

[12]

[13] [14] [15] [16]

[17]

M. Diehl et al. In P. M. Frank, editor, Advances in Control, Highlights of ECC’99, pp. 391–449. Springer, 1999. U.M. Ascher, D.K. Pai, and B.P. Cloutier. Forward dynamics, elimination methods, and formulation stiﬀness in robot simulation. International Journal of Robotics Research, 16(6):749–758, 1997. V. B¨ ar. Ein Kollokationsverfahren zur numerischen L¨ osung allgemeiner Mehrpunktrandwertaufgaben mit Schalt– und Sprungbedingungen mit Anwendungen in der optimalen Steuerung und der Parameteridentiﬁzierung. Master’s thesis, Universit¨ at Bonn, 1984. R.A. Bartlett, A. W¨ achter, and L.T. Biegler. Active set vs. interior point strategies for model predictive control. In Proc. Amer. Contr. Conf., pp. 4229– 4233, Chicago, Il, 2000. R.E. Bellman. Dynamic Programming. University Press, Princeton, 1957. D.P. Bertsekas. Dynamic Programming and Optimal Control, vol. 1. Athena Scientiﬁc, Belmont, MA, 1995. L.T. Biegler, A.M. Cervantes, and A. Waechter. Advances in simultaneous strategies for dynamic process optimization. Chemical Engineering Science, 4(57):575–593, 2002. L.T. Biegler. Solution of dynamic optimization problems by successive quadratic programming and orthogonal collocation. Computers and Chemical Engineering, 8:243–248, 1984. L.T. Biegler. Eﬃcient solution of dynamic optimization and NMPC problems. In F. Allg¨ ower and A. Zheng, editors, Nonlinear Predictive Control, vol. 26 of Progress in Systems Theory, pp. 219–244, Basel Boston Berlin, 2000. Birkh¨ auser. T. Binder, L. Blank, H.G. Bock, R. Bulirsch, W. Dahmen, M. Diehl, T. Kronseder, W. Marquardt, J.P. Schl¨ oder, and O.v. Stryk. Introduction to model based optimization of chemical processes on moving horizons. In M. Gr¨ otschel, S.O. Krumke, and J. Rambau, editors, Online Optimization of Large Scale Systems: State of the Art, pp. 295–340. Springer, 2001. H.G. Bock, M. Diehl, E.A. Kostina, and J.P. Schl¨ oder. Constrained optimal feedback control of systems governed by large diﬀerential algebraic equations. In L. Biegler, O. Ghattas, M. Heinkenschloss, D. Keyes, and B. van Bloemen Waanders, editors, Real-Time and Online PDE-Constrained Optimization. SIAM, 2006. (in print). H.G. Bock and K.J. Plitt. A multiple shooting algorithm for direct solution of optimal control problems. In Proceedings 9th IFAC World Congress Budapest, pp. 243–247. Pergamon Press, 1984. A.E. Bryson and Y.-C. Ho. Applied Optimal Control. Wiley, New York, 1975. A. Cervantes and L.T. Biegler. Large-scale DAE optimization using a simultaneous NLP formulation. AIChE Journal, 44(5):1038–1050, 1998. H. Chen and F. Allg¨ ower. A quasi-inﬁnite horizon nonlinear model predictive control scheme with guaranteed stability. Automatica, 34(10):1205–1218, 1998. J. Denavit and R.S. Hartenberg. A kinematic notation for lower-pair mechanisms based on matrices. ASME Journal of Applied Mechanics, 22:215–221, 1955. M. Diehl. Real-Time Optimization for Large Scale Nonlinear Processes. PhD thesis, Universit¨ at Heidelberg, 2001. http://www.ub.uni-heidelberg.de/ archiv/1659/.

Fast Optimal Robot Control

91

[18] M. Diehl. Real-Time Optimization for Large Scale Nonlinear Processes, vol. 920 of Fortschr.-Ber. VDI Reihe 8, Meß-, Steuerungs- und Regelungstechnik. VDI Verlag, D¨ usseldorf, 2002. Download also at: http://www.ub.uniheidelberg.de/archiv/1659/. [19] M. Diehl, H.G. Bock, and J.P. Schl¨ oder. A real-time iteration scheme for nonlinear optimization in optimal feedback control. SIAM Journal on Control and Optimization, 43(5):1714–1736, 2005. [20] M. Diehl, H.G. Bock, J.P. Schl¨ oder, R. Findeisen, Z. Nagy, and F. Allg¨ ower. Real-time optimization and nonlinear model predictive control of processes governed by diﬀerential-algebraic equations. J. Proc. Contr., 12(4):577–585, 2002. [21] M. Diehl, R. Findeisen, and F. Allg¨ ower. A stabilizing real-time implementation of nonlinear model predictive control. In L. Biegler, O. Ghattas, M. Heinkenschloss, D. Keyes, and B. van Bloemen Waanders, editors, Real-Time and Online PDE-Constrained Optimization. SIAM, 2004. (submitted). [22] M. Diehl, R. Findeisen, F. Allg¨ ower, H.G. Bock, and J.P. Schl¨ oder. Nominal stability of the real-time iteration scheme for nonlinear model predictive control. IEE Proc.-Control Theory Appl., 152(3):296–308, 2005. [23] M. Diehl, R. Findeisen, S. Schwarzkopf, I. Uslu, F. Allg¨ ower, H.G. Bock, E.D. Gilles, and J.P. Schl¨ oder. An eﬃcient algorithm for nonlinear model predictive control of large-scale systems. Part II: Application to a distillation column. Automatisierungstechnik, 51(1):22–29, 2003. [24] M. Diehl, D.B. Leineweber, A.A.S. Sch¨ afer, H.G. Bock, and J.P. Schl¨ oder. Optimization of multiple-fraction batch distillation with recycled waste cuts. AIChE Journal, 48(12):2869–2874, 2002. [25] M. Diehl, L. Magni, and G. De Nicolao. Online NMPC of unstable periodic systems using approximate inﬁnite horizon closed loop costing. Annual Reviews in Control, 28:37–45, 2004. [26] R. Featherstone and D. Orin. Robot dynamics: Equations and algorithms. In Proceedings of the IEEE International Conference on Robotics & Automation, 2000. [27] H.J. Ferreau, G. Lorini, and M. Diehl. Fast nonlinear model predictive control of gasoline engines. In CCA Conference Munich, 2006. (submitted). [28] R. Findeisen, M. Diehl, I. Uslu, S. Schwarzkopf, F. Allg¨ ower, H.G. Bock, J.P. Schl¨ oder, and E.D. Gilles. Computation and performance assesment of nonlinear model predictive control. In Proc. 42th IEEE Conf. Decision Contr., pp. 4613–4618, Las Vegas, USA, 2002. [29] INRIA. Humans toolbox. http://www.inrialpes.fr/bipop/software/humans/, Feb 2005. [30] W. Khalil and D. Creusot. Symoro+: A system for the symbolic modelling of robots. Robotica, 15:153–161, 1997. [31] D. Kraft. On converting optimal control problems into nonlinear programming problems. In K. Schittkowski, editor, Computational Mathematical Programming, vol. F15 of NATO ASI, pp. 261–280. Springer, 1985. [32] D.B. Leineweber. Eﬃcient reduced SQP methods for the optimization of chemical processes described by large sparse DAE models, vol. 613 of FortschrittBerichte VDI Reihe 3, Verfahrenstechnik. VDI Verlag, D¨ usseldorf, 1999. [33] D.B. Leineweber, A.A.S. Sch¨ afer, H.G. Bock, and J.P. Schl¨ oder. An eﬃcient multiple shooting based reduced SQP strategy for large-scale dynamic process

92

[34] [35]

[36]

[37] [38]

[39] [40] [41] [42]

[43] [44]

[45]

[46]

[47]

[48] [49] [50]

[51]

M. Diehl et al. optimization. Part II: Software aspects and applications. Computers and Chemical Engineering, 27:167–174, 2003. P.L. Lions. Generalized Solutions of Hamilton-Jacobi Equations. Pittman, 1982. L. Magni, G. De Nicolao, L. Magnani, and R. Scattolini. A stabilizing modelbased predictive control for nonlinear systems. Automatica, 37(9):1351–1362, 2001. D.Q. Mayne. Nonlinear model predictive control: Challenges and opportunities. In F. Allg¨ ower and A. Zheng, editors, Nonlinear Predictive Control, vol. 26 of Progress in Systems Theory, pp. 23–44, Basel Boston Berlin, 2000. Birkh¨ auser. D.Q. Mayne and H. Michalska. Receding horizon control of nonlinear systems. IEEE Transactions on Automatic Control, 35(7):814–824, 1990. G. de Nicolao, L. Magni, and R. Scattolini. Stability and robustness of nonlinear receding horizon control. In F. Allg¨ ower and A. Zheng, editors, Nonlinear Predictive Control, vol. 26 of Progress in Systems Theory, pp. 3–23, Basel Boston Berlin, 2000. Birkh¨ auser. J. Nocedal and S.J. Wright. Numerical Optimization. Springer, Heidelberg, 1999. L.S. Pontryagin, V.G. Boltyanski, R.V. Gamkrelidze, and E.F. Miscenko. The Mathematical Theory of Optimal Processes. Wiley, Chichester, 1962. PSE. gPROMS Advanced User’s Manual. London, 2000. S.J. Qin and T.A. Badgwell. Review of nonlinear model predictive control applications. In B. Kouvaritakis and M. Cannon, editors, Nonlinear model predictive control: theory and application, pp. 3–32, London, 2001. The Institute of Electrical Engineers. L.O. Santos. Multivariable Predictive Control of Nonlinear Chemical Processes. PhD thesis, Universidade de Coimbra, 2000. L.O. Santos, P. Afonso, J. Castro, N.M.C. de Oliveira, and L.T. Biegler. On-line implementation of nonlinear MPC: An experimental case study. In ADCHEM2000 – International Symposium on Advanced Control of Chemical Processes, vol. 2, pp. 731–736, Pisa, 2000. R.W.H. Sargent and G.R. Sullivan. The development of an eﬃcient optimal control package. In J. Stoer, editor, Proceedings of the 8th IFIP Conference on Optimization Techniques (1977), Part 2, Heidelberg, 1978. Springer. A. Sch¨ afer, P. K¨ uhl, M. Diehl, J.P. Schl¨ oder, and H.G. Bock. Fast reduced multiple shooting methods for nonlinear model predictive control. Chemical Engineering and Processing, 2006. (submitted). V.H. Schulz. Reduced SQP methods for large-scale optimal control problems in DAE with application to path planning problems for satellite mounted robots. PhD thesis, Universit¨ at Heidelberg, 1996. V.H. Schulz. Solving discretized optimization problems by partially reduced SQP methods. Computing and Visualization in Science, 1:83–96, 1998. G.A. Sohl. Optimal Dynamic Motion Planning for Underactuated Robots. PhD thesis, University of California, 2000. O. von Stryk. Numerical solution of optimal control problems by direct collocation. In Optimal Control: Calculus of Variations, Optimal Control Theory and Numerical Methods, vol. 129. Bulirsch et al., 1993. M.J. Tenny, S.J. Wright, and J.B. Rawlings. Nonlinear model predictive control via feasibility-perturbed sequential quadratic programming. Computational Optimization and Applications, 28(1):87–121, April 2004.

Fast Optimal Robot Control

93

[52] S. Terwen, M. Back, and V. Krebs. Predictive powertrain control for heavy duty trucks. In Proceedings of IFAC Symposium in Advances in Automotive Control, pp. 451–457, Salerno, Italy, 2004. [53] A. Toumi. Optimaler Betrieb und Regelung von Simulated Moving Bed Prozessen. PhD thesis, Fachbereich Bio und Chemieingenieurwesen, Universit¨ at Dortmund, 2004. [54] Oskar von Stryck. Optimal control of multibody systems in minimal coordinates. Zeitschrift fur Angewandte Mathematik und Mechanik 78, Suppl 3, 1998. [55] P.-B. Wieber. Mod´elisation et Commande d’un Robot Marcheur Anthropomorphe. PhD thesis, Ecole des Mines de Paris, 2000. [56] P.-B. Wieber. Some comments on the structure of the dynamics of articulated motion. In Proceedings of the Ruperto Carola Symposium on Fast Motion in Biomechanics and Robotics, 2005. [57] J. Zhao, M. Diehl, R. Longman, H.G. Bock, and J.P. Schl¨ oder. Nonlinear model predictive control of robots using real-time optimization. In Proceedings of the AIAA/AAS Astrodynamics Conference, Providence, RI, August 2004.

Stability Analysis of Bipedal Walking with Control or Monitoring of the Center of Pressure D. Djoudi and C. Chevallereau IRCCyN, Ecole Centrale de Nantes, Universit´e de Nantes, UMR CNRS 6597 BP 92101, 1 rue de la No¨e, 44321 Nantes cedex 3, France {Dalila.Djoudi,Christine.Chevallereau}@irccyn.ec-nantes.fr Summary. The objective of this study is to analyze the stability of two control strategies for a planar biped robot. The unexpected rotation of the supporting foot is avoided via the control of the center of pressure or CoP. For the simultaneous control of the joints and of the CoP, the system is under-actuated in the sense that the number of inputs is less than the number of outputs. Thus a control strategy developed for planar robot without actuated ankles can be used in this context. The control law is deﬁned in such a way that only the geometric evolution of the biped conﬁguration is controlled, but not the temporal evolution. The temporal evolution during the geometric tracking is completely deﬁned and can be analyzed through the study of a model with one degree of freedom. Simple conditions, which guarantee the existence of a cyclic motion and the convergence toward this motion, are deduced. These results are illustrated with some simulation results. In the ﬁrst control strategy, the position of the CoP is tracked precisely, in the second one, only the limits on the CoP position are used to speed-up the convergence to the cyclic motion.

1 Introduction The control of many walking robots is based on the notion of center of pressure CoP [11, 12] also called ZMP by Vukobratovic and his co-workers [14, 13]. As long as the CoP remains inside the convex hull of the foot-support, the supporting foot does not rotate and the contact with the ground is guaranteed. Control strategies are often decomposed into a low level and a high level. The low level ensures the tracking of the reference motion, and the high level modiﬁes the reference motion in order to ensure that the CoP remains inside the convex hull of the foot-support. Since the respect of the expected condition of contact with the ground is more important than a tracking error, this kind of control strategy is interesting. In many experimental works, how to modify the reference motion is not

96

D. Djoudi and C. Chevallereau

detailed [11], and it seems that this point has not been studied theoretically. The modiﬁcation of the reference motion has obviously an important eﬀect on the stability of the walking (in the sense of the convergence toward a cyclic motion) and its robustness (in the sense of the reaction of the robot in the presence of perturbation). Our control strategy is based on simultaneous control of the joint and on the evolution of the CoP position. The unexpected rotation of the supporting foot is avoided via the control of the position of the center of pressure. Since the joints and the position of the CoP are controlled simultaneously, the system becomes under-actuated in the sense that the number of inputs is less than the number of outputs. Thus a control strategy developed for planar robots without actuated ankles can be used in this context [9, 2, 4]. An extension of the work of Westervelt et al. [15], for the completely actuated robot based on a prescribed evolution of the ankle torque was proposed [5]. In the proposed study, the position of the CoP is prescribed, not the ankle torque. The control law is deﬁned in such a way that only the geometric evolution of the biped conﬁguration is controlled, but not the temporal evolution. This strategy can be seen as an on-line modiﬁcation of the joint reference motion with respect to time in order to ensure that the position of the center of pressure will be satisfying. The modiﬁcation of the reference motion corresponds to determine the acceleration along a given path1 in the joint space. This modiﬁcation is interesting in the presence of impact, because for all the possible reference motions, the conﬁguration of the robot at impact is the same, and the set of all the reference motions is invariant with respect to impact. As a consequence the impact phase, and the possible variation of the instant of impact have no disturbing eﬀect [3]. Assuming a perfect robot model, and without perturbation, the temporal evolution during the geometric tracking is completely deﬁned and can be analyzed through the study of a model with one degree of freedom. The Poincar´e return map can be used to study the stability of the proposed control law. The practical constraints on the position of the CoP do not imply that this point follows exactly a desired path, but that the position of the CoP evolves between some limits. Thus a second control law is proposed to speed up the convergence to the cyclic motion. The position of the CoP is no longer controlled but only monitored to avoid the rotation of the supporting foot [3]. In this case the control strategy is based on a heuristics proposed by Wieber [16]. In this paper a stability study of this control law is proposed. Section 2 presents the dynamic model of the biped. A planar biped model with massless feet is considered. Section 3 is devoted to the formulation of the ﬁrst control strategy and to the existence of a cyclic motion. In Sect. 4 we present the second control strategy. Some simulation results are presented in Sect. 5 in the case of a precise modeling of the robot and in Sect. 6 in the

1

The time evolution is not speciﬁed for a path.

Stability Analysis of Bipedal Walking with Control of the CoP

97

case of an imprecise modeling. Some properties of the two control strategies are given in Sect. 7. Section 8 concludes the paper.

2 Bipedal Model 2.1 The Biped The biped under study walks in a vertical sagittal xz plane. It is composed of a torso and two identical legs. Each leg consists of two segments and a massless foot. The ankles, the knees and the hips are one-degree-of-freedom rotational frictionless joints. The walk is considered as single support phases separated by impacts (instantaneous double support) with a full contact between the sole of the feet and the ground. The angle of the supporting knee is denoted q1 . The angle of the supporting hip is denoted q2 . The angle of the swing hip is denoted q3 . The angle of the swing knee is denoted q4 . During swing phase the foot is aligned horizontally thus the angle of the swing ankle can be calculated. The supporting ankle angle allows to choose the orientation of the supporting shank with respect to the vertical q5 . Vector q = [q1 , q2 , q3 , q4 , q5 ]T of variables (Fig. 1) describes the shape of the biped during single support. Since the free foot is massless no torque is required at the swing ankle. The torque at the supporting ankle will be treated in a special way thus it is denoted Γa = Γ5 . The torques are grouped into a torque vector Γ = [Γ1 , Γ2 , Γ3 , Γ4 , Γ5 ]T .

Z

CoM

Zg

q2 q 4

q5

q3 q1

Y X

Xg Fig. 1. The studied biped: generalized coordinates

98

D. Djoudi and C. Chevallereau

In the simulation, we use the following biped parameters [2]. The lengths of the thighs and of the shanks are 0.4 m. However, their masses are diﬀerent: 6.8 kg for each thigh and 3.2 kg for each shank. The length of the torso is 0.625 m and its mass is 17.05 kg. The center of mass are placed on the line representing the link in Fig. 1. The distance between the joint actuator and the center of mass is 0.1434 m for the torso, 0.163 m for the shanks, and 0.123 m for the thigh. The moments of inertia of the segments are also taken into account, there values are deﬁned around the joint axis, and there value are 1.8694 kgm2 for the torso, 0.10 kgm2 for the shanks, and 0.25 kgm2 for the thigh. The inertia of the motor of the hip and of the knee are 0.83 kgm2 . The feet is massless and have no inertia. The size of the feet are hp = 0.08 m, lmin = 0.06 m, lmax = 0.2 m (Fig. 2). 2.2 Dynamic Modeling The walking gait is composed of stance phases. A passive impact separates the stance phases. The legs swap their roles from one step to the next one. Thus the study of a step allows us to deduce the complete behavior of the robot. Only a single support phase and an impact model are derived. The Single Support Phase Model Using Lagrange’s formalism, the ith line of the dynamic model for i = 1, . . . , 5 (qi is the ith element of vector q) is: d ∂K ∂P ∂K + = Qi (1) − dt ∂ q˙i ∂qi ∂qi

Fig. 2. The equilibrium of the supporting foot

Stability Analysis of Bipedal Walking with Control of the CoP

99

where K is the kinetic energy and P is the potential energy. The virtual - work δW of the external torques and forces, given by expression δW = Qi δqi = QT δq, deﬁnes the vector Q of the generalized forces. The kinetic energy K is independent of the coordinate frame chosen. Since coordinate q5 deﬁnes the orientation of the biped as a rigid body, the inertia matrix is independent of this variable, it depends only of “internal” variables represented by vector qc = [q1 , q2 , q3 , q4 ]T . The dynamic model can be written: q + h(q, q) ˙ =Γ M (qc )¨

(2)

˙ contains the cenwhere M (qc ) is a (5 × 5) inertia matrix and vector h(q, q) trifugal, Coriolis and gravity forces. The ﬁfth equation of system (1) is: d ∂K ∂P = Γa (3) + dt ∂ q˙5 ∂q5 For our planar biped and our choice of the coordinates in the single support, the term ∂∂K q˙5 is the angular momentum of the biped about the stance ankle A (Fig. 2). We denote this term by σA . Thus we have: ∂K = σA = N (qc )q˙ ∂ q˙5

(4)

where N (qc ) is the ﬁfth line of the inertia matrix M (qc ). ∂P is equal to −mgxg if the abscissa of the stance ankle The expression ∂q 5 is 0, m is the mass of the biped, g is the gravity acceleration. Thus the ﬁfth equation of the dynamic model of the biped in the single support can be written in the following simple form: σ˙ A − mgxg = Γa

(5)

The Reaction Force During the Single Support Phase The position of the mass center of the biped can be expressed as function of the angular coordinates vector q: . / . / fxi (q) xg = (6) fzi (q) zg The vector-function fi (q) = [fxi (q) fzi (q)]T depends on vector q and on the biped parameters (lengths of the links, masses, positions of the centers of mass). The index i denotes the stance leg, for support on leg 1, f1 (q) is used. When leg 1 is on the ground, a ground reaction force, R1 , exists. The global equilibrium of the robot makes it possible to calculate this force:

100

D. Djoudi and C. Chevallereau

. m

x ¨g

/ + mg

z¨g

. / 0 1

= R1

(7)

Equation (7) can also be written: m

∂fx1 (q) ∂ 2 fx1 (q) q¨ + mq˙T q˙ = Rx1 ∂q ∂q 2

∂ 2 fz1 (q) ∂fz1 (q) q¨ + mq˙T m q˙ + mg = Rz1 ∂q ∂q 2 where

∂ 2 fx1 (q) ∂q 2

and

∂ 2 fz1 (q) ∂q 2

(8)

are (5 × 5) matrices.

Equilibrium of the Supporting Foot The supporting foot is exposed to the ground reaction force and the ankle torque −Γa . The equilibrium law gives: −Γa − lRz1 − hp Rx1 = 0

(9)

Thus if the horizontal CoP position is l then the torque at the supporting ankle is, using (7): zg + mg) − hp (m¨ xg ) Γa = −l(m¨

(10)

The horizontal CoP position l is directly deﬁned by the robot dynamics as it can be seen in the following equation obtained by combining equations (4), (5), (6), (8) and (10): q + h0 (q, q) ˙ + lhl (q, q) ˙ =0 (N0 (q) + lNl (q))¨

(11)

with N0 = N (qc ) + mhp

∂fx1 (q) ∂q

∂fz1 (q) ∂q ∂N (qc ) ∂ 2 fx1 (q) q˙ − mgfx1 (q) + mhp q˙T q˙ h0 = q˙T ∂q ∂q 2 ∂ 2 fz1 (q) q˙ + mg hl = mq˙T ∂q 2

Nl = m

The Impact Model When the swing leg 2 touches the ground with a ﬂat foot at the end of the single support of leg 1, an inelastic impact takes place. We assume that the

Stability Analysis of Bipedal Walking with Control of the CoP

101

ground reaction force at the instant of impact is described by a Dirac deltafunction with intensity IR2 . The velocity of foot 2 becomes zero just after the impact. Two kinds of impact can occur depending on whether the stance leg takes oﬀ or not. Here, for simplicity, we study walking with instantaneous double support. Thus at impact the stance leg 1 takes oﬀ and there is no impulsive ground reaction force on leg 1. The robot conﬁguration q is assumed to be constant at the instant of impact, and there are jumps in the velocities. The velocity vectors just before and just after impact, are denoted q˙− and q˙+ respectively. The torques are limited, thus they do not inﬂuence the instantaneous double support. It can be shown that the impact model can be written as [4]: (12) q˙+ = E((q)q˙− ) where (q) is a 5 × 5 matrix, and E is a permutation function describing the legs exchange. For the following single support phase the joints are relabelled in order to study only one dynamic model for single support (SS) and to take into account the change on the supporting ankle. Intensity IR2 of the impulsive reaction force is: ∂f2 (q) ∂f1 (q) IR2 = m (q) − (13) q˙− ∂q ∂q

3 The First Control Law In this study, walking is considered as single support phases with a full foot contact. While this is not a necessary condition for walking, and animals and humans do not enforce this constraint during walking, many control algorithms for bipedal robots enforce this constraint during walking in order to prevent diﬃculties associated with the loss of actuation authority when the foot rotates. To avoid foot rotation, the CoP must be inside the supporting area [13]. In order to ensure this behavior, the CoP position is controlled to follow a desired path ld [11], but as shown in the previous section, the position of the CoP is directly connected to the dynamics of the motion. It is not possible to prescribe independently a desired evolution of the joints q d (t) and of the position of the CoP ld (t). With respect to such a task, the biped can be seen as an under-actuated system, and the control strategy developed for such a system can be used. Thus, the objective of the control law presented in this section is only to track a reference path for q and l rather than a reference motion [4]. A motion diﬀers from a path by the fact that a motion is a temporal evolution along a path. A joint path is the projection of a joint motion in the joint space. The diﬀerence between motion and path are illustrated on Fig. 3 for a two joint-robot. Only a geometrical tracking is desired and a time scaling control [6] is used. A reference joint path is assumed to be known. Thus the desired conﬁguration of q and l for the biped are not expressed as a function of time. But they are

102

D. Djoudi and C. Chevallereau

1 0.9 0.8 0.7

t

0.6 0.5 0.4 0.3 s=1

0.2

0.8 0.6 0.4 0.2

0.1 0

s=0 0.6

0.7

0.8

0.9

0 1

q2

q1

Fig. 3. The dotted lines are two motions (q1 (t), q2 (t)) corresponding to the same path represented by the solid line. A path is a line in the joint space, this line can be graduated as a function of a new variable denoted s, and then can be expressed by (q1 (s), q2 (s)). This function s is deﬁned such that the initial conﬁguration correspond to s = 0, the ﬁnal conﬁguration corresponds to s = 1. Any monotonic function s(t) deﬁnes a motion corresponding to the path q(s). For example s = t/T deﬁnes a motion of duration T . If a joint variable, for example q2 , has a monotonic evolution along the path, the path can also be parametrized by q2 , in this case it can be expressed as q1 (q2 )

function of the scalar path parameter s, a normalized virtual time: q d (s), ld (s). The desired walking of the robot corresponds to an increasing function s(t). This function. s(t) is not known a priori, the set of all the motions that correspond to the desired path is considered. The proposed strategy can be extended without diﬃculty to walking including a rotation about the toe of the supporting foot, since this phase corresponds to a motion such that the position of the center of pressure is imposed. The main diﬃculty is that a sub-phase must be added [5]. 3.1 Choice of a Reference Path The reference path q d (s), ld (s) is designed in order to obtain cyclic motion of the biped. The walk is composed of single supports separated by instantaneous passive impacts. The legs swap their roles from one step to the next one, so the reference path needs only to be deﬁned for a single step. The evolution of the path parameter s along the step k is denoted sk (t), the scalar path

Stability Analysis of Bipedal Walking with Control of the CoP

103

parameter sk must increase strictly monotonically with respect to time from 0 to 1. For 0 < sk (t) < 1, the robot conﬁguration q d (sk ) is such that the swing leg is above the ground. The swing leg touches the ground at sk = 0, 1 exactly. In consequence for any increasing function sk (t) from 0 to 1, the conﬁguration of the biped at impact is the expected one. The control inputs are the torques. The torque acts on the second derivative of q and directly on l via the dynamic model. Thus the reference trajectory q d (sk ) must be twice diﬀerentiable, but no continuity condition exists for ld (sk ). Vectors q d (0) and q d (1) describe the initial and ﬁnal biped conﬁgurations of the biped during a single support. As the legs swap their roles from one step to the following one the desired conﬁgurations are such that q d (1) = E(q d (0)) where E is a permutation function describing the leg exchange. The reference path is deﬁned such that if the reference path is exactly tracked before the impact then the reference path is exactly tracked after the impact. If the reference path is perfectly tracked, before the impact k + 1, d the vector of joint velocities is q˙− = dqds(1) s˙ k (1) and after the impact, q˙+ = dq d (0) ds s˙ k+1 (0).

The velocity at the end and at the beginning of the step are connected by the impact model and the legs exchange (12). Thus we have:

We choose:

dq d (0) dq d (1) s˙ k+1 (0) = E((q d (1)) s˙ k (1)) ds ds

(14)

dq d (0) dq d (1) = E((q d (1)) α) ds ds

(15)

With this choice we have the following equality: s˙ k+1 (0) = For conﬁguration q d (1), and vector dq d (0) ds

d

dq (1) ds

s˙ k (1) α .

the amplitude of the vector

can be modiﬁed by the choice the values of α (but not its direction). This point will be commented in Sect. 5.1. Some hypotheses (no sliding, no rotation of the feet, take-oﬀ of the previous supporting leg) are made on the behavior of the robot at the impact, the corresponding constraints on the joint trajectory can be deduced [4, 7]. 3.2 Deﬁnition of the Control Law The control law must ensure that the joint coordinates follow the joint reference path q d (s) and that the position of the CoP is ld (s). It follows from the deﬁnition of the joints reference path that the desired velocity and acceleration of the joint variables are: dq d (s(t)) s˙ ds d2 q d (s(t)) 2 dq d (s(t)) q¨d (t) = s¨ + s˙ ds ds2 q˙d (t) =

(16)

104

D. Djoudi and C. Chevallereau

The increasing function s(t) deﬁnes the desired motion, but since the control objective is only to track a reference path, the evolution s(t) is free and the second derivative s¨ will be treated as a “supplementary control input”. Thus, the control law will be designed for a system with equal number of inputs and outputs. The control inputs are the ﬁve torques Γj , j = 1, . . . , 5, plus s¨. The chosen outputs are the ﬁve angular variables of vector q(t) − q d (s(t)) and l(t) − ld (s(t)). The control law is a non-linear control law classically used in robotics. But in order to obtain a ﬁnite-time stabilization of one of the desired trajectories, the feedback function proposed by Bhat and Berstein is used [1, 9]. The joint tracking errors are deﬁned with respect to the trajectories satisfying (16): eq (t) = q d (s(t)) − q(t) e˙ q (t) =

dq d (s(t)) s˙ − q(t) ˙ ds

(17)

The desired behavior in closed loop is: q¨ = q¨d +

1 ψ 2

(18)

where ψ is a vector of ﬁve components ψl , l = 1, . . . , 5 with: ψl = −sign(e˙ ql )|e˙ ql |ν − sign(φl )|φl |ν

(19)

1 and 0 < ν < 1, > 0, φl = eql + 2−ν sign(e˙ ql )|e˙ ql |2−ν , ν and are parameters to adjust the settling time of the controller. Taking into account the expression of the reference motion, (18) can be rewritten as:

q¨ =

dq d (s) s¨ + v(s, s, ˙ q, q) ˙ ds

(20)

with

1 d2 q d (s) 2 s˙ + 2 ψ ds2 For the position of the CoP, the desired closed loop behavior is: v(s, s, ˙ q, q) ˙ =

l(t) = ld (s(t)) The dynamic model of the robot is described by eq. (2). The position of the CoP is deﬁned via (11). Thus the control law must be such that: dq d (s) s¨ + v) + h(q, q) ˙ =Γ ds dq d (s) (N0 (q) + ld (s)Nl (q))( s¨ + v) + h0 (q, q) ˙ + ld (s)hl (q, q) ˙ =0 ds M (q)(

(21)

Stability Analysis of Bipedal Walking with Control of the CoP

105

We can deduce that, in order to obtain the desired closed loop behavior, it is necessary and suﬃcient to choose: s¨ =

−(N0 (q) + ld (s)Nl (q))v − h0 (q, q) ˙ − ld (s)hl (q, q) ˙ d

(N0 (q) + ld (s)Nl (q)) dqds(s) Γ = M (q)

dq d (s) s¨ + v + h(q, q) ˙ ds

(22)

(23)

d

If (N0 (q) + ld (s)Nl (q)) dqds(s) = 0, the control law (22)–(23) ensures that q(t) converges to q d (s(t)) in a ﬁnite time, which can be chosen as less than the duration of one step [1, 9], and that l(t) = ld (s(t)). Without initial errors, a perfect tracking of q d (s(t)) and ld (s) is obtained. 3.3 Stability Study Our main goal is to design a control strategy that ensures a stable periodic motion of the biped. The control law (22)–(23) ensures that the motion of the biped converges in a ﬁnite time toward a reference path. The settling time can be chosen to be less than the duration of the ﬁrst step. Since the impact is a geometric condition and due to the characteristics of the joints reference path (Sect. 3.1), any step k begins with sk = 0 and ﬁnishes with sk = 1. Since the control law is designed to converge before the end of the ﬁrst step and since the reference path is such that if the tracking is perfect before the impact, it will be perfect afterward, after the ﬁrst step a perfect tracking is obtained. The biped with control law (22)–(23) follows perfectly the reference path, starting from the second step. Thus: q(t) = q d (s(t)) dq d (s) s(t) ˙ ds dq d (s) d2 q d (s) q¨(t) = s¨(t) + s(t) ˙ 2 ds ds2 q(t) ˙ =

(24)

l(t) = ld (s(t)) These equations deﬁne the zero dynamics corresponding to the proposed control law. To know whether a cyclic motion will be obtained, the behavior of the evolution of s˙ k (t) is studied for k = 2 . . . ∞. The dynamics of s is deduced from the dynamic model (11) with the condition (24). The acceleration s¨ is: (Ns0 (s) + ld (s)Nsl (s))¨ s + hs0 (s, s) ˙ + ld (s)hsl (s, s) ˙ =0 with

(25)

106

D. Djoudi and C. Chevallereau

Ns0 = (N (q d (s)) + mhp

∂fx1 (q) dq d (s) ) ∂q ds

∂fz1 (q) dq d (s) Nsl = m ∂q ds / . T ∂N (q) ∂ 2 fx1 (q) dq d (s) 2 dq d (s) + mhp s˙ hs0 = ds ∂q ∂q 2 ds % & ∂fx1 (q) d2 q d (s) 2 d + N (q (s)) + mhp s˙ ∂q ds2

(26)

− mgfx1 (q d (s)) . / T dq d (s) ∂ 2 fz1 (q) dq ∂fz1 (q) d2 q d (s) 2 + hsl = m s˙ + mg ds ∂q 2 ds ∂q ds2 This equation along with the constraints (24) describe completely the behavior of the system. One single support phase begins with s = 0 and ﬁnishes with s = 1. The evolution of s˙ k+1 during the step k + 1 is uniquely deﬁned by initial value s˙ k+1 (0). The integration of (25) along one step, starting with s˙ k+1 (0), deﬁnes the ﬁnal value s˙ k+1 (1). The single support phases are separated by impact phases; the evolution of the zero dynamics is such that s restarts with s = 0 and s˙ k+1 (0) = s˙ kα(1) (due to the deﬁnition of the reference joint path (15)). Thus the ﬁnal value of s˙ k+1 (1) can be easily deﬁned numerically as a function of s˙ k (1), we deﬁne function ϕ by: s˙ k+1 (1) = ϕ(s˙ k (1)). The existence of a cyclic motion and the convergence to it can be studied via function ϕ as it is classically done using the method introduced by H. Poincar´e [9, 10]. The ﬁxed point of this function deﬁnes the cyclic velocity s˙ c (1), it corresponds to the intersection between the function ϕ and the identity function. If the norm ∆ of the slope of the function ϕ at s˙ c (1) is less than 1, then for an initial state close to the cyclic motion, the biped motion will converge toward the cyclic motion. If the desired evolution of the position of the CoP is piecewise constant, the stability analysis can be conducted mostly analytically [7]. If the desired evolution of the position of the CoP is arbitrary, the stability analysis is conducted numerically in this chapter.

4 The Second Control Law The physical constraint on the position of the CoP is that the position of the CoP is between lmin and lmax but it is not necessary that l(s) follows exactly ld (s). If a cyclic motion corresponding to q d (s), ld (s) exists, it can be interesting to converge quickly toward this cyclic motion deﬁned by s(t) ˙ = s˙ c (t). The corresponding cyclic motion can be deﬁned by the stability study

Stability Analysis of Bipedal Walking with Control of the CoP

107

of the ﬁrst control law. Now we assume that the corresponding cyclic motion is given as a function s˙ c (s) for 0 ≤ s ≤ 1. To achieve this objective, the constraint l(s) = ld (s) can be relaxed to: lmin < l(s) < lmax . To converge toward the cyclic orbit in the phase plane s, s, ˙ we deﬁne an error between the current state and the orbit: ev = s(s) ˙ − s˙ c (s)

(27)

and to nullify this error the desired acceleration s¨d is chosen such that: e˙ v + Kvs ev = 0 where Kvs deﬁnes the convergence rate to the cyclic motion. Thus the desired acceleration is: s¨(s)d =

d(s˙ c (s)) s˙ + Kvs (s˙ c (s) − s(s)) ˙ ds

(28)

But the position l of the CoP, and the acceleration s¨ are linked by the dynamic model. And even if the constraint on l is relaxed, the condition of non-rotation of the feet holds, and l is monitored to be within the domain =]lmin , lmax [ in all the control process. If the same closed loop behavior is desired for the joint variables (22), gives: s¨ =

˙ + l(Nl (q)v − hl (q, q)) ˙ −N0 (q)v − h0 (q, q) d

d

N0 (q) dqds(s) + lNl (q) dqds(s)

(29)

where l must be chosen such that l ∈ . Diﬀerentiating (29) with respect to l shows that s¨ is monotonic with respect to l. Thus the limits lmin < l < lmax can be easily transformed with limits on s¨. For this purpose, the extreme values for s¨ are deﬁned as follows: u1 =

u2 =

˙ + lmin (Nl (q)v − hl (q, q)) ˙ −N0 (q)v − h0 (q, q) d

d

N0 (q) dqds(s) + lmin Nl (q) dqds(s) ˙ + lmax (Nl (q)v − hl (q, q)) ˙ −N0 (q)v − h0 (q, q) d

(30)

d

N0 (q) dqds(s) + lmax Nl (q) dqds(s)

For given values of s, s, ˙ two cases occur depending on whether the denominator can be zero or not for l ∈ . The denominator is zero for d l(s) = −

dq (s) ds dq d (s) Nl (q) ds

N0 (q)

. If for any l such that l ∈ , the denominator of

eq. (29) is not zero, then s¨ is bounded for any acceptable value l and min(u1 , u2 ) < s¨ < max(u1 , u2 ). If for one value l such that l ∈ , the denominator of eq. (29) is zero, then s¨ is unbounded and s¨ cannot be in the interval ] min(u1 , u2 ), max(u1 , u2 )[ with acceptable values of l. Thus the proposed control law is the following: like the previous control law, the reference path q d (s) is tracked using eq. (23) but eq. (22), which corresponds to the constraint l(s) = ld (s), is replaced by the following:

108

D. Djoudi and C. Chevallereau

if −

dq d (s) ds dq d (s) Nl (q) ds

N0 (q)

< lmin or −

dq d (s) ds dq d (s) Nl (q) ds

N0 (q)

> lmax then

 min(u1 , u2 ), if s¨d < min(u1 , u2 )   d s¨ = max(u1 , u2 ), if s¨ > max(u1 , u2 )   s¨d , otherwise if lmin < −

dq d (s) ds dq d (s) Nl (q) ds

N0 (q)

(31)

< lmax then

 min(u1 , u2 ), if min(u1 , u2 ) < s¨d ≤ u12   d s¨ = max(u1 , u2 ), if u12 < s¨ < max(u1 , u2 )   s¨d , otherwise 1 ,u2 ) . This control law ensures a convergence to where u12 = min(u1 ,u2 )+max(u 2 the cyclic motion with a convergence rate deﬁned by Kvs under the constraint l ∈ . The control law (31), (23) ensures that q(t) converges to q d (s(t)) in a ﬁnite time, which can be chosen less than the duration of one step [1, 9], and ensures that l ∈ . The biped with control law (31), (23) follows perfectly the reference path after this ﬁrst step. To know if a cyclic motion will be obtained, the behavior of the evolution of s˙ k (t) is studied for k = 2 . . . ∞ and for an initial velocity s˙ 2 (0). The stability analysis is done numerically like for the ﬁrst control law. The convergence rate to the cyclic motion depends on the choice of the value Kvs . Higher values of Kvs speed up convergence toward the cyclic motion if there is no saturation due to the limits on l.

5 Walking Simulation using Correct Model Parameters 5.1 A Reference Path The proposed control law was tested on the reference path corresponding to the stick-diagram presented in Fig. 4, for the biped presented in Fig. 1. The joint path q d (s) is deﬁned by a fourth order polynomial evolution with respect to s. This reference path has been deﬁned to produce an optimal cyclic motion for the robot Rabbit [2], this robot has the same physical property that the robot described in Sect. 2 but Rabbit has no feet (hp = 0, l = 0). As the studied robot has feet, and a linear evolution of the position of the CoP is considered, the existence of a cyclic motion is not ensured and if it exists it is of course not optimal. For the robot without feet, the optimization process is described in [8]. The reference path is described by an instantaneous double support conﬁguration

Stability Analysis of Bipedal Walking with Control of the CoP

109

Fig. 4. The stick diagram of the desired trajectory. The conﬁguration of the robot are drawn for s = 0, 0.05, 0.1, 0.15 . . . , 0.95, 1. Thus a sequence of pictures of the robot are given. The desired motions of the robot are such that the conﬁguration of the robot coincides at some instant to each picture, but it is not imposed that these instants are equally distributed in the period of one step d

(1) q d (1), the ﬁnal direction of the joint velocity q ds , an intermediate single support conﬁguration q d (0.5), and α. The initial double support conﬁguration is deﬁned by permutation: q d (0) = Eq d (1). The direction of the initial velocity is deﬁned by equation (15). Then the desired path is determined by a polynomial 4th order function of s connecting these conﬁgurations and velocities. The integral of the norm of the torque for the cyclic motion is minimized for a given advance velocity. The free leg tip must be above a sinusoidal function with a maximum of 5 cm. The limit of the actuator are taken into account (maximal torque less than 150 Nm). The reference path corresponding to the Fig. (4) is obtained for given advance velocity vel = 1.41 ms−1 . The optimal solution is such that: q d (1) = [5.73◦ 185.08◦ 40.43◦ 133.33◦ 25.81◦ ]T , q d (1) = [3.57◦ s−1 32.60◦ s−1 − 61.60◦ s−1 0.09◦ s−1 29.50◦ s−1 ]T , q d (0.5) = ds ◦ [19.97 161.22◦ 42.51◦ 154.93◦ 17.72◦ ]T and α = 1.98

5.2 The First Control Law For this joints path, a linear evolution of the CoP position is chosen. When s varies from 0 to 1, ld varies from −0.06 m to 0.18 m. The control law imposes that q(s) = q d (s), l(s) = ld (s) after the ﬁrst step. The stability of the complete system is determined by the evolution of s(t). It

110

D. Djoudi and C. Chevallereau 6

5

4

s˙

3

2

1

s

0

−1 0

0.2

0.4

0.6

0.8

1

Fig. 5. The phase plane for the zero dynamics (25) on single support: s˙ against s

can be described in a phase plane s, s˙ for 0 ≤ s ≤ 1 initialized with diﬀerent values for s. ˙ For example, the phase plane is shown in Fig. 5. For a suﬃciently high initial velocity s(0), ˙ successful stepping pattern can be achieved. At low initial velocity s(0), ˙ the robot falls back. Taking the impact phase into account (here α = 1.98), the Poincar´e return map s˙ k+1 (1) = ϕ(s˙ k (1)) is drawn in Fig. 6. For the example the cyclic motion is such that s˙ c (1) = 3.9 s−1 . The corresponding average motion velocity is vel = 1.5 m/s. The slope of function ϕ is estimated numerically: ∆ = 0.68; it is less than 1, thus the proposed control law is stable. The minimal value and the maximal value of the velocity s˙ k (1) such that the step can be achieved are deﬁned numerically. For smaller initial velocities the biped falls back, for higher velocities the biped takes oﬀ since the normal ground reaction vanishes. Assuming no modeling error and initializing the state of the biped out of the periodic orbit (with an initial velocity 60% higher than the cyclic value), the results of one simulation for 20 walking steps are illustrated in Fig. 7. The convergence toward a cyclic motion can be shown for the ﬁve joints via their evolution in their phase plane. For example the evolution of the trunk is shown in Fig. 7-a. This convergence is also illustrated via the evolution of the position of the CoP with respect to time in Fig. 7-b. For each step, this evolution is linear from −0.06 m to 0.18 m, but the duration of the step varies. At the beginning;, the steps are faster and then a cyclic behavior is obtained. Figure 7-c presents the time-history of s, ˙ it clearly converges toward a cyclic motion, the ﬁnal value of s˙ before each impact is the cyclic value obtained on the Poincar´e map.

Stability Analysis of Bipedal Walking with Control of the CoP

111

8

s˙ k+1(1)

6

4 cyclic motion

2

0 0

1

2

3

4

5

6

7

8

s˙ k (1)

q˙5 − q˙1 − q˙2

Fig. 6. The Poincar´e map: s˙ k+1 (1) = ϕ(s˙ k (1)) Trunk evolution in its phase plane 0.5

0

q5 − q1 − q2 + π

−0.5 0.26

0.27

0.28

0.29

(a) 6

0.1

s˙

l

4

0

−0.1 0

t [s]

2

4 (b)

6

2 0

2

4

6

t [s]

(c)

Fig. 7. The convergence toward a cyclic motion is observed in simulation with the proposed control law, without modeling error. (a) The trunk evolution is drawn in its phase phase (the absolute trunk velocity with respect to the absolute trunk orientation), at impact the velocity changes but not the orientation. It tends toward a limit cycle. (b) During each step, the horizontal position of the CoP with respect to time l(t) evolves from −0.06 m to 0.18 m. The duration of the step tends toward a constant value. (c) s(t) ˙ tends toward a cyclic motion

112

D. Djoudi and C. Chevallereau 6

5

4

s˙

3

2

1

s

0

−1 0

0.2

0.4

0.6

0.8

1

Fig. 8. The phase plane for the zero dynamics on single support, (31), for the second control law with Kvs = 20

5.3 The Second Control Law The second control law was tested on the same reference trajectory q d (s). The desired evolution of s(s) ˙ is the cyclic motion corresponding to the previous control law. The control law imposes some constraints q(s) = q d (s) that are assumed to be perfectly respected. The free dynamics that results from imposing these constraints on the system conﬁguration are described by s, s˙ and eq. (31) can be represented in the phase plane. The phase plane is shown in Fig. 8 for Kvs = 20. The convergence toward the cyclic motion is clear when Figs. 5 and 8 are compared. When Kvs = 20, for initial velocities varying from 1.4 to 2.8, the cyclic motion is reached in one step. This feature gives a horizontal behavior of the Poincar´e map about the ﬁxed point. The motion can be initiated with a lower velocity s(0) ˙ than for the ﬁrst control law because when the current motion converges toward the cyclic motion, it helps prevent the biped from falling back. The control strategy is properly illustrated by the evolution of l(s) corresponding to the evolution of the biped for various initial velocities s(0) ˙ in Fig. 9. When the real motion of the biped is slower than the cyclic one, the position of the CoP is moved backwards to increase the motion velocity until the limit lmin is reached. When the real motion of the biped is faster than the cyclic one, the position of the CoP is moved forwards to decrease the motion velocity until the limit lmax is reached. With a high gain, the position of the CoP is on the limit almost all the time The single support phases are separated by impact phases. The Poincar´e return maps can be deduced and are presented in the Fig. 10, for Kvs = 2 and Kvs = 20.

Stability Analysis of Bipedal Walking with Control of the CoP

113

0.25

0.2

0.15

l

0.1

0.05

s

0

−0.05

−0.1 0

0.2

0.4

0.6

0.8

1

Fig. 9. The evolution of the position of the CoP l(s), for various initial velocities s(0), ˙ for the second control law with Kvs = 20 8 7

s˙ k+1(1)

6 5 4 3 2

K =20 vs

Kvs=2

1 0 0

1

2

3

4

5

6

7

8

s˙ k (1)) Fig. 10. The Poincar´e return map for the second control law with Kvs = 2 (solid line) and Kvs = 20 (dotted line), s˙ k+1 (1) is shown against s˙ k (1))

Since this control law is deﬁned to obtain convergence toward the cyclic motion corresponding to the ﬁrst control law, the ﬁxed point of the Poincar´e maps is the same (see Figs. 6, 10). The minimal and maximal values of the velocity s˙ k (1) such that the step can be achieved are deﬁned numerically. It can be noted that the minimal initial velocity is lower for the second control strategy than for the ﬁrst one. With Kvs = 2, at the ﬁxed point the slope is

D. Djoudi and C. Chevallereau

q˙5 − q˙1 − q˙2

114

Trunk evolution in its phase plane 0.4

q5 − q1 − q2 + π

0.2 0

−0.2 −0.4 −0.6 0.26

0.27

0.28

0.29

(a) 0.2

6 5

0.1

s˙

l

4

0

t [s] −0.1 0

2

4 (b)

6

3 2 1 0

2

(c)

4

6

t [s]

Fig. 11. The convergence toward a cyclic motion is observed in simulation with the second control law, with Kvs = 20, without modeling error. (a) The trunk evolution is drawn in its phase phase (the absolute trunk velocity with respect to the absolute trunk orientation), it tends toward a limit cycle. (b) The horizontal position of the CoP with respect to time l(t) is bounded. It tends toward the same cyclic evolution as in Fig. 7(b). (c) s(s) ˙ tends toward to the same cyclic motion as in Fig. 7(c)

about ∆ = 0.23; it is less than the value obtained for the ﬁrst control law thus the convergence is faster. For Kvs = 20 the convergence is so fast that the slope is close to horizontal at the ﬁxed point, in one step the cyclic motion is almost joined. When the initial velocity is far beyond the cyclic one, the constraint on l produces a saturation on s¨ almost all the time, thus almost the same behavior is obtained with Kvs = 2 or Kvs = 20. Assuming no modeling error and initializing the state of the robot out of the periodic orbit (with an initial velocity 60% higher than the cyclic value), the results of one simulation for 20 walking steps of the robot are illustrated in Fig. 11. The convergence toward a cyclic motion can be shown for the trunk via its evolution in its phase plane (Fig. 11-a). In one step the cyclic motion is reached. This convergence is also illustrated via the evolution of the position of the CoP with respect to time (Fig. 11-b). To slow down the motion, for the ﬁrst step, the position of the CoP stays on the front limit (lmax ). After the evolution of the CoP corresponds to the desired cyclic one, it is linear from −0.06 m to 0.18 m. Figure 11-c presents the evolution of s˙ with respect to time, it clearly converges toward the desired cyclic motion.

Stability Analysis of Bipedal Walking with Control of the CoP

115

6 Control of Walking with Imprecise Model Data In practice the robot parameters are not perfectly known. We assume that we have some errors on the masses and consequently on the inertia moments of the robot links. We simulate the following case of errors: • the mass errors are: +10% for the thighs, +30% for the shanks and +50% for the trunk. The error on the inertia moment of the trunk is +30%; • since the reference path is designed with a false model, the velocity after the impact is not the expected one; • as the position l of the CoP is calculated via the dynamic model, l(s) will not be exactly ld (s). This choice of errors is arbitrary. We choose that the real robot is heavier than the model used in the control law, this point is commented. 6.1 The First Control Law

q˙5 − q˙1 − q˙2

Initializing the state of the robot in the same conditions as in 5.2; the behavior obtained for 20 walking steps is presented in Fig. 12. Some tracking errors exist particularly at the beginning of each step due to the eﬀect of impact, thus the path followed is not exactly the expected one (but the tracking errors Trunk evolution in its phase plane 0.5

0

q5 − q1 − q2 + π

−0.5 0.26

0.27

0.28

0.29

(a) 0.2

6 5

s˙

l

0.1

0

t [s]

−0.1 0

5 (b)

4 3 2 0

5

10

15

s

(c)

Fig. 12. The convergence toward a cyclic motion is observed in simulation with the proposed control law, with modeling error. (a) The trunk evolution is drawn in its phase phase (the absolute trunk velocity with respect to the absolute trunk orientation), it tends toward a limit cycle. (b) The horizontal position of the CoP with respect to time l(t) tends toward a cyclic evolution diﬀerent from Fig. 11(b). (c) s(s) ˙ tends toward a cyclic motion diﬀerent from the motion in Fig. 7(c)

116

D. Djoudi and C. Chevallereau

are cyclic). The convergence toward a cyclic motion is shown for the trunk evolution via its phase plane in Fig. 12-a. This convergence is also illustrated via the evolution of s˙ with respect to s in Fig. 12-c, it clearly converges toward a stable cyclic motion. The cyclic motion is close to the expected one but not exactly the same, because it is the result of the motion of the CoP and of the dynamic model. Since the real robot is heavier than the robot’s model used, we have greater ground reaction forces; consequently the real evolution l of the CoP in Fig. 12-b varies between extreme values smaller in absolute value than the desired values. The diﬀerence between l(s) and ld (s) is higher for large value of s. ˙ In this case there is no problem because constraints of equilibrium of the supporting foot are always satisﬁed. Otherwise if the real robot was lighter than the modeled one, the CoP could be outside the sole and the constraints of equilibrium of the supporting foot could be violated. So a security margin is necessary when the minimum and the maximum values for the CoP evolution are deﬁned. The best way is to deﬁne lmin and lmax with some margins with respect to real size of the foot (see Fig. 2). 6.2 The Second Control Law In order to illustrate some robustness properties of the second control law proposed in (Sect. 4), we test the same modeling error as in Sect. 6. • Since the reference path is designed with a false model, the velocity after the impact is not the expected one; • In the case of perfect modeling the control law (31) assumes that the limits on s¨ corresponds to lmin < l < lmax . But this relation is based on the dynamic model, since the dynamic model is not perfectly known, this transformation will induce some errors. A simulation of 20 walking steps is presented in Fig. 13. The biped state is initialized out of the periodic orbit (with an initial velocity 60% higher than the cyclic value). The convergence toward a cyclic motion can be shown via the trunk evolution in its phase plane in Fig. 13-a; some errors can be observed at the impact times. The convergence toward the cyclic motion can be also shown in Fig. 13-b via the evolution of the CoP with respect to time. The evolution of the CoP is not the expected one even if the evolution of s˙ converges clearly toward the expected cyclic motion with the end of the second step (Fig. 13-c and 12-c). In the presence of modeling errors, the two control laws will not give the same cyclic behavior. Due to the second control law, s˙ will converge toward s˙ c , and the average velocity of the robot does not change, which is not the case for the ﬁrst control law.

q˙5 − q˙1 − q˙2

Stability Analysis of Bipedal Walking with Control of the CoP

117

Trunk evolution in its phase plane 0.5

q5 − q1 − q 2 + π

0

−0.5 0.26

0.265

0.27

0.275

0.28

0.285

0.29

0.295

0.3

15

s

(a) 0.2

6 5

0.1

s˙

l

4 t [s]

0

3 2

−0.1 0

2

4 (b)

6

1 0

5

10 (c)

Fig. 13. The convergence toward a cyclic motion is observed in simulation with the second control law, with Kvs =20, with modeling errors. (a) The trunk evolution is drawn in its phase phase (the absolute trunk velocity with respect to the absolute trunk orientation), it tends toward a limit cycle. (b) The horizontal position of the CoP with respect to time l(t) is bounded. It tends toward a cyclic evolution diﬀerent from Fig. 11(b). (c) s(s) ˙ tends toward to the same cyclic motion as in Fig. 7(c)

7 Discussion Section Even if the stability studies for the two proposed control laws are conducted numerically for the examples, based on the analytical study of the robot without feet [4] and on the case of a desired piecewise evolution of the position of the center of pressure [7], and also based on numerous simulations, some general conclusions can be given: • For the ﬁrst control law, the choice of ld (s) has a large eﬀect on the existence of a cyclic motion and on the average velocity of the robot. If the position of the CoP is moved forward, the average velocity of the cyclic motion is slowed down. This property is limited: if the position of the CoP is too much forward, no cyclic motion exists. In order that the robot walks faster, a simple solution is to move the desired evolution of the CoP backward. • The stability property of the ﬁrst control strategy is essentially due to the fact that the vertical velocity of the center of mass is directed downward just before the impact [4], [7]. Thus this property depends essentially on the choice of q d (s).

118

D. Djoudi and C. Chevallereau

• A larger variation of ld (s) during one step has two eﬀects, the basin of attraction of the control law is slightly increased, the convergence rate is slightly decreased. • For the ﬁrst control law, the control speed s˙ is not directly controlled, as shown in Fig. 5, but only stabilizes step by step. The impact phase has a key role in this stabilization. For the example, if the single support phase can be achieved, s˙ increases non linearly during the single support and decreases linearly during the impact phase, thus a stable speed is reached as in a passive walking. • When the desired joint references and the desired position of the center of pressure are deﬁned, since they are not function of time, we do not have to worry about the dynamic consistency. The joints reference need only to be twice diﬀerentiable and to satisfy the start and stop conditions corresponding to the impact model. The second derivative of s is calculated to satisfy the dynamic consistency. • In the development of the control, a ﬁnite time controller is deﬁned in eq. (18), to insure a fast convergence to the zero dynamic manifold. Such a controller is not required for the simulation and experiments. The dynamic model is used to calculate the position of the CoP and the admissible limits, for the experiments because it implies that the dynamic model must be “correctly” known. The robustness tests (Sect. 6) have demonstrated that an acceptable behavior can be obtained in the presence of an imprecise model. • For the second control law, an arbitrary function s˙ c (s) can be chosen even if ss˙˙ cc (1) (0) = α. If this function s˙ c (s) is not consistent with the constraint on the dynamic model (lmin < l(s) < lmax ), the closed loop system will converge to an evolution “close” to s˙ c (s) but consistent with the constraint on the dynamic model. This can be used to choose faster or slower motion. For the proposed example, if we choose s˙ c = 1, we obtain a cyclic motion with an average velocity of 0.51 ms−1 , the CoP position is in the forward part of the feet and often on the toe limit. If we choose s˙ c = 4 we obtain a cyclic motion with an average velocity of 1.5 ms−1 , the CoP position is often in the limit of the foot. • The proposed control laws can be extended to walking including rotation of the foot about the toe [5]. • We hope that the second control strategy can be directly used for robot walking in 3D, even if the position of the CoP is limited in the sagittal and frontal plane.

8 Conclusion For a planar biped, the proposed control strategy consists in the tracking of a reference path instead of a reference motion for the joints and for the position of the CoP. The biped adapts its temporal evolution according to the dynamic

Stability Analysis of Bipedal Walking with Control of the CoP

119

constraint that relies the position of the CoP and the joint acceleration. In this context a complete study has been presented. The conditions of stability are inequalities. Thus a certain robustness is naturally contained in the proposed control strategy. In spite of tracking errors and/or modeling errors, the behavior of the biped converges to a cyclic motion. In the presence of modeling errors, the obtained cycle is slightly modiﬁed with respect to the predicted cycle, but stable walking is obtained as it has been observed in simulation. Two control strategies have been proposed. In the ﬁrst case, the CoP is constrained to be a function of the robot conﬁguration and the geometric evolution of the joints are controlled, but the temporal evolution is free; the natural convergence toward a cyclic motion is used. In the second case, the convergence to the cyclic motion is forced by using the CoP as a control input to correct for errors in the conﬁguration speed, s, ˙ and the limits on the CoP position are used lmin < l < lmax .

References [1] S. P. Bhat and D. S. Bernstein. Continuous ﬁnite-time stabilization of the translational and rotational double integrators. IEEE Transaction on Automatic Control, 43(5):678–682, 1998. [2] C. Chevallereau, G. Abba, Y. Aoustin, F. Plestan, E. R. Westervelt, C. Canudas-de Witt, and J. W. Grizzle. Rabbit: A testbed for advanced control theory. IEEE Control System Magazine, 23(5):57–78, 2003. [3] C. Chevallereau and A. Lounis. On line reference trajectory adaptation for the control of a planar biped. In 5th international Conference on Climbing and Walking Robots CLAWAR, pp. 427–435, Paris, France, 2002. [4] C. Chevallereau, Formal’sky A. M., and D. Djoudi. Tracking of a joint path for the walking of an under actuated biped. Robotica, 22(1):15–28, 2004. [5] Jun Ho Choi and J.W. Grizzle. Planar bipedal walking with foot rotation. In paper submitted to ACC, 2005. [6] O. Dahl and L. Nielsen. Torque-limited path following by online trajectory time scaling. IEEE Trans. on Automat. Contr, 6(5):554–561, 1990. [7] D. Djoudi and C. Chevallereau. Stability analysis of a walk of a biped with control of the zmp. In IROS’05, 2005. [8] D. Djoudi, C. Chevallereau, and Y. Aoustin. Optimal reference motions for walking of a biped robot. In ICRA 05, 2005. [9] J. W. Grizzle, G. Abba, and F. Plestan. Asymptotically stable walking for biped robots: analysis via systems with impulse eﬀects. IEEE Trans. on Automat. Contr., 46:51–64, 2001. [10] J. Guckenheimer and P. Holmes. Nonlinear Oscillations, Dynamical Systems, and Bifurcations of Vector Fields. Springer-Verlag, 1985. [11] K. Hirai, M. Hirose, Y. Haikawa, and T. Takenaka. The development of honda humano¨ı¿ 21 robot. In Proc. of the IEEE International Conference on Robotics and Automation, pp. 1321–1326, Leuven, Belgium, 1998.

120

D. Djoudi and C. Chevallereau

[12] F. Pfeiﬀer, K. Loﬄer, and M. Gienger. The concept of jogging johnie. In IEEE Int. Conf. on Robotics and Automation, 2002. [13] M. Vukobratovic and B. Borovac. Zero moment point -thirty ﬁve years of its live. Int. Journal of Humanoid Robotics, 1(1):157–173, 2004. [14] M. Vukobratovic, B. Borovac, D. Surla, and D. Stokic. Biped Locomotion. Springer-Verlag, 349p, 1990. [15] E.R. Westervelt, J.W. Grizzle, and D.E. Koditschek. Hybrid zero dynamics of planar biped walkers. IEEE Transaction on Automatic Control, 48(1):42–56, 2003. [16] P.-B. Wieber and C. Chevallereau. Online adaptation of reference trajectories for the control of walking systems. Robotics and Autonomous Systems, 54:559– 566, 2006.

Multi-Locomotion Control of Biped Locomotion and Brachiation Robot T. Fukuda1 , M. Doi1 , Y. Hasegawa2 , and H. Kajima1 1

2

Department of Micro-Nano Systems Engineering, Nagoya University Furo-cho 1, Chikusa-ku, Nagoya, 464-8603, JAPAN {fukuda, doi}@mein.nagoya-u.ac.jp Department of Intelligent Interaction Technologies, University of Tsukuba Tennodai 1-1-1, Tsukuba, Ibaraki, 305-8573, JAPAN [email protected]

Summary. This paper ﬁrst introduces a multi-locomotion robot with high mobility and then proposes Passive Dynamic Autonomous Control (PDAC) for the comprehensive control method of multiple types of locomotion. PDAC is the method to take advantage of the robot inherent dynamics and to realize natural dynamic motion. We apply PDAC to a biped walk control. On the assumption that the sagittal and lateral motion can be separated and controlled individually, each motion is designed based on the given desired step-length and period. In order to stabilize walking, the landing position control according to the status is designed. In addition, a coupling method between these motions, which makes the period of each motion identical, is proposed. Finally, we show that the multi-locomotion robot realizes the 3-dimensional dynamic walking using the PDAC control.

1 Introduction In recent years there have been many successful researches that focus on dynamic and skillful motions inspired by animal dexterity [13, 15, 16, 21, 27]. However, in general, they were mainly focused on a single type of motion, such as biped or quadruped locomotion. On the other hand, many animals, such as primates, use a primary form of locomotion but switch to other types depending on their surroundings, situation and purpose. For instance, a gorilla has high mobility in a forest by selecting a bipedal walking in a narrow space, a quadrupedal walking on rough terrain and a brachiation in the forest canopy. Inspired by high mobility of an animal, we have developed a anthropoid-like “Multi-locomotion robot” that can perform several types of locomotion and choose the proper one on an as-need basis (Fig. 1) [12]. A development of a bio-inspired robot which has multiple types of locomotion for high mobility is challenging, because other problems arise in addition to research topics on humanoid robot study. One is a comprehensive control architecture that is

122

T. Fukuda et al. Brachiation

Biped Locomotion

Quadruped Locomotion

Fig. 1. Concept of the multi-locomotion robot

capable to achieve multiple types of locomotion. A common control architecture should be designed when the robot achieves a seamless transient motion connecting one locomotion to another such as a transient from trot to gallop, because a transient motion between typical types of locomotion can not be realized by fusing control signals from the corresponding controllers. Based on this notion, we have proposed a novel method named Passive Dynamic Autonomous Control (PDAC) [2, 3] that achieves not only a bipedal walk but also a quadrupedal walk. This paper focuses on the PDAC algorithm and control method for the bipedal walk. A lot of research of ZMP-based control [31] has been presented [19, 28]. However, ZMP-based control could not realize an eﬃcient locomotion since it does not take advantage of the robot inherent dynamics. To solve this problem, it is necessary to develop a dynamics-based method. Some researchers proposed a method to use the robot dynamics directly by making the pointcontact between a robot and the ground [7, 14, 21, 33]. Miura and Shimoyama [17] presented a stilt-like biped and control method to stabilize the gaits by changing the robot posture at foot-contact. Kajita et al. [13] proposed a control method based on the conserved quantity introduced due to a horizontal COG (Center Of Gravity) trajectory. Goswami et al. [6, 29] reported a method to realize quasi-passive walking on the horizontal ground. Grizzle and Westervelt et al. [8, 20, 32] proposed a control method of an underactuated planar robot with a trunk and proved its stability. Although some of these point-contact methods actually realized smooth dynamic walking, their walking was 2-dimensional or that of a robot without trunk. Thus, the main goal of a biped walk is to propose the new control method based on point-contact and realize 3-dimensional dynamic walking of a multiple link robot with a trunk.

Multi-Locomotion Control of Biped Locomotion and Brachiation Robot

123

In this paper, we introduce the novel control method named Passive Dynamic Autonomous Control (PDAC). PDAC assumes the two following premises: 1) point-contact 2) interlocking. The second premise means that the angles of the robot joints are connected with the angle around contact point. Although this concept was proposed ﬁrst by Grizzle et al. [8], we propose another new method to control the robot dynamics by means of PDAC. The approach of PDAC is to describe the robot dynamics as a 1-DOF autonomous system, which is the dynamics around the contact point. This approach makes it possible to calculate the period from foot-contact to next foot-contact (we term this foot-contact period hereinafter), hence the foot-contact period of the lateral motion and that of the sagittal one can be made identical. Each motion is designed by means of PDAC based on the assumption that the sagittal and lateral motions can be separated. After that, by keeping the conservative quantity of the autonomous system, the walking motion is stabilized. In addition, we propose a coupling method of each motion to make each foot-contact period identical. Finally, by means of the proposed method, 3-dimensional natural dynamic walking based on the robot inherent dynamics is achieved. In the following section, the multi-locomotion robot is introduced and then PDAC is explained in detail in Sect. 3. The 3-dimensional walking is designed by means of PDAC in Sect. 4. Section 5 describes the experimental results. Finally, Sect. 6 is conclusion.

2 Multi–Locomotion Robot The dimensions of the multi-locomotion robot we developed is designed based on those of a gorilla, and therefore the robot is called “Gorilla Robot III”. Figure 2 shows the overview of Gorilla Robot III and its link structure. This robot is about 1.0 [m] tall, weighs about 22.0 [kg], and consists of 25 links and 26 AC motors including two grippers. The real-time operating system VxWorks (Wind River Systems Inc) runs on a Pentium III PC for processing sensory data and generating its behaviors. The rate gyroscope and force sensor attached at each wrist measures the angular velocity around the grasping bar to calculate the pendulum angle during the brachiation, and reaction forces from grasping bars in order to judge whether the robot successfully grasps the bar or not, respectively. Some photo sensors are attached on the sole in order to perceive foot-contact. This robot has been designed to perform biped locomotion, quadruped locomotion and brachiation. We also consider the intermediate motion between a bipedal and quadrupedal walk in order to realize seamless transfer from a bipedal walk to a quadrupedal walk and from a quadrupedal walk to a bipedal walk without pause. As the ﬁrst step, we designed the controller for both locomotion using the same algorithm “PDAC”. The snapshot of the quadrupedal walk is shown in Fig. 3, and a bipedal walk is shown in Sect. 5. Brachiation is an interesting form of locomotion performed by long-armed apes by using

124

T. Fukuda et al. 350 mm

280 mm

450 mm

360 mm

225 mm 355 mm 210 mm

Fig. 2. Gorilla Robot III

their arms to swing from branch to branch. This motion is a dynamic and dexterous action routinely performed by some kinds of apes [26, 30]. Fukuda et al. developed a six-link brachiation robot [4] as a pioneering research analyzing dynamics of brachiation, “Brachiator II” [5, 9, 18, 22, 23] that is 2-link underactuated system like “Acrobot” [1, 27], and “Brachiator III” [10, 24] that achieves three-dimensional brachiation with redundant mechanisms. Based on these studies, we designed over-hand and side-hand motions of “Gorilla Robot II” [12], using a motion learning algorithm, and “Gorilla Robot III” achieves a continuous brachiation shown in Fig. 4 by implicitly using the PDAC method in locomotion action control.

3 Passive Dynamic Autonomous Control 3.1 Target Dynamics The concept of PDAC is the same as what Grizzle et al. [8] has proposed. We begin with the two following premises. First, the contact state between a robot and the ground is point-contact. Second, robot joints are interlocked with the angle around the contact point. The ﬁrst premise means that the ﬁrst joint of a robot, i.e. the ankle joint of the stance leg, is passive. The second means that the angles of active joints are described as a function of the angle around the contact point. Assuming that PDAC is applied to the serial n-link rigid robot shown in Fig. 5, these two premises are expressed as follows: τ1 = 0 Θ = [θ1 , θ2 , · · · , θn ]T = [f1 (θ), f2 (θ), · · · , fn (θ)]T = f (θ) ,

(1) (2)

Multi-Locomotion Control of Biped Locomotion and Brachiation Robot

125

Fig. 3. Snapshots of the quadrupedal walking of PDAC. The step length is about 0.09 [m] and velocity is about 0.176 [m/s]

where θ is the angle around the contact point in the absolute coodinate system. Since it has no eﬀect on the robot dynamics due to point-contact, level ground is assumed, therefore θ1 = f1 (θ) = θ. The dynamic equations of this model are given by d

1 ∂ T M (Θ)Θ˙ − Θ˙ M (Θ)Θ˙ − G(Θ) = τ , dt 2 ∂Θ

(3)

where M (Θ) = [M1 (Θ), M2 (Θ), · · · , Mn (Θ)]T , Θ = [θ1 , θ2 , · · · , θn ]T , G(Θ) = ∂ = [ ∂θ∂ 1 , ∂θ∂ 2 , · · · , ∂θ∂n ]T . [G1 (Θ), G2 (Θ), · · · , Gn (Θ)]T , τ = [τ1 , τ2 , · · · , τn ]T , ∂Θ Since in this model the dynamic equation around the contact point has no term of the Coriolis force, it is given as d dt

M1 (Θ)Θ˙ − G1 (Θ) = τ1 .

(4)

By diﬀerentiating Eq. (2) with respect to time, the following equation is acquired,

126

T. Fukuda et al.

(1) t=1.00[s]

(2) t=1.49[s]

(3) t=2.16[s]

(4) t=3.31[s]

(5) t=3.97[s]

(6) t=5.13[s]

(7) t=5.95[s]

(8) t=6.94[s]

(9) t=7.77[s]

(10) t=8.76[s]

(11) t=9.74[s]

(12) t=10.57[s]

(13) t=11.40[s]

(14) t=12.39[s]

(15) t=13.21[s]

(16) t=14.70[s]

Fig. 4. Snapshots of continuous brachiation. All bars are set at regular intervals of 0.4 m and at the same height of 2.7 m

θn mn

y

τn

m2 θ2

o

J2

τ2

J1 θ1

Jn

m1

τ1 x

Fig. 5. Mechanical model of the serial n-link rigid robot. θi and τi are the angle and the torque of the ith joint respectively. mi and Ji are the mass and the moment of inertia of the ith link respectively

Multi-Locomotion Control of Biped Locomotion and Brachiation Robot

% &T ∂f1 (θ) ∂f2 (θ) ∂f (θ) ˙ ∂fn (θ) Θ˙ = θ= , ,··· , θ˙ . ∂θ ∂θ ∂θ ∂θ

127

(5)

Substituting Eqs. (1), (2) and (5) into Eq. (3) yields the following dynamic equation, d M (θ)θ˙ = G(θ) , (6) dt where df (θ) M (θ) := M1 f (θ) (7) dθ (8) G(θ) := G1 f (θ) . By multiplying both sides of Eq. (6) by M (θ)θ˙ and integrating with respect to time, the dynamics around the contact point is obtained as follows: d M (θ)θ˙ M (θ)θ˙ dt = M (θ)G(θ)θ˙ dt (9) dt 0 1 ⇐⇒ θ˙ = 2G(θ)M (θ) dθ . (10) M (θ) Assuming that the integration in right side of Eq. (10) is calculated as , G(θ)M (θ) dθ = D(θ) + C, Eq. (10) is described as the following 1-DOF autonomous system, 1 1 2 D(θ) + C (11) θ˙ = M (θ) := F (θ) . (12) In this paper, we term Eqs. (11) and (12) the target dynamics. 3.2 Dynamics Interlocking As mentioned previously, PDAC is based on the two premises: passivity and interlocking. These premises make it possible to describe the whole robot dynamics as a 1-DOF autonomous system, owing to which the simple and valid controller based on the robot dynamics can be composed. However, interlocking of joint angles has the possibility to create a problem that the robot vibrates and the controller loses its stability during locomotion, especially at foot-contact, since if the passive joint vibrates, all of other active joints also do. In order to solve this problem, all of the robot joints are controlled according to the desired dynamics of each joint derived from the interlocking function Eq. (2) and the target dynamics Eq. (12) as follows: ∂fi −1 F fi (θi ) θ˙i = ∂θ

(i = 1, 2, 3, · · · ) .

(13)

128

T. Fukuda et al.

These desired dynamics are independent from each other, thus it is necessary to connect the desired dynamics of the active joints with the target dynamics in order to prevent the whole walking motion being broken in case of error between the target dynamics and the actual dynamics of θ. Hence, we deﬁne the connection between the target dynamics and the active joints. The controller decides the desired angular velocities of each joint as described below, (14) θ˙1d = F f1−1 (θ1 ) ∂f i F fi−1 (θi ) + ki fi (θ) − θi (i = 2, 3, · · · ) (15) θ˙id = ∂θ ⇐⇒ Θ˙ d := F (Θ) ,

(16)

where ki is the strength of connection determined experimentally since its value has little eﬀect on the robot dynamics. As for humanoid robots, the ground slope at the contact point is deduced from the angle of the ankle joint of the swing leg at foot-contact, and θ is calculated from θ1 and the ground slope. The remarkable point is that if there is no error such as model error or disturbance, the second term of Eq. (15) is constantly zero and the actual dynamics of θ is identical with the target dynamics. Figure 6 shows the block diagram of PDAC of bipedal locomotion. The control loop including a robot (enclosed by the dotted line in Fig. 6) has no input, thus it can be considered that the control system is autonomous. This autonomy makes it possible to realize natural dynamic motion based on the inherent dynamics of a robot. The loop described by the broken line is executed only at the moment of foot-contact. In this loop, the target dynamics of the next step is determined according to both the desired parameters such as walking velocity and the robot status, then F is updated. Since this updating compensates the error between the previous target dynamics and the actual ones around the contact point, it is possible to realize stable walking. 3.3 PDAC Constant Since as mentioned previously, the target dynamics is autonomous, in addition, independent of time, it is considered as a kind of conservative system. Therefore, it is conceivable that the target dynamics has a conserved quantity. As for PDAC, it is the constant of integration in right side of Eq. (10). That is, C in Eq. (11) is the conserved quantity of the target dynamics, which is named PDAC Constant. It is clear that PDAC Constant is decided in accordance with initial condition and that the robot motion is generated as it is kept constant. In order to stabilize walking, the controller updates the target dynamics according to PDAC Constant. This method to update is presented later.

Multi-Locomotion Control of Biped Locomotion and Brachiation Robot

129

Desired Velocity

Dynamics generator

θc,θc

Θ = F (Θ)

d

Θ

Angular velocity control

Robot

at foot contact

updating F

Θ ,Θ

Θ Autonomous

Fig. 6. Block diagram of PDAC of bipedal locomotion. θc and θ˙c are the angle and the angular velocity of θ1 at foot-contact respectively

4 Bipedal Walking Control In this paper, it is assumed that lateral motion and sagittal one can be separated and controlled independently since lateral side-to-side rocking motion is quite small and step-length in the sagittal plane is relatively short. Although both motions are composed independently, the period from foot-contact to next foot-contact (foot-contact period) in both planes are necessarily identical. We design each motion by means of PDAC by giving both the desired step-length, λd , and desired foot-contact period, T d , and propose a coupling method of both motions. In addition, the landing position control is designed based on PDAC. At ﬁrst the sagittal motion control is presented that is followed by the lateral motion control satisfying the condition of the foot-contact period is explained. 4.1 Sagittal Motion Control 3-Link Model For the sake of simplicity, the 3-link model as shown in Fig. 7 is used, i.e. the upper body of the robot is not moved. The dynamic equation of this model is described as Eq. (3) and that of the ankle joint of the stance leg is Eq. (4) where n = 3. The left side of Eq. (4) is described as follows:

130

T. Fukuda et al. θ1 γ

γ

θ3

m3 J3

a3 a2 l1 a1

m1 J1

m2 J2

θ1

θ2

Fig. 7. 3-link model in the sagittal plane. mi , Ji , li and ai are the mass, the moment of inertia, the length of link and the distance from the joint to the link COG of link i respectively. γ is the angle of forward tilting. In the right ﬁgure, θ1 , θ2 and θ3 are the ankle angle of the stance leg, the angle from the stance leg to the the swing leg, the angle to swing the trunk up respectively

M11 (Θ) = J1 + J2 + J3 + m1 a21 + m2 l12 + m2 a22 − 2m2 a2 l1 cos θ2 + m3 l12 + m3 a23 + 2m3 a3 l1 cos(γ − θ3 ) M12 (Θ) = −J2 − m2 a22 + m2 a2 l1 cos θ2

(17) (18)

M13 (Θ) = −J3 − m3 a23 − m3 a3 l1 cos(γ − θ3 ) G1 (Θ) = (m1 a1 + m2 l1 + m3 l1 )g sin θ1 + m2 ga2 sin(θ2 − θ1 ) + m3 ga3 sin(θ1 + γ − θ3 ) ,

(19) (20)

where M1 (Θ) = [M11 (Θ), M12 (Θ), M13 (Θ)]. Interlocking of Sagittal Joints Grizzle et al. [8] used the following interlocking in their previous paper to maintain the angle of the torso at some constant value and to command the swing leg to behave as the mirror image of the stance leg. In this paper, we use the same interlocking, that is, θ1 = f1 (θ) = θ − β θ2 = f2 (θ) = 2θ

(21) (22)

θ3 = f3 (θ) = θ ,

(23)

where β is the ground slope at the contact point (ascent is positive). From Eqs. (21)–(23) and (1), Eq. (6) is Ms (θ) = (J1 − J2 + m1 a21 + m2 l12 − m2 a22 + m3 l12 ) + m3 a3 l1 cos(γ − θ)(24) := E1 + E2 cos(γ − θ) (25) Gs (θ) = (m1 a1 + m2 l1 + m2 a2 + m3 l1 )g sin θ + m3 ga3 sin γ := E3 + E4 sin θ .

(26) (27)

Multi-Locomotion Control of Biped Locomotion and Brachiation Robot

ξ

-

l

-

l+

-i-1θ+ iθ

-

ξ+

131

-i θ+

Fig. 8. Parameters at foot-contact. l− and ξ − are the length and inclination of the inverted pendulum which connects the supporting foot and the COG of the whole robot before impact, while l+ and ξ + are those after impact. i θ− and i θ+ are the angles around the contact point before and after impact of the ith step

Thus,

E1 + E2 cos(γ − θ) E3 + E4 sin θ dθ (28) sin(γθ) cos(2θ − γ) = E2 E 4 − 2 4 +E2 E3 sin(θ − γ) − E1 E4 cos θ + E1 E3 θ + Cs (29)

Ms (θ)Gs (θ)dθ =

:= Ds (θ) + Cs

(30)

where Cs is the integral constant, which is PDAC Constant of the sagittal motion. From Eq. (11), the target dynamics in the sagittal plane is 1 1 (31) 2 Ds (θ) + Cs θ˙ = Ms (θ) := Fs (θ) . (32) From Eqs. (21)–(23), f1−1 (θ1 ) = θ1 + β, f2−1 (θ2 ) = 12 θ2 , f3−1 (θ3 ) = θ3 are obtained, thus the desired angular velocities of sagittal joints are described as follows: θ˙1d = Fs (θ1 + β) ˙θd = 2Fs θ2 + k2 (2θ − θ2 ) 2 2 d θ˙ = Fs (θ3 ) + k3 (θ − θ3 ) . 3

(33) (34) (35)

Foot Contact Model In this paper, it is assumed that foot-contact occurs instantaneously and the angular momentum around the contact point is varied instantly. The angular momentum is described as

132

T. Fukuda et al.

P = Ms (θ)θ˙ .

(36)

Figure 8 depicts some parameters at foot-contact. Assuming that the translational velocity along the pendulum at foot-contact is zero since it is quite small, the angular velocity around the contact point is acquired as follows: l+ P + = l− P − cos(ξ − + ξ + ) l− Ms (i θ− )− ⇐⇒ i θ˙+ = + cos(ξ − + ξ + ) i θ˙− l Ms (i θ+ )+ ⇐⇒ i θ˙+ := Hs i θ˙−

(37) (38) (39)

From this value, and the PDAC Constant at the ith step, i Cs , is obtained as i Cs

=

2 1 Ms i θ+ i θ˙+ − Ds i θ+ . 2

(40)

Desired PDAC Constant Since the target dynamics is the 1-DOF autonomous system, it is possible to calculate the foot-contact period by integrating Eq. (12) with respect to time. The foot-contact period satisfying the desired step-length is calculated as below: θ˙ = Fs (θ) 1 dθ = dt ⇐⇒ Fs (θ)

(41) (42)

ˆ−

i+1 θ

⇐⇒

1 dθ = Tˆs , Fs (θ)

(43)

ˆ+

iθ

λ are the desired value of i θ+ and i+1 θ− that can where i θˆ+ = i+1 θˆ− = sin−1 2l 1 be calculated from the desired step-length. This period is necessarily identical with the desired foot-contact period, thus d

Tˆs = T d .

(44)

In order to generate the stable cyclic walking, the angular velocity around the contact point after impact must be kept constant, that is, ˙+ = i+1 θ˙+ .

iθ

(45)

By solving two conditions, Eqs. (44) and (45), by means of two dimensional approximation of i θˆ+ and i+1 θˆ− , the desired PDAC Constant is determined, Csd = Csd (T d , λd ) .

(46)

Multi-Locomotion Control of Biped Locomotion and Brachiation Robot

133

Stabilization of Sagittal Motion Since the target dynamics is a conservative system, if the PDAC Constant is kept constant, the stability of the motion is guaranteed. In order to stabilize walking, step-length is varied according to the PDAC Constant. This method takes advantage of the loss of angular momentum at foot-contact, that is, if the step-length is long, the loss is high, while if short, it is low. The control strategy is to adjust the step-length of the next step after every foot-contact in accordance with the desired PDAC Constant and the actual one as follows: Csd + Ds (i+1 θ+ ) 2

=

Ds (i+1 θ− ) + i Cs

Ms (i+1 θ− ) ⇐⇒ i+1 θ− = i+1 θ− Csd , i Cs . Ms (i+1 θ+ )

2

Hs2

(47) (48)

Note, however, that since in this paper level ground is assumed, i+1 θ− = + i+1 θ . This stabilizing control makes it possible to keep PDAC Constant in the vicinity of the desired value. Therefore, sagittal motion is kept stable. Here, the point to notice is that the foot-contact period diﬀers from the desired foot-contact period due to stabilization. Hence, it is necessary to control the lateral motion so that the period of lateral motion is identical with the following period of sagittal motion,

i+1 θ

Ts =

−

1 dθ . Fs (θ)

(49)

+ iθ

4.2 Lateral Motion Control Lateral Motion Many reserchers investigated and proposed lateral motion control [11, 14, 25]. In this paper, we design the lateral motion by means of PDAC as depicted in Fig. 9. In order to continue the side-to-side rocking motion, a robot lifts its pelvis in phases (A) and (B). The inverted pendulum whose length is variable is used as the model of the lateral plane since the motion to lift the pelvis is quite small, in addition, the robot posture is varied little thus the motion to lift the pelvis can be considered as lengtherning the pendulum. The lateral motion can be continued in spite of the loss of angular momentum at footcontact by changing the pendulum length at impact. Collision Inverted Pendulum Model The following model shown in Fig. 10 is used as the model of the lateral motion: two inverted pendulums which are opposite each other continue to

134

T. Fukuda et al. Left Foot Contact

Sw

wn

ll Fa

ing

Do

Front View

Up

(B)

(A) Transition

(B)

(A)

: Passive joint n ow ll D

Sw

ing

Up

Fa Transition

Right Foot Contact Right-Leg-Support

Left-Leg-Support

Fig. 9. The lateral motion of lateral-based walk (front view). The inverted pendulum falls oﬀ in phase(A) and swings up in phase(B) collision

(A)

(B)

collision

(B)

(A)

Fig. 10. Motion of CIPM. The collision between the foot and the ground is regarded as that between two pendulums. (A) and (B) correspond to those in Fig. 9

Multi-Locomotion Control of Biped Locomotion and Brachiation Robot

135

w

φ

l

φ2

φ1

O

φR

l0 φ0

OR

l0-∆l φ1

-φ3

-φ2

OR

φL φR

OL

l0+∆l

(a)

OL

(c)

φR

OR

φL

φR

OL

(b) Fig. 11. (a) Trajectory of COG and polar coordinate systems Σ R , Σ L . l and φ denote the length and the angle of an inverted pendulum. (l0 , φ0 ) and (l0 + ∆l, φ1 ) are the coordinates in Σ R at the beginning and ending of phase (A), (l0 −∆l, φ2 ) and (l0 , φ3 ) is that of Σ L of phase (B) respectively. φ˙ 1 , φ˙ 2 denotes the angular velocity at the end of phase (A) and at the beginning of phase (B). (b) Phase portraits of φR and φL (c) Phase portrait of CIPM. The gray tetragon surrounded by the pair of separatrixes is named CIP-Area

rock, iterating the collision between them, which is named Collision Inverted Pendulum Model (CIPM). This CIPM is intuitively like the Newton’s Pendulum inverted. Figure 11 shows the trajectory of COG and two coordinate systems Σ R and Σ L that correspond to the right- and left-leg-support period respectively, and Fig. 11(b) depicts the phase portraits of φR and φL . These two phase portrait’s coalescing yields the phase portrait of CIPM (see Fig. 11(c)). In the phase portrait of CIPM, there is the area in which one has the circular nature between the coordinate systems Σ R and Σ L . In this area, the periodic motion can be realized due to the circular nature.

Interlocking of Lateral Joints The interlocking in the lateral plane is deﬁned as below, P hase(A) : l = fA (φ) = a1 φ + b1 P hase(B) : l = fB (φ) = a2 φ2 + b2 φ + c2 ,

(50) (51)

136

T. Fukuda et al.

where 1 ∆l φ1 − φ0 φ0 = l0 − ∆l φ1 − φ0 1 = ∆l (φ3 − φ2 )2 2φ2 =− ∆l (φ3 − φ2 )2 φ2 − 2φ2 φ3 = l0 − 3 ∆l . (φ3 − φ2 )2

a1 =

(52)

b1

(53)

a2 b2 c2

(54) (55) (56)

l is the monotoneally increasing function of φ meeting the following conditions in phase (A): fA (φ0 ) = l0 and fA (φ1 ) = l0 + ∆l, while in phase (B): fB (φ2 ) = l0 − ∆l, f (φ2 ) = 0 and f (φ3 ) = l0 . The dynamic equation of the angle of an inverted pendulum is described as follows: d 2 ml + J φ˙ = mgl sin φ . (57) dt From the interlocking, Eq. (6) is described as follows: MlN (φ) = mfN (φ)2 + J GlN (φ) = mgfN (φ) sin φ ,

(58) (59)

where the suﬃx N means phase (N) (N=A, B). From Eq. (11), the target dynamics in the lateral plane is 0 1 (60) 2MlN (φ)GlN (φ) dφ φ˙ = MlN (φ) 1 1 := (61) 2 DlN (φ) + ClN MlN (φ) := FlN (φ) , (62) where ClN is the integral constant, which is PDAC Constant of the lateral dynamics. Assuming that the collision between the swing leg and the ground is perfectly non-elastic, the angular velocity of the inverted pendulum after impact is v1 cos (φ1 − φ2 − ζ) (63) φ˙2 = l0 − ∆l 2 a21 + (l + ∆l)2 cos (φ1 − φ2 − ζ) φ˙ 1 = (64) l0 − ∆l (65) := Hl φ˙ 1 ,

Multi-Locomotion Control of Biped Locomotion and Brachiation Robot

where

3 1 3 2 2 ˙ ˙ v1 = (lφ) + l 33 = a21 + (l + ∆l)2 φ˙ φ=φ1 3 3 a1 l˙ 3 −1 −1 = tan ζ = tan . 3 l + ∆l lφ˙ 3

137

1

(66) (67)

φ=φ1

∆l is the control value of the lateral motion. It is calculated from the condition of the beginning of phase(A) and the end of phase(B): FlA (φ0 ) = 0 and FlB (φ3 ) = 0. That is, Dl (φ3 ) − DlB (φ2 ) DlA (φ1 ) − DlA (φ0 ) 2 Hl = B . 2 MlA (φ1 ) MlB (φ2 )2

(68)

∆l is so small that it can be obtained from this equation by means of the linear approximation of ∆l, ∆l = ∆l(φ0 , φ3 ) .

(69)

Finally, it is necessary to determine the desired amplitude of the rocking motion, φd0 , so that the foot-contact period in the lateral plane matches with the desired foot-contact period. This condition is described as below, φ1 φd 0

d

1 dφ + FlA (φ)

φ0 φ2

1 dφ = T d . FlB (φ)

(70)

By means of two dimensional approximation of φ, it is possible to calculate φd0 from Eq. (70), φd0 = φd0 (T d ) .

(71)

By setting φ3 at −φd0 at the beginning of phase (A) of every step, the lateral motion can be stabilized. Coupling with the Sagittal Motion As mentioned previously, it is necessary that the foot-contact period of the sagittal motion and that of the lateral motion are made identical. In case of the adjustment of step-length, the sagittal foot-contact period diﬀers from the desired foot-contact period, thus the lateral motion needs to be varied according to the period of Eq. (49). In order to control the lateral foot-contact period, the foot width is adjusted as shown in Fig. 12. l − ∆l + δl and φ2 + δφ2 are acquired from geometrically. It is assumed that this adjustment is so small that its eﬀect on

138

T. Fukuda et al.

ε l-∆l+δl

φ l+∆l

-(φ2 + l-∆l

Fig. 12. Adjustment of foot width. is the angle to open the swing leg to adjust the foot width. l − ∆l + δl and φ2 + δφ2 are the pendulum length and angle at the beginning of phase (B) after adjustment

the target dynamics in phase (A) can be neglected. By the adjustment, the parameters of the target dynamics in phase (B) are varied as follows: 1

2 (∆l − δl) φ3 − (φ2 + δφ2 ) 2φ2 b2 = − 2 (∆l − δl) φ3 − (φ2 + δφ2 )

a2 =

φ2 − 2(φ2 + δφ2 )φ3 c2 = l0 − 3 2 (∆l − δl) . φ3 − (φ2 + δφ2 )

(72) (73) (74)

The condition that the pendulum pauses at the end of phase (B) is FlB (φ3 ) = 0, hence 2 DlB (φ3 ) − DlB (φ2 + δφ2 ) + M (φ2 + δφ2 )φ˙ 2 = 0 . (75) In addition, the condition that the foot-contact period must satisfy is φ3 φ2 +δφ2

1 dφ + FlB (φ)

φ1 φ3

1 dφ = Ts . FlA (φ)

(76)

The ﬁrst term of the left side in Eq. (76) is the period of phase (B) and the second term is that of the subsequent phase (A). The two conditions of Eq. (75) and (76) have two unknowns, i.e. the adjustment value, , and the pendulum angle at the end of phase (B), φ3 . By solving these two conditions by means of linear approximation of and two dimensional approximation of φ3 , the adjustment value, , can be calculated ˙ Ts ) . = (φ, φ,

(77)

Multi-Locomotion Control of Biped Locomotion and Brachiation Robot

139

Autonomous

φ φd Angular velocity control

Robot ε

updating F

∆l (φ 3 , φ 0 ) d

φd0 (T d )

ε = ε (φ, φ, Ts)

φ, φ at transition of phase

φ = Fl (φ)

φ, φ

Lateral Controller Td Desired Velocity

Ts =

λ

d

∫

i+1

θ

+

i

θ

-

1 dθ Fs(θ)

Coupling

Sagittal Controller d d d Cs (T , λ )

-

i+1

θ (Csd , iCs )

i

Cs ( iθ-, iθ- )

i

θ-, iθ-

Θ

d

Θ = Fs (Θ)

Angular velocity control

Robot

at foot contact

updating F

Θ ,Θ

Θ Autonomous

Fig. 13. Block diagram of the coupling between the sagittal and lateral motions

Figure 13 depicts the block diagram of the algorithm described in the previous sections. At foot-contact, the sagittal controller decides the step-length, i.e. the value of θ at the next foot-contact in order to stabilize the sagittal motion. Next, the foot-contact period of the sagittal motion is calculated by integration. Finally, the lateral controller determines the adjustment value of foot width according to both the sagittal foot-contact period and the present status in the lateral plane. This series of controls can be considered as the landing position control of three dimensional walking since the step-length is adjusted in the sagittal plane and the foot width is adjusted in the lateral plane. The box enclosed by a gray dashed line is the algorithm to decide the desired foot-contact period and step-length so that the energy consumption is minimized. However, this has not been solved and is future work, hence we

140

T. Fukuda et al.

(a)

(b)

(c)

(d)

(e)

(f)

Fig. 14. Snapshots of the walking of PDAC. Each ﬁgure shows the snapshots at (a)1st (b)7th (c)12th (d)16th (e)19th (f)22nd step

give the desired foot-contact period and step-length to the controller directly in this paper.

5 Experiment The experiment of the walking proposed in the previous section on ﬂat and level ground was conducted. Since, in order to start the walking, the robot needs potential energy, we lifted up the lateral pendulum to the position at the beginning of phase (A) and released. In experiment, the robot bends its knee joint of the swing leg so as to prevent the foot being in friction with the ground immediately after foot-contact on the assumption that the eﬀect of knee bending on the robot dynamics can be neglected. The foot of the swing leg is actuated so as to be kept parallel to the ground. The desired step-length is given to be gradually increased within the initial 5 steps up to 0.15[m] and the desired foot-contact period is given at 0.7[s]. In consequence, dynamic and natural walking is realized over 25 steps. The steplength is about 0.15[m] and the walking velocity is about 0.23[m/s]. Figure 14 shows the snapshots of the PDAC walking at the 1st, 7th, 12th, 16th, 19th, 22nd step respectively. The angle and angular velocity of the lower body joints are depicted in Fig. 15 and Fig. 16. As shown in these ﬁgures, smooth dynamics motion is realized periodically.

Multi-Locomotion Control of Biped Locomotion and Brachiation Robot 0.1

qrhL qraL qrhR qraR

0.05 Joint Angle[rad]

141

0

-0.05

-0.1

0

2

4

6

8

10

(a)

Time[s]

0.4

qphL qpkL qpaL qphR qpkR qpaR

0.3

Joint Angle[rad]

0.2 0.1

0

-0.1 -0.2

0

2

4

6

(b)

8

10 Time[s]

Fig. 15. Angle of the joints (a) in the lateral plane (b) in the sagittal plane

142

T. Fukuda et al.

Angular momentum[rad/s]

2

qrhL qraL qrhR qraR

1

0

-1

0

2

4

6

8

10 Time[s]

(a) qphL qpkL qpaL qphR qpkR qpaR

Angular momentum[rad/s]

3

2

1

0

-1

-2.0

0

2

4

6

8

10 Time[s]

(b) Fig. 16. Angular Velocity of the joints (a) in the lateral plane (b) in the sagittal plnae

6 Conclusion This paper ﬁrst introduced a multi-locomotion robot with high mobility and then proposed Passive Dynamic Autonomous Control (PDAC) for the comprehensive control method of multiple types of locomotion. PDAC is the method to take advantage of the robot inherent dynamics and realize natural dynamic motion. We applied PDAC to the biped walk control. On the assumption that

Multi-Locomotion Control of Biped Locomotion and Brachiation Robot

143

the sagittal and lateral motion can be separated and controlled individually, each motion was designed based on the given desired step-length and period. In order to stabilize walking, the landing position control according to the status was designed. In addition, a coupling method between these motions, which makes the period of each motion identical, was proposed. Finally, the 3dimensional dynamic walking whose step-length is about 0.15[m] and velocity is about 0.23[m/s] was realized.

References [1] Matthew D. Berkemeier and Ronald S. Fearing (1999), Tracking Fast Inverted Trajectories of the Underactuated Acrobot, IEEE Trans. on Robotics and Automation, 15(4):740–750. [2] M. Doi, Y. Hasegawa, and T. Fukuda (2004), Passive Trajectory Control of the Lateral Motion in Bipedal Walking, Proc. of IEEE International Conference on Robotics and Automation: 3049–3054. [3] M. Doi, Y. Hasegawa, and T. Fukuda (2004), Passive Dynamic Control of Bipedal Walking, Proc. of IEEE-RAS/RSJ International Conference on Humanoid Robots. [4] T. Fukuda, H. Hosokai and Y. Kondo (1991), Brachiation type of mobile robot, Proc. of IEEE Int. Conf. Advanced Robotics: 915–920. [5] T. Fukuda, F. Saito and F. Arai (1991), A study on the brachiation type of mobile robot(heuristic creation of driving input and control using CMAC), Proc. of IEEE/RSJ Int. Conf. on Intelligent Robots and Systems: 478–483. [6] A. Goswami, B. Espiau and A. Keramane (1997), Limit cycles in a passive compass gait biped and passivity-mimicking control laws, Autonomous Robots 4:274–286, Kluwer Academic Publishers. [7] A.A. Grishin, A.M. Formal’sky, A.V. Lensky and S.V. Zhitomirsky (1994), Dynamic Walking of a Vehicle With Two Telescopic Legs Controlled by Two Drives, The International Journal of Robotics Research, 13(2):137–147. [8] J.W. Grizzle, G. Abba and F. Plestan (2001), Asymptotically Stable Walking for Biped Robots: Analysis via Systems with Impulse Eﬀects, IEEE Trans. on Automatic Control, 46(1):56–64. [9] Y. Hasegawa, T. Fukuda and K. Shimojima (1999), Self-scaling reinforcement learning for fuzzy logic controller – applications to motion control of two-link brachiation robot, IEEE Trans. Industrial Electronics, 46(6):1123–1131. [10] Y. Hasegawa, Y. Ito and T. Fukuda (2000), Behavior coordination and its modiﬁcation on brachiation-type mobile robot, Proc. of IEEE Int. Conf. on Robotics and Automation: 3984–3989. [11] H. Hemami and B.F. Wyman (1979), Modeling and Control of Constrained Dynamic Systems with Application to Biped Locomotion in the Flontal Plane, IEEE Trans. on Automatic Control, AC-24(4): 526–535. [12] H. Kajima, Y. Hasegawa and T. Fukuda (2002), Study on brachiation controller -adjustment method of strength and timing parameters-, Proc. IEEE/RSJ Int. Conf. on Intelligent Robots and Systems: 2497–2502.

144

T. Fukuda et al.

[13] S. Kajita, T. Yamaura and A. Kobayashi (1992), Dynamic Walking Control of a Biped Robot Along a Potential Energy Conserving Orbit, IEEE Trans. on Robotics and Automation, 8(4):431–438. [14] A.D. Kuo (1999), Stabilization of Lateral Motion in Passive Dynamic Walking, The International Journal of Robotics Research, 18(9):917–930. [15] Qinghua Li, Atsuo Takanishi and Ichiro Kato (1992), Learning control of compensative trunk motion for biped walking robot based on ZMP, Proc. IEEE/RSJ Int. Conf. on Intelligent Robots and Systems: 597–603. [16] T. McGeer (1990), Passive dynamic walking, The Int. Journal of Robotics Research, 9(2):62–82. [17] H. Miura and I. Shimoyama (1984), Dynamic Walking of a biped, The International Journal of Robotics Research, 3(2):60–74. [18] J. Nakanishi, T. Fukuda and D.E. Koditschek (2000), A brachiating robot controller, IEEE Trans. on Robotics and Automation, 16(2):109–123. [19] K. Nishiwaki, S. Kagami, Y. Kuniyoshi, M. Inaba and H. Inoue (2002), Online Generation of Humanoid Walking Motion based on a Fast Generation Method of Motion Pattern that Follows Desired ZMP, Proc. of the Int. Conf. Intelligent Robots and Systems: 2684–2689. [20] F. Plestan, J.W. Grizzle, E.R. Westervelt and G. Abba (2003), Stable Walking of a 7-DOF Biped Robot, IEEE Trans. on Robotics and Automation, 19(4):653– 668. [21] M.H. Raibert (1986), Legged Robots that Balance. MIT Press, Cambridge, MA. [22] F. Saito, T. Fukuda and F. Arai (1993), Swing and locomotion control for twolink brachiation robot, Proc. of IEEE Int. Conf. on Robotics and Automation: 719–724. [23] F. Saito, T. Fukuda and F. Arai (1994), Swing and locomotion control for a two-link brachiation robot, IEEE Control Syst. Mag. 14(1):5–12. [24] F. Saito and T. Fukuda (1997), A First Result of The Brachiator III – A New Brachiation Robot Modeled on a Siamang, Artiﬁcial Life V: 354–361, MIT Press. [25] A. Sano and J. Furusho (1991), Realization of Dynamic Quadruped Locomotion in Pace Gait by Controlling Walking Cycle, International Symposium on Experimental Robotics: 491–502. [26] E.L. Simons (1972), Primate Evolution: An Introduction to Man’s Place in Nature, Macmillan. [27] M. Spong, (1995) The swing up control problem for the acrobot, IEEE Control Syst. Mag. 15(1):49–55. [28] A. Takanishi, M. Ishida, Y. Yamazaki and I. Kato (1985), The Realization of Dynamic Walking Robot WL-10RD, Proc. of Int. Conf. Advanced Robotics: 459–466. [29] B. Thuilot, A. Goswami and B. Espiau (1997), Bifurcation and Chaos in a Simple Passive Bipedal Gait, Proc. of Int. Conf. Robotics and Automation: 792–798. [30] James R. Usherwood and John E.A. Bertram (2003), Understanding brachiation: insight from a collisional perspective, The Journal of Experimental Biology, 206:1631–1642. [31] M. Vukobratovic, B. Brovac, D. Surla and D. Stokic (1990), Biped Locomotion: dynamics, stability, control, and application, Springer-Verlag.

Multi-Locomotion Control of Biped Locomotion and Brachiation Robot

145

[32] E.R. Westervelt, G. Buche and J.W. Grizzle (2004), Inducing Dynamically Stable Walking in an Underactuated Prototype Planar Biped, Proc. of Int. Conf. Robotics and Automation: 4234–4239. [33] M. Yamada, J. Furusho and A. Sano (1985), Dynamic control of walking robot with kick-action, Proc. of the ICAR: 405–412.

On the Determination of the Basin of Attraction for Stationary and Periodic Movements P. Giesl1 and H. Wagner2 1

2

Zentrum Mathematik, TU M¨ unchen, Boltzmannstr. 3, D-85747 Garching bei M¨ unchen, Germany [email protected] Motion Science, University of M¨ unster, Horstmarer Landweg 62b, D-48149 M¨ unster, Germany [email protected]

Movements of humans are achieved by muscle contractions. Humans are able to perform coordinated movements even in the presence of perturbations from the environment or of the muscles themselves. But which properties of the muscles and the geometry of the joints are responsible for the stability? Does the stability depend on the joint angle? How large are the perturbations, the muscle-skeletal system can cope with before reﬂexes or controls by the brain are necessary? To answer these questions, we will derive a mathematical model of the muscle-skeletal system without reﬂexes. We present diﬀerent mathematical methods to analyze these systems with respect to the stability of movements and thus provide the mathematical tools to answer the above questions. This paper is a companion paper to [13] where the biological applications of the mathematical methods presented in this paper are discussed in more detail. Stationary and periodic movements are modelled by autonomous and timeperiodic diﬀerential equations. If small perturbations to these movements are corrected by the system, the movement is called stable and the set of these perturbations is called the basin of attraction of the movement. The basin of attraction is the appropriate quantity to describe how stable a movement is, since it measures, how large perturbations to the movement may be, which still are led back to the desired movement. The basin of attraction thus describes the self-stabilizing properties of the muscle-skeletal system without control mechanisms. If a human runs, it is important how uneven the ground may be before he either falls down or has to adjust his muscle activations to the new situation. Let us describe how the paper is organized: In Sect. 1 we present a mathematical model of a single human joint including antagonistic muscles. More

148

P. Giesl and H. Wagner

precisely, we introduce models of a human elbow joint and a human knee including muscular and outer forces as well as the geometry of the joint. In Sects. 2 and 3 we present diﬀerent mathematical methods to analyze the stability and the basin of attraction of a stationary or periodic movement, i.e. for autonomous and periodic systems, respectively. Some examples of these methods in biomechanical applications are given in this paper, and many more are presented in [13] within this volume.

1 Biomechanical Models In Sect. 1.1 we present a mathematical model of the human elbow joint, for more details cf. [7] and [9]. In Sect. 1.2 we describe the model of the human knee joint. Note that the models of other joints and other positions of the arms and legs can be derived in a very similar way. 1.1 A Model of the Elbow Joint Consider the following situation: the upper arm of a person is attached to the body, the elbow pointing downwards. The lower arm (ulna) is free to move in the sagittal plane so that the system is totally described by the angle β between upper and lower arm at the elbow joint. The person holds a load in the hand. Denoting the angular velocity ω = β˙ we obtain the following equation of motion: + β˙ = ω (1) ω˙ = J1 T (t, β, ω) =: f (t, β, ω) where J = 13 mu + ml l2 denotes the moment of inertia, mu and ml denote the mass of the ulna and of the load, respectively, and l is the length of the ulna. The torque T (t, β, ω) = To (t, β, ω) + Tm (t, β, ω) consists of two parts corresponding to the outer forces and the muscle forces. The outer forces include the gravitational forces acting on the arm and on the load in the hand. In the periodic case we assume that the person is walking which gives an additional term corresponding to the periodic vertical acceleration a(t) applied to the system. Altogether, To (t, β, ω) = 1 2 mu + ml (g + a(t)) l sin β, cf. [2]. In our model we consider only the three most important muscles at the elbow joint, namely the extensor muscle triceps brachii and the two ﬂexor muscles biceps and brachioradialis. For each muscle the torque is given by T (t, β, ω) = E(t) · fl (β) · H(β, ω) · h(β), where E(t) ∈ [0, 1] denotes the activation level of the muscle which is assumed to be constant or periodic with respect to t, fl denotes the dependency of the muscle on its length, H denotes the Hill-function modelling the dependency of the muscle on its velocity and h denotes the eﬀective moment arm, reﬂecting the geometry of the joint. We will discuss the force-length function fl and the Hill-function H in more detail.

Basin of Attraction for Stationary and Periodic Movements

149

Force-Length Function We model the force-length functions flbic of the biceps by (2), cf. [9], flbic (β) = f˜l (zbic (lbic (β))) , ˜ arctan[˜b(z − c˜)] + d˜ , f˜l (z) = a z0 [x − lbic (8π/9)] + 1 , zbic (x) = lbic (8π/9) − lbic (π/3)

(2) (3) (4)

˜ ˜ = 0.6, d˜ = 0.49. f˜l of (3) models the function dewhere a ˜ = 1.09 π , b = 15, c 2 2 + k 2 − 2k k cos β scribed in [11], β denotes the joint angle, lbic (β) = kH H U U the length of the muscle depending on personal data such as the distances between the elbow joint and the point where the tendon is attached to the upper and lower arm kH , kU , respectively. zbic denotes a normalized muscle length, cf. [7]. z0 is a muscle-dependent parameter, often z0 = 0.55. The formulas for M. brachioradialis are similar. The extensor muscle of the elbow joint is approximately working around the optimum muscle length. Therefore, the inﬂuence of the force-length relation can be neglected, i.e. we set flext (β) = 1. Hill-Function Hill [10] introduced the following force-velocity relation H(v) =

c − a for v ≥ 0 v+b

(5)

where v ≥ 0 denotes the velocity of the contraction of the muscle and a, b, c are person-dependent constants. The excentric part, i.e. v < 0 is modelled such that H is a C 2 -function, cf. [7] H(v) = A +

C B + for v < 0 v − D (v − D)2

(6)

where A, B, C, D are chosen such that H ∈ C 2 (R2 , R) and limv→−∞ H(v) = 1.5 · H(0), i.e. the muscle can generate 1.5 times the isometric force H(0) for very large negative velocities, cf. [7]. The Formula for T Note that the velocity of the muscle v is connected to the angular velocity ω by v = h(β) · ω, where h denotes the eﬀective moment arm, cf. [7]. Altogether, Tm is given by Tm (t, β, ω) = Eext (t) Hext [hext (β) · ω] hext (β) +Ef lex (t) flbic (β) Hbic [hbic (β) · ω] hbic (β) +Ef lex (t) flbrach (β) Hbrach [hbrach (β) · ω] hbrach (β)

(7)

150

P. Giesl and H. Wagner

Recall that E denotes the (periodic or constant) muscle activation, H the Hill-function, h the eﬀective moment arm and fl the force-length relation. The index denotes the ﬂexor muscles biceps and brachioradialis as well as the extensor muscle. Note that the activation levels of both ﬂexor muscles are assumed to be equal and are denoted by Ef lex (t). Plugging the formulas for To and Tm , cf. (7), in (1) we obtain the following system.  β˙ = ω   

1  1    ω˙ = J 2 mu + ml (g + a(t)) l sin β + Eext (t) Hext [hext (β) · ω] hext (β) (8) +Ef lex (t) flbic (β) Hbic [hbic (β) · ω] hbic (β)    +Ef lex (t) flbrach (β) Hbrach [hbrach (β) · ω] hbrach (β)     =: f (t, β, ω) Special Features For the mathematical analysis we hence study a system of diﬀerential equation of the following form + β˙ = ω (9) ω˙ = f (t, β, ω) where f (t, β, ω) is either (i) independent of t (autonomous system) or (ii) periodic with respect to t, i.e. there is a minimal period T > 0 such that f (t + T, β, ω) = f (t, β, ω) for all (t, β, ω). Note that the partial derivative of the Hill-function with respect to ω is strictly negative, i.e. Hω (β, ω) < 0 holds for all (β, ω) ∈ R2 . Hence, this also holds for f , i.e. fω (t, β, ω) < 0

(10)

for all (t, β, ω). Examples In this paper we study the following two examples: 1. Standing. We assume that the person is not moving (To is independent of t), and holds the weight at constant activation levels Eext and Ef lex (Tm is independent of t) at angle β0 . Thus, f = f (β, ω) is independent of t and Eext and Ef lex are chosen such that f (β0 , 0) = 0 holds, i.e. β(t) = β0 and ω(t) = 0 is a solution. 2. Walking. We assume that the person is walking in a periodic way (To is periodic with respect to t), and holds the weight at a constant level for a person watching from outside. An example for such a situation is a waiter carrying a tray while walking. The tray is supposed not to move, so that the elbow angle must compensate the movement the walking imposes on the shoulder and the arm. This is achieved by suitable periodic activation levels Eext (t) and Ef lex (t). In this case, f = f (t, β, ω) is a periodic function of t, cf. Sect. 3.

Basin of Attraction for Stationary and Periodic Movements

151

1.2 A Model of the Knee Joint We assume that the foot is ﬁxed to the ground, the upper and lower legs are connected by the knee joint, the body mass is concentrated in the hip and the hip can only move on a line perpendicular to the ground. The height of the hip is denoted by x, its velocity by v = x. ˙ In this case, our model includes one extensor and one ﬂexor muscle, and we set flext (x) = flf lex (x) = 1. We neglect the mass of the leg, and we set the Hill-function of the ﬂexor Hf lex (v) = fisof lex . We consider the following equations of motion, cf. [12]    x˙ = v v˙ = Eext (t)Hext [hext (x) · v] hext (x) + Ef lex (t)fisof lex hf lex (x) − mg (11)   =: f (t, x, v), where the eﬀective moment arms and the Hill-function of the extensor are modelled by cext hext (x) = aext + bext − x cf lex hf lex (x) = −af lex − bf lex − x Hext (u) = tanh[−(5.6 + 4.5 tanh(−u − 0.4))u] · fisoext (0.25 tanh(10u) + 0.75) + fisoext , where aext = 0.08, bext = 0.88, cext = 0.035, af lex = 0.01, bf lex = 0.87, cf lex = 0.04, fisoext = fisof lex = 21012. Note that this model approximates the eﬀective moment arms of [12], Fig. 6. These moment arms include a model of Menschik with moving center of rotation in the knee. The length are 44 cm of the thigh and 43 cm for the lower leg. We use this model in Sect. 3.6.

2 Autonomous Systems In this section we study equilibria and their stability of the autonomous system (12), which is (9) being independent of t. + β˙ = ω (12) ω˙ = f (β, ω) Whereas the stability of equilibria can often be checked by the eigenvalues, for the basin of attraction we use Lyapunov functions (Sect. 2.1). The following methods for their construction are discussed: via linearization (Sect. 2.2), using special properties of the equation (Sect. 2.3) and with radial basis functions (Sect. 2.4). Note that for the sake of simplicity the Theorems and Deﬁnitions in this paper are only stated for the two-dimensional phase space x = (β, ω) ∈ R2 , but they hold in most cases also for arbitrary dimensions, i.e. x ∈ Rn .

152

P. Giesl and H. Wagner

2.1 Basin of Attraction, Stability and Lyapunov Functions The equilibria of (12), i.e. solutions which are constant in time, are the zeros (β, ω) of the right-hand side vector ﬁeld of (12), i.e. ω = 0 and f (β, ω) = 0. Hence, equilibria are points (β0 , 0) which satisfy f (β0 , 0) = 0. For the muscle model this is achieved by suitable activations Eext and Ef lex . We also use the notation x˙ = F (x) , where x = (β, ω) and

F (x)=(

x2 ) f (x1 ,x2 )

(13)

for the equation (12).

The stability of such an equilibrium (β0 , 0) in the hyperbolic case can by studied by the eigenvalues of the Jacobian matrix of ﬁrst derivatives 0 1 , DF (β0 , 0) = fβ (β0 , 0) fω (β0 , 0) which are λ1,2

1 = 2

fω (β0 , 0) ±

1 fω (β0

, 0)2

+ 4fβ (β0 , 0)

if fω (β0 , 0)2 + 4fβ (β0 , 0) ≥ 0 (real eigenvalues) and 1 1 2 λ1,2 = fω (β0 , 0) ± i −fω (β0 , 0) − 4fβ (β0 , 0) 2 otherwise (complex eigenvalues). In both cases, the real parts of both eigenvalues are strictly negative, if and only if fβ (β0 , 0) < 0, since fω (β0 , 0) < 0 holds by (10). Hence, if fβ (β0 , 0) < 0, then the equilibrium is asymptotically stable. From now on we will assume f (β0 , 0) = 0 and fβ (β0 , 0) < 0. We seek to determine the basin of attraction of the asymptotically stable equilibrium (β0 , 0) =: x0 . Deﬁnition 1. The basin of attraction A(x0 ) of an asymptotically stable equilibrium x0 for the ordinary diﬀerential equation x˙ = F (x) is deﬁned by t→∞

A(x0 ) = {ξ ∈ R2 | x(t) −→ 0} where x(t) denotes the solution of x˙ = F (x) with initial value x(0) = ξ. A powerful method to determine the basin of attraction is the method of a Lyapunov function v : R2 → R. The main property of a Lyapunov function is that it is decreasing along solutions. v is decreasing along solutions, if and only if the orbital derivative v , i.e. the derivative of v along solutions of the diﬀerential equation, is negative. The formula for the orbital derivative follows from the chain rule

Basin of Attraction for Stationary and Periodic Movements

153

3 3 3 3 d (13) 3 3 v(x(t))3 = ∇v(x(t)), x(t) ˙ = ∇v(ξ), F (ξ) = v (ξ) . 3 dt t=0 t=0 Theorem 1 (Lyapunov function). Let x0 be an equilibrium of x˙ = F (x). Let v ∈ C 1 (R2 , R) be a function and K ⊂ R2 be a compact set with neighborhood B such that ◦

1. x0 ∈ K, 2. v (x) = ∇v(x), F (x) < 0 holds for all x ∈ K \ {x0 }, 3. K = {x ∈ B | v(x) ≤ R} with a constant R ∈ R. Then K ⊂ A(x0 ). We sketch the proof of Theorem 1: If we start at x(0) with v(x(0)) ≤ R, then the solution satisﬁes v(x(t)) ≤ R for all t ≥ 0 since v is decreasing along solutions, i.e. solutions stay in the set K of Theorem 1. Since the function v is bounded from below, v(x(t)) will tend to a constant value and the orbital derivative v (x(t)) will tend to 0. Thus, the solution tends to the only point where v (x) = 0 holds, namely to the equilibrium x0 , and x(0) belongs to the basin of attraction A(x0 ). Thus, level sets of v provide a tool to determine a subset of the basin of attraction. Although there are many existence theorems for Lyapunov functions, their explicit construction without knowledge of the solutions of x˙ = F (x) is a diﬃcult problem. Several methods for the calculation of Lyapunov functions are presented in the next sections. 2.2 Linearization We consider the linearized system at the equilibrium point x0 , namely x˙ = DF (x0 )(x − x0 ). This is a linear system and, thus, one can easily calculate a Lyapunov function of the form v(x) = (x − x0 )T C(x − x0 ), where the positive deﬁnite matrix C is the unique solution of the matrix equation DF (x0 )T C + CDF (x0 ) = −I. Note that the orbital derivative is given by v (x) = (x − x0 )T [DF (x0 )T C + CDF (x0 )](x − x0 ) = − x − x0 2 . The function v is not only a Lyapunov function for the linearized system, but also for the nonlinear system in a neighborhood of x0 , cf. Lemma 1. We call such a function a local Lyapunov function, since this neighborhood can be very small and thus the set K of Lemma 1 may be a very small subset of the basin of attraction A(x0 ). Thus, other methods will be discussed in the following sections. Lemma 1. Under the above assumptions on the Lyapunov function v(x) = (x − x0 )T C(x − x0 ) of the linearized system, there is a compact set K with ◦

neighborhood B such that x0 ∈ K, v (x) = ∇v(x), F (x) < 0 holds for all x ∈ K \ {x0 } and K = {x ∈ B | v(x) ≤ R} with R > 0.

154

P. Giesl and H. Wagner

2.3 Special Lyapunov Function In many applications, a Lyapunov function is calculated using special properties or physical insight into the system of diﬀerential equation considered. Also in our case one can use the special structure (12) and the information on the negative sign of fω , cf. (10), to obtain a Lyapunov function, cf. [9]. Theorem 2. Let f ∈ C 1 (R2 , R) satisfy fω (β, ω) < 0 for all (β, ω). Moreover, assume that (β0 , 0) is an equilibrium point of (12) with fβ (β0 , 0) < 0. Then β V (β, ω) = −

˜ 0) dβ˜ + 1 ω 2 f (β, 2

(14)

0

is a Lyapunov function such that 1. V (β, ω) < 0 for all (β, ω) with ω = 0, 2. V attains a local minimum at (β0 , 0). The integral in (14) can be calculated explicitly in the case of our model (8), cf. [9]. Using the Lyapunov function V we obtain again a subset S of the basin of attraction by Theorem 1 through level sets of V . Corollary 1. Let the assumptions of Theorem 2 hold. Let (β0 , 0) be an equilibrium with β0 ∈ (0, π). Moreover, let β0 < β1 < π be such that (β1 , 0) is an equilibrium. Set S := {(β, ω) | V (β, ω) < V (β1 , 0)} ∩ (0, π) × R If S is connected and compact, and f (β, 0) = 0 holds for all (β, 0) ∈ S \ {(β0 , 0)}, then S ⊂ A(β0 , 0). The set S is in general larger than the set K of Lemma 1. In particular, it covers the whole β-axis up to the next unstable equilibrium (β1 , 0). A similar Corollary holds also for an equilibrium (β2 , 0) with β2 < β0 . 2.4 Approximation via Radial Basis Functions A Lyapunov function is characterized by its negative orbital derivative. In this general approach to construct Lyapunov functions via radial basis functions we consider Lyapunov functions V1 and V2 with certain negative orbital derivatives, namely V1 (x) = − x − x0 2 deﬁned for x ∈ A(x0 ) and V2 (x) = −c < 0 deﬁned for x ∈ A(x0 ) \ {x0 }. These equations for the orbital derivatives are linear ﬁrst-order partial diﬀerential equations. The method seeks to ﬁnd an approximate solution vi , i = 1, 2 of the partial diﬀerential equation which is close enough to the solution Vi such that

Basin of Attraction for Stationary and Periodic Movements

155

the orbital derivative vi is negative. Then vi is itself a Lyapunov function and can be used to determine a subset of the basin of attraction by Theorem 1. The approximation is achieved using radial basis functions [8], which is a meshfree approximation method using a certain ansatz for the approximating function v. More precisely, for x, y ∈ R2 we set v(x) =

N j=1

3 3 αj ∇y Ψ (x − y), F (xj )33

(15) y=xj

where Ψ (x) = ψ( x ) denotes a ﬁxed radial basis function and XN = {x1 , . . . , xN } ⊂ R2 a grid. The coeﬃcients αj are chosen such that v satisﬁes the equation on the grid, i.e. v1 (xj ) = V1 (xj ) = − xj − x0 2 or v2 (xj ) = V2 (xj ) = −c for all j = 1, . . . , N . The explicit calculation of αj is easily achieved solving a system of linear equations. The density of the grid points used for the approximation determines the error of the approximation. The following error estimates are obtained, using Wendland’s functions as radial basis functions, cf. [14] and [8]. Theorem 3 shows that if the grid is dense enough, then vi (x) < 0 holds. Finally, by level sets of the function vi and Theorem 1, we can ﬁnd a subset of the basin of attraction. Note that we can modify the method by not only prescribing the values of vi on the grid XN but, additionally, the values of vi on another grid ΞM , e.g. on a level set of the local Lyapunov function. Thus, one can ﬁnd subsets of the basin of attraction which cover each compact set in the basin of attraction, cf. [8]. Theorem 3 (Error estimate). Consider x˙ = F (x), and let f ∈ C σ (R2 , R) where σ ≥ σ ∗ := 32 + k and k ∈ N denotes the parameter of the Wendland function. Let K be a compact set with K ⊂ A(x0 ) \ {x0 }. There is a constant c∗ such that for all grids XN := {x1 , . . . , xN } ⊂ K with ﬁll distance h in K (i.e. h := supx∈K inf xj ∈XN x − xj ) |v (x) − V (x)| ≤ c∗ hκ holds for all x ∈ K ,

(16)

where κ = 12 for k = 1 and κ = 1 for k ≥ 2. v ∈ C 2k−1 (R2 , R) is the approximation of V satisfying V (x) = − x − x0 2 or V (x) = −c with respect to the grid XN using Wendland’s function as radial basis function, i.e. Ψ (x) = ψl,k (µ x ), µ > 0 and l := k + 2. The estimate (16) in Theorem 3 implies v (x) ≤ V (x) + c∗ hκ ≤ − x − x0 2 + c∗ hκ for V (x) = − x − x0 2 . If the grid is dense enough (i.e. the ﬁll distance h is small enough), then v (x) < 0 holds except for a small neighborhood of x0 . The same is true for the approximation of the function V (x) = −c since this function is not deﬁned at x = x0 . This means, that we have to solve the problem diﬀerently near x0 : here we consider some local Lyapunov function, e.g. the Lyapunov function of Sect. 2.2, in a neighborhood of x0 . Details of

156

P. Giesl and H. Wagner

the method can be found in [8] and [4]. An application of the method to the elbow joint is given in [3]. Note that the error estimate of this method requires a certain smoothness of the function F . In particular, the minimal requirement is F ∈ C 3 (R2 , R) with the parameter of Wendland’s function k = 1. However, the Hill-function is in general not C 3 at zero, the turning point from concentric to excentric c − a for behaviour. The Hill-function is assumed to be of the form H(v) = v+b v ≥ 0 (concentric part, cf. (5)). It can be modelled in a smooth way for v < 0: for H ∈ C 2 cf. (6), and for H ∈ C 3 cf. [3]. Even if one uses a Hill-function model which is only C 1 or C 2 at zero, the method works in examples, but the error estimates cannot be proved, cf. [3] for examples and further discussion.

3 Periodic Systems In this section we study periodic movements which lead to time-periodic diﬀerential equations. We deﬁne the basin of attraction for periodic orbits (Sect. 3.1). In order to check their stability one can use Floquet theory (Sect. 3.2). To ﬁnd a subset of their basin of attraction one seeks to ﬁnd a Lyapunov function, e.g. one of the above Lyapunov functions of an adjacent autonomous system (Sect. 3.3), which can be extended by radial basis functions (Sect. 3.4). An alternative method to the use of Lyapunov functions is Borg’s criterion, where the exact position of the periodic movement is not required (Sect. 3.5), cf. [6]. Finally, Borg’s method and Floquet theory are applied to periodic movements of the human knee (Sect. 3.6). 3.1 Basin of Attraction and Lyapunov Functions We consider the time-periodic system x˙ = F (t, x) ,

(17)

where F (t + T, x) = F (t, x) for all t ≥ 0 and T > 0 is minimal with this property. The simplest solution of (17) is a periodic orbit Ω = {(t, x ˜(t)) ∈ ˜(t) is a solution of (17) with x ˜(t + T ) = x ˜(t) for all t ≥ 0 ST1 × R2 }, where x and ST1 denotes the circle of radius T . The phase space is now the cylinder ST1 × R2 . Deﬁnition 2. The basin of attraction A(Ω) of the periodic orbit Ω = {˜ x(t) ∈ ST1 × R2 }, where x ˜(t) is a solution of the time-periodic ordinary diﬀerential equation x˙ = F (t, x) is deﬁned by t→∞

A(Ω) = {(τ, ξ) ∈ ST1 × R2 | x(t) − x ˜(t) −→ 0} , where x(t) denotes the solution of x˙ = F (t, x) with initial value x(τ ) = ξ.

Basin of Attraction for Stationary and Periodic Movements

157

The basin of attraction can again be calculated using a Lyapunov function v ∈ C 1 (ST1 ×R2 , R). A Lyapunov function is still a function which is decreasing along solutions, i.e. a function with negative orbital derivative. Note, however, that the orbital derivative v (τ, ξ) has now the following expression: 3 3 d v(t, x(t))33 v (τ, ξ) = dt t=τ 3 3 3 = ∇x v(t, x(t)), x(t) ˙ + vt (τ, ξ) 3 t=τ

(17)

= ∇x v(τ, ξ), F (τ, ξ) + vt (τ, ξ) .

If v (τ, ξ) < 0, then level sets of v provide a tool to determine a subset of the basin of attraction. Note that the following theorem, similarly to the corresponding Theorem 1 in the autonomous case, does not make any assumptions about the stability of x ˜(t). Theorem 4 (Lyapunov function, periodic case). Let Ω = {(t, x ˜(t)) ∈ ST1 × R2 } be a periodic orbit of x˙ = F (t, x). Let v ∈ C 1 (ST1 × R2 , R) be a function and K ⊂ ST1 × R2 be a compact set with neighborhood B such that ◦

1. Ω ⊂ K, 2. v (t, x) = ∇x v(t, x), F (t, x) + vt (t, x) < 0 holds for all x ∈ K \ Ω, 3. K = {(t, x) ∈ B | v(t, x) ≤ R} with a constant R ∈ R. Then K ⊂ A(Ω). 3.2 Stability and Floquet Theory We perform a transformation of the system (17) which transforms the periodic solution x ˜(t) to the zero solution, i.e. (t, y) := (t, x − x ˜(t)), and consider the transformed system y˙ = G(t, y) ,

(18)

where G(t, y) = F (t, y + x ˜(t)) − x ˜˙ (t). y(t) = 0 is the (periodic) solution of (18) which corresponds to the periodic solution x ˜(t) of (17), and its stability is the same as the stability of x ˜(t) with respect to (17). In order to study the stability, we consider again the linearization of (18) near the periodic solution. However, the stability cannot be checked as easily as in the autonomous case; one has to use Floquet theory. Let us ﬁrst consider a linear system of the form y˙ = G(t)y , where G(t + T ) = G(t) is a (2 × 2)-matrix for each t.

(19)

158

P. Giesl and H. Wagner

Theorem 5 (Floquet). Each fundamental matrix X(t) of (19) has a representation of the form X(t) = P (t)eBt where P (t + T ) = P (t) is a t-periodic, (2 × 2)-matrix-valued function and B is a (2 × 2)-matrix. The eigenvalues of B are called Floquet exponents. If the real part of all Floquet exponents is negative, then the zero solution of (19) is asymptotically stable. Hence, the Floquet exponents play a similar role for the determination of the asymptotic stability as the eigenvalues of the Jacobian matrix evaluated at the equilibrium in the autonomous case. Now we return to the nonlinear problem y˙ = G(t, y). Theorem 6 (Stability, periodic case). Consider (18) and assume that = 0 for all t ∈ ST1 . If the G(t, y) = G(t)y + H(t, y), where limy→0 H(t,y) y real part of all Floquet exponents of the linearized equation y˙ = G(t)y is negative, then the zero solution of (18) is asymptotically stable. Hence, in order to determine the stability one has to ﬁnd the linearization of the system, i.e. G(t), and in the next step one has to calculate the matrix B and its eigenvalues for the linearized system y˙ = G(t)y. This is done numerically for the example of two periodic knee movements, cf. Sect. 3.6 and Table 1. 3.3 Linearization of an Adjacent Autonomous System We seek to construct a Lyapunov function for the time-periodic system y˙ = G(t, y). In most examples the determination of the linearized system y˙ = G(t)y and thus also the local Lyapunov function can only be obtained numerically. In this section, however, we consider the special situation that the time-periodic system is “near” to an autonomous system and we use the local Lyapunov function of the autonomous function as local Lyapunov function of the periodic system. More precisely, we assume in this section that there is a parameter λ ∈ R such that ˜ y, λ), where G(t, ˜ y, 0) is autonomous (independent of t) y˙ = G(t, y) = G(t, and |λ| is small; the precise conditions are summarized in Proposition 1. In Sect. 2.2 we have used a Lyapunov function of the linearized system which turned out to be a local Lyapunov function also for the nonlinear system. The idea for time-periodic systems in this section is to consider the adjacent autonomous system and then to use the Lyapunov function V for the linearized autonomous system. The following proposition, cf. e.g. [2], shows that this local Lyapunov function V is a Lyapunov function for the original time-periodic system in a neighborhood of the zero solution, provided that |λ| is small enough.

Basin of Attraction for Stationary and Periodic Movements

159

Proposition 1. Consider the system y˙ = a(y) + b(t, y, λ)

(20)

where a ∈ C 1 (R2 , R2 ), b ∈ C 1 (ST1 × R2 × R, R2 ) and the following properties hold: 1. a(0) = 0, 2. b(t, 0, λ) = 0 for all t ∈ ST1 and λ ∈ R, 3. b(t, y, 0) = 0 for all t ∈ ST1 and y ∈ R2 . Then y = 0 is a solution of (20) for all λ ∈ R. Let V (t, y) = V (y) = y T Cy be the local Lyapunov function of the autonomous linearized system y˙ = Dy a(0)y according to Lemma 1. If |λ| is small enough, then V (t, y) < 0 holds for all (t, y) ∈ ST1 × R2 with 0 < y ≤ δ, where δ = δ(λ). Conditions 1 and 2 ensure that 0 is a solution for all λ ∈ R and Condition 3 shows that for λ = 0 the system is autonomous. Note that this Lyapunov function is independent of t and thus level sets of V are cylinders in the space ST1 × R2 . Since this function is only a local Lyapunov function, i.e. the orbital derivative is negative only in a small neighborhood y ≤ δ of the zero solution, we seek to ﬁnd a Lyapunov function for a larger set using radial basis functions. 3.4 Approximation via Radial Basis Functions As in Sect. 2.4 we seek to construct a Lyapunov function for the zero solution Ω := {(t, 0) | t ∈ ST1 } by approximation via radial basis functions, cf. [2]. In order to use a similar approach as in Sect. 2.4 we add a diﬀerential equation, namely t˙ = 1, and study the following (autonomous) three-dimensional problem + t˙ = 1 y˙ = G(t, y). Note that the periodicity is reﬂected by (t, y) ∈ ST1 × R2 . Similarly to Sect. 2.4 we consider Lyapunov functions with certain negative orbital derivatives, namely V (t, y) = − y 2 deﬁned for (t, y) ∈ A(Ω), which is a linear partial diﬀerential equation. The method seeks to ﬁnd an approximate solution v of this partial diﬀerential equation which is close enough to the solution V such that the orbital derivative v is negative. Then v is itself a Lyapunov function and can be used to determine a subset of the basin of attraction. Fix a radial basis function Ψ (t, y) where (t, y) ∈ R3 with compact support, e.g. Wendland’s function cf. [14], and a grid XN = {η1 , . . . , ηN } with ηi = (ti , yi ) ∈ [0, T ) × R2 . The ansatz for the approximating function v(t, y) is then

160

P. Giesl and H. Wagner

N

4 αj

∇(τ,z) Ψ (t − τ, y − z),

m∈Z j=1

1 G(tj + mT, yj )

5 3 3 3 3

(τ,z)=(tj +mT,yj )

Note that the summation over m is zero except for ﬁnitely many m due to the compact support of Ψ and reﬂects the fact that t ∈ ST1 . In particular, by this ansatz v(t + T, y) = v(t, y) holds for all t ≥ 0. The coeﬃcients αj are chosen such that v satisﬁes the equation on the grid XN , i.e. v (tj , yj ) = V (tj , yj ) = − yj 2 holds for all j = 1, . . . , N . The explicit calculation of αj is achieved solving a system of linear equations. Again, we have to solve the problem diﬀerently near y = 0: here we consider some local Lyapunov function, e.g. the Lyapunov function of Sect. 3.3, in a neighborhood of 0. Details of the method and an application to the elbow joint are given in [2]. Note that also in the time-periodic case a modiﬁcation of the method such that also the function values of v are prescribed on a level set of the local Lyapunov function or on (t, y) = (t, 0) improve the results. The ansatz is then altered to a mixed ansatz, cf. [2]. An application to biomechanics is also found in [2]. We consider a person who walks and assume that the walking induces a displacement of the shoulder and the upper arm of the form λ sin νt. Hence, the acceleration acting on the system corresponding to an outer force is given by a(t) = −λν 2 sin νt. Hence the equation of motion reads, cf. (8)  β˙ = ω   

1  1 2    ω˙ = J ( 2 mu + ml ) · (g − λν sin νt) · l · sin β +Eext (t)Hext [hext (β) · ω] hext (β)    +Ef lex (t)flbic (β)Hf lex [hbic (β) · ω] hbic (β)     +Ef lex (t)flbrach (β)Hf lex [hbrach (β) · ω] hbrach (β) We are interested in a solution where the position of the hand is constant relatively to the ground. This periodic solution is given by β ∗ (t) = arccos(− λl sin νt). The example studied in [2] assumes a constant ﬂexor activation Eext = 0.7. The activation Ef lex (t) is chosen such that β ∗ (t) is a solution of the system. In this example only a small amplitude λ = 0.5 cm is considered, so that the construction of the local Lyapunov function as in Sect. 3.3 succeeds. The other parameters are ν = 2π and ml = 5 kg. 3.5 Borg’s method In this section we apply a generalization of Borg’s method, cf. [1], to the problem of the determination of the basin of attraction of a periodic orbit of a general time-periodic system (17), cf. [6]. In contrast to the method of Lyapunov functions, the knowledge of the position of the periodic orbit x˜(t) is not required. However, we will see, that it can be helpful also for this method.

Basin of Attraction for Stationary and Periodic Movements

161

Borg’s condition means that two adjacent solutions approach each other with respect to a Riemannian metric. This can easily be checked by the sign of a certain function LM (t, x), cf. (21) of Theorem 7. Deﬁnition 3. The matrix-valued function M ∈ C 1 (ST1 × R2 , R2×2 ) will be called a Riemannian metric, if M (t, x) is a symmetric and positive deﬁnite matrix for each (t, x) ∈ ST1 × R2 . The Riemannian metric M (t, x) deﬁnes a point-dependent scalar product through v, w(t,x) = v T M (t, x)w. The usual Euclidean metric is obtained by M (t, x) = I. A set is called positively invariant, if solutions starting in this set, remain in the set for all positive times. Deﬁnition 4. A set K ⊂ ST1 × R2 is called positively invariant if for all (τ, ξ) ∈ K, we have (t, x(t)) ∈ K for all t ≥ 0, where x(t) denotes the solution of the initial value problem x˙ = F (t, x), x(τ ) = ξ. Theorem 7. (cf. [6]) Consider the equation x˙ = F (t, x), where F ∈ C 1 (ST1 × R2 , R2 ). Let ∅ = K ⊂ ST1 × R2 be a connected, compact and positively invariant set. Let M be a Riemannian metric in the sense of Deﬁnition 3. Moreover, assume LM (t, x) < 0 for all (t, x) ∈ K, where LM (t, x) :=

max

w∈R2 ,wT M (t,x)w=1

LM (t, x; w)

& % 1 LM (t, x; w) := wT M (t, x)Dx F (t, x) + M (t, x) w, 2

(21) (22)

and M (t, x) denotes the matrix with entries ∂Mij (t, x) ∂Mij (t, x) mij = + Fk (t, x) ∂t ∂xk 2

k=1

which is also the orbital derivative of M (t, x). Then there exists one and only one periodic orbit Ω ⊂ K, which is exponentially asymptotically stable. Moreover, for its basin of attraction K ⊂ A(Ω) holds, and the largest real part −ν0 of all Floquet exponents of Ω satisﬁes −ν0 ≤ −ν := max LM (t, x) . (t,x)∈K

For the proof one considers two solutions x(t) and y(t) with adjacent initial values and deﬁnes their time-dependent distance with respect to the 1/2

. Since Riemannian metric A(t) := (y(t) − x(t))T M (t, x(t))(y(t) − x(t)) LM (t, x) < 0, the solutions approach each other and the distance A(t) decreases exponentially. Thus, all solutions in K have the same ω-limit set characterizing the long-time behaviour, since K is connected. The study of a Poincar´e-map shows that this ω-limit set is a periodic orbit.

162

P. Giesl and H. Wagner

The application of the method poses similar problems as in the case of Lyapunov functions: it is known that for each exponentially asymptotically stable periodic orbit there exists a Riemannian metric such that (21) holds, cf. [6], but there is no general construction method available. Moreover, we have to ﬁnd a positively invariant set K. Assume now again that the periodic orbit is known, which is the case in many biomechanical applications. There are two cases, where [6] gives suggestions for a Riemannian metric: (i) if F (t, x) is “near” to an autonomous system – then one can use the metric of the autonomous system; this is sim˜(t)) are ilar to the approach of Sect. 3.3. (ii) if the eigenvalues of Dx F (t, x real, distinct and negative for all t ∈ ST1 . In this case, denote by S(t) a smooth matrix-valued function on ST1 , such that its columns are eigenvectors ˜(t)) for each t. Note that multiplication with S(t) corresponds to a of Dx F (t, x transformation such that the eigenvectors are the new coordinate axes. Then M (t) = (S −1 (t))T S −1 (t) is Riemannian metric. In this case (21) and (22) turn out to have the following form, cf. [6]

−1 T −1 S max w (t)D F (t, x)S(t) + (S ) (t)S(t) w , (23) LM (t, x) = x 2 w∈R ,w=1

where (S −1 ) (t) denotes the orbital derivative of S(t). Formulas for the evaluation of the maximum in (23) are available, cf. [6]. Moreover, since the Riemannian metric only depends on t, we can easily determine a positively invariant set. Proposition 2. Let Ω = {(t, x ˜(t)) ∈ ST1 × R2 } be a periodic orbit and let M (t) be a Riemannian metric which only depends on t. Moreover assume LM (t, x) < 0 for all (t, x) ∈ K r where r > 0 and ˜(t)]T M (t)[x − x ˜(t)] ≤ r2 } . K r := {(t, x) ∈ ST1 × R2 | [x − x Then Ω is exponentially asymptotically stable, and K r ⊂ A(Ω) holds. If LM is negative for all points of the periodic orbit, then Theorem 7 implies that the periodic orbit is exponentially asymptotically stable and we obtain an upper bound for the largest real part of all Floquet exponents. Since then LM is negative also in a neighborhood of the periodic orbit, we can determine a subset of its basin of attraction using Proposition 2. 3.6 Example: Periodic Movements of the Knee For the example of a human knee-joint, cf. Sect. 1.2, we consider the periodic solution x ˜(t) := x0 + ∆x sin(t) , ¨˜(t) we have where x0 := 0.77 and ∆x := 0.07. With v˜(t) := x ˜˙ (t) and a ˜(t) := x the following formula for the extensor activation Eext (t) as a function of the (given) ﬂexor activation Ef lex (t)

Basin of Attraction for Stationary and Periodic Movements

163

–1

0.9 –1.2

0.8

–1.4

0.7

–1.6

0.6

–1.8 –2

0.5

–2.2

0.4 –2.4

0.3 –2.6

0

1

2

3

4

5

6

0

1

2

t

3

4

5

6

t

Fig. 1. (i) Uniform activation. Left: the activation functions Ef lex (t) (prescribed sine-function, black) and Eext (t) (calculated using (24), grey). Right: the function ˜(t)). The maximum value is −0.95 LM (t, x

Eext (t) =

x(t)) mg + a ˜(t) − Ef lex (t)fisof lex hf lex (˜ H[hext (˜ x(t))˜ v (t)]hext (˜ x(t))

(24)

to ensure that x ˜(t) is a solution of (11). We consider a ﬂexor activation of the following form: Ef lex (t) := −d1 sin(t + d2 cos(t)) + d3 , where • Example (i): d1 = 0.35, d2 = 0 and d3 = 0.6 (cf. Fig. 1 – uniform activation) • Example (ii): d1 = 0.3, d2 = 1.2 and d3 = 0.6 (cf. Fig. 2 – non-uniform activation). The Jacobian of the right-hand side at time t is given by 0 1 D(x,v) F (t, x ˜(t), v˜(t)) = fx (t, x ˜(t), v˜(t)) fv (t, x ˜(t), v˜(t)) 0 1 =: fx (t) fv (t) and its eigenvalues at time t are 2 1 λ1,2 (t) = fv (t) ± fv (t)2 + 4fx (t) . 2 The eigenvalues are real, distinct and negative, if and only if  fv (t) < 0   fx (t) < 0   2 fv (t) + 4fx (t) > 0

164

P. Giesl and H. Wagner

0.9

–0.5 0.8

–1

0.7

0.6

–1.5

0.5

–2

0.4

–2.5 0.3 0

1

2

3

4

5

6

0

1

2

3

4

5

6

t

t

Fig. 2. (ii) Non-uniform activation. Left: the activation functions Ef lex (t) (prescribed, black) and Eext (t) (calculated using (24), grey). Right: the function ˜(t)). The maximum value is −0.12 LM (t, x

holds for all t. Note that fv (t) < 0 holds for all t because of the property Hu (u) < 0 of the Hill-function. For both examples (i) and (ii) these conditions are satisﬁed for all t. Fig˜(t)), ures 1 and 2 show the activation functions and the functions LM (t, x which turn out to be negative for all times. Hence, the periodic orbit is asymptotically stable for both examples by Theorem 7. The uniform activation, example (i), gives an upper bound of −0.95 for the largest real part of the Floquet exponents, whereas the example (ii) gives an upper bound of −0.12. We also calculate the Floquet exponents numerically, cf. Sect. 3.2, which takes a considerably longer time, but gives a sharper result on the largest real part of the Floquet exponents: again, the uniform activation (i) leads to a more negative Floquet exponent than the activation in example (ii), cf. Table 1. Thus, a uniform activation seems to be favourable concerning the stability. Table 1. The largest real part of the Floquet exponents for the two examples. Borg’s method A. provides an upper bound for the largest real part of the Floquet exponents. The numerical calculation B. gives an approximation of the exact value of the largest real part of all Floquet exponents. However, it requires a large number of time-steps corresponding to a division of the interval [0, 2π] to calculate the matrix P (t) and B, cf. Sect. 3.2 Example (i) Uniform (ii) Non-uniform

A. Borg’s method −0.95 −0.12

B. numerical calculation steps −3.43 −1.60

37670 55972

Basin of Attraction for Stationary and Periodic Movements

165

0.06

0.04

0.06 0.02

0.04 0.02 v

v

0

0 –0.02

–0.02 –0.04

6

–0.04

5

–0.06

4 0.84

3t 0.82

0.8

–0.06

2

0.78

0.76 x

0.74

1 0.72

0.7 0

0.7

0.72

0.74

0.76

0.78

0.8

0.82

0.84

x

Fig. 3. A subset of the basin of attraction for example (i), uniform activation. Left: (t, x, v)-space. Right: projection on the (x, v)-plane

Using Proposition 2 we calculate subsets of the basin of attraction for the uniform activation, example (i), shown in Fig. 3 in the (t, x, v)-space and projected to the (x, v)-space. The subset obtained with this method, however, is very small. A numerical calculation of solutions with diﬀerent initial conditions shows that the real basin of attraction is much larger and contains the whole part of the phase space shown in Fig. 3. The advantage of Borg’s method, however, is that the conditions can be checked easily and fast, whereas the calculation of many solutions with diﬀerent initial conditions requires much more time.

4 Conclusion In this paper we presented mathematical models of the human elbow and knee joint. We discussed diﬀerent methods to analyze the stability and the basin of attraction of stationary and periodic movements. These mathematical methods serve as tools to answer biological questions as posed in the introduction. The properties of the muscles and the geometry of the joints which are responsible for the stability are the positive slope of the force-length function (elbow joint, cf. [7]) and the moving center of rotation (knee joint, cf. [12]). A high co-activation also stabilizes the system, cf. [7] and [5]. The stability of stationary movements depends on the position of the joint angle: for the elbow small angles are stable, whereas large angles are unstable. The dependency on the angle and the co-activation is also reﬂected in the size of the basin of attraction, cf. [13]. For periodic movements, general answers to these questions are more diﬃcult to obtain, but we hope that the methods presented in this paper serve to analyze periodic movements in more detail in the future. For an

166

P. Giesl and H. Wagner

analysis of the self-stabilizing properties of biological systems with emphasis on the biological reasons and implications, cf. also [13].

References [1] Borg G. (1960). A condition for the existence of orbitally stable solutions of dynamical systems. Kungl. Tekn. H¨ ogsk. Handl. 153:1–12 [2] Chen J. (2004). Stability of the movements of the elbow joint. MSc Thesis, Technical University of Munich [3] Ernst M. (2005). Analyse der Stabilit¨ at und der Einzugsbereiche nichtlinearer dynamischer Systeme am Beispiel eines Armmodells. Diploma Thesis, University of Jena [4] Giesl P. (2004). Approximation of domains of attraction and Lyapunov functions using radial basis functions. In: Proceedings of the NOLCOS 2004 conference Stuttgart, Germany vol. II, 865–870 [5] Giesl P. (2003). The optimal elbow angle for acrobatics – stability analysis of the elbow with a load. In: PAMM Proc. Appl. Math. Mech. 3:68–71 [6] Giesl P. (2004). Z. Anal. Anwendungen 23(3):547–576 [7] Giesl P., Meisel D., Scheurle J., Wagner H. (2004). Journal of Theoretical Biology 228(1):115–125 [8] Giesl P. (2004). Construction of Lyapunov functions using radial basis functions. Habilitation Thesis, Technical University of Munich [9] Giesl P., Wagner H. (submitted) Lyapunov functions and the basin of attraction for a single-joint muscle-skeletal model [10] Hill A. (1938). Proc Royal Soc. (B) London 126:136–195 [11] Murray W., Buchanan T., Scott D. (2000). J. Biomech. 33:943–952 [12] Wagner H. (1999). Zur Stabilit¨ at der menschlichen Bewegung. PhD Thesis, J.W. Goethe-Universit¨ at, Frankfurt a.M. [13] Wagner H., Giesl P. (2006). Self-stability in Biological Systems, this volume [14] Wendland H. (1998). J. Approx. Theory 93:258–272

Task-Level Control of the Lateral Leg Spring Model of Cockroach Locomotion J. Lee1 , A. Lamperski2 , J. Schmitt3 , and N. Cowan1 1

2

3

Johns Hopkins University, Baltimore, MD {jsl,ncowan}@jhu.edu California Institute of Technology, Pasadena, CA [email protected] Oregon State University, Corvallis, OR [email protected]

Summary. The Lateral Leg Spring model (LLS) was developed by Schmitt and Holmes to model the horizontal-plane dynamics of a running cockroach. The model captures several salient features of real insect locomotion, and demonstrates that horizontal plane locomotion can be passively stabilized by a well-tuned mechanical system, thus requiring minimal neural reﬂexes. We propose two enhancements to the LLS model. First, we derive the dynamical equations for a more ﬂexible placement of the center of pressure (COP), which enables the model to capture the phase relationship between the body orientation and center-of-mass (COM) heading in a simpler manner than previously possible. Second, we propose a reduced LLS “plant model” and biologically inspired control law that enables the model to follow along a virtual wall, much like antenna-based wall following in cockroaches.

1 Introduction For decades, researchers have posited low-dimensional spring-mass models to describe the COM dynamics and ground reaction forces in a broad variety of running animals [2, 4, 9, 11, 12, 19]. In order to understand the complex body mechanics of running animals, they have simpliﬁed the problem by decoupling the mechanics into the sagittal and horizontal planes. For animals whose locomotion occurs primarily in the sagittal plane, the locomotion dynamics have been modeled as a spring-loaded inverted pendulum (SLIP) [2, 16, 26, 27]. Insects, whose motion occurs primarily in the horizontal plane, have dynamics that have been approximated by a lateral leg spring (LLS) model [23, 24]. Results of the LLS suggest that the mechanical structure of an insect may be used to produce stable periodic gaits when running at high speeds, without relying solely on proprioceptive reﬂexes and detailed neural feedback for stability.

168

J. Lee et al.

The LLS models insect locomotion, speciﬁcally that of the cockroach Blaberus discoidalis. Cockroaches run using an alternating tripod gait [4]. Experiments have shown that the forces produced by this tripod of legs can be well represented by a single eﬀective leg [10, 15]. Since the total mass of the legs of the insect is less than 6% of the total mass, the LLS model approximates each alternating tripod as a single massless, spring-loaded virtual leg that attaches to the midline of the body at a point called the center of pressure (COP). As illustrated in Fig. 1, the COP is oﬀset from the center of mass (COM) by a displacement, d, where d may lie in front of the COM (d > 0) or behind the COM (d < 0). The model assumes that the foot pivots freely without slipping about its attachment to the ground, rfoot , and that the leg can rotate freely about the COP. This implies that no moments about the foot or COP can be generated, and forces will be applied to the body along the length of the leg. A full stride for the model consists of a left and right step phase. A step phase begins with the relaxed spring extended at an angle ±β0 with respect to the body centerline. The body moves forward, compressing and extending the elastic spring, until the spring returns to its original length, at which point the leg is lifted, the next leg is placed down, and the cycle repeats. Changes in the foot placements between left and right step phases result in a hybrid dynamical system. Systems with piecewise-holonomic constraints such as these can display asymptotic stability [21]. For gaits encountered in the LLS model, periodic motions exhibit neutral eigendirections due to energy conservation and SE(2) invariance. Therefore, stability is partially asymptotic in the sense that perturbations to periodic orbits in the direction of the eigenvectors of conserved quantities and symmetries do not grow or decay, but simply result in movement to a diﬀerent, stable gait. Gaits in the LLS model display partial asymptotic stability in the heading direction and angular velocity as a result of angular momentum trading between left and right step phases. The mechanical structure of the model therefore self-stabilizes the locomotion system [23]. If d < 0 then the gaits are asymptotically stable in heading and angular velocity, i.e. the body approaches straight trajectories if the trajectory begins in the basin of attraction for the stable periodic orbit. If d = 0, the periodic orbits exhibit neutral stability in angular velocity and asymptotic stability in heading. If d > 0, periodic orbits are unstable. To show stability, one takes Poincar´e sections at the beginning of a full stride, and numerically approximates the ﬁxed points and eigenvalues of the linearized return map. While the energetically conservative ﬁxed and moving center of pressure models of [23, 24] reproduce many salient features of the kinematics and forces exhibited experimentally by Blaberus discoidalis, detailed comparisons illuminate limitations of the LLS. In particular, the ﬁxed COP models previously investigated consider only COPs on the fore-aft body axis, and consequently only produce sinusoidal variations in θ; in contrast, the animal produces cosinusoidal variations [28]. This is due to the fact that a ﬁxed COP located

Task-Level Control of the LLS Model of Locomotion

169

behind the center of mass along the fore-aft body axis is only capable of producing a positive or negative moment, rather than a moment that changes sign during each step phase. Additionally, fore-aft and lateral force proﬁle magnitudes are typically reversed in comparison to those observed experimentally. Allowing the leg attachment point to vary from fore to aft in the moving COP model serves to address the qualitative discrepancies in the moment and yawing proﬁles. However, while qualitatively correct yaw and moment proﬁles are produced by the model, quantitative comparisons reveal that the variations in each remain an order of magnitude smaller than those observed experimentally. An activated muscle model introduced by Schmitt and Holmes [25] attempts to correct the moment and yawing oscillations by introducing hip torques and muscle activation. While these authors obtained correct moment proﬁles in this manner, they are obtained at the expense of increased model complexity and inverted fore-aft force proﬁles. The goal of this paper is to modify the LLS model to better match the actual cockroach, with as few parameters as possible, and to extend it to serve as a plant model for control. To compare our model to the previous LLS, we consider features salient to cockroach locomotion, such as stability, body motion kinematics, forces and moments, stride frequency, etc. For control, we use a biologically inspired antenna-like measurement [6, 7, 8], and show numerically that the closed-loop system dynamics asymptotically track a virtual wall in the environment. In addition, the controller maintains the LLS model’s energy conserving nature.

2 Dynamics and Simulation of an Enhanced LLS Model The goal of this research is to control the LLS model from step-to-step to achieve a locomotion objective such as following along a wall and avoiding obstacles in a planar environment. Using a controlled form of the LLS as a “plant model” may provide insights into our longer term objective of controlling a legged robot such as RHex [1], Sprawl [5], or Whegs [20]. It is known, for example, that RHex exhibits a dynamically similar gait in both the sagittal and horizontal planes to a cockroach. Toward that end, this section explores the eﬀects of COP placement and movement on the steady-state dynamics of the LLS model. The goal is to uncover the simplest possible mechanism to match biological data, while still providing the possibility for control. 2.1 LLS Dynamics with 2-D COP Placement for a Left Step We propose an alternative (or a simpler) solution to the moving COP; laterally oﬀset the ﬁxed COP (i.e. position the COP in the positive x-direction of the body frame {B}). This has a similar eﬀect as the moving COP scheme; the leg generates a clockwise torque during the ﬁrst half of a step, and an opposite

170

J. Lee et al.

θ

{F}

q3

M ν ψ

ζ

COM

η

d β

rfoot {U}

fL

{B} δ

rcom

M

q2 fR

fL

COP

M fL q0

q1

y x {U}

Fig. 1. Left: A schematic model of the LLS model, showing the coordinates used for expressing Hamilton’s equations. Right: Illustration of multi-step dynamics. The dynamics of each left step phase are given by (4). A right step can be achieved by ﬁrst ﬂipping the states about the y-axis, integrating the left step map, and then ﬂipping back. Breaking this chain in the correct place leads to a single “integrate and ﬂip” return map, f (q) := M fL (q), that will simplify controller design

torque during the last half, assuming the body angle, θ, is greater than zero at the start of a left-leg step. In order to validate our alternative solution, we represent the position of the COP during the left-leg step as: / . / . b1 + c1 (ψ − θ) d1 , (1) = b2 + c2 (ψ − θ) d2 where d1 and d2 are along the x and y-axis of the LLS body frame {B}, and ψ and θ are shown in Fig. 1. In this representation, we allow the COP to be either ﬁxed (c1 = c2 = 0) or moving (c1 = 0 or c2 = 0) from any oﬀset (b1 and b2 ) in the body frame {B}. This freedom allows us to test diﬀerent COP placement protocols, including the case where the COP moves backwards while oﬀset to the side [28]. This representation implicitly assumes that during the next right-leg step, the COP position will be mirrored about the y-axis about the body frame {B}. If d1 = 0, then we have the equation introduced in [23]. Consider the generalized coordinates r = (ζ, ψ, θ), as depicted in Fig. 1. For the left step phase, the Hamiltonian of the LLS system implemented with a linear spring is H=

p2ψ p2ζ k(η − l0 )2 p2θ + + + 2m 2mζ 2 2I 2

(2)

where ζ, ψ, k, l0 , I, and m denote the distance from the foot placement to the COM, the angle from the foot placement to the COM, the linear spring stiﬀ-

Task-Level Control of the LLS Model of Locomotion

171

ness, the relaxed leg length, the moment of inertia, and the mass, respectively. The length of the leg is given in terms of the COP location by η = b21 + b22 + ζ 2 + φ 2b1 c1 + 2b2 c2 + (c21 + c22 )φ 1/2 . (3) + 2ζ (b1 + c1 φ) cos φ + (b2 + c2 φ) sin φ Hamilton’s equations of motion with our new COP and the linear leg spring model are given by p2ψ pζ k(η − l0 ) ζ˙ = , p˙ζ = ζ + (b1 + c1 φ) cos φ + (b2 + c2 φ) sin φ , − 3 m mζ η ) p k(η − l ψ 0 b1 c1 + b2 c2 + (c21 + c22 )φ ψ˙ = , p˙ψ = − mζ 2 η + ζ(b2 + c1 + c2 φ) cos φ − ζ(b1 − c2 + c1 φ) sin φ , pθ p˙θ = −p˙ψ , θ˙ = , I (4) where φ ψ − θ. We assume when a step commences, the spring is uncompressed, η = l0 . Because the spring starts at and returns to rest length at step transitions, no step-to-step impacts dissipate energy, and thus energy is conserved in the LLS model. 2.2 Hybrid Step-to-Step Dynamics The generalized coordinates r = (ζ, ψ, θ) and their conjugate momenta, pr , provide a convenient set of local coordinates for expressing the within-step Hamiltonian dynamics (4) of the LLS. However, they provide an inconvenient representation when considering the step-to-step dynamics because they depend on the frame {F } that moves every step. As a remedy, we follow [23], ˙ ∈ S ⊂ R3 are the “interand use q = (s, g) ∈ S × SE(2), where s = (v, δ, θ) nal” states, and g ∈ SE(2) is the pose. The speed, v, is the magnitude of the COM velocity, and the relative heading, δ, is the angular diﬀerence between the orientation, θ, and the angle of the COM velocity vector (see Fig. 1). The local coordinates (θ, x, y) parameterize SE(2) without singularities through the usual relationship,   cos θ − sin θ x   g =  sin θ cos θ y  (5) 0 0 1 (written as a homogeneous transformation matrix) so we conﬂate the two when convenient and often write g = (θ, x, y) in an abuse of notation. The dynamical equations can be recast using the state variables q, which we omit for simplicity of presentation. Instead, we consider the state qi , i = 0, 1, 2 . . .

172

J. Lee et al.

as the discrete state, where qk = (sk , gk ) corresponds to the state at the beginning of the k th step. If h ∈ SE(2) and q = (s, g) ∈ S × SE(2) then we deﬁne the left action of SE(2) on S × SE(2) by hq = (s, hg), where hg is the group product on SE(2). At the beginning of the k th step (for k even), the leg is at rest length, η = l0 , and the leg touch-down angle starts at β0 relative to the y-axis of the body frame {B}, β = β0 . This information, together with the state qk = (sk , gk ), uniquely determines the initial conditions for integration of Hamilton’s equations. When η again reaches the spring rest length l0 , the hybrid system transitions to the right step, as described below. The ﬁnal values of (r, pr ) at the end of the k th step uniquely determine the states qk+1 = (sk+1 , gk+1 ), used to start the subsequent step. Thus, the left step dynamics map the state fL : qk → qk+1 according to a simple change of variables into coordinates (r, pr ), followed by integration of Hamilton’s equations. By inspection of Hamilton’s equations (4), note that the left-step mapping is left invariant under rigid transformations of the initial condition, since the equations are not functions of (x, y), and θ never shows up without −ψ, both of which are with respect to the world frame. Hence, fL (s, hg) = hfL (s, g). Note that this implies that qk+1 = (sk+1 , gk+1 ) = fL (s, gk ) = gk fL (s, e), where e ∈ SE(2) is the identity. Let {Ak } denote the location of the body frame at the beginning of the k th step. In other words, gk is the transformation from {Ak } to the world frame {U }. For k odd, the right leg is down, and Hamilton’s equations (4) are identical, so long as we express them in terms of a left-handed frame. We do this by taking a mirror image around the y-axis of frame {Ak } at the beginning and end of the k th step (k odd), to write down the right step map in terms of the left one. This can be expressed in terms of local coordinates ˙ θ, x), integrating the left step map, ˙ θ, x, y)T as ﬁrst “ﬂipping” (δ, θ, q = (v, δ, θ, and then ﬂipping back, namely fR (q) = M fL M q), where M = diag {1, −1, −1, −1, −1, 1} . (6) Note that M M = I. We chose to ﬂip about the y-axis for notational simplicity, but in principle any left-handed frame would work. This mapping leaves the right step map left-invariant under SE(2). For ﬁnding symmetric steady-state gaits, it will be convenient to deﬁne a special step-to-step return map that amounts to an “integrate and ﬂip” (see Fig. 1, Right). For a complete stride that includes a left step and then a right step, the stride-to-stride mapping is given by fL−R = fR ◦ fL , namely fL−R : q → M fL (M fL (q)) = (f ◦ f )(q),

where

f (q) := M fL (q) .

(7)

This approach eliminates the need to distinguish between left and right steps for control purposes. Note, however, that f is not left-invariant, even though both fL and fR are left-invariant. The resulting state evolution is given simply by

Task-Level Control of the LLS Model of Locomotion

qk+1 = f (qk ) ,

173

(8)

keeping in mind that for odd steps, the value for qk in this step-to-step formulation has already been “ﬂipped”. 2.3 Simulation Methods We simulated the LLS model using Matlab and the convention discussed in Sect. 2.2; for every right-step, convert it to a left-leg step, simulate the within step dynamics, and then convert it back to a right-leg step. This enabled us to specify the COP position using (1) and integrate the equation of motion (4) without the explicit representations of a left or right step in the equations. We used Matlab’s ode45 with time varying step size to integrate the equations of motion. The integration terminated as soon as the compressed leg returned back to its relaxed length l0 . To specify a moving COP, we selected bi and di (kT ), i = 1, 2 where di (kT ) denotes the COP position at the start of k-th step. To meet this restriction, ci is allowed to vary at each step, although it shouldn’t vary at an equilibrium point.4 ˙ x)T using the LevenbergWe found the equilibrium point q0 = (v, δ, θ, θ, Marquardt method in Matlab’s fsolve function. While ﬁxing the state v to a desired value, the function minimized the error diﬀerence of a step, f (q) − q. We also found the stride-to-stride Jacobian, Astride , and step-to-step Jacobian, Astep , about the equilibrium point using a central diﬀerence approximation. The ith columns are given by [Astride ]i = (fL−R (q + ei ) − fL−R (q − ei ))/2 and [Astep ]i = (f (q + ei ) − f (q − ei ))/2, where = 1 × 10−6 and ei is the i-th column of 5 × 5 identity matrix. In Sect. 3.2, we discuss the LLS stability from the eigenvalues of Astride , while in Sect. 4, we use Astep to control the LLS model. Unless otherwise noted, we used the following parameters and measurements of death-head cockroaches, Blaberus discoidalis, used in [19, 22, 24]: m = 0.0025 kg, I = 2.04 × 10−7 kg m2 . The choices for l0 , k, v, and β0 were chosen to satisfy constraints on the stride length (Ls = 0.02 − 0.025 m) and stride frequency (fs = 10 Hz), and generally fell in the ranges k = 2.25 − 3.5 N m−1 , l0 = 0.008 − 0.015 m, β0 = 0.8 − 1.2 rad, di = 0.002 m, and v = 0.2 − 0.25 m/s.

3 Analysis of COP Placements 3.1 Eﬀects of Various COP Placements In order to match the LLS system with an actual cockroach data (Fig. 2), we need to understand the eﬀects of bi and ci (or di (kT )) on the overall system. 4

Instead, the values for bi and ci can be speciﬁed directly [24]. This causes di (kT ) to change depending on the quantity (ψ(kT ) − θ(kT )).

174

J. Lee et al.

Fig. 2. Left: A stride of the original LLS model (d1 ≡ 0) with a ﬁxed COP (solid) and a moving COP (dashed). The used parameters for the ﬁxed COP are: v = 0.226 cm/s, k = 2.4 N/m, β0 = 1 rad, l0 = 0.0102 m, d2 = −0.0025 m; for the moving COP are: v = 0.2235 m/s, k = 3.52 N/m, β0 = 1.125 rad, l0 = 0.0082 m, d2 = 0.0025 m → −0.0025 m. Right: Experimental measurements of Blaberus discoidalis from several sources, [14, 15, 18]; ﬁgure from [22]. (Notice, since the right ﬁgure doesn’t start from t = 0, the stride period is roughly the same between the two ﬁgures)

To do so, our initial attempt is to consider various protocols for the COP placements: (a) Increment (b) Increment (c) Increment (d) Increment (e) Increment

d1 while d2 = 0; d2 while d1 = 0; the amplitude of a moving d2 while d1 = 0; the oﬀset of a moving d2 while d1 = 0; d1 while d2 is moving;

Figure 3 illustrates these protocols schematically for a left step; for a right step, the COP path is mirrored about the body y-axis. For each protocol and their parameter increments, we found the corresponding equilibrium points and simulated a full stride (starting with a left step) from the equilibrium points. The results from the simulations are shown in Figs. 4, 5, and 6. For each incrementing parameter, we plotted the result using diﬀerent shades of {B}

y x (a)

(b)

(c)

(d)

(e)

Fig. 3. COP placement protocols for a left step with respect to the body frame {B}. The solid dots indicate ﬁxed COP positions; the arrows indicate the direction, magnitude, and oﬀset of moving COPs

Task-Level Control of the LLS Model of Locomotion

175

(a) Increment d1 = {−0.2, . . . , 0.2 cm} while d2 = 0.

(b) Increment d2 = {−0.2, . . . , 0.2 cm} while d1 = 0. Fig. 4. See text for description of each COP protocol. The parameters used in this ﬁgure are: m = 0.0025 kg, I = 2.04 × 10−7 kg m2 , k = 3.52 N m−1 , v = 0.2275 m/s, l0 = 0.0127 m, β0 = 1.12 rad (or 64.2 ◦ ). Note, unlike Fig. 2, these ﬁgures have scaled units (e.g. cm and mN) for clarity

176

J. Lee et al.

(c) Increment the amplitude of moving d2 from 0 to 0.2 cm while d1 = 0.

(d) Increment the oﬀset of moving d2 from 0 to 0.2 cm while d1 = 0. Fig. 5. The parameters used here are: m = 0.0025 kg, I = 2.04 × 10−7 kg m2 , k = 3.52 N m−1 , v = 0.2235 m/s, l0 = 0.0082 m, β0 = 1.125 rad

Task-Level Control of the LLS Model of Locomotion

177

(e) Increment d1 = {−0.2, . . . , 0.2 cm} (solid) while d2 (dashed) is moving from 0.2 to −0.2 cm. Fig. 6. The parameters used here are: m = 0.0025 kg, I = 2.04 × 10−7 kg m2 , k = 3.52 N m−1 , v = 0.2235 m/s, l0 = 0.0082 m, β0 = 1.125 rad

gray. The ﬁrst two columns of a subﬁgure shows the COM velocity and legspring forces in lateral and forward directions (i.e. x and y directions in the inertial frame {U }), the body angle, and the moment. The last column shows d1 (solid line) and/or d2 (dashed line) as a function of time, COM path, and the eigenvalues as a function of the incrementing parameter. The rest of the parameters (i.e. k, v, β0 , and l0 ) were chosen to closely match the stride length and frequency of cockroach data [22]. Protocol (a): Fixed COP on lateral axis. Figure 4(a) shows results of a simulated LLS model in which we ﬁxed the COP at various positions along the x-axis of the body frame {B}. As desired, when d1 > 0, the proﬁles of the body angle, θ, and the moment waveforms resemble actual cockroach data (Fig. 2), as well as that of the moving COP proposed by Schmitt and Holmes [24] (reproduced in Fig. 2, Left). Note that the positive cosinusoidal waveforms of the body angle (which agrees with the biological data) for a ﬁxed COP only occur when d is on the positive x-axis of {B}. Figure 4(a) indicates that the increase in |d1 | ampliﬁes the body angle and the moment waveforms while the other measurements, including the stride length and frequency, remain relatively constant. This isolated eﬀect of d1 will be useful later on when we

178

J. Lee et al.

ﬁt the data to another waveform. In addition, the eigenvalue plot shows that the system becomes unstable when d1 < 0 and stable when d1 > 0. Protocol (b): Fixed COP on fore-aft axis. As a comparison to the previous result, Fig. 4(b) shows the eﬀects of diﬀerent locations d2 for a ﬁxed COP. Although the body angle is sinusoidal (not cosinusoidal, like the cockroach), the location of d2 does have a larger impact on the magnitude of body angle and the stability of the system (steeper slope for the moving eigenvalue) than d1 in the previous protocol. We speculate that one cause of this diﬀerences in impact level is due to the large value of β0 ; since β0 = 1.12 > π/4, the leg force is oriented primarily in the lateral direction rather than the fore-aft direction. Thus, changes in d2 cause greater moment arm changes than the equal changes in d1 . We will utilize this eﬀect in Sect. 4 by using d2 as our control input. We also note that the body velocity (and position) and foot forces of this ﬁgure matches the previous ﬁgure. Although not shown, as the ﬁxed COP position traverses in this neighborhood, without the restriction of d1 = 0 or d2 = 0, the body velocity and foot force waveforms remain relatively constant. On the other hand, the waveforms for the body angle and the moment go through phase and amplitude changes. Protocol (c): Incrementing magnitude of a moving COP on the fore-aft axis. For fore-aft COP motion along the body frame y-axis, Fig. 5(c) shows the eﬀects of changing the magnitude of COP motion. Unlike the previous protocols, varying the magnitude of a moving COP causes large changes to all the kinematics, step length, and step frequency. Although it is not shown here, further increase in magnitude (also observed in [22]) or reversing the direction (i.e. aft to fore) of the moving COP drives the system unstable. Protocol (d): Forward Shifting of a Moving COP. Figure 5(d) shows the eﬀects of shifting a moving COP in y direction in {B}. It shows that, as the oﬀset b2 increases (or decreases, although not shown), the body loses its cosinusoidal waveform and eventually becomes unstable. We emphasize that the system does not go unstable as soon as the oﬀset b2 > 0 nor b2 < 0. Also the instability does not necessarily occur even though the moving COP remains in front of the COM most of the time. Along with Protocol (c), we introduce one possible explanation of these results in Sect. 3.3. Protocol (e): Lateral Shifting of a Moving COP. Lastly, we look at the result of incrementing the lateral oﬀset to a moving COP, as shown in Fig. 6(e). The result resembles that of Protocol (a) in Fig. 4(a); the changes in d1 mostly aﬀect the magnitude of body angle and moments, but the waveforms all remain qualitatively the same shape. Also, the increase in d1 has ampliﬁed the waveforms, and the moving COP has stabilized the system even with d1 < 0, in contrast to the results of Fig. 4(a). From the results from these protocols, we conclude that we can achieve the desired cosinusoidal waveforms by laterally oﬀsetting a ﬁxed COP or moving COP. However, both cases produce body angle and moment variations that remain an order of magnitude smaller than those of a cockroach. This can be remedied with a very large – possibly non-physical – COP oﬀset of d1 =

Task-Level Control of the LLS Model of Locomotion

179

Fig. 7. Maximum eigenvalue (neglecting two invariant unity eigenvalues for energy and direction) of the linearized return map as a function of our new, two dimensional COP locations. The dark gray indicates the parameter regime of maximum stability and the neutral stability occurs when the contour reads 1. Eigenvalues greater that 3 are empty. The parameter values used are: v = 0.25 m/s, k = 2.4 Nm−1 , l0 = 0.01 m, β0 = 1 rad, and −0.02 m < d1 , d2 < 0.02 m

0.025 m ≈ 2l0 and d2 = 0, which means that the virtual foot touchdown position will be far oﬀ to the positive x-axis in {B} along with the COP. The resulting magnitude of the body angle was about 2 ◦ (or 0.035 rad) with the moment of 0.3 × 10−4 N m. This is within an order of magnitude of the cockroach variations 5.7◦ (or 0.1 rad) and 1 × 10−4 N m in Fig. 2. 3.2 Stability as a Function of Fixed COP Position Figures 4(a) and 4(b) showed the stability plot of the LLS with a ﬁxed COP along the x and y-axis of {B}, respectively. Figure 7 shows a contour plot of the maximum non-unity eigenvalues as a function of more general 2D ﬁxed COP positions. Note that the neutrally stable (i.e. max λ = 1) gait corresponding to (d1 , d2 ) = (0, 0) found by Schmitt and Holmes [23] lies along a neutral stability contour through the origin of the d-plane. There is a large stable region (max λ < 1) “inside” the neutral stability contour and an abrupt area of instability in the lower-right corner of the plot. Notice that the stable region (max λ < 1) extends to a part of d2 > 0 region for d1 > 0. This indicates that we can achieve stability for the ﬁxed COP that is in front of COM, as long as it is suﬃciently oﬀset to the right (d1 > 0). We also notice that around the origin, the gradient of the eigenvalues is greater in the direction of y-axis than x-axis of {B}. This hints that a small displacement of the ﬁxed COP in d2 should give us a greater control than that of d1 . We utilize this notion in Sect. 4.

180

J. Lee et al.

Our long-term goal is to match the LLS to biological or robotic locomotion performance, possibly using the LLS as a plant model for control. Therefore, we examined the equilibrium state values, δ ∗ , θ∗ , and θ˙∗ , in Fig. 8, as a recipe for future comparisons to biological and robotic systems. As expected, the two contours θ∗ = 0 and θ˙∗ = 0 indicate purely oddly symmetric (sinusoidal) and evenly symmetric (cosinusoidal) yaw motions, respectively, and these symmetries only occur on those contours. 3.3 Comparing Fixed vs. Moving COP Models From the observations above, we consider the relationship between a fore-aft moving COP and a ﬁxed, laterally oﬀset COP. These two scenarios generate similar waveforms; in fact, using very similar parameters, we can nearly match the body motions and forces using these two strategies, as shown in Fig. 9. To ﬁnd a good match, we relied on trial and error, using Figs. 4(a) and 5(c) as a guide to adjust d1 and d2 and we referred to [22] to adjust l0 , β0 , and k. As shown, the body angle (yaw) motions match nearly exactly, while for the other measurements, the ﬁxed COP exceeded the moving COP somewhat in magnitude, although the results are qualitatively similar. We compare the moving COP model to a model with a ﬁxed COP on the positive x-axis of {B}, as follows. As the LLS moves through a left step, the leg intersects the body centerline at a point that moves fore-aft, as depicted in Fig. 10(a). Suppose there is another LLS system with a moving COP that traces out the same path, and has the same foot touchdown position as the ﬁxed COP case. With appropriate parameters (and possibly a nonlinear leg spring), the ﬁxed COP LLS model might approximate the moving COP model. By approximating the moving COP with the ﬁxed COP in this way, we can predict which moving COP protocols might be stable on the basis of the stability contour map (Fig. 7, Sect. 3.2). Using this approach, we address below (without formal proofs) unanswered questions from Sect. 3.1. In Protocol (c), we considered increasing the magnitude of a moving COP. We approximate this case using the eﬀective ﬁxed COP and eﬀective β0 shown in Fig. 10(b) and (c); a larger magnitude can be created by moving the eﬀective ﬁxed COP in the x direction and/or decreasing the value of leg touchdown angle β0 . From Fig. 4(a), we saw that the increase in d1 for a ﬁxed COP improved stability and ampliﬁed the body angle and moment, which agrees with increasing the magnitude of fore-aft motion in the moving COP, as shown in Fig. 5(c). Similarly, a moving COP that is shifted forward, as in Protocol (d), can be approximated by shifting the eﬀective ﬁxed COP forward, as shown in Fig. 10(d). Figure 8(a) shows that the eﬀective ﬁxed COP will ﬁrst be stable, but eventually it will be unstable as the oﬀset increases further. Earlier, we indicated that the system became unstable when the moving COP moved from back to front (i.e. aft to fore) along the body centerline. As Fig. 10(e) shows, the eﬀective ﬁxed COP would then be placed on the left side of the body centerline which, according to Fig. 8(a), would probably

Task-Level Control of the LLS Model of Locomotion

181

Fig. 8. Contour map of the maximum non-unity eigenvalue and the equilibrium points δ ∗ , θ∗ , and θ˙∗ . The parameter values are the same as Fig. 7

Fig. 9. Comparison between moving and ﬁxed COP. m = 0.0025 kg, I = 2.04 × 10−7 kg m2 , k = 3.52 N m−1 , β0 = 1.125 rad. Moving COP (dashed) v = 0.2235 m/s, l0 = 0.0082 m, d1 = 0 m, d2 = 0.002 → −0.002 m. Fixed COP (solid) v = 0.2275 m/s, l0 = 0.0128 m, d1 = 0.005 m, d2 = 0 m

182

J. Lee et al.

(a)

(b)

(c)

(d)

(e)

Fig. 10. Comparison between a moving COP and an eﬀective ﬁxed COP during a left step under diﬀerent protocols. Fixed COP is denoted by a ﬁlled dot, and the moving COP is denoted by a gray arrow

make the system unstable. This also suggests we can achieve stability for a forward moving COP if we choose our oﬀsets carefully. In Protocol (e), we increased the lateral oﬀset of a moving COP. We can represent this simply by laterally shifting the eﬀective ﬁxed COP which is similar to Protocol (a), Fig. 4(a). Indeed that is what we observed in Fig. 6(e). This explains why the system remained stable when d1 < 0; the eﬀective ﬁxed COP position was to the right of the COM (d1 > 0). This implies that for cockroaches, if their mechanics limit the magnitude of d2 , i.e. they cannot have a large c2 , then an increase d1 will achieve the desired stability, or vice versa; this would explain the shift in the moving COP observed in cockroaches [28]. In summary, the moving COP model is more complex than the ﬁxed COP, but they have similar performance in matching biological data. Thus, in the next section, we assume the COP is ﬁxed to the right of the COM within each step, but let the controller adjust the location of the COP between steps.

4 LLS Control: Wall Following In addition to their remarkable stability, cockroaches also exhibit extraordinary maneuverability. For example the American cockroach, Periplaneta americana, follows walls using tactile feedback from their antenna, making up to 25 turns per second in response to jagged walls in the environment [3, 6]. Despite its simplicity, the LLS model captures many salient features of the dynamics and stability of steady-state cockroach locomotion. Building on these strengths, we explored using the LLS as a “plant model” for control. Schmitt and Holmes [24] tested the idea of moving the COP to steer locomotion. They noted that brieﬂy moving the COP in front of the COM generates large turns of 20–70◦ . Other possible control parameters, such as the spring stiﬀness, leg length, and step-length can also be used for steering, but Full et al. [13] contend that moving the COP is the most eﬀective, and least fragile. Moreover, moving the COP for steering seems to be consistent with animal turning behavior [17].

Task-Level Control of the LLS Model of Locomotion

183

4.1 LLS Plant Model In Sect. 3, we compared the eﬀects of moving the COP within a step, versus keeping the COP ﬁxed. Both models can, with appropriate parameters, demonstrate asymptotic stability in the relative heading, δ, and angular ve˙ but neutral stability in running speed, v ∈ R+ , orientation θ, and x locity, θ, (if we’re running in the y direction of {U }). As discussed above, the moving COP adds complexity but provides very little advantage over the laterally oﬀset but ﬁxed COP model when it comes to matching steady-state cockroach data. Therefore, we explored using step-to-step adjustments of the COP as an input to control the overall body location in g ∈ SE(2). Because there are no energy losses between steps due to impacts, the controlled LLS remains piecewise Hamiltonian and energy conserving. Initially, we explored control laws that varied d1 , d2 , β0 , and combinations thereof. We found that a highly eﬀective control scheme was to ﬁx β0 and place the nominal COP to the right of the COM (for left steps), varying only the fore-aft COP location (d2 ) from step-to-step. This is consistent with biological observations that rapid maneuvering in cockroaches occurs with large changes in the fore-aft COP [17]. Speciﬁcally, we used the step-to-step control input dk = αe1 + e2 uk ,

where

e1 = [ 1, 0 ]T , e2 = [ 0, 1 ]T ,

(9)

k is the stride number, uk ∈ R is the control input, and α is a scalar. As shown in Sect. 2, selecting α > 0 ensures that for uk ≡ 0, the system is ˙ and neutrally stable in v and g. The result asymptotically stable in δ and θ, is a step-to-step return map, qk+1 = f (qk , uk ) ,

(10)

that is no diﬀerent from the step-to-step uncontrolled LLS in (8), except that between steps the COP location can be adjusted according to (9). 4.2 Antenna-Based Output Function We assume that the LLS controller will have at its disposal proprioceptive measurements at each step, such as its angular velocity, θ˙k , and relative heading, δk , but not necessarily its position and orientation relative to our arbitrarily assigned world reference frame, {U }. Therefore, in order for the LLS to achieve some task level goal in SE(2), it needs exteroceptive feedback. For this, we derive inspiration from nature, and assume the LLS has a simpliﬁed antenna that measures its distance from a surface in the environment as depicted in Fig. 11. Our antenna model follows [6, 7, 8] and assumes the antenna estimates the distance, γ, from the body centerline to a “wall” – in this case the yaxis – ahead of the COM a known, ﬁxed preview distance, . Under these assumptions

184

J. Lee et al.

y {U}

x

Fig. 11. Left: Multiple exposures of the cockroach P. americana negotiating a set of obstacles at high speed, using feedback from its antennae [6]. Right: A simpliﬁed model of an antenna as a distance-to-wall sensor

γ = tan θ − x sec θ .

(11)

The above equation (11) relates the LLS pose to the antenna measurement, γ. We make no attempt to avoid collisions of the LLS with the virtual wall and for simplicity, our controller will drive the LLS to align itself directly on top of the y-axis, though this can easily be extended to drive the LLS to follow at an oﬀset distance from the wall. Together, the proprioceptive and exteroceptive measurements yields the measurement function:

˙ γ T . (12) h(q) := δ, θ,

4.3 Reduced Return Map To simplify controller analysis and design, we reduced the model, by using translational symmetry and energy conservation, as follows. Recall that the left- and right-step mappings, fL and fR , are invariant to SE(3), but the stepto-step return map, f = M fL , is not. However, that mapping is invariant to pure y motions (had we chosen a diﬀerent left-handed frame, translational invariance would have been in the direction of the axis of symmetry of the reﬂection to that frame). This was by design: our goal for control is wall following, and for simplicity, we have chosen to follow the y-axis. In addition, the output mapping, h, is y-translation invariant. Thus we removed the y equation by setting y = 0 at the beginning of each step. Naturally, we ignored the ﬁnal value of y when ﬁnding an equilibrium as we did in Sect. 2. To remove v note that H=

1 1 1 mv 2 + I θ˙2 + k(η − l0 )2 = H0 ≡ constant . 2 2 2

(13)

So, at each step % v=

2 m

1 1 H0 − I θ˙2 − k(η − l0 )2 2 2

Thus we deﬁned a transformation

&1/2 .

(14)

Task-Level Control of the LLS Model of Locomotion

˙ θ, x) → (v, δ, θ, ˙ θ, x, 0) TH : (δ, θ,

185

(15)

that assigns y = 0 and computes v from (14). Note that TH is invertible and −1 is the transformation that simply removes the v and y coordinates. Then, TH we deﬁne the reduced variables and mapping ˙ θ, x) and f r (q r , uk ) = T −1 f (TH (q r ), uk ) . (16) q r = (δ, θ, k k H 4.4 Linearized Return Map, Controllability and Observability As a preliminary control task, we chose to have the LLS follow a virtual “wall” coincident with the y-axis. To ﬁnd an equilibrium, we used similar techniques to those described in Sect. 2 to ﬁnd equilibrium trajectories, ensuring that x = 0 at the beginning and end of each step, corresponding to exact wall following. The result was an equilibrium q ∗ , such that q ∗ = f r (q ∗ , 0). To address controllability, we numerically linearized the return map around a nominal equilibrium trajectory, to obtain the local return map ek+1 = Aek + Buk zk = Cek

(17)

where

∂f r 33 ∂f r 33 B = , (18) 3 3 ∂q r qr =q∗ ,u=0 ∂u qr =q∗ ,u=0 and ek = qkr − q ∗ . The linearized output matrix can be derived analytically,   100 0 % & ∂h   C= = 0 1 0 0  (19) ∂q r qr =q∗ 0 0 −1 A=

but to date we only have numerical approximations to A and B. The reduced system (A, B, C) is stabilizable and observable for the parameters m = 0.0025 kg, I = 2.04 × 10−7 kg m2 , l0 = 0.01 m, k = 2.25 N/m, β0 = 0.8 rad, and a nominal COP oﬀset of α = 0.0025 m. 4.5 Antenna-Based Control Strategy Because the system is completely observable, state feedback is possible; however, we found that the following simple output feedback to be quite eﬀective: uk = Kzk ,

(20)

where zk is the system output from (17). The closed loop system dynamics are governed by the ek+1 = (A + BKC)ek , so to ﬁnd a good choice for the gain, K, we evaluated the eigenvalues of the system matrix (A + BKC). Amidst a variety of possible feedback gains, we selected K = [ 0, 0.001, 0.5 ], which lead to complex conjugate pairs of closed loop eigenvalues at −0.4643 ± j0.2607 and 0.3478 ± j0.4827. A demonstration of this controller, executed on the full nonlinear LLS dynamics, is shown in Fig. 12.

186

J. Lee et al.

Fig. 12. Simulation of the controlled LLS following the y-axis using the feedback control law (20). In this control law, COP lies nominally along the body x, namely d = [ 0.25cm, 0 ]T (for the left step). The output feedback controller (20) varies the COP in the d2 direction. In the simulation, the LLS starts out rotated 30◦ counterclockwise from the y-axis, and 2 cm to the right. The ﬁgure on the right shows COM (◦), COP (∗), and foot (×) positions at the start of each step

5 Conclusion In this paper, we revised the LLS model to achieve the same phase relationship between the θ and δ as a real cockroach, using a ﬁxed and laterally oﬀset COP. Also we investigated how the COP location governed the overall system stability, and related the ﬁxed COP model to the moving COP model presented by Schmitt and Holmes [23]. For control purposes, we reduced the system state to four dimensions, using translation symmetry and energy conservation. We then applied a very simple output-based feedback strategy to update the COP location between strides based on an antenna-like measurement. Using this controller, the reduced system dynamics were linearly stable. In the future, we will explore using the LLS model as a control template [12] for horizontal-plane control of running robots.

References [1] R. Altendorfer, N. Moore, H. Komsuoglu, M. Buehler, H. B. Brown Jr., D. McMordie, U. Saranli, R. J. Full, and D. E. Koditschek. RHex: A biologicallyinspired hexapod runner. Autonomous Robots, 11(3):207–213, 2001.

Task-Level Control of the LLS Model of Locomotion

187

[2] R. Blickhan and R. J. Full. Similarity in multilegged locomotion: Bouncing like a monopode. J Comp Physiol A, 173:509–517, 1993. [3] J. M. Camhi and E. N. Johnson. High-frequency steering maneuvers mediated by tactile cues: antenna wall-following in the cockroach. J Exp Bio, 202:631– 643, 1999. [4] G. A. Cavagna, N. C. Heglund, and C. R. Taylor. Walking, running, and galloping: Mechanical similarities between diﬀerent animals. In T. J. Pedley, editor, Scale Eﬀects in Animal Locomotion, Proceedings of an International Symposium, pp. 111–125. Academic Press, New York, USA, 1975. [5] J. G. Cham, S. A. Bailey, J. E. Clark, R. J. Full, and M. R. Cutkosky. Fast and robust: Hexapedal robots via shape deposition manufacturing. The International Journal of Robotics Research, 21(10), 2002. [6] N. J. Cowan, J. Lee, and R. J. Full. Dynamical control of antennal wall following in the American cockroach. In The Society for Integrative and Comparative Biology, p. 126, San Diego, CA, January 2005. [7] N. J. Cowan, J. Lee, and R. J. Full. Task-level control of rapid wall following in the American cockroach. J Exp Biol, 209:1617–1629, 2006. [8] N. J. Cowan, E. J. Ma, M. Cutkosky, and R. J. Full. A biologically inspired passive antenna for steering control of a running robot. In International Symposium on Robotics Research. Siena, Italy, 2003. [9] M. H. Dickinson, C. T. Farley, R. J. Full, M. R. Koehl, R. Kram, and S. L. Lehman. How animals move: An integrative view. Science, 288(5463):100–106, 2000. [10] R. J. Full, R. Blickhan, and L. H. Ting. Leg design in hexapedal runners. J Exp Biol, 158:369–390, 1991. [11] R. J. Full and C. T. Farley. Musculoskeletal dynamics in rhythmic systems – A comparative approach to legged locomotion. In J. M. Winters and P. E. Crago, editors, Biomechanics and Neural Control of Movement, pp. 192–205. Springer-Verlag, New York, 2000. [12] R. J. Full and D. E. Koditschek. Templates and anchors: neuromechanical hypotheses of legged locomotion on land. J Exp Bio, 202(23):3325–3332, 1999. [13] R. J. Full, T. M. Kubow, J. Schmitt, P. Holmes, and D. E. Koditschek. Quantifying dynamic stability and maneuverability in legged locomotion. Integrative and Comparative Biology, 42(1):149–157, 2002. [14] R. J. Full and M. S. Tu. Mechanics of six-legged runners. J Exp Bio, 148:129– 146, 1990. [15] R. J. Full and M. S. Tu. Mechanics of a rapid running insect: two-, four-, and six-legged locomotion. J Exp Bio, 156:215–231, 1991. [16] F. Iida and R. Pfeifer. “Cheap” rapid locomotion of a quadruped robot: Selfstabilization of bounding gait. In F. Groen, N. Amato, A. Bonarini, E. Yoshida, and B. Krose, editors, Intelligent Autonomous Systems 8, pp. 642–649. IOS Press, 2004. [17] D. L. Jindrich and R. J. Full. Many-legged maneuverability: Dynamics of turning in hexapods. J Exp Bio, 202(12):1603–1623, 1999. [18] R. Kram, B. Wong, and R. J. Full. Three-dimensional kinematics and limb kinetic energy of running cockroaches. J Exp Bio, 200:1919–1929, 1997. [19] T. M. Kubow and R. J. Full. The role of the mechanical system in control: a hypothesis of self-stabilization in hexapedal runners. Philosophical Transactions of

188

[20]

[21] [22]

[23] [24]

[25] [26] [27] [28]

J. Lee et al. the Royal Society of London Serieseoleol B-Biological Sciences, 354(1385):849– 861, 1999. R. D. Quinn, G. M. Nelson, R. J. Bachmann, D. A. Kingsley, J. Oﬃ, and R. E. Ritzmann. Insect designs for improved robot mobility. In Berns and Dillmann, editors, Conference on Climbing and Walking Robots, pp. 69–76, Karlsruhe, Germany, 2001. Professional Engineering Publications. A. Ruina. Nonholonomic stability aspects of piecewise-holonomic systems. Rep. Math. Phys., 42(1-2):91–100, 1998. J. Schmitt, M. Garcia, R. C. Razo, P. Holmes, and R. J. Full. Dynamics and stability of legged locomotion in the horizontal plane: a test case using insects. Biol Cybern, 86:343–353, 2002. J. Schmitt and P. Holmes. Mechanical models for insect locomotion: dynamics and stability in the horizontal plane I. Theory. Biol Cybern, 83:501–515, 2000. J. Schmitt and P. Holmes. Mechanical models for insect locomotion: dynamics and stability in the horizontal plane-II. Application. Biol Cybern, 83:517–527, 2000. J. Schmitt and P. Holmes. Mechanical models for insect locomotion: active muscles and energy losses. Biol Cybern, 89:43–55, 2003. W. J. Schwind and D. E. Koditschek. Approximating the stance map of a 2-dof monoped runner. Journal of Nonlinear Science, 10(5):533–568, 2000. A. Seyfarth, H. Geyer, M. Gunther, and R. Blickhan. A movement criterion for running. J Biomech, 35(5):649–655, May 2002. L. H. Ting, R. Blickhan, and R. J. Full. Dynamic and static stability in hexapedal runners. J Exp Biol, 197:251–269, 1994.

Investigating the Use of Iterative Learning Control and Repetitive Control to Implement Periodic Gaits R.W. Longman1 and K.D. Mombaur2 1

2

Dept. of Mechanical Engineering, Columbia University, New York, 10027, USA [email protected] IWR, Universit¨ at Heidelberg, Im Neuenheimer Feld 368, 69120 Heidelberg, Germany [email protected]

Summary. In the next few years considerable eﬀort will be expended to make humanoid robots that can do true dynamic walking, or even running. One may numerically compute a desired gait, e.g. one that has been optimized to be asymptotically stable without feedback. One would normally give the gait as commands to the controllers for the robot joints. However, control system outputs generally diﬀer from the command given, and the faster the command changes with time, the more deviation there is. Iterative learning control (ILC) and repetitive control (RC) aim to ﬁx this problem in situations where a command is repeating or periodic. Since gaits are periodic motions, it is natural to ask whether ILC/RC can be of use in implementing gaits in hardware. These control concepts are no substitutes for feedback control but work in combination with them by adjusting the commands to the feedback controllers from a higher level perspective. It is shown that the gait problem does not precisely ﬁt either the ILC or the RC problem statements. Gait problems are necessarily divided into phases deﬁned by foot strike times, and furthermore the durations of the phases are not the same from cycle to cycle during the learning process. Several methods are suggested to address these issues, and four repetitive control laws are studied numerically. The laws that include both position and velocity error in the updates are seen to be the most eﬀective. It appears that with appropriate reﬁnement, such generalized RC laws could be very helpful in getting hardware to execute desired gaits.

1 Introduction In the last few years, many humanoid and biped walking robots have been built executing periodic or quasi-periodic gaits [1, 2]. So far such robots are rather slow moving compared to their biological counterparts, and the traditional control approach keeps them as close as possible to a quasistatic equilibrium during the motion, e.g. [5, 25]. As research progresses into

190

R.W. Longman and K.D. Mombaur

making robots that do true dynamic walking or running, in addition to dealing with the dynamic behavior of the nonlinear multibody robot system, it will become necessary to address the imperfect dynamics of any feedback control system that is used to attempt to execute the chosen periodic gait. In recent years, the ﬁelds of iterative learning control (ILC) and repetitive control (RC) have appeared [3, 19], the goal of which is to improve the performance of feedback control systems by adjusting the commands given to them. In principle, these techniques could be combined with any chosen feedback control concept. ILC and RC have now developed to the point that commercial use of the methods has started to appear. Recently, robots have been delivered to Daimler-Chrysler in Detroit using ILC to improve performance. And RC is now being used at the factory to learn a command for each track of computer disk drives. The improved performance allows the tracks to be closer together, and hence the disks can store more data. Similar methods are being used to speed up chip manufacturing, allowing the manufacturing hardware to operate faster while maintaining the needed precision, and hence increase productivity. ILC suddenly began to develop quickly in 1984 motivated by robots performing repetitive operations in a manufacturing environment. Each time the task is performed, the robot restarts from the same initial conditions. When a feedback control system is given a tracking command, the response is not the same as the command, even under ideal circumstances with perfect measurements and no plant noise disturbances. Generally, the faster the requested motion in the tracking command, the larger the discrepancy between what is asked for and what is produced. The purpose of ILC is to use the error observed in the previous run (or repetition) to adjust the command in the current run, aiming to converge to that command that actually produces the desired trajectory. ILC asks the control system to execute commands that are not what you want the controllers to do, so that they actually do what you want them to do. RC is a closely related type of control. Instead of making repeated runs of a desired ﬁnite time trajectory, each time starting from the same initial condition, RC aims to perfectly execute a periodic command, or to perfectly execute a constant command in the presence of a periodic disturbance (or to execute a periodic command with a periodic disturbance, each having the same period). The RC law learns from the measured error in the previous period (or cycle) instead of the previous run, adjusting the command in the current period, aiming to get to zero tracking error. Transients can propagate from one period to the next in RC, but cannot go from one run to the next in ILC, and this results in the two problems having diﬀerent conditions for stability, i.e. for convergence to zero tracking error. As gait research progresses from relatively slow robot walking motions to full dynamic walking, and then to running, the issues of imperfect execution of high speed commands by feedback control systems will become a serious issue. The desired periodic gaits are commands to the feedback controllers for

ILC and RC for Periodic Gaits

191

each robot joint. If the discrepancy between the commanded trajectory and the trajectory executed by the feedback controllers is large enough, the robot will fall. Since ILC and RC are new ﬁelds of control theory that address how to make feedback control systems actually perform desired repeating or periodic motions, it is natural to ask whether ILC or RC can be used to implement high speed gaits. It is the purpose of this paper to make an initial evaluation of what ILC/RC can do for this problem, and to put forward some concepts that might address the issues that are raised.

2 Feedback Control System Errors that ILC/RC can Fix Consider the performance of typical feedback control systems executing a time varying command. Suppose one wishes to control the output y(t) of a ﬁrst order system (the plant) dy/dt + ay = w + v where w(t) is a variable we can manipulate to change y, e.g. we can apply a torque to a robot link to make the output angle change. Typically, whatever variable we can manipulate, nature can also inﬂuence with various disturbances, e.g. in the case of a robot link, gravity can supply a torque history as the link follows the desired path, and v denotes such disturbances. Now consider applying a proportional controller to make the output follow a command. Then w(t) = Ke(t) = K(yC (t) − y(t)) and the manipulated variable w is proportional to the measured error e(t), the command yC (t) minus the measured output y(t). The performance is then predicted by the closed loop diﬀerential equation dy(t) + (K + a)y(t) = KyC (t) + v(t) dt

(1)

whose solution is −(K+a)t

y(t) = e

t y(0)+

−(K+a)(t−τ )

e 0

t KyC (τ )dτ +

e−(K+a)(t−τ ) v(τ )dτ (2)

0

The middle term on the right is the part of the solution that is responding the command we give the system. But it is not equal to the command. Instead it is a convolution integral of the command, creating a form of weighted average of all the commands we have given the system in the past. The weighting factor decays going backward in time, so that more recent commands are more important in the weighted average. Therefore, for any command that is changing with time, the feedback control system does this convolution integral of what we asked it to do, not what we asked it to do. And the faster the command changes with time, the more eﬀect the averaging has on the result. The ﬁrst term on the right represents transients and is the response to initial conditions. The last term on the right gives the eﬀect of disturbances on the

192

R.W. Longman and K.D. Mombaur

performance of the control system. If the system has the same disturbance every time we give the command, then this disturbance is also a source of repeating error. It is the purpose of ILC and RC to converge to a command yC (t) that is no longer equal to the desired output yD (t), but one which has the property that the system output given by the right hand side of (2) is in fact equal to yD (t). In the case of ILC the command has the property that over a ﬁnite time interval starting at time zero, the right side of (2) converges to the desired trajectory as the repetitions progress. In the case of RC, the command has the property that as time increases, the right hand side converges to the desired trajectory. Both ﬁx deterministic errors in following a command, and also cancel any disturbances that repeat. ILC also learns to handle the ﬁrst term on the right, since it is present in every run, while RC learns to get zero error as time goes to inﬁnity, and for an asymptotically stable system the ﬁrst term disappears with time. The iterations solve an inverse problem of ﬁnding the input that produces the desired output. In the simple example above, one can directly solve this inverse problem, e.g. using the desired output yD (t) for the output in (1) the command needed is: 1 dyD (t) − (K + a)yD (t) + v(t) yC (t) = (3) − K dt There are usually diﬃculties with this in practice. First, when done in discrete time, the inverse problem is usually ill-conditioned [11]. Second, if the external disturbance v(t) is an important aspect of the problem, one needs to know this function which may be diﬃcult. And third, the solution is only as good as the model is. ILC and RC ﬁnd an inverse solution iterating with the real world behavior, instead of the model, without needing to know v(t), and without totally relying on a model.

3 ILC/RC Background The most eﬀective ILC/RC design methods are based on linear systems theory, and the discussion presented here is limited to this approach. Results have been generated for doing nonlinear ILC on equations having the form of multibody dynamic systems. These results are likely to be not as practical as the linear design methods. First, they rely on all of the dynamics in the physical system having the multibody dynamics form, which is often not the case when actuators and sensors and eﬀective feedback controllers are included. Second, they create very complex control laws that are more complicated to implement. And third, linear methods as in [4] can converge to tracking errors that approach the minimum possible error, the repeatability level of the system, and do so relatively quickly without requiring the complicated modeling of the nonlinear system. Figure 1 shows the robot used in [4], and

ILC and RC for Periodic Gaits

193

Fig. 1. Robotics Research Corporation robot, and RMS error vs. repetitions using an ILC law

also the tracking error for each ILC repetition for each robot link following a high speed trajectory versus repetitions. The error is decreased by a factor of roughly 1000 in about 12 runs. Note that the ﬁnal error level is actually below the repeatability level of the robot when measured on a day to day basis, so the ILC is ﬁxing errors of the size of how diﬀerent the robot behaves from one day to the next. No amount of modeling could predict such errors. The fact that ILC does not rely heavily on a model allows it to ﬁx such errors. Nevertheless, in gait problems one might want to revisit the question of the usefulness of using fully nonlinear methods, or at least using some form of feedback linearization. There is a related issue for robot applications. One may consider creating an ILC or RC law that has multiple inputs and outputs, one for each of the joints variables. On the other hand, it is much simpler to use a decentralized ILC or RC approach, that applies a separate independent law to each of the separate feedback control systems for each robot joint as if there were no coupling between joints in the nonlinear dynamics. Again, the results in [4] are obtained using decentralized ILC, suggesting that the simple decentralized approach can be very eﬀective in robot applications. Both ILC and RC must necessarily be implemented by digital control methods, because the control updates are based on data from a previous repetition or a previous period, and therefore must be measured and stored in a computer or microprocessor. One will normally use a zero order hold on the input signal that the ILC or RC adjusts. Consider ILC. The objective is to perform a ﬁnite time trajectory, and get zero error at the sample times, i.e. we want the output y(kT ) to converge to the desired output yD (kT ) for k = 1, 2, 3, . . . , N . Here T is the sample time interval of the digital control system, and the desired trajectory is N steps long. The error is e(kT ) = yD (kT ) − y(kT ). The simplest form of ILC is based on pure integral control concepts being applied in repetitions to each of the time steps of the problem. Stated in words for a robot link, if the robot link were 2 degrees too low at a certain time step in the last run or repetition, then add 2 degrees, or 2 degrees

194

R.W. Longman and K.D. Mombaur

times a learning gain ψ, to the command this repetition. Mathematically, this is written as (4) uj (kT ) = uj−1 (kT ) + ψej−1 ((k + 1)T ) where j is the current repetition number, j − 1 is the previous repetition. Based on the ILC description above, u represents the command to the feedback control system, but we note that some of the ILC literature accomplishes the learning by modifying the error signal going into the controller or the manipulated variable coming out of the controller instead, and then the u in (4) either represents the error signal from the comparator or the w output of the controller as discussed above (1) [20]. The +1 in the argument of the error is introduced to account for the one time step delay going through the feedback control system (or the plant equations), i.e. there is usually a one time step delay from the time step in which one changes the command (or the manipulated variable) to the ﬁrst time step in the output where a resulting change is observed. The computations of the command (or manipulated variable) history to use in the next run can be made between runs, computed in a batch mode. The RC equivalent of this learning law is used when one wants to execute a periodic command, and this time the period is N time steps long. The mathematical expression of the corresponding RC law becomes u(kT ) = u((k − N )T ) + ψe((k − N + 1)T )

(5)

Instead of looking back to a previous repetition, one looks back one period. Note that unlike ILC which makes a batch update of the command history for the next repetition, RC normally runs with updates made every time step, in real time. ILC law (4) is almost always stable for suﬃciently small gains ψ, but the learning transients are very likely to be impractical [8]. However, there is an important exception to this that occurs when the sample time of the ILC updates is suﬃciently long that the system comes close to a steady state response before the next update arrives. RC law (5) is usually unstable. In both cases the error may decrease very substantially in the ﬁrst few iterations or periods, and then the error starts to grow [9, 10]. It can be that one is satisﬁed with this level of improvement and simply stops the process when the error is a minimum. To improve performance and obtain stability robustness one normally generalizes the above laws to include a dynamic compensator in place of the gain ψ, and introduces a zero-phase low-pass ﬁlter cutoﬀ of the learning [4, 24, 22, 21, 8]. Equations (4) and (5) can take the form

(6) uj = F uj−1 + Lej−1 U (z) = F (z)z −N [U (z) + L(z)E(z)]

(7)

In (6) the underbar indicates a column matrix of the history of the associated variable in a repetition, and F and L are matrices representing the low pass

ILC and RC for Periodic Gaits

195

ﬁlter and the compensator, respectively. Equation (7) for RC converts to the z-transform domain with F (z) and L(z) being the transfer functions of the cutoﬀ ﬁlter and the compensator.

4 Dynamic Models for Walking Motions The purpose of this section is to present the mathematical models of walking to which the concepts of ILC/RC will be applied. We start by giving the general form of dynamic walking models, and then present the speciﬁc stiﬀlegged biped walker used for the numerical tests later in this paper. Mathematical models of gaits involve distinct model phases with possibly diﬀerent degrees of freedom, each described, in the general form, by a diﬀerent set of diﬀerential equations. These can be ordinary diﬀerential equations (ODEs) q(t) ˙ = v(t) v(t) ˙ = a(t) = M

(8) −1

(q(t), p) · f (q(t), v(t), w(t), p)

(9)

In these equations, the vector q contains the position variables of the system, and v the corresponding velocities; together they form the vector of state variables xT = (q T , v T ). The vector y used in the context of ILC and RC is typically equal to q. The scalar t is the physical time, a the vector of accelerations, w(t) are the input torques or forces of the robot, and p is the vector of model parameters (like geometric or inertial data). M denotes the mass matrix, and f the vector of forces. Alternatively, depending on the choice of coordinates, one may obtain a system of diﬀerential-algebraic equations (DAE) of index 3 for some or all phases M (q(t), p) · a = f (q(t), v(t), u(t), p) − GT (q(t), p)λ gpos (q(t), p) = 0

(10) (11)

with the Lagrange multipliers λ, the constraint equations gpos , and their par∂gpos . We formulate the DAEs in the equivalent index 1 tial derivatives G = ∂q form with invariants q(t) ˙ = v(t)

v(t) ˙ = a(t) f (q(t), v(t), w(t), p) a M (q(t), p) G (q(t), p) = γ(q(t), v(t), p) λ G(q(t), p) 0

with the abbreviation

(12) (13)

T

(14)

gpos = g(q(t), p) = 0

(15)

gvel = G(q(t), p) · q(t) ˙ =0.

(16)

196

R.W. Longman and K.D. Mombaur

γ(q(t), v(t), p) = −v T

d G(q(t), p) v. dq

(17)

Phase boundaries are implicitly deﬁned by the roots of switching functions si (t, q(t), v(t), p) = 0 .

(18)

At these switching points, there may be discontinuities in the right hand side of the linear system, i.e. ∆f (q, v, w, p), ∆γ(q, v, p) (which translates into discontinuities in the accelerations ∆a), or even in the velocities, ∆v(t, q, v, w, p), i.e. in the state variables themselves. Walking problems also involve a number of complex linear and nonlinear, coupled and decoupled equality and inequality constraints; e.g. the periodicity constraints on the state variables (or a ˜(0). The cycle time Tcycle is generally a priori subset thereof) x ˜(Tcycle ) = x unknown. In this paper, we investigate the simple example of a planar stiﬀlegged biped walker with two degrees of freedom. The state variables of this robot are the stance leg angle φ1 and the swing leg angle φ2 , and the corresponding velocities xT = (φ1 , φ2 , φ˙ 1 , φ˙ 2 ). The robot has two torque actuators - one corresponding to each degree of freedom - the ﬁrst one w1 (t) at the hip, and the second one w2 (t) at the ankle to replace the action of a foot with an actuated ankle joint. For repetitive control problems it is convenient to introduce a second set of state variables corresponding to the torques with x ¯T = (θ1 , θ2 , θ˙1 , θ˙2 ), where θ 1 = φ1 − φ2

(19)

θ 2 = φ1

(20)

This model can be considered as an extension of the classical passive-dynamic stiﬀ-legged bipeds of McGeer [12]. The robot is shown in Fig. 2. It is characterized by three free parameters p = (m, l, c)T with

Fig. 2. Sketch of stiﬀ-legged biped investigated in this paper

ILC and RC for Periodic Gaits

197

Fig. 3. Two steps of periodic reference solution of stiﬀ-legged biped

• the mass of each leg, denoted by m • the leg length l • the relative location of the leg’s center of mass measured from the hip, c. Using these three parameters, the moment of inertia Θ of a leg is deﬁned as Θ=

1 2 ml (1 + 2c2 − 2c) . 6

(21)

One cycle of the gait model includes one step of the robot followed by a leg switch, and not a full physical gait cycle consisting of two steps (as presented in Fig. 3). Applying periodicity constraints to this model assures the generation of equal right and left steps, which would not necessarily be the case otherwise. One cycle of this model consists of one continuous phase describing the forward swing and a discrete phase including the sudden change of velocities at foot contact and the leg switch. The dynamic equations of this robot model are φ¨1 =F (22) M· φ¨2 with mass matrix

198

R.W. Longman and K.D. Mombaur

M=

m11 m12

(23)

m21 m22

with m11 = 2ml2 + Θ − 2ml2 c + ml2 c2

(24)

m12 = ml2 c sin φ2 sin φ1 − ml2 c cos φ2 cos φ1 m21 = ml2 c sin φ2 sin φ1 − ml2 c cos φ2 cos φ1

(25) (26)

m22 = ml2 c2 + Θ

(27)

and force term     F =   

−mφ˙ 22 l2 c sin φ2 cos φ1 + mφ˙ 22 l2 c cos φ2 sin φ1 +2mgl sin φ1 − mgl sin φ1 c + w1 + w2 −mφ˙ 21 l2 sin φ1 c cos φ2 + mφ˙ 21 l2 cos φ1

       

(28)

c sin φ2 − mglc sin φ2 − w1 The end of the step is determined by the equation s(t, x, p) = φ1 + φ2 = 0

(29)

The torques at hip and ankle are produced by feedback control systems using proportional control and rate feedback following a commanded trajectory θi,C : wi (t) = K1 (θi,C − θi ) − K2 θ˙i

(30)

More details about this robot model as well as a description of possible extensions using springs and dampers in parallel with the torque actuators are given in [18].

5 Open Loop Stable Gaits Previous research by the authors established that it is possible to have running gaits that are open-loop stable, meaning that they will return to the stable gait after small enough disturbances to position and velocity. This is accomplished without any feedback adjustment of the torque histories being applied to each joint. In the motions of ballet dancers and athletes, one suspects that there is often some inherent stability of the motions used, and we see that in running, hopping and somersaults this is also possible for robots [14, 17, 16]. In is generally preferable to create systems that are open loop stable and use feedback to augment the stability, than to rely totally on feedback to stabilize the motion. Pilots prefer to ﬂy airplanes that do not immediately go unstable if there is a failure in the attitude control system. With this in mind, we consider implementing an open loop stable gait to test the principles of ILC and RC. Numerically solving for such gaits for

ILC and RC for Periodic Gaits

199

actuated robots is not trivial and requires an appropriate selection of model parameters p and driving torques w(t). We have developed special numerical optimization techniques [17, 18] that can produce self-stabilizing solutions for walking robots. They have been applied to stabilize a series of diﬀerent monopod and biped robots [15], one new example is given in another paper in this proceedings volume [13]. As a stability measure, we use the spectral radius of the Jacobian of the Poincar´e map which is associated with the periodic solution. If the spectral radius is smaller than one, the solution is asymptotically stable, and if it is larger than one, the solution is unstable. For the stiﬀ-legged biped described above, we have determined an open loop stable solution that is characterized by a spectral radius of 0.7, well below one, but which is also very eﬃcient and requires only small torque inputs. The parameters p of this solution are m = 1 kg, l = 0.1 m, and c = 0.25; the cycle time is 0.4586 s. The initial values are xT0 = (0.25, −0.25, −1.943, −2.688)T . The corresponding torque inputs as well as the trajectories of the angles and rates are shown in Fig. 4. More information about the solution, as well as the objective functions used to create it, are given in [18].

6 Learning to Execute Open Loop vs. Closed Loop Stable Gaits 6.1 Problem Statement One can pose a couple of diﬀerent kinds of gait problems that might beneﬁt from use of ILC or RC: • Problem 1. As discussed above, one expects that there are beneﬁts to using gaits that are open loop stable, so that there is already some inherent stability to the motion. An open loop stable solution obtained numerically gives a torque input history for each joint, the resulting output history or gait, and its derivative. The next step is to design a feedback control system for each link, since ILC and RC normally adjust the command to feedback control systems. The objective of the ILC/RC is to succeed in making the control system in hardware execute the chosen gait, i.e. the chosen time histories for each joint angle. • Problem 2. If one were not concerned with open loop stable solutions, one could include the feedback control system equations with the robot dynamic equations, and design the gait using as inputs the position command histories given to the feedback control system instead of the torques applied to the robot dynamics. One can include the controller gains as parameters to be optimized (as well as the original model parameters p above) while ﬁnding the gait based on a chosen optimality criterion. Of course, the solution is most likely not open loop stable. Because the feedback controller equations have been used in the design of the gait, the hardware would

200

R.W. Longman and K.D. Mombaur

Fig. 4. Torques and states for open-loop stable solution

actually perform the gait desired if the model used for both the robot dynamics and the control system dynamics perfectly represent the behavior of the hardware. The ILC/RC might be used to perfect the execution of the gait by ﬁxing errors caused by inaccurate or incomplete modeling of the robot dynamics, actuators, and control system components. 6.2 Implementation Issues For both problems the ILC/RC result serve as the basis for implementing the desired motion on level ground. With suﬃcient stability perhaps this is all that one needs for reasonably level ground. For more uneven terrain, one may next try to design an outer loop that adjusts the commands to the feedback control systems to handle such things as uneven ground. The outer loop is

ILC and RC for Periodic Gaits

201

analogous to the upper level trajectory generator in industrial robots and might use additional sensor information such as vision feedback to modify the gait mode to adapt to the terrain. This paper will consider issues in addressing Problem 1. ILC/RC are methods of solving inverse problems: given a desired output of some dynamic system or component, ﬁnd the input that would produce that output. It is normally done with the physical robot hardware, but of course one can also use a mathematical model. This gives rise to three possible levels of implementing ILC/RC: (i) Do the iterations on the hardware. (ii) Do the iterations on a computer using a model. If the model is good enough, this can work when the solution is applied to the hardware. (iii) Do (ii) to get an initial command to give in hardware, and then continue the ILC/RC iterations in hardware to correct for any deﬁciencies in the model. The numerical studies reported below, directly illustrate (i) where one presumes the computer model is functioning as the real world model. They also automatically illustrate the process one goes through in performing (ii). And then by introducing some changed parameter, i.e. inaccurately modeled parameter, and continuing the learning process one can illustrate (iii). There are two short cuts for accomplishing (ii), one of which simply eliminates the need for (ii) altogether. These are: use of torque feedforward in the control system design, and do an inverse problem on the controller alone, instead of the complete system. 6.3 Torque Feedforward Perhaps the most logical implementation of the open loop stable gait solution is to use torque feedforward as in Fig. 5. The solution is a torque history w(t), a desired output history or gait, yD (t), and its derivative y˙ D (t). Since we consider a decentralized implementation, there is a separate controller for each joint angle with its own desired output history yD (t). If the actuator actually applies this torque to the robot links, and we give yD (t) as the command yC (t), and if the robot model was perfect, then the error signal e(t) would be zero in the block diagram. Then the feedback only starts to function if there is some deviation from the desired trajectory. Several comments apply: (1) The actuator may not apply the torque we want. If it is a DC motor with voltage input being adjusted by the controller (and the feedforward signal) it will not apply the intended torque, but if one can use current as the input it will accomplish the goal if one knows the motor constants correctly. In order to try to do this, one often uses a current feedback loop around the motor. In addition, the back electro motive force (emf) introduces a rate feedback inherent in the motor. Hence, the actuator has dynamics of its own and will not exactly apply the intended torque.

202

R.W. Longman and K.D. Mombaur

Fig. 5. Feedback control system using torque feedforward

(2) In positioning systems it is often desirable to use rate feedback, meaning the feedback loop takes a measurement of the output, makes a separate measurement of the output rate, which is multiplied by a constant and then the two are added together to produce the signal subtracted from the command to get the “error”, e(t). If this is being done, then one must compute what this signal would be when the output is the desired output, and use it as the command given the control system. (3) The approach totally avoids solving any inverse problem. But it does not ﬁx any errors related to use of an imperfect model or an imperfect actuator, although the feedback loop may make partial adjustments. Then one can apply ILC/RC in hardware to ﬁx remaining errors.

6.4 Inverting Controller Equations Consider the feedback control block diagram of Fig. 6 including the rate feedback that is typically used in robotics. The usual ILC/RC application solves the inverse problem, given the desired output y(t) = yD (t), ﬁnd input yC (t) to produce it. In the process of having solved for the desired periodic gait, we know more than just the desired output, we also know its derivative, and the torque w(t). Therefore, we can instead solve the inverse problem for the blocks introduced for control: given output w(t), desired position history yD (t) and its velocity (which together determine the feedback signal) ﬁnd yC (t). In the examples below, we use an idealized proportional control system with rate feedback, and in this simple case doing the suggested inverse problem is simple and immediate. Suppose that the actuator can be represented by a simple gain, and this gain can be combined with the proportional control gain in the controller block, and the product called K1 . The feedback signal is yD (t) + K2 y˙ D (t), where K2 is the rate feedback gain. Then

Fig. 6. Feedback control system using rate feedback

ILC and RC for Periodic Gaits

w(t) = K1 (yC (t) − [yD (t) + K2 y˙ D ])

203

(31)

Substituting the computed open loop stable time histories, one can solve for the needed command yC (t) to produce the gait. In simple situations one can do this. In the real world the control system design is likely to be more complicated, requiring inversion of dynamic equations, which is the domain of ILC/RC. Classical control system designers are likely to introduce a low pass ﬁlter, possibly as a noise ﬁlter, possibly as a way to control the bandwidth of the system to avoid exciting vibrations. They are likely to introduce compensators to modulate the frequency response behavior, which introduces extra dynamics with both poles and zeros. And, as discussed above, the actuator can have some dynamics. Just introducing the back emf of a motor puts in a rate feedback loop feeding the motor, which is missing in the block diagram. If one has a full computer model of all of this, one can aim to solve this inverse problem, and ILC/RC might again be an appropriate method to use.

7 Some Non-Standard ILC/RC Problems The gait problem does not immediately ﬁt the ILC or RC descriptions. The following two non-standard ILC/RC problems address some of the issues related to gait problems. Timing Belt Drive Problem. Figure 7 shows a double reduction timing belt drive system that might be used in a copy machine when one needs to have a very uniform velocity of the output shaft. Using a well designed velocity feedback control system, the frequency content of the velocity error is given in Fig. 8 (left) [6, 7]. All of the peaks are related to inaccuracies in the gearing, and include errors that have the periods of one rotation for each shaft, and each belt, including fundamental and harmonics. In addition, the large peaks at 80 Hz and 240 Hz are at the frequencies for tooth meshing of each timing belt. Because gearing is involved, all of these frequencies have a common period which can be used by a repetitive control system to eliminate the peaks. The best experimental result is shown in Fig. 8 (right) where all of the peaks have been eliminated. However, this problem does not completely

Fig. 7. Double reduction timing belt drive system

204

R.W. Longman and K.D. Mombaur

Fig. 8. Frequency spectrum of velocity error of timing belt system using feedback only (left) and at repetition 50 using batch phase cancellation RC (right)

ﬁt the standard repetitive control assumptions, because as the velocity varies, so does the period. The error to be corrected is actually periodic with the output shaft rotation angle, not with time. To address this issue, these experiments used an index pulse on the output shaft to know when the next period started. The data from the previous period was taken at evenly spaced time steps, and sometimes had more steps than the correct number, and sometimes had less. Some ad hock rule was used to decide what to do when there was missing data for the update. If the period had varied more, one might have made some adjustments to match the time scales for each period. If one has measurements of the angle at each time step, one could do interpolation in the data to explicitly make updates related to each angle instead of each time. There are in fact many repetitive control applications that have this same modeling diﬃculty. Cam Problem. Figure 9 (top) shows a cam follower system driven by a DC motor, nominally at a constant speed. The cam is normally designed with its lift curve, the following dwell, the return curve, and the subsequent dwell, all assumed to be made with a prescribed constant cam rotation rate. Of course, as the cam rotates the lever arm to the contact point lengthens and shortens making the resistance to rotation vary with cam angle. The change in resistance to motion is a disturbance to any speed control system. This means that the resulting lift displacement history, dwell durations, and return history as a function of time are not the intended histories or durations. In addition, there are imperfections in the machining of the cam. Suppose we would like to ﬁx these errors by using the velocity control system for the shaft rotation rate to speed up and slow down the cam in such a way as to have the lift and return curves be the intended functions of time, and have the dwell parts be the desired time durations.

ILC and RC for Periodic Gaits

205

Fig. 9. Cam follower system, and comparison of desired cam displacement (solid line) and displacement after 50 repetitions

This problem has several unusual aspects. First, there are four phases, the rise, the dwell, the return, and the following dwell. Second, during the dwell phases presumably the cam has a constant radius so that there is no change in lift, but even if this is not true one is not able to ﬁx the problem by changing the speed. So the only objective to be accomplished in the dwell phases is to make sure they have the right time durations, so that the next phase starts at the right time. Reference [23] reports both simulations and experiments in learning to get the desired output curves. What was done was to learn the lift phase ﬁrst. Once it converged to a satisfactory error level, the iterations to learn the next phase started. The dwell phases computed the average velocity of the dwell from the end minus start angles, divided by the end minus start times. The error in this average velocity was multiplied by a gain such as ψ and added to the command given in the previous learning cycle to produce the command during the dwell this cycle. Figure 9 (bottom) shows an example of learning iterations after the lift part of the curve has been learned, and iterations are being made to get the top dwell right. Errors in the return are being ignored until the learning of the top dwell is complete. It is fortunate in this problem that the initial conditions for the phase being

206

R.W. Longman and K.D. Mombaur

learning are not heavily dependent on whether the previous phase has been learned already, because the dwell phases allow time for decay of transients. Hence, this problem was treated as a sequence of ILC problems for each phase learning one at a time. The simple learning law of equation (4) was used, but with a chosen time step lead in the error term, creating a linear phase lead compensation [8]. No attempt was made to use a zero-phase low-pass ﬁlter to stabilize the process. This was done partly for simplicity, and partly because of nonlinearities in the system. A relatively long sample time for the ILC updates was used to slow the growth of the instability, and the learning process was stopped before the instability started to become evident in the behavior.

8 Approaches to Handling the Non-Standard Nature of Gait Problems Applying ILC or RC to the gait problem has some of the same issues as the above problems, but introduces additional diﬃculties as well. The equations are highly nonlinear, and include jump discontinuities. It could be a challenging problem to deal with the nonlinear nature in some direct way, and one would prefer to try to approach the problem as in the nonlinear cam problem, using a simple learning law that might improve the error substantially before an instability sets in, and then stop the learning when the error is a minimum. Note however, that such methods are more successful in ILC than RC, and the gait problem seems to have more relationship to RC problems. We comment that learning high frequency components can be slow and can create stability problems, and discontinuities even in the derivative of variables being controlled introduce some high frequency components. The problem has distinct phases with ground impacts denoting the start of a new phase. As in the timing belt problem, the period or duration of a phase varies with each cycle, the index pulse or impact indicating when the next cycle or phase starts. As in the cam problem the duration of each phase is important. The phases in the cam problem used diﬀerent learning laws and started with reasonably repeating initial conditions when learned in sequence, so treatment as a sequence of ILC problems worked. But like RC the initial conditions for each period or phase in the gait problem do not repeat until convergence, indicating use of an RC formulation. The RC control laws from phase to phase need not be particularly diﬀerent, but the fact that the phases have diﬀerent duration may introduce jumps in the learning process across phase boundaries, the same points where there can be jump discontinuities in velocities. It is not clear how these jumps will aﬀect the learning process. In the next section several diﬀerent learning laws will be studied. One immediate issue to consider is the question of how to look back at the current phase in the previous cycle to pick which data point is most relevant to the current update. The standard RC looks back at the corresponding time step (modiﬁed by the usual one step time delay). But since the duration is diﬀerent from one run to the next,

ILC and RC for Periodic Gaits

207

perhaps it would be better if one looked at data corresponding to the same percentage of time in the cycle.

9 Numerical Investigation of Possible Learning Schemes In this section we present four diﬀerent approaches to the repetitive control of walking motion and discuss their eﬀects on the example of the stiﬀ-legged biped described in Sect. 4 with the simple feedback controller (30). Each law describes an algorithm to compute the inputs yC,j (kT ) to the feedback controller at sample times k (k = 1, ....Nj ) of cycle j, depending on the inputs of the previous cycle yC,j−1 , errors of the previous cycle etc. All these laws have in common that they rely on a synchronization of the phase with a signiﬁcant event occurring once per cycle – in this case the touchdown of the swing foot, which is also very easy to detect in practice. This event is starting a new phase with relative time in the phase equal to zero. Therefore – even though the problem is closer to RC - we prefer to use a notation which is more of ILC type with the sampling time index k reset to 0 in every cycle. Note that this is purely by mathematical convenience and does not inﬂuence results. All RC laws presented depend on one or more gains that can be tuned and typically have a large impact on the performance. We have investigated some sets of constants for each law (without doing a real optimization); and we present for each law the best constants we have found so far. No proof of convergence is given for these learning laws. Given the nonlinear nature of the problem with multiple phases and jump discontinuities it might be very diﬃcult to establish such proofs. However, as noted in [8], good learning transients can be more important than stability (i.e convergence of the learning scheme). Furthermore [9] demonstrates that unstable learning laws can be very useful in applications. In order to allow a comparison of the diﬀerent laws, we display the following result plots: • for each law, the error histories of angles θ1 and θ2 over time (shifted by the duration of the ﬁrst cycle), see Figs. 10, 12, 14, and 16. • for each law, a comparison of the outputs for angle θ1 , the corresponding reference trajectories, and the commanded trajectory at the beginning (t = 0 s...2 s) and at the end (t = 18 s...20 s) of the investigated learning process, see Figs. 11, 13, 15, and 17 • the RMS errors for each cycle of all four laws, in overview Fig. 18 • the development of cycle times αj over a number of cycles for all four laws, in Fig. 19.

R.W. Longman and K.D. Mombaur

Errors (RC Law 1)

208

0.1 0.05 0 -0.05 -0.1 -0.15 -0.2 -0.25 -0.3 -0.35

theta1 theta2

0

2

4

6

8 10 12 14 16 18 20 time

Fig. 10. Error histories of angles θ1 and θ2 using RC law 1

9.1 RC Law 1

with

yC,j (kT ) = yC,j−1 (kT ) + ψ1 ej−1 ((k + 1)T ) ej−1 (kT ) = yD (kT ) − yj−1 (kT )

(32) (33)

This ﬁrst law is the most simple and straightforward one: The error term ej−1 that is used to correct the input of the system only compares the actual trajectory of the previous cycle and the reference trajectory at identical time points. The knowledge about the duration of the previous cycle is not used. The constant ψ1 is chosen as 0.1. As shown in Fig. 10, the law works quite well – despite its simplicity – to reduce the position variable errors (especially the RMS error of the relative hip angle θ1 is reduced, while the error of the absolute stance leg angle θ2 goes up a little again). However, this law does a poor job correcting the wrong initial cycle times (Fig. 19). Figure 11 gives some more details about the learning process, since it shows a comparison of the actual output angle θ1 , the desired angle θ1,D , and the commanded angle θ1,C at 2 diﬀerent intervals of the learning process. The upper picture shows the ﬁrst few cycles: as in all other cases we start by commanding the desired trajectory, with the result that the output trajectory is far oﬀ. The lower picture shows cycles between 18 and 20 s (after roughly 36–40 cycles modiﬁed by law 1), with the desired and actual trajectory being quite close, and the commanded trajectory being quite diﬀerent. These pictures also show the adjustment of phase times (the reference trajectory does a step change to zero after termination of the step, while the actual output step is still not ﬁnished) with a large error (18%) in the beginning, and a smaller, but still a signiﬁcant diﬀerence (8%) at the end.

ILC and RC for Periodic Gaits

209

0.6 0.4 0.2 0 -0.2 -0.4 -0.6 0

0.5

1 time

1.5

2

0.6 0.4 0.2 0 -0.2 -0.4 -0.6 18

18.5

19 time

19.5

20

Fig. 11. Comparison of output trajectory (solid line), reference trajectory (dashed lines) and commanded trajectory (crosses) for angle θ1 using RC law 1, at beginning (top ﬁgure) and end (bottom ﬁgure) of learning process

9.2 RC Law 2

with

yC,j (kT ) = c1 (αj−1 ) · (yC,j−1 (αj−1 kT ) + ψ1 ej−1 ((k + 1)T ) (34) ej−1 (kT ) = yD (kT ) − yj−1 (αj−1 kT ) (35) Tcycle,j−1 αj−1 = . (36) Tcycle,ref

In this law, the constant ψ1 is again 0.1. This second law uses the factor αj−1 to introduce information about the previous cycle time and by computing the error between corresponding (and not identical) points of the current and reference cycle. For the evaluation of the right hand side of eqn. (35), linear interpolation is used between sample points. But it is important to note that while this error computation may be the more logical one, it does not punish

210

R.W. Longman and K.D. Mombaur

0.15 Errors (RC Law 2)

0.1 0.05 0 -0.05 -0.1 -0.15

theta1 theta2

-0.2 0

2

4

6

8 10 12 14 16 18 20 time

Fig. 12. Error histories of angles θ1 and θ2 using RC law 2 0.6 0.4 0.2 0 -0.2 -0.4 -0.6 0

0.5

18

18.5

1 time

1.5

2

0.6 0.4 0.2 0 -0.2 -0.4 -0.6 19 time

19.5

20

Fig. 13. Comparison of output trajectory (solid line), reference trajectory (dashed lines) and commanded trajectory (crosses) for angle θ1 using RC law 2, at beginning (top) and end (bottom) of learning process

ILC and RC for Periodic Gaits

211

Errors (RC Law 3)

0.1 0.05 0 -0.05 -0.1 -0.15

theta1 theta2

-0.2 0

2

4

6

8 10 12 14 16 18 20 time

Fig. 14. Error histories of angles θ1 and θ2 using RC law 3

any errors of the cycle time; in fact a linearly scaled slower or faster cycle would actually lead to an error of zero. So it is important to introduce some correcting factor for wrong αj−1 which we do in the form of c1 , which is a function of αj−1 that has to satisfy c1 (1) = 1. Again, there are obviously many possible choices, and in this case we have set it to c1 (αj−1 ) = √

1 . αj−1

(37)

As shown in Fig. 19, this law does a better job than the ﬁrst one in correcting the cycle duration. The reduction of position errors is roughly the same as for the ﬁrst law with the inverse eﬀect on the two degrees of freedom: this time errors of the stance leg angle are corrected better than errors of the relative hip angle (see Fig. 18). However, as in the case of law 1, there is no continuous reduction in one of the position errors, and the development beyond the investigated time frame is unclear. But we expect that it would be possible to improve the performance of this law with a tuned factor c1 (α). There is however another possibility to improve the adjustment of cycle times (instead of using factor c1 ) which is the inclusion of error terms on velocity level. The performance of this approach is investigated in the following two RC laws, numbers 3 and 4. 9.3 RC Law 3

yC,j (kT ) = yC,j−1 (αj−1 kT ) + ψ1 epos,j−1 ((k + 1)T ) with

+ψ2 evel,j−1 ((k + 1)T ) epos,j−1 (kT ) = ej−1 (kT ) = yD (kT ) − yj−1 (αj−1 kT )

(38) (39)

evel,j−1 (kT ) = y˙ D (kT ) − y˙ j−1 (αj−1 kT )) and αj−1 as above.

(40) (41)

212

R.W. Longman and K.D. Mombaur 0.6 0.4 0.2 0 -0.2 -0.4 -0.6 0

0.5

1 time

1.5

2

0.6 0.4 0.2 0 -0.2 -0.4 -0.6 18

18.5

19 time

19.5

20

Fig. 15. Comparison of output trajectory (solid line), reference trajectory (dashed lines) and commanded trajectory (crosses) for angle θ1 using RC law 3, at beginning (top) and end (bottom) of learning process

0.2 Errors (RC Law 4)

0.15 0.1 0.05 0 -0.05 -0.1 theta1 theta2

-0.15 -0.2 0

2

4

6

8 10 12 14 16 18 20 time

Fig. 16. Error histories of angles θ1 and θ2 using RC law 4

ILC and RC for Periodic Gaits

213

0.8 0.6 0.4 0.2 0 -0.2 -0.4 -0.6 -0.8 0

0.5

18

18.5

1 time

1.5

2

0.8 0.6 0.4 0.2 0 -0.2 -0.4 -0.6 19 time

19.5

20

Fig. 17. Comparison of output trajectory (solid line), reference trajectory (dashed lines) and commanded trajectory (crosses) for angle θ1 using RC law 4, at beginning (top) and end (bottom) of learning process

This law stems from the above RC law 2 skipping the leading factor c1 , but adding another correcting term which is proportional to the velocity errors. The constants are chosen as ψ1 = 0.1, and ψ2 = 0.01. As the Figs. 14, 15, 18 and 19 show, this law works extremely well both in correcting state errors and cycle duration. The cycle duration αj is correct to 3 digits after only 3 cycles. The diﬀerence between the desired and actual output angle θ1 is barely visible after 18 s (in the lower part of Fig. 15). 9.4 RC Law 4

yC,j (kT ) = yC,j−1 (αj−1 kT ) + ψ1 epos,j−1 ((k + 1)T ) with

+ψ2 (1 − αj−1 )2 evel,j−1 ((k + 1)T ) αj−1 , epos,j−1 , epos,j−1 as above.

(42) (43)

RC law 4 is a modiﬁed version of law 3 with an additional factor in front of the velocity error term. The motivation behind this was to avoid asking too

214

R.W. Longman and K.D. Mombaur

RMS error of theta1

0.25 Law1 Law2 Law3 Law4

0.2 0.15 0.1 0.05 0 0

5 10 15 20 25 30 35 40 45

RMS error of theta2

Cycle # 0.2 0.18 0.16 0.14 0.12 0.1 0.08 0.06 0.04 0.02 0

Law1 Law2 Law3 Law4

0

5 10 15 20 25 30 35 40 45 Cycle #

Fig. 18. RMS errors of θ1 and θ2 for RC laws 1–4

much from the RC controller, namely to correct errors in 2n state variables (the positions and velocities) while only modifying n input variables (the commanded position histories). The constants chosen in this case are ψ1 = 0.1 and ψ2 = 8.0. As Figs. 16, 17, 18 and 19 show, the performance of this law is also very good, comparable to that of law 3. In the particular case investigated here, law 4 seems to do slightly better on the absolute stance angle, and law 3 does better on the realtive hip angle corrections. While the start of the learning process according to law 4 (Fig. 17) clearly shows a diﬀerent behavior than in the case of law 3, there is a clear resemblance of the commanded trajectories after 20s of the learning processes following laws 3 and 4. 9.5 Discussion of Simulation Results Four diﬀerent methods of RC for gaits have been studied here. One can either learn from the error in the previous cycle for the corresponding time step (as implemeted in law 1), or for the corresponding percent of the time used for that phase in that cycle (laws 2–4). The second approach is expected to signiﬁcantly improve the size of deviations tolerated by the algorithm before the process goes unstable. However, since (without any other correcting terms)

ILC and RC for Periodic Gaits

Relative Cycle Times

1.2

215

Law1 Law2 Law3 Law4

1.15 1.1 1.05 1 0.95 0.9 0

5 10 15 20 25 30 35 40 45 Cycle #

Fig. 19. Development of cycle times for RC laws 1–4

it does not penalize the error in the duration of the phase, one would like to introduce some extra aspect to the learning process to do so, e.g. introduce a factor depending on the relative cycle change (as in law 2). The two most promising of the four RC approaches for gait problems that we investigated seemed to be the two laws that were based on both position and velocity errors in the previous cycle (laws 3 and 4). For the computation of errors and new commands the duration of the previous cycle (relative to the desired reference cycle time) was explicitely taken into account in both cases. In digital control one would not normally include both terms (position and velocity errors) because the number of input variables, i.e. the commands given for each time step, are not enough to independently control both the position and the velocity of the output at each time step. Hence in law 4 we included (in contrast to law 3) a cancellation of the velocity error term in the case of correct cycle time adjustment. After these ﬁrst results for a speciﬁc walker and a speciﬁc feedback control law, it is hard to judge which of the two approaches might perform better in general. We think that both laws deserve further investigation on more test examples, also including more extensive studies of the most suitable choices of gains in the laws.

10 Conclusions The concepts of ILC and RC aim at improving the performance of feedback control systems by adjusting the commands given to them for the execution of a repetitive or cyclic maneuver. The purpose of this paper is twofold: ﬁrst, to discuss the general issues of transferring the ideas of ILC/RC to gait problems; and second, to present particular implementations in the form of four RC laws applied to a simple robot model with simple feedback control laws. It has been shown that the problem of ﬁxing errors in hardware execution of periodic gaits does not perfectly ﬁt the problem formulations for either ILC

216

R.W. Longman and K.D. Mombaur

or RC, but is closest to that of RC. The gait problem diﬀers in that it must be separated into phases that start at foot strike, and that the durations of the phases can vary each cycle until reaching convergence to the desired gait. Four diﬀerent methods of addressing these extra issues have been presented. The results are summarized in Sect. 9.5 above. The two most promising RC approaches investigated included both an update on the command based on the error of the previous cycle, and a second update term based on the velocity error, where we also studied including a cancellation of the velocity term in the case of correct cycle time adjustment. Both laws deliver excellent results of adjusting cycle time and eliminating tracking errors. We intend to perform further investigations along these lines, involving other robot models and combining the concept of RC with other underlying feedback control systems used in contemporary walking robots. We note that ILC and RC are notorious for exhibiting substantial decay in the error followed eventually by growth of the error, and much of the literature seeks ways to address this problem. No attempt has been made here to determine whether the RC laws result in asymptotical convergence to the desired solution when applied to the nonlinear robot dynamic equations. However, even if the laws are unstable, they may be very useful in practice. One simply uses the RC law to decrease the error, and turns it oﬀ when the error starts to grow – an approach that is used in the computer disk drive industry to good eﬀect. Our results suggest that with appropriate modiﬁcations it will be possible to use repetitive control concepts to signiﬁcantly improve the execution of chosen periodic gaits by real walking robots.

Acknowledgments The ﬁrst author would like to express his thanks to the Alexander von Humboldt Foundation for a research award that partially supported the research presented here. The second author gratefully acknowledges ﬁnancial support by the State of Baden-W¨ urttemberg through the Margarete von Wrangell program.

References [1] Proceedings of the IEEE-RAS International Conference on Humanoid Robots (Humanoids, 2005). IEEE, Tsukuba, Japan, 2005. [2] K. Berns. Walking machines catalogue. http://www.fzi.de/ipt/WMC/walkingmachines-katalog/walking-machines-katalog.html, 2000. [3] Z. Bien and J.-X. Xu, editors. Iterative Learning Control: Analysis, Design, Integration, and Applications. Kluwer Academic Publishers, Boston, 1998.

ILC and RC for Periodic Gaits

217

[4] H. Elci, R. W. Longman, M. Phan, J.-N. Juang, and R. Ugoletti. Discrete frequency based learning control for precision motion control. In Proceedings of the 1994 IEEE International Conference on Systems, Man, and Cybernetics, pp. 2767–2773, San Antonio, TX, Oct. 1994. [5] Honda. Asimo. http://asimo.honda.com, 2006. [6] Y. P. Hsin, R. W. Longman, E. J. Solcz, and J. de Jong. Experimental comparisons of four repetitive control algorithms. In Proceedings of the 31st Annual Conference on Information Sciences and Systems, pp. 854–860, Johns Hopkins University, Department of Electrical and Computer Engineering, Baltimore, Maryland, 1997. [7] Y. P. Hsin, R. W. Longman, E. J. Solcz, and J. de Jong. Experiments bridging learning and repetitive control. Advances in the Astronautical Sciences, 95:671– 690, 1997. [8] R. W. Longman. Iterative learning control and repetitive control for engineering practice. International Journal of Control, 73(10):930–954, 2000. [9] R. W. Longman and Y.-C. Huang. Use of unstable repetitive control for improved tracking accuracy. Adaptive Structures and Composite Materials: Analysis and Applications, ASME, AD-45, MD-54:315–324, Nov. 1994. [10] R. W. Longman and Y.-C. Huang. The phenomenon of apparent convergence followed by divergence in learning and repetitive control. Intelligent Automation and Soft Computing, Special Issue on Learning and Repetitive Control, 8(2):107–128, 2002. [11] R. W. Longman, Y.-T. Peng, T. Kwon, H. Lus, R. Betti, and J.-N. Juang. Adaptive inverse iterative learning control. Advances in the Astronautical Sciences, 114:115–134, 2003. [12] T. McGeer. Passive dynamic walking. International Journal of Robotics Research, 9:62–82, 1990. [13] K. D. Mombaur. Performing open-loop stable ﬂip-ﬂops – an example for stability optimization and robustness analysis of fast periodic motions. In Fast Motions in Robotics and Biomechanics – Optimization and Feedback Control, Lecture Notes in Control and Information Science. Springer, 2006. [14] K. D. Mombaur, H. G. Bock, J. P. Schl¨ oder, and R. W. Longman. Human-like actuated walking that is asymptotically stable without feedback. In Proceedings of IEEE International Conference on Robotics and Automation, pp. 4128–4133, Seoul, Korea, May 2001. [15] K. D. Mombaur, H. G. Bock, J. P. Schl¨ oder, and R. W. Longman. Open-loop stability – a new paradigm for periodic optimal control and analysis of walking mechanisms. In Proceedings of IEEE CIS-RAM 04, Singapore, 2004. [16] K. D. Mombaur, H. G. Bock, J. P. Schl¨ oder, and R. W. Longman. Selfstabilizing somersaults. IEEE Transactions on Robotics, 21(6), Dec. 2005. [17] K. D. Mombaur, R. W. Longman, H. G. Bock, and J. P. Schl¨ oder. Open-loop stable running. Robotica, 23(01):21–33, January 2005. [18] K. D. Mombaur, R. W. Longman, J. P. Schl¨ oder, and H. G. Bock. Optimizing spring-damper design in human-like walking that is asymptotically stable without feedback. 2006, submitted. [19] K. Moore and J.-X. Xu. Special issue on iterative learning control. International Journal of Control, 73(10), July 2000. [20] S. J. Oh, R. W. Longman, and Y.-P. Hsin. The possible block diagram conﬁgurations for repetitive control to cancel periodic plant or measurement dis-

218

[21]

[22]

[23]

[24]

[25]

R.W. Longman and K.D. Mombaur turbances. In Proceedings of the 2004 AIAA/AAS Astrodynamics Specialist Conference, Providence, RI, Aug. 2004. B. Panomruttanarug and R. W. Longman. Frequency based optimal design of FIR zero-phase ﬁlters and compensators for robust repetitive control. Advances in the Astronautical Sciences, 123, to appear. B. Panomruttanarug and R. W. Longman. Repetitive controller design using optimization in the frequency domain. In Proceedings of the 2004 AIAA/AAS Astrodynamics Specialist Conference, Providence, RI, Aug. 2004. N. Phetkong, M. S. Chew, and R. W. Longman. Morphing mechanisms part 1: Using iterative learning control to morph cam follower motion. American Journal of Applied Sciences, 2(5):897–903, 2005. A. M. Plotnik and R. W. Longman. Subtleties in the use of zero-phase lowpass ﬁltering and cliﬀ ﬁltering in learning control. Advances in the Astronautical Sciences, 103:673–692, 1999. Sony. Qrio – Sony Dream Robot. http://www.sony.net/SonyInfo/QRIO/, 2006.

Actuation System and Control Concept for a Running Biped T. Luksch, K. Berns, and F. Fl¨ orchinger Technische Universit¨ at Kaiserslautern, Germany {luksch,berns}@informatik.uni-kl.de, [email protected]

Summary. Dynamic walking with two-legged robots is still an unsolved problem of todays robotics research. Beside ﬁnding mathematical models for the walking process, suitable mechanical designs and control methods must be found. This paper presents concepts for the latter two points. As biological walking makes use of the elastic properties of e.g. tendons and muscles, a joint design using a pneumatic rotational spring with adjustable stiﬀness is proposed. Equations to model the spring’s dynamics as well as the supporting sensor systems and electronics are presented. For controlling the robot a behaviour-based approach is suggested.

1 Introduction Looking at todays two-legged robots and their way of walking, one might get the impression that there is still a long way to go before human-like running will be achieved. Sereval reasons for this can be found: most bipeds walk in a static manner shifting their weight carefully to always maintain a stable position; no dynamic eﬀects are taken advantage of. Energy consumption is much higher compared to natural movements as no energy from the dynamics of the system is reused. More often than not classical robot mechanics known from industrial applications are used for construction whereas more exotic concepts like elastic elements are rarely involved. The control algorithms applied are often based on a complete physical model of the robot and its environment and do seldom allow the freedom of unknown ground conditions or other external disturbances. Fast running motion is impossible for most of these machines not only because the occurring impacts could destroy the mechanics but also because the problem of controlling fast dynamical biped locomotion in unstructured environment is still unsolved. This paper presents the ﬁrst steps of the development of a two-legged walking robot addressing some of the above topics, namely the actuation system, the system architecture and a behaviour-based control concept. As some of the ideas in the design originate from natural human walking, some aspects of what biology has come up with should be mentioned.

220

T. Luksch et al.

It has been shown that mammals use the elastic components in their legs (tendons, ligaments and muscles) to run economically, while maintaining consistent support mechanics across various surfaces. Representations of the running leg as a simple spring have described the mechanics of a running leg remarkably well. McMahon and Cheng describe the spring leg having two stiﬀnesses: kleg as the actual leg stiﬀness and kvert as the eﬀective vertical stiﬀness of the runner [12]. Kerdok et al. examine the eﬀect of diﬀerent ground stiﬀnesses on the energetics and mechanics of human running [8]. Elastic elements seem to play a crucial role in natural fast locomotion concerning energy consumption, robustness and simpleness of control. This paper is structured as follows: Section 2 discusses some of the more biologically motivated research eﬀorts found in the literature. Mechanical aspects as well as control architectures are looked at. The following section will introduce the concepts of a leg design for fast locomotion. Several aspects like actuation, elastic elements or electronics will be mentioned with the focus on the knee construction as one of the most important joints. Section 4 will address the control problem. A behaviour-based architecture is proposed and ﬁrst results of periodic movement control in simulation are presented. The paper concludes with a summary and outlook.

2 State-of-the-Art Several projects can be found in the literature trying to apply more ideas from nature than most other walking robot eﬀorts. This section will introduce some of these research projects, highlighting ﬁrst the mechanical aspects, followed by control designs for fast or dynamically walking robots. 2.1 Mechanics Most contributions to constructing walking machines that are able to move energy eﬃciently, run or even jump include elastic element to store energy, absorb impacts or support actuators in producing high torques. This follows the concepts found in nature as mentioned in the previous section. The Robotics and Multibody Mechanics Group of D. Lefeber in Brussels is building the biped robot Lucy (Fig. 1(c)) featuring artiﬁcial pneumatic muscles [17], [5]. Recent eﬀorts in the ﬁeld of actuated passive walking have produced an actuator called MACCEPA (The Mechanically Adjustable Compliance and Controllable Equilibrium Position Actuator, Fig. 1(a)). This low cost design is able to set a desired angle with an adjustable compliance or stiﬀness. The low weight, easy control and low stiﬀness make it an appropriate choice for actuated passive walking, but will not allow precise or highly dynamical movement. The passive dynamic walker Veronica (Fig. 1(b)) featuring these actuators is currently under development.

Actuation System and Control Concept for a Running Biped

221

Fig. 1. (a) The MACCEPA actuator – (b) Veronica – (c) Lucy

Fig. 2. (a) PANTER leg – (b) Series Elastic Actuator – (c) Spring Flamingo

Another project using ﬂuidic pneumatic muscles is running at FZI in Karlsruhe. A prototype leg called Panter (Fig. 2(a)) for fast running locomotion has been constructed [1]. Further research is done on the dynamics of the artiﬁcial muscles and their behaviour using a quick release mechanism [9]. Among the advantages of these actuators are implicit compliance, behaviour close to the biological muscle and high forces. On the other side they need compressed air supply and are not easy to control. Already in the 1980s researchers at MIT developed the Series Elastic Actuator [18] (Fig. 2(b)). This linear electric actuator is coupled with a springdamper system. Walking robot prototypes constructed with these actuators include the Spring Flamingo (Fig. 2(c)) and the biped walker M2 [14]. The Series Elastic Actuator is now distributed by Yobotics, Inc1 . A few four-legged walking machines with elastic elements are developed by F. Iida at R. Pfeifer’s institute in Zurich. The main research interest lies in the 1

www.yobotics.com

222

T. Luksch et al.

self-stabilisation process. Experiments are carried out using robots equipped with servomotors in combination with springs controlled by synchronized oscillators [7]. 2.2 Control In Japan many research projects on biped walking exist which mostly follow the traditional concepts of robot construction and control. Honda is developing the robot Asimo and recently managed a running-like locomotion mode. But these motions and trajectories are completely pre-calculated and easily disturbed by small unknown obstacles. Similar projects outside Japan have been done by e.g. F. Pfeiﬀer (Johnnie, [13]) or the Kaist company (KHR-1, [10]). Another group working on the dynamical pre-calculation of joint trajectories is D. Lefeber’s group in Brussels. Walking and running trajectories for biped robots have been calculated [16]. The running cycle has been divided into three phases including a ﬂight phase without foot contact. The resulting force trajectories for each joint are to be combined with the output of stability reﬂexes. The implicit control of passive dynamic walkers is emerging from their sophisticated mechanics. Limb lengths and weights are chosen in a way that the robot can walk down a slop without any actuators. First in-depth experiments and theoretical examinations on this topic have been done by T. McGeer [11]. Recent work includes the research projects at TU Delft by M. Wisse et al. [19]. Some machines of this group are equipped with additional actuators like ﬂuidic muscles to substitute the potential energy gained from the slope, e.g. the robot Denise as shown in Fig. 3(a). Another example is the machine by A. Ruina et al. at Cornell University (Fig. 3(b)) using springs and DC motors [4]. The control algorithms involved are straight forward as they only have to

Fig. 3. (a) Denise from TU Delft – (b) Cornell Biped – (c) BioLeg2 from Jena

Actuation System and Control Concept for a Running Biped

223

support the implicit control of the mechanics and have even been shown to be managed by a computational unit as simple as the LEGO Mindstorms RCX computer. The walking laboratory of A. Seyfarth at Jena is examining the physics of human walking, running and jumping [15] as well as the control of legged robots. It has been shown that a rather simple control strategy can suﬃce to create walking or running behaviour if the mechanics implicitly supports it. Several small single leg prototypes have been build (e.g. Fig. 3(c)) including only one actuator driving the hip movement and a spring generating the knee motions [6]. The research cited above as well as contributions by biologists, e.g. [20], seem to suggest that the control of walking robots can drastically be simpliﬁed by clever mechanics (Embodiment). Especially including elastic elements could result in a more natural and faster walking behaviour. The following sections introduce considerations on building and controlling such a two-legged walking machine.

3 Leg Prototype for Fast Locomotion This section introduces a possible mechatronical construction for a two-legged walking machine. After discussing preconditions and the resulting design decisions, special attention is given to the knee layout focusing on a pneumatic spring unit. Some considerations on the necessary electronics and sensor systems follow. 3.1 Design of an Elastic Joint Actuation System The long term and admittingly ambitious goal of this research project is to build a walking robot that can stand, walk, run and jump. A ﬁrst prototype will consist only of a single leg with an actuated joint in the hip and the knee. As the knee is the most demanding joint in a biped locomotion system, the design of an elastic knee joint is the initial task to solve. The limb lengths are assumed to be 50 cm, the body mass to be 20 kg. As jumping will put the most stress on the mechanics this task will serve as calculation basis. Knee Design The knee design has to meet several requirements: • The construction should allow the lower leg to freely swing back without using any additional energy. To achieve this either a direct DC motor without gear (Fig. 4(b)) or a motor with gear, a clutch and additional loop control (Fig. 4(c)) could be used. As a design decision is to put as much implicit control in the mechanics as possible the ﬁrst variant would

224

T. Luksch et al.

Fig. 4. (a) serial spring – (b) par. spring, direct drive – (c) par. spring, gearbox. (with M: DC motor, G: gearbox, S: adjustable spring, S’: stiﬀ spring, B: bearing)

be preferable. But as there are not DC motors on the market producing the high torques needed, a compromise will probably be implemented in the ﬁrst testbed. • The system should be able to deal with hard impacts. Again a gear box would be the second choice as it would have to be decoupled with e.g. a spring of high stiﬀness and a clutch (S’ in Fig. 4(c)). • To support the motor in producing enough torque an elastic element is to be included to store energy. Such a spring could be mounted serial (Fig. 4(a)) or parallel (Fig. 4(b), (c)) to the motor. In the ﬁrst case the motor would have to counter-act to the spring energy. The second variant has the disadvantage of a ﬁxed equilibrium position. But as the system is to be energy eﬃcient, the second variant is chosen. The spring mounted parallel to the drive should possess a variable stiﬀness to adapt to the current situation. In the case of normal walking the spring should be soft to allow the leg to swing forward and harder before ground impact and support phase. In the case of jumping the spring should be stiﬀ for the whole cycle. These precondition and the goal to achieve a compact actuator unit mainly located in the joint itself led to the development of a pneumatic rotatory spring with variable volume. Pneumatic Spring The schematic design of the pneumatic spring can be seen in Fig. 5(b). The piston is attached to the shaft in the middle of the spring and compresses the volume V2 with inceasing rotation φ. Variable separations can be added to decrease the initial volume V1 (φ = 0). The separations are inserted at angles k where a smaller k results in higher stiﬀness of the spring. Each variable separation can be switched open by an outlet valve. This way the working volume can be changed and the spring’s stiﬀness can be adapted.

Actuation System and Control Concept for a Running Biped

225

Fig. 5. (a) ﬁrst prototype – (b) schematic drawing of pneumatic spring

The following calculation are based on several assumptions: the testbed robot (Fig. 5(a)) will consist of only one leg; the body mass m of the robot is 20 kg; the limb length l is 500 mm; the robot should be able to jump 20 cm high; the spring should be able to store the impact energy of a 70 cm fall. The torque Mm (φ) on the knee joint created by the body mass given a knee angle φ can be given as 1 Mm = mg sin( φ)l 2

(1)

resulting in a maximum torque of 85 Nm at a deﬁned maximum angle of 120 degree. The aim of the pneumatic spring is to compensate this static torque. Starting from the ideal gas equation pV = M RT we assume the adiabatic case, i.e. there is no energy transferred as heat to the environment. This leads to the adiabatic state equation following the Poisson Law: κ V1 κ κ κ pV = const ⇒ p1 V1 = p2 V2 ⇒ p2 = p1 V2 with κ being the adiabatic exponent (κ = 1.67 for one atom gases like argon, κ = 1.4 for two atom gases and κ = 1 for non-adiabatic processes). To calculate the pressure p2 we have to know the start volume V1 = V2πk and the start pressure p1 : k κ V p2 = p1 2π κ k−φ 2π V with V being the volume of the complete spring. The resulting torque only depends on this pressure p2 and the radius r and the applying lever h:

226

T. Luksch et al.

Fig. 6. (a) static torque Mm and spring torque Mspring – (b) optimal values of k

Mspring = 0.110−8 p2 πr2 h From this torque and the static torque Mm an optimal placement k of the variable separation can be calculated at which the torque Mm is compensated. Figure 6(a) visualizes the two torques. The resulting intersection curve representing the optimal choices for k can be calculated as kopt (φ) =

1−

φ 0.11010 mg sin( p1 πr 2 h

φ+φ0 π

)l

− κ1

and is plotted in Fig. 6(b). This nearly linear correlation can be approximated with multiple switchable variable separations. Further calculations have been done for the dimensioning of the shaft and sealing. The resulting design is shown in Fig. 7. Present development focuses on the integration of the motor and the pneumatic spring following Fig. 4(c). A brushless DC motor by Macon serves as actuator combined with a low gear transmission ratio. This unit is coupled with the driving axis by a clutch and an additional spring of high stiﬀness. Position encoders are placed at the motor axis and the driving joint axis. The pneumatic spring is coupled parallel to the motor.

Fig. 7. Prototypical design of the pneumatic spring

Actuation System and Control Concept for a Running Biped

227

Fig. 8. (a) DSP board with power electronics – (b) pressure measurement sensor

3.2 Sensor Systems and Electronics Fast locomotion does not only challenge the mechanics but also the electronic and computational subsystems. To successfully move at high speed several sensor systems have to be processed by controlling electronics and software. Sensor Systems To measure the pressure in the pneumatic spring as described in Sect. 3.1 a pressure sensor system has been developed capable of handling the occurring pressures (Fig. 8(b)). A programmable microchip for temperature dependent calibration as well as an A/D converter are directly integrated on the sensor board. High deﬁnition optical position encoders are used to measure the joint and shaft angles. Power Electronics To drive the DC motors power electronics have been developed and modiﬁed for high currents. The board shown as plugable module in Fig. 8(a) is capable of supplying 10 Ampere at 24 Volt and can drive brushed or brushless motors. For short periods it can even deliver higher currents. The circuit is further able to measure motor currents, this way providing a statement about the produced motor torque. DSP/CPLD Board for Fast Closed-Loop Control All sensor and actuator subsystems are connected to a DSP/CPLD board that was already designed for previous projects. Multiple boards can be connected via CAN-Bus to a embedded main computer. The DSP is able to manage the low-level control algorithms at high speed as the Motorola DSP 564803 is well suited for complex control and ﬁltering tasks. Higher level control algorithms are computed on the embedded PC. The next section will provide information on these control algorithms.

228

T. Luksch et al.

4 Behaviour Control for a Running Biped This section will introduce a concept for the control of a fast moving biped. A behaviour-based architecture is proposed for the levels above the basic torque or position control and will be brieﬂy described in the following section. To support the inherent periodic movement of the mechanics, basic periodic motion units are introduced. Some ﬁrst results from the periodic movements are presented and further steps for the implementation are discussed. 4.1 Basic Behaviour Units The proposed architecture as introduced in [2] has been successfully applied on a four-legged walking machine as well as to wheel-driven mobile indoor and outdoor platforms. The basic computational unit is a behaviour module as shown in Fig. 9. Each module processes sensor inputs e to calculate a resulting action u. This output can be inﬂuenced by two values, the activation or motivation ι coming from higher level behaviours, and the inhibition i used by lower level behaviours. In addition each behaviour features two special meta information signals, the activity a ∈ [0, 1] stating the current amount of action produced by the behaviour; and the target rating r ∈ [0, 1] corresponding to the current evaluation of the situation relative to the behaviour’s goal. The actual functionality of the behaviour is deﬁned by the transfer function F (e, ι, i) = u. All behaviours are arranged in a hierarchical network. The coordination of competing behaviours is solved by fusion nodes. These nodes calculate a resulting action based on the behaviours meta signals. Either arbitration or fusion of the values is possible. Further details on the behaviour-based architecture can be found in e.g. [2] or [3]. This behaviour architecture is to be used as reactive control layer fusioned with the basic periodic leg movement. Possible reﬂexes include posture stabilisation or reactions to disturbances.

Fig. 9. The basic behaviour module

Actuation System and Control Concept for a Running Biped

229

Fig. 10. (a) sample kinematic of human-like leg – (b) disturbed oscillation

4.2 Periodic Motion Units In order to support the periodic movement of the legs while walking, running or jumping, an oscillator-like unit can be used. Such a periodic behaviour or action unit should supply the actuator with additional torque on top of the natural movement to maintain the locomotion. A tool has been developed to test diﬀerent oscillator strategies on diﬀerent leg kinematics. The description of the legs can be given in an extended Denavit-Hartenberg format. The direct kinematic problem is automatically solved to be able to investigate foot point trajectories. Coupled periodic action units can be attached to the individual joints to generate walking movements in a simulation environment. These movements can be disturbed to observe how the control units can snap back to their oscillating behaviour. Figure 10 shows the generated 3D model of a sample kinematic with 6 degrees of freedom and the oscillator behaviour during a disturbed joint movement. It can be observed how the current joint angle movement (thick line) is accelerated (ﬁrst disturbance) or slowed down (second disturbance) depending on the current deviation to fall back to the reference oscillation (thin line).

5 Summary and Outlook This paper presented the mechanical design and ﬁrst control concepts for a running biped. For fast running or jumping locomotion that is still energy eﬃcient it is crucial to exploit the natural leg movement as done by passive walkers and to store energy during motion. For this task a rotatory pneumatic spring has been developed to be integrated into the actuation unit. It has an adjustable stiﬀness while being able to generate high torques and to withstand

230

T. Luksch et al.

the high stress emerging during jumping. The actuation unit should be able to meet the demands of a knee joint and with only slight modiﬁcations be used as a hip joint. The next research steps will be to build the proposed prototype with actuated hip and knee and to compare its performance with the results from dynamic simulation. Control algorithms following the described oscillation principles combined with a reactive control based on the behaviour-based architecture of Sect. 4.1 will be tested on the prototype leg. Future work will include the design of an elastic foot with two degrees of freedom and following that a two legged 8 DOF prototype.

References [1] J. Albiez, T. Kerscher, F. Grimminger, U. Hochholdinger, R. Dillmann, and K. Berns. Panter – prototype for a fast-running quadruped robot with pneumatic muscles. In 6th International Conference on Climbing and Walking Robots (CLAWAR), Catania, Italy, September 17–19 2003. [2] J. Albiez, T. Luksch, K. Berns, and R. Dillmann. An activation-based behavior control architecture for walking machines. The International Journal on Robotics Research, vol. 22:203–211, 2003. [3] J. Albiez, T. Luksch, K. Berns, and R. Dillmann. A behaviour network concept for controlling walking machines. In 2nd International Symposium on Adaptive Motion of Animals and Machines (AMAM), Kyoto, Japan, March 4–8 2003. [4] S. Collins, A. Ruina, R. Tedrake, and M. Wisse. Eﬃcient bipedal robots based on passive-dynamic walkers. Science, 307:1082–1085, 2005. [5] F. Daerden and D. Lefeber. Pneumatic artiﬁcial muscles: actuators for robotics and automation. European Journal of Mechanical and Environmental Engineering, 47(1):10–21, 2002. [6] H. Geyer, A. Seyfarth, and R. Blickhan. Spring-mass running: simple approximate solution and application to gait stability. Journal of Theoretical Biology, 232(3):315–328, 2004. [7] F. Iida and R. Pfeiﬀer. Self-stabilization and behavioral diversity of embodied adaptive locomotion. In Embodied Artiﬁcial Intelligence, pp. 119–129. Springer, 2003. [8] A. Kerdok, A. Biewener, T. McMahon, P. Weyand, and H. Herr. Energetics and mechanics of human running on surfaces of diﬀerent stiﬀness. J. Appl. Physiol., 92:469–478, 2002. [9] T. Kerscher and et al. Flumat – dynamic modelling of ﬂuid muscles using quick-release. In 3rd International Symposium on Adaptive Motion in Animals and Machines, 2005. [10] J.-H. Kim and J.-H. Oh. Walking control of the humanoid platform khr-1 based on torque feedback control. In IEEE International Conference on Robotics and Automation (ICRA), 2004. [11] T. McGeer. Passive dynamic walking. International Journal of Robotics Research, 9:62–82, 1990. [12] T. McMahon and G. Cheng. The mechanics of running: How does stiﬀness couple with speed? J. Biomech., 23(1):65–78, 1990.

Actuation System and Control Concept for a Running Biped

231

[13] F. Pfeiﬀer, K. L¨ oﬄer, and M. Gienger. The concept of jogging johnnie. In Proceedings of the IEEE International Conference on Robotics and Automation, 2002. [14] J. Pratt. Exploiting inherent robustness and natural dynamics in the control of bipedal walking robots. In PhD thesis. MIT Press, Cambridge, 2000. [15] A. Seyfarth, H. Geyer, M. G¨ unther, and R. Blickhan. A movement criterion for running. J. Biomech., 35(5), 2002. [16] J. Vermeulen. Trajectory Generation for Planar Hopping and Walking Robots: An Objective Parameter and Angular Momentum Approach. PhD thesis, Vrije Universiteit Brussel, May 2004. [17] B. Verrelst, R. Van Ham, B. Vanderborght, F. Daerden, and D. Lefeber. The pneumatic biped “lucy” actuated with pleated pneumatic artiﬁcial muscles. Autonomous Robots, 18:201–213, 2005. [18] M. M. Williamson. Series elastic actuators. Technical report, Massachusetts Insitute of Technology, Artiﬁcial Intelligence Lab, 1995. [19] M. Wisse. Essentials of dynamic walking, Analysis and design of two-legged robots. PhD thesis, TU Delft, 2004. [20] H. Witte, H. Hoﬀmann, R. Hackert, C. Schilling, M. Fischer, and H. Preuschoft. Biomimetic robotics should be based on functional morphology. J. Anat., 204:331–342, 2004.

Dynamical Synthesis of a Walking Cyclic Gait for a Biped with Point Feet S. Miossec1 and Y. Aoustin2 1

2

Joint Japenese-French Robotics Laboratory, Intelligent Systems Research Institute, National Institute of Advanced Science and Technology, AIST Tsukuba Central 2, Umezono 1-1-1, Tsukuba, 305-8568 Japan [email protected] Institut de Recherche en Communications et Cybern´etique de Nantes, U.M.R. 6597 Ecole Centrale de Nantes, CNRS, Universit´e de Nantes, 1 rue de la No¨e, BP 92101, 44321 Nantes Cedex 3, FRANCE [email protected]

Summary. This paper deals with a methodology to design optimal reference trajectories for walking gaits. This methodology consists of two steps: (i) design a parameterized family of motions, and (ii) determine the optimal parameters giving the motion that minimizes a criterion and satisﬁes some constraints within this family. This approach is applied to a ﬁve link biped, the prototype Rabbit. It has point feet and four actuators which are located in each knee and haunch. Rabbit is underactuated in single support since it has no actuated feet and is overactuated in double support. To take into account this under-actuation, a characteristic of the family of motions considered is that the four actuated joints are prescribed as polynomials in function of the absolute orientation of the stance ankle. There is no impact. The chosen criterion is the integral of the square of torques. Diﬀerent technological and physical constraints are taken into account to obtain a walking motion. Optimal process is solved considering an order of treatment of constraints, according to their importance on the feasibility of the walking gait. Numerical simulations of walking gaits are presented to illustrate this methodology.

1 Introduction For more than thirty years walking robots and particularly the bipeds have been the objects of research. For example Vukobratovic and his co-author [1] have proposed in 1968 their famous Zero-Moment Point (ZMP ), for the analysis of a biped gait with feet. In 1977, optimal trajectories [2] were designed for a bipedal locomotion using a parametric optimization. Formal’sky completely characterized the locomotion of anthropomorphic mechanisms in [3] in 1982. Sutherland and Raibert proposed their principle about virtual legs for walking robots in the paper [4] in 1983. Currently Humano¨ıds such as Honda biped in

234

S. Miossec and Y. Aoustin

[5] and HRP2 biped in [6] (Humano¨ıd Robotics Project 2), which are probably on the technological point-of-view the most advanced biped robots, lead to many popular demonstrations of locomotion and interaction with their environment. In parallel, some research is done on legged robots with less degrees of freedom. Here it is worked with the control, the model and the reference trajectories to design walking bipedal gaits more ﬂuid. See for example [7] where a biped with telescopic legs is studied, [8] where the famous dog Aibo from Sony is used to design biped gaits, [9] where an intuitive approach is developed for a biped locomotion or [10] where an accurate analysis of the gravity eﬀects is made to give necessary and suﬃcient conditions to ensure a cyclic walking gait for a biped without feet. In this paper, the eﬀorts are focused on the design by a parametric optimization of a walking gait. This approach necessitates two steps: (i) design a parameterized family of motions, and (ii) determine the optimal parameters giving the motion that minimizes a criterion and satisﬁes some constraints within this family. The motion obtained is later used as a reference motion. This approach is applied to a planar ﬁve-link biped without feet and with four actuators only. The family of motions considered is composed of a singlesupport phase and a double-support phase, with no impact. The minimization criterion is the integral over the motion of the square of torques. Therefore it is a criterion of torque minimization. The originality of the present work is double: • To overcome the underactuated characteristic of the biped, the four variables deﬁned as polynomials in single support are function of another generalized coordinate, the absolute orientation at the stance leg ankle. This allows to deﬁne the conﬁgurations of the biped during the single support phase, while the dynamics of the not controlled degree of freedom are still unknown. In double support, two actuated joints are also prescribed as functions of the absolute orientation at the stance leg ankle, which is a polynomial function in time. • There is a classiﬁcation and a treatment of constraints according to their importance on the feasibility of the walking gait. This paper does not address the stability of the motion obtained. The reader may refer to [11] which gives conditions of stability of the non controlled degree of freedom during the single support phase, and additionally a measure of this stability. It has been proved that the presence of the double support phase practically guarantees the stability. This article is organized as follows: the dynamical model of the biped under interest is presented in Sect. 2 for the single and the double-support phase. Section 3 is devoted to the deﬁnition of the family of reference trajectories, their constraints and their parameters. The calculation of the criterion in torque during the single support and the double support, and the optimization process to determine the optimal parameters are presented in Sect. 4. Some

Dynamical Synthesis of a Walking Cyclic Gait

235

simulation results are shown Sect. 5. Section 6 contains our conclusion and perspectives.

2 Dynamic Model 2.1 Presentation of the Biped and Notations A planar ﬁve-link biped is considered and is composed by a torso and two identical legs with knee and point feet (see Fig. 1 for a diagram of the studied biped). There are four identical motors, which drive the haunches and the knees. We note Γ = [Γ1 , Γ2 , Γ3 , Γ4 ]T the torque vector, q = [α, δ T ]T =[α, δ1 , δ2 , δ3 , δ4 ]T the vector composed of the orientation of the stance leg and the actuated joint variables, and X = [q T , xt , yt ]T the vector of generalized coordinates. The components (xt , yt ) deﬁne the position of the center of gravity of the trunk. 2.2 A Reduced Model The optimization process to determined reference trajectories, which will be presented in the next sections leads to many CPU operations. Therefore the strategy was to use a reduced model that needs less computations. To obtain this reduced model, we consider that the contact between the leg tip 1 and

xt , yt

d2

Leg 2

d3

d4

Leg 1

d1

a y

R1 x

S

Fig. 1. Biped in the sagittal plane

R2

236

S. Miossec and Y. Aoustin

the ground is acting as a pivot: there is no take oﬀ and no slipping. Then the biped conﬁguration is described with vector q only. This model is reduced by comparison to a more general model that would be written with vector X. We obtain the reduced model by using Lagrange’s equations: A(δ)¨ q + H(q, q) ˙ + Q(q) = DΓ Γ + D2 (q)R2

(1)

where A(δ)(5×5) is the symmetric positive inertia matrix of the biped. As the kinetic energy of the biped is invariant under a rotation of the world frame [12], and viewed that α deﬁnes the orientation of the biped, the 5 × 5-symmetric positive inertia matrix is independent of this variable, i.e. A = A(δ). Vector H(q, q)(5 ˙ × 1) represents the centrifugal, Coriolis eﬀects, and Q(q)(5 × 1) is the gravity eﬀects vector. DΓ (5 × 4) is a constant matrix composed of ones and zeros. D2 (q) is the 5 × 2-Jacobian matrix converting the ground reaction in the leg tip 2 into the corresponding joint torques. Taking into account Coulomb dry and viscous frictions, Γ has the following form T T q) ˙ − Fv DeΓ q˙ (2) Γ = Γu − Γs sign(DeΓ where Γs (4 × 4) and Fv (4 × 4) are diagonal matrices representing respectively the dry friction and the viscous friction. Γu is the motors torque vector when considering the joint friction. In the case of double support, the point foot 2 is in contact with the ground. Then the position variables q, the velocity variables q, ˙ and the acceleration variables q¨ are constrained. In order to write these relations, we deﬁne the position, velocity and acceleration of the point foot 2 in an absolute frame. The position of the point foot 2 is noted d2 (X). By diﬀerentiation of d2 (X) we obtain the relation between the velocity V2 = (V2x V2y )T of the point foot 2 and q, ˙ (3) V2 = De2 (q)T q˙ . By another diﬀerentiation we obtain the relation between the acceleration V˙ 2 = (V˙ 2x V˙ 2y )T of the point foot 2 and q¨, V˙ 2 = De2 (q)T q¨ + D˙ e2 (q)T q˙ = De2 (q)T q¨ + Ce2 (q, q) ˙ .

(4)

Then the contact constraints for the point foot 2 with the ground are given by the three vector-matrix equations:    d2 (X) = const ,      V2 = 0 , (5)        V˙ 2 = 0 . These vector-matrix equations (5) mean that the position of the point foot 2 remains constant, and then the velocity and acceleration of the point foot 2 are zero.

Dynamical Synthesis of a Walking Cyclic Gait

237

During the double-support phase, both legs are in contact with the ground. Then the dynamic model is formed of both vector-matrix equations (1) and (5). During the single-support phase on leg 1, the dynamic model is simply written as (1) with the ground reaction for foot 2 in the air is R2 = (0 0)T . Model (1) allows us to compute the torques and the dynamic model of α easier (10). However, it is not possible to take into account a single-support on the leg 2 with (1). Furthermore we cannot calculate the ground reaction with model (1) only. We add the two following equations, obtained from the Newton’s second law at the center of mass G of the biped + Mx ¨G = R1x + R2x (6) M y¨G = R1y + R2y − M g where M is the mass of the biped and (xG , yG ) are the coordinates of G.

3 Deﬁnition of the Walk and Its Constraints Our objective is to design a cyclic bipedal gait. There are two aspects for this problem. The deﬁnition of a parameterized family of reference trajectories and the method to determine a particular solution in this restricted space. This section is devoted to the deﬁnition of the parameterized family of reference trajectories. The optimal process to choose the best solution of parameters from the point of view of a given criterion will be described in the next section. The parameterized family of reference motions is such that one degree of freedom, which changes monotonically during a step composed of a singlesupport phases and a double-support phases, will be used as a variable to deﬁne the other degrees of freedom. These special solutions lead to a particular simple dynamical model of the biped in single support which can be calculated from (1). An impactless bipedal gait is considered because in [13] numerical results proved that the insertion of an impact with this walking gait for the studied biped is a very diﬃcult challenge. The condition found to obtain no impact was simply that the velocity of free foot must reach the ground with zero velocity. After the choice of parameters, the constraints will be determined. In the following, indices “ss” and “ds” respectively indicate the single-support phase and the double-support phase. 3.1 Restrictions of Motion Considered in Single Support During the single support, the biped has ﬁve degrees of freedom. With the four actuators for the biped, only four output variables can be prescribed. Then the biped is underactuated in single support. In previous experiments, see for example, [7, 14, 15], researchers observed that for most of walking gaits of biped robots the ankle angle α of the stance leg changes absolutely monotonically during the single-support phase. Therefore, it is possible to use

238

S. Miossec and Y. Aoustin

the angle variable α instead of time t as an independent variable during the single-support phase of the bipedal gait. As a consequence α like time will have to be monotonic. But this choice will not eliminate potentially optimal motions in the space in which we seek for solutions, since so far all motions observed were satisfying this property. Thus the four joint variables δj are prescribed as polynomial functions of the ankle angle α, δj,ss (α) (j = 1, . . . , 4). The behavior of α is governed by the dynamic model (1). To deal with the underactuation the advantage of this approach is that the complete set of conﬁgurations is deﬁned during the motion of the biped and it is not necessary to anticipate a duration for the single-support phase, which is the result of the integration of (1). The order of these polynomial functions (7) is chosen at four to specify initial, ﬁnal and intermediate conﬁgurations, plus initial and ﬁnal joint velocity variables, δj,ss (α) = aj0 + aj1 α + aj2 α2 + aj3 α3 + aj4 α4 .

(7)

Let us note that it would be possible to prescribe other variables as Cartesian variables. But to avoid the problems of singularity of the inverse geometric model in the single-support phase, we prefer to work with angular variables only. However some authors, for example [2, 16], use Cartesian coordinates of the hip for the deﬁnition of the bipedal gait. The joint variables are then prescribed. However since the biped is underactuated the evolution of the angle α must be such that the biped motion satisﬁes the dynamic model. Considering the relations (7) we introduce for the variables of the reference motion q = q(α) the following temporal derivatives q(α, ˙ α) ˙ = q ∗ α˙ (8) q¨(α, α, ˙ α) ¨ = q∗ α ¨ + q ∗∗ α˙ 2 ˙ where the notation ()∗ means partial derivative with respect to α, and the () ∗ ∗ ∗ ∗ ∗ T represent derivation with respect to time. Then we have q = [1 δ1 δ2 δ3 δ4 ] and q ∗∗ = [0 δ1∗∗ δ2∗∗ δ3∗∗ δ4∗∗ ]T . By calculating the angular momentum of the biped at the ﬁxed point S (see Fig. 1), we obtain the general form σ=

4

fi (δ1 , δ2 , δ3 , δ4 )δ˙i + f5 (δ1 , δ2 , δ3 , δ4 )α˙ .

(9)

i=1

We can obtain two ﬁrst order diﬀerential equations on σ and α (see [15])  σ˙ = −M g (xG (α) − xS )    (10)  σ   α˙ = . f (α)

Dynamical Synthesis of a Walking Cyclic Gait

239

M is the biped mass, g the acceleration of gravity, xG (α) and xS are respectively the horizontal component of the positions of the biped’s mass center and of the foot of the stance leg. The ﬁrst equation of (10) comes from the dynamic momentum equation at S when eliminating q from (7). The second equation of (10) follows from (9) when eliminating q and q˙ using (7) and (8). This diﬀerential system (10) is equivalent to the ﬁrst line of (1). By identiﬁcation, it is possible to determine f (α) and xG (α) from (1). The simple model (10) completely deﬁnes the dynamic behavior of the biped in single support for the reference motion. From (10) we can deduce that (see [17]) σ˙ =

dσ σ 1 dσ 2 1 dσ α˙ = = = −M g (xG (α) − xS ) . dα dα f (α) 2 dα f (α)

Finally this calculation leads to the relation due to [17] dσ 2 = −2M g (xG (α) − xS ) f (α) . dα

(11)

If α is strictly monotone, the integration of (11) gives α σ − 2

2 σiss

= −2M g

(xG (s) − xS ) f (s)ds

(12)

αiss

where σiss is the angular momentum at the beginning of single support characterized by the initial value αiss . Then the dynamic of the biped is completely 2 2 = α˙ 2 f 2 (α) − α˙ iSS f 2 (αiSS ) deﬁned from (10) as function of Φ(α) = σ 2 − σiss such as 2 2 Φ(α) + f (αiSS )2α˙ iSS . (13) α˙ = − f (α) α ¨ is obtained from the second equation of (10) α ¨=

M g (xG (α) − xS ) + σf ˙ (α) − σ f˙(α) =− f 2 (α) f (α)

df (α) 2 ˙ dα α

.

(14)

From the solution of the diﬀerential equation in α (11) and using relations (13) and (14) the numerical simulation to ﬁnd the optimal motion and the calculation of constraints will be easier. The authors of [17] showed that system (10) behaves like an inverted pendulum. Therefore the only possible non-monotone behavior would be that the biped fall back if the initial velocity of single support is not suﬃcient. The condition to ensure the monotony of α has been added as a constraint in the optimization process, see (18). 3.2 Restrictions of Motion Considered in Double Support In double support, the biped has three degrees of freedom. With its four actuators, the biped is over actuated. Hence the motion of the biped is completely

240

S. Miossec and Y. Aoustin

deﬁned with three prescribed degrees of freedom. For a question of convenience for the use of the inverse geometric model, the ankle angle α and both joint variables, δj (j = 1, 2) are prescribed. A polynomial function in time of third-order (15) is chosen to deﬁne α. In a concern to be homogeneous with the single support phase we deﬁne both joint angular variables δj , as polynomial functions of third-order in α. Then initial and ﬁnal conﬁgurations, and initial and ﬁnal velocities can be deﬁned for these three prescribed variables. The duration of the double-support phase is a parameter. Hence we get   

α(t) = a0 + a1 t + a2 t2 + a3 t3 ,

  δj (α) = aj0 + aj1 α + aj2 α2 + aj3 α3 .

(15)

It should be noted that there is no diﬀerential equations needed for the deﬁnition of the motion, since the biped is over-actuated in double support. 3.3 Optimization Parameters A boundary value problem has to be solved to design this cyclic bipedal gait with successive single and double-support phases. This boundary value problem depends on parameters to prescribe the initial and ﬁnal conditions for each phase. Taking into account the conditions of continuity between the phases and the conditions of cyclic motion we will enumerate now in detail the minimal number of parameters which are necessary to solve this boundary value problem on a half step k (a half step is considered as a single support and a double support). 1. Seven parameters are needed to deﬁne the initial and ﬁnal conﬁgurations in double support. The parameters αids , δ1,ids , θids , αf ds , δ1,f ds , θf ds and d, the distance between both tips of stance legs in double support are chosen. The use of the absolute orientation of the trunk, θ (see Fig. 2) instead of δ2,f ds is easier and does not change the problem. 2. Time Tds of the double support is given as a parameter. 3. The initial velocity of the biped in single support is prescribed by only three ∗ ∗ ∗ ∗ , δ2,iss . The velocities δ3,iss and δ4,iss are deduced parameters α˙ iss , δ1,iss taking into account the null velocity of the leg tip which takes oﬀ. 4. The ﬁnal velocity of the biped in single support is prescribed by only three ∗ ∗ ∗ ∗ parameters α˙ f ss , δ1,f ss , δ2,f ss . The velocities δ3,f ss and δ4,f ss are deduced taking into account the absence of impact of the swing leg tip on the ground, which is equivalent to a null velocity of this tip. 5. With the chosen order for the polynomial functions (7) (fourth order) it is necessary to specify ﬁve conditions for each function δj,ss , j = 1, . . . , 4. Then the ﬁfth coeﬃcient is calculated by deﬁning an intermediate conﬁguration. Let intermediate conﬁguration in single support be determined with the ﬁve following parameters αint , δ1,int , θint and the coordinates

Dynamical Synthesis of a Walking Cyclic Gait

241

q A

G

d

R1y

R2y B

R1x

R2x

Fig. 2. Biped in the sagittal plane (the point G is the center of mass of the biped)

(xp,int and yp,int ) of the swing leg tip. The angle αint is ﬁxed equal to αiss + αf ss . 2 Then ﬁnally the vector of parameters has eighteen coordinates ∗ p = [Tds , αids , δ1,ids , θids , αf ds , δ1,f ds , θf ds , d, α˙ iss , δ1,iss , ... ∗ ∗ ∗ δ2,iss , αf ss , δ1,f ss , δ2,f ss , δ1,int , θint , xp,int , yp,int ] .

3.4 Constraints Constraints have to be considered to design nominal gait. We will present them according to their importance on the feasibility of the walking gait. • First, no motion is possible if the distance d(A, B) between the tip of leg 2 and the hip joint, for initial and ﬁnal conﬁgurations of the double support and the intermediate conﬁguration of the single support, is such that d(A, B) > 2 × l

(16)

242

S. Miossec and Y. Aoustin

where l is the common length of the femur and the tibia. In other words, there is no solution with the geometrical model to compute δ3 and δ4 . • Constraint (16) is also taken into account during the motion of the biped in double support. The maximum value of d(A, B) in function of α is considered. • The mechanical stops of joints for initial, intermediate and ﬁnal conﬁgurations of each phase and during the motion are  −260◦ < (δ2 )min , (δ2 )max < −110◦          ◦ ◦    −260 < (δ2 − δ3 )min , (δ2 − δ3 )max < −110            

−230◦ < (δ1 )min ,

(δ1 )max < −127◦

−230◦ < (δ4 )min ,

(δ4 )max < −127◦

The notation ()max and ()min stands respectively for the maximum and minimum value over one step. • In double support the monotony condition for variable α is imposed max α(t) ˙ <0.

(17)

t∈[0,Tds ]

• In single support, the monotony condition for variable α is imposed by the inequality 2 Φmin + f (αiss )2α˙ iss >0 (18) where Φmin = minα∈[αiss ,αf ss ] Φ(α). • In single support it is fundamental to avoid the singularity f (α) = 0 to simulate one step. Then we deﬁne the following constraint min

α∈[αiss ,αf ss ]

f (α) > 0 .

(19)

Now the following constraints can be violated during the optimization process to simulate a half step. However they are important for experimental objectives. The optimization process will ensure their veriﬁcation. • Each actuator has physical limits such that  ∗ ˙1 |)  |Γ (α)| − Γ (| δ  max 1  max      ∗ ˙  |Γ2 (α)| − Γmax (|δ2 |)  max  ∗ ˙ ˙  |Γ3 (α)| − Γmax (|δ2 − δ3 |)    max     ∗ ˙  |Γ4 (α)| − Γmax (|δ4 |) max

<0 <0 (20) <0 <0

Dynamical Synthesis of a Walking Cyclic Gait

243

Function Γmax (∗) can be deduced from a template, torque actuator/velocity, given by the actuator manufacturer. • We must take into account constraints on the ground reaction Rj = ( Rjx Rjy )T in the tip of the stance leg j, j = 1 in single support and j = 1, 2 in double support. The ground reaction must be inside a friction cone deﬁned by the friction coeﬃcient µ. This is equivalent to write both inequalities Rjx − µRjy < 0 −Rjx − µRjy < 0. By summing these two inequalities, the condition of no take oﬀ is deduced ⇒ Rjy > 0 .

(21)

• There is also a constraint on the swing leg tip to avoid an impact with the ground during its transfer. This constraint is deﬁned by a parabola function % 2 & x (α) min − 1 ymax y(α) − d2 α∈[αiss ,αf ss ] where (x, y) are the coordinates of the swing leg tip and ymax is the maximum height of the parabola. • Optimal motions are deﬁned for diﬀerent velocities with the constraint d = v(Tss + Tds )

(22)

where d is the distance between the tips of stance legs (see Fig. 2), v is the desired average velocity of the biped, and Tss is the time of the singlesupport phase. The calculation of time Tss of the single-support phase is αfss 1 dα given by Tss = ˙ αiss α

4 Optimal Walk Many values of parameters presented in Sect. 3 can give a periodic bipedal gait satisfying constraints (16)–(22). Then a parametric optimization process, minimizing a criterion under nonlinear constraints, is possible to ﬁnd a particular nominal motion. Let us deﬁne this optimization process min C(p)

(23)

p

gi (p) ≤ 0

i = 1, 2, . . . , n

where p is the vector of parameters, C(p) is the criterion to minimize with n constraints gi (p) ≤ 0 to satisfy. We give now some details about the way to calculate the criterion during the single-support phase and the double-support phase, and about the optimization process.

244

S. Miossec and Y. Aoustin

4.1 Criterion To ﬁnd the nominal motion criterion CΓ , which is a torque minimizing criterion, is considered  αf ss  Tss+Tds Tds ΓTΓ 1 1 T T dµ + Γ Γ dt = Γ Γ dt CΓ = (24) d d α˙ 0

αiss

0

where Tss and Tds are the times of single support and double support. For electrical motors such as DC motors the torque is usually proportional to the induced current. Then the criterion CΓ represents the losses by Joule eﬀects to cover distance d, see [18, 19]. To consider an energy minimizing criterion, it would only be necessary to add the losses by friction in the joints. 4.2 Single-Support Phase From calculation of the integral term (12) using the polynomial functions (7), 2 we obtain Φ(α) = σ 2 − σiss . Velocity α˙ and acceleration α ¨ can be obtained with relations (13) and (14). We then have determined the dynamics of the under actuated biped in single support for a reference trajectory. The torques are determined from the four last equations of (1) q + H25 (q, q) ˙ + Q25 (q) = DΓ 25 Γ A25 (δ)¨

(25)

where A25 (4 × 5), H25 (4 × 5) and DΓ 25 (4 × 4) are the submatrices of A, H and DΓ , Q25 (4 × 1) is the subvector of Q. The invertible matrix DΓ 25 allows to determine the torque vector Γ . The ground reaction Ri = (Rix , Riy ) at the tip of the stance leg i are calculated using the equations (6). 4.3 Double-Support Phase From relations (15) α(t), α(t) ˙ and α(t) ¨ are calculated as polynomial functions of time ﬁrst at each time step, then δj (α), δ˙j (α) and δ¨j (α) (j = 1, 2) are determined. There is an inﬁnit set of solutions for the torques to realize the double support, because the biped is overactuated. Only three generalized coordinates, for example α(t), δ1 and δ2 , are necessary to describe the motion completely. Then, we can parameterize the solution of torques as function of one variable. To ﬁnd this variable we consider equation (6) and the equation of the angular momentum theorem applied at the leg tip 1. The equation of the angular momentum theorem in double support is equivalent to equation (10) but with the eﬀect of ground reaction force of foot 2. It is also equivalent to the ﬁrst line of model (1). This additional equation reads q + H1 (q, q) ˙ + Q1 (q) = −dR2y A1 (δ)¨

(26)

Dynamical Synthesis of a Walking Cyclic Gait

245

where A1 (1 × 5) and H1 (1 × 5) are the ﬁrst line of A and H, Q1 (1 × 1) is the ﬁrst element of Q. Term d is the distance between the two leg tips on the ground. Component R2x does not appear in equation (26) because the ground is assumed to be horizontal and plane. From the second line of (6) and (26), for a given acceleration of the biped there is only one solution for R1y and R2y , independent of the torques. The torques only inﬂuence R1x and R2x . For this reason, a solution for the torques can be found as function of R1x or R2x as parameter. Let us choose R2x and deﬁne the minimization problem with the associated constraint on component R2x min Γ ∗T Γ ∗ R2x

 −µR1y − R1x     −µR1y + R1x  −µR2y − R2x    −µR2y + R2x

≤0 ≤0

(27)

≤0 ≤0.

The choice of the particular solution of this optimization problem is because it is also the solution that minimizes the criteria (24). With the four last lines of the vector-matrix equations (1) and (2) a relation between torques Γ ∗ and R2x can be written Γ ∗ = J − KR2x

(28)

with K = DΓ−125 D2x 25 and ˙ + Q25 (q) − D2y 25 R2y ) + Γs sign(DΓT q) ˙ + Fv DΓT q. ˙ J = DΓ−125 (A25 q¨ + H25 (q, q) The solution R2x optΓ which minimizes the square of the torques without ∂Γ ∗ constraints is given when Γ ∗T ∂R = 0. Considering equation (28) R2x optΓ 2x is given by KT J (29) R2x optΓ = T . K K Deﬁning a minimum value R2xinf and a maximum value R2xsup , the constraints on R2x can be written under the simple form, R2xinf ≤ R2x ≤ R2xsup

(30)

Then a solution for the minimization problem (27) is given by three cases • if R2x inf ≤ R2x optΓ ≤ R2x sup then R2x = R2x optΓ , • if R2x optΓ ≤ R2x inf then R2x = R2x inf , • if R2x sup ≤ R2x optΓ then R2x = R2x sup . In the case where there is no solution, i.e R2xinf ≥ R2xsup , we choose R2x to minimize the violation of constraints such as

246

S. Miossec and Y. Aoustin

R2x inf + R2x sup . 2 In this last situation, the constraints are not satisﬁed. However, the optimization process will tend to satisfy the constraints of the motion, and the ﬁnal solution will always satisfy R2xinf ≤ R2xsup . This violation will only occur during the optimization process. R2x =

4.4 Optimization Algorithm The algorithm NPSOL, see [20] is used to solve this optimization problem with its nonlinear constraints. The sequence of treatment of constraints according to their importance is described Fig. 3. From level 0 to level 4, the constraints must be satisﬁed to simulate one step. Other constraints as the maximum velocity of the biped, the torques limits are considered in level 5. Sometimes, while solving the problem (23), the optimization process can ask a value of the criterion or the constraints in a point p0 where they are not deﬁned. Therefore an intermediate optimization process is started to ﬁnd another point pM , the closest from p0 . For example if constraints gi (p0 ) ≤ 0, i = 1, 2, . . . , m0 are not satisﬁed, pM is determined as the solution of the problem min p0 − p p

(31) gi (p) ≤ 0 i = 1, 2, . . . , m0 . Then the constraints not deﬁned at the point p0 will be computed at the point pM . And using gradient information at pM , an interpolated value will be determined at p0 . This interpolation ensures that constraints and criteria are continuous and diﬀerentiable functions, even at the boundary of their space of deﬁnition. This is a necessary condition for the optimization program to solve this modiﬁed problem. During the optimization process the constraints can be violated. But it tends to satisfy the constraints at the end of the optimization. Since we add in the problem the constraints specifying the sub-space where all constraints and criterion are deﬁned, at the end of the optimization the walking motion will be deﬁned and satisfy all the constraints. The only situation where the algorithm will not ﬁnd a solution that satisﬁes constraints is if there is no such solution (if we ask for a walk too fast and the actuators are not suﬃcient to do it, for example) or if the problem is not convex. Indeed the algorithm used is a local optimization algorithm. For a non convex problem, it will probably ﬁnd only a local non feasible solution, whereas other feasible solution exists. However, we have tried many random initial conditions for the optimization process and always found the same optimal solution that satisﬁed constraints. We can therefore assume that our problem is convex. To solve the intermediate optimization problem (31) and the general optimization problem (23), the gradient in function of the vector of parameters p of the criterion and constraints is necessary. To obtain an eﬃcient algorithm, these gradients were calculated analytically.

Dynamical Synthesis of a Walking Cyclic Gait Level 0 Constraints of existence of solutions for the inverse geometrical model for the boundary configurations

Satisfied Level 1 Monotony of a for limit conditions of single support

Satisfied Level 2 Constraint on the singularity f(a)=0

Satisfied Level 3 Constraint of monotony of the velocity of a in single support

Satisfied Level 4 Constraints of existence of solutions for the inverse geometrical model in double support

Satisfied Level 5

Other constraints and criterion

Fig. 3. Sequence of constraints to satisfy before the step can be deﬁned

247

248

S. Miossec and Y. Aoustin

5 Simulation Results Figures 4–7 are devoted to a chosen motion velocity for a biped which equals 0.3 m/s. Figure 4 shows that the needed torques for this trajectory are inside the template, motor torque/velocity, given by the manufacturer. The normal components of the ground reactions as functions of time, during one step are presented in Fig. 5. The constraint of unilateral contact on the leg tip 2 is active because the ﬁxed limit 20 N is reached in the tip of leg 2 during the double-support phase. The double-support phase begins after time 0.93 s. Figure 6 shows as functions of time the evolutions of joint variables δ1 , δ2 , δ3 and δ4 in single-support phase and double-support phase. Let us remark that the discontinuities in the graphes mark the limit between the singlesupport phase and the double-support phase. These discontinuities are not due to an impact (only an impactless motion is considered). These discontinuities appear in the graphes of Figs. 5–7 because the role of both legs are exchanged at the beginning of the double-support phase. Figure 7 shows the behavior of the variable α, which is monotone as expected. The discontinuity at the end of the single-support phase (time 0.93 s) is due to the exchange of the role of both legs. 140 Knee 1 Haunch 1 Haunch 2 Knee 2

120

Couples [N.m]

100

80

60

40

20

0

0

2

4

6

8

10

12

14

Joint Velocity DΓT q˙ [rad/s]

Fig. 4. Velocity versus torque for knee i and haunch i, (i = 1, 2) are inside the template, motor torque/velocity, deﬁned by the limit values 140 N.m and 12 rad/s

Dynamical Synthesis of a Walking Cyclic Gait

249

500

450

Rjy , j = 1, 2 [N]

400

350

300

250

200

150

100

50

0

Leg 1 in SS Leg 1 in DS Leg 2 in DS 0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.2

1.4

Time [s]

Fig. 5. Normal components in the stance leg tips 20

0

Joint variables δi [degrees]

-20

-40

-60

-80

-100

-120

-140

-160

-180

0

0.2

0.4

0.6

0.8

1

Time [s]

Fig. 6. Evolution of joint variables δ1 , δ2 , δ3 et δ4

250

S. Miossec and Y. Aoustin 5

0

α [degrees]

−5

−10

−15

−20

−25

0

0.2

0.4

0.6

0.8

1

1.2

1.4

Time [s]

Fig. 7. Evolution of α in function of time 4

2.2

x 10

2 1.8

CΓ [N 2 .m.s]

1.6 1.4 1.2 1 0.8 0.6 0.4 0.2 0

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

Motion velocity [m/s] Fig. 8. CΓ in function of several motion velocities for the biped

1

Dynamical Synthesis of a Walking Cyclic Gait

251

In conclusion, for the velocity 0.3 m/s of the biped an optimal motion is feasible according to the constraints. Other velocities of walk for the biped have been tested with success. In Fig. 8 discrete values of criterion CΓ are presented versus the velocity of motion. The evolution of discrete criterion CΓ versus velocity of motion is more regular if the optimal walks are obtained without taking into account Coulomb friction. This is due to the fact that the convergence for the case with friction is not very good, since torques are not smooth. For superior velocities a running gait is more appropriate, (see for example numerical experiments in the paper [18]).

6 Conclusion An optimization process is proposed to design optimal bipedal gaits for a ﬁve-link biped. The walking gaits are composed of single-support phases and double-support phases, but with no impact. The criterion minimized is the integral of the square of the torques. A sequential procedure is done, taking into account the constraints according to their importance realizing a walk step. Coulomb frictions, which are nonlinear and discontinuous functions, are taken into account because their contribution cannot be neglected. A possible improvement would be to do a piecewise linear approximation of the Coulomb friction around the discontinuity point of the friction force for a null joint velocity. Currently the main drawback of the optimization method we used is that it is not exactly adapted to our problem. Our problem is a semiinﬁnite problem, that is an optimization problem with constraints that must be satisﬁed over an interval. We have then adapted our problem by considering the constraints over an interval only at their most constraining point. The optimization problem we then solve is with non-smooth constraints. But we obtained convergence even if NPSOL was not designed to cope with such non-smooth problems. To solve our problem, we would like to consider an optimization algorithm that can take into account a variable number of constraints in the future. Indeed, the number of maximum and minimum where we considered the semi-inﬁnite constraints can change during the optimization process. We hope also to experiment on prototype Rabbit these reference trajectories and to extend also this work to a walking biped with more degrees of freedom.

References [1] M. Vukobratovic and D. Juricic. Contribution to the synthesis of biped gait. In Proc. of the IFAC Symp. Technical and Biological on Control, Erevan, USSR, 1968. [2] V. V. Beletskii and P. S. Chudinov. Parametric optimization in the problem of bipedal locomotion. Izv. An SSSR. Mekhanika Tverdogo Tela [Mechanics of Solids], (1):25–35, 1977.

252

S. Miossec and Y. Aoustin

[3] A. M. Formal’sky. Locomotion of Anthropomorphic Mechanisms. Nauka, Moscow [In Russian], 1982. [4] I. E. Sutherland and M. H. Raibert. Machines that walk. Scientiﬁc American, 248:44–53, 1983. [5] K. Hirai, M. Hirose, and T. Haikawa. The development of honda humanoid robot. In Proc. of the IEEE Conf. on Robotics and Automation, pp. 1321– 1326, 1998. [6] K. Kaneko, F. Kanehiro, S. Kajita, H. Hirukawa, T. Kawasaki, M. Hirata, K. Akachi, and T. Isozumi. Humanoid robot hrp-2. In Proc. of the IEEE Conf. on Robotics and Automation, pp. 1083–1090, 2004. [7] A. A. Grishin, A. M. Formal’sky, A. V. Lensky, and S. V. Zhitomirsky. Dynamic walking of a vehicle with two telescopic legs controlled by two drives. Int. J. of Robotics Research, 13(2):137–147, 1994. [8] F. Zonfrilli, M. Oriolo, and T. Nardi. A biped locomotion strategy for the quadruped robot sony ers-210. In Proc. of the IEEE Conf. on Robotics and Automation, pp. 2768–2774, 2002. [9] J. Pratt, C. M. Chew, A. Torres, P. Dilworth, and G. Pratt. Virtual model control: an intuitive approach for bipedal locomotion. International Journal of Robotics Research, 20(2):129–143, 2001. [10] C. Chevallereau, G. Abba, Y. Aoustin, F. Plestan, E.R. Westervelt, C. Canudas de Wit, and J. W. Grizzle. Rabbit: a testbed for advanced control theory. IEEE Control Systems Magazine, 23(5):57–79, 2003. [11] S. Miossec and Y. Aoustin. A simpliﬁed stability study for a biped walk with under and over actuated phases. International Journal of Robotics Research, 24(7):537–551, 2005. [12] M. W. Spong and M. Vidyasagar. Robots Dynamics and Control. John Wiley, 1989. ´ [13] S. Miossec. Contribution a ` l’Etude de la Marche d’un bip`ede. PhD thesis, Universit´e Nantes France, 2004. [14] A. M. Formal’sky. Ballistic Locomotion of a Biped. Design and Control of Two Biped Machines. Human and Machine Locomotion, Ed. A. Morecki and K. Waldron. Springer-Verlag, 1997. [15] Y. Aoustin and A. M. Formal`sky. Control design for a biped: Reference trajectory based on driven angles as functions of the undriven angle. Journal of Computer and Systems Sciences International, 42(4):159–176, 2003. [16] P. H. Channon, S. H. Hopkins, and D. T. Pham. Derivation of optimal walking motions for a bipedal walking robot. Robotica, 10(3):165–172, 1992. [17] C. Chevallereau, A. Formal’sky, and D. Djoudi. Tracking of a joint path for the walking of an under actuated biped. Robotica, 22(1):15–28, 2003. [18] C. Chevallereau and Y. Aoustin. Optimal reference trajectories for walking and running of a biped. Robotica, 19(5):557–569, 2001. [19] L. Roussel, C. Canudas de Wit, and A. Goswami. Generation of energy optimal complete gait cycles for biped. In Proc. of the IEEE Conf. on Robotics and Automation, pp. 2036–2042, 2003. [20] P. E. Gill, W. Murray, M. A. Saunders, and M. H. Wright. User’s guide for NPSOL 5.0.

Performing Open-Loop Stable Flip-Flops – An Example for Stability Optimization and Robustness Analysis of Fast Periodic Motions K. Mombaur IWR, Universit¨ at Heidelberg, Im Neuenheimer Feld 368, 69120 Heidelberg, Germany [email protected] Summary. For fast motions in biomechanics and robotics, stability and robustness against perturbations are critical issues. The faster a motion the more important it is to exploit the system’s natural stability properties for control. The stability of a periodic motion can be measured in terms of the spectral radius of the monodromy matrix. We optimize this stability criterion for a given robot topology, using special purpose optimization methods and leaving the model parameters, actuator inputs, trajectory start values and cycle time free to be determined by the optimization. This approach allows us to create simulations of robots that can move stably without any feedback. In order to analyze the robustness of a resulting periodic motion, we propose two methods, the ﬁrst of which relies on forward simulations using perturbed start data and parameters while the second is based on the pseudospectra of the matrix. As a new example for a fast open-loop stable motion that has been produced by stability optimization, we present a biped gymnastics robot performing repetitive ﬂip-ﬂops (i.e. back handsprings). A similar model has previously been shown capable of performing open-loop stable running motions and repetitive somersaults.

1 The Role of Open-Loop Stability in Fast Motions For fast walking and running motions, stability is a crucial property. Maintaining static stability throughout the motion is obviously not possibly for fast walking and running robots with just one or two legs. In contrast, those robots need to be allowed to tip and fall and swing, but the resulting motions have to be overall dynamically stable. Dynamically stable motions are characterized by the fact that they persist even in the presence of perturbations which always exist in a real world environment. The more complex the system becomes and the faster a motion gets, the more diﬃcult is the task of stabilizing the motion by means of appropriate feedback control systems. A comparison of the speed of running motions in biology and robotics still shows a huge gap: despite all technological process in recent years, the world’s

254

K. Mombaur

Fig. 1. A ﬂip-ﬂop (back handspring) motion in gymnastics (taken from [9], with permission)

fastest robots are still far behind the world’s fastest track runners which run the 100 meters at average speed of nearly 37 km/h. One explanation for this is that biological feedback and signal processing of a running person based on senses and neurons is still far more sophisticated than any technical feedback system implemented on a robot. However, it seems even more important, that fast biological motions exploit the natural stability properties of the system much better than robots which typically rely on the traditional approach of trajectory playback. In biology, fast motions seem to be to some extent inherently stable, or self-stabilizing, which signiﬁcantly reduces the online feedback eﬀort. In robotics, the idea of exploiting natural stability properties has been introduced mainly through the ﬁeld of passive-dynamic walking (compare e.g. the work of McGeer [12, 13] or Ruina and coworkers [8]). Passive-dynamic robots walk down inclined slopes in an amazingly natural looking manner, without any motors, powered only by gravity, and are fully open-loop stable without any feedback. There are diﬀerent approaches to transfer the ideas of passive-dynamics to actuated walking and running motions (e.g. Collins et al. [6]). There only exist very few entirely open-loop stable actuated robots today, e.g. the onelegged hopping robots of Ringrose [21] and Wei et al. [24], and they rely mainly on simple stabilization measures like a large curved foot in combination with a low center of mass. There are also more complex robots that combine an exploitation of self-stabilizing eﬀects and feedback control, see e.g. the works of Buehler [4] or Pratt [20]. Very fast open-loop stable multi-legged robots have been built inspired by cockroaches (Cham et al. [5]). The idea of open-loop stability or exploitation of natural stability has been addressed by diﬀerent authors of this symposium, e.g.: • Martijn Wisse presented his robot Denise which is a quasi-passive walking robot with a very natural gait based on very little actuation and feedback • Heiko Wagner and Peter Giesl explore the open-loop stability present in biological systems especially in the muscle actuators • John Schmitt describes the SLIP model which can be passively stable and for which the basin of stability can be enlarged by very simple feedback.

Stability Optimization and Robustness Analysis for a Flip-Flop Robot

255

We approach the issue of producing fast open-loop stable motions from an optimal control perspective. The objective of this paper is twofold: • to summarize a systematic procedure based on numerical optimization that we developed [14, 15], to produce robot conﬁguration and fast periodic motions that are fully open-loop stable, and to analyze their robustness. These numerical methods are meant to be used already in the design phase of the robots because they help to determine adequate geometric and inertial properties; • to present one new speciﬁc example optimized with these methods: a twolegged robot performing repetitive ﬂip-ﬂops i.e. back handsprings. This paper presents an extension of the work done in cooperation with Bock, Schl¨ oder and Longman [17, 16]. Altogether this research shows that very diﬀerent types of open-loop stable motions can be produced using these numerical techniques. Since robustness is another crucial characteristic of a motion – besides stability – we also present diﬀerent methods that allow one to numerically asses the robustness of a solution, i.e. to quantitatively determine the size of the tolerated perturbations. In detail, the following steps are necessary in order to produce such a purely open-loop stable robot conﬁguration and motion: • Step 1: Choose the basic robot conﬁguration and motion • Step 2: Establish a mathematical model of the robot motion • Step 3: Determine open-loop stable solutions by means of optimization of all model parameters and free input variables • Step 4: Analyze robustness of solution. The individual steps will be described in the following sections of this paper for the speciﬁc example of the biped robot performing repetitive ﬂip-ﬂops. A special focus is on the stability optimization in step 3, where the underlying numerical methods and the results of these methods for the robot under investigation will be presented.

2 First Step: Choosing the Robot Conﬁguration and Motion – The Flip-Flop Example In a ﬁrst step the basic characteristics of the robot conﬁguration and the motion to be stabilized need to be picked. In detail, that means that one has to select 1. the robot topology: i.e. determine the number of bodies, their basic shapes, the types of connections (or joints) between those bodies, the types of passive elements (like springs or dampers), the number and types of actuators etc.

256

K. Mombaur

2. the motion of the robot: i.e. describe the gait to be performed (like walking, running, trot, gallop etc.) or any other type of periodic motion (like jumping with ﬂip etc.); this also includes a determination of the logical order of phases and of the natural phase-separating conditions etc. 3. the free quantities in the model: i.e. choose those model parameters, actuator inputs and characteristics of the trajectory that qualify for a modiﬁcation by the optimization procedures. In the present paper, we investigate a periodic ﬂip-ﬂop – or back handspring motion of a planer robot trying to mimic sequences of the human gymnastics movement (Fig. 1). Going along the items listed above, the following selections have been made: 1. The robot consists of a large trunk (in a bird-like horizontal orientation), two thin telescopic legs with point feet and two arms which are assumed to be identical to the legs in shape and size (compare Fig. 2). Since both arms and both legs act exactly in parallel for the planar motion considered, this original model can be substituted by a model with just one leg (pointing down from the trunk) and just one arm (pointing up, see Fig. 4). This model is in fact almost identical to the biped robot model that we have investigated in earlier publications and that we have proven capable of open-loop stable periodic running motions [17] and open-loop stable periodic somersaults with alternating single leg contacts [16] (also compare Fig. 3). The only diﬀerence is that one leg now has to be pointed upwards in order to be interpreted as an arm. Each leg/arm is connected to the trunk by a hinge driven by a torque and a parallel torsional spring-damper element. The lower part of the leg/arm is assumed to be massless, and the two parts of each leg/arm are connected by an actuated spring-damper element (series elastic actuator).

Fig. 2. A robot model performing ﬂip-ﬂop motions

Stability Optimization and Robustness Analysis for a Flip-Flop Robot

257

Fig. 3. Two-legged robot capable of periodic running and somersault motions

Fig. 4. Robot model used for ﬂip-ﬂop computations with one arm and one leg

2. As shown in Fig. 1, for the human ﬂip-ﬂop motion the following phases can be identiﬁed: foot contact phase – ﬂight phase – hand contact phase – ﬂight phase. Since our robot model is characterized by several symmetries not present in the human gymnast (legs and arms are identical, as well as the associated actuators, and the trunk is symmetric with respect to its horizontal plane), we can assume that foot and arm contacts have identical eﬀects and reduce the model to half a physical cycle, i.e. just consider two phases: a contact phase followed by a ﬂight phase. The natural condition

258

K. Mombaur

ending ﬂight and starting the contact phase is satisﬁed if the respective foot or hand reaches zero height with a negative vertical velocity. The contact phase ends if the spring in the telescoping leg/arm regains its normal length after compression. 3. In order to optimize the open-loop stability of the ﬂip-ﬂip robot motion the following quantities are left free to be modiﬁed by the optimization procedures: • Most of the model parameters i.e. of the design variables that can be a priori chosen for a robot at design time and are ﬁxed throughout the motion. Model parameters for this robot model are trunk mass and inertia mb and Θb , leg/arm mass and inertia ml and Θl , distance between centers of mass of trunk and leg d, leg rest length l0 , torsional spring and damper constants ktors and btors , rest location of torsional spring ∆φ, and translational spring and damper constants k and b. For the computations presented here mb and l are ﬁxed for scaling reasons, the rest is left free. Note that all quantities correspond to the substitute model (Fig. 4). • All time-varying system inputs, i.e. the time histories of the forces and torques produced by motors and other types of actuators and acting on the model are also left free to be determined by optimization. Out of the four actuators described above – two torques utors between the hip and the leg and arm, respectively, and two series elastic actuators uSEA in the telescopic part of the leg/arm – only three are active in the half cycle considered. • There are a few remaining free quantities describing the trajectory of the robot, like the initial values (at the beginning of a cycle) of all positions and velocities and the period time. The rest of the trajectory is implicitly determined by the dynamics and free quantities described above, like the parameters and the input histories. Even though all these quantities are free in optimization, they are subject to physically reasonable bounds.

3 Second Step: Establishing a Mathematical Model of the Robot Motion After the general selections described in the previous section have been made, a detailed mathematical model of the robot and the motion has to be set up. From a mathematical perspective, gait models result in hybrid systems which include continuous motion phases with highly complex nonlinear diﬀerential equations and discrete “phases” (of duration zero) with sudden – discontinuous – changes in the state variables of the system1 . The number of degrees of freedom, the number of state variables, as well as the number of free control 1

State variables x for a mechanical system are all position and velocity variables used for description

Stability Optimization and Robustness Analysis for a Flip-Flop Robot

259

variables u(t) (i.e. actuator inputs in physical terms) can be diﬀerent in each phase. Typically, the points of phase change do not take place at given times but only depend implicitly on time via the states of the system. In order to fully describe the motion of the ﬂip-ﬂop robot, we have to establish the following sets of equations: • diﬀerential equations for the leg contact phase • diﬀerential equations for the ﬂight phase • discrete equations for discontinuity “phase” after touchdown. We use the same set of state variables for the description of all phases, namely ˙ where xb and yb q = (xb , yb , φb , φl , φa )T , and the corresponding velocities q, are the position coordinates of the trunk center of mass in the vertical plane, and φb , φl and φa are the orientations of trunk, leg and arm. During the ﬂight phase, these position variables correspond to the ﬁve degrees of freedom; and the reduction during contact phase is handled by additional constraints. The coordinates of the centers of mass of the leg and arm (xl , yl ) and (xa , ya ) can be eliminated using the distance parameter d and the respective angle by φl or φa . The motion in the ﬂight phase is described by the following set of ordinary diﬀerential equations:    x ¨b m 0 0 ml d cos φl ml d cos φa    y¨  0 m 0 ml d sin φl ml d sin φa   b         φ¨b  = 0 0 θ 0 0 b      ¨  2 0  ml d cos φl ml d sin φl 0 θl + ml d   φl  2 ml d cos φa ml d sin φa 0 0 θ l + ml d φ¨a   ml d(sin φl φ˙ 2l + sin φa φ˙ 2a )     −ml d(cos φl φ˙ 2l + cos φa φ˙ 2a ) − mg   -2   ˙ b − φ˙ l ) (1)   u − k (φ − φ − ∆φ) − b ( φ tors,i tors b l tors i i i=1    −u  ˙ ˙ tors,1 − ml gd sin φl + ktors (φb − φl − ∆φ) + btors (φb − φl )   −utors,2 − ml gd sin φa + ktors (φb − φa − ∆φ) + btors (φ˙ b − φ˙ a ) where m is the total mass m = mb + 2ml and utors,1 and utors,2 are the torques between trunk and leg or arm, respectively. For compactness of notation, we use φl1 = φl and φl2 = φa in the third line of eqn. (1). The leg length l is ﬁxed to l0 during the major part of the ﬂight phase (since the foot is massless) and depends on the other coordinates during the contact phase (with the leg) as follows: yb l= ⇒ (2) cos φl sin φl ˙ y˙ b l˙i = + yb (3) φl . cos φl cos2 φl (equivalently for arm contact).

260

K. Mombaur

The series elastic actuator (compare Pratt et al. [19]) active in the contact leg uSEA,l ≥ 0 actively changes the spring’s length ∆l =

yb − r + r − uSEA,l − l0 cos φl

(4)

uSEA,l is equal to zero at touchdown and must be positive at liftoﬀ to compensate for the energy loss in the damper. In order to model foot contact, we use a constraint-based approach instead of contact forces: friction is assumed to be large enough to guarantee instantaneous stopping of the contact point without sliding. This would reduce the degrees of freedom of the system by two, but at the same time the previously ﬁxed leg length becomes variable and changes under the inﬂuence of the SEA spring-damper forces. In sum, this leads to reduction of degrees of freedom by one which is described by the additional kinematic constraint in velocity space (5) x˙ b + (yb + yb tan2 φl ) φ˙ l + tan φl y˙ b = 0 . The corresponding equation on the acceleration level is used to establish the diﬀerential-algebraic equations of index 1 describing the contact phase:    x ¨b m 0 0 ml d cos φl ml d cos φa 1     y  0 m 0 ml d sin φl ml d sin φa tanφl   ¨b   ¨     φb   0 0 θb 0 0 0     ¨= m d cos φ m d sin φ 0 2 2 θ + m d 0 y (1 + tan φ ) l l l l l b l   φl   l      ml d cos φa ml d sin φa 0 0 θ l + ml d 2 0 φ¨a  1 tan φl 0 yb (1 + tan2 φl ) 0 0 λ   2 2 ml d(sin φl φ˙ l + sin φa φ˙ a ) + (Fk + Fd ) sin φl   −ml d(cos φl φ˙ 2l + cos φa φ˙ 2a ) − mg − (Fk + Fd ) cos φl     2   ˙ ˙   i=1 utors,i − ktors (φb − φli − ∆φ) − btors (φb − φli )   (6)  −utors,1 − ml gd sin φl + ktors (φb − φl − ∆φ) + btors (φ˙ b − φ˙ l )      −utors,2 − ml gd sin φa + ktors (φb − φa − ∆φ) + btors (φ˙ b − φ˙ a ) −2 · cos−2 φl φ˙ l (y˙ b + yb tan φl φ˙ l ) with spring and damper forces Fk and Fd yb − l0 − uSEA,1 ) cos φl tan φl ˙ y˙ b Fd = b ( + yb φl ) . cos φl cos φl

Fk = k (

(7) (8)

The solution of these equations must lie on the invariant described by the velocity equation (5); this is guaranteed by including this equation in the computations of the discrete phase (see below).

Stability Optimization and Robustness Analysis for a Flip-Flop Robot

261

Before addressing this discrete phase, we brieﬂy state all phase switching conditions: • Phase change from contact phase to ﬂight pase, i.e. liftoﬀ, occurs when the spring length in the leg is equal to the (modiﬁed) rest length slif tof f = l0 + uSEA,1 −

yb =0 cos φl

(9)

and, at the same time, the trunk has a positive vertical speed clif tof f = y˙ b > 0 .

(10)

• Touchdown, i.e. phase change from ﬂight phase to the following contact phase (arm contact) occurs when the height of the prospective contact point is equal to zero stouchdown = yb − l0 cos φa = 0 .

(11)

The vertical speed of the contact point at touchdown must be negative: ctouchdown = y˙ b + l0 sin φa φ˙ a < 0 .

(12)

Discrete phases are used to describe sudden changes of the state variables due to collisions etc.; in the model the time of such an event is assumed to be zero. The lift-oﬀ phase transition is assumed to be smooth, so no discrete phase is inserted at this point. However, touchdown is generally non-smooth. The velocity of the contact point is instantaneously set to zero, and the discontinuity propagates to all other parts of the system. A set of ﬁve algebraic equations is used to determine the ﬁve unknown velocities after impact with the arm: • non-sliding ground contact combined with spring-damper action: x˙ contact = x˙ b + l0 cos φa φ˙ a + y˙ b tan φa + yb φ˙ a tan2 φa = 0

(13)

• conservation of angular momentum of trunk about hip: Htrunk,hip = Θb φ˙ b = const .

(14)

• conservation of angular momentum of the leg (remaining in free swing phase) about the hip Hswingleg,hip = (Θl + ml d2 )φ˙ l = const . • conservation of angular momentum of full robot about prospective contact point Hrobot,contact = Θb φ˙ b − mb (yb − yc )x˙ b + mb (xb − xc )y˙ b +Θl φ˙ l − ml (yl − yc )x˙ l + ml (xl − xc )y˙ l +Θl φ˙ a − ml (ya − yc )x˙ a + ml (xa − xc )y˙ a = const . (15)

262

K. Mombaur

with xc = xb + l0 sin φa yc = yb − l0 cos φa .

(16) (17)

• conservation of translational momentum in direction of prospective stance arm (considering spring-damper-force) m(x˙ b sin φa − y˙ b cos φa ) − Fkd = const .

(18)

The model presented so far is identical (except for notational details) for all robots of the same family, the running robot, the somersault robot and the ﬂip-ﬂop robot. However, using additional inequality constraints, it is possible to distinguish one type of motion from the other and impose the desired type. For ﬂip-ﬂops these constraints are e.g. • at foot contact, the foot is pointing down and forward while the arm is pointing up and backwards • leg and arm are always on diﬀerent sides of the trunk which must be assured by appropriate collision avoidance constraints • leg and arm have a negative rate during the whole cycle (i.e. are continuously rotating backwards) Also the periodicity constraints (i.e. coupled equality constraints) applied to the model are diﬀerent for each type of motion. In order to obtain the desired periodic ﬂip-ﬂop with symmetric leg and arm contact half-cycles, the following modiﬁed periodicity constraints have to be imposed: yb (0) = yb (T ) φb (0) = φb (T ) − π φl (0) = φa (T ) − 2π φa (0) x˙ b (0) y˙ b (0) φ˙ b (0) φ˙ l (0)

= φl (T ) = x˙ b (T ) = y˙ b (T ) = φ˙ b (T ) = φ˙ a (T )

(19)

φ˙ a (0) = φ˙ l (T ) Additionally, the following inequality constraints need to be satisﬁed for any type of motion: • clearance of swing leg and arm (height of lowest point larger than zero) • contact avoidance of all bodies.

Stability Optimization and Robustness Analysis for a Flip-Flop Robot

263

4 Numerical Stability Optimization Methods for Step 3 We address the task of step 3 to determine open-loop stable solutions by means of numerical optimization. A problem like this can not be solved by standard optimization or optimal control methods due to • the complexity of the robot gait models described above, namely the nonsmoothness in the dynamics (i.e. the hybrid system property) • the diﬃculties produced by the choice of the optimization criterion stability (which is another source of non-smoothness of an even more complicated type, see below). In our previous research we therefore have developed numerical methods for the optimization of open-loop stability of general hybrid periodic systems. The method described here depends on a two-level approach splitting the problem into periodic gait generation and stabilization of the periodic system (for details see [14, 15]). This approach has already been applied to many types of problems and robot examples and has always delivered stable solutions [14, 17, 16]. Figure 5 gives an overview of the two-level procedure. The tasks to be performed in each level will be presented in the next two paragraphs. 4.1 Solution of Periodic Optimal Control Problem The task of the lower level – or inner optimization loop is to ﬁnd actuator patterns, initial values and cycle time leading to a periodic trajectory while the set of parameters is ﬁxed by the outer loop for each inner loop computation. The choice of those variables is governed by energy consumption considerations (in terms of actuator inputs u). We also have imposed a lower bound on Outer optimization loop

Inner optimization loop Stability optimization min φstab modify model parameters (mass, inertia, geometry ...)

Solution of periodic optimal control problem minimize energy for given parameters modify initial values, actuator inputs, cycle time

Fig. 5. Sketch of two-level stability optimization procedure

264

K. Mombaur

the trunk forward speed at all points, and bounds on the leg inclination angle at touchdown and liftoﬀ instants. Together with the equations of motion, the periodicity constraints and phase switching conditions, box constraints on all variables etc., this leads to a multi-phase optimal control problem of the following form: T ||u||22 dt

min

x,u,T

(20)

0

s. t.

x(t) ˙ = fj (t, x(t), u(t), p) x(τj+ ) = h(x(τj− ))

or DAE

gj (t, x(t), u(t), p) ≥ 0 for t ∈ [τj−1 , τj ] , j = 1, . . . , nph , τ0 = 0, τnph = T req (x(0), .., x(T ), p) = 0 rineq (x(0), .., x(T ), p) ≥ 0 .

(21) (22) (23)

(24) (25)

We solve this problem using a variant of the optimal control code MUSCOD (Bock & Plitt [3], Leineweber [10]) suited for periodic gait problems. It is based on • a direct method for the solution of the optimal control problem (also termed a “ﬁrst discretize then optimize method”: instead of using arbitrary (i.e. inﬁnite dimensional) control functions u(t), we restrict the controls to a discretized space described by a ﬁnite set of parameters. For numerical eﬃciency, we use functions with local support, in this case piecewise constant functions on a grid with m intervals. • a multiple shooting state parameterization: the basic idea of this technique is to split the long integration interval [0, T ] into many smaller ones and to introduce the values of the state variables x at all those grid points as new variables sij (compare Fig. 6). The original boundary value problem is thus transformed into a set of initial value problems with corresponding continuity conditions between the integration intervals. For numerical reasons the multiple shooting grid is chosen identical to the control grid described above. The multiple shooting approach is very favorable for a number of reasons. The phase order and switching structure, which is generally known for walking problems, can be easily prescribed in this context. The rough knowledge that one usually has about the trajectory in the case of walking and running can be exploited in the generation of starting data for the multiple shooting points. Since the integration intervals are much shorter than in the original problem, the chances for ﬁnding a solution of the initial value problem obtaining suﬃciently accurate derivatives increase signiﬁcantly, even if all values are still far from the ﬁnal solution.

Stability Optimization and Robustness Analysis for a Flip-Flop Robot

265

x 1

....

2

....

i

x (t i-1 ,s i-1 ,q

i-1

m

,p)

..... s0

s1

s2

si-1

si

sm-1

sm t

t0

t1

t2

t i-1

ti

t m-1

tm

Fig. 6. Multiple Shooting state parameterization

Using these discretization techniques, the original inﬁnite dimensional optimal control problem is transformed into a ﬁnite dimensional nonlinear programming problem which is of large dimension but very structured and can therefore be solved eﬃciently by a tailored SQP algorithm exploiting the structure of the problem (also compare Leineweber et al. [10] [11]). The treatment of the dynamical model equations is not part of the discretized optimal control problem; this task must however be handled in parallel in order to provide the required information for the evaluation of objective functions, continuity constraints and the derivatives thereof. For this task, fast and reliable integrators are used that also include a computation of sensitivities based on the techniques of internal numerical diﬀerentiation (for details see Bock [2]). 4.2 Stability Optimization of Periodic Solution In the outer loop of the optimization procedure the open-loop stability of the periodic optimal problem solution is optimized by adjusting the model parameters. Stability is deﬁned in terms of the spectral radius of the Jacobian C of the Poincar´e map – also termed the monodromy matrix – associated with the periodic solution (see e.g. [22]). If the spectral radius is smaller than one, the solution is asymptotically stable, and if it is larger than one, the solution is unstable. We have proven that this criterion based on linear theory and typically applied to simple smooth systems can also be used to demonstrate the stability of solutions of a nonlinear multiphase system with discontinuities (Mombaur et al. [15]). The computation of the monodromy matrix corresponding to a periodic solution of the hybrid system can be performed eﬃciently by reusing sensitivity information computed during the optimal control problem solution. The following steps are required:

266

K. Mombaur

a) For continuous model phases, a simple chain rule multiplication of the sensitivity matrices on the individual multiple shooting intervals must be performed: Cq,v (t0 , tm ) = Cq,v (tm−1 , tm ) · ... · Cq,v (t1 , t2 ) · Cq,v (t0 , t1 ) .

(26)

b) At the points of discontinuity ts – like the touch-down or lift-oﬀ of the robot – an “update” of the sensitivity information has to be performed. This update term has to take into account that these discontinuity points are not ﬁxed in time, but implicitly depend on the states and parameters and will therefore experience a time shift under the inﬂuence of a perturbation of these variables. It is computed according to the following formula (compare Bock [2], von Schwerin et al. [23]): 1 Uq,v = (∆f − Jt − Jq,v flef t (ts )) · (sq , sv )T + I + Jq,v . s˙

(27)

f , q, and v are deﬁned as in Sect. 2, and s is the relevant switching function with partial derivatives sq and sv and total derivative with respect to time s. ˙ Subscripts lef t and right always denote quantities before and after the switching point, respectively. I is the identity matrix. J is the state variable jump function with the partial derivatives Jt and Jq,v . ∆f is the right hand side change. The full monodromy matrix including the update becomes Cx := Cq,v (t0 , tm ) = Cq,v (ts , tm )Uq,v Cq,v (t0 , ts )

(28)

c) If the models include non-periodic variables, a subsequent reduction to the periodic subspace is necessary to compute the matrix to be used in optimization. The eigenvalues of this matrix which is quadratic and nonsymmetric can be computed using a standard QR algorithm as available in Lapack (Anderson et al. [1]); these eigenvalues may be real or complex. We use the spectral radius as objective function of our optimization min |λ(C(p))|max , p

(29)

with the intention to decrease it below one. This is a diﬃcult optimization criterion for diﬀerent reasons: • The maximum eigenvalue function of the non-symmetric matrix C is nondiﬀerentiable and possibly even non-Lipschitz at points where multiple eigenvalues coalesce. • The determination of the matrix C involves the computation of ﬁrst order sensitivities of the discontinuous trajectories (see above). • The function is non-convex and typically has several local minima.

Stability Optimization and Robustness Analysis for a Flip-Flop Robot

267

Any gradient-based optimization method would thus require second order derivatives of the trajectory which are extremely hard to compute, especially due to the discontinuities in the dynamics. For all reasons mentioned, a direct search method has proven to be a very good choice for the solution of this outer loop optimization problem. Direct search methods are optimization methods that solely use function information and neither compute nor explicitly approximate derivatives. We have implemented a modiﬁcation of the Nelder-Mead algorithm [18] which is based on a polytope with n + 1 vertices for optimization in n-dimensional space. According to the function information collected at its vertices the polytope expands in directions promising descent and contracts in bad directions. In contrast to the original method, we allow for multiple expansions in a promising direction, we use a diﬀerent direction of contraction, and we only apply full polytope shrinking after multiple one-dimensional contractions. In addition, we consider the diﬀerent nature of optimization variables by appropriate scaling of the initial polytope, we use a modiﬁed termination criterion, and we rely on a restart procedure as globalization strategy. In contrast to the original Nelder-Mead method, our algorithm can directly handle box constraints on the optimization variables not requiring a penalty function. Although there is no theoretical convergence proof for this direct search method in the case of non-smooth systems, it has delivered excellent results with spectral radii below one in all the computational examples that we have applied it to so far.

5 Results of Step 3: Open-Loop Stable Repetitive Flip-Flops Applying these stability optimization methods to the model of the periodic ﬂip-ﬂop established in step 2, leads to the open-loop stable motion visualized in Fig. 7. It is characterized by a maximum eigenvalue of magnitude 0.807. The robot is characterized by the following model parameters: mb = 2.0, Θb = 1.0, ml = 0.848, Θl = 0.0174, d = 0.24, l0 = 0.5, ktors = 10.07, ∆φ = 0.969, btors = 7.41, k = 701.1, and b = 23.16 (all in ISO units). Note that all parameter values are given for the substitute model; going back to the original model with two legs and two arms requires cutting in half ml , Θl , and all spring and damper constants. Figures 8 and 9 show the state variable histories and actuator inputs for this open-loop stable solution. The initial values of the trajectory are xb (0) = 0.0, yb (0) = 0.439, φb (0) = 0.2, φl (0) = 0.5, φa (0) = 4.273, x˙ b (0) = 5.745, y˙ b (0) = −1.549, φ˙ b (0) = −9.659, φ˙ l (0) = −8.598, φ˙ a (0) = −8.804. The cycle time is T = 0.3 s with phase times Tf light = 0.15 s and Tcontact = 0.15 s (implicating that both phase times reach their lower limits as speciﬁed). The direction associated with the non-periodic variable xb is eliminated from the monodromy matrix, such that the matrix considered for stability computations has dimension nine. The periodic mapping described by this

268

K. Mombaur

Fig. 7. Open-loop stable ﬂip-ﬂops produced by optimization

matrix also includes the shift of arm and leg and of trunk orientation expressed in the modiﬁed periodicity constraints (19). The computations resulted in the nine eigenvalues, by magnitude: |λ1 | = 0.807 |λ2 | = 0.389 |λ3 | = 0.225 |λ4 | = 0.797 |λ5 | = 0.644 |λ6 | = 0.665 |λ7 | = 2 · 10−6 |λ8 | = 0 |λ9 | = 0

Stability Optimization and Robustness Analysis for a Flip-Flop Robot

269

Fig. 8. State variable trajectories of open-loop stable ﬂip-ﬂop solution

The two eigenvalues of zero come from the fact that the degrees of freedom of the robot are reduced from ﬁve to four during the contact phase (i.e. from ten to eight independent directions in state space). This also implies that perturbations associated with this lost degree of freedom are naturally damped

270

K. Mombaur

Fig. 9. Actuator inputs producing ﬂip-ﬂop motion in Fig. 8

out which is represented by a rank reduction by two of the monodromy matrix, and thus by two zero eigenvalues. Another eigenvalue happens to be very small for this particular solution, but it is not analytically or numerically zero in the general case.

6 Fourth Step: Robustness Analysis of Solution A maximum eigenvalue with a magnitude below one guarantees stability of the solution against small perturbations in the initial value. But how small is small? The absolute size of the maximum eigenvalue only gives some information about the speed of decay over time of the perturbations applied, not about the absolute size of the perturbations that are possible before the robot falls down. An analysis of the stability margins of a solution, i.e. of its robustness, is also very important in order to asses the quality of a solution. Stability of a solution is a prerequisite for robustness, but stability does not necessarily imply robustness. There is no straightforward way to numerically compute the robustness of a solution including all nonlinear eﬀects in terms of a single function, but we

Stability Optimization and Robustness Analysis for a Flip-Flop Robot

271

present two quick numerical tests that can help to evaluate the robustness of a given solution. 6.1 Test 1: Simulation of Perturbed Solutions We can determine stability margins numerically by applying one-dimensional perturbations to the initial values of the trajectory and simulating the resulting behavior of the system checking if it stumbles or if it returns to the periodic motion. This test includes all nonlinear eﬀects associated with the system and the motion including: • nonlinearities of the model equations • closeness to constraints, especially to phase-switching constraints (perturbations may cause new phase switching structures to appear) As an example, we investigate here the eﬀect of perturbations of the initial vertical velocity y(0). ˙ The boundary of the basin of attraction of the stable solution is above +30% of the reference value. Figure 10 shows the recovery process after a perturbations of +30%. 6.2 Test 2: Pseudospectra of Monodromy Matrix Another completely diﬀerent way of assessing the robustness of a solution is to look at the pseudospectra of the computed monodromy matrix. The pseudospectra can be deﬁned in terms of the spectra of all nearby, i.e. perturbed matrices: Λ (C) = {z ∈ C : z ∈ Λ(C + E)for some E with||E|| < }

(30)

where Λ denotes the spectrum of a matrix. For more information on pseudospectra, including other, equivalent deﬁnitions and useful tools, see the Pseudospectra Gaitway by Embree and Trefethen [7]. Pseudospectra capture the fact that nonsymmetric matrices, i.e. matrices without an orthogonal basis of eigenvectors may exhibit transients or other types of eﬀects which are diﬀerent from the asymptotic behavior predicted by the eigenvalues. The pseudospectrum is used to deﬁne measures of robust stability and can help to qualitatively compare diﬀerent solutions. However, one has to keep in mind that this analysis only includes information about the sensitivity of the spectral radius with respect to matrix entries and not about the sensitivities of the matrix entries with respect to the free optimization variables. The pseudospectrum associated with the most stable solution of the ﬂipﬂop robot is given in Fig. 11.

272

K. Mombaur 16

6.5

14 6

12 x_dot

x_b

10 8

5.5 5

6 4

4.5

2 0

4 0

1

2

3

0

1

0.5 0.48 0.46 0.44 0.42 0.4 0.38 0.36 0.34 0.32 0.3

3

2

3

2

3

2

3

1 0.5 0 -0.5 -1 -1.5 -2 -2.5 1

2

3

0

1

t

t -8

0

-9

-0.5

-10 phi_b_dot

phi_b

2

2

0

-1 -1.5

-11 -12

-2

-13

-2.5

-14

-3

-15 0

1

2

3

0

1

t

t

1 0.5 phi_l_dot

0 phi_l

3

1.5

0.5

-0.5 -1 -1.5 -2 -2.5 0

1

2

-3 -4 -5 -6 -7 -8 -9 -10 -11 -12 -13 -14

3

0

1

t

t

4.5 4 3.5 phi_a_dot

3 phi_a

2 t

y_dot

y_b

t

2.5 2 1.5 1 0.5 0 0

1

2 t

3

-8 -9 -10 -11 -12 -13 -14 -15 -16 -17 -18 0

1 t

Fig. 10. Eﬀect of a perturbation (of y(0) ˙ by +30%) on most stable trajectory of ﬂip-ﬂop robot (stable base solution is shown by solid line, perturbed solution by dashed line)

Stability Optimization and Robustness Analysis for a Flip-Flop Robot

273

Fig. 11. Pseudospectra of monodromy matrix corresponding to most stable ﬂip-ﬂop solution computed with Eigtool [7]

7 Conclusions We have presented an animated biped robot that is capable of performing repetitive open-loop stable ﬂip-ﬂop motions. This adds a new type of motion to the range of periodic motions that are possible without any feedback. Producing these simulations was only possible by means of special purpose numerical stability optimization methods that also have been brieﬂy described in this paper. In addition, two numerical tests for the robustness of the solution have been presented.

Acknowledgments I would like to thank the anonymous reviewer of [16] who suggested to investigate the open-loop stability of ﬂip-ﬂops as a next example. Financial support by the Margarete von Wrangell program of the State of Baden-W¨ urttemberg is gratefully acknowledged.

References [1] E. Anderson, Z. Bai, C. Bischof, J. Demmel, J. Dongarra, J. Du Croz, A. Greenbaum, S. Hammarling, A. McKenney, S. Ostruchov, and D. Sorensen. Lapack Users’ Guide. SIAM, 1995.

274

K. Mombaur

[2] H. G. Bock. Randwertproblemmethoden zur Parameteridentiﬁzierung in Systemen nichtlinearer Diﬀerentialgleichungen. In Bonner Mathematische Schriften 183. Universit¨ at Bonn, 1987. [3] H. G. Bock and K.-J. Plitt. A multiple shooting algorithm for direct solution of optimal control problems. In Proceedings of the 9th IFAC World Congress, Budapest, pp. 242–247. International Federation of Automatic Control, 1984. [4] M. Buehler. Dynamic locomotion with one, four and six-legged robots. Journal of the Robotics Society of Japan, 20(3):15–20, April 2002. [5] J. G. Cham, S. A. Bailey, J. E. Clark, R. J. Full, and M. R. Cutkosky. Fast and robust: Hexapedal robots via shape deposition manufacturing. International Journal of Robotics Research, 21(10-11):869–882, 2002. [6] S. H. Collins, A. L. Ruina, R. Tedrake, and M. Wisse. Eﬃcient bipedal robots based on passive-dynamic walkers. Science, 307:1082–1085, 2005. [7] Mark Embree and Nick Trefethen. Pseudospectra gaitway. web.comlab.ox. ac.uk/ projects/pseudospectra/index.html, 2005. [8] M. Garcia, A. Ruina, M. Coleman, and A. Chatterjee. Some results in passivedynamic walking. In Biology and Technology of Walking, pp. 268–275. Euromech Colloquium 375, Munich, 1998. [9] Kurt Knirsch. Lehrbuch des Ger¨ at- und Kunstturnens, vol. 1. Knirsch-Verlag, 1997. [10] D. B. Leineweber, I. Bauer, H. G. Bock, and J. P. Schl¨ oder. An eﬃcient multiple shooting based reduced SQP strategy for large-scale dynamic process optimization – part I: theoretical aspects. Comput. Chem. Engng, 27:157–166, 2003. [11] D. B. Leineweber, A. Sch¨ afer, H. G. Bock, and J. P. Schl¨ oder. An eﬃcient multiple shooting based reduced SQP strategy for large-scale dynamic process optimization – part II: software aspects and applications. Comput. Chem. Engng, 27:167–174, 2003. [12] T. McGeer. Passive dynamic walking. International Journal of Robotics Research, 9:62–82, 1990. [13] T. McGeer. Passive dynamic biped catalogue. In R. Chatila and G. Hirzinger, editors, Proceedings of the 2nd International Symposium of Experimental Robotics, Toulouse. Springer-Verlag, New York, 1991. [14] K. D. Mombaur. Stability Optimization of Open-loop Controlled Walking Robots. PhD thesis, University of Heidelberg, 2001. www.ub.uni-heidelberg. de/archiv/1796. VDI-Fortschrittbericht, Reihe 8, No. 922, ISBN 3-18-392208-8. [15] K. D. Mombaur, H. G. Bock, J. P. Schl¨ oder, and R. W. Longman. Openloop stable solution of periodic optimal control problems in robotics. ZAMM – Journal of Applied Mathematics and Mechanics / Zeitschrift f¨ ur Angewandte Mathematik und Mechanik, 85(7):499–515, July 2005. [16] K. D. Mombaur, H. G. Bock, J. P. Schl¨ oder, and R. W. Longman. Selfstabilizing somersaults. IEEE Transactions on Robotics, 21(6), Dec. 2005. [17] K. D. Mombaur, R. W. Longman, H. G. Bock, and J. P. Schl¨ oder. Open-loop stable running. Robotica, 23(01):21–33, January 2005. [18] J. A. Nelder and R. Mead. A simplex method for function minimization. Computer Journal, 7:308–313, 1965. [19] G. A. Pratt and M. M. Williamson. Series Elastic Actuators. In Proceedings of IROS, pp. 399–406, Pittsburgh, 1995.

Stability Optimization and Robustness Analysis for a Flip-Flop Robot

275

[20] J. E. Pratt. Exploiting Inherent Robustness and Natural Dynamics in the Control of Bipedal Walking Robots. PhD thesis, Massachusetts Institute of Technology, 2000. [21] R. P. Ringrose. Self-stabilizing running. Technical report, Massachusetts Institute of Technology, 1997. [22] J. La Salle and S. Lefschetz. Stability by Liapunov’s Direct Method. Academic Press, 1961. [23] R. v. Schwerin, M. J. Winckler, and V. H. Schulz. Parameter estimation in discontinuous descriptor models. In D. Bestle and W. Schiehlen, editors, IUTAM Symposium on Optimization of Mechanical Systems, pp. 269–276. Kluwer Academic Publishers, 1996. [24] T. E. Wei, G. M. Nelson, R. D. Quinn, H. Verma, and S. L. Garverick. Design of a 5-cm monopod hopping robot. In Proceedings of IEEE International Conference on Robotics and Automation, pp. 2828–2823, 2000. http://www.ﬂuggart.cwru.edu/icra2000/icra2000.html.

Achieving Bipedal Running with RABBIT: Six Steps Toward Inﬁnity B. Morris1 , E.R. Westervelt2 , C. Chevallereau3 , G. Buche4 , and J.W. Grizzle1 1

2

3

4

Control Systems Laboratory, EECS Department, University of Michigan, Ann Arbor, Michigan 48109-2122, USA {MorrisBJ,Grizzle}@umich.edu Locomotion and Biomechanics Laboratory, Department of Mechanical Engineering, The Ohio State University, Columbus, Ohio 43210, USA [email protected] IRCCyN, Ecole Centrale de Nantes, UMR CNRS 6597, BP 92101, 1 rue de la No¨e, 44321 Nantes, cedex 03, France [email protected] LAG, Ecole Nationale d’Ing´enieurs Electriciens de Grenoble, BP 46, 38402 St Martin d’H`eres, France [email protected]

Summary. This paper develops a class of bipedal running controllers based on the hybrid zero dynamics (HZD) framework and discusses related experiments conducted in September 2004 in Grenoble, France. In these experiments, RABBIT, a ﬁve-link, four-actuator, planar bipedal robot, executed six consecutive running steps. The observed gait was remarkably human-like, having long stride lengths (approx. 50 cm or 36% of body length), ﬂight phases of signiﬁcant duration (approx. 100 ms or 25% of step duration), an upright posture, and an average forward rate of 0.6 m/s. A video is available at [7, 17]. In the time allotted for experiments, stability of the gait could not be validated. To put the results into context, background information on hybrid robot modeling, control philosophy, and gait optimization techniques accompany ﬁnal experimental observations. An additional discussion about some unmodeled dynamic and geometric eﬀects that contributed to implementation diﬃculties is given.

1 Introduction Designed and built as a platform to explore legged locomotion, RABBIT, a ﬁve-link, four-actuator, planar, bipedal robot [3, 2] (see Fig. 1(a)), has provided a means to test conceptually new approaches to underactuated, active, dynamic walking and running. Since March 2003, RABBIT has been used to experimentally verify a mathematical framework for the systematic design,

278

B. Morris et al. q5 ycm xcm

q2 q1

q4

θs (q)

y2

y1 x1

x2

q3 (i) (a) RABBIT

(ii)

(iii)

(b) Phases of running and coordinate conventions.

Fig. 1. RABBIT and the diﬀerent phases of running with coordinate conventions labeled. In (b), the robot is shown (i) at the end of the stance phase; (ii) during ﬂight; and (iii) at the beginning of the stance phase just after landing. To avoid clutter, the coordinate conventions have been spread out over the stance and ﬂight phases. Angles are positive in the counter clockwise direction. The stance leg is indicated in bold

analysis, and optimization of controllers that induce stable walking gaits in N -link planar bipedal robots with one degree of underactuation [8, 20]. The next challenge for RABBIT is to achieve stable running. In previous experiments, walking controllers for RABBIT acted by enforcing virtual constraints, which are holonomic constraints used to coordinate link movements throughout a gait. The stability of such walking motions were analyzed on the basis of the hybrid zero dynamics (HZD) of walking, with the conclusions of theory supported by experimental results [18]. Recent work in [5] extends the method of virtual constraints and the notion of an HZD to encompass the analysis of running in robots such as RABBIT. The developed control strategy is hybrid with both continuous and event-based actions and leads to the deliberate creation of an HZD of running and an accompanying stability test. This paper summarizes a ﬁrst attempt to use RABBIT to experimentally validate the theory of stable running presented in [5]. To facilitate implementation, the controller hypotheses of [5] are slightly relaxed, leading to controllers that are easier to design but which still lend the closed-loop system to a reduced-dimensionality stability test. Both discrete and continuous actions of the modiﬁed hybrid control law are discussed in detail. To put the results into context, background information on hybrid robot modeling, control philosophy, and gait optimization techniques accompany ﬁnal experimental observations. An additional discussion about some unmodeled dynamic and geometric eﬀects that contributed to implementation diﬃculties is given.

Achieving Bipedal Running with RABBIT: Six Steps Toward Inﬁnity

279

1.1 Related Work on Running Machines Nearly twenty years ago, Raibert developed a series of 2D and 3D running machines that are more commonly called hoppers [16]. These hoppers employed a three part controller regulating hopping height, foot touchdown angle, and body attitude correction. The successful use of such a controller is tied to the morphology of the hopper: a massive body with signiﬁcant inertia at the end of an actuated, compliant, prismatic leg. Recently, there have been a number of successful demonstrations of running in robots with a diﬀerent morphology: bipeds with revolute knees. A variety of control approaches were used. In late 2003, both Iguana Robotics and Sony announced (separate) experimental demonstrations of running in robots with revolute knees. Iguana Robotics’ controller was based on central pattern generators (CPGs) and Sony’s was based on the zero moment point (ZMP). In early 2004, running was announced for HRP-2LR [13] using a controller based on a technique of resolved momentum. In December 2004, Honda’s robot, ASIMO, achieved running at 3 km/h (0.8 m/s) with a 50 ms ﬂight phase using a controller based on posture control. A year later, ASIMO ran at a new top speed of 6 km/h. 1.2 Outline The remainder of the paper is a self-contained description of the theoretical development and hardware modiﬁcations leading up to one example of an experiment in which RABBIT took six consecutive running steps.5 Echoing [5, Sects. III and IV], Sect. 2 develops an open-loop model for RABBIT.6 Section 3 discusses a modiﬁcation of the control law given in [5] that is based on relaxed hypotheses. Philosophy and motivation of the modiﬁed control law are given in Sect. 3.1 with a detailed development of the hybrid controller in Sects. 3.2 to 3.6. The resulting closed-loop model of RABBIT and its stability properties are discussed in Sects. 3.7 and 3.8. Beyond this, Sect. 4 outlines a method for the design of stable gaits using constrained nonlinear optimization and includes a numerical example. Section 5 presents results from the ﬁrst experimental implementation of running on RABBIT and a discussion outlining a number of possible reasons why stable running was not observed. Conclusions are drawn in Sect. 6.

5

6

In a number of experiments, RABBIT achieved ﬁve or six running steps before tracking errors exceeded software bounds. One example is examined in detail. The content of Sect. 2 is based entirely on [5, Sects. III and IV] and is included for completeness.

280

B. Morris et al.

2 Modeling 2.1 Assumptions and Terminology RABBIT is modeled as a planar robot with ﬁve rigid massive links connected by (four) actuated, frictionless, revolute joints. The model is subdivided into two legs with identical physical properties, and a torso. The legs each consist of two links, a thigh and a shank, and are connected to each other and the single-link torso at the hips. Let qb := (q1 , q2 , q3 , q4 ) be the vector of actuated body coordinates, q5 be the unactuated coordinate of the robot’s absolute orientation, and xcm and ycm be the cartesian coordinates that give the horizontal and vertical positions of the robot’s center of mass. See Fig. 1(b) for a depiction of the robot’s morphology and coordinate convention. The robot is said to be in ﬂight phase when neither leg is in contact with the ground, and in stance phase when one leg is in stationary contact with the ground. During stance, the leg contacting the ground is called the stance leg and the other is called the swing leg. The transition from stance to ﬂight is called takeoﬀ and the transition from ﬂight to stance is called landing. In this context, (steady-state) running is deﬁned as a sequence of alternating stance and ﬂight phases that is symmetric with respect to the left and right legs stride-to-stride.7 2.2 Dynamics of Flight and Stance In the ﬂight phase, the robot has 7 DOF with generalized coordinates qf := (qb , q5 , xcm , ycm ) . The equations of motion for this phase may be derived using the method of Lagrange and written in the following form: Df (qb )¨ qf + Cf (qb , q˙f )q˙f + Gf (qf ) = Bf u ,

(1)

where Df is the inertia matrix, the matrix Cf contains Coriolis and centrifugal terms, and Gf is the gravity vector. Introducing the state vector xf := (qf , q˙f ) , the model (1) is expressed as x˙ f = ff (xf ) + gf (xf )u .

(2)

The state space is taken as Xf = T Qf = {xf = (qf , q˙f ) | qf ∈ Qf , q˙f ∈ IR7 }, where the conﬁguration space Qf is a simply-connected, open subset of IR7 corresponding to physically reasonable conﬁgurations of the robot. In the stance phase, the stance leg end is ﬁxed and, therefore, xcm and ycm are no longer independent coordinates. Accordingly, the robot in stance 7

The chosen deﬁnition of running is fundamental to the following model and controller development. Other authors have deﬁned running based on the motion of the center of mass or the reaction force proﬁle on the stance leg, for example see [14].

Achieving Bipedal Running with RABBIT: Six Steps Toward Inﬁnity

281

phase has 5 DOF with generalized coordinates q := (qb , q5 ) . Similar to the ﬂight phase, the equations of motion may be written as Ds (qb )¨ q + Cs (qb , q) ˙ q˙ + Gs (q) = Bs u .

(3)

Note that (3) may be obtained by subjecting (1) to the constraint that one leg end is in contact with the ground. Choosing the state vector xs := (q , q˙ ) , the model (3) may be expressed as x˙ s = fs (xs ) + gs (xs )u .

(4)

The state space is taken as Xs = T Qs = {xs = (q , q˙ ) | q ∈ Qs , q˙ ∈ IR5 }, where the conﬁguration space Qs is a simply-connected, open subset of IR5 corresponding to physically reasonable conﬁgurations of the robot. 2.3 Transitions Landing, the transition from the ﬂight phase to the stance phase, is modeled as a rigid impact. During this instantaneous event impulsive reaction forces from the ground bring the velocity of the tip of the advancing leg to zero without causing it to rebound or slip. In addition, at the moment of landing, the robot’s conﬁguration remains unchanged, but joint velocities change instantaneously [11]. The post-impact joint velocities8 are given by a function [5, Eq. (21)], ˜ − , q˙− ) . q˙+ = ∆(q f f

(5)

Since the gait is assumed to be symmetric from stride to stride (with respect to the left and right legs) a state relabeling matrix R is used to swap leg deﬁnitions (redeﬁne the coordinates) at landing. The ﬂight-to-stance transition operator, including state relabeling, is therefore deﬁned as . / R q− + s − xs = ∆f (xf ) := . (6) ˜ − , q˙− ) R∆(q f

f

This transition operator is applied when the end of the advancing leg touches the ground, that is, when y2 = 0 (see Fig. 1(b)). Deﬁne the function, Hfs : T Qf → IR by Hfs (xf ) = y2 , so that Hfs (xf ) = 0 characterizes the transition hypersurface surface Sfs within T Qf . The transition from stance to ﬂight is also modeled as an instantaneous event, but one on which positions and velocities are unchanged, 8

+ The terms x− := (qf− , q˙f− ) and x+ , q˙+ ) refer to the system state s := (q f − , q˙− ) and just before and just after the landing event. The terms x− s := (q + + x+ f := (qf , q˙f ) refer to the system state just before and just after the takeoﬀ event. The addition of the superscript “ ∗ ” (such as x+∗ f ) indicates reference to the value at steady-state, i.e., on the periodic orbit.

282

B. Morris et al.

.

q−

/



    fcm (q − ) f −  .  . / x+ = ∆ (x ) := s s f   q˙−   ∂ − − f (q ) q ˙ ∂q cm

(7)

where fcm (q) := (xcm (q), ycm (q)) gives the location of the center of mass. The transition from stance into ﬂight is treated as a control decision because the ﬂight phase is initiated (at will) by accelerating the stance leg oﬀ the ground. The transition hypersurface, Ssf , is a level set of a function Hsf (xs ) : T Qs → IR that is chosen by the control designer. 2.4 A Hybrid, Open-Loop Model of Running The stance and ﬂight dynamic models may be represented compactly, along with their transition models, as a discrete-event system with two charts (terminology taken from [9]). This open-loop hybrid model is speciﬁed by charts Σf and Σs where for (i, j) ∈ {(f, s), (s, f)}, Σi = {Xi , Fi , Sij , Tij }, 1. Xi is a state manifold, which is 10 dimensional in stance and 14 dimensional in ﬂight; 2. Fi is a ﬂow on the state manifold, a diﬀerential equation describing the in-phase motion on Xi ; 3. Sij is a switching hypersurface, a hypersurface of Xi corresponding to a transition from one state manifold to another; and 4. Tij is a transition map giving initial conditions for the next continuous phase. In this notation, the open-loop hybrid model is

Σf :

Σs :

 Xf = T Qf       Ff : (x˙ f ) = ff (xf ) + gf (xf )u      

Sfs = {xf ∈ T Qf | Hfs (xf ) = 0} s − Tfs : x+ s = ∆f (xf )

 Xs = T Qs       Fs : (x˙ s ) = fs (xs ) + gs (xs )u      

(8a)

Ssf = {xs ∈ T Qs | Hsf (xs ) = 0} f − Tsf : x+ f = ∆s (xs ).

(8b)

Achieving Bipedal Running with RABBIT: Six Steps Toward Inﬁnity

283

3 Control Methodology 3.1 Summary and Philosophy The overall philosophy of HZD control is to use the freedom available in feedback design to achieve a parameterized family of closed-loop systems whose stability analysis is analytically tractable. This allows the use of numerical optimization to search among the family of closed-loop systems to ﬁnd those that yield a desired behavior, such as stable running at a pre-determined speed with upper bounds on peak actuator power and the coeﬃcient of static friction between the leg end and the ground. Parameterization is achieved through the use of virtual constraints in both the stance and ﬂight phases. Perfect enforcement of virtual constraints results in low-dimensional surfaces that are invariant under the diﬀerential equations of the closed-loop model9 and are also invariant under the transition maps.10 To achieve invariance at landing, a deadbeat action is incorporated in the ﬂight phase controller that steers the robot to land in a pre-determined conﬁguration, while respecting conservation of angular momentum about the robot’s center of mass. This hybrid controller creates a one DOF HZD that allows the stability of a running motion to be analyzed in closed form on the basis of a one-dimensional Poincar´e map. In the ﬁrst running experiment attempted on RABBIT, there was not sufﬁcient time11 to implement completely the controller of [5]. The controller that was implemented used virtual constraints in both the stance and ﬂight phases, but the deadbeat action of the ﬂight phase controller was not implemented to regulate the ﬁnal conﬁguration of the robot at touchdown. Instead, to account for the changing conﬁguration of the robot at touchdown, the transition controller of [19] was adopted12 . Key points of the related analysis are highlighted in Sect. 3.8. 3.2 Preliminaries on Virtual Constraints Since RABBIT has four independent actuators (two at the hips and two at the knees), four virtual constraints may be imposed in both the stance and 9

10

11 12

“Invariant” in this sense means that if the diﬀerential equation is initialized on the constraint surface, then its solution remains on the constraint surface until a transition occurs. “Invariant” in this sense means that if the solution is on the ﬂight phase (resp. stance phase) constraint surface at touchdown (resp. takeoﬀ), then after transition the solution will be contained in the stance phase (resp. ﬂight phase) constraint surface. A total of two weeks were available to perform the experiments. The transition controller of this paper takes into account the joint angles of the robot at touchdown but not the joint angular velocities. As a result a true HZD of running is not created, and the resulting analysis of Sect. 3.8 (based on [8]) is modiﬁed accordingly.

284

B. Morris et al.

ﬂight phases. To deﬁne them, consider a function pair {θ(q), hd (θ)}, where θ : Q → IR is a scalar function of the conﬁguration variables, and hd : IR → IR4 is a function giving the desired conﬁguration of the actuated joints as a function of θ. The virtual constraints are expressed as outputs of (8), y = h(q) := qb − hd ◦ θ(q) ,

(9)

which are zeroed by the action of a state feedback controller. The design of such a controller is a well-understood, standard problem of nonlinear control [12]. For purposes of design, the virtual constraints are parameterized [5]. For notational convenience, the stance phase and ﬂight phase virtual constraints will be parameterized separately by as and af , respectively. These parameter sets, which lie in the parameter spaces As := IRns and Af := IRnf , may be updated at takeoﬀ and landing events but are otherwise constant. With this notation, the virtual constraints for stance and ﬂight are, respectively,13 y = qb − hd,s [as ](θs (q))

(10a)

y = qb − hd,f [af ](θf [af ](qf )) .

(10b)

3.3 Stance Phase Control The controller for the stance phase acts by updating the parameters as and by enforcing the virtual constraints (10a). As a result of enforcing the virtual constraints, in stance phase, the robot behaves as an unactuated 1 DOF system whose properties may be tuned by choosing diﬀerent constraint parameters. Apart from diﬀerent boundary conditions on the virtual constraints, this control is identical to the walking controllers developed in [18, 20]. The stance phase parameter vector, as , may be expressed as as := (as,0 , as,1 , . . . , as,ms −1 , as,ms , θs− , θs+ ) ,

(11)

where ms ≥ 3, as,i ∈ IR4 for i ∈ {0, 1, . . . , ms − 1, ms }, and θs− , θs+ ∈ IR. Note that ns = 4 (ms + 1) + 2. The terms θs− and θs+ are the values of the function θs (q) evaluated at the end and the beginning of the stance phase. In [18, 20], hd is expressed in terms of B´ezier polynomials. Here, a slightly diﬀerent class of polynomials14 is used that satisfy the following: hd,s [as ](θs+ ) = as,0 d + dθs hd,s [as ](θs ) 13

14

= as,1

d − dθs hd,s [as ](θs )

= as,ms −1

hd,s [as ](θs− ) = as,ms .

(12)

Terms that are constant during the continuous phases of motion, and potentially updated at phase transitions, will be considered parameters and enclosed in square brackets. Any class of smooth functions satisfying these properties may be used to deﬁne virtual constraints.

Achieving Bipedal Running with RABBIT: Six Steps Toward Inﬁnity

285

The stance phase virtual constraints are imposed on the dynamics by using a control us : Xs × As → IR4 that drives (10a) to zero in ﬁnite time. The speciﬁc assumptions are as in [8, 20]. 3.4 Flight Phase Control The development of the ﬂight phase controller is similar to that of the stance phase controller. The key diﬀerence is the choice of θf in (10b) to be a function of the position of the center of mass. The ﬂight phase parameter vector, af , is deﬁned as af := (af,0 , af,1 , . . . , af,mf −1 , af,mf , x+ ˙+ cm,f , x cm,f , Tf ) ,

(13)

˙+ where mf ≥ 3, af,i ∈ IR4 for i ∈ {0, 1, . . . , mf −1, mf }, and x+ cm,f , x cm,f , Tf ∈ IR. + Note that nf = 4 (mf + 1) + 3. The terms x+ , x ˙ , and T are, respectively, f cm,f cm,f the horizontal position of the center of mass at the beginning of the ﬂight phase, the horizontal velocity of the center of mass at the beginning of the ﬂight phase, and the estimated15 duration of the ﬂight phase. The ﬂight phase virtual constraints (10b) are given by + 1 xcm − xcm,f , (14) θf [af ](qf ) := Tf x˙ + cm,f and hd,f [af ], which, as in the stance phase, is a smooth, vector-valued function that satisﬁes hd,f [af ](0) = af,0 d dθf

hd,f [af ](0) = af,1

d dθf

hd,f [af ](1) = af,mf −1 (15) hd,f [af ](1) = af,mf .

For a given stride, let tf denote the elapsed time within the ﬂight phase. By conservation of linear momentum, x˙ + cm,f is constant during ﬂight, which + + implies tf = (xcm − xcm,f )/x˙ cm,f . As a result, θf = tf /Tf is a valid substitute for (14), and for this reason, the given ﬂight phase virtual constraints are said to be time scaled. Flight phase virtual constraints are enforced using any smooth state feedback controller uf : Xf × Af → IR4 that drives (10b) to zero exponentially quickly. Note that ﬁnite-time convergence is not used in the ﬂight phase. A ﬁnitetime controller is used in the stance phase to render the stance phase constraint surface ﬁnite-time attractive so that the analysis of running will be similar to that of walking [8]. For further discussion of this point, refer to Sect. 3.8. 15

− Calculation of Tf requires the height of the center of mass at landing, ycm,f , to be known a priori, which is only possible if the virtual constraints are exactly enforced throughout the ﬂight phase.

286

B. Morris et al.

3.5 Transition Control: Landing In the event that landing occurs with the state of the robot not satisfying the virtual constraints, the control parameters of the subsequent stance phase, as , are updated to ensure that the conﬁguration of the robot satisﬁes qb − hd,s [as ](θs+ ) = 0. The parameter updates are governed by the diﬀerentiable function wfs : Sfs → As , such that for as = wfs (x− f ), as,0 = qb+ as,1 = a∗s,1 .. . as,ms −1 = a∗s,ms −1

θs+ = θs (q + ) θs− = θs−∗

(16)

as,ms = a∗s,ms .

−∗ In the above, q + is calculated using ∆sf (x− and a∗s,i ∈ IR4 , f ), and the terms θs i ∈ {1, . . . , ms − 1, ms } are constant parameters chosen during design. If the stance phase ﬁnite-time controller can satisfy the virtual constraints (10a) before the liftoﬀ event occurs, and the parameter updates obey (16), then the stance phase will terminate with qb − hd,s [as ](θs− ) = 0, or equivalently, with q − = q −∗ .

3.6 Transition Control: Takeoﬀ At takeoﬀ, the parameters of the ﬂight phase virtual constraints, af , are updated so that the duration of the planned motion of the robot is equal to the estimated ﬂight time. Parameter updates are governed by a continuously diﬀerentiable function wsf : Ssf → Af , such that for af = wsf (x− s ), af,0 = a∗f,0 af,1 = a∗f,1 .. .

− x+ cm,f = (fcm (q ))1 ∂fcm − − (q x˙ + = ) q ˙ cm,f ∂q 1

af,mf −1 = a∗f,mf −1 af,mf = a∗f,mf 1 + −∗ + + (y˙ cm,f )2 − 2g(ycm,f − ycm,f ) y˙ cm,f + . Tf = g g

(17)

−∗ where g is the magnitude of the acceleration of gravity and ycm,f is the height of the center of mass at the end of the ﬂight phase, on the limit cycle. The terms a∗f,i ∈ IR4 , i ∈ {0, 1, . . . , mf − 1, mf } are parameters chosen during design. Initiation of the takeoﬀ event is a control decision, designated to occur when θs (q) = θs− . In the closed-loop model the switching hypersurface is Ssf = {(xs , as ) ∈ Xs × As | Hsf (xs , as ) = 0} where Hsf (xs , as ) := θs (q) − θs− .

Achieving Bipedal Running with RABBIT: Six Steps Toward Inﬁnity

287

3.7 Resulting Closed-Loop Model of Running To form the closed-loop model of running, the state space of the open-loop model, (8), is enlarged to include the parameters of the ﬂight and stance phases. Deﬁne the augmented state spaces X¯f := T Qf ×Af and X¯s := T Qs ×As with elements given by x ¯f := (qf , q˙f , af ) and x ¯s := (q , q˙ , as ) . The closed-loop dynamics may then be written as . / (x ) + g (x )u (x , a ) f f f f f f f f f¯f (¯ xf ) := (18a) 0nf ×1 . / (x ) + g (x )u (x , a ) f s s s s s s s xs ) := f¯s (¯ . (18b) 0ns ×1 The zero vectors reﬂect that the virtual constraint parameters do not change during the continuous phases of running. The impact maps, in which the parameters are updated, are modiﬁed to include the parameter update laws, wfs and wsf : . / ∆sf (x− f ) s − ¯ ∆f (¯ xf ) := (19a) wfs (x− f ) . / ∆fs (x− ) s f − xs ) := . (19b) ∆¯s (¯ wsf (x− s ) The closed-loop hybrid model is then  X¯f = T Qf × Af       F¯f : (x ¯˙ f ) = f¯f (¯ xf ) Σcl,f : s ¯  Sf = {(xf , af ) ∈ X¯f | Hfs (xf ) = 0}      ¯s + Tf : x ¯s = ∆¯sf (¯ x− f )  X¯s = T Qs × As       F¯s : (x ¯˙ s ) = f¯s (¯ xs ) Σcl,s :  S¯sf = {(xs , as ) ∈ X¯s | Hsf (xs , as ) = 0}      ¯f ¯f x− Ts : x ¯+ s ). f = ∆s (¯

(20a)

(20b)

3.8 Existence and Stability of Periodic Orbits The Poincar´e return map is a well-known tool for determining the existence of periodic orbits and their stability properties (see Fig. 2). For its use in hybrid systems, see for example [6, 8, 10, 15]. Its application to periodic orbits of (20) can be carried out using the results in [8] and a construction presented in

288

B. Morris et al.

∆(x− ) P (x)

x

x x∗ S

x− P (x− )

+

φ(t, x)

∆(S)

S

φ(t, x∗ )

(a) Poincar´e return map for ODE systems

− φ(t, ∆(x )) (b) Poincar´e return map for systems with impulse eﬀects

Fig. 2. Illustration (a) shows the geometric interpretation of a Poincar´e return map P : S → S for an ordinary diﬀerential equation (non-hybrid) system as event-based sampling of the solution near a periodic orbit. The Poincar´e section, S, may be any co-dimension one (hyper) C 1 -surface that is transversal to the periodic orbit. Illustration (b) shows the geometric interpretation of a Poincar´e return map for a system with impulse eﬀects. A periodic orbit exists when P (x− ) = x−

[5, Thm. 1]. Since thorough development of these ideas would consume more space than is available, the main ideas are only sketched. The ﬁrst step is to construct a system with impulse eﬀects (that is, a single-chart hybrid model) that has the same Poincar´e map as (20). Following [5, Eq. (62)], deﬁne + x ¯˙ (t) = f¯s (¯ x(t)) x− (t) ∈ S¯ (21) Σcl : ¯ x− (t)) x ¯ x ¯+ (t) = ∆(¯ ¯− (t) ∈ S, where S¯ := S¯sf , ∆ := ∆¯sf ◦ Pf , and Pf is the ﬂow of the closed-loop ﬂight phase model (see [5, Eq. (59)]). In words, this system consists of the diﬀerential equation of the closed-loop stance phase model of (20) and a generalized impact map ∆¯ that includes the transition map from stance to ﬂight, the ﬂight phase dynamics, and the impact map from ﬂight to stance. The generalized impact map is the result of event-based sampling of the solution of (20) at takeoﬀ events. Because the virtual constraints in the stance phase are achieved with a continuous ﬁnite-time controller [1], the reduction technique of [8, Thm. 2] is applicable. Because the parameter updates in the stance phase can be computed in terms of the state of the robot at takeoﬀ, the analysis of periodic orbits can be reduced to the computation of a one-dimensional restricted Poincar´e map, ρ, having Ssf as a Poincar´e section.

Achieving Bipedal Running with RABBIT: Six Steps Toward Inﬁnity

289

4 Design of Running Motions with Optimization 4.1 Optimization Parameters To design a running motion, a numerical routine is used to search the parameter spaces As and Af for a set of parameters that results in a desirable gait (periodic orbit of (20)). Common requirements on the gait are achieved by incorporating constraints into the numerical search. Such constraints address actuator limits, allowable joint space, and unilateral ground-contact forces. The constraints also ensure steady-state running at a certain speed and overall eﬃciency of the gait. For the experiments reported here, the gait was designed using an optimization approach that combined the ideas of [4] and [20]; the optimization was performed directly on the parameters of the virtual constraints in order to simultaneously determine a periodic running motion and a controller that achieves it. This is in contrast with the approach of [5] where virtual constraints are designed by regression against optimal, pre-computed, periodic trajectories. Virtual constraints are assumed to be identically satisﬁed on the periodic orbit, which has two consequences: ﬁrst, the integration of the closed-loop system dynamics can be performed using the stance and ﬂight phase zero dynamics (see [5] for details), resulting in short computation times; and second, the virtual constraint parameters, as and af , are not independent. Once the independent parameters have been identiﬁed, standard numerical optimization routines may be used to search for desirable gaits. The implementation of such a procedure is outlined in the following subsections. 4.2 Boundary Conditions of the Virtual Constraints The transition maps of takeoﬀ and landing can be used to identify redundancies between the virtual constraint parameter vectors as and af . Given the state corresponding to the end of the limit-cycle stance phase, −∗ −∗ , q˙ ), the state at the beginning of the subsequent ﬂight phase x−∗ s = (q −∗ = (qf+∗ , q˙f+∗ ) = ∆fs (x−∗ and x+∗ to may be computed as x+∗ s ). For both xs f f satisfy the virtual constraints of their respective phases, the following relations must hold, −∗ ˙ −∗ +∗ /θs a∗f,0 = qb,f a∗s,ms −1 = q˙b,s (22) −∗ +∗ ∗ a∗s,ms = qb,s a∗f,1 = q˙b,f Tf , which are derived by applying (12), (14), (15), and (17) to (10). These are the boundary conditions associated with the liftoﬀ event of the periodic orbit. The +∗ +∗ , q˙ ), can state of the robot at the beginning of the stance phase, x+∗ s = (q −∗ be related to the state at the end of the previous ﬂight phase, xf = (qf−∗ , q˙f−∗ ), s −∗ by the landing map, x+∗ s = ∆f (xf ), to yield the following additional design constraints,

290

B. Morris et al.

Table 1. Independent and dependent terms used in optimization. The choice of the independent terms is non-unique and depends on the speciﬁc optimization procedure. The parameters below correspond to the algorithm in Sect. 4.3, which is one straightforward method to ensure the boundary conditions of the virtual constraints are met Terms of Optimization Independent Dependent x−∗ f ∗ ∗ as,2 , . . . , as,ms θs−∗ ∗ ∗ af,2 , . . . , af,mf −2

∈ ∈ ∈ ∈

IR14 IR4 IR IR4

+∗ a∗s,0 = qb,s +∗ ˙ +∗ a∗s,1 = q˙b,s /θs

θs+∗ a∗s,1 x+∗ s ∗ af,0 , a∗f,1 a∗f,mf −1 , a∗f,mf +∗ ∗ xcm,f , x˙ +∗ cm,f , Tf − xf a∗s,0 ,

∈ ∈ ∈ ∈ ∈ ∈ ∈

IR IR4 IR10 IR4 IR4 IR IR14

−∗ ∗ a∗f,mf −1 = q˙b,f Tf −∗ a∗f,mf = qb,f .

(23)

The update law presented here enforces fewer boundary conditions than the update law of [5]. The extra boundary conditions associated with takeoﬀ are already satisﬁed by (22), but those of landing are not met by (23); they are more diﬃcult to satisfy due to conservation of angular momentum in the ﬂight phase. The main theoretical result of this paper is that invariance of the ﬂight and stance phase constraint surfaces over the landing event is not a necessary condition for achieving provably stable running. As noted earlier, relaxing this condition makes running motions signiﬁcantly easier to design. 4.3 Optimization Algorithm Details Trial gaits for the running experiments were generated using the constrained nonlinear optimization routine fmincon of MATLAB’s Optimization Toolbox. Three quantities are involved in optimization: J, a scalar cost function to be minimized on the periodic orbit, EQ, a vector of equality constraints, and IN EQ, a vector of inequality constraints. The following is a description of the optimization procedure that was implemented. The independent and dependent terms16 of optimization are given in Table 1. Note that when the will be a point looptimizer terminates with the constraints satisﬁed, x+∗ s cated on a closed-loop periodic orbit and the virtual constraints will be given by (11) and (13).

16

“Terms” is used to describe those variables used in optimization; these are diﬀerent from the parameters of the virtual constraints.

Achieving Bipedal Running with RABBIT: Six Steps Toward Inﬁnity

291

Algorithm 1. Select x−∗ = (qf−∗ , q˙f−∗ ), the state corresponding to the end of the ﬂight f phase. = 2. Using the ﬂight-to-stance transition function, ∆sf , calculate x+∗ s (q +∗ , q˙+∗ ), the state corresponding to the beginning of the subsequent stance phase. 3. Calculate θs+∗ by (16) and a∗s,0 , a∗s,1 by (23). 4. Select a∗s,2 , . . . , a∗s,ms , and θs−∗ to complete the stance phase parameter vector as . 5. Using parameters as and the initial condition x+∗ s , integrate the equations of motion of stance and apply the stance-to-ﬂight transition operator, ∆fs , = (qf+∗ , q˙f+∗ ). to obtain x+∗ f ∗ ˙ +∗ 6. Calculate af,0 , a∗f,1 by (22); a∗f,mf −1 , a∗f,mf by (23); and x+∗ cm,f , x cm,f , and ∗ Tf by (17). 7. Select a∗f,2 , . . . , a∗f,mf −2 to complete the ﬂight phase parameter vector af . 8. Using parameters af , and initial condition x+∗ f , integrate the equations of . motion of ﬂight to obtain x− f 9. Evaluate J, EQ, and IN EQ. 10. Iterate Steps 1 to 8 until J is (approximately) minimized, each entry of EQ is zero, and each entry of IN EQ is less than zero. 4.4 An Example Running Motion A sample running gait designed by the above algorithm is now presented. A stick diagram of this motion is given in Fig. 3(a). The stability analysis outlined in Sect. 3.8 was applied to the resulting running motion. Figure 3(b) gives the restricted Poincar´e map, which indicates that the motion is locally exponentially stable. The gait was designed to minimize the integral of torque squared per distance traveled, with the following constraints: Equality Constraints, EQ −∗ • error associated with ﬁnding a ﬁxed point ||x− f − xf || • deviation from the desired running rate • required frictional forces at the leg ends are zero just before takeoﬀ and just after landing (to prevent slipping at these transitions)

Inequality Constraints, INEQ • magnitude of the required torque at each joint less than 100 Nm • knee angles to lie in (0◦ , −70◦ ) and hip angles to lie in (130◦ , 250◦ ) (see Fig. 1(b) for measurement conventions) • minimum height of the swing foot during stance greater than 7 cm

292

B. Morris et al. 1.6

340

1.4

330

ζi+1 = ρ(ζi )

1.2

(m)

1 0.8 0.6 0.4 0.2 0

320 310 300 290 280

−0.5

0

0.5

1

1.5

270

(m) (a) Stick diagram

ζ ∗ = 303 ζmin = 275 280 290 300 310 320 330 340

ζi (b) Poincar´e map

Fig. 3. Stick diagram and Poincar´e map for the example running motion (rate − 2 ) /2 at the end of 0.58 m/s). Poincar´e map constructed by evaluating ζ := (σs,1 − successive stance phases, where σs,1 is the angular momentum about the stance leg end just before liftoﬀ. The ﬁxed point, ζ ∗ = 303, is located at the intersection of ρ and the identity map ζi = ζi+1 , and corresponds to an equilibrium running rate of 0.58 m/s. The slope of the graph at ζ ∗ is dρ/dζ ≈ 0.67, indicating exponential stability

• required coeﬃcient of friction of the stance phase less than 0.7 • ﬂight time greater than or equal to 25% of total gait duration • landing foot impacts the ground at an angle of approach less than 45◦ from vertical • joint angular velocities less than 5 rad/s

5 Experiment 5.1 Hardware Modiﬁcations to RABBIT Prior to the experiment reported here, only walking experiments had been performed with RABBIT. To prepare for the task of running, four hardware modiﬁcations were made. The ﬁrst modiﬁcation was the inclusion of prosthetic shock absorbers in the shanks. It was speculated that with shock absorbers the landing would cause less wear and tear on the harmonic drive gear reducers that form RABBIT’s hip and knee joints. The inclusion of shock absorbers added approximately 5 cm to each shank. The second modiﬁcation was the installation of force sensitive resistors into RABBIT’s point feet. These devices allowed for more accurate measurement of the touchdown time than did the previously installed mechanical contact switches. Since these sensors suﬀer from signiﬁcant drift, their signals were numerically diﬀerentiated to make easier the detection of impact events.

Achieving Bipedal Running with RABBIT: Six Steps Toward Inﬁnity

293

The last two modiﬁcations were the bolting of aluminum u-channel stock along each thigh and the widening of the hips. Both of these changes were made to help prevent ﬂexing of the legs in the frontal plane. Signiﬁcant ﬂexing was witnessed during the ﬁrst several experimental trials of running. This problem was more pronounced in running than in walking because of the greater impact forces associated with landing. On several occasions RABBIT “tripped itself” during a stance phase of running when the swing leg passed by the stance leg (the legs knocked against each other). This came about because RABBIT was designed to have its legs close together to better approximate a planar biped. 5.2 Result: Six Running Steps After completing hardware modiﬁcations and successfully reproducing previous walking experiments, running experiments were conducted. A number of experimental trials resulted in RABBIT taking several human-like17 running steps. One such trial, which was an implementation of the example running motion of Sect. 4.4, will be discussed here. For this experiment, motion was initiated by an experimenter who pushed the robot forward, into the basin of attraction of a walking controller that induced walking with an average forward walking rate of 0.8 m/s. RABBIT then achieved stable walking, followed by a transition to running in a single step, followed by 6 running steps. After the sixth step, the experiment was terminated by the control software when the tracking error limit of 0.3 radians was exceeded for the stance knee angle. Examination of collected data suggests that tracking error resulted from actuator saturation18 . Data also show the swing leg extremely close to the ground at the moment the experiment was terminated, suggesting the swing leg may have, in fact, struck the ground contributing additional tracking error. A plot of estimated19 foot height is given in Fig. 4. Average stride duration for the steps was 431 ms. Flight times, observed as those portions of Fig. 4 where neither leg is at zero height, lasted an average of 107 ms (25% of the stride). Videos of the experiment and many additional data plots are available at [7, 17].

17

18

19

A human-like gait is considered to be characterized by an upright posture, a torso leaning slightly forward, and a long step length. See [18] for a description of the PD controllers used to enforce the virtual constraints. When RABBIT is in ﬂight, there is no accurate way to determine hip height. A sensor was mounted to record boom pitch angle, but due to ﬂexing of the boom, these data were inaccurate. During the stance phase this lack of sensing is not a problem because the end of the stance leg is always at zero height.

B. Morris et al.

estimated height (m)

294

0.16 0.12 0.08 0.04 0 18.5

19

19.5

20

20.5

21

21.5

time (sec) Fig. 4. Estimated height of the point feet with RABBIT’s left foot indicated in bold. Flight phases occur when neither foot is at zero height

5.3 Discussion Several problematic issues related to RABBIT’s hardware did not appear until running was attempted. (For a discussion of general implementation issues of walking including unmodeled eﬀects of the boom, gear reducers, and an uneven walking surface see [18].) Future running experiments–whether on RABBIT or another, similar mechanism–should take into account the following issues. Boom Dynamics The perturbing eﬀects of the boom were found to be much more signiﬁcant during ﬂight phases than during stance phases. When RABBIT is modeled as a planar system, an analysis of the three-dimensional mechanics shows that the contribution of the boom to the center of mass dynamics is signiﬁcant. Speciﬁcally, q5 is no longer, in general, a cyclic variable during ﬂight. However, if boom masses are appropriately distributed, the parabolic motion of the center of mass, as modeled in a planar system, is recovered. Unfortunately, this special mass distribution was impossible because RABBIT does not have a counterweight system. Walking Surface The walking surface was also a source of problems. This surface–consisting of rubber over elevated plywood supported on the edges by a wood frame– was originally built to provide a uniform, level surface. Although the surface appears uniform, walking experiments demonstrated otherwise. It was found that the surface has “fast” and “slow” areas corresponding to varying ﬂoor stiﬀness and coeﬃcient of friction.

Achieving Bipedal Running with RABBIT: Six Steps Toward Inﬁnity

295

Limited Joint Space For safety, RABBIT’s joints have hard stops that limit its joint space, which, for example, prevent the shank from contacting the thigh. Although the available joint space was suﬃcient for walking, it became a signiﬁcantly limiting factor in the design of running gaits. These hard stops prevented the swing leg from being folded close to the hip, which is a natural and desirable motion that minimizes the leg’s rotational inertia.

6 Conclusion A novel approach to the control of running in planar bipeds and its ﬁrst experimental implementation on RABBIT have been presented. The control law is hybrid, consisting of continuous actions in the stance and ﬂight phases, and discrete actions at the transitions between these phases. In the stance and ﬂight phases, the controller coordinates the relative motions of the robot’s links by imposing virtual constraints at the actuated joints. At the transition from stance to ﬂight, the controller adjusts the virtual constraints for the ﬂight phase as a function of estimated ﬂight duration to ensure that the former swing leg is advanced properly to take up its role as the next stance leg. At the transition from ﬂight to stance, the controller updates the virtual constraints of the stance phase to account for the orientation of the robot at landing. For the nominal periodic running motion, the parameters of the virtual constraints are determined by numerical optimization in order to meet actuator power limits, friction bounds, joint limits, etc. For running experiments, RABBIT’s mechanical and electrical systems were modiﬁed: shock absorbers were added to the shanks; the ground contact sensors were improved; the stiﬀnesses of legs in the frontal plane were increased; and the hips were widened. The main theoretical result of this paper was the development of a running controller that is based on the HZD methodology, but easier to design and implement while still resulting in a reduced-dimensionality stability test. The main experimental result of this paper was the physical realization of six consecutive running steps with a human-like gait and identiﬁcation of hardware diﬃculties of running with RABBIT that were not present in walking.

Acknowledgments The work of J.W. Grizzle and B. Morris was supported by NSF grant ECS0322395. The work of E.R. Westervelt was supported by NSF grant CMS0408348.

296

B. Morris et al.

References [1] S. P. Bhat and D. S. Bernstein. Finite-time stability of continuous autonomous systems. SIAM J. Contr. Optim., 38:751–66, 2000. [2] G. Buche. ROBEA Home Page. http://robot-rabbit.lag.ensieg.inpg.fr/ English/. [3] C. Chevallereau, G. Abba, Y. Aoustin, F. Plestan, E. R. Westervelt, C. Canudas de Wit, and J. W. Grizzle. RABBIT: a testbed for advanced control theory. IEEE Control Systems Magazine, 23(5):57–79, October 2003. [4] C. Chevallereau and Y. Aoustin. Optimal reference trajectories for walking and running of a biped robot. Robotica, 19(5):557–69, September 2001. [5] C. Chevallereau, E. R. Westervelt, and J. W. Grizzle. Asymptotically stable running for a ﬁve-link, four-actuator, planar bipedal robot. International Journal of Robotics Research, 24:431–64, 2005. [6] B. Espiau and A. Goswani. Compass gait revisited. In Proc. of the IFAC Symposium on Robot Control, Capri, Italy, pp.839–46, September 1994. [7] J. W. Grizzle. Jessy Grizzle’s publications. http://www.eecs.umich.edu/ ∼grizzle/papers/robotics.html. [8] J. W. Grizzle, G. Abba, and F. Plestan. Asymptotically stable walking for biped robots: Analysis via systems with impulse eﬀects. IEEE Transactions on Automatic Control, 46:51–64, January 2001. [9] J. Guckenheimer and S. Johnson. Planar hybrid systems. In Hybrid Systems II, Lecture Notes in Computer Science, pp. 203–25. Springer-Verlag, 1995. [10] I. A. Hiskens. Stability of hybrid limit cycles: application to the compass gait biped robot. In Proc. of the 40th IEEE Conf. Dec. and Control, Orlando, FL, December 2001. [11] Y. H¨ urm¨ uzl¨ u and D. B. Marghitu. Rigid body collisions of planar kinematic chains with multiple contact points. International Journal of Robotics Research, 13(1):82–92, 1994. [12] A. Isidori. Nonlinear Control Systems: An Introduction. Springer-Verlag, Berlin, third edition, 1995. [13] S. Kajita, T. Nagasaki, K. Kaneko, K. Yokoi, and K. Tanie. A hop towards running humanoid biped. In Proc. of the 2004 IEEE International Conference on Robotics and Automation, pp. 629–35, 2004. [14] T. A. McMahon, G. Valiant, and E. C. Frederick. Groucho running. Journal of Applied Physiology, 62(6):2326–37, June 1987. [15] S. G. Nersesov, V. Chellaboian, and W. M. Haddad. A generalization of Poincar´e’s theorem to hybrid and impulsive dynamical systems. Int. J. Hybrid Systems, 2:35–51, 2002. [16] M. H. Raibert. Legged robots that balance. MIT Press, Cambridge, MA, 1986. [17] E. R. Westervelt. Eric Westervelt’s publications. http://www.mecheng.osu. edu/∼westerve/publications. [18] E. R. Westervelt, G. Buche, and J. W. Grizzle. Experimental validation of a framework for the design of controllers that induce stable walking in planar bipeds. International Journal of Robotics Research, 23(6):559–82, 2004.

Achieving Bipedal Running with RABBIT: Six Steps Toward Inﬁnity

297

[19] E. R. Westervelt, J. W. Grizzle, and C. Canudas de Wit. Switching and PI control of walking motions of planar biped walkers. IEEE Transactions on Automatic Control, 48(2):308–12, February 2003. [20] E. R. Westervelt, J. W. Grizzle, and D. E. Koditschek. Hybrid zero dynamics of planar biped walkers. IEEE Transactions on Automatic Control, 48(1):42–56, January 2003.

Velocity-Based Stability Margins for Fast Bipedal Walking J.E. Pratt1 and R. Tedrake2 1

2

Florida Institute for Human and Machine Cognition [email protected] Massachusettes Institute of Technology [email protected]

Summary. We present velocity-based stability margins for fast bipedal walking that are suﬃcient conditions for stability, allow comparison between diﬀerent walking algorithms, are measurable and computable, and are meaningful. While not completely necessary conditions, they are tighter necessary conditions than several previously proposed stability margins. The stability margins we present take into consideration a biped’s Center of Mass position and velocity, the reachable region of its swing leg, the time required to swing its swing leg, and the amount of internal angular momentum available for capturing balance. They predict the opportunity for the biped to place its swing leg in such a way that it can continue walking without falling down. We present methods for estimating these stability margins by using simple models of walking such as an inverted pendulum model and the Linear Inverted Pendulum model. We show that by considering the Center of Mass location with respect to the Center of Pressure on the foot, these estimates are easily computable. Finally, we show through simulation experiments on a 12 degree-of-freedom distributed-mass lower-body biped that these estimates are useful for analyzing and controlling bipedal walking.

1 Introduction “How stable is your robot?” is a fundamental yet challenging question to answer, particularly with fast moving legged robots, such as dynamically balanced bipedal walkers. With many traditional control systems, questions of stability and robustness can be answered by eigenvalues, phase margins, loop gain margins, and other stability margins. However, legged robots are nonlinear, under-actuated, combine continuous and discrete dynamics, and do not necessarily have periodic motions. These features make applying traditional stability margins diﬃcult. In this paper we deﬁne stability for a biped simply as whether or not the biped will fall down. We focus on velocity-based stability since we believe that regulating the velocity of the Center of Mass is the most challenging

300

J.E. Pratt and R. Tedrake

subtask for human-like bipedal walking. Regulating velocity is a challenging subtask due to the extended period during a natural gait that the Center of Mass velocity is underactuated (the actuators cannot produce an arbitrary acceleration on the Center of Mass). For example, once the body has traveled far enough away from the foot, the only course of action that can stabilize the Center of Mass velocity is to take a step. Other requirements such as regulating virtual leg length and body orientation, and swinging the swing leg, can be met through traditional control system techniques since these subtasks are fully actuated during the majority of the gait. We argue that the most crucial thing for regulating velocity in bipedal walking is the ability to place the foot of the swing leg in a proper location that allows for maintaining or reducing velocity on subsequent steps. A suﬃcient condition for being able to maintain or reduce velocity is the ability to eventually come to a stop. Thus we deﬁne stability margins that estimate the likelihood that a biped can “Capture” its kinetic energy and stop over a given number of steps. We deﬁne a Capture Point as a point that can be stepped to in order to stop. We propose the “N-Step Capture Stability Margin” which gives an indication of the degree to which a biped can stop in N steps. For most practical purposes if a walking biped cannot stop within several steps, it is probably close enough to falling to consider it unstable. Hence being, say, 10-Step Capturable is a suﬃcient, and close to necessary, stability condition. Exact computation of these stability margins is diﬃcult since the dynamics of bipedal walking is complex. However, we can compute these margins for simpliﬁed walking models. These simpliﬁed walking models give useful approximations to the real values of the margins, which we demonstrate through control experiments on a simulated 12 degree-of-freedom lower-body biped with distributed mass. The robot can recover from being pushed by stepping to a Capture Point computed from the simpliﬁed walking models. It can step to desired foothold locations by guiding the Capture Point to the desired stepping point and stepping once the Capture Point reaches it.

2 Stability Deﬁnition In this paper, we deﬁne stability for a biped in terms of whether or not the biped will fall down. However, the concept of falling down is diﬃcult to precisely deﬁne. For example, sitting down on the ﬂoor and slipping down onto the ﬂoor might result in the exact same trajectories and end state but one is considered falling and the other is considered sitting, with the only diﬀerence being intent. For the purposes of this paper, to eliminate complications rising from such concepts as intent, let us deﬁne a fall as follows. Deﬁnition 1 (Fall). When a point on the biped, other than a point on the feet of the biped, touches the ground.

Velocity-Based Stability Margins for Fast Bipedal Walking

301

This deﬁnition allows for such things as holding a hand railing to not be considered a fall, by pushing the burden onto a precise deﬁnition of the ground and of the feet. However, in this paper, we only consider situations in which the ground is everything that is not part of the biped and the feet are the two commonly known appendages at the end of the legs. Turning now to the question of whether the biped will fall, consider the deterministic closed-loop dynamics of the biped in the general form x˙ = f (x, t), where x ∈ N is the state vector. Deﬁne a subset F ⊂ N which includes all conﬁgurations of the robot for which some part of the robot other than the feet is touching the ground. The basin of attraction of F , which we will call the “Basin of Fall”, deﬁnes all of the states of the robot that eventually lead to a fall: Deﬁnition 2 (Basin of Fall). Subset of state space that leads to a fall. B ⊂ N , x(t) ∈ B ⇒ ∃∆t ≥ 0 s.t. x(t + ∆t) ∈ F The complement of the Basin of Fall is the Viability Kernel of Wieber [32]. Whether or not the robot will fall down, and hence whether or not the robot is stable, can now be precisely deﬁned. Deﬁnition 3 (Stable). A biped is stable if and only if the state of the robot is not inside the Basin of Fall, B. Note that for a real-world biped in a non-deterministic environment, the Basin of Fall may be the entire state-space, as all bipeds will eventually fall given enough time. Also, note that for a biped that has regions of chaotic gait, the Basin of Fall may be Uncomputable as determining whether some states are in the Basin of Fall may be Undecideable [26]. In addition, for the system to be deterministic, the dynamics, f , and the state vector, x, must contain full information about the environment such as the ground proﬁle. Encoding the entire environment for all time is prohibitive in general. Therefore, to ensure computability, non-zero volume of stable states, and feasible deﬁnition of the state and environment, one may wish to consider the state of the biped to be stable if it does not lead to a fall after some reasonable ﬁnite amount of time, τ , and deﬁne the Time-Limited Basin of Fall as follows: Deﬁnition 4 (Time-Limited Basin of Fall). Subset of state space that leads to a fall within a ﬁnite amount of time, τ . Bτ ⊂ N , x(t) ∈ Bτ ⇒ ∃∆t, 0 ≤ ∆t ≤ τ s.t. x(t + ∆t) ∈ F For human-like walking, approximately 1 minute is a reasonable horizon to consider since it is highly unlikely that a biped would be in a state where a fall is inevitable but the biped can stagger around before the fall for a whole minute. Computing the Basin of Fall, B, is conceptually simple, but computationally expensive. Given an accurate closed-loop dynamic model and a discretized state space, one can use dynamic programming to determine B. Let V (x) be

302

J.E. Pratt and R. Tedrake

the value function encoding the time until the biped falls. Initialize V to zero for states that are falling states and inﬁnity for all other states. Then use the update rule V (xn ) = min(V (xn ), V (xn+1 ) + ∆t) where state xn dynamically leads to state xn+1 and ∆t is the time taken to transition from state xn to xn+1 . In the worst case N 2 updates would have to occur, where N is the number of discrete states. If the dynamics can be simulated backward in time, then only N updates need be made if the states are visited recursively, starting from falling states. In a companion paper [28], we will use similar ideas to explore stochastic stability margins for legged locomotion. A stability margin that logically follows from the previous discussion is the distance to the Basin of Fall, called the Viability Margin by Wieber [32]. + /B minx ∈B (|x − x |) if x ∈ (1) MV iability (x) = − minx ∈B / (|x − x |) if x ∈ B By deﬁnition the Viability Margin is both a necessary and suﬃcient indicator of stability and thus would be a good stability margin for a biped. However, it does have drawbacks. It does not take into consideration what disturbances we would like the biped to be robust to, since distance in state space does not necessarily correlate to real-world disturbances. It does not take into consideration the dynamics of the system in moving from the current state to the nearest boundary state, since Cartesian nearness does not necessarily correlate with dynamic nearness. It assigns importance to each degree of freedom based upon its units of measure. It is diﬃcult to compute. And ﬁnally, it does not give us much insight into why a biped is stable or why it falls. Some of these problems could be mitigated by such things as weighting the state variables based on importance, for each state computing the magnitude of a given disturbance required to transition the biped into a Basin of Fall state, etc. However, these additions would only add to the computational complexity of computing this margin. Additionally, perfect dynamic models are impossible to attain for real systems, and to compute the entire Basin of Fall during experimental trials would require an infeasible number of trials, many of which could result in damage to the biped. Due to these limitations, instead of attempting to compute the entire Basin of Fall, in this paper we endeavor to develop heuristic stability margins that approximate whether a state is in the Basin of Fall. These margins, such as the N-Step Capture Margin, give an indication of the ability of the biped to come to a stop within a given number of steps. Previous stability margins, such as the static stability margin, are similar heuristic approximations, but are typically too conservative. In the next section we discuss desirable characteristics for heuristic bipedal walking stability margins.

Velocity-Based Stability Margins for Fast Bipedal Walking

303

3 Desirable Characteristics of Stability Margins An ideal stability margin for a biped would act as a fortune teller. It would tell us when the biped is going to fall down next, what the cause will be, and how it can be prevented. If the biped is not going to fall down, the margin would indicate the closest the biped will be to falling down in the next step or so, at what point during the gait this occurs, and how much extra disturbance it could handle. While such omniscience is infeasible for anything but the simplest systems, some reasonable characteristics we may desire for stability margins include: • Necessary. If the stability margin is outside an acceptable threshold of values, the robot will fall down. • Suﬃcient. If the stability margin is inside the acceptable threshold of values, the robot will not fall down. • Comparable. Two control algorithms should be comparable for stability based on their relative stability margins. • Measurable and Computable. One should be able to measure the relevant state variables and estimate the stability margin on-line in order to use it for control purposes. • Meaningful. The stability margin should answer relevant questions as to why the robot fell. It should correlate with the degree of robustness to disturbances, such as noise, terrain irregularities, and external forces or impulses. The Viability Margin is necessary, suﬃcient, and allows comparisons. However, its main drawback is that it is very diﬃcult to compute. Various heuristic stability margins, which are much easier to compute, have been used in analyzing and controlling bipeds. In the next section we review some of the margins that are commonly used for bipedal walking and discuss how well they achieve these desirable characteristics. In the subsequent sections we introduce some heuristic stability margins which we believe more accurately measure the stability of a biped as deﬁned by Deﬁnition 3.

4 Review of Stability Margins for Bipedal Walking While there have been many proposed ways to deﬁne stability for a bipedal walking robot, we argue that many of these do not adequately address the desired characteristics described above. Here we review eigenvalues of Poincare return maps [14], the Zero Moment Point (ZMP) criterion [30], the foot rotation indicator (FRI) [7], and change of angular momentum [1, 20] as potential stability margins in light of the desired characteristics.

304

J.E. Pratt and R. Tedrake

4.1 Eigenvalues of Poincare Return Maps For a periodic system, trajectories in state space will cycle and return each cycle to a point in a slice of state space. Deviations from a periodic limit cycle will return to the cycle, and for small deviations, typically follow a linear relation, (2) Xn+1 = KXn , where X is the vector of deviations from the ﬁxed point that the limit cycle passes through and K is a linear return matrix. One of the eigenvalues of K, corresponding to the evolution of the orbit, will be 1.0. If the magnitude of the remaining eigenvalues of K are all less than one, then the limit cycle is stable. Thus, the magnitude of the largest eigenvalue of the return map, disregarding the eigenvalue corresponding to the orbit, is a suitable stability margin for a periodic system. Measuring the eigenvalues of Poincare return maps is commonly used for analyzing Passive Dynamic Walking robots [16, 8, 27, 2, 4, 29] and was used by Miura and Shimoyama [17] to analyze their Biper robots. However, using eigenvalues of Poincare return maps assumes periodicity and is valid only for small deviations from a limit cycle. While most bipedal systems, including humans, indeed seem to be periodic, there is nothing about the bipedal walking problem that requires periodicity. In particular, a biped’s motion is not periodic when walking over discontinuous rough terrain, or when abruptly changing speed or direction of travel. In addition, large disruptions from a limit cycle, such as when being pushed, cannot be analyzed using this technique as it assumes small deviations. Therefore, while eigenvalue magnitudes of Poincare return maps may be suﬃcient for analyzing periodic bipedal walking, they are not suﬃcient for analyzing nonperiodic motions, and are not necessary for analyzing bipedal walking in general. 4.2 Zero Moment Point (ZMP) and Foot Rotation Indicator (FRI) The Zero Moment Point (ZMP) is the location on the ground where the net moment generated from the ground reaction forces has zero moment about two axes that lie in the plane of the ground [30]. The ZMP when used in control algorithm synthesis for bipedal walking robots typically is computed analytically based upon desired trajectories of the robot’s joints. As long as the ZMP lies strictly inside the support polygon of the foot, then these desired trajectories are dynamically feasible. If the ZMP lies on the edge of the support polygon, then the trajectories may not be dynamically feasible. During playback of the desired joint trajectories, the actual ZMP is measured from force sensors in the foot or by observing accelerations of all the joints [10]. Then deviations between the precomputed and actual ZMP are typically used to modify the joint trajectories [34, 9]. The ZMP is equivalent to the Center of Pressure (CoP) but is commonly used to mean the analytically computed point based on the state and acceleration of the robot whereas

Velocity-Based Stability Margins for Fast Bipedal Walking

305

the CoP is commonly used to mean the point measured from ground reaction forces. The Foot Rotation Indicator (FRI) point [7] is the point on the ground where the net ground reaction force would have to act to keep the foot stationary given the state of the biped and the accelerations of its joints. If the foot is stationary, then the FRI, the ZMP, and the CoP are all the same point. If the foot is experiencing rotational acceleration, then the ZMP and CoP are on an edge of the support polygon, and the FRI is outside the support polygon. Therefore the FRI is a more general form of the ZMP and provides both a positive and negative margin when used for control and analysis purposes. The ZMP stability margin is the distance from the ZMP to the nearest edge of the convex hull of the support polygon. In its typical use it measures how much room for error there is in achieving the desired trajectories of the robot. If the ZMP reaches the edge of the support polygon, then the desired Center of Mass trajectory may no longer be dynamically feasible. Maintaining the ZMP inside the support polygon is not a necessary condition for stable walking. During toe-oﬀ in human walking and in walking of the robot Spring Flamingo [23], the ZMP stability criterion is violated, the FRI point lies outside the foot, and the foot rotates. For bipeds with point feet [11, 24, 5, 31], and Passive Dynamic Walkers with curved feet, when on one support foot, the ZMP, FRI, and CoP have little value as they are all simply the location of the foot, and the ZMP criterion is always violated. Maintaining the ZMP inside the support polygon is also not a suﬃcient condition for stable walking since a biped can fall down while its ZMP remains in the center of its foot. As an example, if all the joint torques are set to zero, the robot will collapse while the ZMP stays inside the foot. The ZMP criterion does become a suﬃcient criterion for stable walking when coupled with a particular class of control algorithms, typically those that rely on stiﬀ tracking of predetermined joint trajectories [13, 34, 9, 19]. One way to state the ZMP criterion is: Given desired state variable trajectories that are consistent with the dynamics and that predict the ZMP staying inside the support polygon, a trajectory tracking control algorithm can stably track those trajectories as long as the ZMP does indeed remain inside the support polygon. In these terms the ZMP criterion is a suﬃcient criterion for stable walking and has become a very powerful tool for trajectory generation and veriﬁcation of the dynamic feasibility of trajectories during execution. However, its utility has perhaps led to its overuse, resulting in the majority of bipedal robots relying heavily on prerecorded trajectories and stiﬀ joint control to achieve those trajectories. Such stiﬀ joint control of prerecorded trajectories typically leads to poor robustness to pushes and unknown rough terrain, relies on a ﬂatfooted gait, and makes it diﬃcult to incorporate natural dynamic mechanisms that have shown their utility in Passive Dynamic Walkers [16, 8, 27, 2, 4, 29], and a growing number of powered bipeds [22, 3, 33]. Since the ZMP criterion is a suﬃcient condition for stable walking only when coupled with a particular class of control system, as pointed out by

306

J.E. Pratt and R. Tedrake

Abdallah and Goswami [1] the ZMP criterion should be viewed as a controllability criterion that assesses the dynamic feasibility of a desired trajectory through state-space. It is important to note that the ZMP, CoP, and FRI are not functions of the state variables of the robot, but rather require knowledge of either the joint torques and/or the joint accelerations. Thus, they themselves cannot be viewed as state variables, and thus are not good candidates for stability margins that are a function of the state of the robot. Instead, for this paper and in previous work [23, 22], we view the Center of Pressure as a control input that can be arbitrarily and instantaneously changed, within the limits of actuator bandwidth, force limits, and the limitation that it stays inside the support polygon. 4.3 Angular Momentum Motivated by observations that humans appear to regulate angular momentum about the Center of Mass when standing, walking, and running, researchers [1, 20] have suggested that angular momentum about the Center of Mass (refered to as spin angular momentum) of a biped should be minimized throughout a motion. They argue that the spin angular momentum should be used when needed to balance, such as when lunging to prevent a fall after being pushed, or windmilling one’s arms when standing on a balance beam. The amount of angular momentum that is available is limited by joint angle limits, joint speed limits, and joint power limits. By minimizing angular momentum when not required, the biped has a reserve it can draw on when necessary. Minimizing spin angular momentum is not a necessary condition for stable walking. One can walk while violently thrashing his or her upper body mass around. It is not graceful, nor eﬃcient, and perhaps grace and eﬃciency are the primary reasons humans do not usually walk in this fashion. Minimizing spin angular momentum is also not a suﬃcient condition for stable walking, as a biped can fall over while maintaining an angular momentum of zero. Therefore, we believe angular momentum about the Center of Mass, taken purely on its own, is not a good indicator of whether a biped will fall and hence not a good stability margin by itself. However, the reserve in spin angular momentum that can be utilized to help recover from a push or other disturbance is important, as there is a coupling between angular momentum rate change and linear momentum rate change, and hence speed. This coupling arises from the fact that the net angular momentum rate change of the biped, about the Center of Pressure, is only modiﬁed by gravity (Fig. 1). Since the ground reaction force, by deﬁnition, acts through the Center of Pressure it does not aﬀect the angular momentum about this point. The angular momentum dynamics about the Center of Pressure can therefore be written as H˙ tot = mgl sin θ1 ,

(3)

Velocity-Based Stability Margins for Fast Bipedal Walking

307

Fig. 1. Bipedal model for sagittal plane dynamics. l is the virtual leg length from the Center of Pressure to the Center of Mass. θ1 is the angle from vertical to the virtual leg. F is the ground reaction force, with component Fl along the virtual leg and F⊥ perpendicular to it. m and Hcm are the mass and angular momentum about the Center of Mass of the biped. g is the gravitational acceleration constant

where Htot is the total angular momentum about the Center of Pressure, m is the mass of the robot, g is the gravitational acceleration, l is the distance from the Center of Pressure to the Center of Mass, and θ1 is the angle from the Center of Pressure to the Center of Mass with vertical being zero. The total momentum about the Center of Pressure consists of the angular momentum of the Center of Mass rotating about the Center of Pressure, plus the spin angular momentum about the Center of Mass: Htot = H0 + Hcm = ml2 θ˙1 + Hcm .

(4)

Diﬀerentiating, we get H˙ tot = mgl sin θ1 , = ml2 θ¨1 + 2mll˙θ˙1 + H˙ cm .

(5)

The ﬁrst term, ml2 θ¨1 is the acceleration of the Center of Mass penduluming around the Center of Pressure. The second term, 2mll˙θ˙1 encodes the coupling of distance to Center of Mass and rotational velocity (which makes a person rotate faster when they pull their legs in on a tire swing). Since only gravity has an inﬂuence on H˙ tot , equation 5 demonstrates the coupling between acceleration of spin angular momentum about the Center of Mass, H˙ cm , and acceleration of the Center of Mass about the Center of Pressure, θ¨1 . A clockwise acceleration of internal inertia will create a counterclockwise acceleration of the Center of Mass penduluming over the Center of Pressure

308

J.E. Pratt and R. Tedrake

and vice versa. That is what leads to the counterintuitive strategy of lunging your upper body forward to prevent falling forward. Since spin angular momentum is useful for catching balance in this way, having a reserve is useful. However, instead of using the reserve of spin angular momentum as a stability margin by itself, in Sect. 7 we propose using it as it relates to reducing the velocity of the biped. In the next section we justify our focus on velocity for analyzing bipedal stability.

5 Bipedal Walking and Stability There are ﬁve subtasks that a biped performs while walking. T1 T2 T3 T4 T5

Maintain body orientation within a reasonable bound. Maintain virtual leg length within a reasonable bound. Swing the swing leg. Transfer support from one support leg to the other. Regulate Center of Mass velocity.

For human-like bipedal walking, subtasks T1 through T3 are achievable through traditional control means as long as the support leg is ﬁrmly planted and joint torque limits that prevent foot slippage are observed. While there are occasions during the gait cycle where these tasks are not fully controllable, such as perhaps at the end of toe-oﬀ, we argue that there is plenty of opportunity during a normal bipedal walking gait to control orientation, virtual leg length, and leg swing using traditional techniques that depend on full controllability and full observability. Subtask T4, transfer of support, is diﬃcult when attempted using traditional high-gain joint position control techniques due to the over-constrained kinematics that the resultant closed loop kinematic structure presents [25]. However, when low impedance force control techniques are used [23], the system is no longer over-constrained and smooth transfer of support can occur relatively easily. We contend that the most diﬃcult subtask in bipedal walking from a controllability point of view is subtask T5, regulating the Center of Mass velocity. It is this subtask that makes bipedal walking both an interesting and challenging problem. It is the most diﬃcult subtask as the degrees of freedom that contribute to the velocity vector are under-actuated from a continuous dynamics point of view. Once the Center of Mass projection on the ground moves even a small distance out of the support polygon of the feet, then a signiﬁcant amount of angular acceleration of internal mass, such as windmilling the arms or lunging the upper body, must occur to bring the Center of Mass back. Beyond a certain distance, the Center of Mass cannot be brought back at all and the biped must take a step. Thus velocity can be regulated only through a combination of the continuous dynamics and the discrete dynamics.

Velocity-Based Stability Margins for Fast Bipedal Walking

309

It is this lack of actuation and the requirement of taking a step to continue walking that leads many to describe walking as a sequence of controlled falling. We can illustrate the diﬃculty of regulating velocity versus orientation and leg length by looking at the dynamics of walking during single support. To simplify the discussion, we consider only sagittal plane dynamics (Fig. 1), but our discussion extends to 3D dynamics. The dynamics of the rotation of the mass about the Center of Pressure (5) is H˙ tot = mgl sin θ1 = ml2 θ¨1 + 2mll˙θ˙1 + H˙ cm

(6)

where l is the virtual leg length from the Center of Pressure to the Center of Mass, θ1 is the angle from vertical to the virtual leg, m and Hcm are the mass and angular momentum about the Center of Mass of the biped, and g is the gravitational acceleration constant. As discussed in Sect. 4, humans tend to regulate their angular momentum about the Center of Mass, and thus the magnitude of H˙ cm is kept relatively low. Suppose H˙ cm = 0. Since the virtual leg length, l is always positive, then equation 6 can be rewritten as θ¨1 = −c1 l˙θ˙1 + c2 sin θ1

(7)

where c1 and c2 are always positive. If θ˙1 and θ1 have the same sign, then the magnitude of θ1 must always be increasing, assuming |l|˙ < ∞ and −π < θ1 < π. In physical terms, this means that if the Center of Mass is moving away from the Center of Pressure, it cannot be stopped without either moving the location of the Center of Pressure (instantaneoulsy changing the value of θ1 ), or by accelerating internal inertia. However, once the Center of Mass is beyond the support polygon of the foot, changing the Center of Pressure alone cannot stop further motion of the Center of Mass. At that point, only accelerating internal inertia (H˙ cm = 0), or taking a step can prevent a fall. Due to joint range of motion, velocity, and torque limits, the amount of opportunity for using angular momentum to catch balance is limited. Therefore, during a large portion of a human-like walking gait, the only way to prevent a fall is to take another step. Turning to the dynamics of the virtual leg length from the Center of Pressure to the Center of Mass we have 2 m¨l = mlθ˙1 − mg cos θ1 + Fl

(8)

where Fl is the component of the ground reaction force along the line from the Center of Pressure to the Center of Mass and is typically positive since the total ground reaction force must be positive and lie inside the friction cone. Fl is a function of the leg actuator forces and as long as the leg isn’t straight can be arbitrarily and instantaneously set to any positive value, assuming ideal force-source actuators, as long as it doesn’t result in slipping on the ground. 2 Since mg is typically larger than mlθ˙1 on the Earth at typical walking speeds,

310

J.E. Pratt and R. Tedrake

one can achieve a large range of negative ¨l, and an even larger range of positive ¨l, bounded only by the strength of the leg. Therefore, during a large percentage of the walking gait, the virtual leg length, l is fully controllable. Turning to the dynamics of the angular momentum about the Center of Mass we have H˙ cm = lF⊥ (9) where F⊥ is the component of the ground reaction force perpendicular to the line from the Center of Pressure to the Center of Mass. F⊥ is a function of the leg actuator forces and can be arbitrarily and instantaneously set to any value, assuming ideal force-source actuators, as long as it doesn’t result in slipping on the ground. Since a large range of positive and negative values of H˙ cm can be achieved during a large portion of the walking gait, angular momentum about the Center of Mass is fully controllable. Note that in this discussion we do not break out the components of Hcm due to their complexity, nor do we consider the motion of internal degrees of freedom. We assume that all the internal degrees of freedom have an actuator associated with them, and only consider their net eﬀect on the walking motion as it couples through Hcm . Achieving any orientation of internal body parts, or swinging the swing leg along a trajectory in internal coordinates is thus achievable with traditional control means. Note that in this discussion we use θ˙1 as our velocity variable (and could use θ˙2 for a 3D analysis). However, any two variables that are independent of virtual leg length velocity, l˙ could be used. Except for a fall, the virtual leg should never lie in the horizontal plane. Therefore, x˙ and y˙ would also be a suitable choice in the above analysis. As long as we choose a deﬁnition of a velocity vector that along with the virtual leg vector spans the 3D space, the above analysis can be modiﬁed to show that velocity is not fully controllable for a large percentage of a human-like walking gait. We have shown that virtual leg length and body orientation are fully controllable during a large portion of a natural gait, with the main limitation in their control being the requirement that the foot doesn’t slip on the ground. In contrast, velocity is not controllable with continuous dynamics once the Center of Mass has moved away from the support polygon. For human-like walking, velocity is controllable only through a combination of continuous dynamics and discrete dynamics (i.e. taking a step). This lack of controllability is why we consider velocity regulation the most challenging part of walking and why we focus on stability margins were velocity is central.

6 Capture Points and Capture Regions A key capability required for robust 3D walking is the ability to place the swing leg foot at an appropriate position, such that the Center of Mass can come to rest over the foot. We call such a point a “Capture Point”. Before deﬁning a Capture Point, we deﬁne a Capture State and a Safe Feasible Trajectory:

Velocity-Based Stability Margins for Fast Bipedal Walking

311

Deﬁnition 5 (Capture State). State in which the kinetic energy of the biped is zero and can remain zero with suitable joint torques. Deﬁnition 6 (Safe Feasible Trajectory). Trajectory through state space that is consistent with the robot’s dynamics, is achievable by the robot’s actuators, and does not contain any Falling States. Note that the Center of Mass must lie above the support polygon in a Capture State. We now deﬁne a Capture Point: Deﬁnition 7 (Capture Point). For a biped in state x, a Capture Point, p, is a point on the ground where if the biped covers p, either with its stance foot or by stepping to p in a single step, and then maintains its Center of Pressure to lie on p, then there exists a Safe Feasible Trajectory that ends in a Capture State. Note that since height, swing, orientation, and velocity are dynamically coupled, the location of a Capture Point is dependent on the trajectory through state-space before and after swinging the leg and thus is not a unique point. Therefore, there exists a Capture Region on the ground such that if the Center of Pressure is placed inside this region, then the biped can come to a stop for some height and orientation trajectory. Deﬁnition 8 (Capture Region). The set of all Capture Points. The size of the Capture Region is highly dependent on how fast the swing leg can swing to a Capture Point before the biped has accelerated and the point is no longer a Capture Point. The faster the swing leg, the larger the area of the Capture Region. Therefore, all else being equal, a biped with a faster swing leg will have a higher margin of stability than one with a slower swing leg. If constraints are placed on the subsequent height, swing, orientation, and internal motion of the robot, then the size of the Capture Region will be reduced. For example, we can deﬁne a momentum-regulated capture region as the set of all Capture Points that exist when H˙ cm is regulated during the subsequent motion. With enough constraints, the Capture Region may be reduced to a single point, or perhaps even vanish. Note that our deﬁnition of Capture Point also implies that the swing leg can reach the point. Let us deﬁne an Unreachable Capture Point as a point that is not a Capture Point, but would be if the swing leg did not have kinematic constraints. The Unreachable Capture Region is then the set of all Unreachable Capture Points and there is no intersection between the Unreachable and Reachable Capture Regions. While being able to stop in one step implies stability by deﬁnition 3, it is overconservative. Being able to stop in two steps is a less conservative estimate of stability and hence we consider Two-Step Capture Points.

312

J.E. Pratt and R. Tedrake

Deﬁnition 9 (Two-Step Capture Point). A point on the ground, p, such that if the biped swung its swing leg to cover p with its foot and maintained its Center of Pressure to lie on p, then there exists a Safe Feasible Trajectory, such that at some state along the trajectory, there exists a Capture Point. Deﬁnition 10 (Two-Step Capture Region). The set of all Two-Step Capture Points. We can now deﬁne an N-Step Capture Point recursively: Deﬁnition 11 (N-Step Capture Point). A point on the ground, p, such that if the biped swung its swing leg to cover p with its foot and maintained its Center of Pressure to lie on p, then there exists a Safe Feasible Trajectory, such that at some state along the trajectory, there exists an N-1-Step Capture Point. Deﬁnition 12 (N-Step Capture Region). The set of all N-Step Capture Points. If an N-Step Capture point exists, then we say that the biped is “N-Step Capturable”. As N approaches ∞, the N-step Capture Region converges to the area on the ground that the foot can be placed at without resulting in an eventual fall. Note that the above deﬁnitions require the maintenance of the Center of Pressure at a Capture Point, which is not necessary nor typical in walking. More general deﬁnitions would allow the Center of Pressure to move inside the foot. However, we conjecture that such deﬁnitions would result in equivalent Capture Regions, i.e. if a biped can stop by taking a step and then moving its Center of Pressure around its foot, then the biped can stop by taking the same step and maintaining the Center of Pressure at a single point inside the foot. We chose to maintain the Center of Pressure at a Capture Point in our deﬁnitions for computational reasons since doing so reduces the potential action space by two degrees-of-freedom. However, in controlling a biped, there is no such requirement. The stability margins we propose in the next Section will give a measure of how much opportunity there is to stop based on the ability to reach a Capture Region with the swing leg. Knowing where the leg can swing to will be important and so we deﬁne the Reachable Region as follows. Deﬁnition 13. Reachable Region: Region on the ground that can be reached kinematically by any point on the bottom of the swing foot.

7 Proposed Stability Margins We now propose some velocity-based stability margins for bipedal walking that are based on Capture Regions. Because the Capture and Reachable Regions depend on the current state of the robot, x, these margins are scalar functions of x.

Velocity-Based Stability Margins for Fast Bipedal Walking

313

• Zero-Step Capture Margin. M (x, 0): Maximum distance from points in the Capture Region to their closest edge of the support polygon if the Capture Region and support polygon intersect. Otherwise, the negative distance from the Capture Region to the support polygon. • One-Step Capture Margin. M (x, 1): Maximum distance from points in the Capture Region to their nearest boundary of the Reachable Region if the Capture Region is non-empty. Otherwise, the negative distance from the Unreachable Capture Region to the Reachable Region. • N-Step Capture Margin. M (x, N ): Maximum distance from points in the N-Step Capture Region to their nearest boundary of the Reachable Region if the N-Step Capture Region is non-empty. Otherwise, the negative distance from the Unreachable N-Step Capture Region to the Reachable Region. Note that the Zero-Step Capture Margin is equivalent to the traditional static stability margin when velocity is negligible and angular momentum is not used for capturing balance. For a moving biped, it is more appropriate than the traditional margin, since the Center of Pressure needs to be placed near the Capture Point rather than the ground projection of the Center of Mass to stop the biped. The above margins assume that any dynamically feasible trajectory through the state space is acceptable. Variants of the above margins can be deﬁned that impose constraints on the trajectories, control system, etc. For example, angular-momentum-regulated versions of these margins such as the AngularMomentum-Regulated N-Step Capture Margin, M (x, N, |H˙ cm | < H˙ cmmax ), can be deﬁned as above, but where the Capture Region is computed with the assumption that angular momentum is limited for capturing balance. These margins are useful for determining whether the robot is Capturable without having to rely on drastic measures such as lunging or windmilling. All of the above margins are in terms of how close the biped is to being able to stop. Determining if a biped is able to stop over a number of steps is appropriate, since if a biped cannot slow down and eventually stop, then its state is likely in the Basin of Fall, except for some metastable border states on the boundary of the Basin of Fall. We conjecture that in the limit as N approaches inﬁnity, N-Step Capturable implies stability as deﬁned by Deﬁnition 3. For most practical purposes, if the biped cannot stop in several steps, it is probably close enough to falling to consider it unstable. Therefore, though still not a completely necessary criterion, being say 10-step Capturable should be a suitable stability criterion. The above stability margins are all in terms of Cartesian distance of various regions on the ground to the Reachable Region. This distance is related to the ability to get the swing leg to the region and also the time before the region moves and is no longer reachable if it isn’t stepped to rapidly. One could also deﬁne other margins in terms of the area of a given region. These

314

J.E. Pratt and R. Tedrake

margin would give an indication of the degree of accuracy required in placing the swing leg in the region. For example, • N-Step Area-Based Capture Margin. Marea (x, N ): Area of the NStep Capture Region if the N-Step Capture Region is non-empty. Otherwise, the negative distance from the Unreachable N-Step Capture Region to the Reachable Region. We have now suggested a number of potential stability margins that can be used for analyzing and controlling bipedal walking. In terms of the desirable characteristics listed in Sect. 3, the margins listed above are suﬃcient conditions for stability, allow comparisons between diﬀerent walking algorithms, and are meaningful. While none of them, except for boundary cases requiring extreme computationally complexity, are tight necessary conditions, they are tighter necessary conditions than previously proposed stability margins. These margins are measurable and computable. For N = 1, estimates with low computational complexity are easy to derive, as described in the next section. As N increases, the computational requirements likely increase, but may be reasonable for small N . Deciding which margin is the most desirable and what degree of stability is required depends on tradeoﬀs between performance and safety. For example, if the biped is walking carefully over stepping stones, then having a large One-Step Area-Based Capture Margin may be desirable. If the biped is a gymnast on a balance beam, then being Angular-Momentum-Regulated OneStep Capturable is important since lunging or windmilling to prevent a fall results in deductions. And if the biped is an Olympic Race Walker, being able to stop in a small number of steps is less important than speed, so a small degree of ∞-Step Capture Margin is all that may be required. In the next section we will discuss methods for estimating Capture Point locations and some of these stability margins.

8 Estimating Stability Margins In this section we describe some methods for estimating our proposed stability margins. 8.1 Estimating One-Step Capture Points We can derive estimates of the location of Capture Points by using inverted pendulum models that approximate walking. Using a constant length inverted pendulum model (Fig. 2, left side) and equating initial and ﬁnal energy, we have 12 mv 2 + mgh0 = mgl, where m is the mass, v is the velocity of the mass, g is the gravitational constant, h0 is the initial height above the ground, and l is the virtual leg length. Let sˆ be the unit vector parallel to the ground that

Velocity-Based Stability Margins for Fast Bipedal Walking

315

Fig. 2. Simple models for estimating the location of Capture Points

points in the same direction as the Center of Mass velocity. The Capture Point will lie somewhere on the line deﬁned by the projection of the Center of Mass on the ground and sˆ: xc = rc sˆ, where xc is the location of the Capture Point with respect to the ground projection of the Center of Mass and rc is the distance from the Center of Mass projection to the 2 Capture Point. Because rc , l, and h0 form a right triangle, we have rc = l2 − h20 . Solving for rc we get 0 h0 v2 rc = v + 2 (10) g 4g This model assumes that the leg length stays constant as the Center of Mass follows an arc, coming to rest above the Capture Point. Another model, referred to as the Linear Inverted Pendulum model [11, 12], assumes that the Center of Mass height stays constant (Fig. 2 – right side). Using this method to compute the location of a Capture Point results in an even simpler equation. In order to counter gravity, the vertical force on the mass must be mg. Because ground reaction forces can only act on the line between the Center of Pressure and the Center of Mass, similar triangles are formed and we have x mg Fx = =⇒ Fx = x mg h0 h0

(11)

where Fx is the horizontal force on the mass, and x is the distance from the mass to the Capture Point. Because the mass moves at a constant height, the energy absorbed while moving above the Capture Point will be the integral of the force times the displacement: rc E= 0

mg F dx = h0

rc x dx =

mg 2 r 2h0 c

(12)

0

Equating initial and ﬁnal energies, we have 12 mv 2 = get 0 h0 rc = v g

mg 2 2h0 rc .

Solving for rc we (13)

316

J.E. Pratt and R. Tedrake

Equations (10) and (13) give estimates for two diﬀerent Capture Points, one based on the Center of Mass following an arc, and one based on it following a straight horizontal line. Because the Center of Mass typically follows a ﬂattened arc in bipedal walking, a more relevant Capture Point for a typical gait will likely fall between those two bounds. For parameter values typical in human walking, these two bounds are within 15% of each other. The above estimates assumed that the swing leg could instantaneously arrive at the Capture Point. Since it may take some time for the swing leg to get to the Capture Point, we should estimate where the Capture Point will be when the swing leg arrives. If we have an estimate on the time remaining for swing, we can estimate the predicted Capture Point location using the Linear Inverted Pendulum model. Figure 3 illustrates the key variables in estimating the predicted Capture Point location. First we estimate the Center of Mass ¨ = hg0 x. Since trajectory. From the previous analysis, Fx = mg h0 x. Therefore, x this is a linear equation, it can be solved in closed form [11, 12] x = C0 ewt + C1 e−wt , x˙ = D0 ewt + D1 e−wt 1 where w =

g h0

(14)

and 1 v0 1 v0 x0 + , C1 = x0 − , 2 w 2 w 1 1 D0 = (wx0 + v0 ) , D1 = (−wx0 + v0 ) 2 2 C0 =

(15)

The equations for y are identical, with the proper substitutions. Given the estimated swing time, we can estimate the location and velocity of the Center of Mass at the end of swing using Equation (15). Using Equation (13) we then can estimate the location of the Capture Point at the end of swing. As the leg is swinging, we can update the predicted Capture Point location and adjust the swing leg trajectory to land in the desired location. The above estimates of Capture Points were for point mass models. We can estimate a Capture Region using the above equations to compute the center of the region and then models for using angular momentum to determine a region around the center. Suppose we have no limit in how fast we can change our angular momentum (H˙ cm is unbounded), but we do have a limit on the angular momentum about the Center of Mass that we have in “reserve”, Hreserve . Also suppose we have a limit on the duration, τreserve that the biped can achieve that angular momentum before running into joint limits and having to “pay it back” by stopping the rotation. We conjecture that once it has been determined to use reserve angular momentum to catch balance, immediately using all that is available as rapidly as possible is the strategy that results in the recovery from the worst conditions. If we assume that the change in angular momentum is achieved instantaneously through an impulsive torque, then the change in rotational velocity

Velocity-Based Stability Margins for Fast Bipedal Walking

317

Fig. 3. Evolution of the Center of Mass and a Capture Point from time t to t + ∆t. xcop (t) is the location of the Center of Pressure; xcom (t) and vcom (t) are the location and velocity of the Center of Mass; and xc (t) is the location of the Capture Point

of the Center of Mass about the Center of Pressure can be determined by integrating Equation (6) for an inﬁnitesimal amount of time. The result is ∆θ˙1 = −

1 Hreserve ml2

(16)

Given the new velocity, we can estimate the Center of Mass trajectory during the next τreserve seconds using Equation (14). At the end of this time, the angular momentum must be “paid back” before joint angle limits are violated. We can then repeat the above steps to estimate the ﬁnal Center of Mass position and velocity. To estimate the Capture Region, one can use a search, guessing points on the boundary and checking with the above procedure to see if the Center of Mass can be captured over that point with the reserves of angular momentum. After several points on the boundary are determined, then the Capture Region can be estimated with a best-ﬁt curve. 8.2 Estimating N-Step Capture Points Estimating N-Step Capture Points can be achieved through a brute-force search algorithm. Even with a large state space, the search can be relatively fast if N is small and a discrete step-to-step transition function, S is known, xn+1 = S(xn , xstep , P)

(17)

where xn is the state of the step n, xstep is the location of the point that the biped steps to, and P are parameters that govern the motion of the step. The

318

J.E. Pratt and R. Tedrake

Linear Inverted Pendulum analysis described previously provides one such example of a discrete step-to-step transition function, with the time of step being the only input parameter. Since it is likely that, on ﬂat ground, an N-Step Capture Region is a connected, and perhaps even convex area on the ground, a search for the boundary of the Capture Region can be performed when considering the ﬁrst xstep . After the ﬁrst step, then steps should only be considered that are likely optimal for stopping as quickly as possible. If a One-Step Capture Point exists, then the step should be there. If not, then the step should be as quickly as possible as far as possible toward the Unreachable Capture Region. 8.3 Quick Estimates on the Number of Steps Required for a Stop We can estimate the approximate number of steps required for a biped to stop by estimating how much the biped can slow down each step using the Linear Inverted Pendulum model. The amount of energy absorbed during mg rstep and the amount of energy returned during acceleration deceleration is 2h 0 mg is 2h0 raccel where rstep is the step length from Center of Mass projection on the ground to the Center of Pressure, and raccel is the distance from the trailing support leg’s Center of Pressure to the Center of Mass projection on the ground. If swing can happen quickly enough, raccel can be zero. However, when the swing leg takes long enough to swing that exchange of support cannot be achieved before the Center of Mass passes over the trailing Center of Pressure, then raccel = 0. raccel can be estimated with (14), given the required swing time. With the net change in energy we can then determine the velocity on the next step, 2 = vn2 − vn+1

g 2 g r + r2 h0 step h0 accel

(18)

We can iterate on Equation (18) to estimate the number of steps required to stop the biped. Note that a biped will always be able to slow down if the swing leg is fast enough to have a non-symmetric stance in which the Center of Pressure is in front of the Center of Mass more than it is behind it (rstep > raccel ). When the swing leg takes too long to swing (rstep < raccel ) the biped necessarily continues to speed up and eventually falls down. If raccel = 0, then the number of steps required to stop is N = grh20 v 2 . If step

raccel = 0 then N >

h0 v2 . 2 grstep

For typical human parameters at fast walking

speeds, grh20 v 2 ≈ 1 meaning a human at top speed usually requires 1 or 2 step steps to come to a stop but typically not more. This estimate is easily veriﬁed through simple experiments in which one person walks fast while another calls out “stop” at a random time. Note that a biped with its mass concentrated in its body, and no limit to the torques at its joint, should be able to walk extremely fast along a straight line trajectory. The only limit is how quickly it can swing its leg

Velocity-Based Stability Margins for Fast Bipedal Walking

319

and transfer support. But with massless legs, it should be able to swing and transfer instantaneously. Along a ﬂattened arc trajectory, the speed would be limited by the requirement of the centripetal acceleration of the Center of Mass being less than the acceleration of gravity. At higher velocities, the ground reaction force falls below zero and the biped becomes airborne. For humans walking at Earth’s gravity, this limitation is not the limiting factor in top speed [22]. However, at the gravity of the moon, this limitation is relevant and explains why astronauts preferred to hop on the moon rather than walk.

9 Standing Balance Strategies Using our proposed stability deﬁnitions we can now propose an algorithm for selecting an appropriate strategy to regain balance after being disturbed when standing. The algorithm below is for the reﬂex phase of balance recovery [1]. Once the robots velocity is captured, than a recovery phase can begin to restore the posture of the robot. 1. if Momentum-Regulated Zero-Step Capturable, then use the Center of Pressure to maintain balance. 2. else if Zero-Step Capturable, then use angular momentum to capture balance by lunging or windmilling. 3. else if Momentum-Regulated One-Step Capturable, then take a step to a Momentum-Regulated Capture Point and use the Center of Pressure to maintain balance after the step. 4. else if One-Step Capturable, then take a step to a Capture Point and use angular momentum to capture balance after the step. 5. else if N-Step Capturable, take a sequence of steps to capture balance. 6. else take a running step and/or fall. The ﬁrst two strategies have been well documented with human walking and are often refered to as the “ankle strategy” and the “hip strategy” [15]. We prefer to call them the “Center of Pressure strategy” and the “Angular Momentum Strategy”. All of the strategies are easily observable in easily performable balance experiments. Using the stability margins proposed in this paper, it should be possible to hypothesize what amount of disturbance is required to cause a human to switch from one strategy to another one.

10 Control Algorithms 10.1 Control Algorithms for Push Recovery The balance strategies listed above can be easily implemented in a control algorithm for push recovery. For such an algorithm, it is not important to accurately compute the various stability margins. What is more important is

320

J.E. Pratt and R. Tedrake

to rapidly determine which balance recovery method to attempt. If a step is required, reaction speed is critical and therefore, determining a good place to step is more appropriate than deliberating on the optimal place to step. In a simulation study, we developed an algorithm that attempts to stay balanced on one foot. The simulated robot uses modulation of the Center of Pressure in order to keep the Capture Point, which is estimated every control cycle using the Linear Inverted Pendulum model, inside the foot. The commanded Center of Pressure location, xCoP is computed using a linear controller on the desired Capture Point location, xcapture , xCoP = xcapture + k(xcapture − xcapture )

(19)

where k is the controller gain parameter. If the computed Center of Pressure location falls outside the foot, then it is changed to the nearest point on the foot’s edge that lies on the line from the desired Capture Point to the actual Capture Point. When a signiﬁcant push occurs, the Capture Point moves outside of the foot. The biped approximates how long it will take to swing the leg to the point and uses that time in the estimate of where the Capture Point will be after swing. If this Capture Point is reachable, then the biped moves its Center of Pressure to the point on the foot nearest the Capture Point to minimize further acceleration and then takes a step to the Capture Point to regain balance. If this Capture Point is unreachable, then the algorithm assumes that if a 2-Step or N-Step Capture Region does exist, that it is near the boundary of the Reachable Region nearest the Unreachable Capture Point. Therefore, the biped will take a step as far as possible towards the Capture Point and then determine a new Capture Point to attempt to step to. This algorithm works fairly well, validating the Capture Point estimates. When the robot is pushed in such a way that a Capture Point is reachable, the robot does regain balance on that step most of the time. This indicates that even if the estimated Capture Point is not a Capture Point, its margin of error is within the size of the foot. 10.2 Control Algorithms for the Stepping Stones Problem Now consider the problem of taking steps to desired locations. We refere to this problem as the One-Step Stepping Stones problem, since it is similar to crossing a pond over stones considering only one step at a time. In a simulation study, we developed an algorithm for one-step stepping stones that uses the Center of Pressure on the support foot to guide the Capture Point out of the foot toward the desired stepping location when told to take a step. The swing leg then swings to the desired stepping location and is loaded once the estimated Capture Point becomes coincident with the desired step location. Figure 4 shows time elapsed snapshots of the simulated biped using this algorithm. Note that since the robot steps to a Capture Point,

Velocity-Based Stability Margins for Fast Bipedal Walking

321

Fig. 4. Time elaspsed snapshots of simulated 12 degree-of-freedom lower-body biped walking over discrete steps while remaining One-Step Capturable. Snapshots are taken at one second intervals

at all times it is One-Step Capturable. If the robot is not told to step to the next desired location, it will stop and balance on one leg. Since the desired footholds are staggered, the motion is not periodic and hence Poincare return maps could not be used to demonstrate stability of this simulation. Since there is a toe-oﬀ phase, the ZMP criterion is violated on every step and is therefore not applicable to this simulation. Even though the robot is One-Step Capturable throughout the simulation, the motion is dynamic, fairly natural-looking, moderate speed, and includes toe-oﬀ and double-support phases. If the robot is told the next step in advance, it does not need to stop over the current step and can continue at a moderate velocity. Therefore, being One-Step Capturable does not imply stopping or even imply slow walking. In this example the small performance degradation is well worth it in order to get the safety margin that being One-Step Capturable provides. In current work we are investigating how speed can be increased by looking two or three steps ahead and guiding the capture point on a path that curves by the ﬁrst stepping stone and then on to the second stepping stone as the ﬁrst step is being taken.

11 Discussion and Future Work 11.1 Probabilistic Stability Margins In this paper, we have deﬁned stability assuming a deterministic system. However, bipeds should be considered nondeterministic, since ground variations, sensor noise, and external disturbances are impossible to precisely model. Most stability margins handle nondeterminism by relating to the tolerance to a particular unknown disturbance. This is the case for phase margins and gain margins in traditional linear control and is the case for many margins for

322

J.E. Pratt and R. Tedrake

bipedal walking, such as the static stability margin and the margins introduced in this paper. These margins typically give comparative indications of robustness to terrain, noise, and disturbances (the larger the margin, the greater the disturbance that can be tolerated). They sometimes are an indication of the magnitude of the largest single disturbance that can be tolerated. However, they usually do not indicate the probability of instability given a particular disturbance distribution. In a companion paper [28], we explore stochastic stability margins for legged locomotion. 11.2 Proofs of Conjectures In this paper we have stated several conjectures without proof. While these conjectures seem logical, it would be very useful to prove them since they are relied on for both the theory and the computational implementation in this paper. These conjectures are: • In the limit as N approaches inﬁnity, N-Step Capturable implies stability as deﬁned by Deﬁnition 3. • The Capture Region remains the same whether or not the constraint is enforced that the Center of Pressure remains at a Capture Point during a step. • For ﬂat terrain, the Capture Region will be a connected, and perhaps even convex, area on the ground. • Once it has been determined to use reserve angular momentum to catch balance, immediately using all that is available as rapidly as possible is the strategy that results in the recovery from the worst conditions. 11.3 Generalization In this paper we considered bipedal walking only. For a multi-legged robot or animal, similar deﬁnitions should apply. The diﬀerence would be that instead of requiring the foot to cover a Capture Point, the resultant convex hull of the support polygon would need to cover a Capture Point. For running, the diﬀerence would be in the Reachable Region of the swing foot. The Reachable Region would be larger, and in addition to being kinematically constrained, it would also be constrained by the leg strength and take-oﬀ velocities.

References [1] Muhammad Abdallah and Ambarish Goswami. A biomechanically motivated two-phase strategy for biped upright balance control. IEEE International Conference on Robotics and Automation (ICRA), 2005. [2] Michael J. Coleman. A Stability-Study of a Three-Dimensional PassiveDynamic Model of Human Gait. PhD thesis, Cornell University, 1998.

Velocity-Based Stability Margins for Fast Bipedal Walking

323

[3] Steven H. Collins, Andy Ruina, Russ Tedrake, and Martijn Wisse. Eﬃcient bipedal robots based on passive-dynamic walkers. Science, 307:1082–1085, February 18 2005. [4] Steven H. Collins, Martijn Wisse, and Andy Ruina. A three-dimensional passive-dynamic walking robot with two legs and knees. International Journal of Robotics Research, 20(7):607–615, July 2001. [5] E.R. Dunn and R.D. Howe. Foot placement and velocity control in smooth bipedal walking. pages 578–583. IEEE International Conference on Robotics and Automation (ICRA), 1996. [6] Elena Garcia, Joaquin Estremera, and Pablo Gonzalez de santos. A classiﬁcation of stability margins for walking robots. In Proceedings of the International Conference on Climbing and Walking Robots (CLAWAR), 2002. [7] A. Goswami. Postural stability of biped robots and the foot rotation indicator (FRI) point. International Journal of Robotics Research, 18(6), 1999. [8] A. Goswami, B. Espiau, and A. Keramane. Limit cycles and their stability in a passive bipedal gait. pages 246–251. IEEE International Conference on Robotics and Automation (ICRA), 1996. [9] K. Hirai, M. Hirose, Y. Haikawa, and T. Takenaka. The development of Honda humanoid robot. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pp. 1321–1326, 1998. [10] Qiang Huang, Kazuhito Yokoi, Shuuji Kajita, Kenji Kaneko, Hirohiko Arai, Noriho Koyachi, and Kazuou Tanie. Planning walking patterns for a biped robot. IEEE Transactions on Robotics and Automation, 17(3):280–289, June 2001. [11] S. Kajita and K. Tani. Study of dynamic biped locomotion on rugged terrainderivation and application of the linear inverted pendulum mode. volume 2, pp. 1405–1411. IEEE International Conference on Robotics and Automation (ICRA), 1991. [12] Shuuji Kajita, Fumio Kanehiro, Kenji Kaneko, Kazuhito Yokoi, and Hirohisa Hirukawa. The 3d linear inverted pendulum mode: a simple modeling for a biped walking pattern generation. pages 239–246. IEEE International Conference on Intelligent Robots and Systems (IROS), 2001. [13] T. Kato, A. Takanishi, H. Jishikawa, and I. Kato. The realization of the quasidynamic walking by the biped walking machine. In A. Morecki, G. Bianchi, and K. Kedzior, editors, Fourth Symposium on Theory and Practice of Walking Robots, pp. 341–351. Warsaw: Polish Scientiﬁc Publishers, 1983. [14] Hassan K. Khalil. Nonlinear Systems. Prentice Hall, 3rd edition, December 2001. [15] A.D. Kuo and F. E. Zajac. Human standing posture: multijoint movement strategies based on biomechanical constraints. Progress in Brain Research, 97:349–358, 1993. [16] Tad McGeer. Passive walking with knees. pages 1640–1645. IEEE International Conference on Robotics and Automation (ICRA), 1990. [17] H. Miura and I. Shimoyama. Dynamic walk of a biped. International Journal of Robotics Research, 3(2):60–74, 1984. [18] Jun Morimoto and Christopher Atkeson. Minimax diﬀerential dynamic programming: An application to robust biped walking. Advances in Neural Information Processing Systems, 2002.

324

J.E. Pratt and R. Tedrake

[19] Hun ok Lim and Atsuo Takanishi. Waseda biped humanoid robots realizing human-like motion. pages 525–530. IEEE International Workshop on Advanced Motion Control, 2000. [20] Marko Popovic, Amy Englehart, and Hugh Herr. Angular momentum primitives for human walking: Biomechanics and control. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, 2004. [21] Marko Popovic, Ambarish Goswami, and Hugh Herr. Ground reference points in legged locomotion: Deﬁnitions, biological trajectories and control implications. International Journal of Robotics Research, 24(12):1013–1032, Dec 2005. [22] Jerry Pratt. Exploiting Inherent Robustness and Natural Dynamics in the Control of Bipedal Walking Robots. PhD thesis, Computer Science Department, Massachusetts Institute of Technology, 2000. [23] Jerry Pratt and Gill Pratt. Intuitive control of a planar bipedal walking robot. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), 1998. [24] Jerry Pratt, A. Torres, Peter Dilworth, and Gill Pratt. Virtual actuator control. IEEE International Conference on Intelligent Robots and Systems, 1996. [25] C.L. Shih and W.A. Gruver. Control of a biped robot in the double-support phase. IEEE Transactions on Systems, Man and Cybernetics, 22(4):729–735, 1992. [26] Michael Sipser. Introduction to the Theory of Computation. Course Technology, second edition, 2005. [27] Adam C. Smith and Matthew D. Berkemeier. Passive dynamic quadrupedal walking. In Proceedings of the 1997 IEEE Conference on Robotics and Automation, vol. 1, pp. 34–39. IEEE, April 1997. [28] Russ Tedrake and Jerry E. Pratt. Probabilistic stability in legged systems and the mean ﬁrst passage time (FPT) stability margin. In progress, 2005. [29] Russ Tedrake, Teresa Weirui Zhang, Ming-fai Fong, and H. Sebastian Seung. Actuating a simple 3D passive dynamic walker. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), vol. 5, pp. 4656– 4661, New Orleans, LA, April 2004. [30] Miomir Vukobratovic and Branislav Borovac. Zero-moment point – thirty ﬁve years of its life. International Journal of Humanoid Robotics, 1(1):157–173, 2004. [31] E. R. Westervelt, J. W. Grizzle, and D. E. Koditschek. Zero dynamics of underactuated planar biped walkers. IFAC-2002, Barcelona, Spain, pp. 1–6, Jul 2002. [32] Pierre-Brice Wieber. On the stability of walking systems. In Proceedings of the International Workshop on Humanoid and Human Friendly Robots, 2002. [33] Martijn Wisse. Essentials of dynamic walking; Analysis and design of two-legged robots. PhD thesis, Technische Universiteit Delft, 2004. [34] J. Yamaguchi, N. Kinoshita, A. Takanishi, and I. Kato. Development of a dynamic biped walking system for humanoid – development of a biped walking robot adapting to the humans’ living ﬂoor. pages 232–239. IEEE International Conference on Robotics and Automation (ICRA), 1996.

Nonlinear Model Predictive Control and Sum of Squares Techniques T. Raﬀ, C. Ebenbauer, R. Findeisen, and F. Allg¨ ower Institute for Systems Theory and Automatic Control, University of Stuttgart, Germany {raff,ce,findeise,allgower}@ist.uni-stuttgart.de Summary. The paper considers the use of sum of squares techniques in nonlinear model predictive control. To be more precise, sum of squares techniques are used to solve at each sampling instant a ﬁnite horizon optimal control problem which arises in nonlinear model predictive control for discrete time polynomial systems. The combination of nonlinear model predictive control and sum of squares techniques is motivated by the successful application of semideﬁnite programming in linear model predictive control. The advantages and disadvantages of applying sum of squares techniques to nonlinear model predictive control are illustrated on a small example.

1 Introduction Model predictive control (MPC) is a popular control strategy, especially suitable for the control of multivariable systems subject to input and state constraints [1, 26, 16]. The basic idea of model predictive control is to determine the control input by repeatedly solving at each time instant a ﬁnite horizon optimal control problem. In general one distinguishes between linear and nonlinear model predictive control. In linear model predictive control, the optimal control problem is usually solved for a linear system subject to linear input and state constraints and a quadratic performance index. In contrast to linear model predictive control, nonlinear model predictive control (NMPC) allows the consideration of a nonlinear system, possible nonlinear input and state constraints, and a possible nonquadratic performance index in the optimal control problem setup. Model predictive control, especially linear model predictive control, is widely used in industry [24]. One reason why linear model predictive control is more successful than nonlinear model predictive control is that the optimal control problem with linear constraints and a quadratic performance index is a convex quadratic optimization problem for discrete time linear systems. The key advantage in this case is that any local minimum is also a global minimum and that eﬃcient numerical algorithms exist

326

T. Raﬀ et al.

which guarantee to converge to the global minimum [5]. In nonlinear model predictive control, the ﬁnite horizon optimal control problem is in general a nonconvex optimization problem. Nonconvex optimization problems are often hard to solve and rather a local minimum than a global minimum is obtained [16]. The purpose of this paper is to solve the nonlinear and in general nonconvex optimization problem that stems from the ﬁnite horizon optimal control problem in nonlinear model predictive control for discrete time polynomial systems via sum of squares techniques [15, 20, 25]. Sum of squares techniques are based on the sum of squares decomposition which is an eﬃcient method to check if a multivariate polynomial can be decomposed into a sum of squares since it can be solved via semideﬁnite programming [5]. The advantages and disadvantages of this combination are discussed on a small example. Furthermore, an introductory review of nonlinear model predictive control, sum of squares techniques, and polynomial optimization is given and an extension of the results presented in [25] to the class of implicit discrete time polynomial systems is introduced. The remainder of the paper is organized as follows: In Sect. 2, nonlinear model predictive control is reviewed and in Sect. 3 an overview of the sum of squares decomposition of multivariate polynomials and polynomial optimization is given. In Sect. 4, polynomial optimization is applied to nonlinear model predictive control for discrete time polynomial systems. Finally, conclusions are stated in Sect. 5. 1.1 Notation A polynomial p in x-= [x1 , . . . , xn ] is a ﬁnite linear combination of monoα1 α αn c x = mials, i.e., p(x) = α α α cα x1 ...xn , where cα -n∈ R and α = [α1 , . . . , αn ], αi ∈ N0 . The degree of p is deﬁned as d = i=1 αi . The set of all polynomials in x = [x1 , . . . , xn ] with real coeﬃcients is written as R[x]. A polynomial vector ﬁeld f : Rn → Rn , f (x) = [f1 (x), . . . , fn (x)]T is a vector ﬁeld with fi (x) ∈ R[x]. A polynomial p is called positive deﬁnite, if p(0) = 0 and p(x) > 0, ∀x ∈ Rn \ {0} and positive semideﬁnite if p(x) ≥ 0, ∀x ∈ Rn .

2 Nonlinear Model Predictive Control In this part of the paper, the basic idea and the mathematical setup of nonlinear model predictive control is introduced in Sect. 2.1 and Sect. 2.2 respectively. Furthermore, Sect. 2.3 provides a review of nonlinear model predictive control schemes that asymptotically stabilize nonlinear systems subject to input and state constraints.

Model Predictive Control and Sum of Squares

327

2.1 Basic Idea of Model Predictive Control The motivation of model predictive control is to stabilize a system by solving the inﬁnite horizon optimal control problem min u · |k s.t.

∞

F (xi|k , ui|k )

(1a)

i=k

xi+1|k = f (xi|k , ui|k ), ui|k ∈ U, xi|k ∈ X .

xk|k = xk ,

(1b) (1c)

In the inﬁnite horizon optimal control problem, a system (1b) with the initial condition xk is asymptotically stabilized while a performance index (1a) subject to input and state constraints (1c) is minimized. The performance index (1a) often stems from economical and ecological considerations of the considered control problem. However, problem (1) is in general very diﬃcult to solve. For unconstrained linear systems with a quadratic performance index, the closed solution of the inﬁnite horizon optimal control problem can be determined. In this case, the solution of (1) is the well-known linear quadratic regulator. One possibility to obtain a computationally tractable approximation of (1), i.e., for constrained nonlinear systems with a nonquadratic performance index, is to consider the ﬁnite horizon optimal control problem min u · |k s.t.

k+N −1

F (xi|k , ui|k )

(2a)

i=k

xi+1|k = f (xi|k , ui|k ), ui|k ∈ U, xi|k ∈ X ,

xk|k = xk ,

(2b) (2c)

where N is the so called prediction horizon. The basic idea of model predictive control is the following one: At each time instant k, the ﬁnite horizon optimal control problem (2) is solved on-line with the current measurement of the system state xk as an initial condition. To obtain a feedback, only the ﬁrst control action of the optimal control sequence obtained from the ﬁnite horizon optimal control problem (2) is applied to the system (2b). These steps, i.e., solving the optimal control problem (2), applying the ﬁrst control action to the system, and updating the initial condition by the current measurement of the system state, are repeated continuously in model predictive control. Due to online optimization in model predictive control, this control strategy is suitable to control linear as well as nonlinear multivariable systems with input and state constraints. However, the repeated solution of the ﬁnite horizon optimal control problem (2) leads to stability problems of the closed loop system [3]. Therefore, diﬀerent approaches have been developed to achieve closed loop stability. An introductory overview of model predictive control schemes with guaranteed closed loop stability is given in the next sections.

328

T. Raﬀ et al.

2.2 Setup of Nonlinear Model Predictive Control Before discussing diﬀerent nonlinear model predictive control schemes with guaranteed closed loop stability, the mathematical setup is introduced. Consider the stabilization problem for nonlinear discrete time systems described by the diﬀerence equations xk+1 = f (xk , uk )

(3)

subject to the input and state constraints uk ∈ U, xk ∈ X ,

(4)

where xk ∈ Rn is the state and uk ∈ Rm the control input of the system. Furthermore, assume that the equilibrium point to be stabilized is at the origin, i.e., the vector ﬁeld f satisﬁes f (0, 0) = 0. The set X is a closed subset of Rn and the set U a compact subset of Rm , both containing the origin. Suppose that the full state xk is available at time instant k. Then the ﬁnite horizon optimal control problem that has to be solved at each time instant k in nonlinear model predictive control is given by min ν

s.t.

E(xk+N |k ) +

k+N −1

F (xi|k , ui|k )

i=k

xi+1|k = f (xi|k , ui|k ), ui|k ∈ U, xi|k ∈ X ,

xk|k = xk

i ∈ [k, k + N − 1] i ∈ [k, k + N − 1]

(5)

xk+N |k ∈ E , where J(ν, xk ) = E(xk+N |k ) +

k+N −1

F (xi|k , ui|k )

(6)

i=k

is the cost function, ν the control sequence ν = [uk|k , . . . , uk+N −1|k ]T , N the length of the prediction and the control horizon, and xi|k the predicted system state at time instant i that is obtained by applying the control input sequence uk|k , . . . , uk+i−1|k to the system (3) from the initial condition xk . Furthermore, the stage cost F : Rn × Rm → R is a positive deﬁnite function and satisﬁes F (0, 0) = 0. The terminal cost E : Rn → R satisﬁes E(0) = 0 and E is the terminal region. The optimal solution to the ﬁnite horizon optimal control problem (5) is denoted by ν = [uk|k , . . . , uk+N −1|k ]T . The actual control action of the nonlinear model predictive controller is the ﬁrst element of ν , i.e., uk = uk|k . The ﬁnite horizon optimal control problem (5) diﬀers from the optimal control problem (2) because of the terminal cost E and the terminal region E. In the next section a summary is given on how to choose the terminal cost E and the terminal region E in order to achieve closed loop stability.

Model Predictive Control and Sum of Squares

329

2.3 Stability of Nonlinear Model Predictive Control A wide variety of approaches have been developed to achieve asymptotic closed loop stability [1, 9, 16]. All these approaches are based, implicitly or explicitly, on three ingredients: A terminal cost E, a terminal region E, and a locally stabilizing control law uk = ϕ(xk ). Many of these approaches are covered by the following theorem [16]: Theorem 1. The closed loop system is asymptotically stable if the optimal control problem (5) is feasible at the ﬁrst time instant and the following assumptions are satisﬁed for a terminal cost E, a terminal region E, and a locally stabilizing control law uk = ϕ(xk ): [A1] [A2] [A3] [A4] [A5]

E(xk ) > 0, ∀xk ∈ Rn \ {0} E ⊆ X, 0 ∈ E ϕ(xk ) ∈ U, ∀xk ∈ E f (xk , ϕ(xk )) ∈ E, ∀xk ∈ E E(f (xk , ϕ(xk ))) − E(xk ) ≤ −F (xk , ϕ(xk )), ∀xk ∈ E.

A proof of Theorem 1 can be found, e.g., in [16]. In the following, a number of approaches are described based on Theorem 1 which ensure closed loop stability. Zero Terminal State Constraint. An approach to guarantee closed loop stability is to choose E(xk ) = 0, E = {0}, and ϕ(xk ) = 0 [14], i.e., one obtains the so called zero terminal state constraint xk+N |k = 0 .

(7)

If the optimal control problem (5) is feasible at the ﬁrst time instant, all assumptions of Theorem 1 are satisﬁed and closed loop stability is achieved. The advantage of this method is the conceptual simplicity. The disadvantage of the zero terminal constraint is that it makes the resulting optimal control problem (5) in general diﬃcult to solve and a long prediction horizon is needed to obtain feasibility at the ﬁrst time instant. Terminal Region. Another approach, called dual mode model predictive control [17], is to choose E(xk ) = 0, E = {xk ∈ Rn |xTk P xk ≤ α}, and uk = ϕ(xk ) = Kxk as a locally stabilizing linear state feedback. The positive deﬁnite matrix P of the terminal region E and the linear feedback matrix K are determined by a particular procedure in such a way that the assumptions of Theorem 1 are satisﬁed [17]. The dual mode refers to two diﬀerent controllers that are applied in diﬀerent regions of the state space depending on the state being inside or outside the terminal region E. If the state is outside the terminal region E, the model predictive controller is used to steer the state inside the terminal region E. If the state is inside the terminal region E, the linear feedback uk = Kxk is applied. Hence, closed loop stability is achieved

330

T. Raﬀ et al.

by switching between the model predictive controller and the linear controller. From a computational point of view, this approach is more attractive than the zero terminal state constraint (7) since the terminal state is only required to lie in a region around the origin. A drawback of this approach is the switching between the model predictive controller and the linear controller which makes the implementation complicated. Terminal Cost and Terminal Region. In the so-called quasi inﬁnite horizon model predictive control scheme [6], the terminal cost E is chosen as E(xk ) = xTk P xk , the terminal region E as E = {xk ∈ Rn |xTk P xk ≤ α}, and uk = ϕ(xk ) = Kxk as a locally stabilizing linear state feedback. The terminal cost E, the terminal region E, and the linear feedback uk = ϕ(xk ) = Kxk are calculated oﬀ-line by a procedure described in [6] so that the assumptions of Theorem 1 are satisﬁed. The underlying idea of the quasi inﬁnite horizon model predictive control scheme is to approximate the inﬁnite horizon optimal control problem (1) by the terminal cost E. Due to this approximation it can be shown that this scheme has a better performance in comparison with the zero terminal constraint model predictive controller or the dual model model predictive controller. Furthermore, the terminal region E of the quasi inﬁnite horizon model predictive controller is better suited for computational purposes than a zero terminal state constraint. Note that not all nonlinear model predictive control schemes with guaranteed closed loop stability are reviewed. For a detailed and more rigorous treatment of model predictive control schemes with guaranteed stability for discrete time and continuous time systems, see e.g. [16] and the references quoted therein. In summary, model predictive control is suitable to control linear and nonlinear multivariable systems subject to state and input constraints. Furthermore, closed loop stability can be guaranteed by adding, e.g., a terminal cost E or a terminal region E to the ﬁnite horizon optimal control problem (2). The price to pay in model predictive control is that one has to solve at each time instant a ﬁnite horizon optimal control problem, i.e., in general a nonlinear and possible nonconvex optimization problem. Typically, specially tailored optimizers [2, 8] are used to solve the optimization problem in nonlinear model predictive control which enable to use this control strategy for practical control problems. For discrete time polynomial systems, the resulting optimization problem is a polynomial optimization problem if the objective function and the constraints are polynomial functions. One possible solution approach to solve polynomial optimization problems is to translate the original optimization problem to a semideﬁnite program via the theory of moments [15] or the theory of sum of squares polynomials [20]. This solution approach is in the following applied to solve the ﬁnite horizon optimal control problem in nonlinear model predictive control for discrete time polynomial systems. In the next section, a review of sum of squares techniques and polynomial optimization is given and in Sect. 4 polynomial optimization is applied to nonlinear model predictive control.

Model Predictive Control and Sum of Squares

331

3 Sum of Squares Techniques In this section, an introduction to the sum of squares decomposition of multivariate polynomials and polynomial optimization is given. The background of the sum of squares decomposition is presented in Sect. 3.1. Furthermore, the concept of polynomial optimization is introduced in Sect. 3.2. Finally, Sect. 3.3 gives an example to polynomial optimization. 3.1 Sum of Squares The sum of squares decomposition is a method to check if a multivariate polynomial can be decomposed into a sum of squares. Since a sum of squares polynomial is always positive semideﬁnite, one can use the sum of squares decomposition as a suﬃcient condition to check if a multivariate polynomial is positive semideﬁnite. The question whether a polynomial in several variables can be written as sum of squares has a long history and goes back to Hilbert [12]. From a computational point of view, one can show that the general problem of checking if a polynomial is positive semideﬁnite is a NP-hard problem, i.e., the problem is hard to solve. Today, questions about positive polynomials are investigated in the ﬁeld of real algebraic geometry. In the following, some facts about sum of squares polynomials and some computational aspects of the sum of squares decomposition are summarized. More details about sum of squares decomposition and its application in control theory can be found in [19, 29, 22]. Deﬁnition 1. A polynomial - p of degree d = d1 + ... + dn , di ∈ N0 with real coeﬃcients cd , i.e., p(x) = d1 +...+dn ≤d cd xd11 ...xdnn , is a sum of squares (SOS) if there exists a ﬁnite number of polynomials pi such that p can be written as p(x) = p2i (x) . (8) i

Hilbert proved that not every positive semideﬁnite polynomial can be written as a sum of squares. However, Artin’s result in 1927, which was the answer to Hilbert’s 17th problem posed in 1900, states that every positive semideﬁnite polynomial is a sum of squares of rational functions [22]. The next theorem, which is central for computational purposes, gives the answer to the question, when a polynomial is a sum of squares [7]: Theorem 2. A polynomial p of degree 2d has a sum of squares decomposition if and only if there exists a positive semideﬁnite matrix Q such that p(x) = mT (x)Qm(x) ,

(9)

where m is the vector of all monomials in x1 , . . . , xn of degree less or equal such to d, i.e., m(x) = [1, x1 , x2 , . . . , xn , x1 x2 , . . . , xdn ]T . There exist n+d n monomials.

332

T. Raﬀ et al.

This representation theorem, which is based on the so called “Gram matrix” method, tells that all sum of squares polynomials can be parameterized by the set (convex cone) of positive semideﬁnite matrices. The next theorem [27], which basically extends Artin’s result, gives a link between sum of squares polynomials and positive deﬁnite polynomials. Theorem 3. If p is a positive deﬁnite polynomial, then x r p(x) is a sum of squares polynomial for a suﬃciently large integer r. This theorem is helpful if positive deﬁniteness of a polynomial must be checked with the help of a sum of squares decomposition. Furthermore, it is often useful and suﬃcient to check positive semideﬁniteness of a polynomial p on a subset in Rn constrained by the set of points {x ∈ Rn : p1 (x) ≥ 0}, where p1 is a polynomial. In the theory of quadratic forms, this is usually done via Finsler’s lemma or the S-procedure [4]. An analogous type of argument does the same for polynomials (cf. e.g. [18]): Lemma 1. Let p1 be a polynomial. A polynomial p is positive semideﬁnite on the set {x ∈ Rn : p1 (x) ≥ 0}, if there exists a positive semideﬁnite polynomial q such that p(x) − q(x)p1 (x) is positive semideﬁnite. It should be mentioned that there also exist representation theorems for (strictly) positive polynomials on compact semialgebraic sets, i.e., compact sets deﬁned by polynomial inequalities [13, 22]. These representation theorems [13, 22] play a decisive role in the polynomial optimization theory [15, 20] which is outlined in Sect. 3.2. However, what makes the above results especially appealing from a computational point of view is the fact that these results can be eﬃciently and reliably solved on a computer, namely, with the help of semideﬁnite programming. In particular, in [19], the gap between the Gram matrix method and semideﬁnite programming was bridged by showing that the existence of a sum of squares decomposition of a polynomial can be decided by solving a semideﬁnite program. Semideﬁnite programming, which may be viewed as a generalization of linear programming, is a convex optimization problem with a linear objective function under linear matrix inequality constraints. Semideﬁnite programs have very nice properties in optimization theory and practice [30, 5] and are very successful in linear control design and combinatorial optimization [4]. Combining the sum of squares decomposition and semideﬁnite programming, solutions for the following system of aﬃne polynomial inequalities can be obtained [19, 29]: Nj

qi (x)pij (x) ≥ rj (x)

i=1 Nj i=1

(10) q˜i (x)˜ pij (x) = r˜j (x) ,

Model Predictive Control and Sum of Squares

333

where j = 1, . . . , M , pij , p˜ij , rj , and r˜j are given polynomials and qi and q˜i are unknown (to be determined) polynomials, for which the degrees have to be speciﬁed. This system of polynomial equations and inequalities can be solved by a sum of squares relaxation, namely by solving Nj

qi (x)pij (x) − rj (x) = SOS

i=1 Nj

(11) q˜i (x)˜ pij (x) − r˜j (x) = 0 ,

i=1

where SOS stands for an arbitrary sum of squares polynomial. If one would like to enforce positive deﬁniteness instead of semideﬁniteness, e.g., p(x)q(x) > 0, then this can be done for example via p(x)q(x) − r(x) = SOS, where r is a positive deﬁnite polynomial, e.g. r(x) = εxT x, ε > 0. Note, that for example with the freely available Matlab toolbox SOSTOOLS [21] the equations and inequalities of type (11) can be easily solved. In summary, it should be noted that the sum of squares decomposition is a relaxed condition, i.e., a suﬃcient condition for positive semideﬁniteness. Hence there may exist solutions to the above polynomial system of equations and inequalities, although no solution can be found by using the sum of squares decomposition. Furthermore, the degrees of the unknown polynomials qij and q˜ij must be speciﬁed a priori. In the next section, the sum of squares techniques are applied to solve polynomial optimization problems. 3.2 Polynomial Optimization In recent years, major advances in polynomial optimization have been made. Two approaches based on semideﬁnite programming have been developed. One approach uses the theory of moments [15] and the other approach the theory of sum of squares [20, 28]. These methods are strongly related because the theory of moments is, in some sense, dual to the theory of nonnegative polynomials. Both approaches build a sequence of semideﬁnite programs of increasing size. The optima of these semideﬁnite programs converge monotically to the global optimum for the original nonconvex polynomial optimization problem. In the following, the basic idea of polynomial optimization via the sum of squares approach is outlined. Consider the optimization problem min

x∈K

p0 (x) ,

(12)

where p0 (x) ∈ R[x] and K = {x ∈ Rn : pi (x) ≤ 0, i = 1, . . . , m} is a compact set described by multivariate polynomial inequalities, i.e., pi (x) ∈ R[x], i = 1, . . . , m. Furthermore, some pi satisﬁes, on its own, the condition that {x ∈ Rn : pi (x) ≤ 0} is compact. Under these assumptions, a representation theorem [23] can be established as follows:

334

T. Raﬀ et al.

Theorem 4. If the assumptions as stated above are satisﬁed, then any polynomial p0 strictly positive on the set K can be written as p0 (x) = q0 (x) −

m

qi (x)pi (x) ,

(13)

i=1

where qi , i = 0, . . . , m, are all sum of squares polynomials. Unfortunately, the degrees of the sum of squares polynomials qi , i = 0, . . . , m, are unknown and must be chosen suﬃciently high. However, Theorem 4 plays a central role since it allows to consider the following equivalent polynomial optimization problem [20, 28] max s.t.

t p0 (x) − t +

m

qi (x)pi (x) ≥ 0,

(14)

∀x

i=1

where pi , i = 0, . . . , m, are given polynomials and qi , i = 1, . . . , m, are unknown sum of squares polynomials. The basic idea of the optimization problem (14) is shown in Fig. 1 for an unconstrained polynomial optimization problem in one variable. Instead of minimizing the polynomial p0 directly, t is maximized while satisfying the inequality in (14). Since the unknowns t, qi , i = 1, .., m, enter aﬃne in the polynomial optimization problem (14), it can be solved using the sum of squares decomposition of multivariate polynomials introduced in the previous subsection. If the degrees of the sum of squares polynomials qi , i = 1, . . . , m, have not been chosen suﬃciently high, then t is a lower bound for p , where p is the global optimum of the original polynomial optimization problem (12). However, increasing the degrees of the sum of squares polynomials qi , i = 1, . . . , m, the lower bound t converges arbitrarily close to p . The price to pay for improving the lower bound t by increasing the degrees of the sum of squares polynomials is that the number of decision variables and the size of the semideﬁnite program of the polynomial optimization problem (12) grow exponentially. The theory of moments [15] relates to the theory of nonnegative polynomials [20, 28] by considering the dual problem of (14). It can be shown that the dual problem of (14) is the semideﬁnite program obtained by solving the

p0(x) t>t

p0(x)

p0(x) t=t

t
x

0

x

0

Fig. 1. Basic idea of polynomial optimization

x

Model Predictive Control and Sum of Squares

335

original polynomial optimization problem (12) using the theory of moments. What remains to be discussed is the detection of the global optimum and the extraction of the optimal solution. In [11], a rank condition is proposed to detect the global optimum. The extraction of the optimal solution is based on a Cholesky decomposition and on an eigenvalue problem. For a more thorough discussion of polynomial optimization, consult [11, 15, 20] and the references quoted therein. Note, that with the freely available Matlab software toolbox GloptiPoly [10] polynomial optimization problems of type (12) can be easily solved. In the following, a small example is given to illustrate the possibilities of polynomial optimization before applying polynomial optimization to nonlinear model predictive control in Sect. 4. 3.3 Example Consider the polynomial optimization problem 2x41 − 4x21 + x1 + x22 + 4

min

x1 ,x2

−

s.t.

3 3 3 3 ≤ x1 ≤ , − ≤ x2 ≤ , 2 2 2 2

where the objective function p0 (x) = 2x41 − 4x21 + x1 + x22 + 4 has one local and one global minimum as shown in Fig. 2. Using GloptiPoly [10] to solve the semideﬁnite programming relaxations described above, the computed lower bound is t = 0.9705 and the extracted solutions are x1 = −1.0575 and x2 = 0. In this example, the lower bound t = 0.9705 is equal to the global minimum of the original problem. This example illustrates that the global minimum of a

12 10

p0(x)

8 6 4 2 0 2 0 x2

−2

−2

0

−1

1

x1

Fig. 2. Nonconvex polynomial objective function p0

2

336

T. Raﬀ et al.

nonconvex polynomial optimization problem can be eﬃciently computed via semideﬁnite programming.

4 Sum of Squares in Nonlinear Model Predictive Control In this section, sum of squares techniques and polynomial optimization are applied to nonlinear model predictive control for the class of discrete time polynomial systems. In Sect. 4.1 and Sect. 4.2, it is shown that the ﬁnite horizon optimal control problem in nonlinear model predictive control can be formulated as a polynomial optimization problem and can be solved with the polynomial optimization techniques presented in the Sect. 3. Furthermore, the applicability of polynomial optimization in nonlinear model predictive control is shown in Sect. 4.3. 4.1 Polynomial Optimization Formulation of NMPC In the following it is shown that the ﬁnite horizon optimal control problem (5) in nonlinear model predictive control can be formulated as a polynomial optimization problem for the class of discrete time polynomial systems. In the following it is assumed that the system (3) is polynomial, i.e., f : Rn × Rm → Rn is a polynomial vector ﬁeld and f (0, 0) = 0. Furthermore, the stage cost F : Rn ×Rm → R is assumed to be a polynomial map that satisﬁes F (0, 0) = 0. The terminal penalty term E : Rn → R is assumed to be also a polynomial map with E(0) = 0 and the terminal region E can be described by a polynomial inequality. In order to formulate the ﬁnite horizon optimal control problem (5) as a polynomial optimization problem, the predicted states xk+1|k , . . . , xk+N |k must be computed from the future inputs uk|k , . . . , uk+N −1|k and the initial condition xk . To write (5) in a compact form, the vector T ξ = xk+1|k , xk+2|k , ... , xk+N |k

(15)

for the predicted states is introduced. Iterating system (3) yields ξ = H(ν, xk )

(16)

with 

h1 (uk|k , xk )





f (xk , uk|k )



    h2 (uk|k , uk+1|k , xk )   f (xk+1|k , uk+1|k )       H(ν, xk ) =  =  .. ..     . .     hN (uk|k , . . . , uk+N −1|k , xk ) f (xk+N −1|k , uk+N −1|k )

Model Predictive Control and Sum of Squares

337

and ν = [uk|k , . . . , uk+N −1|k ]T . Note, that hi , i = 1, . . . , N , are multivariate polynomial functions in ν for a ﬁxed initial condition xk . Using (16), the cost function (6) can be rewritten as J(ν, xk ) = F (xk , uk|k ) + ... + F (hN −1 (uk|k , . . . , uk+N −2|k , xk ), uk+N −1|k ) + E(hN (uk|k , . . . , uk+N −1|k , xk ))

(17)

= p0 (ν) . Hence, J(ν, xk ) is a multivariate polynomial function in ν for a ﬁxed state xk , i.e., p0 (ν) ∈ R[ν]. Next, the constraints of the optimization problem (5) must be expressed in terms of polynomial inequalities. For simplicity, suppose that the constraints are given as X := {xk ∈ Rn |xmin ≤ xk ≤ xmax } U := {uk ∈ Rm |umin ≤ uk ≤ umax } ,

(18)

where xmin , xmax , umin , umax are constant vectors. Therefore, the constraints can be written as / . H(ν, xk ) ≤0, L 1 . / (19) ν M ≤0, 1 where



I1

−I  1   0   0 L=   ..  .   0  0

0 ... 0 −xmax



0 ... 0 xmin    I1 ... 0 −xmax   −I1 ... 0 xmin  ,  .. .. ..  . ... . .   0 ... I1 −xmax   0 ... −I1 xmin



I2

−I  2   0   0 M =   ..  .   0  0

0 ... 0 −umax



0 ... 0 umin    I2 ... 0 −umax   −I2 ... 0 umin   ,  .. .. ..  . ... . .   0 ... I2 −umax   0 ... −I2 umin

and I1 and I2 are identity matrices of appropriate dimensions. Therefore, (19) deﬁnes 2(n + m)N polynomial inequality constraints pi (ν) ∈ R[ν], i = 1, . . . , 2(n + m)N , for a ﬁxed state xk . Combining (17), (19), and the fact that the terminal region E is described by an additional polynomial inequality, e.g., p2(n+m)N +1 (ν) ≤ 0 , one obtains the following theorem [25]:

(20)

338

T. Raﬀ et al.

Theorem 5. The ﬁnite horizon optimal control problem (5) can be formulated as a polynomial optimization problem of the form min

ν∈K

p0 (ν)

(21)

for discrete time polynomial systems, if K = {ν ∈ Rm · N : pi (ν) ≤ 0, i = 1, . . . , 2(n+m)N +1} is a compact set described by the multivariate polynomial inequalities pi (ν) ∈ R[ν], i = 1, . . . , 2(n + m)N + 1. 4.2 Polynomial Optimization Formulation of NMPC for Implicit Discrete Time Polynomial Systems In the following it is brieﬂy outlined that polynomial optimization can also be applied to solve the ﬁnite horizon optimal control problem in nonlinear model predictive control for implicit discrete time polynomial systems, i.e., for systems of the form f (xk+1 , xk , uk ) = 0 . The basic idea in this case is to introduce the vector  f (xk+1|k , xk , uk|k )   f (xk+2|k , xk+1|k , uk+1|k )   W (ξ, ν) =  f (xk+3|k , xk+2|k , uk+2|k )  ..  . 

(22)       ,   

(23)

f (xk+N |k , xk+N −1|k , uk+N −1|k )

T where the variables ξ = xk+1|k , xk+2|k , ... , xk+N |k are considered now as independent variables. Therefore, the cost function (6) can be rewritten as J(ν, xk ) = F (xk , uk|k ) + ... + F (xk+N −1|k , uk+N −1|k )

(24)

+ E(xk+N |k ) = p0 (ξ, ν) , where p0 (ξ, ν) is a multivariate polynomial function in the variables ξ and ν for a ﬁxed state xk , i.e., p0 (ξ, ν) ∈ R[ξ, ν]. In the same way one can proceed writing the state and input constraints and the terminal region as polynomial inequalities in ξ and ν. Therefore, one has 2(n + m)N + 1 polynomial inequalities of the form pi (ξ, ν) ≤ 0,

i = 1, . . . , 2(n + m)N + 1

(25)

Model Predictive Control and Sum of Squares

339

with pi (ξ, ν) ∈ R[ξ, ν], i = 1, . . . , 2(n + m)N + 1, for a ﬁxed state xk . Finally, one has to restore the relation between xk+i|k and xk+j|k by introducing the constraint W (ξ, ν) 2 = 0 ,

(26)

-n 1 where x = ( i=1 x2i ) 2 denotes the Euclidian norm of x ∈ Rn . Note, that equality (26) can be rewritten as two polynomial inequalities of the form p2(n+m)N +2 (ξ, ν) = W (ξ, ν) 2 ≤ 0 p2(n+m)N +3 (ξ, ν) = W (ξ, ν) 2 ≥ 0 .

(27)

Therefore, the ﬁnite horizon optimal control problem for implicit discrete time polynomial systems can be solved via the polynomial optimization techniques presented in Sect. 3. This result is summarized in the next theorem. Theorem 6. The ﬁnite horizon optimal control problem (5) can be formulated as a polynomial optimization problem of the form min

(ξ,ν)∈K

p0 (ξ, ν)

(28)

for implicit discrete time polynomial systems of the form (22), if K = {(ξ, ν) ∈ Rn · N × Rm · N : pi (ξ, ν) ≤ 0, i = 1, . . . , 2(n + m)N + 3} is a compact set described by the multivariate polynomial inequalities pi (ξ, ν) ∈ R[ξ, ν], i = 1, . . . , 2(n + m)N + 3. Due to Theorem 5 and Theorem 6, the polynomial optimization techniques presented in Sect. 3 can be applied to nonlinear model predictive control for discrete time polynomial systems. In the next section, the applicability of sum of squares techniques in nonlinear model predictive control is demonstrated on a small example. 4.3 Example In the following, polynomial optimization is used to solve the ﬁnite horizon optimal control problem in nonlinear model predictive control for discrete time polynomial systems as stated in Theorem 5. Consider the stabilization problem of the second order polynomial system [6] described by the diﬀerence equations xk+1 = f (xk , uk ) , where (1)

(2)

xk = [xk , xk ]T ,

340

T. Raﬀ et al.



(1)

(2)

(1)



xk + 0.1xk + 0.05(1 + xk )uk

. f (xk , uk ) =  (1) (2) (2) 0.1xk + xk + 0.05(1 − 4xk )uk This system is unstable and its linearization around the origin is stabilizable but not controllable. Furthermore, the input has to satisfy the constraint U := {uk ∈ R| − 2 ≤ uk ≤ 2}. To stabilize the system, a nonlinear model predictive controller is designed based on the ﬁnite horizon optimal control problem (5) with a quadratic stage cost / . T 0.5 0 xk + 0.1u2k , F (xk , uk ) = xk 0 0.5 a prediction and control horizon of the length N = 4, a terminal penalty term E(xk ) = xTk P xk , and a terminal region E = {xk ∈ R2 |xTk P xk ≤ α}. Using the quasi inﬁnite horizon method [1, 6] to ensure stability of the closed loop system, the matrix P and the parameter α are chosen as . / 4.5 1.8 P = , α = 0.3 . 1.8 4.5 The resulting polynomial MPC ﬁnite horizon optimal control problem was solved using GloptiPoly [10]. Figures 3 and 4 show the initial response of the closed loop system with an initial condition x0 = [−0.3, −0.9]T . It can be seen that the closed loop trajectories converge to the origin without violating the input constraints. Hence, this example demonstrates that in principle the polynomial optimization techniques based on semideﬁnite programming 0.1 0 −0.1

xk ,xk

(1) (2)

−0.2 −0.3 −0.4 −0.5 −0.6 −0.7 −0.8 −0.9 0

5

10

15

20

25

30

Normalized Time (1)

Fig. 3. Closed loop state trajectories xk

(2)

(black) and xk

(gray)

Model Predictive Control and Sum of Squares

341

2 1.8 1.6 1.4

u

k

1.2 1

0.8 0.6 0.4 0.2 0 0

5

10

15

20

25

30

Normalized Time

Fig. 4. Control input uk

can be used to solve the ﬁnite horizon optimal control problem in nonlinear model predictive control for discrete time polynomial systems. Finally, some computational properties of the resulting polynomial optimization problem are summarized in Table 1. This table shows some optimization parameters of the polynomial optimization problem obtained using Matlab 6.5 and GloptiPoly on a Linux PC with a 3 GHz Intel Pentium 4 processor and 1.5 GB RAM. From Table 1 it can be seen that for a prediction horizon N greater than N = 4 the size of the semideﬁnite program (SDP) increases strongly. The reason for this increase is that the problem size of polynomial optimization problems growths exponentially with the number of variables [11, 15]. Therefore, these techniques can be only applied to small and medium size problems. In order to make these techniques applicable to larger polynomial optimization problems and thus to model predictive control for real world application, it is necessary to develop numerical methods which exploit the system structure, e.g., sparsity and symmetry of polynomial optimization problems. Summarizing, the key advantage of applying sum of squares techniques in nonlinear model predictive control is that a global minimum of a possible nonconvex ﬁnite horizon optimal control problem is obtained via semideﬁnite programming. This property can avoid unacceptable performance degradation Table 1. Optimization parameters Prediction horizon N =3 N =4 N =5 Number of variables 3 4 5 SDP decision variables 455 495 3003 Relaxation 6 4 5

342

T. Raﬀ et al.

in nonlinear model predictive control. The main disadvantage of this approach is that it is computationally very demanding using currently available algorithms and therefore only applicable to small and medium size problems.

5 Conclusion In this paper it was shown that sum of squares techniques and polynomial optimization can be applied to solve ﬁnite horizon optimal control problems in model predictive control for the class of discrete time polynomial control systems. Polynomial optimization problems can be solved via semideﬁnite programming based on the theory of sum of squares polynomials and on the theory of moments. The key advantage of this approach is that a global minimum of a possible nonconvex optimal control problem in model predictive control is obtained via semideﬁnite programming. This property can avoid unacceptable performance degradation in model predictive control. The applicability of polynomial optimization in nonlinear model predictive control was demonstrated on an example. However, this example also shows that a straightforword application of sum of squares techniques in model predictive control is only applicable to small and medium size polynomial optimization problems, because polynomial optimization problems are computationally very demanding using currently available algorithms.

References [1] F. Allg¨ ower, T.A. Badgwell, J.S. Qin, J.B. Rawlings, and S.J. Wright. Nonlinear Predictive Control and Moving Horizon Estimation – An Introductory Overview. In Advances in Control: Highlights of European Control Conference 1999, pp. 391–449. Springer, 1999. [2] L.T. Biegler. Eﬃcient Solution of Dynamic Optimization and NMPC Problems. In Nonlinear Model Predictive Control, Birkh¨ auser, pp. 219–244, 2000. [3] R.R. Bitmead, M. Gevers, I.R. Petersen, and R.J. Kaye. Monotonicity and Stabilizability Properties of Solutions of the Riccati Diﬀerence Equation: Propositions, Lemmas, Theorems, Fallacious Conjectures and Counterexamples. Systems and Control Letters, 5:309–315, 1985. [4] S. Boyd, L. El Ghaoui, E. Feron, and V. Balakrishnan. Linear Matrix Inequalities in System and Control Theory. SIAM, 1994. [5] S. Boyd and L. Vandenberghe. Convex Optimization. Cambridge University Press, 2004. [6] H. Chen and F. Allg¨ ower. A Quasi-Inﬁnite Horizon Nonlinear Model Predictive Control Scheme with Guaranteed Stability. Automatica, 34:1205–1218, 1998. [7] M.D. Choi, T.Y. Lam, and B. Reznick. Sums of Squares of Real Polynomials. In Proceedings of Symposia in Pure Mathematics 58, pp. 103–126, 1995. [8] M. Diehl, R. Findeisen, Z.K. Nagy, H.G. Bock, J.P. Schl¨ oder, and F. Allg¨ ower. Real-time Optimization and Nonlinear Model Predictive Control of Processes

Model Predictive Control and Sum of Squares

[9]

[10]

[11]

[12] [13] [14]

[15] [16]

[17]

[18]

[19]

[20]

[21]

[22] [23] [24] [25]

343

Governed by Diﬀerential-Algebraic Equations. Journal of Process Control, 4:577–585, 2002. R. Findeisen, L. Imsland, F. Allg¨ ower, and B.A. Foss. State and Output Feedback Nonlinear Model Predictive Control: An Overview. European Journal of Control, 9:190–206, 2003. D. Henrion and J.B. Lasserre. {GloptiPoly} – Global Optimization over Polynomials with Matlab and SeDuMi, 2003. Available from http://www.laas.fr/ henrion/software/gloptipoly/. D. Henrion and J.B. Lasserre. Detecting Global Optimality and Extracting Solutions in GloptiPoly. In D. Henrion and A. Garulli, editors, Positive Polynomials in Control. Springer, 2004. D. Hilbert. Gesammelte Abhandlungen. Springer Verlag, 1932. T. Jacobi and A. Prestel. Distinguished Representations of Strictly Positive Polynomials. Journal f¨ ur reine und angewandte Mathematik, 532:223–235, 2001. S. Keerthi and E. Gilbert. Optimal Inﬁnite-Horizon Feeback Laws for a General Class of Constrained Discrete-Time Systems: Stability and Moving-Horizon Approximations. Journal of Optimization Theory and Applications, 57:265–293, 1988. J.B. Lasserre. Global Optimization with Polynomials and the Problems of Moments. SIAM Journal on Optimization, 11:796–817, 2001. D.Q. Mayne, J.B. Rawlings, C.V. Rao, and P.O.M. Scokaert. Constrained Model Predictive Control: Stability and Optimality. Automatica, 36:789–814, 2000. H. Michalska and D.Q. Mayne. Robust Receding Horizon Control of Constrained Nonlinear Systems. IEEE Transactions on Automatic Control, 38:1623–1633, 1993. A. Papachristodoulou and S. Prajna. On the Construction of Lyapunov Functions using the Sum of Squares Decompostion. In Proccedings of 41th IEEE Conference on Decision and Control, USA, pp. 3482–3487, 2002. P.A. Parrilo. Structured Semideﬁnite Programs and Semialgebraic Geometry Methods in Robustness and Optimization. PhD thesis, California Institute of Technology, 2000. P.A. Parrilo and B. Sturmfels. Minimizing Polynomial Functions. In S. Basu and L. Gonzalez-Vega, editors, Algorithmic and Quantitative Real Algebraic Geometry, vol. 60, pp. 83–99. DIMACS, 2000. S. Prajna, A. Papachristodoulou, and P. A. Parrilo. {SOSTOOLS} – Sum of Squares Optimization Toolbox for Matlab, 2002. Available from http://www.cds.caltech.edu/sostools. A. Prestel and C.N. Delzell. Positive Polynomials. From Hilbert’s 17th Problem to Real Algebra. Springer Verlag, 2000. M. Putinar. Positive Polynomials on Compact Semi-Algebraic Sets. Indiana University Mathematical Journal, 42:969–984, 1993. S.J. Qin and T.A. Badgwell. A Survey of Industrial Model Predictive Control Technology. Control Engineering Practice, 11:733–764, 2003. T. Raﬀ, R. Findeisen, C. Ebenbauer, and F. Allg¨ ower. Model Predictive Control for Discrete Time Polynomial Control Systems: A Convex Approach. In Proceedings of the 2nd IFAC Symposium on System, Structure and Control, Mexico, pp. 158–163, 2004.

344

T. Raﬀ et al.

[26] J.B. Rawlings. Tutorial Overview of Model Predictive Control. IEEE Control Systems Magazine, 20:38–52, 2000. [27] B. Reznick. Uniform Denominators in Hilbert’s Seventeenth Problem. Mathematische Zeitschrift, 220:75–97, 1995. [28] N.Z. Shor. Class of Global Bounds of Polynomial Functions. Tekhnicheskaya Kibernetika, 6:9–11, 1987. [29] B. Sturmfels. Solving Systems of Polynomial Equations. American Mathematical Society, 2002. [30] M.H. Wright. The Interior-point Revolution in Optimization: History, Recent Developments, and Lasting Consequences. Bulletin of the American Mathematical Society, page Electronically published, 2004.

Comparison of Two Measures of Dynamic Stability During Treadmill Walking M. Schablowski1 and H.J. Gerner1 Stiftung Orthop¨ adische Universit¨ atsklinik Heidelberg [email protected]

Summary. Stability of biped walking is an important characteristic of legged locomotion. Whereas clinical investigations often relate increased variability to decreased stability, there are only few studies examining stability aspects directly. On the other hand, various papers from the ﬁeld of robotics are dedicated to the question: how can the stability of legged locomotor systems be quantiﬁed? Particularly, when it comes to realizing fast motions in robots, the question of maintaining dynamic stability is of utmost importance. The current paper presents a theoretical comparison of several measures for dynamic stability – namely Floquet multipliers and Local Divergence Exponents (LDE). The sensitivity of these parameters to changes in speed of human treadmill locomotion is investigated. Experimental results show that two diﬀerent types of stability with respect to speed dependence seem to exist. Short term LDE and Floquet multipliers consider the stability over a period of one stride, which seems to be optimal at intermediate walking speeds. Long term LDE quantify stability of movement trajectories over multiple strides. This type of stability decreases with speed and may be one reason for changing gaits from walking to running at a certain speed value.

1 Introduction Stability of biped walking is an important characteristic of legged locomotion [1]. Studies which investigate this issue from a clinical point of view mainly target the question: how is an upright gait maintained in the presence of internal and external perturbations [2, 3, 4]?. Particularly, the eﬀect of macroscopic perturbations challenging the musculoskeletal system’s capabilities of balance control is studied. The measures being used for determining the degree of stability in this context are commonly based on the vertical projection of the center of mass onto the base of support [5]. When analyzing steady-state locomotion in the absence of external large scale perturbations, a diﬀerent notion of stability comes into play. Here, the question is: how stably does the movement system reproduce the trajectories of the state-variables from one cycle of motion to the other? This issue can be

346

M. Schablowski and H.J. Gerner

investigated by using tools from the theory of nonlinear dynamical systems like Floquet multipliers [6] and Lyapunov exponents [7, 8]. Both of these measures quantify the eﬀect of microscopic perturbations on the movement trajectories. Microscopic perturbations arise from extrinsic circumstances like irregular surfaces as well as from internal sources like inaccuracies during movement generation. Analyzing the impact of such microscopic perturbations during movement generation is essential for a deeper understanding of neuromuscular control in the context of nonlinear dynamical system theory [9]. The present study investigates dynamic stability of treadmill walking in healthy subjects with special consideration of the inﬂuence of walking speed. In particular, for the ﬁrst time a comparison of two similar measures of dynamic stability – Floquet multipliers and Local Divergence Exponents – for this type of movement is undertaken.

2 Methods The following subsections brieﬂy review the mathematical basis of the measures of dynamic stability used here and the experimental setup used for data acquisition. Speciﬁcally, an adaption of the calculation of Floquet-Multipliers to treadmill walking and the method used for the estimation of Lyapunov exponents are explained. 2.1 Dynamic Stability Measures Modiﬁed Floquet-Multipliers Let γ be a periodic orbit of some ﬂow Φt in n arising from a nonlinear ﬂow f (x). We then ﬁrst take a local cross section Σ ⊂ n of dimension n-1. Σ deﬁnes a hypersurface which needs not to be planar but has to be chosen such that the ﬂow is everywhere transverse to it [10]. Further on, only those points where the Φt intersects Σ are considered (“Poincar´e map”). Thereby it is assumed that the oscillation period Tp is ﬁxed. The Floquet-Multipliers represent a measure of how the intersecting points vary from one period of oscillation to the next. For application of Floquet multipliers to gait analysis one starts from a discrete state space description of the human movement system [11]: x(k) = [q1 (k), . . . , qS (k), q˙1 (k), . . . , q˙S (k)]T

(1)

For investigating dynamic stability of gait, possible state variables q and q˙ are primarily the joint angles and the trajectories of selected points on the body surface. The number of state variables S depends on the measurement system at hand and the desired degree of diﬀerentiation of the model. Applying a Poincar´e mapping to equation (1) leads to the following formulation:

Dynamic Stability of Treadmill Walking

xn+1 = Pp (xn )

347

(2)

Pp represents a nonlinear vector function, which maps the state space vector x at point in time p of the oscillation period n to the state space vector at the same point in time during period (n + 1) (p is arbitrary, yet ﬁxed during the oscillation period). In the following only strictly periodic solutions of equation (2) are considered: x∗p = Pp (x∗p )

(3)

For investigating the stability of the equilibrium point x∗ against inﬁnitesimal perturbations δx, Pp is linearized around x∗ as follows:

δxn+1

∂Pp

(x∗p ) ∂x = Jp δxn

Jp =

(4) (5)

Here Jp is the Jacobian (also called Floquet or monodromy) matrix [10]. Due to the linearization, now, instead of the nonlinear vector function Pp the linear matrix Jp needs to be estimated from the measurement data (see below). Stability of the equilibrium point x∗p of the linear system (5) is determined through the Eigenvalues βsp , s = 1, . . . , 2S of Jp . The Eigenvalues βsp are also called Floquet Multipliers. x∗p is stable if |βsp | < 1 for all s. In case of stability p | dominates the system the Eigenvalue with the maximum absolute value |βmax behavior. Floquet multipliers were introduced into the ﬁeld of gait analysis in [12] and were applied to the investigation of dynamic stability of post-polio gait [11, 6]. In these studies, overground walking was investigated and in particular the transition into the dynamical equilibrium was analyzed. For this purpose, the ﬁrst four steps starting from stand were analyzed under the assumption that dynamic equilibrium was achieved after the fourth step. In this case, Floquet multipliers quantify stabilization of gait after a very large perturbation (the resting position). During treadmill walking however, dynamic stability against inﬁnitesimal deviations from dynamic equilibrium is of interest. Therefore, a diﬀerent deﬁnition of the dynamic equilibrium needs to be assumed. In this paper, this equilibrium is deﬁned as the mean value of the state space vector x at point in time p over all strides of a particular trial: x∗p,s =

N −1 1 xs (p + n · Tp ), N n=0

s = 1, . . . , 2S

(6)

Deviations from x∗p for all N strides are calculated by: δxn = x(p + n · Tp ) − x∗p ,

n = 0, . . . , N − 1

(7)

The elements Jp,ns of the Jacobian Jp now are estimated from a system of linear equations:

348

M. Schablowski and H.J. Gerner

δxn+1 =

2S

Jp,ns · δxns ,

n = 1, . . . , N − 1

(8)

s=1

The N − 1 equations of system (8) describe the linear evolution of the deviations from stride n to stride n + 1. For this reason, the minimum number of strides that need to be measured is determined by the dimension of the state space model. A necessary condition for a unique solution of equation (8) is 1 : N ≥ 2S + 1

(9)

This relation is not mentioned explicitly in [6], however, condition (9) is fulﬁlled implicitly by the experimental protocol (the ﬁrst 3 strides from 16 trials are accumulated to a measurement series of 48 strides, the 4th stride is the equilibrium [6]). In theory, the Floquet multipliers are independent from the point in time p when the Poincar´e map is being calculated. In experimental practice however, there are nevertheless diﬀerences when considering diﬀerent points in time during the gait cycle. For this reason the authors in [6] use the times of maximum knee ﬂexion, because these, according to the authors, lead to the most consistent results. These diﬀerences result from slight errors in the detection of the points in time from experimental data but also from the fact human gait is not strictly periodic. Therefore, in the current paper a modiﬁed version of the Floquet multipliers is introduced which is based on the dynamic equilibrium during treadmill walking as deﬁned in equation (3). The model of the movement system consists of the three sagittal joint angles for the left and the right leg and the corresponding temporal derivatives (dimension of model 2S = 12). The derivatives at points in time k are estimated as follows: q(k) ˙ = (q(k + 1) − q(k − 1))/2

(10)

In order to eliminate the dependence on the point in time when the Poincar´e maps are calculated, the Floquet multipliers were modiﬁed by taking the average values of the Floquet multipliers over all K points of the normalized gait cycle (αstr ): K p αstr = 1/K βmax (11) p=1

Local Divergence Exponents Lyapunov exponents quantify the development of trajectories in state space after an initial inﬁnitesimal perturbation. They are a measure for the sensitivity of a system to such perturbations and express the degree of predictability 1

A suﬃcient condition is that the determinant of the system (8) must be diﬀerent from 0.

Dynamic Stability of Treadmill Walking

349

Fig. 1. Development of initial perturbations with the system dynamics and its quantiﬁcation through local divergence exponents (adapted from [13])

of a dynamical system (see Fig. 1). For a complete description of the eﬀects of a perturbation, the system development needs to be described in all directions of the reconstructed state space yielding multiple Lyapunov exponents. The number of these exponents is equal to the dimension of the embedding space and altogether they set up the so called Lyapunov spectrum. In practice, calculation of the full spectrum is diﬃcult and analysis is restricted to the largest Lyapunov exponent λmax , which dominates the behavior of the system. In this paper, the algorithm described in [14] using the modiﬁcation from [13] is applied. In contrast to other implementations [15], this algorithm uses all the points of a time series and thus allows for reliable estimates for short measurements. In order to eliminate dynamical correlations, a Theiler window of one stride duration was applied during reconstruction of the state space [13, 16]. The maximum ﬁnite time Lyapunov exponent λmax is deﬁned as d(t) = d0 eλmax t

(12)

where d(t) is the mean distance between neighboring trajectories at time t. d0 is the initial distance between the reference points xj and their nearest neighbors xk . The nearest neighbor is calculated as follows [14]: d0 = min xj − xk , xk

j, k = 1, . . . , M , j = k

(13)

Here, M is the number of reference trajectories. This deﬁnition of the Lyapunov exponent λmax is valid only in the dual limit t → ∞ and d0 → 0. For practical application with ﬁnite sampling times, the λmax are approximated by their ﬁnite estimated values λ∗ , which are calculated as follows [13, 14]: ln dj (i) ≈ λ∗ · i · ∆t + ln d0

(14)

In equation (14) dj (i) designates the jth pair of nearest neighbors after i discrete time steps, which is an expression for local divergence. Since Lyapunov

350

M. Schablowski and H.J. Gerner 0

−1 −1.5 −2

Walking Speed 0.2 m/s 0.4 m/s 0.6 m/s 0.8 m/s 1.0 m/s 1.2 m/s 1.4 m/s

−2.5

j

ln Left Ankle Flex

−0.5

−3 −3.5 −4 0

1

2

3

4

5 # Strides

6

7

8

9

10

Fig. 2. Local divergence for the left ankle ﬂexion angle of one subject at diﬀerent walking speeds (vT M = 0.2, . . . , 1.4 m/s )

exponents are a measure of the mean changes of d0 over time, the dj (i) are averaged over all M reference trajectories [14]: d(i) =

1 ln dj (i) ∆t

(15)

Here, · denotes the mean over all values of j. The mean local divergence d(i) over the number of strides for the left ankle joint angle of one subject is depicted in Fig. 2. The line coding in the graph corresponds to seven individual walking speeds between 0.2 und 1.4 m/s. In order to allow for direct comparison of the curves belonging to diﬀerent speeds, the x-axis was normalized to the duration of one stride as described in [17]. The oscillations during the ﬁrst stride are caused by diﬀerences in the density of points when following trajectories in the diﬀerent directions of the state space. λ∗ is derived from these curves [13, 14]. In order to distinguish the estimated values λ∗ from the theoretically deﬁned Lyapunov exponents λmax , they are also called local divergence exponents [13]. The estimation depends on selecting a range with constant slope in the curves in Fig. 2 which is diﬃcult to realize with suﬃcient accuracy in experimental data [14]. Therefore, an approach ﬁrst described by [13] was used here: two diﬀerent time scales were analyzed, where short term LDE (denoted as λST ) were calculated over the ﬁrst stride and long term LDE (λLT ) were computed for the strides 4–10 (Fig. 3). Using these procedures, local scaling exponents were calculated for a series of measurements obtained from an experimental protocol described in the next subsection.

Dynamic Stability of Treadmill Walking −1.5

0

λ =0.54

λLT=0.094

ST

ln Left Ank Flex

351

−0.5

−2

−1 −2.5 −1.5 −3

−3.5

−2

0

0.5 # Strides

1

−2.5

4

5

6

7 # Strides

8

9

10

Fig. 3. Linear regression for deriving local divergence exponents from the curves in Fig. 2 [13]

Theoretical Comparison of the Two Measures Both, the LDE as well as the Floquet multipliers represent geometrical measures, which quantify the development of two neighboring trajectories with the system dynamics. Regarding their application to the investigation of dynamic stability of human gait [13, 18, 6, 12, 11], however, there are several important diﬀerences to note: Peridocity: Floquet multipliers are based on the assumption of a strictly periodical system, whose attractor is analyzed at speciﬁc points in time [12]. The LDE do not require the system to be periodic. Linearization: Floquet multipliers are the Eigenvalues of a linearized transformation matrix describing the system dynamics between selected periodic points in time. The LDE do not rely on any kind of linearization. Modelling: Floquet multipliers are based on an a priori model of the human movement system [6], which determines the dimension of the state space. The embedding dimension which is used for calculating the LDE is derived from experimental data during further analysis. Therefore, Floquet multipliers inherently describe dynamic stability of the entire movement system as it is captured by the model. The LDE on the other hand quantify stability of the individual joint angle trajectory. 2.2 Experimental Protocol of Gait Data Acquisition Using a commercial motion analysis system (Motion Analysis, Santa Rosa, CA) kinematic data from 10 subjects with no prior history of gait disorders were acquired. Three-dimensional trajectories of retroﬂective markers placed on anatomical landmarks as prescribed in [19] were recorded and transformed into joint angles [20]. For each subject, seven trials of 90 s each with ﬁxed

352

M. Schablowski and H.J. Gerner

Fig. 4. Experimental setup for acquisition of gait data during treadmill walking

speeds ranging from vT M = 0.2, 0.4, . . . , 1.4 m/s were recorded. Additionally, three trials at slow, medium and fast walking speeds were performed. In order to avoid systematic errors, the order of speed was randomized. Whereas there are reference data for walking overground [21, 22, 23], here, for the ﬁrst time a normative base of kinematic movement data is provided for treadmill walking. Parts of these results were published in [24].

3 Results 3.1 Sagittal Plane Kinematics The mean joint angles for the seven ﬁxed walking speeds are displayed in Fig. 5. Assuming symmetry between the left and right side of the body, the data from both sides were averaged to yield one single time series per subject and speed. The most prominent changes with increasing speed are the increasing range of motion and a phase shift of the extreme values in the curves. This corresponds to the results described in [23]. On the basis of these joint angle curves, the two stability measures introduced in the previous section were calculated. 3.2 Local Dynamic Stability Floquet-Multipliers In Fig. 6 the modiﬁed Floquet multipliers αstr as deﬁned in the previous section for all trials are plotted over walking speed (black squares). The solid

Dynamic Stability of Treadmill Walking Flex 40

353

v = 0.2 m/s v = 0.4 m/s v = 0.6 m/s

20 Hip

v = 0.8 m/s v = 1.0 m/s

0

v = 1.2 m/s v = 1.4 m/s

Ext −20 Flex 80

Knee

60 40 20 0 Ext −20 Flex 20 Ankle

10 0 −10 −20 Ext −30

0

20

40 60 % Gaitcycle

80

100

Fig. 5. Changes in the sagittal plane kinematics of the lower limb joints with varying walking speed 1 Raw Data Regression 95% Konf Int

0.9

α

str

0.8

0.7

0.6 r2 = 0.24 0.5

0.4

0.2

0.4

0.6

0.8 1.0 1.2 1.4 Walking Speed [m/s]

1.6

1.8

2.0

Fig. 6. Dynamic stability of treadmill walking as described by the averaged Floquet multiplier (αstr ) plotted over walking speed

354

M. Schablowski and H.J. Gerner

line delineates the quadratic regression curve over the range of speeds covered by the measurements. The regression explains 24% of variance in the raw data (r2 = 0.24). Hence, the relationship is not very strong, yet nevertheless signiﬁcant (p < 0.05) indicating a certain dependence of this stability measure on walking speed with a weak local minimum at intermediate speeds. Local Divergence Exponents The course of local dynamic stability over walking speed for the sagittal joint angles of the lower limbs is depicted in Fig. 7. The long term LDE (λLT ) are shown in the left column, the short term values (λST ) are displayed in the right column. Quadratic regression curves are shown; the r2 -values give the proportion of explained variance in the data. The individual graphs contain the accumulated values for both the left and the right side joint angles. In general, the regression curves indicate that the two estimates of local dynamic stability depend on walking speed. However, there appears to exist a signiﬁcant diﬀerence in speed dependence on the two time scales: the λST express a minimum at intermediate speeds for the hip joint and no clear tendency in the two remaining joints. The λLT , which describe the long term stability, continuously increase towards larger speed values. 1.5

0.2 λLT Hip

r2 = 0.56

0.1

1 λST Hip

0.3

0.5

0 2

r = 0.21 Raw Data

−0.1

0

Mean

0.3

1.5

95% Confidence

1 λST Knee

λLT Knee

0.2 0.1

0.5

0 2

r = 0.38

2

r = 0.10

−0.1

0

0.3

1.5 2

λLT Ankle

0.2

1

0.1 0

−0.1

λST Ankle

r = 0.14

0.5 r2 = 0.46 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 2.0 Walking Speed [m/s]

0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 2.0 Walking Speed [m/s]

0

Fig. 7. Speed dependence of local divergence exponents including quadratic regression curves

Dynamic Stability of Treadmill Walking

355

1.3 Raw data Mean 95% Confidence

1.2

λST Sacral_Vel

1.1 1 0.9 0.8 0.7 0.6

r2 = 0.47

0.5 0.4

0.2

0.4

0.6

0.8 1.0 1.2 1.4 Walking Speed [m/s]

1.6

1.8

2.0

Fig. 8. Speed dependence of short term stability expressed in normal walking 0.1 r2 = 0.47

λLT Sacral_Vel

0.08 0.06 0.04 0.02 0 −0.02

Raw data Mean 95% Confidence

0.2

0.4

0.6

0.8 1.0 1.2 1.4 Walking Speed [m/s]

1.6

1.8

2.0

Fig. 9. Speed dependence of long term stability expressed in normal walking

In order to further investigate these diﬀerent tendencies and to allow for comparison with the results obtained in [25], the local divergence exponents here were also calculated for the time derivatives of the sacral marker position on both time scales. The main reason for using these velocity data instead of the position is to eliminate drift eﬀects in the absolute marker positions during walking on the treadmill [25]. The course of the regression curves in the Figs. 8 and 9 conﬁrms the diﬀerences in speed dependence for the two diﬀerent time scales: whereas the speed dependence of short term stability expresses a distinct minimum at intermediate speeds (≈ 1.3 m/s), the long term values show a clear increase towards higher speeds. The percentage of variance explained in both cases

356

M. Schablowski and H.J. Gerner

reaches almost 50%. Since the basic dynamic properties of the system are in principal not dependent on the measurement modalities chosen for analysis, this supports the idea of two diﬀerent eﬀects of speed on stability of walking. 3.3 Experimental Comparison of the Two Measures The two measures for dynamic stability applied here are both based on a geometrical point of view with respect to the system dynamics. However, as shown in Sect. 2.1, there are several diﬀerences in the way the measures are deﬁned. The fact that they both aim at quantifying dynamic stability of walking calls for an analysis of the relation between their values. Table 1 gives a list of the linear correlations between the modiﬁed Floquet multipliers introduced in Sect. 2.1 and the Local Divergence exponents. The individual cells of the table contain the correlation coeﬃcients coded as gray values. Darker gray cells contain large positive correlations, lighter gray cells display negative values. Statistically signiﬁcant coeﬃcients are printed in bold font, non signiﬁcant correlations stand in parentheses. Table 1. Linear Correlations between the diﬀerent measures for dynamic stability

Leaving out λST Knee there are essentially three blocks containing correlations with diﬀerent signs: the features x1 to x5 are weakly positively correlated with each other and negatively with the features x6 to x9 . The latter are strongly correlated to each other. The absolute value of the correlation coeﬃcients increases signiﬁcantly from the ﬁrst to the third block.

4 Discussion The results presented in the previous section comprise two measures of dynamic stability of walking. Theoretical comparison of the two measures as described in Sect. 2 illustrates diﬀerences which arise from the mathematical deﬁnitions of the two measures. However, the diﬀerence in speed dependance

Dynamic Stability of Treadmill Walking

357

shown in Sect. 3 demonstrates that these theoretical diﬀerences are not fully replicated by the experimental application. Whereas theory suggests grouping the measures according two their mathematical deﬁnition – Floquet versus Lyapunov –, the experiments performed here emphasize the importance of the time scale on which the measures are based. This ﬁnding is supported by the data listed in Table 1 as follows: since block x1 – x5 mainly consists of the λST which are calculated over the ﬁrst stride, it makes sense to assume that this block indicates short term stability. αstr is positively correlated with λST Hip and λST V Sacral V el. This coincides well with the fact that this measure is based on the assumption of a strictly periodic system (see Sect. 2.1) and consequently measures short term stability as well. The second block which consists of the λLT describes some sort of long term stability that is captured over several strides. The diﬀerences in modeling between αstr and λ seem to make no diﬀerence here. From this, it may be argued that capturing the full kinematics of the entire movement system may be not necessary to describe dynamic stability of walking. Both the measures describing short term stability of periodic walking motion over one stride (αstr and λST ) exhibit a tendency towards maximal stability at intermediate speeds, whereas long term stability seems to decrease continuously towards faster speeds2 . The minima in αstr and λST,Hip agree with the results from [26] where a maximum in stability for the head marker trajectory at the preferred walking speed was found. The average self-selected speed of the subjects analyzed here was around 1.02 m/s which does not coincide with the location of the minimum in λST for the hip (≈ 1.4 m/s). This together with the fact that the quadratic speed dependency is only weak corresponds well with the conclusion of the authors in [26] that (short term) stability is only one criterion among others determining the self selected walking speed. The signiﬁcant increase in λLT with walking speed indicates increasing instability during faster gaits. This corresponds well with the interpretation that the local divergence exponents quantify the capability of the locomotor system to compensate for slight irregularities from stride to stride. This capability is increasingly challenged at higher gait speeds which leads to decreased stability of motion. The highest speed values adopted by the subjects in this study are close to the walk-run-transition (mean value vT M = 1.89±0.08 m/s, compared e.g. to vT M = 2.17 m/s in [27]). According to the theory of nonlinear dynamical systems, this transition can be interpreted as switching to a diﬀerent system attractor. Therefore, following the reasoning illustrated in [28], it can be hypothesized that this increasing instability is an important factor in changing gaits [28, 29, 27]. In contradiction to the results obtained here, the authors in [25] found a continuous increase on both time scales. The authors investigated the derivative with respect to time of the three dimensional trajectory of a 2

Larger values of λ indicate decreasing stability.

358

M. Schablowski and H.J. Gerner

marker in the area of the sacral vertebral column. A potential reason for the diﬀerence in the results is the smaller range of speeds investigated there (vpref , vpref ± 20%, vpref ± 40%). The correlation between λLT and walking speed which was shown here also supports the results described in [7]. The authors demonstrated that older subjects show signiﬁcantly higher values in their local divergence exponents compared to a younger reference group. This was interpreted as an indicator of increased instability of walking with age. At the same time, the older group showed signiﬁcantly lower walking speeds, a fact that was not examined in more detail by the authors. The dependence of the local divergence exponents on walking speed, which was found here, underlines the explanation found by the authors that, in fact, aging leads to increased walking instability. This increase is most probably not only a consequence of the diﬀerence in walking speed [7]. An interesting question is why the speed dependence of the sacral marker velocities is much more pronounced than that of the joint angles. There are two possible explanations for this: on the one hand, the diﬀerences could be related to the way the stability measures are determined. There are several marker trajectories connected in a nonlinear way during the the calculation of the joint angles, which might lead to slight nonstationarities. On the other hand, it can be hypothesized that there are diﬀerent control mechanisms involved in controlling the position/velocity of the sacral marker in comparison to the joint angles. The fact that virtually all values of λ have positive values and as such indicate chaos does not necessarily mean that, gait itself is chaotic [13]. As pointed out in [30] stochastic noise can indicate chaos where in fact there is none. A sound investigation of this question can be undertaken using the method of surrogate data [30].

5 Conclusion The results presented here complement data from previous studies [25]. Two diﬀerent mechanisms regarding dynamic stability of locomotion seem to exist: the minimum in λST at intermediate speed values suggests that short term stability may serve as an optimization criterion in the locomotor system. The increased local dynamic stability (λLT ) towards slower speeds may explain the tendency towards slower speeds in many gait pathologies. On the other hand, the increasing instability at higher speeds may be one reason for the transition from walking to running.

Acknowledgment The author would like to thank all the subjects who volunteered in this study.

Dynamic Stability of Treadmill Walking

359

References [1] R. Full, T. Kubow, J. Schmitt, P. Holmes, and D. Koditschek, “Quantifying dynamic stability and maneuverability in legged locomotion,” Integrative and Comparative Biology, vol. 42, pp. 149–157, 2002. [2] D. Krebs, D. Goldvasser, J. Lockert, L. Portney, and K. Gill-Body, “Is base of support greater in unsteady gait?” Physical Therapy, vol. 82, no. 2, pp. 138–47, Feb 2002. [3] M. Sliwinski, S. Sisto, M. Batavia, B. Chen, and G. Forrest, “Dynamic stability during walking following unilateral total hip arthroplasty,” Gait and Posture, vol. 19, no. 2, pp. 141–7, Apr 2004. [4] M. Hahn and L. Chou, “Can motion of individual body segments identify dynamic instability in the elderly?” Clinical Biomechanics, no. 8, pp. 737–44, Oct 2003. [5] M. Vukobratovic and J. Stepanenko, “On the stability of anthropomorphic systems,” Mathematical Biosciences, vol. 15, pp. 1–37, 1972. [6] Y. Hurmuzlu, C. Basdogan, and D. Stoianovici, “Kinematics and dynamic stability of the locomotion of post-polio patients,” Journal of Biomechanical Engineering, vol. 118, no. 3, pp. 405–11, Aug 1996. [7] U. H. Buzzi, N. Stergiou, M. J. Kurz, P. A. Hageman, and J. Heidel, “Nonlinear dynamics indicates aging aﬀects variability during gait.” Clinical Biomechanics, vol. 18, no. 5, pp. 435–443, Jun 2003. [8] J. Dingwell et al., “Local dynamic stability versus kinematic variability of continuous overground and treadmill walking,” Journal of Biomechanical Engineering, vol. 123, no. 1, pp. 27–32, Feb 2001. [9] D. Sternad, “Debates in dynamics: A dynamical systems perspective on action and perception,” Human Movement Science, vol. 19, pp. 407–423, 2000. [10] J. Guckenheimer and P. Holmes, Nonlinear Oscillations, Dynamical Systems, and Bifurcations of Vector Fields, 3rd ed., ser. Applied Mathematical Sciences, F. John, J. Marsden, and L. Sirovich, Eds. New York ; Berlin ; Heidelberg ; Tokyo [u.a.]: Springer-Verlag, 1990, vol. 42. [11] Y. Hurmuzlu, C. Basdogan, and J. Carollo, “Presenting joint kinematics of human locomotion using phase plane portraits and poincare maps,” Journal of Biomechanics, vol. 27, no. 12, pp. 1495–9, Dec 1994. [12] Y. Hurmuzlu and C. Basdogan, “On the measurement of dynamic stability of human locomotion,” Journal of Biomechanical Engineering, vol. 116, no. 1, pp. 30–36, 1994. [13] J. Dingwell et al., “Nonlinear time series analysis of normal and pathological human walking,” Chaos, vol. 10, no. 4, pp. 848–863, Dec 2000. [14] M. Rosenstein, J. Collins, and C. DeLuca, “A practical method for calculating largest lyapunov exponents from small data sets,” Physica D, vol. 65, pp. 117– 134, 1993. [15] A. Wolf, J. Swift, H. Swinney, and J. Vastano, “Determining lyapunov exponents from a time series,” Physica D, vol. 16, pp. 285–317, 1985. [16] J. Theiler, “Spurious dimensions from correlation algorithms applied to limited time-series data,” Physical Reviews A, vol. 34, pp. 2427–2432, 1986. [17] J. Dingwell, “Variability and nonlinear dynamics of continous locomotion: Applications to treadmill walking and diabetic peripheral neuropathy,” PhD Thesis, The Penyslvania State University, 1998.

360

M. Schablowski and H.J. Gerner

[18] J. Dingwell et al., “Increased variability of continuous overground walking in neuropathic patients is only indirectly related to sensory loss,” Gait and Posture, vol. 14, no. 1, pp. 1–10, Jul 2001. [19] M. Kadaba, H. Ramakrishnan, and M. Wootten, “Measurement of lower extremity kinematics during level walking,” Journal of Orthopaedic Research, vol. 8, no. 3, pp. 383–92, May 1990. [20] Motion Analysis, OrthoTrak5.0 – Gait Analysis Software Reference Manual, 3617 Westwind Blvd., Santa Rosa, CA 95403, 2001. [Online]. Available: http://www.motionanalysis.com/ ¨ [21] L. Larsson, P. Odenrick, B. Sandlund, P. Weitz, and P. Oberg, “The phases of the stride and their interaction in human gait,” Scandinavian Journal of Rehabilitation Medicine, vol. 12, no. 3, pp. 107–12, 1980. ¨ ¨ [22] T. Oberg, A. Karsznia, and K. Oberg, “Basic gait parameters: reference data for normal subjects, 10-79 years of age.” Journal of Rehabilitation Research and Development, vol. 30, no. 2, pp. 210–223, 1993. [23] ——, “Joint angle parameters in gait: reference data for normal subjects, 10– 79 years of age.” Journal of Rehabilitation Research and Development, vol. 31, no. 3, pp. 199–213, Aug 1994. [24] M. Schablowski, “Speed dependence of local dynamic stability of treadmill walking,” in Proceedings of the 3rd International Symposium on Adaptive Motion in Animals and Machines (AMAM), Technische Universit¨ at Illmenau, 2005. [25] C. Marin and J. Dingwell, “Local dynamic stability improves at slower walking speeds,” in Proceedings fo the 28th Annual Meeting of the American Society of Biomechanics. ASB, 2004. [26] K. Holt, S. Jeng, R. Ratcliﬀe, and J. Hamill, “Energetic cost and stability during human walking at the preferred stride frequency,” Journal of Motor Behavior, vol. 27, no. 2, pp. 164–178, 1995. [27] F. Diedrich and W. Warren, “Why change gaits? dynamics of the walk-run transition,” Journal for Experimental Psychology: Human Perception and Performance, vol. 21, no. 1, pp. 183–202, Feb 1995. [28] J. Brisswalter and D. Mottet, “Energy cost and stride duration variability at preferred transition gait speed between walking and running,” Canadian Journal of Applied Physiology, vol. 21, no. 6, pp. 471–80, Dec 1996. [29] L. Li, “Stability landscapes of walking and running near gait transition speed,” Journal of Applied Biomechanics, vol. 16, no. 4, Nov 2000. [30] J. Theiler, S. Eubank, A. Longtin, B. Galdrikian, and J. Farmer, “Testing for nonlinearity in time series: The method of surrogate data,” Physica D, vol. 58, pp. 77–94, 1992.

Simple Feedback Control of Cockroach Running J. Schmitt Oregon State University, Corvallis, Oregon [email protected]

Summary. The spring loaded inverted pendulum model (SLIP) has been shown to accurately model sagittal plane locomotion for a variety of legged animals. Tuned appropriately, the model exhibits passively stable periodic gaits using either ﬁxed leg touchdown angle or swing-leg retraction protocols. In this work, we investigate the relevance of the model in insect locomotion and develop a simple feedback control law to enlarge the basin of stability and produce stable periodic gaits for both the point mass and rigid body models. Control is applied once per stance phase through appropriate choice of the leg touchdown angle. The control law is unique in that stabilization is achieved solely through direct observation of the leg angle and body orientation, rather than through feedback of system positions, velocities, and orientation.

1 Introduction The spring loaded inverted pendulum (SLIP) [1, 2, 3, 4] has emerged as a template of locomotion dynamics in the vertical plane for a large number of animals. In the model, the combination of legs animals use during each stance phase is idealized as a single eﬀective leg represented by an elastic spring. Experimental research on animal locomotion has shown that running animals use multiple legs as one [5], and that the resulting body motion is well represented by this simple template [1, 2, 6, 7]. For suitable model parameters, both the rigid body and point mass models produce self-stabilizing periodic gaits, with or without the inclusion of simple feedforward control methodologies [8, 9, 10]. Much of the previous research on the SLIP model has utilized a ﬁxed angle leg reset policy, where the leg touchdown angle remains constant for each stance phase. While this leg touchdown protocol produces stable periodic gaits for the point mass and rigid body models, the basin of stability for the rigid body model remains quite small and stability is only achieved for suitably tuned model parameters [8]. Additionally, use of a similar ﬁxed angle leg reset protocol in a three dimensional spatial SLIP model produces only unstable periodic gaits [11]. In this work, we investigate the performance of

362

J. Schmitt

the two dimensional SLIP model in modeling insect locomotion and, looking forward to developing simple stabilizing controllers for the spatial SLIP model, construct a simple feedback control law to expand the basin of stability for periodic orbits of both the point mass and rigid body SLIP models. The paper is structured as follows. In Sect. 2, we brieﬂy review the rigid body vertical plane SLIP formulation of [8]. While we primarily focus on the point mass model, extensions to motions of the rigid body model are also considered in Sect. 4.2. In Sect. 3, we investigate the stability of periodic gaits of the SLIP model with model parameters similar to those of the cockroach Blaberus discoidalis. We ﬁnd that the ﬁxed angle leg reset policy utilized in previous works produces predominantly unstable periodic gaits for these model parameters. These results, in conjunction with the stability results of the spatial SLIP model for a similar leg touchdown protocol, prompt an investigation into the stability properties of alternate leg touchdown protocols. Speciﬁcally, we begin by analyzing a leg touchdown protocol that places the next leg down at the same angle, relative to the inertial frame, that the previous leg was lifted. Neglecting gravity during the stance phase, we show that such a protocol produces neutrally stable period two as well as period one orbits. Since one of the eigenvalues of the period one orbits is negative one, continuity arguments suggest that leg placement protocols between these extremes will produce stable periodic gaits. We therefore introduce a simple, adaptive leg touchdown angle control law in Sect. 4.1 that connects both this leg touchdown protocol as well as the ﬁxed angle leg reset policy, and show that the control law improves the stability properties of periodic gaits. We brieﬂy consider the stability of periodic orbits of the rigid body model under a similar control law in Sect. 4.2, and numerically show that while inclusion of this control law produces stable gaits, periodic gaits are only necessarily achieved by incorporating delay feedback control into the control law.

2 Review of SLIP Model Formulation The SLIP model, illustrated in Fig. 1, consists of a rigid body of mass m and moment of inertia I. A pair of legs are attached at a frictionless pin joint P in the body, displaced a distance d from the center of mass, where d can take either sign. While the SLIP model has primarily been used in investigating the motions of larger animals, in this work we examine its relevance in insect locomotion, in particular the locomotion of the cockroach Blaberus discoidalis. Cockroaches run in an alternating tripod gait, with three legs down during each stance phase. Experiments have shown that the forces produced by these legs during the stance phase are well represented by a single eﬀective leg. Since the mass of the legs of the insect comprise less than 6% of the total mass, we therefore model the tripod of legs by a single, massless eﬀective leg represented by a linear, elastic spring of nominal length l.

Simple Feedback Control of Cockroach Running

e

e2

2

P d η

z

ζ

ψ β y

(a)

363

θ

e1

d z

P δ n+1

vn+1

e

1

r β

LO

TD

n+1

n

y

(b)

rF

Fig. 1. SLIP rigid body model formulation, illustrating coordinate systems and relevant quantities at (a) leg lift-oﬀ and (b) leg touchdown

If both legs are attached at the same point P in the body, a full stride consists of a stance and ﬂight phase, since left and right stance phases are indistinguishable. The stance phase begins when the leg, extended at its nominal length l, touches the ground at an angle βnT D with respect to the inertial frame. Superscripts of TD and LO denote values at touchdown and lift-oﬀ respectively, whereas subscripts identify the speciﬁc stance phase. The foot placement remains ﬁxed during the stance phase and is represented by a moment free pin joint. Under the inﬂuence of gravity and its own momentum, the body moves forward in the y direction during the stance phase, compressing and extending the elastic leg. When the leg returns to its nominal length, it is lifted from an angle βnLO with respect to the inertial frame and a ﬂight phase ensues. Simple ballistic dynamics govern the ﬂight phase, and the next stance TD with phase begins when the leg touches the ground, placed at an angle βn+1 respect to the inertial frame. While feedforward control is required to place each leg in anticipation of the next stance phase, no energy is required to move the leg to the prescribed position since the leg has no mass. As a result, the system is passive and energy is globally conserved, since no impacts or impulses occur. The equations of motion for the stance phase are derived in [8] and summarized here, implemented with a linear spring leg: l mζ¨ = mζ ψ˙ 2 − mg cos(ψ) − k(1 − )(ζ + d cos(θ + ψ)) η l 2mζ ζ˙ ψ˙ + mζ 2 ψ¨ = mg sin(ψ) + k(1 − )dζ sin(θ + ψ) (1) η l I θ¨ = k(1 − )dζ sin(θ + ψ) η where η, ζ, ψ, θ, and k denote the leg length, distance from the foot placement to the center of mass, angle ζ makes with the vertical inertial axis, body rotation, and spring stiﬀness respectively. The ﬂight phase dynamics are governed by simple ballistic dynamics which may be integrated to yield

364

J. Schmitt

1 y(t) = y LO + y˙ LO t, z(t) = z LO + z˙ LO t − gt2 , θ(t) = θLO + θ˙LO t . 2

(2)

The composition of the stance and ﬂight phase dynamics result in a piecewise-holonomic system. While systems with piecewise-holonomic constraints can display asymptotic stability [12], they are often best described in terms of partial asympototic stability. In these cases, the corresponding periodic motions typically exhibit some neutral eigendirections (with eigenvalue = 0 or Floquet multiplier = 1) often associated with conserved quantities or symmetries such as energy conservation or rotational invariance. As in previous analyses for the conservative horizontal and vertical plane models [8, 13], perturbations to stable gaits in the direction of the eigenvector(s) of these conserved quantities or symmetries do not grow nor decay, but result in the attainment of a new gait.

3 Periodic Gaits, Stability and Control Many prior SLIP model analyses focus on gait properties for a ﬁxed angle leg reset model, where the leg is reset to its original touchdown angle at the beginning of each stance phase. For model parameter ranges consistent with larger animals, these studies illustrate the ability of the model to produce passively stable gaits. While we initially examine gaits produced for a ﬁxed angle leg reset model with parameters set to those typical of the roach Blaberus discoidalis, we subsequently investigate simple feedback control laws that prescribe the leg touchdown angle and enhance stability. In all cases, simulations are developed and performed using the RungeKutta integrator ode45 available in Matlab. As in previous work [8, 14, 15], we determine periodic orbits and their stability through use of a Poincar´e map [16], with a Poincar´e section deﬁned at the instant of leg touchdown. Fixed points of the mapping represent periodic gaits in the continuous system, and are identiﬁed by a Newton-Raphson iteration. As illustrated in Fig. 1, we simplify the stability analysis through the deﬁnition of our Poincar´e map in ˙ β), where v is the center terms of variables of a new coordinate frame (v, δ, θ, θ, of mass velocity, δ is the velocity heading angle measured clockwise from the inertial horizontal axis, θ is the body rotation and θ˙ is the angular velocity. Stability of ﬁxed points of the mapping is determined by examining the eigenvalues of the linearization of the Poincar´e map about the ﬁxed point. If any eigenvalue is greater than unity the periodic orbit is unstable and if all non-unity eigenvalues remain within the unit circle, the periodic orbit is stable. 3.1 Fixed Angle Leg Reset Gaits We begin by investigating the point mass SLIP model with parameters similar to those of the cockroach Blaberus discoidalis: spring stiﬀness (k) of 20 N/m,

Simple Feedback Control of Cockroach Running

365

1 0.9 0.8 0.7

δ (rad)

0.6 0.5 0.4 0.3 0.2 0.1 0 0.1

0.2

0.3

0.4

0.5 0.6 v (m/s)

0.7

0.8

0.9

1

Fig. 2. Point mass SLIP gait families (vnT D , δnT D ) for βnT D = 1.1 − 1.3. Gait stability is indicated by dotted and solid lines for unstable and stable gaits respectively. Model parameters used in calculating the gaits are similar to those of Blaberus discoidalis, as described in the text. Gait families for increasing βnT D are obtained as one moves from right to left in the plot

leg length (l) of 0.015 m, body mass (m) of 0.0025 kg, and a leg touchdown angle (βnT D ) between 1.1 − 1.3 radians. These parameter values are chosen to produce reasonable leg compressions (less than 50%) during the stance phase, as well as to match experimental stride length and frequency results [17]. While the theoretical results neglect gravity during the stance phase so that angular momentum is conserved, gravity is included in both stance and ﬂight phases in all simulations conducted in this work. In this section, we consider only the point mass SLIP model with d = 0; extensions to the rigid body model with d = 0 and an adaptive control law are presented in Sect. 4.2. Using Newton-Raphson routines in conjunction with the model simulation, we obtain a one parameter family of periodic gaits, depending upon the body touchdown velocity, v. The gait family is initially obtained for a nominal value of βnT D = 1.20, which best matches the experimental stride length and frequency results. Gait families for values of βnT D = 1.1 − 1.3 are subsequently obtained to see if changes in gait stability occur for these alternate values, even though stride length and frequency results do not necessarily match those observed experimentally. As illustrated in Fig. 2, almost all periodic gaits obtained over this range of leg touchdown angles are unstable. A small range of stable periodic gaits exist for leg touchdown values between 1.1−1.11, although the minimum touchdown speeds obtained in these instances exceed the preferred operating speed of 0.25 m/s of Blaberus discoidalis.

366

J. Schmitt

3.2 Alternate Leg Angle Touchdown Protocols The stability properties of the SLIP model with parameters similar to those of Blaberus discoidalis, in conjunction with the instability results of the ﬁxed angle leg reset policy in the three dimensional spatial SLIP model, prompt an investigation into alternate leg touchdown protocols. A problem with the ﬁxed angle leg reset policy is the propensity for the model to trip. During each ﬂight phase, the body must attain a height suﬃcient to place the next leg down at the constant touchdown angle. Many initial conditions produce gaits that do not satisfy this condition, leading to gaps in the mapping where no periodic gaits can exist, as found in [8]. A simple means of eliminating these gaps is to TD = βnLO . Even place the next leg down at the previous leg liftoﬀ angle, βn+1 in the absence of a ﬂight phase, a model utilizing this leg touchdown protocol on a level surface will not stumble, although the forward velocity may be insuﬃcient to compress the elastic spring and move the body forwards past the foot placement point. TD LO = βn Properties of Period Two Gaits with βn+1

Implementing a leg touchdown protocol where the new leg touchdown angle equals the previous leg lift-oﬀ angle primarily produces asymmetric period TD TD = βnT D = βn+1 , although period one gaits also exist, due to two gaits, βn+2 the symmetry of leg touchdown and lift-oﬀ angles in such gaits, βnT D = βnLO . We construct a Poincar´e map, through the use of conservation laws, to determine necessary conditions for periodic orbits, as well as stability properties of those orbits. Analytically, as in previous studies [11, 8], we assume that leg forces dominate gravity forces during the stance phase. As in [8], we use a mixed approximation in computing the stance map, where we concurrently neglect the eﬀect of gravity during stance to retain conservation of angular momentum, but retain the eﬀect of gravity in computing the velocity mapping to enforce energy conservation. The variations in the leg angle swept during stance, ∆ψ, therefore reduce to those examined in the lateral leg spring model [14, 13], and the analyses conducted therein apply. We compute the touchdown velocity mapping directly from energy conservation, with the zero potential energy level deﬁned at the initial touchdown height as TD 2 m(vn+2 ) m(vnT D )2 TD = + mgl(sin(βn+2 ) − sin(βnT D )) 2 2 1 TD T D )) . vn+2 = (vnT D )2 + 2gl(sin(βnT D ) − sin(βn+2

(3) (4)

We compute the velocity heading angle mapping using conservation of linear momentum, conservation of angular momentum and conservation of energy. Since the next leg touchdown angle equals the previous leg lift-oﬀ angle in

Simple Feedback Control of Cockroach Running

367

this protocol, conservation of energy between the lift-oﬀ and touchdown conTD TD LO = vnLO and vn+2 = vn+1 . Conservation of linear ditions necessarily yields vn+1 momentum during the ﬂight phase TD TD vn+1 cos(δn+1 ) = vnLO cos(δnLO )

(5)

TD TD LO LO vn+2 cos(δn+2 ) = vn+1 cos(δn+1 )

(6)

TD TD LO therefore yields δn+1 = −δnLO and δn+2 = −δn+1 . Since gravity is neglected during stance, angular momentum is conserved during each stance phase

ml2 vnT D sin(βnT D − δnT D ) = ml2 vnLO sin(βnLO + δnLO ) TD TD TD LO LO LO sin(βn+1 − δn+1 ) = ml2 vn+1 sin(βn+1 + δn+1 ). ml2 vn+1

(7) (8)

Utilizing the relationships for the velocities, velocity heading angles and leg touchdown protocol developed above, however, we ﬁnd that the magnitude of angular momentum remains constant across all stance phases for this protocol TD TD TD ml2 vnLO sin(βnLO + δnLO ) = ml2 vn+1 sin(βn+1 − δn+1 ) LO LO LO TD TD TD ml2 vn+1 sin(βn+1 + δn+1 ) = ml2 vn+2 sin(βn+2 − δn+2 ).

(9) (10)

As a result, we ﬁnd TD TD TD ml2 vn+2 sin(βn+2 − δn+2 ) = ml2 vnT D sin(βnT D − δnT D )

(11)

such that the velocity heading angle map can be expressed as TD TD δn+2 = βn+2 − sin−1 (

vnT D sin(βnT D − δnT D )) . TD vn+2

(12)

Stance phase geometry and the leg touchdown protocol produce a relationship for the leg touchdown angle mapping as TD LO TD βn+2 = βn+1 = π − ∆ψ2 − βn+1

= π − ∆ψ2 − (π − ∆ψ1 − βnT D ) = ∆ψ1 − ∆ψ2 + βnT D

(13)

where ∆ψ1 and ∆ψ2 represent the leg angle swept during the ﬁrst and second stance phase respectively. The full period two Poincar´e map is therefore comprised by equations (4), (12), and (13), and a periodic orbit for this mapping requires ∆ψ1 = ∆ψ2 . Ignoring gravity, the formulas for each ∆ψ can be constructed from conservation of angular momentum and energy during the stance phase, as in [14], to yield

368

J. Schmitt

l 1

∆ψ1 = ζb1

ζ

l 1

∆ψ2 = ζb2

ζ

2vnT D l sin(βnT D − δnT D )dζ

(14)

((vnT D )2 − k/m(ζ − l)2 )ζ 2 − l2 (vnT D )2 sin2 (βnT D − δnT D ) TD TD TD 2vn+1 l sin(βn+1 − δn+1 )dζ T D )2 sin2 (β T D − δ T D ) (a − k/m(ζ − l)2 )ζ 2 − l2 (vn+1 n+1 n+1

TD 2 TD ) + 2gl(sin(βn+1 ) − sin(βnT D )) a = (vn+1

(15) (16)

where ζb1 , ζb2 are the largest positive roots of the equation(s) 2 2 ml2 (vnT D )2 sin2 (βnT D − δnT D ) + k(ζb1 − l)2 ζb1 − mζb1 (vnT D )2 = 0

(17)

TD 2 TD TD 2 ) sin2 (βn+1 − δn+1 ) + k(ζb2 − l)2 ζb2 ml2 (vn+1 = 1 . (18) 2 [(v T D )2 + 2gl(sin(β T D ) − sin(β T D ))] mζb2 n n+1 n+1

Here, ζb1 , ζb2 represent the distance between the center of mass and the foot placement point when ζ˙ = 0. For the point mass system, in which ζ = η, this represents the state of maximal spring compression. For consistency, gravity is included in the energy calculation at leg touchdown, but not during the stance phase, in the formulation of ∆ψ2 . Conservation of energy between the start of the ﬁrst and second stance phases yields a = (vnT D )2 , and from equations TD TD TD sin(βn+1 −δn+1 ) = vnT D sin(βnT D −δnT D ). Substituting these (7–9), we ﬁnd vn+1 values into the expression for ∆ψ2 in (15) shows that the integrands of (1415) are equivalent. Similar arguments can be used to show that equations (17) and (18) are equal, resulting in ζb2 = ζb1 . By ignoring gravity during stance, we ﬁnd that ∆ψ1 necessarily equals ∆ψ2 such that the period two mapping reduces to TD vn+2 = vnT D

(19)

TD δn+2 = δnT D

(20)

TD βn+2 = βnT D .

(21)

Therefore, under these assumptions, this leg touchdown protocol necessarily produces period two gaits. The eigenvalues of the mapping are all unity, indicating that these gaits are neutrally stable. Numerical simulations verify this result for a large number of gaits, even when gravity is included during the stance phase. The limited eﬀect of gravity during the stance phase observed here is restricted to this particular leg touchdown protocol and model. Specifically, this leg touchdown protocol ensures, through conservation of energy and conservation of linear momentum, that orbits of the system are reﬂectionsymmetric about the midpoint of the ﬂight phase. As a result, while angular momentum is not conserved during either stance phase, the net angular impulse delivered by gravity during the ﬁrst stance phase is counteracted by an equal and opposite net angular impulse during the second stance phase. For

Simple Feedback Control of Cockroach Running

369

other leg touchdown protocols, especially those relating to period one orbits, gravity can have a signiﬁcant eﬀect on orbits and orbital stability, as discussed in [8, 18]. TD LO = βn Properties of Period one Gaits with βn+1 TD Neutrally stable period one gaits, with βn+1 = βnT D = βnLO , exist as a subset of the family of period two gaits analyzed previously. The eigenvalues of the period two mapping in this case represent the square of each eigenvalue in the period one map, suggesting that eigenvalues in the associated period one mapping are ±1. We pursue a stability analysis in this section to determine if one of the eigenvalues is negative one for this leg touchdown protocol. If an eigenvalue is negative one, then continuity suggests that a leg touchdown TD TD = βnT D and βn+1 = βnLO that protocol exists between the extremes of βn+1 produces stable periodic gaits. TD = βnLO The period one Poincar´e map for the leg touchdown protocol βn+1 is a composition of the stance and ﬂight phase maps. The full stride map is constructed similarly to the period two map, using conservation of energy, conservation of linear momentum and conservation of angular momentum to yield 1 TD = (vnT D )2 + 2gl(sin(βnT D ) − sin(∆ψ + βnT D )) (22) vn+1 TD = π − ∆ψ − 2βnT D + δnT D δn+1 TD βn+1

= π − ∆ψ −

βnT D

(23)

.

(24)

As in [8], consistency requires that we neglect gravity in computing the liftoﬀ velocity in the heading angle map, since gravity is neglected in the computation of the leg sweep angle, ∆ψ, during stance. Neglecting gravity in this computation yields vnLO = vnT D , resulting in the heading angle map presented above. A period one orbit therefore requires ∆ψ = π − 2βnT D , which implies through the stance phase geometry that βnLO = βnT D . We construct the Jacobian of the Poincar´e map evaluated at the ﬁxed point as   ∂∆ψ ∂∆ψ ∂∆ψ 1 + b ∂v b ∂δ b(2 + ∂β TD TD TD ) n n n     ∂∆ψ ∂∆ψ ∂∆ψ   1 − −(2 + ) − (25) T D T D T D ∂vn ∂δn ∂βn     ∂∆ψ ∂∆ψ ∂∆ψ − ∂δ −1 − ∂β − ∂v TD TD TD n

n

n

where b=

gl cos(βnT D ) . vnT D

(26)

We calculate the eigenvalues of the Jacobian, evaluated at the ﬁxed point, in the Appendix as

370

J. Schmitt

λ1,2 = 1 λ3 = −1 −

(27) ∂∆ψ ∂∆ψ ∂∆ψ − TD + b TD ∂βnT D ∂δn ∂vn

(28)

where ∆ψ is computed as in equation (14). While ignoring gravity during the stance phase enables ∆ψ to be computed analytically, as in [14], it results in a complex expression in terms of incomplete elliptic integrals. As in [13], we instead utilize the Schwind-Koditschek approximation [18] to compute ∆ψ and the associated derivatives. Using this approximation, the computation of ∆ψ can be approximated by 2lvnT D sin(βnT D − δnT D )(l − ζb ) (29) ∆ψ = 1 2 T D 2 2 2 2 T D 2 T D T D ˆ ˆ ˆ ζ ((vn ) − k/m(ζ − l) )ζ − l (vn ) sin (βn − δn ) 3ζb + l . ζˆ = 4

(30)

While straightforward, computing the derivatives required in the eigenvalue expression is lengthy and left to the Appendix. Using the ﬁxed points computed for the gait family with βnT D = 1.2, we numerically compute the third eigenvalue using equation (28) and the relationships for the derivatives detailed in the Appendix. This computation reveals that the third eigenvalue has a maximum deviation of 10% from −1 across all the gaits of the gait family. Purely numerical computation of the Jacobian and associated eigenvalues for each gait reveal that the third eigenvalue is equal to −1 in each case. The diﬀerence between the numerical results and our analytical approximation results from neglecting gravity during stance as well as the approximation used for the computation of ∆ψ and the associated derivatives. However, taken together, these results show that periodic gaits utilizing this leg touchdown protocol are neutrally stable, with one eigenvalue equal to negative one.

4 An Adaptive Control Law The results of the analyses for the period two and period one orbits under the TD = βnLO are used to guide the construction of an leg touchdown protocol βn+1 adaptive control law for period one orbits. 4.1 Control of the Point Mass SLIP Model We begin by deriving a general period one Poincar´e map with varying leg touchdown angle, using conservation of energy, conservation of linear momentum during the ﬂight phase, and conservation of angular momentum during the stance phase as

Simple Feedback Control of Cockroach Running TD vn+1 = TD cos(δn+1 )=

1 0

371

T D )) (vnT D )2 + 2gl(sin(βnT D ) − sin(βn+1

(31)

(vnT D )2 + 2gl(sin(βnT D ) − sin(βnLO )) cos(g(· · · )) T D )) (vnT D )2 + 2gl(sin(βnT D ) − sin(βn+1

(32)

TD βn+1 = f (βnT D , βnLO , vnT D , δnT D )

(33)

where g(βnT D , vnT D , δnT D ) = βnLO + sin−1 (

vnT D sin(δnT D − βnT D )) vnLO

βnLO = π − ∆ψ − βnT D 1 vnLO = (vnT D )2 + 2gl(sin(βnT D ) − sin(βnLO )) .

(34) (35) (36)

TD = βnT D = βnLO While a period one orbit of the mapping must satisfy βn+1 TD and ∆ψ = π − 2βn , the stability of an orbit depends upon the leg touchdown protocol utilized in response to perturbations from the periodic orbit. TD = βnT D ) with model parameters Using the ﬁxed angle leg reset policy (βn+1 similar to those of Blaberus discoidalis primarily produces unstable gaits with a single eigenvalue greater than unity, whereas using the leg touchdown proTD = βnLO necessarily produces neutrally stable periodic gaits with tocol βn+1 an eigenvalue of −1. Continuity therefore suggests that the stability of the periodic gait should vary continuously for leg placement protocols between these two extremes. An adaptive control law that incorporates both of these leg touchdown protocols and satisﬁes the leg angle symmetry condition for a periodic orbit is given by TD = βnLO + c(βnT D − βnLO ) βn+1

(37)

where c is an arbitrary constant. Since gait symmetry requires βnT D = βnLO for a periodic gait, c only changes the stability of a gait, since a periodic gait remains periodic for any value of c. In particular, we expect that the unstable gaits observed in the SLIP model will stabilize as c is decreased from unity (the ﬁxed angle leg reset policy, βnT D = βnLO ), since the unstable eigenvalue must enter the unit circle and tend towards −1 as c tends towards zero (the TD = βnLO protocol). βn+1 We investigate the stability properties of periodic gaits utilizing this feedback control law using numerically computed periodic orbits and eigenvalues, with model parameters set to values similar to those of Blaberus discoidalis. We begin by analyzing the eﬀect changing c has on the eigenvalues of a representative gait family, as illustrated in Fig. 3. As illustrated in the second panel, decreasing c from unity shifts the non-unity eigenvalue curve downwards, stabilizing an increasing number of gaits until almost all gaits are stabilized for c = 0.1. We note, however, that as c decreases, some gaits, while still stable, will be relatively less stable as the eigenvalue determining stability tends towards negative one. This is illustrated in the third panel for a single orbit,

J. Schmitt 1

10

4

0.8

8

3

6

0.4

(a)

1

2

0.2 0

2

4

λ

0.6 λ

δ (rad)

372

0

0 0

0.5 v (m/s)

1

−2 0.2 (b)

0.4 v (m/s)

0.6

−1

0

0.5 c

1

(c)

Fig. 3. (a) SLIP periodic gait family and stability for β = 1.2, with other parameters set to those of Blaberus discoidalis, as described in the text (b) Gait family eigenvalues for c = 1, 0.7, 0.5, 0.3, 0.1, 0, with curves moving downwards as c decreases. (c) Eigenvalue variation as c varies from 1 to 0 for the periodic gait (*) identiﬁed in (a)

denoted with a * in the ﬁrst panel, where we observe that an almost linear eigenvalue variation occurs with changes in c. As c decreases from unity, the unstable gait becomes less unstable, stabilizes for c < .46, and becomes more stable until c reaches 0.23, at which point the eigenvalue begins to grow in magnitude as it approaches −1 at c = 0. As c passes through zero, we ﬁnd that the eigenvalue passes through −1, indicating a ﬂip bifurcation. Therefore, for c < 0, we ﬁnd that the period one orbit once again becomes unstable. We illustrate the performance of the control law, for c = 0.3, in Fig. 4. While a ﬁxed angle leg reset policy is incapable of producing a stable periodic orbit with these model parameters, we see that the inclusion of this simple control law leads to stabilization to a periodic orbit within several stance phases. How quickly a new stable gait is obtained depends upon how close the eigenvalue is to zero, which depends upon c and the model parameters. We note that variations in β T D utilized in stabilizing the orbit remain quite small. It is important to clarify the deﬁnition of asymptotic stability that is being utilized in this context. Since energy conservation and translational invariance naturally produce unity eigenvalues, we apply the deﬁnition of asymptotic stability used by Coleman et al. [19] and Coleman and Holmes [20]. In this less restrictive deﬁnition of asymptotic stability, a periodic orbit is asymptotically stable if perturbations result in the convergence to a nearby periodic gait. Perturbations to a periodic gait for our system and control result in the attainment of a new, stable gait, due to the partial asymptotic stability of these gaits and the coupling of motions introduced through the inclusion of the control law. Since the model remains conservative under this control law, we also illustrate the changes in gait stability for a constant energy surface, as illustrated in Fig. 5. The gait family illustrated therein is determined by choosing an en-

0.24

0.15

0.23

0.1

Vertical velocity (m/s)

Fore−aft velocity (m/s)

Simple Feedback Control of Cockroach Running

0.22 0.21 0.2 0.19 0.18

0

0.1

0.2

0.3

373

0.05 0 −0.05 −0.1

0.4

0

0.1

0.2

t

0.3

0.4

0.3

0.4

t

0.015 1.28

0.0145 (rad)

0.013 0.0125 0.012

1.26

TD

0.0135

β

z (m)

0.014

1.24

1.22 0

0.05 y (m)

0.1

0

0.1

0.2 t

Fig. 4. Stabilization of the SLIP model to a periodic gait for c = 0.3, with parameters are set to values similar to the cockroach Blaberus discoidalis. Plus signs denote the start and end of the stance phase and stars denote the start of the stance phase

ergy value and varying βnT D , thereby implicitly determining vnT D , and ﬁnding the associated δnT D that determines a periodic orbit by a Newton-Raphson iteration. The gaits for this constant energy represent the periodic orbits which the model can settle on, assuming no perturbations occur that change the energy of the system. As c decreases, we see that an increasing number of gaits stabilize, until all gaits are stable at c = 0.1. To be clear, however, the deﬁnition of asymptotic stability utilized in this work means that perturbations to a particular periodic orbit that do not result in a change in energy do result in the state asymptotically converging to a nearby periodic orbit of the gait family. Obviously, perturbations to a conservative system that result in a change in the total system energy necessarily result in the convergence to a new periodic gait that belongs to a diﬀerent gait family. Perturbations that would typically be encountered in practice would tend to fall into this latter category rather than the former. In fact, Altendorfer [21] notes that the small range of touchdown speeds evidenced for a constant energy surface necessitates changes in the total system energy if the controlled system is to exhibit a useful range of touchdown velocities. Achieving asymptotic stability in the more traditional sense to perturbations within the energy surface (i.e. returning to the original periodic orbit) requires a measure of knowledge of the periodic orbit that we wish to stabilize.

374

J. Schmitt

Fig. 5. Periodic gaits and stability for the SLIP model as a function of c, for a constant energy surface corresponding to an insect running at 0.25 m/s at a height of .014 m. All other parameters are similar to those of Blaberus discoidalis, as described in the text. (a) Periodic gaits for c = 0.0 − 1.0, with dotted and solid lines denoting unstable and stable periodic gaits respectively (b) Eigenvalue variation for the periodic gaits of panel (a)

The ﬁxed angle leg reset gaits that exhibit this property for larger animals accomplish this naturally since the leg touchdown protocol deﬁnes a desired leg touchdown angle and therefore a desired touchdown velocity, assuming a constant energy. We note that we have constructed a control law similar in structure and content to that presented in this work which includes a depenTD . This alternate control law produces dence on a desired touchdown angle, βdes asymptotically stable periodic gaits in the more traditional sense, as explained above. The performance of this alternate control law will be investigated in detail in a future work. Finally, returning to the original gait family plot of Fig. 2, we illustrate how this control law enlarges the basin of attraction, as illustrated in Fig. 6. As c decreases, an increasing number of periodic gaits on all gait family curves stabilize. Considering that perturbations to an orbit will typically occur transverse to the constant energy surface, this control law therefore enables the model to successfully recover from a large range of perturbations. As a result, implementing this control expands the stability basin of the SLIP model. 4.2 Control of the Rigid Body SLIP Model Displacing the leg attachment point from the center of mass in the SLIP model couples the translational and rotational dynamics and introduces pitching. Use of the same control law formulation, with leg angles deﬁned in the inertial frame, is examined brieﬂy in this section. Preliminary simulations of the rigid body SLIP model suggest that the control law implementation stabilizes

1

1

0.8

0.8

0.6

0.6

δ (rad)

δ (rad)

Simple Feedback Control of Cockroach Running

0.4 0.2 0

0.4 0.2

0

0.2

(a)

0.4 0.6 v (m/s)

0.8

0

1

0

0.2

0.4 0.6 v (m/s)

0.8

1

0

0.2

0.4 0.6 v (m/s)

0.8

1

(b) 1

1

0.8

0.8

0.6

0.6

δ (rad)

δ (rad)

375

0.4 0.2 0

0.4 0.2

0

0.2

0.4 0.6 v (m/s)

0.8

0

1

(c)

(d)

Fig. 6. SLIP gait families for = 1.1 − 1.3 for (a) c = 0.7 (b) c = 0.5 (c) c = 0.3 and (d) c = 0.1. Gait stability is indicated by dotted and solid lines for unstable and stable gaits respectively. Model parameters used in calculating the gaits are similar to those of Blaberus discoidalis, as described in the text βnT D

the system, but does not necessarily produce period one gaits, since the quasiperiodic gaits observed in [8] also appear, as illustrated in Fig. 7. However, the recurrent nature of quasi-periodic orbits enables the use of chaos control methods to enforce stabilization to period one gaits. Speciﬁcally, a delay feedback controller [22] is implemented to enforce stabilization to period one orbits. Delay feedback control incorporates the diﬀerence between the value of a state variable at one instant and one period delayed, such that the control eﬀect vanishes when the periodic orbit is attained. In our system, delay feedback control is implemented by including a dependence in the control law on the pitch angle as follows TD LO = βnLO + c(βnT D − βnLO ) + c2 (θnLO − θn−1 ). βn+1

(38)

The eﬀect of the control law and the control law with delay feedback included is illustrated in Figs. 7 and 8. In both cases, the rigid body SLIP model is simulated from the same set of initial conditions (v = 0.25, δ = 0.2, θ = 0, θ˙ = −0.2). Figure 7 illustrates that while the inclusion of the leg angle control does

376

J. Schmitt 0.25

0.015

0.23 0.22

0.05

0.01 z (m)

0.24

Vertical velocity (m/s)

Fore−aft velocity (m/s)

0.1

0

0.005

−0.05

0.21

−0.1 0.2

0

0.5

1

0

0.5 t

1.25

0.01

1.24

0.005

1.23

(rad)

0.015

0

0.1

0.2

y (m)

1.22

TD

0

0

β

θ (rad)

t

1

−0.005

1.21

−0.01

1.2

−0.015

0

0.5

1 t

1.19

0

0.5

1 t

Fig. 7. Rigid body SLIP simulation with the standard leg angle feedback control implemented, with d = 0.001, c = 0.1, I = 1.86 × 10−7 and initial conditions as speciﬁed in the text. Model parameters used in calculating the gaits are similar to those of Blaberus discoidalis, as described in the text. Plus signs (+) denote the start and end of each stance phase, whereas stars (*) denote the start of each stance phase

stabilize the system, the resulting orbit is quasi-periodic rather than period one. While period one gaits may be obtained through the use of this control scheme, they are not necessarily achieved. Conversely, Fig. 8 illustrates the eﬀect of the control law with delay feedback control included. As illustrated, the system not only stabilizes, but stabilizes to a period one orbit. Simulations initiated from a wide range of initial conditions provide similar results. As in previous analyses, the translational and rotational coupling present in the equations of motion, as well as in the control law, lead to the attainment of a new gait in response to perturbations. While further investigation is required to quantify the eﬀects of these control laws on the stability of the rigid body SLIP model, these preliminary results suggest that the control laws presented here will expand the very small basin of attraction identiﬁed for this model in previous studies [8].

Simple Feedback Control of Cockroach Running 0.25

377

0.015

0.23 0.22

0.05

0.01 z (m)

0.24

Vertical velocity (m/s)

Fore−aft velocity (m/s)

0.1

0

0.005

−0.05

0.21

−0.1 0.2

0

0.5

1

0

0.5

t

1 t

0

0

0.1

0.2

y (m)

−3

x 10

1.25

10

1.23

5

βTD (rad)

θ (rad)

1.24

1.22 1.21

0

1.2 −5

0

0.5

1 t

1.19

0

0.5

1 t

Fig. 8. Rigid body SLIP simulation with delayed feedback control, with d = 0.001, c = 0.1, c2 = 1.25, I = 1.86 × 10−7 and initial conditions as speciﬁed in the text. Model parameters used in calculating the gaits are similar to those of Blaberus discoidalis, as described in the text. Plus signs (+) denote the start and end of each stance phase, whereas stars (*) denote the start of each stance phase

5 Conclusion In this work, we investigate the applicability of the SLIP model to insect locomotion, speciﬁcally that of the cockroach Blaberus discoidalis. We ﬁnd that unlike previous results obtained for the SLIP model when applied to larger animals, periodic gaits produced for these model parameters remain largely unstable over a wide range of leg touchdown angles. This, in conjunction with results indicating that a ﬁxed angle leg reset policy employed in a spatial SLIP model produces only unstable gaits, prompts an investigation into alternate leg touchdown protocols and their eﬀects on gait stability. In particular, we show that a leg angle touchdown protocol that places the next leg down at TD = βnLO , necessarily produces neutrally stathe previous leg lift-oﬀ angle, βn+1 ble period two gaits, and is capable of producing neutrally stable period one gaits with an eigenvalue equal to negative one. Continuity arguments between this leg touchdown protocol and the ﬁxed angle leg reset protocol are used to develop a feedback control law based on inertial leg touchdown angles that stabilizes these unstable periodic gaits. Numerically computed gait families

378

J. Schmitt

verify that lowering the parameter c in this control law increases the number of stable periodic gaits, therefore expanding the basin of stability. Implementing the same control law in a rigid body SLIP formulation also produces stable gaits, but not necessarily periodic gaits, since quasi-periodic gaits also appear. Utilizing the pitch angle as delay feedback control in the control law forces stabilization to period one orbits. The control laws developed in this work are unique in that they: a) do not require knowledge of pre-existing periodic orbits or linearization about those orbits to achieve control b) require relatively simple feedback measurements rather than full state feedback for implementation c) apply control once per stance phase rather than continuously during the stance phase and d) can adapt to perturbations by changing to a diﬀerent stable periodic gait that is more suitable to the new environment. Qualitatively, it appears that the eﬀectiveness of the control law results from implicit information about the system angular momentum that is present in a reading of the leg lift-oﬀ angle. Quantitative exploration of this hypothesis will be conducted in future work. Additionally, it appears that these control laws may also have applicability in both the horizontal plane and spatial SLIP models. Preliminary simulations utilizing two control laws specifying the leg placement angles of a point mass spatial SLIP model appear to, at least for some parameter ranges, produce stable periodic gaits. We plan to investigate these applications further in later works.

References [1] R. Blickhan. The spring-mass model for running and hopping. J. Biomechanics, 11/12:1217–1227, 1989. [2] R. Blickhan and R.J. Full. Similarity in multi-legged locomotion: bouncing like a monopode. J. Comp. Physiol. A, 173:509–517, 1993. [3] G.A. Cavagna, N.C. Heglund, and C.R. Taylor. Mechanical work in terrestrial locomotion: two basic mechanisms for minimizing energy expenditure. Am. J. Physiol., 233 (5):R243–R261, 1977. [4] T.A. McMahon and G.C. Cheng. The mechanics of running: how does stiﬀness couple with speed? J. Biomechanics, 23 (suppl 1):65–78, 1990. [5] R.J. Full and M.S. Tu. Mechanics of a rapid running insect: two-, four- and six-legged locomotion. J. Exp. Biol., 156:215–231, 1991. [6] C.T. Farley, J. Glashenn, and T.A. McMahon. Running springs: speed and animal size. J. Exp. Biol., 185:71–86, 1993. [7] R.J. Full, R. Blickhan, and L.H. Ting. Leg design in hexpedal runners. J. Exp. Biol., 158:369–390, 1991. [8] R. Ghigliazza, R.M. Altendorfer, P. Holmes, and D. Koditschek. A simply stabilized running model. SIAM Journal on Applied Dynamical Systems, 2(2):187– 218, 2003. [9] A. Seyfarth, H. Geyer, M. Gunther, and R. Blickhan. A movement criterion for running. J. of Biomechanics, 35:649–655, 2002.

Simple Feedback Control of Cockroach Running

379

[10] A. Seyarth, H. Geyer, and H. Herr. Swing-leg retraction: a simple control model for stable running. J. Exp. Biology, 206:2547–2555, 2003. [11] J. Seipel and P. Holmes. Running in three dimensions: analysis of a point-mass sprung-leg model. Intl. J. Robotics Research, 24(8), 2005. [12] A. Ruina. Non-holonomic stability aspects of piecewise holonomic systems. Reports on Mathematical Physics, 42(1/2):91–100, 1998. [13] J. Schmitt and P. Holmes. Mechanical models for insect locomotion: Stability and parameter studies. Physica D, 156(1–2):139–168, 2001. [14] J. Schmitt and P. Holmes. Mechanical models for insect locomotion: Dynamics and stability in the horizontal plane – Theory. Biological Cybernetics, 83(6):501–515, 2000. [15] J. Schmitt and P. Holmes. Mechanical models for insect locomotion: Dynamics and stability in the horizontal plane – Application. Biological Cybernetics, 83(6):517–527, 2000. [16] J. Guckenheimer and P. Holmes. Nonlinear Oscillations, Dynamical Systems, and Bifurcations of Vector Fields. Springer-Verlag, New York, NY, 1990. [17] L.H. Ting, R. Blickhan, and R.J. Full. Dynamic and static stability in hexapedal runners. J. Exp. Biol., 197:251–269, 1994. [18] W.J. Schwind and D.E. Koditschek. Approximating the stance map of a 2 dof monoped runner. J. Nonlinear Science, 10(5):533–568, 2000. [19] M. Coleman, A. Chatterjee, and A. Ruina. Motions of a rimless spoked wheel: a simple 3d system with impacts. Dynamics and stability of systems, 12(3):139– 169, 1997. [20] M. Coleman and P. Holmes. Motions and stability of a piecewise holonomic system: the discrete chaplygin sleigh. Regul. Chaotic Dyn., 4:55–77, 1999. [21] R. Altendorfer, R. Ghigliazza, P. Holmes, and D. Koditschek. Exploiting passive stability for hierarchical control. In Proceedings of the Fifth International Conference on Climbing and Walking Robots (CLAWAR 2002), pp. 81–85, London, 2002. Professional Engineering Publishing Limited. [22] K. Pyragas. Continuous control of chaos by self-controlling feedback. Phys. Lett. A, 170:421–428, 1992.

Appendix The eigenvalues of the period one Poincar´e map are determined from det(λI − DP ), where I is the identity matrix and DP is the Jacobian matrix of (25). Evaluating the determinant and simplifying yields ∂∆ψ ∂∆ψ ∂∆ψ ∂∆ψ ∂∆ψ )(λ2 + ( T D + T D )λ − 1 − − TD ) + ∂vnT D ∂βn ∂δn ∂βnT D ∂δn ∂∆ψ ∂∆ψ ∂∆ψ ∂∆ψ ( λ −1)b T D T D + (λ − 1)(2 + )b =0. (A-1) ∂vn ∂δn ∂βnT D ∂vnT D

( λ −1 − b

Factoring a (λ − 1) out of all terms yields

380

J. Schmitt

∂∆ψ ∂∆ψ ∂∆ψ )(λ + 1 + + TD ) + ∂vnT D ∂βnT D ∂δn ∂∆ψ ∂∆ψ ∂∆ψ ∂∆ψ ∂∆ψ b + 2b T D + b T D T D ) = 0 . T D T D ∂δn ∂vn ∂vn ∂βn ∂vn

( λ −1)((λ − 1 − b

(A-2)

Expanding further and simplifying yields (λ − 1)2 (λ + 1 +

∂∆ψ ∂∆ψ ∂∆ψ + TD − b TD ) = 0 ∂βnT D ∂δn ∂vn

(A-3)

which yields the expression for the eigenvalues in (27–28). ∂∆ψ ∂∆ψ ∂∆ψ From (28), it is clear that we need to calculate ∂β T D , ∂v T D , ∂δ T D , in order n n n to determine the value for the relevant eigenvalue that determines stability. The approximation of ∆ψ presented in (29–30) can be simpliﬁed further as p q p = 128lvnT D sin(βnT D − δnT D )(l − ζb ) √ q = (3ζb + l) s 9k (ζb − l)2 )(3ζb + l)2 s = (16(vnT D )2 − m − 256l2 (vnT D )2 sin2 (βnT D − δnT D ) .

∆ψ =

(A-4) (A-5) (A-6) (A-7)

Since ∆ψ is a function of ζb , in evaluating the required partial derivatives, we need ∂v∂ζTbD , ∂δ∂ζTbD , ∂β∂ζTbD , all of which may be determined implicitly from (17) n n n as mvnT D (ζb2 − l2 sin2 (βnT D − δnT D )) ∂ζb = ∂vnT D ζb (−m(vnT D )2 + k(2ζb − l)(ζb − l))

(A-8)

∂ζb ml2 (vnT D )2 sin(2(βnT D − δnT D )) = T D ∂δn 2ζb (−m(vnT D )2 + k(2ζb − l)(ζb − l))

(A-9)

∂ζb ml2 (vnT D )2 sin(2(βnT D − δnT D )) . = − ∂βnT D 2ζb (−m(vnT D )2 + k(2ζb − l)(ζb − l))

(A-10)

Given these partial derivatives, we can calculate the expressions for the required derivatives of ∆ψ from the quotient rule ∂∆ψ q p˙ − pq˙ = ∂() q2

(A-11)

where () represents vnT D , δnT D , βnT D and (˙) denotes derivatives with respect to these variables, which can be expressed as ∂p ∂ζb = 128l sin(βnT D − δnT D )((l − ζb ) − vnT D T D ) T D ∂vn ∂vn

(A-12)

Simple Feedback Control of Cockroach Running

381

∂p ∂ζb = −128lvnT D (cos(βnT D − δnT D )(l − ζb ) + T D sin(βnT D − δnT D )) (A-13) ∂δnT D ∂δn ∂p ∂ζb = 128lvnT D (cos(βnT D − δnT D )(l − ζb ) − sin(βnT D − δnT D )) (A-14) T D ∂βn ∂βnT D and (3ζb + 1)(t ∂v∂ζTbD + w − u sin(βnT D − δnT D )) ∂q ∂ζb √ n √ = 3 s + ∂vnT D ∂vnT D s

(A-15)

(3ζb + 1)(t ∂δ∂ζTbD − x ∂δ∂ζTbD + uvnT D cos(βnT D − δnT D )) ∂q ∂ζb √ n n √ = 3 TD s + ∂δnT D ∂δn s (A-16) ∂ζb ∂ζb TD TD TD + 1)(t − x − uv cos(β − δ (3ζ b TD TD n n n )) ∂q ∂ζb √ ∂βn ∂βn √ = 3 s + ∂βnT D ∂βnT D s (A-17) where 9k (ζb − l)2 ) m u = 512l2 vnT D sin(βnT D − δnT D ) 18k ∂ζb (ζb − l) T D )(3ζb + l)2 w = (32vnT D − m ∂vn 18k (ζb − l)(3ζb + 1)2 . x= m t = 6(3ζb + l)(16(vnT D )2 −

(A-18) (A-19) (A-20) (A-21)

Running and Walking with Compliant Legs A. Seyfarth1 , H. Geyer1 , R. Blickhan2 , S. Lipfert1 , J. Rummel1 , Y. Minekawa1 , and F. Iida1,3 1

2 3

Locomotion Laboratory, University of Jena, Dornburger Str. 23, D-07743 Jena, Germany www.lauflabor.de Science of Motion, University of Jena, Seidelstr. 20, D-07749 Jena, Germany Artiﬁcial Intelligence Laboratory, University of Zurich, Andreasstrasse 15, CH-8050 Zurich, Switzerland

It has long been the dream to build robots which could walk and run with ease. To date, the stance phase of walking robots has been characterized by the use of either straight, rigid legs, as is the case of passive walkers, or by the use of articulated, kinematically-driven legs. In contrast, the design of most hopping or running robots is based on compliant legs which exhibit quite natural behavior during locomotion. Here we ask to what extent spring-like leg behavior could be useful in unifying locomotion models for walking and running. In so doing, we combine biomechanical experimental and computer simulation approaches with theoretical considerations and simple legged robots. We have found that (1) walking and running result from mechanical stability which corresponds to the experimentally observed gait dynamics, (2) running is a subset of stable movement patterns for high system energies, and (3) walking with knee ﬂexion during stance can result from passive leg mechanics with elastic structures spanning the joints.

1 Introduction What are the common design and control principles of legged locomotion? On the one hand, we must consider the internal leg function: the number of leg segments, the arrangements of muscles, ligaments and other soft tissue within the leg and the appropriate muscle activation patterns in order to generate a desired leg behavior. On the other hand, we need to integrate the leg into an encompassing control system – the global leg function – such that we can observe a “complete” movement pattern including legs and the supported body. This means that if we have simple, biologically meaningful models describing the leg behavior

384

A. Seyfarth et al.

Fig. 1. Leg function is divided into global and internal leg functions. Global leg function describes the leg control based on a given internal leg function. The internal leg function addresses issues of design and control of a segmented leg itself. To identify control strategies we analyze the mechanical stability of selected leg designs at diﬀerent levels of leg function

in a desired movement task, we can use these leg templates (i.e. simpliﬁed models, [7]) to derive the required leg control strategy. A very simple description of the leg behavior is provided by the springmass model. Here, the force generated by the leg during the stance phase is assumed to be proportional to the amount of leg compression, i.e. the more the leg shortens the larger the corresponding leg force. Despite its simplicity this model is a very powerful tool to predict movement strategies or jumping performance [22]. It can help to better understand the role of leg segmentation or muscle function during fast movements. Furthermore, spring-like leg operation seems to ease required control action even in highly dynamic situations with reduced sensory perception. In legged locomotion we observe a sudden change in leg behavior between walking and running. This gait transition occurs in most animals at about the same dimensionless speed at Froude number equal to approximately 0.4, which is calculated using the equation v 2 /gl (where v is the forward speed, g is the gravitational acceleration, and l is the leg length, [14, 26, 8, 18]). This suggests the possibility that common, underlying mechanical principles may exist for legged systems. If this is true, mechanical design methodologies could be derived which would yield systems equally capable of walking and running gaits, depending on the selected system condition (e.g. system energy) or movement strategy. In order to change gait, either the mechanical system could change its behavior in an internal and self-organized fashion or the overlaying control strategy could initiate the gait transition. This might point to distinct movement primitives (i.e. programs) which could be used to select gaits. In order to control a given mechanical system we could apply simple feedback control approaches (e.g. a PD controller). In this paper we rather suggest a diﬀerent approach. First, we build simpliﬁed mechanical models either on a computer

Running and Walking with Compliant Legs

385

(i.e. simulation models) or as physical models (i.e. robots) and explore their behavior for a variety of reasonable initial conditions and model parameters. Second, we compare the predictions of diﬀerent proposed models with experiments of human or animal locomotion. Finally, we investigate the inﬂuence of the model parameters on the movement performance and try to estimate the best control strategy in order to improve system stability with the lowest possible sensory eﬀort. In this paper we describe a series of simple models of internal and global leg functions and compare them with the behavior of two experimental legged robots. We will start with two models addressing internal leg behavior, a threesegment model with elastic joints and a two-segment model with an extensor muscle. Afterwards, we present models describing the global leg function in spring-mass running and walking. Finally, we explore the behavior of the simple legged robots, imitating walking and running.

2 Internal Segmentation of the Leg In order to operate in a spring-like fashion, a segmented leg must be able to compress and extend stably. With two segments, a leg would have no problem doing so as long as the internal leg joint is operating in a spring-like fashion as well. Leg compression directly translates into joint ﬂexion, which in turn results in higher joint torque and consequently, in increased leg force. The only problem could be in the generation of linear leg spring behavior as observed in human and animal locomotion. Therefore, the joint spring should be nonlinear in terms of the torque-angle characteristics, i.e. being more compliant at low compressions and being stiﬀer at larger joint ﬂexion. If we extend the two-segment leg by one segment we obtain a three-segment leg similar to the human leg, with foot, shank and thigh. Now, leg compression can result in diﬀerent outcomes. Let us assume both internal leg joints (e.g. ankle and knee) to be equally stiﬀ and let us further assume completely symmetric leg geometry (equal segment lengths, equal initial joint angles). We would then expect a transformation of leg compression into equal ﬂexion of the two joints. Interestingly, however, this is not the case (Fig. 2). In the three segment leg, stability requires a local minimum of the mechanical system energy with respect to variations in the joint angle conﬁguration (ankle and knee) and a given leg length (e.g. distance hip to toe). At a certain amount of leg compression however, this local minimum in system energy changes into a local maximum when loading both joints equally. Consequently, symmetric bending of both joints becomes unstable. Even perfect adjustment of the joint springs cannot prevent asymmetric joint behavior if a certain critical amount of leg compression is exceeded [25]. Here, one leg joint starts to extend whereas the second joint ﬂexes rapidly. As a consequence, the leg force is not shared equally between the joints and the leg stiﬀness drops. This situation would lead to high local stresses at the ﬂexing joint which could

386

A. Seyfarth et al.

Fig. 2. Three-segment model with equal segment lengths (L1=L2=L3) and two joint springs of equal stiﬀness and equal static equilibrium angles (A). After a certain amount of leg compression (B) symmetric bending of both joints becomes unstable. Depending on inﬁnitely small perturbations the two joints proceed by rotating asymmetrically, with one extending and the other ﬂexing (C)

result in structural damage or failure in a real leg. To avoid this disastrous behavior, diﬀerent measures can be introduced. One strategy is to steer the leg joint movements by a kinematic control approach, whereby the joint angle is constrained along a desired trajectory. This intervention may work at low energies through the use of a high-bandwidth controller to counteract the system dynamics. At faster leg movements however, a more systematic modiﬁcation is required. Stability analysis of the three-segment leg reveals that diﬀerent solutions exist to avoid this intrinsic instability (Fig. 3). Strategy A: The joint spring characteristics are slightly non-linear, e.g. the joint stiﬀness increases with joint ﬂexion. If one joint ﬂexes more than the other, the increased joint stiﬀness compensates for the mechanical disadvantage caused by the joint’s increased ﬂexion. The model predicts a higher nonlinearity in joints whose conﬁguration at static equilibrium is characterized by greater extension (e.g. knee compared to ankle). This prediction is conﬁrmed by experimental results. The required nonlinearity of the joint torque characteristics might be provided by the nonlinear stress-strain characteristics of tendons and aponeuroses connecting the muscle to the skeleton as found in human running or jumping [13, 21]. Strategy B: The leg segmentation is asymmetric, i.e. the outer leg segments are diﬀerent in length. The model predicts that stable leg operation can be achieved if the joint adjacent to the shorter outer segment (e.g. ankle in humans) is more ﬂexed compared to the other leg joint. This ﬁnding is in agreement with human leg design.

Running and Walking with Compliant Legs

387

Fig. 3. Five strategies to avoid instability in an elastic three-segment leg: (A) nonlinear joint springs, (B) asymmetric leg segmentation and leg joint conﬁguration, (C) biarticular elastic structures, (D) joint constraints, e.g. heel contact, and (E) operation in a bow conﬁguration

Strategy C: The risk of out-of-phase joint function due to mechanical instability can be reduced by adding biarticular elastic structures. In the case of asymmetric leg segmentation (Strategy B) it is suﬃcient to have only one biarticular spring which ﬂexes the more extended joint (knee) and extends the more ﬂexed joint (ankle). The risk of over-extension of the ankle joint is largely avoided by the ﬂexed ankle conﬁguration (about 80–120 degrees) during human locomotion. A biarticular antagonist of the m.gastrocnemius is not required and does not exist in nature. Strategy D: Even if all previous strategies (A–C) fail to prevent unstable leg operation, there is still another hard-built safety measure to avoid overextension of the more extended leg joint (the knee): the mechanical constraint of joint ﬂexion due to a skeletal structure, e.g. the calcaneus with the heel pad. If the activity of the plantar ﬂexors is not suﬃcient the heel strike prevents overextension of the knee which would have serious consequences in many athletic movements like running or long jumping. Strategy E: Finally, there is a strategy to achieve stable leg operation by using a very diﬀerent nominal leg conﬁguration by swapping the leg joints (from a zigzag or Z-conﬁguration) to the same side with respect to the leg axis (bow or C-conﬁguration). The leg conﬁguration is safe and simple to control at cost of reduced limb stiﬀness and reduced leg force. It can be found in the upper limbs of humans or in spiders. All of these strategies guarantee parallel joint operation in a three segment leg and can be found in nature. It is important to realize that these measures are not exclusive and are often implemented in a highly redundant fashion. Similar to the air bags in our cars, the leg includes several design and control strategies to avoid mechanical instability potentially leading to serious damages of the musculo-skeletal system. Elastic joint behavior in itself does not guarantee stable leg operation during contact. However, for the identiﬁed

388

A. Seyfarth et al.

leg design and control strategies (Fig. 3) the control of the highly nonlinear segmented leg could be simpliﬁed. The results demonstrate that spring-like leg operation can be a key for better understanding the architecture and function of biological legs. On the joint level, spring-like behavior (joint stiﬀness and nominal angle) can be adapted based on neuromuscular mechanisms [6], [10]. In turn, if all required measures are undertaken to guarantee stable leg operation, spring-like leg operation can result at various loading conditions.

3 Generation of Muscle Activity Periodic movement patterns as observed in legged locomotion require a cyclic action of the muscles within the body. The time series of the corresponding muscle activation pattern could be the result of diﬀerent mechanisms. One possibility is direct control of muscle activation by supraspinal commands. In this case the exact timing of all muscles would require a high processing (i.e. high-bandwidth) capacity in the brain. A diﬀerent approach would be the generation of periodic movement patterns in rhythm generators located in the spinal cord (central pattern generators, [12]). Then the higher control could be reduced to coordinate these pattern generators. Another possibility is the generation of the required muscle activation based on the dynamics of the musculo-skeletal system and sensory feedback to the spinal cord [27]. Such a control strategy could further relax the neural control eﬀort and could take advantage of positive side-eﬀects of the muscle-reﬂex dynamics. In a simulation study [10], we asked which proprioceptive reﬂex loop would be capable of generating the required muscle stimulation STIM(t) for steady state hopping in place (Fig. 4). Leg geometry is reduced to two massless leg segments with one Hill-type extensor muscle spanning the leg joint. The body is represented by a point mass on top of the upper segment (Fig. 4). Muscle stimulation is assumed to be the sum of a given central command, STIM0, and a potential reﬂex contribution (ampliﬁed and time delayed sensory signal based on muscle length, muscle velocity or muscle force). We found that steady state hopping is possible (1) with an optimized stimulation pattern STIM0(t) or (2) based on a constant STIM0 and positive length or positive force feedback. The predicted maximum hopping height employing positive force feedback is 83% of that calculated using an optimal muscle stimulation pattern and turned out to be robust with respect to simulated external perturbations (e.g. changed ground properties, Fig. 4C). At moderate hopping heights, an almost spring-like leg operation is predicted (Fig. 4B). The simulation results indicate that the generation of the extensor muscle activity in hopping or running tasks could be facilitated by positive force feedback. Instead of giving the muscle a precisely timed activation pattern, the task is now executed based on a constant stimulation (STIM0) and proper integration of proprioceptive signals into the activation of the extensor-motoneuron. The control eﬀort is therefore largely reduced and the

Running and Walking with Compliant Legs

389

A

B

¿ Fig. 4. (A) Two-segment leg model with one extensor muscle and proprioceptive feedback. The activity of the α-motoneuron driving the extensor muscle (STIM ) is assumed to be the sum of a central command (STIM0 ) and a contribution of the reﬂex loop based on diﬀerent sensory signals (muscle length, muscle velocity or muscle force). Muscle stimulation signals STIM(t) with and without sensory feedback for stable hopping in place are calculated. (B) The best hopping performance based on sensory feedback is achieved with positive force feedback and a constant bias signal STIM0. Here, spring-like leg behavior is found. (C) The muscle-reﬂex dynamics are robust with respect to environmental changes like hopping on a dissipative substrate (sand)

leg behavior is more robust with respect to perturbations. Furthermore, even with little or no passive compliance (as would be provided by tendons, for instance) the muscle-reﬂex dynamics produced spring-like leg behavior. In that respect leg stiﬀness is an emergent steady-state behavior in cyclic hopping (or running) based on the neural integration of sensory information and will therefore adapt to environmental changes detected by the sensory organs. In the previous sections two simpliﬁed models for the internal leg function are introduced. Both models give new insights into how spring-like legs could be designed and controlled. It turned out that compliant leg operation is not just useful to store elastic energy; it also helps to make a segmented leg safe and robust when faced with external perturbations. In the following two sections we will deal with the global leg function, i.e. how a spring-like leg can be controlled to obtain stable locomotion as observed in running or walking. We provide evidence that spring-like leg operation may

390

A. Seyfarth et al.

also be useful in facilitating the global leg function, i.e. the method in which the body utilizes limbs for stable locomotion.

4 Running with Elastic Legs The movement of a single leg during walking and running is characterized by a series of contact and swing phases. At lower speeds humans generally choose to walk whereas at higher speeds running is preferred. During the stance phase of running, the leg compresses until midstance and then extends until the leg leaves the ground. The force generated by the leg is approximately proportional to the amount of leg compression [4, 3]. This relation provides the basis for the concept that leg stiﬀness is the parameter describing that the ratio between leg force and leg compression remains constant. This concept leads to the spring-mass model which describes the movement of the center of mass based on a spring-like leg operation during the stance phase of running [1]. For certain combinations of leg stiﬀness k and leg angle of attack α0 cyclic motion of the center of mass (COM) can be observed (Fig. 5A). Interestingly, cyclic movement can also be achieved for slightly diﬀerent initial conditions.

Fig. 5. Spring-mass running. (A) For certain combinations of leg stiﬀness k and angle of attack α0 a cyclic movement of the center of mass can be found. (B) Diﬀerent angles of attack can result in stable running patterns (here α0 = 67o , 68o with leg stiﬀness k = 20 kN/m, body mass m = 80 kg, leg length l = 1 m). Steeper angles overshoot a step resulting in failure at the proceeding step (α0 = 69o ) whereas ﬂatter angles decelerate the movements until forward velocity goes to zero (α0 = 66o ). With leg retraction, e.g. increasing α0 prior to landing, the angular adjustment can vary much more without losing running stability (not shown here, for details see [24])

Running and Walking with Compliant Legs

391

The movement of the center of mass approaches a steady state after a couple of steps without modifying the landing leg angle α0 (Fig. 5B). However, even when modifying the angle of attack α0 stable running can be observed, although the COM trajectory has adapted slightly. Hence, spring-mass running is self-stabilizing and robust with respect to changes in the initial conditions and model parameters (e.g. leg angle of attack, leg stiﬀness, system energy). The adjustment of leg stiﬀness and angle of attack is not unique. At a given running speed, diﬀerent combinations are possible (Fig. 5B) and result in speciﬁc running styles; for instance, in terms of step length or step frequency. This prediction agrees with experimental results [23]. With increasing speed, the range of successful combinations of leg angle and leg stiﬀness is even increased. In contrast, at low speed (less than 3 m/s) no stable running is predicted with the constant angle of attack control policy at any leg stiﬀness. However, animal or human running reveals that the leg angle is not kept constant prior to landing [5]. In fact, a backward rotation of the leg with respect to the body is observed. Introducing this early leg retraction we ﬁnd an increased stability in spring-mass running at low speeds [24]. Even larger variations in internal or external conditions (e.g. a change in ground level of 50 percent leg lengths) can be managed when leg retraction is used. In this section we introduced two global leg control strategies for running with compliant legs: constant angle of attack and leg retraction. We found that for a given system energy spring-mass running is stable for various combinations of leg stiﬀness k and angle of attack α0 . For higher running speeds and by employing leg retraction the region of stable running (leg stiﬀness and leg angle adjustment) is largely enhanced. In a recent study, the spring-mass model for running was extended to a rigid body model in the vertical plane and analyzed based on an analytical approximation neglecting gravity during the stance phase [11]. The results support the identiﬁed strategies for stable running with a ﬁxed angle of attack policy. Furthermore, it is argued that at high running speeds the domains of attraction become smaller (i.e. the system is less robust with respect to perturbations of the center of mass trajectory) leading to a demand for more elaborated control methods. One possible method could involve the adaptation of leg stiﬀness to ﬂight time duration (e.g. due to muscle preactivation) similar to the strategy of leg retraction. Therefore, an integration of muscular mechanisms into the analysis of running stability would be helpful.

5 From Running to Walking So far only a single leg was considered in the analysis of spring-mass running. As a next step we generalize the model to bipedal locomotion. Does the concept of spring-like leg operation hold only for running with single support phases? What happens to the system dynamics if more than one leg is in contact with the ground at the same time?

392

A. Seyfarth et al.

Fig. 6. Spring-Mass Walking. (A) For certain combinations of leg stiﬀness k and angle of attack α0 a cyclic movement of the center of mass can be found. (B) The single leg force patterns (upper line: vertical force, lower line: horizontal force) resemble that found in human and animal walking [9]

In a previous simulation study we investigated the behavior of a bipedal spring-mass model [9]. This model consists of two massless springs (leg stiﬀness k ) and a point mass representing the center of mass (COM). During the single support phase one leg spring remains in contact with the ground while the other is positioned at a constant angle of attack. If the COM reaches the corresponding landing height of the second leg before the ﬁrst leg leaves the ground a double support phase occurs (Fig. 6A). Although slightly more complex than the previous running model, stable solutions can again be found using diﬀerent combinations of leg stiﬀness k and angle of attack α0 . In contrast to the single leg model, stable solutions with double support phases can only be found for low system energies (forward speed lower than about 1.4 m/s). The single leg forces predicted by the bipedal spring-mass model are very close to the observed patterns in human and animal walking (Fig. 6B). The corresponding maximum walking speed is only slightly above the preferred walking speed observed in humans. This suggests higher control eﬀorts at higher walking speeds. Hence, mechanical stability and therefore a relaxed control (rather than metabolic considerations) could be important criteria to explain the preferred walking speed. At high energies (forward speed larger than about 3 m/s) the previously observed running pattern is found if the leg is allowed to contact the ground after take-oﬀ of the opposite leg. Thus, the bipedal spring-mass model can both stable walking and running on a single mechanical system. There is an energetic gap between both gaits which can only be accessed by a more complicated control strategy (e.g. leg retraction of the swing leg). The model suggests that walking and running are two natural and self-stabilizing behaviors of a simple mechanical system. This concept has strong implications on the design and control of legged systems in general. Leg compliance is not only a strategy to facilitate control; it might also be the origin of the existence of

Running and Walking with Compliant Legs

393

natural gaits. Furthermore, it resolves the issue of collision avoidance which was addressed recently in studies on inverted pendulum walking [20]. In the last sections we examined consequences of compliant leg operation on a global scale. We found that the two fundamental gaits of legged locomotion are natural behaviors of elastic legs attached to a common center of mass. In the next section simple legged robots are introduced. How does the behavior of a constructed leg compare to the conceptional models for legged locomotion presented in the last four sections?

6 Exploring Simple Legged Robots Elastic leg behavior can elucidate design and control strategies used in legged systems. However, all models presented so far are based on computer simulations or analytical calculations. While the model-based predictions have been compared to results from biology, as discussed in Sects. 3, 4 and 5, it was decided that further validation under real-world conditions was necessary. Therefore, we built a series of very simple legged robots to better understand the pros and cons of our theoretical models, examining their validity and the underlying assumptions, and to examine any overlooked elements. What should a simple legged robot look like? A good example can be found in the pioneering work of Raibert and his coworkers [19]. These robots are made of elastic legs (pogo sticks) which are controlled in such a manner as to regulate hopping height, body speed and pitch at desired values. We found the construction of a pogo stick leg still “delicate” from the mechanical point of view (e.g. due to constraint forces perpendicular to the leg axis) and decided consequently to start with a simple two-segment leg (Fig. 7A), instead. 6.1 Robot Testbed with Elastic Two-Segment Leg Our goal is to explore the natural dynamics of a two-segment elastic leg during forward hopping. The movement of the robot (Fig. 7A) is constrained by a metal boom which allows the robot to only move in vertical and horizontal direction. Body pitch movements are not allowed to keep the mechanical system as simple as possible. The focus of this approach is to better understand how legged systems are organized. The movement of the leg is driven by a servo motor between the body and the upper leg segment (thigh). The motor introduces a sinusoidal oscillation deﬁned by oscillator frequency f, angular amplitude A and oﬀset angle O. Taking a maximum angular velocity ωM AX of the motor (1 rotation/s) into account, the amplitude A can be calculated depending on the frequency f with A = ωM AX /(2πf ) .

(1)

394

A. Seyfarth et al.

A

C

B

D

Fig. 7. (A) The “Fujubot” robot, with an elastic two-segment leg kinematically driven at a sinusoidal trajectory using a hip-mounted servo motor. (B) Experimental data for stable robot hopping. The angle of the upper segment (phi) is predeﬁned by the motor control. At touch-down (dotted vertical lines) retraction of the upper segment is brieﬂy interrupted, but recommences immediately afterwards. (C) Experimental data on human walking and (D) running at 2 m/s

This robot follows the rapidly prototyped, minimalist approach to design (“cheap design” robots, [16]), avoiding high-end or high-precision components and advanced control approaches. For instance, the upper leg segment does not follow the desired angular trajectory of the servo motor (Fig. 7B). Every time the leg hits the ground leg retraction is interrupted. This is due to the fact that the leg joint ﬂexes shortly after landing impact. Comparing this observation with experimental data in human running and walking we ﬁnd the same phenomenon. At a forward speed of 2 m/s the relative timing of protraction and retraction of the upper limb is very similar between walking and running. Both in the robot and in human running touch-down occurs shortly after the initiation of limb retraction. This is in agreement with the predicted role of leg retraction for stability [24]. To investigate the inﬂuence of an enforced leg retraction after touch-down, we implemented a higher torque motor using very stiﬀ coupling of the motor

Running and Walking with Compliant Legs

395

Fig. 8. (A) Dependency of hopping direction on control parameters (oscillation frequency f, oﬀset angle O) of the servo motor. (B) With no hip actuation the leg force FLEG merely depends on joint torque M and leg geometry. (C) Leg force is enhanced compared to (B) if the hip retracts actively (hip extension torque). The opposite is true if leg retraction is pointing to the left, in which case the leg force is reduced

to the hip joint. As a result the boom keeping the upper body upright broke. Consequently, a more compliant coupling was inserted between the motor and the hip joint, imitating the biological function of tendons in hip muscles. The robot demonstrates a variety of behaviors depending on the selected oscillator frequency f and oﬀset angle O. Surprisingly, at low oscillation frequencies, the hopping direction is not as expected, namely opposite to the leg joint (Fig. 8A). This movement could well be compared to that of a hopping bird. This behavior is not very sensitive to changes in the control parameters. At higher frequencies (above 6 Hz) a more human-like movement is observed. Then, the leg joint points forward (similar to a human knee). At an intermediate region hopping in place is observed with no substantial horizontal movement. Why does the robot change its movement direction depending on the selected control frequency? To approach this question the eﬀect of active limb retraction on the leg dynamics is considered in Figs. 8B and 8C. For simplicity, we focus on a static approach neglecting all dynamic eﬀects, i.e. due to segmental accelerations, joint damping, or torques at the foot point. With no retraction (zero hip torque), leg force is directly dependent on limb conﬁguration. This is a consequence of the rotational spring which relates joint torque to joint angle. If the hip is actively contributing to limb retraction (Fig. 8C), leg force is increased or decreased depending on the geometrical relation between leg joint torque and hip torque. In bird-like hopping, leg force is reduced whereas in human-like hopping the force is increased. As a consequence of this increased (or decreased) leg force, the natural frequency of the hopping system is

396

A. Seyfarth et al.

changed, i.e. an increased leg force to some extent imitates a stronger (stiﬀer) leg associated with a higher step frequency. This is a well known dependency for the spring-mass model. Hence, with compliant legs movement direction can be encoded as a frequency signal. At this level, the detailed trajectories of the limb segments are not required to control the diﬀerent directions of movement. Stable hopping is robust with respect to variations in the control parameters (oscillation frequency, oﬀset angle). This is in line with the self-stabilizing mechanisms of spring-mass running. It remains to be investigated in detail how the control parameters inﬂuence the angle of attack and the eﬀective leg stiﬀness. Furthermore, we found that elastic joint behavior is important in dealing with impacts (e.g. touch-down) avoiding damage to the actuator. A simple harmonic oscillation in the hip is suﬃcient to obtain stable hopping movement. The observed protraction and retraction plots of the upper limb are very similar to those observed in human walking and running. This encouraged us to build bipedal robots imitating the hip strategy of the one-legged hopping robot. Based on this experimental platform we will extend the concept of spring-like hopping to compliant walking. 6.2 Bipedal Robot The bipedal spring-mass model indicates that compliant legs may facilitate stable running and walking. Does this mean that walking is just running with double support phases? We approach this question by comparing experimental data on human walking and running with the behavior of a simple bipedal robot. We will demonstrate that leg compliance is useful in generating stable walking movements. The analysis of the three-segment model (Sect. 1) indicates that elastic joint operation may lead to a synchronous operation of the leg joints. Such behavior can be found in human running (Fig. 9) where during stance both knee and ankle joint ﬂex and extend in parallel. In walking this situation is not found: the knee joint extends during midstance and the ankle joint extends only at the late portion of the stance phase when the knee joint has returned to ﬂexing. The desired function of a mechanical spring to store and release elastic energy does not seem to be fulﬁlled by a walking leg. The high level of coordination between knee and ankle joint in running allows an eﬃcient push-oﬀ phase after midstance. The biarticular m.gastrocnemius transfers the rotational energy of the knee joint to the ankle which in turn is capable of generating a rapid leg extension. In walking this coupling is not found. The extension of the knee joint does not lead to a push-oﬀ phase because the ankle joint does not follow the knee extension. In the extended conﬁguration the knee is not able to contribute to leg lengthening but it has a signiﬁcant contribution to leg rotation. In fact, the eﬀect of thigh retraction on leg retraction is reduced by knee extension and (later) supported by knee ﬂexion. As a result the thigh is already protracting before the leg leaves the ground (Fig. 9A).

Running and Walking with Compliant Legs

397

Fig. 9. (A) Leg kinematics in human walking and running at the preferred transition speed (2 m/s). (B) Knee and ankle joint kinematics and vertical ground reaction forces corresponding to (A). (C) Muscle activity vs. muscle length of m.gastrocnemius medialis (GASm) of one subject during walking (41 steps, circles) and running (45 steps, crosses) at 2 m/s. Thick lines represent mean tracings for walking and running. Muscle length is calculated based on knee and ankle angle data [15]

If the rotational energy during knee extension (at midstance in walking) is not used for push-oﬀ, how can this energy be reused for locomotion? With continuous thigh retraction during stance phase (e.g. due to active hip retraction) the m.gastrocnemius gets stretched and can use the knee rotation at a later time to contribute to ankle extension. Hence, the rotational energy of the knee during midstance could still be reused for push-oﬀ triggered by the amount of leg rotation during stance phase. Therefore, we hypothesize that in walking the nominal length of the biarticular m.gastrocnemius should be longer than when compared to running. To test this hypothesis we analyze the activity of m.gastrocnemius medialis (Fig. 9C) and compare it to the estimated length of the muscle [15]. We ﬁnd

398

A. Seyfarth et al.

Fig. 10. (A) The “JenaWalker” bipedal robot, equipped with compliant legs. (B) Joint kinematics of knee and ankle joint and ground reaction forces (GRF) during human walking and (C) during robot walking. Dotted curves indicate opposite leg

that muscle activity is dependent on muscle length and gait. In walking the EMG is active at greater muscle length as compared to running. This indicates that walking could take advantage of elastic biarticular structures spanning knee and ankle joint as also suggested in a previous simulation study [2]. To test this idea the bipedal “JenaWalker” robot was developed (Fig. 10A, [17]). In order to investigate this gait-speciﬁc interplay between ankle and knee joint we decided to use three-segment legs. A biarticular spring was installed between knee and ankle joints, simulating the function of m.gastrocnemius. Furthermore, two additional “muscles” were required in the leg: one foot ﬂexor (m.tibialis anterior) and one biarticular knee extensor and hip ﬂexor (m.rectus femoris). This combination of elastic structures in a three-segment leg turned out to be suﬃcient to generate stable locomotion. In agreement with our experimental data on human walking we kept the hip control the same as in the hopping robot (i.e. a simple harmonic oscillation). An example of the leg kinematics and the ground reaction forces for stable walking is given in Fig. 10C. The robot is able to reproduce the experimentally observed knee and ankle joint kinematics (Fig. 10B). It is important to note that no eﬀort was made to optimize the leg kinematics or the ground reaction forces to ﬁt biological data. The criterion used for optimization is a steady periodic movement pattern. The robot is able to walk, hop and run speciﬁed by a corresponding motor adjustment. For instance, walking is robust with respect to changing step frequencies. Running is only possible for high oﬀset angles with the legs operating in front of the body. This is due to the fact that the leg cannot bend freely during the swing phase. With synchronous hip function, hopping was found.

Running and Walking with Compliant Legs

399

The legged robots presented here demonstrate that stable walking and running can be observed with compliant legs and simple harmonic oscillations at the hip joint. The movements are robust although both design and control are very simple. In the three-segment leg, the function of knee and ankle joint is gait-speciﬁc. The biarticular muscles could play an important role in synchronizing the internal leg function depending on the selected gait. Running can be considered as a gait with fast leg compression and fast leg rotation. The leg rotation (retraction) is mainly provided by hip extension and knee ﬂexion (plus late ankle extension). Due to synchronized operation of knee and ankle, the high joint angular velocities are slowed down in the biarticular m.gastrocnemius. Walking is a gait with slow leg compressions and slow leg rotation. Leg rotation is reduced by knee extension during midstance but supported by knee ﬂexion at early and late stance phase. As a consequence, the upper limb can already start protraction while the leg is still on the ground. This allows the hip control to still follow a harmonic oscillation despite a duty factor larger 0.5. Due to the out-of-phase operation of knee and ankle, the slow joint movements are accelerated in the biarticular m.gastrocnemius. Further research is required to better understand these intersegmental dynamics in walking and running at diﬀerent speeds and environmental conditions.

7 Conclusions In this paper we summarized several simple biomechanical models and “cheap” design robots describing legged locomotion. The common design principle was to reduce the systems to a minimal conﬁguration which allows for a systematic investigation of the underlying mechanisms of legged locomotion. The introduction of compliant structures and the search for self-stabilizing mechanisms revealed to be eﬀective tools to identify natural movement patterns and relaxed control strategies. It turned out that walking and running can be described as two natural movement patterns of one mechanical system with elastic legs. The organization of the segmented leg is largely supported by elastic structures spanning one or more joints. For properly designed legs, the control is largely simpliﬁed and could be reduced to an adjustment of hip oscillators in our legged robots. In the future, we aim to identify leg designs which are equally suited for both human-like walking and running. Therefore, a better understanding of the gait selecting mechanisms will be required.

References [1] Blickhan, R. (1989) The spring-mass model for running and hopping. J. Biomech. 22, 1217–1227.

400

A. Seyfarth et al.

[2] van den Bogert, A. J. (2003) Exotendons for assistance of human locomotion. Biomedical Engineering Online, 2–17. [3] Cavagna, G. A., Heglund, N. C. and Taylor, C. R. (1977) Mechanical work in terrestrial locomotion: two basic mechanisms for minimizing energy expenditure. Am. J. Physiol. 233, 243–261. [4] Cavagna, G. A., Saibene, F. P. and Margaria, R. (1964) Mechanical work in running. J. Appl. Physiol. 19, 249–256. [5] De Wit, B., De Clercq, D., and Aerts, P. (2000). Biomechanical analysis of the stance phase during barefoot and shod running. J. Biomech. 33, 269–278. [6] Feldmann, A. (1966) Functional tuning of the nervous system during control of movement or maintenance of a steady state posture. II controllable parameters of the muscles. Biophysics 11: 565–578. [7] Full, R. F. and Koditschek D. (1999) Templates and anchors: neuromechanical hypotheses of legged locomotion on land. J. Exp. Biol. Vol. 202, Issue 23, 3325– 3332. [8] Gatesy, S. and Biewener, A. (1991) Bipedal locomotion eﬀects of speed, size and limb posture in birds and humans. J. of Zool. 224, 127–147. [9] Geyer, H. (2005) Simple models of legged locomotion based on compliant limb behavior. PhD Thesis. University of Jena. [10] Geyer, H., Seyfarth, A. and Blickhan, R. (2003) Positive force feedback in bouncing gaits? Proc. R. Soc. Lond. B 270, 2173–2183. [11] Ghigliazza, R., Altendorfer, R., Holmes, P., and Koditschek, D. (2003) A simply stabilized running model. SIAM Journal on Applied Dynamical Systems, 2(2), pp. 187–218. [12] Grillner, S. (1986) in Wenner-Gren International Symposium Series Vol. 45. Neurobiology of Vertebrate Locomotion (eds. Grillner, S., Stein, P. S. G., Stuart, D. G., Forssberg, F. and Herman, R. M.) 505–512 (Macmillan, London). [13] Guenther, M. and Blickhan, R. (2002) Joint stiﬀness of the ankle and the knee in running. J. Biomech. 35, 1459–1474. [14] Hayes, G. and Alexander, R. M. (1983) The hopping gaits of crows (corvidae) and other bipeds. J. Zool. Lond. 200, 205–213. [15] Hof, A. L., van Zandwijk, J. P. and Bobbert, M. F. (2002) Mechanics of human triceps surae muscle in walking, running and jumping. Acta Physiol Scand 174, 17–30. [16] Iida, F. (2005) Cheap design approach to adaptive behavior: walking and sensing through body dynamics. AMAM conference, Ilmenau. [17] Iida, F., Minekawa, Y., Rummel, J., and Seyfarth, A. (2006) Toward humanlike biped robot with compliant legs. Intelligent Autonomous Systems 9. Arai, T. et al. (Eds.). IOS Press, 820–827. [18] Kram, R., Domingo, A., and Ferris, P. F. (1997) Eﬀect of reduced gravity on the preferred walk-run transition speed. J. Exp. Biol. 200, 821–826. [19] Raibert, M (1986) Legged robot that balance. MIT press. [20] Ruina, A., Bertram, J. E. A., and Srinivasana, M. (2005) A collisional model of the energetic cost of support work qualitatively explains leg sequencing in walking and galloping, pseudo-elastic leg behavior in running and the walk-torun transition. J. Theor. Biology, 237, 170–192. [21] Seyfarth, A., Blickhan, R., and van Leeuwen, J. L. (2000) Optimum take-oﬀ techniques and muscle design for long jump. J. Exp. Biol. 203, 741–750.

Running and Walking with Compliant Legs

401

[22] Seyfarth, A., Friedrichs, A., Wank, V., and Blickhan, R. (1999) Dynamics of the long jump. J. Biomech. 32, 1259–1267. [23] Seyfarth, A., Geyer, H., Guenther, M., and Blickhan, R. (2002). A movement criterion for running. J. Biomech. 35, 649–655. [24] Seyfarth, A., Geyer H., and Herr, H. (2003) Swing-leg retraction: a simple control model for stable running. J. Exp. Biol. 206, 2547–2555. [25] Seyfarth, A., Guenther, M., and Blickhan, R. (2001). Stable operation of an elastic three-segmented leg. Biological Cybernetis 84, 365–382. [26] Thorstensson, A. and Roberthson, H. (1987) Adaptations to changing speed in human locomotion: speed of transition between walking and running. Acta Physiol. Scand. 131, 211–214. [27] Zehr, E. P. and Stein, R. B. (1999) What functions do reﬂexes serve during human locomotion? Prog. Neurobiol. 5, 185–205.

Self-stability in Biological Systems – Studies based on Biomechanical Models H. Wagner1 and P. Giesl2 1

2

Biomechanics and Motor Control, WWU M¨ unster, Horstmarer Landweg 62b, 48149 M¨ unster, Germany [email protected] Zentrum Mathematik, TU M¨ unchen, Boltzmannstr. 3, 85747 Garching bei M¨ unchen, Germany [email protected]

Summary. Mechanical properties of complex biological systems are non-linear, e.g. the force-velocity-length relation of muscles, activation dynamics, and the geometric arrangement of antagonistic pair of muscles. The control of such systems is a highly demanding task. Therefore, the question arises whether these mechanical properties of a muscle-skeletal system itself are able to support or guarantee for the stability of a desired movement, indicating self-stability. Self-stability of single joint biological systems were studied based on eigenvalues of the equation of motions and the basins of attraction were analysed using Lyapunov functions. In general, we found selfstability in single muscle contractions (e.g. frog, rat, cui), in human arm and leg movements, the human spine and even in the co-ordination of complex movements such as tennis or basketball. It seems that self-stability may be a general design criterion not only for the mechanical properties of biological systems but also for motor control.

1 Introduction The basis for human and animal motion and locomotion are co-ordinated muscle contractions. Even for very simple movements, a huge number of muscles must be controlled. Therefore, we may ask how humans and animals are able to control such complex neuromusculoskeletal systems. Especially for humans, the easiest way to get an answer is to ask somebody. But we would not expect a meaningful answer because the motor control system acts almost without conscious control. The muscles must generate suﬃcient forces and moments at the joints. However, these forces and moments must be ﬁne tuned in such a way that they can react upon sudden perturbations. This ﬁne tuning may be guaranteed by mono- and poly-synaptic reﬂexes with negative and positive feedback-loops [5]. On the other hand, the mechanical properties of a musculoskeletal system itself may support or even guarantee for suﬃcient sta-

404

H. Wagner and P. Giesl

bility [7, 4]; in these cases motion or locomotion is self-stabilized [10, 2, 1]. Simple biomechanical models, e.g. single muscle contractions or one degree of freedom joint models, can be self-stabilized. But even if these subsystems are self-stabilized the global motion of the multi-body system may still be unstable. However, the global control of a locally self-stabilized system is easier compared to a locally unstable system. The purpose of this paper is to analyze and summarize the self-stabilizing properties of biological systems, i.e. single muscle contractions, single joint movements, and more complex arrangements like the human spine. This paper is a companion paper to Giesl and Wagner (this issue) where the mathematical details to analyse stability and basins of attraction of biomechanical models are given.

2 Single Muscle Quick Release Contractions

muscle length

external load

The ﬁrst step in analyzing self-stability of musculoskeletal systems was to study whether mechanical properties of muscles themselves may provide selfstability. A simple method to investigate the self-stabilizing properties of dissected muscles is a quick-release experiment. In quick-release experiments dissected muscles are loaded with an external weight or force, which will be released suddenly [8]. Typically, after the release the muscles contracted and found a new equilibrium at shorter muscle lengths, indicating that the systems were stable (Fig. 1). As a next step, the quick-release experiments were described and simulated by an equation of motion of a biomechanical model. Based on the equation of motion the stability could be analysed by the eigenvalues of the system. For the given experiments the eigenvalues were negative indicating self-stability. Furthermore, because of the simplicity of the model, the system could be

t0

time

Fig. 1. Schematical representation of a quick-release experiment. At time t0 the external load was released (upper row) and the muscle contracted until a new equilibrium was found between the external force and the muscle force

Self-stability in Biological Systems

405

analysed analytically. From this it could be shown that the classical muscle properties, i.e. the force-velocity relation, force-length relation, are suﬃcient to provide self-stability. As the mechanical properties of dissected muscles support self-stability, the question arise whether muscles within a geometrical arrangement of joints are still able to achieve self-stability.

3 One Degree of Freedom Joint Models In models consisting of a muscle and a simple one-degree of freedom rotational joint the interactions between the muscle and joint properties inﬂuence the stability of the system. In general, the inner muscle moment arm as well as the moment arm according to the external force vector depended on the ﬂexion angle of joints. The individual shapes of these dependencies were inﬂuencing the mechanical stability of the system. We performed quick-release experiments with the elbow-joints of rats and cuis, while the extensor muscles were stimulated [8]. Here again, the systems found new equilibriums after the release of the external loads, indicating stability. A stability analysis based on a biomechanical model resulted in negative eigenvalues, indicating asymptotic stability likewise. For ﬂexor muscles of an elbow joint the moment arms can be calculated from simple trigonometric assumptions, whereas the moment-arms of extensor muscles depends on individual geometrical arrangements. Based on an analytical analysis of the eigenvalues (Giesl and Wagner, this issue) it ext of the geometric function hext with can be estimated that the derivative ∂h∂β respect to the ﬂexion angle β must be positive to support stability. This is guaranteed around elbow ﬂexion angles below ca. 90◦ (Fig. 2).

0.04 0.03

0.02

ble

sta

sta

ble

un

moment arm [m]

0.05

0.01

0

0

20

40

60

80

100

120

140

160

180

flexion angle [°]

Fig. 2. Representation of the ﬂexor moment arm at an elbow joint (length humerus = 0.27 m, length ulna = 0.26 m, insertion of the muscle at the ulna = 0.048 m). The slope of the curve indicates that stability is supported for ﬂexion angles below ca. 90◦

406

H. Wagner and P. Giesl Stable ββ = 70°

Stable ββ = 80°

Unstable ββ = 90°

angular velocity [°/s]

200

100

0

-100

-200 60

70

80

90

100

elbow flexion angle [°]

60

70

80

90

100

elbow flexion angle β [°]

60

70

80

90

100

elbow flexion angle β [°]

Fig. 3. The basins of attraction of equilibrium points in the phase-plane were calculated with Lyapunov-functions of the systems (Giesl and Wagner this issue). The basins of attraction (black lines – extensor activation 50% of maximum voluntary contraction) were reduced with increasing elbow angles (left: equilibrium at β0 = 70◦ ; middle: β0 = 80◦ ; right: β0 = 90◦ ). Furthermore, the basins of attraction were reduced with decreasing level of co-activation (thin gray lines – extensor activation 25% of maximum voluntary contraction)

Experiments with humans supported this result [6]. Here, we determined individual muscle properties of the ﬂexor and extensor muscles. Then the subjects performed quick-release contractions [12]. As a result, the subjects found new equilibriums at lower elbow ﬂexion angles, which was in accordance with the animal experiments. The stability analysis of the experiments resulted in negative eigenvalues for ﬂexion angles below ca. 90◦ , indicating stability. Furthermore, basins of attraction were calculated based on the theory of Lyapunov functions. We found considerable large basins of attraction at low elbow ﬂexion angles and unstable situations for more extended elbows (Fig. 3). Furthermore, the areas of the basins of attraction depended on the co-activation level of the antagonistic muscles. Finally, we analysed simple vertical oscillations of a human leg model [10, 11]. Here additionally, the self-stability was supported by a moving center of rotation at the knee joint, as well as a co-activation of bi-articular muscles, i.e. rectus and biceps femoris muscles. While introducing a joint it is much more complicated to achieve selfstability. Therefore, several solutions to support the stability could be found in biological systems, e.g. co-activation of bi-articular muscles, moving center of rotations.

4 Varying Center of Rotation Model In the previous section we have discussed the stabilizing behaviour of simple biomechanical models of extremities, i.e. elbow joint and knee joint. These

Self-stability in Biological Systems

407

models have a well deﬁned location of the instantaneous center of rotation. In the following we will discuss models with simple one-degree of freedom rotational joints and varying center of rotations. As an example, we may think of lateral ﬂexions of the human lumbar spine. Here we can not deﬁne one single center of rotation but depending on the intermuscular co-ordination of local inter-vertebral muscles the center of rotation will vary between the lowest (L5S1) and the highest functional unit (L1-L2). As a simpliﬁcation we analyzed rotations at one functional unit, e.g. L1-L2, while the other units were assumed to be stiﬀ. How interactions between diﬀerent joints of models with more than one degree of freedom may inﬂuence the stability of the system cannot be answered with the present simpliﬁed model. Is it still possible to self-stabilize such a model with a single pair of antagonistic muscles? First, we described the antagonistic muscles with a Hill-type model including a force-velocity relation, but excluding a force-length relation [9]. As a result two stable areas existed; one around L5-S1 with negative attachment angles of the muscles, e.g. obliquus internus muscle, and another one more cranially for positive attaching angles of the muscles, e.g. obliquus externus or multiﬁdus muscles. But it was impossible to stabilize every center of rotation with only one antagonistic muscular arrangement. Therefore, we improved the model and included a force-length relation such that the muscle was acting on the ascending limb. Now, it was possible to self-stabilize the system at every location of the center of rotation (Fig. 4). We calculated the minimum physiological cross-sectional area (PCSA) of the acting muscles that still can stabilize the system. The physiological cross-sectional area is nearly proportional to the maximum isometric force of a muscle, therefore, a low minimum value of PCSA indicates that only low muscular force is necessary to stabilize the system. For oblique muscle arrangements a minimum physiological cross-sectional area (PCSA) between 50 and 80 cm2 was found, whereas, muscles acting in parallel to the spine were able to stabilize the system with only 7 cm2 . Introducing additional antagonistic muscles could not reduce this minimum value of the PCSA.

5 Discussion The purpose of this paper was to analyze and summarize the self-stabilizing properties of biological systems. We tried to draw a line from single muscle contractions, single joint movements, to more complex arrangements like the human spine. Single muscle contractions could be stabilized based on typical shapes of bio-mechanical properties, i.e. the force-velocity relation and the force-length relation of skeletal muscles. It could be shown analytically, that the typical shape of the force-velocity relation was essential for the stabilisation of single muscle contractions [8]. Furthermore, if acting on the ascending limb of the force-length relation, the typical shape of the active and passive force-length relation supports the self-stability of muscles. Especially for muscle lengths above the optimum

408

H. Wagner and P. Giesl

Potential function [1/s²]

PCSA=7 cm²

PCSA=50 cm²

500

100 320

L2-L3

L1-L2

300

0

L2-L3 L3-L4

280

180

200

Potential function [1/s²]

PCSA =50 cm²

0 -100

L -L L4-L5 3 4 L2-L3 L5-S1

-300

220

140

160

160

180

200

Flexion angle [°]

-1000

L3-L4

180

PCSA =9 cm²

280

L3-L4 L2-L3 L4-L5

260

L5-S1

240

-1500

L1-L2 -2000 220 140

200

160

340

L1-L2 L2-L3 L3-L4

300

180

200

PCSA =10 cm²

300 280

L2-L3 L3-L4 L4-L5 L5-S1

260

L4-L5 L5-S1

L1-L2

220

Flexion angle [°]

PCSA =8 cm²

260

200

L5-S1

L2-L3

Flexion angle [°]

220

L1-L2 -500

L4-L5 L5-S1

Flexion angle [°]

100

-500

-200

L5-S1

160

L3-L4

L4-L5

-100

240 140

00

L1-L2

0

L4-L5 260

PCSA=80 cm² PCSA=8

240

L1-L2 220

160

180

200

Flexion angle [°]

160

180

200

Flexion angle [°]

160

180

200

Flexion angle [°]

Fig. 4. Lyapunov-functions for diﬀerent antagonistic muscle arrangements describing the lumbar spine. The centers of rotations were located at L1-L2, L2-L3, L3-L4, L4-L5, and L5-S1 [3]. For all situations a minimum PCSA for self-stability was calculated. The x-axis shows the ﬂexion angle of the segment, while 180◦ represented the vertical position. The small icons represent the muscular arrangements of the diﬀerent models and the minimum PCSA for each model is given at the top

length the passive properties were important. In sub-maximal contractions, the activation level of the muscle changes the slope of the force-velocity and force-length relation and thus changes the stability of the system. Therefore, sub-maximal co-ordination patterns in physiological motions and locomotion inﬂuences the self-stability of the system [11]. If the muscle was not dissected the geometrical arrangement of joints inﬂuenced the self-stability behaviour. The ﬂexion angle of an elbow joint eﬀected the stability of the system. Extending the elbow more than about 90◦ results in an unstable situation [6]. This simple geometrical dependency may inﬂuence simple movement tasks, e.g. imagine a waiter who should not spill the water in the glass while moving. But also throwing tasks are inﬂuenced by these geometric relations. Compare throwing a basketball with juggling. Whereas in the ﬁrst case the basketball will be released with a nearly extended elbow

Self-stability in Biological Systems

409

joint, in the second case the juggling-ball leaves the hand with a more ﬂexed elbow joint. Especially the stabilisation of the human spine is a challenging task. Here the location of the center of rotation may vary depending on the activation pattern of intervertebral muscles. They are inﬂuencing the stiﬀness around the spine and therefore the location of the instantaneous center of rotation. However, the simulations support the assumption that even for this complicated situation the muscles can guarantee for the self-stabilizing function of the spine. Without changing the activation patterns of the trunk muscles, it seems to be possible to stabilize lateral ﬂexions at diﬀerent centers of rotations [9]. This analysis of the self-stabilizing behaviour of biological systems may inﬂuence diﬀerent scientiﬁc areas, e.g. robotics and prosthetics, and it may hopefully have an eﬀect for the medicine and physiotherapy. A profound understanding of the self-stabilizing properties of biological systems is important while investigating the motor control of complex movements of the whole body. Although the models analyzed here were very simple, we may assume that self-stability seems to be an important criterion for the evolution of humans and animals. If the basin of attraction of an equilibrium point or an envisioned trajectory is considerably large this may oﬀer a great advantage for motor control systems. A self-stable system can be controlled much easier with simple reﬂexes compared to an unstable system. If the system is risking to move out oﬀ the basin of attraction a simple reﬂex may be suﬃcient to push it back into the stable basin. Especially for fast movements, which do not require a high precision, the neuronal system can be unburdened. Considering the control of legged robots this may reduce the requirements on the precision of the sensors and the controller systems.

6 Acknowledgement We would like to thank A. Liebetrau. She calculated the numerical analyses of the human spine model with varying center of rotation.

References [1] Blickhan R, Seyfarth A, Geyer H, Grimmer S, Wagner H, G¨ unther M (accepted) Intelligence by mechanics. Proc. Roy. Soc. Lond. A. [2] Blickhan R, Wagner H, Seyfarth A (2003). Brain or muscles? (S. G. Pandalai, Ed.) Recent Research Developments in Biomechanics. Transworldresearch, Trivandrum, India. 1:215–245. [3] Bogduk, N (2000) Klinische Anatomie von Lendenwirbels¨ aule und Sakrum. Berlin, Springer. [4] Dickinson M H, Farley C T, Full R J, Koehl M A, Kram R, Lehman S (2000) How animals move: an integrative view. Science 288(5463):100–106.

410

H. Wagner and P. Giesl

[5] Geyer H, Seyfarth A, Blickhan R (2003) Positive force feedback in bouncing gaits? Proc R Soc Lond B Biol Sci 270(1529):2173–2183. [6] Giesl P, Meisel D, Scheurle J, Wagner H (2004) Stability analysis of the elbow with a load. J Theor Biol 228(1):115–125. [7] Loeb G E, Brown I E, Cheng E J (1999) A hierarchical foundation for models of sensorimotor control. Exp Brain Res 126(1):1–18. [8] Siebert T, Wagner H, Blickhan R (2003) Not all oscillations are rubbish: Forward simulation of quick-release experiments. JMMB 3(1):107–122. [9] Wagner H, Anders C, Puta C, Petrovitch A, M¨ orl F, Schilling N, Witte H, Blickhan R (2005) Musculoskeletal properties support stability of the lumbar spine. Pathophysiology 12: 257–65. [10] Wagner H, Blickhan R (1999) Stabilizing function of skeletal muscles: an analytical investigation. J Theor Biol 199(2):163–179. [11] Wagner, H, Blickhan R (2003) Stabilizing Function of Antagonistic Neuromusculoskeletal Systems – an Analytical Investigation. Biol Cybern 89:71–79. [12] Wagner H, Giesl P, Blickhan R (subm.) Musculoskeletal stabilization of arm movements – complex or real –. Journal of Mechanics in Medicine and Biology.

Holonomy and Nonholonomy in the Dynamics of Articulated Motion P.-B. Wieber INRIA Rhˆ one-Alpes, 38334 St-Ismier Cedex, France [email protected]

Summary. Walking, running or jumping are special cases of articulated motions which rely heavily on contact forces for their accomplishment. This central role of the contact forces is widely recognized now, but it is rarely connected to the structure of the dynamics of articulated motion. Indeed, this dynamics is generally considered as a complex nonlinear black-box without any speciﬁc structure, or its structure is only partly uncovered. We propose here to precise this structure and show in details how it shapes the movements that an articulated system might realize. Some propositions are made then to improve the design of control laws for walking, running, jumping or free-ﬂoating motions.

1 Introduction Improving the technology of sensors, actuators, computing power, mechanical design, might still be necessary in order to achieve faster and more reliable motions than what can be observed today in humanoid robotics, but better control law designs will probably be one of the key points. The design of feedback laws heavily relies on the understanding that we have of the underlying dynamics, and there may still be room for improvement here. Walking, running or jumping are special cases of articulated motions that strongly depend on contact forces for their accomplishment. This central role of the contact forces is widely recognized now, but it is rarely connected to the structure of the dynamics of articulated motion. Indeed, this dynamics is generally considered as a complex nonlinear black-box without any speciﬁc structure, or its structure is only partly uncovered. We propose here to precise this structure and show in details how it shapes the movements that an articulated system might realize. The existence of a structure in the dynamics of articulated motion has often been recognized [4, 7, 21, 25, 29], but it has never been analyzed as thoroughly as in [18]. Even in a study as precise as what can be found in [21, 20], it is not completely clear that behind the “d’Alembertian wrench” of

412

P.-B. Wieber

the system studied there lies in fact its Newton and Euler equations, as will be stated here. Most of all, the holonomy and nonholonomy of the kinetic momenta and their implications for the locomotion of articulated systems has been rarely if not never discussed outside the works of the authors of [18], with the only exception of space robotics [16, 19]. The analysis that we are going to propose here is therefore deeply inspired by what can be found in [18]. Now, this brilliant work has been made in the framework of Lie algebras, a very powerful framework for high-level analysis of dynamical systems, but which may hide somehow the details appearing in the “real” equations to the reader who doesn’t speak this language ﬂuently. The main point of the present article is therefore to rederive these results without the use of Lie algebras. Doing so calls for an unusual way to derive the dynamics of articulated systems, through the use of Gauss’s principle. We’re going therefore to present this principle and how it can be used to derive the dynamics of articulated systems in Sect. 2. This original way of deriving this dynamics will be helpful then in Sect. 3 in order to precise its inner structure. What this structure implies for the movements that articulated systems can realize will be studied then in Sect. 4, where nonholonomy makes its ﬁrst appearance. Nonholonomy will be the main topic then of Sect. 5, where some implications of this phenomenon for the locomotion of articulated systems are put to light, and where propositions are made to make use of it in order to improve the design of control laws for walking, running, jumping or free-ﬂoating motions.

2 Gauss’s Principle and the Dynamics of Articulated Motion 2.1 Gauss’s Principle Gauss’s principle, equivalent to d’Alembert’s one, can be seen as an extension of the principle of virtual work to the dynamical case. It states that the acceleration of a set of solids subject to some constraints deviates the least possible from the acceleration that it would have had without the constraints [23, 27]. This deviation is measured with the following kinetic metric: 1 1 (¨ xk − x ¨k )T mk (¨ xk − x ¨k ) + (ω˙ k − ω˙ k )T Ik (ω˙ k − ω˙ k ) , (1) D= 2 2 k

with x ¨k and ω˙ k the translation and rotation accelerations of the k th solid, mk its mass, Ik its inertia matrix expressed at its center of mass, and x ¨k and ω˙ k the translation and rotation accelerations that it would have had without the constraints, that is the solutions of the classical Newton and Euler equations, ¨k = fk , mk x Ik ω˙ k − (Ik ωk ) × ωk = τk ,

(2)

Holonomy and Nonholonomy in the Dynamics of Articulated Motion

413

where fk and τk are the forces and torques acting on this solid. Note that the Euler equation is expressed in a frame attached to the solid, as well as the velocity ωk , reason why there is a gyroscopic term (Ik ωk ) × ωk . 2.2 The Dynamics of Articulated Motion Considering now a set of solids constrained to move together by a set of articulations, their dynamics can be computed with the help of Gauss’s principle. The constraints induced by the articulations can be expressed implicitly by describing the positions of the diﬀerent solids of the system in a compact way through a conﬁguration vector q ∈ Rn . Their velocities and accelerations can be related then to the vectors q˙ and q¨ with the help of translation and rotation jacobians: x˙ k = Jtk (q) q˙ , (3) ωk = JRk (q) q˙ and

x ¨k = Jtk (q) q¨ + J˙tk (q, q) ˙ q˙ , ω˙ k = JRk (q) q¨ + J˙Rk (q, q) ˙ q˙ .

(4)

Introducing these relations in the deﬁnition (1) of the deviation D and solving the Newton and Euler equations (2) for x ¨k and ω˙ k , the optimality condition for the minimization of this deviation turns into (we skip these calculations which are completely straightforward)

with M (q) =

∂D = M (q) q¨ + N (q, q) ˙ q˙ − F = 0 , ∂ q¨

(5)

T T Jtk mk Jtk + JRk Ik JRk ,

(6)

T T T Jtk mk J˙tk + JRk Ik J˙Rk − JRk (Ik JRk q) ˙ × JRk ,

(7)

T T Jtk fk + JRk τk .

(8)

k

N (q, q) ˙ =

k

F=

k

We end up therefore with a classical Lagrangian description of the dynamics of a system of articulated bodies with an inertia matrix M (q), nonlinear dynamical eﬀects N (q, q) ˙ q˙ and generalized forces F acting on the system. Note that we’re taking some liberties in (7) and in the following with the notation of the cross-product by considering that given a vector v ∈ R3 , the notation (v)× means in fact multiplying by the classical anti-symmetric matrix   0 −v3 v2    v3 0 −v1  . −v2 v1 0

414

P.-B. Wieber

The point here is that even though the deﬁnition (6) of the inertia matrix is identical to what can be found in usual robotics textbooks [15, 26], it is not the case for the deﬁnition (7) of the nonlinear eﬀects. Indeed, these nonlinear eﬀects are generally presented through Christoﬀel symbols that completely hide the structure that can be seen here, and this structure is going to be very useful in analyzing the dynamics of articulated motion. For example, one can observe immediately with the deﬁnitions here that the matrix M˙ − 2N is anti-symmetric.

3 Inner Structure of the Dynamics of Articulated Motion 3.1 The Structure of the Conﬁguration Vector In the case of locomoting or free-ﬂoating articulated systems, the conﬁguration vector q ∈ Rn introduced in the previous section stitches in fact together three very distinct informations,   qˆ   q =  x0  , (9) θ0 where qˆ describes the positions of the articulations of the system, and x0 and θ0 the position and the orientation of a reference frame attached to one solid of the system. This structure of the conﬁguration vector can be found then in all the kinematic and dynamic equations of the system, to begin with the translation and rotation jacobians that were introduced in (3). Indeed, if we consider the ˆ˙k and ω ˆ k of the k th solid with respect to translation and rotation velocities x the reference frame that we have just introduced, they must be composed with the velocities x˙ 0 and ω0 of this reference frame itself in order to obtain the total translation and rotation velocities x˙ k and ωk of the solid. This is done through the following classical composition rules (remember that the rotation velocities ω are expressed in local frames): ˆ˙k + x˙ 0 + (R0 (θ0 ) ω0 ) × (xk −x0 ) , x˙ k = R0 (θ0 ) x ˆ k + R0 (θ0 ) ω0 Rk (q) ωk = Rk (q) ω with R0 (θ0 ) and Rk (q) the orientation matrices of the k th solid and the reference frame with respect to the inertial frame. Now, if we use the fact that ˆ˙ ˆ˙k and ω ˆ k are solely related to the vector q, the velocities x ˆ˙k = Jˆtk (ˆ q ) qˆ˙ , x ω ˆ k = JˆRk (ˆ q ) qˆ˙ ,

Holonomy and Nonholonomy in the Dynamics of Articulated Motion

415

and the fact that the rotation velocity ω0 of the reference frame can be related to the angular velocity θ˙0 , ω0 = JR0 (θ0 ) θ˙0 , these composition rules turn into (with shortened notations) x˙ k = R0 Jˆtk qˆ˙ + x˙ 0 − (xk −x0 ) × R0 JR0 θ˙0 , ωk = JˆRk qˆ˙ + RkT R0 JR0 θ˙0 , where we can observe that the translation and rotation jacobians that were introduced in (3) exhibit a structure corresponding exactly to the structure (9) of the conﬁguration vector: −(x −x ) × R J Jtk = R0 Jˆtk 3×3 k 0 0 R0 JRk = JˆRk with

3×3

and

3×3

3×3

RkT R0 JR0

(10)

a zero and an identity matrix.

3.2 Back to Newton and Euler Equations T T Replacing this structure of the jacobians Jtk and JRk in (6)–(8), we obtain a structure of the inertia and non-linear eﬀects matrices and of the generalized forces that corresponds once again exactly to the structure (9) of the conﬁguration vector:   T T Jˆtk R0T mk Jtk + JˆRk Ik JRk     (11) M (q) =  mk Jtk  ,   k T JR0 R0T (xk −x0 ) × mk Jtk + Rk Ik JRk

N (q, q) ˙ =   T T T Jˆtk R0T mk J˙tk + JˆRk Ik J˙Rk − JˆRk (Ik JRk q) ˙ × JRk      , (12)  mk J˙tk   k T T ˙ ˙ JR0 R0 (xk −x0 ) × mk Jtk + Rk Ik JRk − Rk (Ik JRk q) ˙ × JRk   T T Jˆtk R0T fk + JˆRk τk     F= (13)  fk  .   k T JR0 R0T (xk −x0 ) × fk + Rk τk The dynamics (5) can be split therefore in three lines, each one with a very speciﬁc structure. Particularly interesting are the two last ones: with the help of relations (3) and (4), the line in the middle gives

416

P.-B. Wieber

mk x ¨k =

k

fk

k

and the last line gives T R0T (xk − x0 ) × mk x ¨k + Rk Ik ω˙ k − Rk (Ik ωk ) × ωk = JR0 k T R0T JR0

(xk −x0 ) × fk + Rk τk

k T what, putting aside the multiplication by JR0 R0T , corresponds to an equality between the dynamical momentum of rotation of the whole system and the sum of all the torques applied to it, both expressed with respect to x0 in an absolute reference frame. What appears here are therefore simply a Newton and an Euler equation for the whole system.

3.3 Forces Acting on the System We’re going to consider three diﬀerent types of generalized forces, those which are most generally found acting on systems of articulated bodies: the gravity Fg , the control forces Fu and the contact forces Fc . The gravity forces and torques acting on each solid are simply fk = mk g and τk = 0, where g is simply the vector of the gravity ﬁeld. Replacing this in (13) gives   T Jˆtk R0T mk g     (14) Fg =  mk g  .   k T R0T (xk −x0 ) × mk g JR0 Now, we can observe from the structure (10) of the translation and rotation jacobians that we obviously have     0 0     Jtk  g  = g and JRk  g  = 0 0 0 so that a short inspection of (11) leads to the fact that   0   Fg = M (q)  g  . 0 This implies that the dynamics (5) of the system under the action of gravity can be written as:

Holonomy and Nonholonomy in the Dynamics of Articulated Motion

417



  0    M (q) q¨ −  g  + N (q, q) ˙ q˙ = 0 . 0 Here lies the obvious observation that even on an articulated system, the action of gravity is nothing but a linear acceleration in the direction of the gravity ﬁeld. Concerning control forces, we’ll consider that the only ones acting on the system are internal forces acting between the diﬀerent solids of the system, coming from the action of muscles or actuators on the articulations of the system. In this case, the application of Newton’s law of action and reaction leads us immediately to the conclusion that their sum (13) is of the form   u   (15) Fu =  0  , 0 and we’re not going to precise any deeper the structure of the vector u for the analysis undertaken here. Concerning the contact forces between the system and its environment, very diﬀerent models exist [3] and we will focus only on the fact that whatever the model, there are always similar constraints on their direction and amplitude due to unilaterality and limited friction. We will gather all these limitations in a vector inequality relating these forces to the position of the system: (16) A(q, Fc ) ≤ 0

4 Motions that an Articulated System can Realize 4.1 With only Control Forces If we introduce the center of mass xG of the system, xG =

1 mk xk with m = mk , m k

k

the Newton equation of the whole system appears to be simply fk . mx ¨G =

(17)

k T If we get rid of the multiplication by JR0 R0T in the Euler equation and if we add to it mk x ¨k = (x0 − xG ) × fk (x0 − xG ) × k

k

418

P.-B. Wieber

in order to express the momentum of rotation and the sum of torques with respect to the center of mass instead of x0 , we obtain (xk −xG )×mk x ¨k +Rk Ik ω˙ k −Rk (Ik ωk )×ωk = (xk − xG )×fk +Rk τk , k

k

(18) what is equal to . / d (xk − xG ) × mk x˙ k + Rk Ik ωk = (xk − xG ) × fk + Rk τk , dt k

k

where the left hand side appears to be simply the derivative of the kinetic momentum of rotation of the system. This way, we can see that if the system is under the action of only the control forces (15), we have the obvious conservation of the kinetic momenta:

m x˙ G = Constant , (xk − xG ) × mk x˙ k + Rk Ik ωk = Constant .

k

If the system starts with a zero velocity, these constants are zero and the ﬁrst equation implies that whatever the control forces employed, the center of mass of the system will remain unmoved. The implications of the second equation are more subtle since it is a nonholonomic constraint, a relation between the velocities of the bodies of the system that doesn’t imply a relation between their positions and orientations: it constrains the movements that the system can realize, but not the positions that it can reach. We’re going to spend more time in Sect. 5 on the many implications of this nonholonomy, but we can already stress that notwithstanding this conservation of the kinetic momentum of rotation, the position of the articulations qˆ and the orientation of the system θ0 can be controlled together to any desired value, with the only action of muscles or actuators on the articulations of the system: controlling the articulations of the system is enough to control also its orientation. 4.2 With Gravity Forces If the system is under the action of the gravity forces (14) in addition to the control forces considered earlier, we can observe that the only modiﬁcation to the movements of the system is that its center of mass will be linearly accelerated along the gravity vector instead of staying idle: x ¨G = g . The conservation of the kinetic momentum of rotation is unchanged, and so is the conclusion about its nonholonomy, and so is therefore the fact that

Holonomy and Nonholonomy in the Dynamics of Articulated Motion

419

Fig. 1. Falling back on the feet thanks to nonholonomy [6]

controlling the articulations of the system is enough to control also its orientation. This can be observed in the most usual example of articulated system under the action of gravity and muscles, a cat falling which always manages to fall back on its feet. Figure 1 shows the similar case of a dog, and a close inspection of this stop-motion allows to understand how the rotation of the body is undertaken even though the kinetic momentum of rotation is kept unchanged to zero: limbs are moved back and forth with diﬀerent lengths, and therefore with diﬀerent inertial properties. We’re going to focus again later on this very simple principle. 4.3 With Contact Forces Of the three types of forces considered in Sect. 3.3, the contact forces appear therefore to be the only one able to generate movements of the center of mass of the system in any way other than falling down. The locomotion of articulated systems completely rely therefore on the availability of contact forces. Now, we have seen that these contact forces are limited because of unilaterality and limited friction, as has been resumed in the general inequality (16). Every movement undertaken by an articulated system has therefore to comply with these limitations. Research in biped locomotion has been extensively focusing on this question, and in diﬀerent ways, either focusing explicitly on the dynamic momenta that appear in the Newton and Euler equations (17)–(18) as in the Resolved Momentum Control approach [10, 11], in the Zero Moment Point analysis [8, 28] and in other similar works [9, 13, 22], or treating more globally the force allocation problem directly in the Lagrangian Equation (5) as in [7, 14, 29]. Since we have seen that this Lagrangian equation embeds explicitly the Newton and Euler equations of the system, we can conclude that these two ways of approaching the problem are exactly equivalent. The contact forces are also the only ones which can have an eﬀect on the momentum of rotation of the system and therefore potentially remove, within the bounds of the inequalities (16), all the dynamical constraints that existed on the movements of articulated systems. This point is less crucial

420

P.-B. Wieber

Fig. 2. 22 degrees of rotation of the whole body in the sagittal plane induced by 24 steps of the walking pattern recorded in [31] and replayed on a simple free-ﬂoating biomechanical model, what corresponds to a complete turn in 393 steps

though than the problem of moving the center of mass since we have seen that the nonholonomy of this momentum of rotation allows for a control of the orientation of the system through the control of its articulations. This is going to be the main topic of the next section.

5 Some Implications on the Locomotion of Articulated Systems 5.1 Nonholonomy of the Momentum of Rotation When Walking We have seen in Sect. 4.2 that the nonholonomy of the momentum of rotation can be observed in the case of the very speciﬁc movements that cats and dogs realize in order to fall back on their feet. It is in fact a very general phenomenon that can be observed even when walking. When replaying for example the walking pattern recorded in [31] on a simple biomechanical model with no external forces and therefore with a kinetic momentum of rotation constantly equal to zero, we can indeed observe a rotation of the whole body happening in the sagittal plane (Fig. 2). Note that a general property of such nonholonomic constraints is that the outcome of the movement doesn’t depend on its speed, but solely on its shape, so we can measure the nonholonomy here as an amount of rotation of the whole body for each step of walking, depending solely on the shape of this step, regardless of its actual speed. In this case, that gives 1/393rd of a complete turn of the body per step accomplished. A more accurate measurement of this phenomenon can be obtained on a robotic system such as the HRP-2 robot, for which the inertial properties and

Holonomy and Nonholonomy in the Dynamics of Articulated Motion

421

0.007

0.006

0.005

0.004

0.003

0.002

0.001

0.000 0.00

0.04

0.08

0.12

0.16

0.20

0.24

0.28

Fig. 3. Fraction of a complete turn in the sagittal plane of the whole body of the HRP-2 robot induced by a step of walking as a function of the height of this step (in meters), with up to 1/159th of a complete turn for a 25 cm high step

the movements actually realized can be known with very good precision. We can accurately measure then the amount of rotation of the whole body as a function of the shape of the step, for example its height in Fig. 3, reaching here 1/159th of a complete turn of the body for a 25 cm high step. Note that this phenomenon is intrinsic to the movements of the legs when walking, back and forth with diﬀerent lengths in order to avoid undesired contacts with the ground, and therefore back and forth with diﬀerent inertial properties, just as what has been observed in the case of the dog falling in Fig. 1. Keeping the body upright when walking necessarily calls therefore for a second phenomenon in order to counterbalance this rotation. One can imagine the arms making the exact inverse of the movements of the legs, with diﬀerent lengths when moving back and forth, but this is not what can be observed in natural walking patterns, without mentionning the cases when the arms don’t even move back and forth, when holding a heavy object or when keeping arms crossed. The counterbalancing phenomenon that can be observed is in fact a non-zero mean kinetic momentum of rotation in the direction opposite to the rotation induced by the legs’ movements (Fig. 4). With these observations in mind, the proposition made in [22] of controlling the kinetic momentum of rotation of a walking system to keep it to zero appears to be problematic: a zero kinetic momentum of rotation appears to be incompatible with walking,

422

P.-B. Wieber

2

1

0

−1

−2

0

1

2

3

4

5

6

7

8

9

Fig. 4. Kinematic momentum of rotation of the HRP-2 robot in the sagittal plane (in kg.m2 .s−1 ) as a function of time (in seconds) when executing 25 cm high steps on the ground

and designing a control law to do so will obviously lead to an error either in the tracking of the walking movements or in the tracking of the zero kinetic momentum, both potential sources of instability. It could be tempting then to control this kinetic momentum of rotation to a non-zero value, following for example the framework presented in [11], but the choice of the desired value may not be easy to decide since it should be made according to the speciﬁc shape of the walking pattern being considered. On top of that, the kinetic momentum of rotation can be observed to vary strongly when walking (Fig. 4), so the choice of a constant value might be an unnecessary limitation. But most of all, as we have already observed in Fig. 2, the value of the kinetic momentum of rotation is only scarcely related to the actual rotations that the articulated system is going to realize: deciding a value for this momentum decides in fact almost nothing about the movement to come. It may not be sure therefore whether focusing speciﬁcally on the control of the kinetic momentum of rotation as proposed in [1, 9, 11, 13, 22] is the best option, after all. At least, it is incomplete in controlling the motion of an articulated system.

Holonomy and Nonholonomy in the Dynamics of Articulated Motion

423

5.2 Some Hints for Improving the Control of Walking, Running, Jumping and Free Floating Motions of Articulated Systems The amounts of rotation measured in the previous section, 1/393rd to 1/159th of a complete turn, might look small if not negligible, the main reason that led to the erroneous conclusion in [22] that they might be strictly zero. But we should not forget the example of the cat always falling back on its feet: nonholonomy can be a very precious tool in the control of the locomotion of articulated systems, not to be underestimated. Of course, solely varying the height of the steps as in the previous section is not a serious solution to completely stabilize a walking movement, but this is an indication that varying the shape of the walking pattern can help improve this stability, especially if motions of the whole body are involved, for greater eﬃciency (the arms can be a precious source of inertial eﬀects). At least, this can be a valuable addition to the methods already known for stabilizing walking motions such as varying the step lengths or the speed [30]. Useful in the case of walking, making use of this nonholonomy can become an absolute necessity in the case of running, jumping and free ﬂoating motions, being the only way to control the orientation of the system when contact forces are not available anymore. Now, it is well known as a side eﬀect of a famous theorem due to R.W. Brockett [2] that the complete control of a system with nonholonomic constraints can’t be realized with continuous timeinvariant feedback control laws: discontinuous or time-varying control laws are a necessity in this case [5]. This explains why the use of this nonholonomy is out of reach of the control laws proposed in [12, 17, 25] which are all continuous and time-invariant. Using this nonholonomy is a well established control method in space robotics [16, 19, 24], but the only control law making such a use of this nonholonomy that seems to have been proposed so far for humanoid locomotion is for running, in [4]. There, a time-varying control law is proposed in the ﬂight phase by simply letting an additional degree of freedom in the design of the trajectories be used to control the orientation of the system. We can observe there that calling for discontinuous or time-varying control laws doesn’t necessarily imply complex solutions: local modiﬁcations of the shape of the limb trajectories can be just enough, what can be made even easier from a computational point of view with the help of a library of precomputed motions.

6 Conclusion The core of this short note has been a precise description of the inner structure of the dynamics of articulated systems of bodies, establishing in particular how the Newton and the Euler equations of the whole system are very simply and directly embedded inside its Lagrangian dynamics, implying an immediate equivalence between the approaches that focus on the former equations and

424

P.-B. Wieber

the approaches that focus on the latter one. This whole analysis has been made possible in the ﬁrst place thanks to an original derivation of this Lagrangian dynamics through the use of Gauss’s principle. Conclusions have been derived then concerning the holonomy of the Newton equation and the nonholonomy of the Euler equation, implying the necessity of contact forces for articulated systems to realize translations, but not rotations for which joint forces are enough. A speciﬁc analysis of this latter phenomenon has been undertaken then in the case of walking motions, and propositions have been ﬁnally made to make use of it for improving the design of control laws for stabilizing walking, running, jumping, and more generally every kind of articulated movements on the ground and in the air.

References [1] M. Abdallah and A. Goswami. A biomechanically motivated two-phase strategy for biped upright balance control. In Proceedings of the IEEE International Conference on Robotics & Automation, 2005. [2] R.W. Brockett. Asymptotic stability and feedback stabilization. In Diﬀerential Geometric Control Theory. Birkhauser, 1983. [3] B. Brogliato. Nonsmooth Impact Mechanics. Springer-Verlag, 1996. [4] C. Chevallereau, E.R. Westervelt, and J.W. Grizzle. Asymptotically stable running for a ﬁve-link, four-actuator, planar bipedal robot. International Journal of Robotics Research, 2005. [5] A. De Luca and G. Oriolo. Modelling and control of nonholonomic mechanical systems. Kinematics and Dynamics of Multi-Body Systems, CISM Courses and Lectures, 1995. [6] M. Frizot. Etienne-Jules Marey : Chronophotographe. Nathan, 2001. [7] Y. Fujimoto, S. Obata, and A. Kawamura. Robust biped walking with active interaction control between foot and ground. In Proc. of the IEEE International Conference on Robotics & Automation, 1998. [8] A. Goswami. Postural stability of biped robots and the Foot-Rotation Indicator (FRI) point. International Journal of Robotics Research, 1999. [9] A. Goswami and V. Kallem. Rate of change of angular momentum and balance maintenance of biped robots. In Proceedings of the IEEE International Conference on Robotics & Automation, 2004. [10] K. Harada, S. Kajita, K. Kaneko, and H. Hirukawa. An analytical method on real-time gait planning for a humanoid robot. In International Conference on Humanoid Robotics, 2004. [11] S. Kajita, F. Kanehiro, K. Kaneko, K. Fujiwara, K. Harada, K. Yokoi, and H. Hirukawa. Resolved momentum control: Humanoid motion planning based on the linear and angular momentum. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots & Systems, 2003. [12] S. Kajita, T. Nagasaki, K. Kaneko, K. Yokoi, and K. Tanie. A running controller of humanoid biped hrp-2lr. In Proceedings of the IEEE International Conference on Robotics & Automation, 2005.

Holonomy and Nonholonomy in the Dynamics of Articulated Motion

425

[13] T. Komura, H. Leung, S. Kudoh, and J. Kuﬀner. A feedback controller for biped humanoids that can counteract large perturbations during gait. In Proceedings of the IEEE International Conference on Robotics & Automation, 2005. [14] K. L¨ oﬄer, M. Gienger, and F. Pfeiﬀer. Sensor and control design of a dynamically stable biped robot. In Proceedings of the IEEE International Conference on Robotics & Automation, 2003. [15] R.M. Murray, Z. Li, and S. Sastry. A Mathematical Introduction to Robotic Manipulation. CRC Press, 1994. [16] Y. Nakamura and R. Mukherjee. Exploiting nonholonomic redundancy of space robots through hierarchical Lyapunov functions. IEEE Transactions on Robotics and Automation, 1993. [17] V. Nunez and N. Nadjar-Gauthier. Control strategy for vertical jump of humanoid robots. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots & Systems, 2005. [18] J. Ostrowski and J. Burdick. Geometric perspectives on the mechanics and control of robotic locomotion. In Proc. of the International Symposium on Robotics Research, 1995. [19] E. Papadopoulos. Nonholonomic behaviour in free-ﬂoating space manipulators and its utilization. Nonholonomic Motion Planning, Xu, Y. and Kanade, T. (Eds.), 1993. [20] J. Park. Principle of dynamical balance for multibody systems. Multi-Body Systems and Dynamics, 2005. [21] J. Park, Y. Youm, and W.-K. Chung. Control of ground interaction at the zero-moment point for dynamic control of humanoid robots. In Proceedings of the IEEE International Conference on Robotics & Automation, 2005. [22] M. Popovic, A. Hofmann, and H. Herr. Zero spin angular momentum control: deﬁnition and applicability. In International Conference on Humanoid Robotics, 2004. [23] E.J. Routh. Dynamics of a system of rigid bodies, part 1. Dover Publications, 1905. [24] V. Schulz, R. Longman, and H. Bock. Computer-aided motion planning for satellite mounted robots. M 2 AS, 1998. [25] L. Sentis and O. Khatib. Control of free-ﬂoating humanoid robots through task prioritization. In Proceedings of the IEEE International Conference on Robotics & Automation, 2005. [26] M.W. Spong and M. Vidyasagar. Robot Dynamics and Control. John Wiley & Sons, 1989. [27] N. Tch¨ı¿ 21 aev. M¨ı¿ 21 anique rationnelle. ¨ı¿ 21 itions Mir Moscou, 1993. [28] M. Vukobratovi´c, B. Borovac, D. Surla, and D. Stoki´c. Biped Locomotion : Dynamics, Stability, Control and Application. Springer-Verlag, 1990. [29] P.-B. Wieber. Constrained dynamics and parametrized control in biped walking. In Proceedings of the International Symposium on Mathematical Theory of Networks and Systems, 2000. [30] P.-B. Wieber and C. Chevallereau. Online adaptation of reference trajectories for the control of walking systems. Technical Report 5298, INRIA Rhˆ one-Alpes, 2004. [31] D.A. Winter. Biomechanics and Motor Control of Human Movement, Second Edition. John Wiley & Sons, New York, 1990.

Dynamic Stability of a Simple Biped Walking System with Swing Leg Retraction M. Wisse1 , C.G. Atkeson2 , and D.K. Kloimwieder2 1

2

Delft University of Technology www.dbl.tudelft.nl Carnegie Mellon University www.cs.cmu.edu/∼cga

Summary. In human walking, the swing leg moves backward just prior to ground contact, i.e. the relative angle between the thighs is decreasing. We hypothesize that this swing leg retraction may have a positive eﬀect on gait stability, because similar eﬀects have been reported in passive dynamic walking models, in running models, and in robot juggling. For this study, we use a simple inverted pendulum model for the stance leg. The swing leg is assumed to accurately follow a time-based trajectory. The model walks down a shallow slope for energy input which is balanced by the impact losses at heel strike. With this model we show that a mild retraction speed indeed improves stability, while gaits without a retraction phase (the swing leg keeps moving forward) are consistently unstable. By walking with shorter steps or on a steeper slope, the range of stable retraction speeds increases, suggesting a better robustness. An optimization of the swing leg trajectory of a more realistic model also consistently comes up with a retraction phase, and indeed our prototype demonstrates a retraction phase as well. The conclusions of this paper are twofold; (1) use a mild swing leg retraction speed for better stability, and (2) walking faster is easier.

1 Introduction In human walking, the swing leg moves forward to maximal extension and then it moves backward just prior to ground contact. This backward motion is called “swing leg retraction”; the swing foot stops moving forward relative to the ﬂoor and slightly moves backward before touching the ground. In biomechanics it is generally believed that humans apply this eﬀect (also called “ground speed matching”) in order to reduce heel strike impacts. However, we believe that there is a diﬀerent way in which swing leg retraction can have a positive eﬀect on stability; a fast step (too much energy) would automatically lead to a longer step length, resulting in a larger energy loss at heel strike. And conversely, a slow step (too little energy) would automatically lead to a

428

M. Wisse et al.

shorter step length, resulting in less heel strike loss. This could be a useful stabilizing eﬀect for walking robots. The primary motivation to study swing leg retraction comes from our previous work on passive dynamic walking [15, 4, 19]. Passive dynamic walking [11] robots can demonstrate stable walking without any actuation or control. Their energy comes from walking downhill and their stability results from the natural dynamic pendulum motions of the legs. Interestingly, such walkers possess two equilibrium gaits, a “long period gait” and a “short period gait” [5, 12]. The long period gait has a retraction phase, and this gait is the only one that can be stable. The short period gait has no swing leg retraction. This solution is usually dismissed, as it never provides passively stable gaits. More motivation stems from work on juggling and running, two other underactuated dynamic tasks with intermittent contact. The work on juggling [14] featured a robot that had to hit a ball which would then ballistically follow a vertical trajectory up and back down until it was hit again. The research showed that stable juggling occurs if the robot hand is following a well chosen trajectory, such that its upward motion is decelerating when hitting the ball. The stable juggling motion required no knowledge of the actual position of the ball. We feel that the motion of the hand and ball is analogous to that of the swing leg and stance leg, respectively. Also analogous is the work on a simple point-mass running model [16]. It was shown that the stability of the model was signiﬁcantly improved by the implementation of a retraction phase in the swing leg motion. It has been suggested [13] that this eﬀect also appears in walking. In this paper we investigate the stabilizing inﬂuence of the swing leg retraction speed just prior to heel strike impact. We use a Poincar´e map analysis of a simple point-mass model (Sect. 2). The results are shown in Sect. 3, including a peculiar asymmetric gait that is more stable than any of the symmetric solutions. Section 4 reports that the results are also valid for a model with a more realistic mass distribution. The discussion and conclusion are presented in Sects. 5 and 6.

2 Simulation Model and Procedure The research in this paper is performed with an inverted pendulum model consisting of two straight and massless legs (no body) and a single point mass at the hip joint, see Fig. 1. Straight legged (“compass gait”) models are widely used as an approximation for dynamic walking [7, 6, 9, 8, 5]. 2.1 Stance Leg The stance leg is modeled as a simple inverted pendulum of length 1 (m) and mass 1 (kg) (Fig. 1). It undergoes gravitational acceleration of 1 (m/s2 ) at an angle of γ following the common approach to model a downhill slope

Dynamic Stability of a Simple Biped Walking System

429

m=1 g=1 φ

l=1

θ swing leg retraction

γ

Fig. 1. Our inverse pendulum model, closely related to the “Simplest Walking Model” of [5]

in passive dynamic walking. It has one degree of freedom denoted by θ, see Fig. 1. The foot is a point and there is no torque between the foot and the ﬂoor. The equation of motion for the stance leg is: θ¨ = sin(θ − γ)

(1)

which is integrated forward using a 4th order Runge-Kutta integration routine with a time step of 0.001 (s). 2.2 Swing Leg The swing leg is modeled as having negligible mass. Its motion does not aﬀect the hip motion, except at the end of the step where it determines the initial conditions for the next step. A possible swing leg motion is depicted in Fig. 2 with a dashed line. As is standard with compass gait walkers, we ignore the brief but inevitable foot scuﬃng at midstance. The swing leg motion at the end of the step is a function of time which we construct in two stages. First we choose at which relative swing leg angle φ (See Fig. 1) heel strike should take place, φlc . This is used to ﬁnd a limit cycle (an equilibrium gait), which provides the appropriate step time, Tlc . Second, ˙ The swing leg angle φ is then created as a we choose a retraction speed φ. ˙ linear function of time going through the point {Tlc , φlc } with slope φ. 2.3 Transition The simulation transitions from one step to the next when heel strike is detected, which is the case when φ = −2θ. An additional requirement is that the foot must make a downward motion, resulting in an upper limit for the forward swing leg velocity φ˙ < −2θ˙ (note that θ˙ is always negative in normal walking, and note that swing leg retraction happens when φ˙ < 0). In our simulation, we use a third order polynomial to interpolate between two

M. Wisse et al.

φ and θ (rad)

430

This paper addresses linearized stability; the width of this region is very small, unlike the drawing.

0.6 Swing leg retraction

Nominal heel strike

0.4

Exact swing motion is irrelevant

0.2

Range of stable retraction speeds Continued motion if heel strike is delayed

0 Nominal motion of stance leg θ

-0.2 Nominal motion of swing leg φ

-0.4 0

1

2

3

4

5 time (s)

Fig. 2. The ﬁgure shows an example trajectory and it shows the stable region (hatched area) for retraction speeds. Only the swing leg trajectory around heel strike is important; the swing leg by itself has no dynamic eﬀect on the walking motion other than through foot placement

simulation data points in order to accurately ﬁnd the exact time and location of heel strike. The transition results in an instantaneous change in the velocity of the point mass at the hip, see Fig. 3. All of the velocity in the direction along the new stance leg is lost in collision, the orthogonal velocity component is retained. This results in the following transition equation: θ˙+ = θ˙− cos φ

(2)

in which θ˙− indicates the rotational velocity of the old stance leg, and θ˙+ that of the new stance leg. At this instant, θ and φ ﬂip sign (due to relabeling of the stance and swing leg). Note that in Eq. (2) φ could equally well be replaced with 2θ. The instant of transition is used as the start of a new step for the swing leg controller; in the case that a disturbance would make step n last longer than usual, then the start of the swing leg trajectory for step n + 1 is postponed accordingly. Thus, although the swing leg motion is a time-based trajectory independent of the state of the stance leg, it does depend on foot contact information for the start of the trajectory.

Dynamic Stability of a Simple Biped Walking System

431

.+ θ φ φ

. θ-

Fig. 3. At heel strike the velocity of the point mass is redirected. All velocity along the length of the new stance leg is lost, so that θ˙+ = θ˙− cos φ

2.4 Finding Limit Cycles The model exhibits a limit cycle if the initial conditions of step n + 1 are exactly equal to those of step n. For this model, the only independent initial ˙ The motion of the conditions are the stance leg angle (θ) and its velocity (θ). swing leg is fully trajectory controlled; we assume that it accurately follows the desired trajectory. The ﬁrst step of ﬁnding a limit cycle is to guess initial conditions that are near a hypothesized limit cycle, either through experience or by starting from a known limit cycle for similar parameter values. This provides initial guess {θ0 , θ˙0 }. Then a Newton-Raphson gradient-based search algorithm is applied on the diﬀerence between {θ0 , θ˙0 } and the initial conditions of the next step, which we obtain through forward simulation. The search algorithm terminates when the norm of the diﬀerence is smaller than 1e−9 . The search algorithm uses a numerically obtained gradient J which is also used for the stability analysis as described in the next paragraph. Note that this procedure can ﬁnd unstable as well as stable limit cycles. 2.5 Poincar´ e Stability Analysis The stability of the gait is analyzed with the Poincar´e mapping method, which is a linearized stability analysis of the equilibrium gait. The Poincar´e mapping method perturbs the two independent initial conditions and monitors the eﬀect on the initial conditions for the subsequent step. Assuming linear behavior, the relation between the original perturbations at step n and the resulting perturbations at step n + 1 is captured in the Jacobian matrix J, as in: / . / . ∆θn ∆θn+1 =J (3) ∆θ˙n+1 ∆θ˙n If the magnitudes of both of the eigenvalues of J are smaller than 1, then errors decay step after step and the gait is stable. The eigenvalues could have imaginary parts, as was the case for the passive model [5], but in the model with trajectory control they have no imaginary parts.

432

M. Wisse et al.

2.6 Nominal Gait We have chosen the steady passive gait with a slope of γ = 0.004 rad as a basis of reference for walking motions. For the passive model, γ is the only parameter, and for γ = 0.004 there exists only one unstable equilibrium gait (the “short period solution”) and one stable equilibrium gait (the “long period solution”). We use the latter as our reference gait. The initial conditions and the step time of that gait are listed in Table 1. Table 1. Initial conditions and step time for steady walking of the passive walking model at a slope of γ = 0.004 rad. Note that the initial velocity for the swing leg φ˙ is irrelevant for our study, because the swing leg motion is fully trajectory controlled θ φ = −2θ θ˙ φ˙ step time

0.1534 rad 0.3068 rad −0.1561 rad/s 0.0073 rad/s 3.853 s

3 Results 3.1 Nominal Limit Cycle For the given gait of Table 1 on a given slope of γ = 0.004 rad, the only ˙ how fast is the swing parameter that we can vary is the retraction speed φ; leg moving rearward (or forward, depending on the sign) just prior to heel strike. This parameter has no inﬂuence on the nominal gait, but it does change the behavior under small disturbances as captured by the Poincar´e stability analysis. Note that the swing leg will follow a ﬁxed time-based trajectory independent of the disturbances on the initial conditions. The stability results are shown in Fig. 4; the eigenvalues of J on the vertical axis versus the retraction speed φ˙ at the horizontal axis. A positive value for φ˙ indicates that the swing leg keeps moving forward. A value of zero means that the swing leg is being held at the heel strike value φ = 0.3068 and so the foot comes down vertically. A negative value for φ˙ indicates the presence of a retraction phase. Figure 4 shows that stable gaits emerge for retraction speeds of −0.18 < φ˙ < 0.009, and that the fastest convergence will be obtained with φ˙ = −0.09 since the maximum absolute eigenvalue is minimal at that point. In other words, swing leg retraction is not necessary for stable walking, but errors will deﬁnitely decay faster if the swing leg motion does include a retraction phase. Also, even though some forward swing leg motion (φ˙ > 0) is allowable, this

Dynamic Stability of a Simple Biped Walking System

433

1

|λ|

0.5

0

-0.5

-1

stable retraction speeds -0.2

-0.18

-0.16

-0.14

-0.12

-0.1

-0.08

-0.06

-0.04

-0.02

0

. retraction speed φ (rad/s)

Fig. 4. The graph shows the range of stable retraction speeds. The stability is indicated with eigenvalues of J on the vertical axis. The walking motion is stable if the eigenvalues are between −1 and 1. The horizontal axis contains the retraction ˙ A positive value for φ˙ indicates that the swing leg keeps moving forward, speed φ. a value of zero means that the swing leg is being held at the heel strike value φ = 0.3068 and so the foot comes down vertically. A negative value for φ˙ indicates the presence of a retraction phase. Stable gaits exist for retraction speeds between −0.18 and +0.009 rad/s. In words, this graph shows that relative hip angle should be decreasing around the instant that heel strike is expected

would make the walker operate very close to instability characterized by a rapidly growing eigenvalue. An interesting data point is φ˙ = 0. One of the eigenvalues there is zero (λ1 = 0); any errors in the initial condition θ will be completely eliminated within one step, because it is certain that the step will end with φ = 0.3068 as the swing leg will be held at that value until heel strike occurs. The other eigenvalue can also be calculated manually. Although the derivation is a little more involved, the result simply reads λ2 = cos2 φ. A system with φ˙ = 0 is dynamically equivalent to the “Rimless Wheel” [10, 2]. 3.2 The Inﬂuence of Step Length We repeat the stability analysis of the previous subsection still using the same slope γ = 0.004 but varying the step length of the gait. For example, we choose a much faster and shorter step starting with θ0 = −0.1317. The limit cycle belonging to that value starts with θ˙0 = −0.17 while the step time is 2 s (this is what we tuned for). The resultant eigenvalues are shown in Fig. 5. Clearly there is a much larger range of stable retraction speeds, at the cost of slightly slower convergence. A retraction speed of φ˙ = −0.71 seems optimal, i.e. this is where we ﬁnd the smallest value when taking the maximal absolute values of

434

M. Wisse et al.

1.5 1

|λ|

0.5 0 -0.5 -1 stable retraction speeds -1.5 -1.6

-1.4

-1.2

-1

-0.8

-0.6

-0.4

-0.2

0

. retraction speed φ (rad/s)

Fig. 5. A faster walking motion (T = 2 s) leads to a much larger range of stable retraction speeds

. retraction speed φ

both of the eigenvalues. The absolute eigenvalues are 0.8, i.e. errors decrease 20% per step. Figure 6 provides an overview of the eﬀect of step length on stable range of retraction speeds and the optimal retraction speed and accompanying eigenvalues. The stable range decreases for larger step lengths until it is zero for φ = 0.3155. Above that value no limit cycles exist, because the energy supply from gravity cannot match the large impact losses. Near this value, the walker is operating dangerously close to a state in which it does not have suﬃcient forward energy to pass the apex at midstance, resulting in a fall backward. The main conclusion from this graph is that it is wise to operate well away from a fall backward, i.e. walk fast!

0 Stable retraction speeds -1 -2

Optimal retraction speed For γ = 0.004

-3 0.24

0.25

0.26

0.27

0.28

0.29

0.3

0.31

step length φ

0.32

Fig. 6. Eﬀect of step length on the range of stable retraction speeds for a ﬂoor slope of γ = 0.004. The gray area shows that shorter steps are better. The graph also shows the retraction speed with the smallest eigenvalues

. retraction speed φ

Dynamic Stability of a Simple Biped Walking System 0

435

Stable retraction speeds

-1 Optimal retraction speed

-2 For φ = 0.3068 -3 0

0.002

0.004

0.006

0.008

0.01 0.012 slope angle γ

Fig. 7. Eﬀect of ﬂoor slope on the range of stable retraction speeds for a step length of φ = 0.3068. The gray area shows that steeper slopes (and thus faster steps) are better. The graph also shows the retraction speed with the smallest eigenvalues disturbed step

T + ∆T1

second step

T + ∆T2

Step-down

Fig. 8. A step-down of 0.1% of the leg length results in a slightly longer step time (∆T1 > 0), followed by a signiﬁcantly faster second step (∆T2 < 0)

3.3 The Inﬂuence of the Slope Angle The inﬂuence of the slope angle is similar to the that of the step length. A steeper slope provides more energy input and thus the resultant gait is faster, an eﬀect similar to decreasing the step length. Figure 7 shows how the range of stable retraction speeds depends on the slope angle, for the nominal step length φ = 0.3068. 3.4 Eﬀect of Step-Down Disturbance on Step Time During preliminary presentations of this work [18], we were asked whether the beneﬁcial stability eﬀect of swing leg retraction also holds in the case of a step-down in the ﬂoor. One might think that the step-down has the following adverse eﬀect: the disturbed step takes longer than normal due to the extra step-down height (Fig. 8). The swing leg retraction then results in a shorter -than-normal steplength, while the extra energy should rather have been dissipated with a longer -than-normal steplength. Thus it seems that swing leg retraction worsens the eﬀects of the step-down disturbance. However, we will show here that the opposite is true.

436

M. Wisse et al. 0.1

Tnominal = 2 (s)

∆T

∆T of disturbed step

0.2 0

∆T of second step

-0.1 -0.2 -1.6

-1.4

-1.2

-1

-0.8

-0.6

-0.4

-0.2

.

0

0.2

retraction speed φ 0.8

∆T

Tnominal = 3.853 (s) ∆T of disturbed step

0.4 0

∆T of second step

-0.4 -0.8 -0.18 -0.16

-0.14

-0.12

-0.1

-0.08

-0.06

-0.04

-0.02

0

.

0.02

retraction speed φ Fig. 9. Eﬀect of a step-down of 0.1% of the leg length on the step time of the disturbed step and the step after that (“second step”). Both graphs show that the second step is much more disturbed than the disturbed step itself. The top graph shows that a fast walking gait results in a much smaller step time disturbance than a slow walking gait (bottom graph). Both graphs show that swing leg retraction is beneﬁcial because it reduces the eﬀect of a step-down on the step time of the second step

The most important gait characteristic for a disturbance analysis is the step time. For a simple 2D model, the step time is the most telling characteristic to predict how likely a forward or backward fall is. So, we must investigate the eﬀect of the step-down disturbance on the step time. It turns out that the step after the disturbed step (“second step” in Fig. 8) suﬀers most, see Fig. 9. In this ﬁgure we show the eﬀect of a step down of 0.1% of the leg length. The disturbed step itself has a slightly longer step time, and the second step has a signiﬁcantly shorter step time. From this ﬁgure we draw two conclusions: (1) the step time is considerably less disturbed for the fast gait, so walking faster is better, and (2) the presence of a swing leg retraction phase actually reduces the step time disturbance ∆T , contrary to what one might expect. Note that this analysis was only done for the range of stable solutions from Figs. 4 and 5.

Dynamic Stability of a Simple Biped Walking System

437

3.5 Asymmetric Gait is More Stable The results in the previous sections show that the retraction speed can change the eigenvalues, but it doesn’t ever seem to obtain eigenvalues of all zeros. The explanation is simple; the system uses one control input (the swing leg angle φ at heel strike) with which it must stabilize two states (stance leg angle θ and its ˙ For a discrete control situation like the one at hand, the controller velocity θ). needs at least two interactions with the system before a random disturbance can be eliminated. In other words: although one cannot obtain “deadbeat control” (all eigenvalues zero) within a single step, it should be possible to ﬁnd a deadbeat controller for a succession of two steps. Here we present such a controller for our nominal situation of γ = 0.004 and φ = 0.3068. The previous solutions were all symmetric, i.e. the trajectory of the swing leg was the same each step. We found that a purposefully induced asymmetric gait can result in eigenvalues of all zeros. The swing leg trajectories (one for leg 1 and another for leg 2) are shown in Fig. 10. Leg 1 always goes to φ = 0.3068 and does not have a retraction phase (i.e. the foot comes straight down). Leg 2 always follows a trajectory with a retraction speed of φ˙ = −0.125245. If one calculates the eigenvalues over a series of steps of Leg 1 – Leg 2 – Leg 1 (or more), all eigenvalues are zero. This means that any disturbance will be completely eliminated after three steps of this asymmetric gait. Preliminary research suggests that this “deadbeat” solution even pertains to large errors, although that requires a non-constant retraction speed. We intend to investigate such large-error solutions in the future. Note that in steady gait, the motion of the swing legs is asymmetric (one retracts and the other does not) but the step length and step time are still symmetric. Hence

φ and θ (rad)

0.6

Step 1 Step lengths are the same in undisturbed walking

0.4

Step 2 Retraction speed = -0.125

Retraction speed = 0

0.2 θ

θ

0 -0.2

Jump is artifact due to relabeling stance swing leg

φ

φ

-0.4 0

1

2

3

4

5 time (s)

6

7

8

Fig. 10. An asymmetric gait can lead to two-step deadbeat control, i.e. to two eigenvalues of zero

438

M. Wisse et al.

this solution is conceptually entirely diﬀerent from the “period-two” solutions found in a bifurcation analysis (e.g. [5]). Also note that due to the asymmetry, the eigenvalues cannot be divided up into “one-step” eigenvalues.

4 Automated Optimization on a More Realistic Mass Distribution also Results in Swing Leg Retraction The theoretical results in the previous section are based on a point mass model for walking. One of the main assumptions there is that the mass of the swing leg is negligible. Obviously, in real walking systems this is not true. The reaction forces and torques from a non-massless swing leg will inﬂuence the walking motion. In our experience, the main eﬀect is energy input. Driving the swing leg forward also pumps energy into the gait. A beneﬁt of this eﬀect is that a downhill slope is no longer required, but the question is whether it breaks the stabilizing eﬀect of swing leg retraction. Or, even if it does still help stability, whether the stability gain outweighs the added energetic cost for accelerating the swing leg. We study these questions using a model with a more realistic mass distribution, based on a prototype we are currently experimenting with [1] (Fig. 11). The swing leg trajectory is optimized both for stability and for eﬃciency. The model (Fig. 11) has the same topology as our initial model (Fig. 1). However, instead of a single point mass at the hip, the model now has a distributed mass over the legs, see Table 2. The swing leg follows the desired trajectory with reasonable accuracy using a PD controller on the hip torque: T = k(φ − φdes (t)) + dφ˙

(4)

with gain values k = 1500 and d = 10. The swing leg trajectory is parameterized with two knot points deﬁning the start and the end of the retraction

c m, I

g

w

Fig. 11. Our current experimental biped and a straight-legged model with mass distribution based on the experimental robot, see Table 2

Dynamic Stability of a Simple Biped Walking System

439

Table 2. Parameter values for a model with a more realistic mass distribution in the legs gravity ﬂoor slope leg length leg mass vertical position CoM horizontal position CoM moment of inertia

g γ l m c w I

9.81 m/s2 0 rad 0.416 m 3 kg 0.027 m 0m 0.07 kgm2

phase (Fig. 12). The trajectory before the ﬁrst knot point is a third order spline which starts with the actual swing leg angle and velocity just after heel strike. The trajectory between the two knot points is a straight line. This parametrization provides the optimizer with ample freedom to vary the retraction speed, the nominal step length, and the duration of the retraction phase. The optimization procedure is set up as follows. The model is started with manually tuned initial conditions, after which a forward dynamic simulation is run for 20 simulated seconds. The resulting motion is then rated for average velocity and eﬃciency: cost = Σ20s (wT T 2 + wv (x˙ − x˙ des )2 + wv y˙ 2 )

(5)

φ (rad)

0.6 knot point 1 (φ1, t1)

0.4

3th order spline

0.2 knot point 2 (φ2, t2)

0

-0.2 knot point 0 (uses actual post-impact . φ and φ)

-0.4 0

1

2

3

5

4 time (s)

Fig. 12. The desired trajectory for the swing leg is parameterized with two knot points (four parameters)

440

M. Wisse et al.

. Retraction speed φ (rad/s)

0 -0.2 -0.4 -0.6 -0.8 -1

0

5

10

15

20

25

Random torque level (Nm)

Fig. 13. Retraction speed φ˙ as a function of the level of random torque disturbances. A negative value for φ˙ indicates the presence of a retraction phase. The results are obtained with an optimization algorithm which was initialized with trajectories with a retraction phase for some runs and without one for others. Irrespective of the initialization and the level of disturbances, the optimization always settles into a trajectory with a retraction phase, i.e. φ˙ is always negative just prior to foot contact

with the weight for the torque penalty wT = 0.1, the weight for the velocity penalty wv = 1, and the desired forward velocity x˙ des = 0.3, summed over a trial interval of 20 s. During the motion, random noise with uniform distribution is added to the hip torque. In this way, the model is indirectly rated for robustness; if the noise makes the walker fall, then the resultant average walking velocity is low and so the penalty for not achieving x˙ des is high. A simulated annealing procedure optimized the cost function of (5) by adjusting the four knot point parameters for the swing leg trajectory. Figure 13 shows that for a wide range of noise levels and initialization values, the optimization procedure consistently settles into gaits with a retraction phase. These results fully concur with the theoretical results for the model with massless legs. Therefore, we conclude that the analysis is valid and the conclusion holds: a mild retraction speed is beneﬁcial for the walking stability.

5 Discussion This work was limited to a small error analysis. Our future work consists of analyzing the eﬀect of the swing leg motion when under large disturbances, i.e. an analysis of the basin of attraction must be added to the present linearized stability analysis [19]. We also intend to investigate the eﬀects of an increased model complexity by adding knees, feet, and an upper body. Observations of the gait of our previously developed passive-based walking robots [17] show that they all walk with a retraction phase in the swing leg

0.1 0

441

{

foot postion in sagittal plane (m)

Marker position for motion capture

Dynamic Stability of a Simple Biped Walking System

-0.1 -0.2 -0.3 -0.4 -0.5

swing leg retraction heel strike in all data

-0.6 -0.7 -0.8 -0.9 -0.8 -0.7 -0.6 -0.5 -0.4 -0.3 -0.2 -0.1 0 0.1 time (s)

Fig. 14. Left: our prototype Denise. Right: the graph shows the clear existence of a retraction phase. The graphs shows a measurement of the motion of Denise’s heel with respect to the ﬂoor. We measured over 150 steps from several trials. The data is time-synchronized using heel strike at the end as the reference, and the ﬁnal position after heel strike is deﬁned as 0 meters. Then we calculated the mean and standard deviation, which are shown in the graph

motion. For illustration, Fig. 14 shows the motion of the swing foot with respect to the ﬂoor, as measured with a motion capture system on our most recent prototype Denise [3]. The measurements (an average of over 150 steps) show that there exists a clear retraction phase just prior to heel strike.

6 Conclusion In this paper we research the eﬀect of swing leg retraction on gait stability. The conclusions are straightforward: 1. Walk fast; this decreases the sensitivity of the gait to the swing leg motion just prior to heel contact. 2. Use mild swing leg retraction; by moving the swing leg rearward just prior to heel contact, one avoids the highly unstable eﬀects that occur when the swing leg is still moving forward at heel contact.

Acknowledgements This research was supported by US National Science Foundation grants ECS0325383 and CNS-0224419 and by the NWO, the Netherlands Organization for Scientiﬁc Research. The authors are grateful to Seiichi Miyakoshi for discussions about the concept of swing leg retraction. Also thanks to Brian Moyer at

442

M. Wisse et al.

Mark Redfern’s lab of University of Pittsburgh for the motion capture measurements of robot Denise. Daan Hobbelen helped with the analysis of the step-down eﬀects.

References [1] S. O. Anderson, M. Wisse, C. G. Atkeson, J. K. Hodgins, G. J. Zeglin, and B. Moyer. Powered bipeds based on passive dynamic principles. In Proc. of IEEE/RAS International Conference on Humanoid Robots, pp. 110–116, Tsukuba, Japan, 2005. [2] M. J. Coleman, A. Chatterjee, and A. Ruina. Motions of a rimless spoked wheel: A simple 3D system with impacts. Dynamics and Stability of Systems, 12(3):139–160, 1997. [3] S. H. Collins, A. Ruina, R. L. Tedrake, and M. Wisse. Eﬃcienct bipedal robots based on passive-dynamic walkers. Science, 307:1082–1085, February 18 2005. [4] S. H. Collins, M. Wisse, and A. Ruina. A two legged kneed passive dynamic walking robot. Int. J. of Robotics Research, 20(7):607–615, July 2001. [5] M. Garcia, A. Chatterjee, A. Ruina, and M. J. Coleman. The simplest walking model: Stability, complexity, and scaling. ASME J. Biomech. Eng., 120(2):281– 288, April 1998. [6] A. Goswami, B. Thuilot, and B. Espiau. A study of the passive gait of a compass-like biped robot: symmetry and chaos. Int. J. Robot. Res., 17(12):1282–1301, December 1998. [7] V. T. Inman, H. J. Ralston, and F. Todd. Human Walking. Williams & Wilkinns, Baltimore, 1981. ISBN 0-683-04348-X. [8] S. Kajita and K. Tani. Study of dynamic biped locomotion on rugged terrainderivation and application of the linear inverted pendulum mode. In Proc., IEEE Int. Conf. on Robotics and Automation, pp. 1405–1411. IEEE, 1991. [9] A. D. Kuo. Energetics of actively powered locomotion using the simplest walking model. J. of Biomechanical Engineering, 124:113–120, February 2002. [10] T. McGeer. Powered ﬂight, child’s play, silly wheels, and walking machines. In Proc., IEEE Int. Conf. on Robotics and Automation, pp. 1592–1597, Piscataway, NJ, 1989. [11] T. McGeer. Passive dynamic walking. Int. J. Robot. Res., 9(2):62–82, April 1990. [12] T. McGeer. Passive dynamic biped catalogue. In R. Chatila and G. Hirzinger, editors, Proc., Experimental Robotics II: The 2nd International Symposium, pp. 465–490, Berlin, 1992. Springer–Verlag. [13] S. Miyakoshi, T. Yamamoto, G. Taga, and Y. Kuniyoshi. Bipedal walking simulation of a compass-like robot. In Proc. of The Robotics Society of Japan, pp. 1107–1108, 2000. (in Japanese). [14] S. Schaal and C. G. Atkeson. Open loop stable control strategies for robot juggling. In Proc., IEEE Int. Conf. on Robotics and Automation, pp. 913–918, Piscataway, NJ, 1993. IEEE. [15] A. L. Schwab and M. Wisse. Basin of attraction of the simplest walking model. In Proc., ASME Design Engineering Technical Conferences, Pittsburgh, Pennsylvania, 2001. ASME. Paper number DETC2001/VIB-21363.

Dynamic Stability of a Simple Biped Walking System

443

[16] A. Seyfarth, H. Geyer, and H. Herr. Swing-leg retraction: A simple control model for stable running. J. Exp. Biol., 206:2547–2555, 2003. [17] M. Wisse. Three additions to passive dynamic walking; actuation, an upper body, and 3d stability. In Proc., Int. Conf. on Humanoid Robots, Los Angeles, USA, 2004. IEEE. [18] M. Wisse, C. G. Atkeson, and D. K. Kloimwieder. Swing leg retraction helps biped walking stability. In Proc. of IEEE/RAS International Conference on Humanoid Robots, pp. 295–300, Tsukuba, Japan, 2005. [19] M. Wisse, A. L. Schwab, R. Q. van der Linde, and F. C. T. van der Helm. How to keep from falling forward; elementary swing leg action for passive dynamic walkers. IEEE Trans. on Robotics, 21(3):393–401, 2005.

Lecture Notes in Control and Information Sciences Editors: M. Thoma, M. Morari Further volumes of this series can be found on our homepage: springer.com Vol. 340: Diehl, M.; Mombaur, K. (Eds.) Fast Motions in Biomechanics and Robotics 460 p. 2006 [3-540-36118-9] Vol. 339: Alamir, M. Stabilization of Nonlinear Systems Using Receding-horizon Control Schemes 325 p. 2006 [1-84628-470-8] Vol. 338: Tokarzewski, J. Finite Zeros in Discrete Time Control Systems 325 p. 2006 [3-540-33464-5] Vol. 337: Blom, H.; Lygeros, J. (Eds.) Stochastic Hybrid Systems 395 p. 2006 [3-540-33466-1] Vol. 336: Pettersen, K.Y.; Gravdahl, J.T.; Nijmeijer, H. (Eds.) Group Coordination and Cooperative Control 310 p. 2006 [3-540-33468-8] Vol. 335: Kozáowski, K. (Ed.) Robot Motion and Control 424 p. 2006 [1-84628-404-X] Vol. 334: Edwards, C.; Fossas Colet, E.; Fridman, L. (Eds.) Advances in Variable Structure and Sliding Mode Control 504 p. 2006 [3-540-32800-9] Vol. 333: Banavar, R.N.; Sankaranarayanan, V. Switched Finite Time Control of a Class of Underactuated Systems 99 p. 2006 [3-540-32799-1] Vol. 332: Xu, S.; Lam, J. Robust Control and Filtering of Singular Systems 234 p. 2006 [3-540-32797-5] Vol. 331: Antsaklis, P.J.; Tabuada, P. (Eds.) Networked Embedded Sensing and Control 367 p. 2006 [3-540-32794-0] Vol. 330: Koumoutsakos, P.; Mezic, I. (Eds.) Control of Fluid Flow 200 p. 2006 [3-540-25140-5] Vol. 329: Francis, B.A.; Smith, M.C.; Willems, J.C. (Eds.) Control of Uncertain Systems: Modelling, Approximation, and Design 429 p. 2006 [3-540-31754-6] Vol. 328: Loría, A.; Lamnabhi-Lagarrigue, F.; Panteley, E. (Eds.) Advanced Topics in Control Systems Theory 305 p. 2006 [1-84628-313-2]

Vol. 327: Fournier, J.-D.; Grimm, J.; Leblond, J.; Partington, J.R. (Eds.) Harmonic Analysis and Rational Approximation 301 p. 2006 [3-540-30922-5] Vol. 326: Wang, H.-S.; Yung, C.-F.; Chang, F.-R. HaÞ ˛ Control for Nonlinear Descriptor Systems 164 p. 2006 [1-84628-289-6] Vol. 325: Amato, F. Robust Control of Linear Systems Subject to Uncertain Time-Varying Parameters 180 p. 2006 [3-540-23950-2] Vol. 324: Christoa˛ ˛edes, P.; El-Farra, N. Control of Nonlinear and Hybrid Process Systems 446 p. 2005 [3-540-28456-7] Vol. 323: Bandyopadhyay, B.; Janardhanan, S. Discrete-time Sliding Mode Control 147 p. 2005 [3-540-28140-1] Vol. 322: Meurer, T.; Graichen, K.; Gilles, E.D. (Eds.) Control and Observer Design for Nonlinear Finite and Ina˛ ˛enite Dimensional Systems 422 p. 2005 [3-540-27938-5] Vol. 321: Dayawansa, W.P.; Lindquist, A.; Zhou, Y. (Eds.) New Directions and Applications in Control Theory 400 p. 2005 [3-540-23953-7] Vol. 320: Steffen, T. Control Recona˛ ˛eguration of Dynamical Systems 290 p. 2005 [3-540-25730-6] Vol. 319: Hofbaur, M.W. Hybrid Estimation of Complex Systems 148 p. 2005 [3-540-25727-6] Vol. 318: Gershon, E.; Shaked, U.; Yaesh, I. HaÞ ˛ Control and Estimation of State-muliplicative Linear Systems 256 p. 2005 [1-85233-997-7] Vol. 317: Ma, C.; Wonham, M. Nonblocking Supervisory Control of State Tree Structures 208 p. 2005 [3-540-25069-7] Vol. 316: Patel, R.V.; Shadpey, F. Control of Redundant Robot Manipulators 224 p. 2005 [3-540-25071-9]

Vol. 315: Herbordt, W. Sound Capture for Human/Machine Interfaces: Practical Aspects of Microphone Array Signal Processing 286 p. 2005 [3-540-23954-5] Vol. 314: Gil’, M.I. Explicit Stability Conditions for Continuous Systems 193 p. 2005 [3-540-23984-7] Vol. 313: Li, Z.; Soh, Y.; Wen, C. Switched and Impulsive Systems 277 p. 2005 [3-540-23952-9] Vol. 312: Henrion, D.; Garulli, A. (Eds.) Positive Polynomials in Control 313 p. 2005 [3-540-23948-0] Vol. 311: Lamnabhi-Lagarrigue, F.; Loría, A.; Panteley, E. (Eds.) Advanced Topics in Control Systems Theory 294 p. 2005 [1-85233-923-3] Vol. 310: Janczak, A. Identiacation ˛ of Nonlinear Systems Using Neural Networks and Polynomial Models 197 p. 2005 [3-540-23185-4] Vol. 309: Kumar, V.; Leonard, N.; Morse, A.S. (Eds.) Cooperative Control 301 p. 2005 [3-540-22861-6] Vol. 308: Tarbouriech, S.; Abdallah, C.T.; Chiasson, J. (Eds.) Advances in Communication Control Networks 358 p. 2005 [3-540-22819-5] Vol. 307: Kwon, S.J.; Chung, W.K. Perturbation Compensator based Robust Tracking Control and State Estimation of Mechanical Systems 158 p. 2004 [3-540-22077-1] Vol. 306: Bien, Z.Z.; Stefanov, D. (Eds.) Advances in Rehabilitation 472 p. 2004 [3-540-21986-2] Vol. 305: Nebylov, A. Ensuring Control Accuracy 256 p. 2004 [3-540-21876-9] Vol. 304: Margaris, N.I. Theory of the Non-linear Analog Phase Locked Loop 303 p. 2004 [3-540-21339-2]

Vol. 301: de Queiroz, M.; Malisoff, M.; Wolenski, P. (Eds.) Optimal Control, Stabilization and Nonsmooth Analysis 373 p. 2004 [3-540-21330-9] Vol. 300: Nakamura, M.; Goto, S.; Kyura, N.; Zhang, T. Mechatronic Servo System Control Problems in Industries and their Theoretical Solutions 212 p. 2004 [3-540-21096-2] Vol. 299: Tarn, T.-J.; Chen, S.-B.; Zhou, C. (Eds.) Robotic Welding, Intelligence and Automation 214 p. 2004 [3-540-20804-6] Vol. 298: Choi, Y.; Chung, W.K. PID Trajectory Tracking Control for Mechanical Systems 127 p. 2004 [3-540-20567-5] Vol. 297: Damm, T. Rational Matrix Equations in Stochastic Control 219 p. 2004 [3-540-20516-0] Vol. 296: Matsuo, T.; Hasegawa, Y. Realization Theory of Discrete-Time Dynamical Systems 235 p. 2003 [3-540-40675-1] Vol. 295: Kang, W.; Xiao, M.; Borges, C. (Eds) New Trends in Nonlinear Dynamics and Control, and their Applications 365 p. 2003 [3-540-10474-0] Vol. 294: Benvenuti, L.; De Santis, A.; Farina, L. (Eds) Positive Systems: Theory and Applications (POSTA 2003) 414 p. 2003 [3-540-40342-6] Vol. 293: Chen, G. and Hill, D.J. Bifurcation Control 320 p. 2003 [3-540-40341-8] Vol. 292: Chen, G. and Yu, X. Chaos Control 380 p. 2003 [3-540-40405-8] Vol. 291: Xu, J.-X. and Tan, Y. Linear and Nonlinear Iterative Learning Control 189 p. 2003 [3-540-40173-3]

Vol. 303: Mahmoud, M.S. Resilient Control of Uncertain Dynamical Systems 278 p. 2004 [3-540-21351-1]

Vol. 290: Borrelli, F. Constrained Optimal Control of Linear and Hybrid Systems 237 p. 2003 [3-540-00257-X] Vol. 289: Giarré, L. and Bamieh, B. Multidisciplinary Research in Control 237 p. 2003 [3-540-00917-5]

Vol. 302: Filatov, N.M.; Unbehauen, H. Adaptive Dual Control: Theory and Applications 237 p. 2004 [3-540-21373-2]

Vol. 288: Taware, A. and Tao, G. Control of Sandwich Nonlinear Systems 393 p. 2003 [3-540-44115-8]

Fast Motions in Biomechanics and Robotics: Optimization and Feedback Control (Lecture Notes in Control and Information Sciences)

Fast motions in biomechanics and robotics: optimization and feedback control

Singular Control Systems (Lecture Notes in Control and Information Sciences)

Networked Control Systems (Lecture Notes in Control and Information Sciences)

Ensuring Control Accuracy (Lecture Notes in Control and Information Sciences)

Recent Advances in Control and Optimization of Manufacturing Systems (Lecture Notes in Control and Information Sciences)

Feedback Control, Nonlinear Systems, and Complexity (Lecture Notes in Control and Information Sciences)

Control of Indefinite Nonlinear Dynamic Systems: Induced Internal Feedback (Lecture Notes in Control and Information Sciences)

RAMSETE: Articulated and Mobile Robotics for Services and Technology (Lecture Notes in Control and Information Sciences)

L2-Gain and Passivity Techniques in Nonlinear Control (Lecture Notes in Control and Information Sciences)

Topics in Time Delay Systems: Analysis, Algorithms and Control (Lecture Notes in Control and Information Sciences)

Recent Advances in Learning and Control (Lecture Notes in Control and Information Sciences)

Robustness in Identification and Control (Lecture Notes in Control and Information Sciences)

New Directions and Applications in Control Theory (Lecture Notes in Control and Information Sciences)

Iterative Learning Control: Convergence, Robustness and Applications (Lecture Notes in Control and Information Sciences)

Control and Estimation of Systems with Input Output Delays (Lecture Notes in Control and Information Sciences)

Dynamics, Bifurcations and Control (Lecture Notes in Control and Information Sciences)

Assessment and Future Directions of Nonlinear Model Predictive Control (Lecture Notes in Control and Information Sciences)

Adaptive Dual Control: Theory and Applications (Lecture Notes in Control and Information Sciences)

Active Fault Tolerant Control Systems: Stochastic Analysis and Synthesis (Lecture Notes in Control and Information Sciences)

Chaos Control: Theory and Applications (Lecture Notes in Control and Information Sciences)

The Confluence of Vision and Control (Lecture Notes in Control and Information Sciences)

Variational and Hamiltonian Control Systems (Lecture Notes in Control and Information Sciences)

Nonsmooth Impact Mechanics: Models, Dynamics and Control (Lecture Notes in Control and Information Sciences)

Robot Motion and Control 2009 (Lecture Notes in Control and Information Sciences)

Bifurcation Control: Theory and Applications (Lecture Notes in Control and Information Sciences)

Robust Control via Variable Structure and Lyapunov Techniques (Lecture Notes in Control and Information Sciences)

Analysis and Synthesis of Networked Control Systems (Lecture Notes in Control and Information Sciences)

Stability and Control of Time-delay Systems (Lecture Notes in Control and Information Sciences)

Nonlinear and Adaptive Control: NCN4 2001 (Lecture Notes in Control and Information Sciences)

Fuzzy Control and Filter Design for Uncertain Fuzzy Systems (Lecture Notes in Control and Information Sciences)

Fast Motions in Biomechanics and Robotics: Optimization and Feedback Control (Lecture Notes in Control and Information Sciences)

Fast motions in biomechanics and robotics: optimization and feedback control

Singular Control Systems (Lecture Notes in Control and Information Sciences)

Networked Control Systems (Lecture Notes in Control and Information Sciences)

Ensuring Control Accuracy (Lecture Notes in Control and Information Sciences)

Recent Advances in Control and Optimization of Manufacturing Systems (Lecture Notes in Control and Information Sciences)

Feedback Control, Nonlinear Systems, and Complexity (Lecture Notes in Control and Information Sciences)

Control of Indefinite Nonlinear Dynamic Systems: Induced Internal Feedback (Lecture Notes in Control and Information Sciences)

RAMSETE: Articulated and Mobile Robotics for Services and Technology (Lecture Notes in Control and Information Sciences)

L2-Gain and Passivity Techniques in Nonlinear Control (Lecture Notes in Control and Information Sciences)

Topics in Time Delay Systems: Analysis, Algorithms and Control (Lecture Notes in Control and Information Sciences)

Recent Advances in Learning and Control (Lecture Notes in Control and Information Sciences)

Robustness in Identification and Control (Lecture Notes in Control and Information Sciences)

New Directions and Applications in Control Theory (Lecture Notes in Control and Information Sciences)

Iterative Learning Control: Convergence, Robustness and Applications (Lecture Notes in Control and Information Sciences)

Control and Estimation of Systems with Input Output Delays (Lecture Notes in Control and Information Sciences)

Dynamics, Bifurcations and Control (Lecture Notes in Control and Information Sciences)

Assessment and Future Directions of Nonlinear Model Predictive Control (Lecture Notes in Control and Information Sciences)

Adaptive Dual Control: Theory and Applications (Lecture Notes in Control and Information Sciences)

Active Fault Tolerant Control Systems: Stochastic Analysis and Synthesis (Lecture Notes in Control and Information Sciences)

Chaos Control: Theory and Applications (Lecture Notes in Control and Information Sciences)

The Confluence of Vision and Control (Lecture Notes in Control and Information Sciences)

Variational and Hamiltonian Control Systems (Lecture Notes in Control and Information Sciences)

Nonsmooth Impact Mechanics: Models, Dynamics and Control (Lecture Notes in Control and Information Sciences)

Robot Motion and Control 2009 (Lecture Notes in Control and Information Sciences)

Bifurcation Control: Theory and Applications (Lecture Notes in Control and Information Sciences)

Robust Control via Variable Structure and Lyapunov Techniques (Lecture Notes in Control and Information Sciences)

Analysis and Synthesis of Networked Control Systems (Lecture Notes in Control and Information Sciences)

Stability and Control of Time-delay Systems (Lecture Notes in Control and Information Sciences)

Nonlinear and Adaptive Control: NCN4 2001 (Lecture Notes in Control and Information Sciences)

Fuzzy Control and Filter Design for Uncertain Fuzzy Systems (Lecture Notes in Control and Information Sciences)

Recommend Documents