Harmonic Analysis on Exponential Solvable Lie Groups

Springer Monographs in Mathematics

Hidenori Fujiwara Jean Ludwig

Harmonic Analysis on Exponential Solvable Lie Groups

Springer Monographs in Mathematics

More information about this series at http://www.springer.com/series/3733

Hidenori Fujiwara • Jean Ludwig

Harmonic Analysis on Exponential Solvable Lie Groups

123

Hidenori Fujiwara Kinki University Fukuoka, Japan

Jean Ludwig University of Metz Metz, France

ISSN 1439-7382 ISSN 2196-9922 (electronic) ISBN 978-4-431-55287-1 ISBN 978-4-431-55288-8 (eBook) DOI 10.1007/978-4-431-55288-8 Springer Tokyo Heidelberg New York Dordrecht London Library of Congress Control Number: 2014955480 Mathematics Subject Classification (2010): 22E27, 22E25, 22E30, 43A85, 22E15 © Springer Japan 2015 This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. Exempted from this legal reservation are brief excerpts in connection with reviews or scholarly analysis or material supplied specifically for the purpose of being entered and executed on a computer system, for exclusive use by the purchaser of the work. Duplication of this publication or parts thereof is permitted only under the provisions of the Copyright Law of the Publisher’s location, in its current version, and permission for use must always be obtained from Springer. Permissions for use may be obtained through RightsLink at the Copyright Clearance Center. Violations are liable to prosecution under the respective Copyright Law. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. While the advice and information in this book are believed to be true and accurate at the date of publication, neither the authors nor the editors nor the publisher can accept any legal responsibility for any errors or omissions that may be made. The publisher makes no warranty, express or implied, with respect to the material contained herein. Printed on acid-free paper Springer is part of Springer Science+Business Media (www.springer.com)

Preface

In this book we treat some topics in the theory of unitary representations of solvable Lie groups and we present through various examples the present situation, some open problems and possible directions of research. In the early 1970s Auslander and Kostant [3] succeeded in constructing the unitary dual of connected and simply connected type I solvable Lie groups by holomorphically induced representations and this result was extended by Pukanszky [65] to non-type I solvable Lie groups. These works are landmarks in the representation theory for solvable Lie groups. Nevertheless, it is still difficult even now to study in detail these representations and their applications. For example, concerning the study of induced or restricted representations, we wish to decompose them into irreducibles, construct intertwining operators and understand the related algebras of invariant differential operators. However, we know little about these classical problems even in the case of exponential solvable Lie groups. It is only for nilpotent Lie groups that we have more tools in our hands. The representation theory of semisimple Lie groups differs very much from that of solvable Lie groups. Rich algebraic structures of semisimple Lie groups offer many research tools; the less rigid structure of solvable Lie groups allows only the induction procedure as the unique efficient method to obtain results. Fortunately, for solvable Lie groups, we have the orbit method to understand their unitary duals. The fundamental idea of Kirillov [48] to associate a coadjoint orbit to an irreducible unitary representation allows us to describe problems in representation theory in terms of data coming from the coadjoint orbits in the dual space of the Lie algebra of the group. The authors have studied the orbit method since the 1970s. During this time it seems that some progress has been made for nilpotent Lie groups. On the other hand, almost all the problems one can ask for solvable non-nilpotent Lie groups remain unanswered or unexplored, for instance, questions surrounding holomorphically induced representation. It follows that the materials of this book are mainly the analysis for exponential solvable Lie groups, especially for nilpotent Lie groups. However, if we want to go beyond exponential solvable Lie groups, we necessarily must be confronted with the holomorphically induced representation. Therefore, the v

vi

Preface

book introduces also the Auslander–Kostant theory, because the authors believe that this theory will be an important research subject for the future. This book is an enlargement of the Japanese book [39] written by the first author. The first chapter presents preliminaries to treat unitary representations of solvable Lie groups. Following Professor Sugiura’s book [74], we describe the general theory of Lie groups and Lie algebras which will be needed from the second chapter downward. In the second chapter, we give some generalities on harmonic analysis for locally compact topological groups. In the third chapter, we study the Mackey theory for induced representations which is the background of the orbit method. A good reference is his lecture note [56]. In fourth chapter we study in detail four typical solvable Lie groups. These observations will offer to us a certain perspective on the orbit method. The fifth chapter is somewhat central to this book and we explain in detail the orbit method for exponential solvable Lie groups. As a guideline we use Mackey theory for a minimal non-central normal subgroup. We shall see in the sixth chapter some more details on Kirillov theory for nilpotent Lie groups. There we recognize that we can manipulate more tools in this case. In the seventh chapter we study in detail holomorphically induced representations for exponential solvable Lie groups. From the eighth chapter forward we shall explain some topics on which we have been working. Though the ordering of these topics is not necessarily chronological along the research life of the authors, all topics are closely related to each other. In the eighth chapter we first describe in terms of orbits the canonical central decomposition of monomial representations and restrictions onto subgroups of irreducible unitary representations in the case of exponential solvable Lie groups. In the ninth chapter we examine e-central elements, which are very useful to study invariant differential operators and which were introduced for nilpotent Lie groups in Corwin and Geenleaf [17]. In the tenth chapter we consider the Frobenius reciprocity in the distribution version. In the 11th chapter we explicitly describe by the orbit method the abstract Plancherel formula for monomial representations. Investigating many examples, we try to understand its mechanism. In the 12th chapter we prove for monomial representations the so-called commutativity conjecture due to Duflo, and Corwin and Geenleaf. This conjecture will be translated in the final chapter into a claim concerning the restriction of representations. All topics treated in the last five chapters still have to be studied in the exponential case. We believe that they all present interesting and important but difficult problems, which should be addressed in the near future. The materials of this book are specialized, but they are accessible to researchers and students of graduate schools or universities who intend to be researchers in

Preface

vii

the future. The reader should have a basic background of the theory of Lie groups and Lie algebras, and some elementary knowledge on representation theory. It often happens that proofs by induction become long and tedious since often many cases have to be considered. So occasionally we sketch only a guideline of the proof. Moreover, we are obliged sometimes to omit proofs which need too much preparation, but which can be found in the classical literature. Finally, among many texts concerning the theory of Lie group, Lie algebra and representation, we list some references relating to the materials of this book. On the theory of representation for general topological groups: [20, 49, 56], on representations of nilpotent Lie groups: [16, 64], on representations of solvable Lie groups: [4,10,51] (the principal part of [51] is devoted to a proof of Theorem 5.3.31 in the fifth chapter and addressed to experts). Further, on the enveloping algebra of Lie algebra: [21] and on the orbit method: [50]. Acknowledgement: We are grateful to the referees for the very careful reading and many valuable comments and suggestions. Iizuka, Fukuoka, Japan Metz, France

Hidenori Fujiwara Jean Ludwig

Contents

1

Preliminaries: Lie Groups and Lie Algebras . . . . . . . .. . . . . . . . . . . . . . . . . . . . 1.1 Lie Groups and Lie Algebras . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 1.2 Enveloping Algebra . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 1.3 Unitary Representations.. . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . .

1 1 20 24

2

Haar Measure and Group Algebra . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 2.1 The Haar Measure of a Locally Compact Group . . . . . . . . . . . . . . . . . . . 2.2 The Group Algebra .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 2.3 Representations .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . .

29 29 37 40

3

Induced Representations.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 3.1 Measures on Quotient Spaces. . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 3.2 Definition of an Induced Representation .. . . . . . .. . . . . . . . . . . . . . . . . . . . 3.3 Conjugation of Induced Representations .. . . . . . .. . . . . . . . . . . . . . . . . . . . 3.4 The Imprimitivity Theorem .. . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . .

53 53 59 68 73

4

Four Exponential Solvable Lie Groups . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 4.1 The Group Rn . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 4.2 The Heisenberg Group . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 4.2.1 Induced Representations .. . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 4.3 The “ax C b” Group .. . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 4.3.1 Induced Representations .. . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 4.3.2 The Plancherel Theorem .. . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 4.3.3 d  for the Enveloping Algebra . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 4.4 Grélaud’s Group .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 4.4.1 Induced Representations .. . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 4.4.2 d  for the Enveloping Algebra . . . . . . . .. . . . . . . . . . . . . . . . . . . . 4.4.3 The Imprimitivity Theorem for Grélaud’s Group . . . . . . . . . . 4.4.4 The Plancherel Theorem .. . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . .

83 83 85 87 95 97 100 101 104 108 109 110 114

ix

x

Contents

5

Orbit Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 5.1 Auslander–Kostant Theory . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 5.2 Exponential Solvable Lie Groups . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 5.2.1 Co-exponential Sequence .. . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 5.2.2 The Exponential Mapping for Solvable Lie Groups . . . . . . . 5.3 Kirillov–Bernat Mapping . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 5.4 Pukanszky Condition .. . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . .

117 117 125 125 130 135 159

6

Kirillov Theory for Nilpotent Lie Groups . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 6.1 Jordan–Hölder and Malcev Sequences .. . . . . . . . .. . . . . . . . . . . . . . . . . . . . 6.1.1 Unipotent Representations.. . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 6.1.2 Polynomial Vector Groups.. . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 6.1.3 Unipotent Orbits . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 6.2 Schwartz Spaces .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 6.3 Intertwining Operator for Irreducible Representations . . . . . . . . . . . . . 6.4 Traces and Plancherel Theorem . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 6.5 Parametrization of All Orbits . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . .

167 167 167 170 173 179 193 196 198

7

Holomorphically Induced Representations .f; h; G ) for Exponential Solvable Lie Groups .. . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 7.1 First Trial . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 7.2 Exponential j-Algebras . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 7.3 Exponential Kähler Algebras . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 7.4 Structure of Positive Polarizations.. . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 7.5 Non-vanishing of H.f; h; G/ . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 7.6 Irreducibility and Equivalence of .f; h; G/ . . . .. . . . . . . . . . . . . . . . . . . . 7.7 Decomposition of .f; h; G/ . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . .

209 209 214 227 243 246 268 285

8

Irreducible Decomposition . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 289 8.1 Monomial Representations . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 289 8.2 Restriction of Unitary Representations to Subgroups . . . . . . . . . . . . . . 304

9

e-Central Elements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 317 9.1 Fundamental Result of Corwin and Greenleaf ... . . . . . . . . . . . . . . . . . . . 317 9.2 Monomial Representations and e-Central Elements.. . . . . . . . . . . . . . . 319

10 Frobenius Reciprocity .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 10.1 Frobenius Reciprocity in Distribution Version ... . . . . . . . . . . . . . . . . . . . 10.2 Realization of the Reciprocity in Nilpotent Case . . . . . . . . . . . . . . . . . . . 10.3 Examples of Semi-invariant Distributions . . . . . .. . . . . . . . . . . . . . . . . . . .

333 333 334 337

11 Plancherel Formula . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 11.1 Penney’s Plancherel Formula . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 11.2 Vergne’s Decomposition Theorem . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 11.3 Examples of Penney’s Plancherel Formula . . . . .. . . . . . . . . . . . . . . . . . . . 11.4 Bonnet’s Plancherel Formula . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . .

343 343 349 359 372

Contents

xi

12 Commutativity Conjecture: Induction Case.. . . . . . . .. . . . . . . . . . . . . . . . . . . . 12.1 Toward the Commutativity Conjecture .. . . . . . . . .. . . . . . . . . . . . . . . . . . . . 12.2 Corwin–Greenleaf Functions . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 12.3 Proof of the Commutativity Conjecture .. . . . . . . .. . . . . . . . . . . . . . . . . . . .

383 383 392 416

13 Commutativity Conjecture: Restriction Case . . . . . . .. . . . . . . . . . . . . . . . . . . . 13.1 Kernel of Representation.. . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 13.2 Commutativity of the Algebra D .G/K . . . . . . . .. . . . . . . . . . . . . . . . . . . . 13.3 Commutativity of the Centralizer of Lie Subalgebras .. . . . . . . . . . . . .

431 431 439 450

References .. .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 455 List of Notations .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 459 Index . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 463

Chapter 1

Preliminaries: Lie Groups and Lie Algebras

1.1 Lie Groups and Lie Algebras There are many texts on Lie groups and Lie algebras. Here following Sugiura [74] we gather general preliminaries. As other references we list [45, 62, 70, 71]. Definition 1.1.1. When the set G satisfies the following three conditions, G is called a (real) Lie group: (1) G is a group. (2) G is a real analytic manifold. (3) The group operation G  G 3 .x; y/ 7! xy 1 2 G is an analytic mapping. Definition 1.1.2. Let g be a vector space over the real number field R or the complex number field C. When a bilinear mapping g  g 3 .X; Y / 7! ŒX; Y  2 g is defined and satisfies the following two conditions, g is called a (real) Lie algebra or a complex Lie algebra: (1) ŒX; X  D 0. (2) ŒŒX; Y ; Z C ŒŒY; Z; X  C ŒŒZ; X ; Y  D 0 (Jacobi identity). Let M be a real analytic manifold and p 2 M . We consider a real-valued analytic function in a neighbourhood of p and denote its totality by C ! .p/. If a linear mapping Xp W C ! .p/ ! R satisfies Xp .' / D Xp .'/ .p/ C '.p/Xp . /; it is called a tangent vector at p. We designate its totality by Mp or Tp .M / and call it the tangent space of M at p. Definition 1.1.3. Let M; N be analytic manifolds, and suppose that the mapping  W M ! N is analytic at p 2 M . Then, since f ı  2 C ! .p/ for f 2 C ! ..p//, Y.p/ 2 T.p/ .N / is defined for Xp 2 Tp .M / by the formula © Springer Japan 2015 H. Fujiwara, J. Ludwig, Harmonic Analysis on Exponential Solvable Lie Groups, Springer Monographs in Mathematics, DOI 10.1007/978-4-431-55288-8__1

1

2

1 Preliminaries: Lie Groups and Lie Algebras

Y.p/ .f / D Xp .f ı /; 8 f 2 C ! ..p//: The mapping Xp 7! Y.p/ is linear from Tp .M / to T.p/ .N /, called the differential of  and denoted by .d/p . Definition 1.1.4. Let G be a Lie group. La denotes the left translation and Ra the right translation by a 2 G. Namely, La .g/ D ag; Ra .g/ D ga .g 2 G/: For a; b 2 G, we clearly have La ıRb D Rb ıLa . A vector field X W G 3 g 7! Xg 2 Tg .G/ on G is said to be left-invariant if 

dLg

 h

.Xh / D Xgh

holds for any g; h 2 G. Now let X W g 7! Xg and Y W g 7! Yg be two left-invariant vector fields on a Lie group G. Then, using the fact that the group operation on G is analytic mapping, we see that the vector fields X; Y are analytic. It means that, if f 2 C ! .g/ is analytic in an open neighbourhood U of g, we get analytic functions on U by .Xf /.p/ D Xp .f /; .Yf /.p/ D Yp .f /. So, putting ŒX; Y g .f / D Xg .Yf /  Yg .Xf /; we have ŒX; Y g 2 Tg .G/. In fact, for ';

2 C ! .g/ we simply calculate

ŒX; Y g .' / D Xg .Y .' //  Yg .X.' // D Xg ..Y'/ C '.Y //  Yg ..X'/ C '.X //     D Xg .Y'/  Yg .X'/ .g/ C '.g/ Xg .Y /  Yg .X / D ŒX; Y g .'/ .g/ C '.g/ŒX; Y g . /: Moreover, the vector field g 7! ŒX; Y g on G is left-invariant. Indeed, for a 2 G and f 2 C ! .ag/,     .dLa / ŒX; Y g .f / D ŒX; Y g .f ıLa / D Xg .Y .f ıLa //  Yg .X .f ıLa // D Xg ..Yf /ıLa /  Yg ..Xf /ıLa / D .dLa / .Xg /.Yf /  .dLa / .Yg /.Xf / D Xag .Yf /  Yag .Xf / D ŒX; Y ag .f /:

1.1 Lie Groups and Lie Algebras

3

It is easily checked that the vector space g of all the left-invariant vector fields on G becomes a Lie algebra with respect to the bracket product introduced in this way. Let e be the unit element of G. Through the linear mapping X 7! Xe , g is isomorphic as vector space to Te .G/ and dim g D dim G. Of course we could consider the right-invariant vector fields instead of the left-invariant ones, but nothing new appears since the left and the right transform to each other by the diffeomorphism g 7! g1 of G. Definition 1.1.5. g is called the Lie algebra of the Lie group G. Let G be a Lie group and g its Lie algebra. It is the exponential mapping that connects g and G. We consider an C r map c.t/ from an open I in R to  dinterval  of T G, namely an C r curve in G. Let r  1. The element .dc/t dt c.t / .G/ is t called the tangent vector of the curve c at the point c.t/ and designated by c.t/. P Furthermore, for a vector field X on an open set U in G, a C r curve c .r  1/ in U satisfying c.t/ P D Xc.t / .8 t 2 I / is said to be an integral curve of X . Definition 1.1.6. An analytic homomorphism a from the additive group R to Lie group G is called a one-parameter subgroup of G. In this case a.sCt/ D a.s/a.t/ holds for arbitrary s; t 2 R. If this relation holds only locally in a neighbourhood of 0 2 R, that is to say, there exists a certain ı > 0 such that the above relation holds when s; t; s C t 2 I.ı/, then the analytic mapping a from I.ı/ D ft 2 RI jtj < ıg to G is called a local one-parameter subgroup of G. Proposition 1.1.7. Let a.t/ be a local one-parameter subgroup of Lie group G. Clearly a.0/ D e; a.t/ D a.t/1 . There uniquely exists X 2 g such that a.0/ P D Xe , and a.t/ is an integral curve, passing through e, of the left-invariant vector field X . Proof. For f 2 C ! .a.t//, we see by the left invariance of X that ˇ ˇ ˇ ˇ d d ˇ a.t/.f P /D D f .a.t C s//ˇ f .a.t/a.s//ˇˇ ds ds sD0 sD0 ˇ ˇ     d f ıLa.t / .a.s//ˇˇ D D Xe f ıLa.t / D Xa.t / .f /: ds sD0 Namely, a.t/ is an integral curve of X .



Since an integral curve is given as a solution of a system of ordinary differential equations on a local coordinate system, using the theorem on the existence and the uniqueness of the solution, we get the following. Proposition 1.1.8. For any point g of Lie group G and any element X of the Lie algebra g of G, there exists a certain ı > 0 so that we have a unique integral curve c W I.ı/ ! G of X verifying c.0/ D g. If we write this c as c.tI g/, c.tI g/ D gc.tI e/ holds for arbitrary t 2 I.ı/.

4

1 Preliminaries: Lie Groups and Lie Algebras

Since two one-parameter subgroups coincide globally if they do locally, we have: Lemma 1.1.9. Two one-parameter subgroups a.t/; b.t/ of Lie group G coincide P if a.0/ P D b.0/. An integral curve of a left-invariant vector field turns out to be a local oneparameter subgroup, and extending it to a one-parameter subgroup we establish the following. Theorem 1.1.10. Let G be a Lie group with Lie algebra g. For any element X of g, there exists a unique one-parameter subgroup a.t/ of G satisfying a.0/ P D Xe . Definition 1.1.11. a.t/ in the theorem being written as aX .t/, we define the exponential map exp W g ! G by exp X D aX .1/. Then we easily see the following. Proposition 1.1.12. Let X be any element of the Lie algebra g of Lie group G and let s; t be real numbers. (1) (2) (3) (4) (5) (6)

aX .st/ D asX .t/; aX .t/ D exp.tX /; exp..s C t/X / D exp.sX /exp.tX /; aP X .0/ D Xe ; ˇ d ! ˇ dt f .gexp.tX // t D0 D Xg f; g 2 G; f 2 C .g/; for any one-parameter subgroup c.t/ of G, there exists X 2 g such that c.t/ D exp.tX / .8 t 2 R/:

A homomorphism between Lie groups means an analytic mapping which is a group homomorphism. An isomorphism between Lie groups is defined likewise. The symbol G Š G 0 means that the Lie groups G and G 0 are isomorphic. We denote by Aut.G/ the group formed by all the automorphisms of Lie group G, and call it the automorphism group of G. V being a real or complex vector space of finite dimension, the totality GL.V / of regular endomorphisms of V is a Lie group. A homomorphism from a Lie group G to GL.V / is called a (finite-dimensional) representation of G in V. Let’s make a similar consideration for Lie algebras. A homomorphism of Lie algebras means a linear mapping preserving bracket products. To be isomorphic is designated by the same notation Š. We denote by Aut.g/ the automorphism group of the Lie algebra g. A vector subspace closed under the bracket product is called a Lie subalgebra. V being a real or complex vector space of finite dimension, the totality gl.V / of endomorphisms of V becomes a Lie algebra by the bracket product ŒX; Y  D X Y  YX . A homomorphism from the Lie algebra g to gl.V / is called a (finite-dimensional) representation of g in V.

1.1 Lie Groups and Lie Algebras

5

Proposition 1.1.13. Let G; H be Lie groups with Lie algebras g; h respectively and ' W G ! H a homomorphism. Then, for any X 2 g, there exists a unique Y D .d'/.X / 2 h verifying Y'.g/ D .d'/g .Xg / .8 g 2 G/. And d' W g ! h is a homomorphism. Proof. As before, we denote by e the unit element of G and by e 0 that of H . For any X 2 g, there uniquely exists Y 2 h such that Ye0 D .d'/e .Xe /. Then, we shall see that Y'.g/ D .d'/g .Xg / at any g 2 G. In fact,     Y'.g/ D dL'.g/ e0 .Ye0 / D dL'.g/ e0 ..d'/e .Xe //       D d L'.g/ ı' e .Xe / D d 'ıLg e .Xe /    D .d'/g dLg e .Xe / D .d'/g .Xg /: Let’s put Yj D .d'/.Xj / .j D 1; 2/. For any f 2 C ! .'.g//, ..d'/g ŒX1 ; X2 /f D ŒX1 ; X2 g .f ı'/ D .X1 /g .X2 .f ı'//  .X2 /g .X1 .f ı'// D .Y1 /'.g/ .Y2 f /  .Y2 /'.g/ .Y1 f / D ŒY1 ; Y2 '.g/ f: 

So d' is a Lie algebra homomorphism.

Definition 1.1.14. Let G be a Lie group with Lie algebra g. Let a 2 G, and consider the inner automorphism ia W g 7! aga1 of G by a. We denote by Ad a the differential d.ia / of ia . Ad a is an element of Aut.g/ and Ad is a homomorphism from G to Aut.g/. The representation Ad of G on g is called the adjoint representation of G. We often make use of the following result. Theorem 1.1.15. Let g; h be Lie algebras of Lie groups G; H respectively. Concerning a homomorphism ' from G to H , we have '.exp X / D exp.d'.X // .8 X 2 g/: Proof. We consider a one-parameter subgroup b.t/ D '.exp.tX // .t 2 R/ of H . P D Let e 0 be the unit element of H . As we already saw, using Y 2 h such that b.0/ Ye0 , we have b.t/ D exp.tY /; t 2 R. On the other hand, we put a.t/ D exp.tX /. Since b D 'ıa, P D .db/0 Ye0 D b.0/  D .d'/e .da/0

 

d dt d dt

 0

  D .d'/e .a.0// P D .d'/e .Xe /: 0

Hence Y D .d'/.X /. Therefore, '.exp.tX // D exp.td'.X // for any t 2 R. Putting t D 1, we obtain the desired result. 

6

1 Preliminaries: Lie Groups and Lie Algebras

Theorem 1.1.16. Let G be a Lie group with Lie algebra g. (1) The exponential map exp is an analytic mapping from g to G. (2) The differential .d exp/0 W g0 D T0 .g/ ! Te .G/ at the point 0 is an isomorphism of vector spaces. (3) The exponential map exp induces a diffeomorphism between an open neighbourhood V of 0 in g and an open neighbourhood W of e in G. Proof. We show (1). Let dim G D n. We take a local coordinate system .U; /; .e/ D 0; around the unit element e in G and denote by x D .x1 ; : : : ; xn / the coordinate function in the open set .U / of Rn . We fix a basis fX1 ; : : : ; Xn g of g. If we write on U Xi D

n X j D1

aij

@ .1  i  n/; @xj

aij are analytic functions on U . Using u D .u1 ; : : : ; un / 2 Rn , any X 2 g is uniquely written as X D X.u/ D

n X kD1

uk Xk D

n n X X j D1

! uk akj

kD1

@ : @xj

So, if we write aX .t/ D a.t; u/ D .a1 .t; u/; : : : ; an .t; u//; cj .x; u/ D

n X

uk akj .x/;

kD1

these functions ai .t; u/ satisfy a system of ordinary differential equations d ai .t; u/ D ci .a.t; u/; u/; 1  i  n dt and the initial condition ai .0; u/ D 0: Since ci .x; u/ is an analytic function on U  Rn , the theorem on the existence and uniqueness of a solution for a system of ordinary differential equations and the theorem on the differentiability with respect to parameter u give us the following. For u D .u1 ; : : : ; un /, put kuk D max1i n jui j. Then, for a certain ı > 0, there exists in J.2ı/ D I.2ı/  fu 2 Rn I kuk < 2ıg a unique solution a.t; u/ of this initial value problem  anda is an analytic mapping from J.2ı/ to U . As a.t; u/ D a ı; tıu , an analytic solution a.t; u/ 2 U is finally defined for arbitrary .t; u/ 2 R  Rn satisfying ktuk < ı 2 . In particular, if kuk < ı 2 ,

1.1 Lie Groups and Lie Algebras

7

exp.X.u// D a.1; u/ is analytic with respect to u and belongs to U . Thus if we put U0 D fX.u/ 2 gI kuk < ı 2 g; U0 is an open neighbourhood of 0 in g and exp is an analytic mapping from U0 to U . For any X 2 g, X 2 U0 with a sufficiently large positive integer m.   m m So, exp X D exp X is analytic around X . m Since (3) is immediately obtained from (2) and the inverse function theorem, let’s show (2). The tangent space g0 at 0 of the vector space g becomes g0 D g, when we identify the differential operator in the direction of X at 0, ˇ ˇ d f .tX /ˇˇ ; f 2 C ! .0/; D.X /0 f D dt t D0 with X 2 g. Under this identification, ..d exp/0 X / f D D.X /0 .f ı exp/ D

ˇ ˇ d D Xe f: f .exp.tX //ˇˇ dt t D0

Hence .d exp/0 X D 0 , Xe D 0 , X D 0.



Definition 1.1.17. We denote by log W W ! V the inverse mapping of the mapping exp W V ! W in (3) of the theorem. We take a basis fX1 ; : : : ; Xn g of g and its dual basis fX1 ; : : : ; Xn g. If we put xi D Xi ı log, then x D .x1 ; : : : ; xn / is a local coordinate of G defined on W . Moreover, if we consider the mapping g D exp.y1 X1 /    exp.yn Xn / 7! y D .y1 ; : : : ; yn / 2 Rn ; the value taken at the unit element e of G by the Jacobian of y with respect to x is equal to 1. Therefore, y is also a local coordinate around e. x; y are respectively called the canonical coordinate of the first and the second kind relative to the basis fXi gniD1 . Now let’s see relations existing through the exponential map between the product in a Lie group and the bracket product in its Lie algebra. Let f .t/; g.t/ be numerical or vector-valued functions defined on a neighbourhood of 0 in R and let j  j denote the absolute value or the norm. If there exists such a constant C > 0 that jf .t/j  C jg.t/j holds in some neighbourhood of 0, we write f .t/ D O.g.t// with Landau notation. Using the notations introduced until now: Lemma 1.1.18. Let g 2 G; f 2 C ! .g/, n be a positive integer, and X; X1 ; : : : ; Xn elements of g. ˇ dn ˇ (1) .X n f /.g/ D dt n f .gexp.tX // t D0 . ˇ @n (2) .X1   Xn f /.g/ D @t1 @tn f .gexp.t1 X1 /    exp.tn Xn // ˇt1 DDtn D0 .

8

1 Preliminaries: Lie Groups and Lie Algebras

(3) In a neighbourhood of t D 0, f .gexp.tX // D f .g/ C t.Xf /.g/ C

t2 2 .X f /.g/ C O.t 3 /: 2

Proof. (1), (2) are clear. (3) is obtained by substituting (1) in the Maclaurin expansion 00

F .t/ D F .0/ C F 0 .0/t C

F .0/ 2 t C O.t 3 / 2

of the function F .t/ D f .gexp.tX //.



Theorem 1.1.19. Let X; Y 2 g and t 2 R.   2 (1) exp.tX /exp.tY / D exp t.X C Y / C t2 ŒX; Y  C O.t 3 / .   3 (2) exp.tX /exp.tY /exp.tX / D exp tY C t 2 ŒX;  2Y  C O.t / . 3  (3) exp.tX /exp.tY /exp.tX /exp.tY / D exp t ŒX; Y  C O.t / . Proof. We show (1). By (2) of the preceding lemma, .X m Y n f /.e/ D

ˇ ˇ @mCn ˇ f .exp.tX /exp.sY // ˇ m n @t @s t DsD0

for any f 2 C ! .e/; m; n 2 N. Applying this formula to the Taylor expansion of the function F .t; s/ D f .exp.tX /exp.sY // with two variables and letting r D

p t 2 C s 2 , we get

f .exp.tX /exp.sY // Df .e/ C ..tX C sY /f /.e/ 1 C ..t 2 X 2 C 2tsX Y C s 2 Y 2 /f /.e/ C O.r 3 /: 2 In particular, letting s D t, f .exp.tX /exp.tY // Df .e/ C t..X C Y /f /.e/ C

t2 ..X 2 C 2X Y C Y 2 /f /.e/ C O.t 3 /: 2

From what we have seen until now, there exists a neighbourhood V of 0 in g and for sufficiently small jtj we can uniquely write as exp.tX /exp.tY / D exp.Z.t//; Z.t/ 2 V:

1.1 Lie Groups and Lie Algebras

9

Since Z.t/ D log .exp.tX /exp.tY // is analytic in a neighbourhood of 0 and since Z.0/ D 0, we have the Maclaurin expansion Z.t/ D tA C t 2 B C O.t 3 /; here A; B are elements of g not depending on t. Let’s take a basis fXi gniD1 of g and introduce the canonical coordinate of the first kind x D .x1 ; : : : ; xn / around the unit element e of G. By (3) of the preceding lemma,  1 Z.t/2 xi .e/ C O.t 3 / 2    1 D .tA C t 2 B/xi .e/ C .tA C t 2 B/2 xi .e/ C O.t 3 / 2    t2 2 2 D tA C t B C A xi .e/ C O.t 3 /: 2

xi .exp.Z.t/// D xi .e/ C .Z.t/xi /.e/ C

Comparing this with the above equation, for any 1  i  n, .Axi /.e/ D ..X C Y /xi /.e/     .2B C A2 /xi .e/ D .X 2 C 2X Y C Y 2 /xi .e/: As x is a local coordinate around e, we have Ae D .X C Y /e and A D X C Y by the left invariance. Likewise, 2B C A2 D X 2 C 2X Y C Y 2 and BD

 1 2 1 1 .X C 2X Y C Y 2 /  .X C Y /2 D .X Y  YX / D ŒX; Y : 2 2 2

We show (2). Just like (1), if we put Z.t/ D log.exp.tX /exp.tY /exp.tX // D tA C t 2 B C O.t 3 /; f .exp.Z.t/// D

X

t mCnCp .X m Y n .X /p f / .e/ C O.t 3 / mŠnŠpŠ mCnCp2

t2  2 .X C Y 2 C X 2 D f .e/ C t..X C Y  X /f /.e/ C 2  C 2.X Y  YX /  2X 2 /f .e/ C O.t 3 /    1 2 Y C ŒX; Y  f .e/ C O.t 3 /: D f .e/ C t.Yf /.e/ C t 2 2

10

1 Preliminaries: Lie Groups and Lie Algebras

Setting f D xi .1  i  n/ as above, we get 1 1 A D Y; B C A2 D Y 2 C ŒX; Y  2 2 and B D ŒX; Y . We show (3). If we put Z.t/ D log.exp.tX /exp.tY /exp.tX /exp.tY // D tA C t 2 B C O.t 3 /; f .exp.Z.t/// D

X

t mCnCpCq .X m Y n .X /p .Y /q f / .e/ C O.t 3 / mŠnŠpŠqŠ mCnCpCq2 t 2  2 X C Y 2 C X2 C Y 2 2   C 2.X Y  X 2  X Y  YX  Y 2 C X Y / f .e/ C O.t 3 /

D f .e/ C t..X C Y  X  Y /f /.e/ C

D f .e/ C t 2 ..X Y  YX /f /.e/ C O.t 3 /: Thus, A D 0; B D ŒX; Y .



Definition 1.1.20. For each element X of a real or complex Lie algebra g we define a linear transformation adX on g by .adX /.Y / D ŒX; Y  .Y 2 g/. The Jacobi identity says that the mapping ad is a homomorphism of Lie algebras from g to gl.g/. We call this the adjoint representation of g and its kernel the centre of g. Theorem 1.1.21. Let G be a Lie group with Lie algebra g. (1) (2) (3) (4) (5)

For any g 2 G; X 2 g, g.exp X /g 1 D exp..Adg/X /. Ad W G ! GL.g/ is an analytic mapping. d.Ad/ D ad (cf. Proposition 1.1.13). For any X 2 g, Ad.exp X / D exp.adX /. If G is connected, the kernel of the adjoint representation of G coincides with the centre of G.

Proof. The claim (1) comes from Definition 1.1.14 and Theorem 1.1.15. We show (2). We take a basis fXi gniD1 of g and identify GL.g/ with the set GL.n; R/ P of all the real regular matrices of order n. Namely, if we write .Adg/.Xj / D  niD1 aij.g/Xi for g 2 G, the linear transformation Adg is identified with the matrix aij .g/ . Then by (1) g.exp.tXj //g 1 D exp.t.Adg/Xj / D exp t

n X i D1

! aij .g/Xi

1.1 Lie Groups and Lie Algebras

11

and the left member of this equation is an analytic function of g. Relative to the basis fXi gniD1 , let .U; .x1 ; : : : ; xn // be a canonical coordinate system of the first kind around the unit element e of G. g0 being any element of G and taking t > 0 small enough, we have g.exp.tXj //g 1 2 U as far as g remains in a certain open neighbourhood W of g0 . We get from the above equation that   xi g.exp.tXj //g 1 D taij .g/: Hence aij .g/ is analytic in the open neighbourhood W of g0 and the mapping Ad is analytic on G. We show (3). We put ' D d.Ad/ and let X; Y 2 g; t 2 R. With g.t/ D exp.tX /, Theorem 1.1.15 gives Ad.g.t// D exp.t'.X //: Together with (1) we get ig.t /.exp.tY // D exp.t.Ad.g.t///.Y // D exp.texp.t'.X //Y / D exp.tY C t 2 '.X /Y C O.t 3 //: On the other hand, Theorem 1.1.19 (2) gives ig.t /.exp.tY // D exp.tY C t 2 ŒX; Y  C O.t 3 //: Comparing these two equations, we see for sufficiently small jtj that '.X /Y D ŒX; Y  C O.t/: Setting t ! 0, '.X /Y D ŒX; Y  D .adX /.Y /. Namely ' D ad. The claim (4) comes from (3) and Theorem 1.1.15. We show (5). If g 2 G belongs to the kernel of the adjoint mapping, Definition 1.1.14 means that d.ig / becomes the identity mapping of g and ig is the identity mapping in some neighbourhood of e. As G is connected, ig finally becomes the identity mapping on the whole of G. Thus, ig .a/ D gag 1 D a for any a 2 G. That is to say, ga D ag and g belongs to the centre of G. The inverse inclusion is clear.  We sketch the correspondence between Lie subgroups and Lie subalgebras. Please refer to Sugiura [74] for the details.

12

1 Preliminaries: Lie Groups and Lie Algebras

Definition 1.1.22. Let M be a subset of an analytic manifold N . We say that M is an analytic submanifold of N , if the following two statements hold. (1) M is an analytic manifold. (2) The inclusion map i W M ! N; i.x/ D x .8 x 2 M /; is an analytic map and its differential .d i /p W Mp ! Np is injective at each point p 2 M . Definition 1.1.23. Let G be a Lie group. When a subgroup H of G is an analytic submanifold of G and becomes a Lie group with these two structures, we say that H is a Lie subgroup of G. A connected Lie subgroup is called an analytic subgroup. Theorem 1.1.24. Let G be a Lie group with Lie algebra g. (1) The Lie algebra h of any Lie subgroup H of G is isomorphic to the Lie subalgebra .d i /.h/ of g. Here i W H ! G denotes the inclusion map. Let’s identify h with .d i /.h/. (2) Conversely, for any Lie subalgebra h of g, there exists a connected Lie subgroup H of G the Lie subalgebra of which is h. H is nothing but the maximal connected integral manifold containing e of h. (3) The correspondence H $ h is a bijection between the totality of connected Lie subgroups of G and the totality of Lie subalgebras of g. Here we recall the definitions of solvable Lie algebras and nilpotent Lie algebras. Let g be a Lie algebra. When ŒX; Y  D 0 for any X; Y 2 g, we say that g is commutative or abelian. A linear subspace a of g is an ideal if ŒX; a  a for any X 2 g. Then, the quotient space g=a becomes a Lie algebra by the operation induced from the bracket product of g. This is called the quotient Lie algebra of g by a. Now, letting Dg D Œg; g D

X m Œxi ; yi I xi ; yi 2 g; m  1 ; i D1

Dg is an ideal of g and the quotient Lie algebra g=Dg is commutative. If we set inductively D 0 g D g; D k g D D.D k1 g/ .k D 1; 2;    /; we get a sequence of ideals g D D0g  D1 g  D2 g     : When there exists k such that D k g D f0g, Lie algebra g is said to be solvable. Changing slightly this procedure, if we set inductively C 0 g D g; C k g D Œg; C k1 g .k D 1; 2;    /;

1.1 Lie Groups and Lie Algebras

13

we get a sequence of ideals g D C 0g  C 1g  C 2g     : When there exists k such that C k g D f0g, Lie algebra g is said to be nilpotent. Besides, a Lie group is solvable when its Lie algebra is solvable and the same for the nilpotent case. From the above, nilpotent Lie groups are solvable Lie groups, but the gap between these two categories is very big. Later we shall define the category of exponential solvable Lie groups which will be our object, and survey the orbit method for these groups. Here we collect fundamental properties of nilpotent and solvable Lie groups. Proposition 1.1.25. Let g ¤ f0g be a (real or complex) Lie algebra and z its centre. (1) If g is nilpotent, any Lie subalgebra h and the image of any homomorphism '.g/ are also nilpotent. (2) If g is nilpotent, z ¤ f0g. (3) If the quotient Lie algebra g=z is nilpotent, g is also nilpotent. Proof. For any k 2 N, C k h  C k g and C k '.g/ D '.C k g/. So, we have (1). Besides, if C k g first becomes f0g, then f0g ¤ C k1 g is contained in the centre z and (2) follows. We finally show (3). If C k .g=z/ D f0g, C k g  z and hence C kC1 g D f0g.  Lemma 1.1.26. Let V be a finite-dimensional vector space, and we consider the Lie algebra g D gl.V /. If X 2 g is a nilpotent endomorphism, then adX is a nilpotent linear transformation of g. Proof. The two linear transformations LX .Y / D X Y; RX .Y / D YX of g are nilpotent and mutually commutative and adX D LX  RX . Thus, we can apply the binomial expansion formula to conclude that adX is nilpotent.  Theorem 1.1.27 (Engel). Let V ¤ f0g be a finite-dimensional vector space and g a Lie subalgebra of gl.V /. If every element of g is a nilpotent linear transformation, there exists 0 ¤ v 2 V such that gv D f0g. Proof. We proceed by induction on n D dim g. The assertion is clear when n D 1. Let n  2 and h be a proper Lie subalgebra of g. The preceding proposition implies that adg h is a Lie algebra composed of nilpotent linear transformations of g, which keep h stable. Therefore, adg=h h is a Lie algebra composed of nilpotent linear transformations of g=h. Since dim h < dim g, the induction hypothesis implies the existence of X 2 .g n h/ such that Œh; X   h. Hence, the normalizer n.h/ D fA 2 gI ŒA; h  hg of h in g properly contains h. So, if we take h as a maximal proper Lie subalgebra of g, then n.h/ D g and h is an ideal of g.

14

1 Preliminaries: Lie Groups and Lie Algebras

Here, if dim.g=h/ > 1, taking the inverse image k in g of a one-dimensional Lie subalgebra of g=h, we have h ¨ k ¨ g, which contradicts the maximality of h. Hence dim.g=h/ D 1. Let’s utilize once again the above X 2 .g n h/. If we put W D fw 2 V I hw D f0gg, the induction hypothesis gives W ¤ f0g. On the other hand, h is an ideal of g and so W is even g-invariant. Now if we take an eigenvector v 2 W for the eigenvalue 0 of X jW , v is the element of V we are looking for.  Theorem 1.1.28. For a Lie algebra g, the following are equivalent: (a) g is nilpotent. (b) For any X 2 g, adX is a nilpotent linear transformation. (c) There exists a decreasing sequence of ideals g D g0  g1      gn D f0g; such that dim.gi =gi C1 / D 1 and such that Œg; gi   gi C1 for i D 0;    ; n  1. Proof. (a) ) (b): If we assume C n g D f0g, then .adX /n D 0. (b) ) (a): We proceed by induction on n D dim g. The claim being clear when n D 0, let n  1. The previous theorem says that there exists an element X ¤ 0 of g such that Œg; X  D f0g. Thus the centre z of g is not f0g and ad.g=z/ is formed only by nilpotent linear transformations. Hence the induction hypothesis implies that g=z is nilpotent. Finally, g is nilpotent by Proposition 1.1.25. (a) , (c): This is trivial.  Definition 1.1.29. We say that a sequence M D .hi /niD0 of subalgebras of a Lie algebra g is a Malcev sequence if g D h0  h1      hn D f0g; such that dim.hi =hi C1 / D 1 for i D 0;    ; n  1. We say that the Malcev sequence M passes through a subalgebra k of g, if k D hj for some j 2 f1;    ; ng. A Malcev sequence M is called a Jordan–Hölder sequence, if hj is an ideal of g for every j 2 f1; : : : ; ng. Let g be a Lie algebra. A decreasing sequence g D g0  g1      gm D f0g of ideals of g is called a composition series, if the g-module gi =gi C1 is irreducible for every i 2 f0;    ; m  1g. A Malcev sequence .hj /j of g is called strong, if whenever hj is not an ideal of g, then hj 1 and hj C1 are ideals and the g-module hj 1 =hj C1 is irreducible. Every composition sequence of g can be refined to give a strong Malcev sequence. Remark 1.1.30. Let g be a nilpotent Lie algebra. Let G be a simply connected, connected Lie group with Lie algebra g. Then for any X; Y 2 g, we have that

1.1 Lie Groups and Lie Algebras

15

exp X exp Y D exp.CB.X; Y //; where CB W g  g ! g denotes the Campbell–Hausdorff product 1 1 1 CB.X; Y / D X C Y C ŒX; Y  C ŒX; ŒX; Y  C ŒY; ŒY; X  C    ; 2 12 12 which is a finite expression in the brackets of X and Y , since g is nilpotent. Hence we can define on g a group multiplication .X; Y / 7! X CB Y WD CB.X; Y / which is polynomial in X and Y . This new group .g; CB / is evidently isomorphic to G, since the Lie algebra of .g; CB / is equal to g. Proposition 1.1.31. Let g be a Lie algebra. (1) If g is solvable, any Lie subalgebra h and the image of any homomorphism '.g/ are also solvable. (2) If a is a solvable ideal of g and the quotient algebra g=a is solvable, then g is solvable. (3) If a; b are solvable ideals of g, then a C b is also a solvable ideal. Proof. For any k 2 N, D k h  D k g and D k '.g/ D '.D k g/. So, the assertion (1) follows. We show (2). Letting D n .g=a/ D f0g, D n g  a. Further supposing D m a D f0g, nCm D g D f0g. The claim (3) is obtained by applying (1) and (2) to .a C b/=a Š b=.a \ b/.  Lemma 1.1.32. Let V be a real or complex vector space of finite dimension, g a Lie subalgebra of gl.V / and h an ideal of g. We assume that, for an element ˛ of the dual vector space h , the common eigenspace W D fw 2 V I Y w D ˛.Y /w 8 Y 2 hg of h is not f0g. Then: (1) ˛.Œg; h/ D f0g; (2) gW  W . Proof. We fix 0 ¤ w 2 W and X 2 g. For any 0 ¤ i 2 N let Wi be the subspace of V generated by fw; X w; : : : ; X i 1 wg and W0 D f0g. If we take the minimal positive integer n satisfying WnC1 D Wn , then dim Wn D n; X Wn  Wn and WnCj D Wn for any j  0. Now, for any i 2 N; Y 2 h, YX i w ˛.Y /X i w mod Wi holds. In fact, this is clear for i D 0. If we assume it for i  1, then modulo Wi YX i w D YXX i 1 w D X YX i 1 w C ŒY; X X i 1 w D ˛.Y /X i w C ˛.ŒY; X /X i 1 w ˛.Y /X i w

16

1 Preliminaries: Lie Groups and Lie Algebras

and it holds for i too. From this equation, we see Y Wi  Wi for any i 2 N. Consequently, relative to the basis fw; X w; : : : ; X n1 wg of Wn , the restriction Y jW of Y to W is represented by an upper triangular matrix with diagonal entries ˛.Y /. Therefore its trace is Tr .Y jWn / D n˛.Y /. In particular, ˛.ŒX; Y / D 0 because of 0 D Tr .ŒX; Y jWn / D n˛.ŒX; Y /. (2) The assertion (1) implies YX w D X Y w C ŒY; X w D ˛.Y /X w C ˛.ŒY; X /w D ˛.Y /X w for any X 2 g; Y 2 h; w 2 W . So, X w 2 W .



Theorem 1.1.33 (Lie). Let V ¤ f0g be a complex vector space of finite dimension and g a solvable Lie subalgebra of gl.V /. Then, there exists a common eigenvector w ¤ 0 for g. Namely, there exists ˛ 2 g such that X w D ˛.X /w .8 X 2 g/. Proof. We proceed by induction on n D dim g. The statement being clear when n D 0, we assume n > 0. As g ¤ f0g is solvable, gN D g=Œg; g ¤ f0g. If we take a subspace of codimension 1 in gN and write h its inverse image in g, then h is an ideal of codimension 1 in g. From the induction hypothesis, W D fw 2 V I Y w D ˇ.Y /w; 8 Y 2 hg is not f0g for some ˇ 2 h . Besides, the previous proposition says that W is g-invariant. Now, if we write g D CT ˚ h, T jW has an eigenvector 0 ¤ w 2 W and T w D w;  2 C. Clearly, w is a common eigenvector for g.  Corollary 1.1.34. Let g be a solvable Lie algebra and .; V / a complex finitedimensional representation of gC . Let V D V0  V1      Vm D f0g be a composition series of V . Then dim.Vj =Vj C1 / D 1 for every j D 0; : : : ; m  1. In particular there exists a basis B D fb1 ; : : : ; bm g of V , such that the matrix M.X / of .X / in this basis is upper triangular for every X 2 g. Proof. The simple modules Vj =Vj C1 are necessarily one-dimensional by Theorem 1.1.33. Applying this theorem several times, we find a sequence .Vj /j of g-invariant subspaces of V such that Vj D Rbj ˚ Vj C1 and such that .X /vj D j .X /vj modulo Vj C1 for every X 2 g, for some linear functional j W g ! C, j D 1; : : : ; m  1.  Corollary 1.1.35. Let g be a solvable Lie algebra. Then the ideal Dg D Œg; g of g is nilpotent.

1.1 Lie Groups and Lie Algebras

17

Proof. Let .; V / D .ad; gC / be the adjoint representation of g. Then for every X 2 Dg its matrix M.X / has 0 diagonal entries. Hence ad.X / is nilpotent. This shows that Dg is nilpotent by Engel’s theorem.  Definition 1.1.36. Let g be a solvable Lie algebra. Let .; V / be a finitedimensional complex g-module. We put SV D f  2 gC I 9v 2 V n f0gI .X /v D .X /v; 8X 2 gC g: We call an element of SV an eigenvalue of . To an eigenvalue  of  is associated the eigenspace of V , defined by E .V / D E D f v 2 V I .X /v D .X /v 8X 2 gC g: We also let E.V / D E D

X

E .V /:

2SV

Since g is solvable, SV is nonempty by the theorem of Lie. Definition 1.1.37. Let .; V / be a finite-dimensional complex g-module. A linear form  on gC , or its restriction on g, is called a weight of .; V / if there exists a g-invariant subspace V0 of V and an element v in V nV0 such that for all X in g we have .X /v  .X /v 2 V0 : A weight of .ad; gC / is called a root of gC or g. We denote by RV the set of the weights of .; V / and by R the set of the roots of gC . We fix finally an element T 2 g which is in general position with respect to  and ad, i.e. 0 1 [ [ T 2gn@ ker.  0 / [ ker.  0 /A ¤0 2RV

¤0 2R

and let T ./, resp. T .ad/, be the spectrum of .T /, resp. of ad.T /. The mappings RV ! T ./; res.R ! T .ad/;  7! .T /;

(1.1.1)

are then automatically injective. Let ˚

V D v 2 V I ..T /  .T /IdV /dim.V / .v/ D 0 ;

 2 RV ;

18

1 Preliminaries: Lie Groups and Lie Algebras

and ˚

.gC / D X 2 gC I .ad.T /  .T /IdgC /dim.g/ .X / D 0 ;

 2 R:

We then obtain the Jordan decomposition of V with respect to T : V D

M

V

(1.1.2)

2T ./

and M

gC D

.gC / :

2T .ad/

Definition 1.1.38. Let g be a solvable Lie algebra and let T be an element of g in general position with respect to ad. Let for  2 T .ad/, g WD ..gC / C .gC / / \ g D ..gC / C .gC / / \ g: Evidently g D g for every . Let g1 WD

X

g :

2T .ad/;¤0

Then we have that g D g0 ˚ g1 :

(1.1.3)

Proposition 1.1.39. We have that g  Œg; g for every  ¤ 0;  2 T .ad/. Proof. By definition of .gC / , we see that ad.T /  IdV is nilpotent on .gC / , so for  ¤ 0, ad.T / is invertible on .gC / . Hence .gC / D ŒT; .gC /   ŒgC ; gC :  We have the following bracket relation. Proposition 1.1.40. Let .; V / be a finite-dimensional g-module. For all  in T ./ and all  in T .ad/ we have that .g /.V /  VC : In particular g0 D .gC /0 \ g is a nilpotent subalgebra of g and .g0 /V  V for all  2 T ./.

1.1 Lie Groups and Lie Algebras

19

Proof. Let v 2 V and X 2 .gC / . We choose a composition series .Vi /i of V and let j be the index for which v 2 Vj nVj C1 . Then we observe that  D j .T / since v 2 Vj and so .T /..X /v/  . C /.X /v D .ŒT; X /v  .X /v C .X /..T /v  v/ D .ad.T /X  X /v C .X /..T /v  v/ D .ad.T /X  X /v mod gj C1 : This shows that inductively on i , ..T /  . C /IdV /i ..X /v/ D .ad.T /X  X /i .v/ mod gjC1 : Hence for i big enough, it follows that ..T /  . C /IdV /i ..X /v/ 2 gj C1 : Repeating this game for j C 1, we finally see that ..T /  . C /IdV /k ..X /v/ D 0 for some k 2 N . If g0 is not nilpotent, then there exists a root  ¤ 0 of the Lie algebra g0 , which defines then also a root of g. But this root is 0 on T by definition of g0 . This contradiction tells us that 0 is the only root of g0 , i.e. g0 is nilpotent.  Remark 1.1.41. Let g be a solvable Lie algebra. We can use the decomposition g D g0 ˚ g1 of Definition 1.1.3 to describe the multiplication of the simply connected group G associated to g. Let w be a subspace of g0 such that g0 D w ˚ .g0 \ Œg; g/. Then for X; Y 2 w, we can write X CB Y D .X C Y; P .X; Y //; where P .X; Y / is a polynomial expression in X; Y , since g0 is nilpotent. We now write g as a semi-direct product of w and n WD Œg; g, n being a nilpotent ideal of g. This allows us to define a group multiplication  on G WD w  n by letting 0

.w; n/.w0 ; n0 / WD .w C w0 ; P .w; w0 / CB .e w n CB n0 //:

(1.1.4)

Here the symbol e w for w 2 w denotes the automorphism e w WD exp.ad.w// of the nilpotent Lie algebra n. The group .G; / admits as Lie algebra the semi-direct product w  n with the Lie bracket Œ.w; X /; .w0 ; X 0 / D .0; Œw; w0  C ŒX; X 0  C Œw; X 0  C ŒX; w0 / which is isomorphic to g. Hence the Lie group G, which is simply connected, admits g as its Lie algebra.

20

1 Preliminaries: Lie Groups and Lie Algebras

Proposition 1.1.42. Let g be a solvable Lie algebra, let b be an ideal of g, such that g=b is abelian. Let v  g such that g D v ˚ b. Let G be a simply connected Lie group with Lie algebra g. Then G contains a closed simply connected subgroup B with Lie algebra b and the mapping E W v  B ! GI .v; b/ 7! .exp v/b is a diffeomorphism. Proof. We proceed by induction on the dimension of g. If dim g D 1, there is nothing to prove. If dim v D 1, i.e. v D RX , then we take a simply connected Lie group B 0 with Lie algebra b and we consider the derivation d.U / D adX.U /; U 2 b, of b. This defines a one-parameter group of automorphisms a.t/; t 2 R, of B 0 , where a.t/.exp U / D exp.e t d .U //; U 2 b. We can thus construct a new Lie group G 0 D R  B 0 with the multiplication .s; u/.s 0 ; u0 / WD .s C s 0 ; a.s 0 /uu0 /: The Lie algebra of this group is isomorphic to g and so there exists an isomorphism W G 0 ! G. The subgroup B WD .B 0 / is closed and simply connected with Lie algebra b, since the same is true for f0gB 0  G 0 . Furthermore, exp.sX / D .s; 0/ for every s 2 R. This shows that the mapping E.s; b/ D exp.sX /b D .s; 1 .b// from R  B onto G is a diffeomorphism. Suppose now that dim v > 1. Choose a vector X 2 v and a subspace w in v, such that v D RX ˚ w and let g0 D w C b. Applying the induction hypothesis to g0 , we see that there exists a closed normal simply connected subgroup B of G, such that the mapping R  w  B ! GI .s; U; b/ 7! exp.sX /.exp U /b is a diffeomorphism. Since G=B is abelian, we have that exp.sX /exp U D exp.sX C U /q.s; U /1 ; where .s; U / 7! q.s; U / 2 B is a C 1 mapping. This shows that our mapping .sX C U; b/ 7! exp.sX C U /b D exp.sX /.exp U /q.s; U /b is a diffeomorphism of v  B onto G. 

1.2 Enveloping Algebra In this section, following Dixmier [21] we define the enveloping algebra of a Lie algebra and give the fundamental theorem of Poincaré–Birkhoff–Witt. Let g be a real or complex Lie algebra. We designate the base field by k. Namely, k is R or C. Let T D T 0 ˚ T 1 ˚    ˚ T n ˚    be the tensor product of g. Here T n denotes an n tensor product of g, in particular T 0 D k  1 and T 1 D g. We denote by J the two-sided ideal of T generated by the tensors X ˝ Y  Y ˝ X  ŒX; Y  .X; Y 2 g/:

1.2 Enveloping Algebra

21

The algebra T =J is called the enveloping algebra of g and denoted by U.g/. The composite mapping  W g ! T ! U.g/ is called the canonical mapping of g into U.g/. Thus, for any X; Y 2 g, .X /.Y /  .Y /.X / D .ŒX; Y /: When g is a commutative Lie algebra, U.g/ is nothing but the symmetric algebra S.g/ of the vector space g. We denote the canonical image in U.g/ of the two-sided ideal TC D T 1 ˚ T 2 ˚    of T by UC .g/. Since T D T 0 ˚ TC D k  1 ˚ TC and J  TC , if we denote by U 0 the image of T 0 in U.g/, U.g/ D U 0 ˚ UC .g/ and U 0 D k  1 is one-dimensional. Let’s identify U 0 with k. The associative algebra U.g/ is generated by 1 and the canonical image of g in U.g/. Lemma 1.2.1. Let  be the canonical mapping of g into U.g/. Let A be an algebra with unity,  a linear mapping of g into A such that .X/.Y /  .Y /.X/ D .ŒX; Y / for any X; Y 2 g. Then, there uniquely exists a homomorphism  0 of U.g/ into A satisfying  0 .1/ D 1 and  0 ı D . Proof. U.g/ is generated by 1 and .g/,  0 is unique if it exists. On the other hand, there uniquely exists a homomorphism ' of T into A, which extends  and satisfies '.1/ D 1. As '.X ˝ Y  Y ˝ X  ŒX; Y / D .X/.Y /  .Y /.X/  .ŒX; Y / D 0 for all X; Y 2 g, '.J / D f0g and passing to the quotient ' gives us the desired homomorphism  0 .  We fix a basis fX1 ; X2 ; : : : ; Xn g of g and put Yi D .Xi / .1  i  n/. For a finite sequence I D .i1 ; : : : ; ip / of positive integers from 1 to n, we set YI D Yi1 Yi2   Yip . When i  i1 ; : : : ; i  ip for an integer i , we write i  I . Furthermore, we denote by Uq .g/ the image of T 0 ˚ T 1 ˚    ˚ T q in U.g/. Lemma 1.2.2. Let Y1 ; : : : ; Yp be elements of g and  a permutation of .1; : : : ; p/. Then,     .Y1 /    .Yp /   Y.1/     Y.p/ belongs to Up1 .g/. Proof. It is sufficient to prove the claim when  is the transposition of j and j C 1. In that case, the claim follows from the equality .Yj /.Yj C1 /  .Yj C1 /.Yj / D .ŒYj ; Yj C1 /:



22

1 Preliminaries: Lie Groups and Lie Algebras

From this we immediately see the following. Corollary 1.2.3. The YI for all non-decreasing sequences I of length smaller than or equal to p generate the vector space Up .g/. Let P be the polynomial ring of n variables z1 ; : : : ; zn . We denote by Pi for i  0 the totality of polynomials the degree of which is smaller than or equal to i . Besides, we set zI D zi1 zi2   zip for the finite sequence I D .i1 ; : : : ; ip / of natural numbers between 1 and n. Lemma 1.2.4. For any integer p  0, there uniquely exists a linear mapping fp from the vector space g ˝ Pp to P satisfying the following conditions. Moreover, the restriction of fp on g ˝ Pp1 is fp1 . .Ap / if i  I; zI 2 Pp , fp .Xi ˝ zI / D zi zI ; .Bp / if zI 2 Pq ; q  p, fp .Xi ˝ zI /  zi zI 2 Pq ; .Cp / if zJ 2 Pp1 .p  1/, fp .Xi ˝ fp .Xj ˝ zJ // D fp .Xj ˝ fp .Xi ˝ zJ // C fp .ŒXi ; Xj  ˝ zJ /: Proof. For p D 0, the condition .A0 / means f0 .Xi ˝ 1/ D zi , and the condition .B0 / holds. Let us assume the existence and the uniqueness of the mapping fp1 . If fp exists, its restriction on g ˝ Pp1 satisfies the conditions .Ap1 /; .Bp1 /; .Cp1 / and is equal to fp1 . Therefore, we show that there uniquely exists a linear extension fp of fp1 to g ˝ Pp satisfying the conditions .Ap /; .Bp /; .Cp /. That is to say, we need to define fp .Xi ˝ zI / for non-decreasing sequence I of p elements. When i  I , this is done by the condition .Ap /. Otherwise, we can write I D .j; J /; j < i; j  J . Then, by .Ap1 / and .Cp /, fp .Xi ˝ zI / D fp .Xi ˝ fp1 .Xj ˝ zJ // D fp .Xj ˝ fp1 .Xi ˝ zJ // C fp1 .ŒXi ; Xj  ˝ zJ /: Now, .Bp1 / implies fp1 .Xi ˝ zJ / D zi zJ C w; w 2 Pp1 and by .Ap / fp .Xj ˝ fp1 .Xi ˝ zJ // D zj zi zJ C fp1 .Xj ˝ w/ D zi zJ C fp1 .Xj ˝ w/: Through these processes fp1 is uniquely extended to g ˝ Pp by linearity, and this extension fp satisfies .Ap /; .Bp /. We verify that fp satisfies .Cp / too. From the above construction, when j < i and j  J , the condition .Cp / is satisfied. As ŒXj ; Xi  D ŒXi ; Xj , .Cp / is also satisfied when i < j and i  J , and clearly in the case j D i as well. In consequence, if i  J or j  J , then .Cp / is satisfied. In the other case, we can write J D .k; K/; k  K; k < i; k < j . To simplify the notations, we put from now on fp .X ˝ z/ D X z for X 2 g; z 2 Pp . From the induction hypothesis, Xj zJ D Xj .Xk zK / D Xk .Xj zK / C ŒXj ; Xk zK :

(1.2.1)

1.2 Enveloping Algebra

23

Now we can write Xj zK D zj zK C w; w 2 Pp2 . Because of k  K and k < j , the condition .Cp / is applied to Xi .Xk .zj zK //, to Xi .Xk w/ by the induction hypothesis and hence to Xi .Xk .Xj zK //. Taking Eq. (1.2.1) into account, Xi .Xj zJ / DXk .Xi .Xj zK // C ŒXi ; Xk .Xj zK / C ŒXj ; Xk .Xi zK / C ŒXi ; ŒXj ; Xk zK : Interchanging i and j then taking the difference of these two equations, Xi .Xj zJ /  Xj .Xi zJ / D Xk .Xi .Xj zK /  Xj .Xi zK // C ŒXi ; ŒXj ; Xk zK  ŒXj ; ŒXi ; Xk zK D Xk .ŒXi ; Xj zK / C ŒXi ; ŒXj ; Xk zK C ŒXj ; ŒXk ; Xi zK D ŒXi ; Xj Xk zK C ŒXk ; ŒXi ; Xj zK C ŒXi ; ŒXj ; Xk zK C ŒXj ; ŒXk ; Xi zK D ŒXi ; Xj Xk zK D ŒXi ; Xj zJ :



Lemma 1.2.5. The YI for all non-decreasing sequence I constitute a basis of the vector space U.g/. Proof. We use the notations of the preceding lemma. The lemma says that there exists a bilinear mapping f from g  P to P such that f .Xi ; zI / D zi zI for i  I and f .Xi ; f .Xj ; zJ // D f .Xj ; f .Xi ; zJ // C f .ŒXi ; Xj ; zJ / for any i; j; J . In other words, there exists a representation  of g in P such that .Xi /zI D zi zI when i  I . By Lemma 1.2.1, there exists a homomorphism ' of U.g/ into End.P / such that '.Yi /zI D zi zI when i  I . Therefore, '.Yi1 Yi2   Yip /1 D zi1 zi2   zip when i1  i2      ip . Hence, the YI for non-decreasing sequence I are linearly independent. Together with Corollary 1.2.3, we get the assertion.  Corollary 1.2.6. The canonical map of g into U.g/ is injective. By this corollary we regard from now on that g  U.g/. By what proceeds: Theorem 1.2.7 (Poincaré–Birkhoff–Witt). Let .X1 ; : : : ; Xn / be a basis of the vector space g. When D . 1 ; : : : ; n / runs over Nn , X D X1 1 X2 2   Xn n form a basis of U.g/. Corollary 1.2.8. The algebra U.g/ has no zero divisor.

24

1 Preliminaries: Lie Groups and Lie Algebras

Proof. For D . 1 ; : : : ; n /, we set j j D 1 C  C n . Now computing the product of two non zero elements ` X

AD

m X

c˛ X ˛ ; B D

j˛jD0

dˇ X ˇ

jˇjD0

of U.g/, X

AB D

c˛ dˇ X ˛ X ˇ C

j˛jD`;jˇjDm

0

X

D

@

j jD`Cm

X

1

X

e X

j j<`Cm

X

c˛ dˇ A X C

e X :

j j<`Cm

˛CˇD

Thus, letting AB D 0, the product of two polynomials pD

X

c˛ x ˛ ; q D

j˛jD`

X

dˇ x ˇ

jˇjDm

in the polynomial ring kŒx1 ; : : : ; xn  becomes 0, which is impossible.



Definition 1.2.9. We define a linear map of the vector space g by .X / D X and extend it to an anti-isomorphism of the tensor product T . Since .J /  J , induces an anti-isomorphism ˙ on U.g/, called the principal anti-isomorphism of U.g/. Definition 1.2.10. We denote by S.g/ the symmetric algebra of the vector space g and by S n the set of its homogeneous elements of degree n. Sn being the symmetric group of degree n, the mapping x1 x2   xn 7!

1 X x˛.1/ x˛.2/   x˛.n/ .xj 2 g; 1  j  n/ nŠ ˛2Sn

from S n to Un .g/ gives a bijection of S.g/ onto U.g/, called the symmetrization map.

1.3 Unitary Representations In this section we only touch upon unitary representations of groups; their detailed discussion will be given in the next chapter. In general a representation of a group G means a homomorphism of G to the group of linear automorphisms of a linear space. The first results were obtained by Frobenius at the end of the nineteenth century and

1.3 Unitary Representations

25

were extended, making use of integrals on group manifolds, by Weyl in the 1920s to compact groups. Let M be a space where a group G acts. If we try to develop the analysis on the space M utilizing the action of G, we would be interested in the representations of G. When there exists on M an G-invariant measure , denoting by L2 .M; / the space of all complex-valued functions on M which are square integrable with respect to , we get a unitary representation of G on L2 .M; / by the formula ..g/'/ .m/ D '.g 1 m/ .g 2 G; m 2 M / for any ' 2 L2 .M; /. This construction is in some sense fundamental. For example, letting M D G be a separable locally compact topological group,  a left Haar measure on G, we get the left regular representation of G. The disintegration of this representation would lead to a development from the commutative Fourier analysis to the non-commutative harmonic analysis. Furthermore, H being a closed subgroup of G, G acts on the homogeneous space G=H and by extending this manner of construction we obtain a method to induce a representation of G starting from that of H . It was in the 1940s that the study of these infinite-dimensional unitary representations really began. We only consider locally compact topological groups G satisfying the second countability condition and complex Hilbert spaces H are always supposed to be separable. Let H be a (separable) Hilbert space. A complete orthonormal system fej g1 j D1 of H is called a basis of H. 1 Lemma 1.3.1. Let fxj g1 j D1 and fyk gkD1 be two bases of a Hilbert space H. If we set for a bounded linear operator A of H

Tx .A/ D

1 X

kAxj k2 ; Ty .A/ D

j D1

1 X

kAyk k2 ;

kD1

then Tx .A/ D Tx .A / D Ty .A/. Proof. In fact, Tx .A/ D

1 X j D1

kAxj k2 D

1 1 X X j D1

j.Axj ; yk /j2

kD1

1 1 1 1 X X X @ D j.A yk ; xj /j2 A D kA .yk /k2 D Ty .A / kD1

0

!

j D1

kD1

and in particular Tx .A/ D Tx .A /. Replacing A by A in this relation, Tx .A/ D Tx .A / D Ty .A/. 

26

1 Preliminaries: Lie Groups and Lie Algebras

Definition 1.3.2. Let fej g1 H. A bounded linear j D1 be a basis of a Hilbert space P 2 operator A of H is of Hilbert–Schmidt class if kAk2HS D 1 j D1 kAej k < C1. The preceding lemma confirms that this definition does not depend on the choice of the basis fej g1 j D1 . The number kAkHS is called the Hilbert–Schmidt norm of A. The next lemma is evident from the definition. Lemma 1.3.3. (1) The sum of two operators of Hilbert–Schmidt class is of Hilbert–Schmidt class. (2) The adjoint operator of a Hilbert–Schmidt class operator is of Hilbert–Schmidt class. (3) If A is of Hilbert–Schmidt class and B is any bounded linear operator, AB; BA are of Hilbert–Schmidt class and kABkHS  kBkop kAkHS . Definition 1.3.4. An operator A is of trace class or a nuclear operator if it is written as a finite sum of operators of the form BC , where B; C are of Hilbert–Schmidt class. Lemma 1.3.5. An operator A of trace class is written in the form A D P n  n j D1 cj Aj Aj .cj 2 C/, where fAj gj D1 are Hilbert–Schmidt class operators. Proof. For any bounded linear operators A; B, 4A B D f.A C B/ .A C B/  .A  B/ .A  B/g i f.A C iB/ .A C iB/  .A  iB/ .A  iB/g : Thus, the assertion is clear from the definition.



Definition 1.3.6. Let A be a trace class operator in a Hilbert space H and fej g1 j D1 a P .Ae ; e / is absolutely basis of H. From what we have seen above, the series 1 j j j D1 convergent and its sum does not depend on the choice of the basis fej g1 j D1 . We call this sum the trace of the operator A and denote it by Tr.A/. Definition 1.3.7. A unitary representation of G in H means a homomorphism  of G into the group U.H/ of unitary operators on H such that for any x 2 H the mapping G 3 g 7! .g/.x/ 2 H is continuous relative to the topology of H. The Hilbert space H is called the representation space of , and we agree to denote it by H./ or H . According to the dimension of H./,  is said to be finite- or infinite-dimensional. Definition 1.3.8. An invariant subspace of  means a subspace L of H./ satisfying .g/.L/  L for any g 2 G. When there is no closed invariant subspace other than f0g and H./,  is said to be irreducible. Definition 1.3.9. Let ;  be two unitary representations of G. A bounded linear operator T from H./ to H./ is called a intertwining operator from  to , if it satisfies T ı.g/ D .g/ıT for any g 2 G. We denote by R.; / the vector space of all such operators. When there is as such an intertwining operator which

1.3 Unitary Representations

27

is unitary, we say that  and  are equivalent. To indicate the equivalence we use the notation  ' . The set of all equivalence classes of irreducible unitary O representations of G is called the unitary dual of G and denoted by G. An irreducible unitary representation  of G and its equivalence class are often identified and we write  2 GO etc. By the way, what are standard problems in the representation theory? We may list for example the problems of constructing and decomposing representations, checking the irreducibility and equivalence of representations, computing an explicit intertwining operator between two equivalent representations, analysing particular representation and so on. In order to treat these problems we sketch the direct integral of unitary representations. Two unitary representations ;  of G being given, let H be the direct sum Hilbert space of their spaces. The unitary representation  ˚  on H is defined by the formula . ˚ /.g/.v; w/ D ..g/v; .g/w/; g 2 G; v 2 H./; w 2 H./: We call  ˚  the direct sum of  and . More generally: Definition 1.3.10. Let fi g be a family of unitary representations of G in their P Hilbert spaces fH.i /g. The unitary representation  D i i of G constructed as follows on the Hilbert space H, which is the direct sum of the Hilbert spaces P fH.i /g, is called the direct sum of fi g. The vectors of H D H. i / are i P the sequences v D .vi I vi 2 H.i // such that i kvi k2 < 1 and .g/v D .i .g/vi / .g 2 G/. In particular, when all the i are equivalent to a same repreP sentation , the direct sum i i is also written as n. Here, n 2 f1; 2; 3; : : : ; 1g denotes the number of indices. O Let us extend this sum to integrals. For the details on the Borel structure in G, direct integrals of representations and groups of type I etc., the readers may refer to Mackey’s lecture note [56]. A Borel structure on a set X means the existence of a -algebra defined on X , i.e. a family B of subsets of X , closed under taking the complement and countable unions. A Borel space is a set X equipped with a Borel structure B. The members of B are called Borel sets. The Borel structures we mainly consider here are the -algebras generated by the closed subsets of separable topological spaces and in particular complete separable metric spaces. Let  be a Borel measure on a Borel space X . Let H be a separable Hilbert space and L2 .X; ; H/ the set of all functions f on X with values in H, and satisfying: (1) for R any ' 2 H, .f .x/; '/ is a Borel function of x 2 X , (2) X kf .x/k2 d.x/ < 1. If we identify in L2 .X; ; H/ two functions which coincide almost everywhere with respect to  and define the inner product by

28

1 Preliminaries: Lie Groups and Lie Algebras

Z .f; h/ D

.f .x/; h.x//H d.x/;

X

we get a separable Hilbert space L. Now, let fx I x 2 X g be a field of unitary representations of G such that H .x / D H; .8 x 2 X / and that X 3 x 7! hx .g/ .f .x// ; h.x/i is a measurable function for any g 2 G and f; h 2 L. Then, we get a unitary representation  of G in L by the formula ..g/f / .x/ D x .g/ .f .x// ; g 2 G; f 2 L: This representation  is called the direct integral of fx I x 2 X g and written as R˚  D X x d.x/. The exponential solvable Lie groups G which we will treat later are groups of type I and its unitary representations on separable Hilbert spaces can be uniquely decomposed into a direct integral of irreducible unitary representations. On GO there exists a canonical topology, called the Fell topology, which gives us a canonical O For any unitary representation  of G, there exists a Borel Borel structure on G. O measure  on G and a Borel function m W GO ! N [ f1g such that Z '

˚

GO

m./d./:

We call this the canonical irreducible decomposition of  and the function m the multiplicity function in this decomposition. If there is another canonical irreducible decomposition Z '

˚

GO

m0 ./d0 ./

of , the measures ; 0 are mutually equivalent and the multiplicity functions m; m0 coincide almost everywhere with respect to .

Chapter 2

Haar Measure and Group Algebra

2.1 The Haar Measure of a Locally Compact Group Definition 2.1.1. Let G be a locally compact topological group. We denote by Cc .G/ the vector space of all the continuous complex-valued functions on G with compact support. On this space we put the norm kf k1 WD sup jf .g/j;

g 2 G:

g2G

The completion of Cc .G/ with respect to this norm is the function-space C0 .G/ WD ff W G ! CI f continuous, tending to 0 at 1g: The support of a continuous function defined on a topological space X is by definition the closure supp.f / of the subset fx 2 X I f .x/ ¤ 0g. Definition 2.1.2. We say that a linear functional  W Cc .G/ ! C; is a Borel measure on G, if for every compact subset K of G, there exists a constant CK > 0, such that j.f /j  CK kf k1 for all f with supp.f /  K. A linear functional  on Cc .G/ is said to be positive if .f /  0 for every nonnegative function f 2 Cc .G/. First we remark that such a positive linear functional  maps real-valued functions into real numbers. It is furthermore easy to see that  is in fact a (positive) Borel measure. Indeed, given a compact subset K of G, we can find a non-negative function  in Cc .G/, such that  is equal to 1 on a neighbourhood of K. Hence for every real-valued f 2 Cc .G/ with supp.f /  K, we have that f .g/  kf k1 .g/;

g 2 G;

© Springer Japan 2015 H. Fujiwara, J. Ludwig, Harmonic Analysis on Exponential Solvable Lie Groups, Springer Monographs in Mathematics, DOI 10.1007/978-4-431-55288-8__2

29

30

2 Haar Measure and Group Algebra

and so .kf k1   f /  0, since  is positive. Hence .f /  CK kf k1 , where CK D ./. Replacing f by f , we see that j.f /j  CK kf k1 . Now let f be any element of Cc .G/ supported by K. Multiplying f by complex number ˛ of modulus one, we can suppose that .˛f / is a real number. Let ˛f D a C i b be the decomposition of ˛f into its real part a and its imaginary part b. Then R 3 .˛f / D .a/ C i.b/ implies that .b/ D 0 and so j.f /j D j.˛f /j D j.a/j  CK kak1  CK kf k1 : Hence  is a (positive) Borel measure. By the theorem of Stone (see [44]), there exists on the -algebra generated by the compact subsets of G a unique positive measure d such that Z .f / D

f .g/d.g/;

f 2 Cc .G/:

G

Definition 2.1.3. Let X be a complex vector space. For a function or mapping F W G ! X , set .s/F .t/ WD F .s 1 t/;

s; t 2 G:

We call this action of G on the function-space F .G; X / WD fF W G ! X g the left translation. It is immediately checked that .s/ defines a linear bijection of the vector space F .G; X / and that .st/ D .s/ ı .t/;

.s/1 D .s 1 /;

s; t 2 G:

In particular .s/ leaves the spaces Cc .G/ and C0 .G/ invariant. Definition 2.1.4. We call a Borel measure  on a locally compact group G left-invariant if ..s/f / D .f /; for all s 2 G; f 2 Cc .G/: Proposition 2.1.5. Let  ¤ 0 be a left-invariant positive measure on a locally compact group G. Then .f / > 0 for any non-negative function f 2 Cc .G/ different from 0. Proof. Let 0 ¤ f 2 Cc .G/ be a non-negative function which is annihilated by . There exists then an element u in the support of f and an neighbourhood U of u, such that f .s/  " > 0 for all s 2 U . Let us show that .h/ D 0 for all nonnegative h 2 Cc .G/. Indeed, for such a function h and for every t in a compact neighbourhood K of the support of h, consider the compact neighbourhood Ut WD tU of t. Then the translated function ft WD .t/f is  " on Ut , because for s D tu0 ; u0 2 U , we have that ft .s/ D f .t 1 tu0 / D f .u0 /  ". Since K is compact,

2.1 The Haar Measure of a Locally Compact Group

31

S there exists a finite number of points t1 ; : : : ; tN in K such that N j D1 Utj  K. Then PN the function a WD j D1 ftj has the following property. Take any t 2 K. Since SN K  j D1 Utj , there exists l 2 f1; : : : ; N g such that t 2 Utl and so a.t/ D

N X

ftj .t/  ftl .t/  ":

j D1

Therefore we have for our t that h.t/  khk1  This means that a  But

" h khk1

khk1 khk1 " a.t/: " "

 0 and so .a/ 

" .h/ khk1

 0, since  is positive.

 X X N N N X .a/ D  ftj D ..tj /f / D .f / D 0; j D1

j D1

j D1

because  is left-invariant. Hence .h/ D 0. This implies that .h/ D 0 for every h 2 Cc .G/.  Theorem 2.1.6. Let G be a locally compact group. There exists on G a unique (up to multiplication by a positive constant) left-invariant positive Borel measure (called Haar measure) which we denote by Z .f / WD

f .g/dt;

f 2 Cc .G/:

G

Proof. We shall show only the uniqueness of the Haar measure; a complete proof can be found for instance in [13]. Take two functions f; g 2 Cc .G/. Then it is easy to check that the function Z f g.t/ WD f .s/g.s 1 t/ds; G

is continuous with compact support. Indeed, for t; t 0 2 G, we have that ˇ jf g.t/  f g.t 0 /j D ˇ

Z

Z

ˇ f .s/.g.s 1 t/  g.s 1 t 0 //ds ˇ G

jf .s/jjg.t L 1 s/  g.t L 0

 G

 k.t/gL  .t 0 /gk L 1

1

s/jds

Z jf .s/jds: G

32

2 Haar Measure and Group Algebra

(Here we used the notation FL .s/ WD F .s 1 /;

F 2 F .G; X /:/

Since the function gL is continuous with compact support, it follows that it is uniformly continuous, hence limt 0 !t k.t/g  .t 0 /gk1 D 0. Therefore f g is continuous. Furthermore, if f g.t/ ¤ 0 then necessarily g.s 1 t/ ¤ 0 for at least one s 2 supp.f /. Hence s 1 t 2 supp.g/ and so t 2 supp.f /supp.g/: This shows that supp.f g/  supp.f /supp.g/; since the product of the two compact subsets supp.f / and supp.g/ is compact and hence closed in G. Let us take two left-invariant Borel measures on G, denoted by R d1 g and d2 g. We choose an element h 2 Cc .G/ such that G h.g/d2 g D 1. Then for any f 2 Cc .G/, we have that Z

Z G

h.g/d2 g D

f .s/d1 s G

G

Z Z



1

D

 f .g 1 s/d1 s h.g/d2 g

Z Z

Z

f .s/d1 s D

G

G

Z

f .g s/h.g/d2 g d1 s D G

G

1

Z



f .g / G

h.sg/d1 s d2 g G

by the left invariance of d1 s; d2 g and Fubini’s theorem. Hence Z

Z f .s/d1 s D G

where 'h1 .s/ WD

R

G

f .s 1 /'h1 .s/d2 s;

h.gs/d1 g; s 2 G. Choosing a function q 2 Cc .G/ such that

G

Z q.g/d1 g D 1; G

we get in a similar way that Z

Z f .s/d2 s D G

where 'q2 .s/ WD shows that:

R G

G

f .s 1 /'q2 .s/d1 s;

q.gs/d2 g; s 2 G. Combining the two identities above, this Z

Z f .s/d1 s D G

G

f .s/'h1 .s 1 /'q2 .s/d1 s;

2.1 The Haar Measure of a Locally Compact Group

33

for all f 2 Cc .G/. Therefore 'h1 .s 1 /'q2 .s/ D 1 for all h; q 2 Cc .G/ such that Z Z h.g/d2 g D 1; q.g/d1 g D 1: R

G

G

Fixing q  0 with G q.g/d1 g D 1 and letting s D e, we see that, for every R h 2 Cc .G/ with G h.g/d2 .g/ ¤ 0, Z h.g/ 1 R d1 g D R : h.s/d s q.g/d 2 2g G G G Hence

Z

Z

Z

h.g/d2 g D c G

h.g/d1 g; where c WD G

q.t/d2 t > 0; G

by Proposition 2.1.5. This shows that d2 g D cd1 g.



Definition 2.1.7. RLet G be a locally compact group. For every x 2 G, the linear functional ' 7! G '.gx 1 /dg is positive and left translation invariant. Hence it defines a Haar measure and so by Theorem 2.1.6, there exists a positive number denoted by G .x/ such that Z

'.gx 1 /dg D G .x/

Z

G

'.g/dg; ' 2 Cc .G/: G

We call the positive function x 7! G .x/ on G the modular function. Proposition 2.1.8. Let G be a locally compact group. The modular function is a continuous homomorphism of G into the multiplicative group of the strictly positive real numbers. Proof. We have for any non-negative nonzero element ' 2 Cc .G/, that R G .x/ D

GR '.gx

1

/dg ; x 2 G; G '.g/dg

and this relation tells us that G is a continuous function. Furthermore, for x; y 2 G; ' 2 Cc .G/, we have that Z

Z '.g/dg D

G .xy/ G

1

Z

'.g.xy/ /dg D G

'..gy 1 /x 1 /dg

G

Z

'.gx 1 /dg D G .y/ G .x/

D G .y/ G

Z '.g/dg: G



34

2 Haar Measure and Group Algebra

Proposition 2.1.9. Let G be a locally compact group. Then for any f 2 Cc .G/ we have that Z Z f .g 1 / G .g 1 /dg D f .g/dg: G

G

R

Proof. The integral f 7! G f .g 1 / G .g 1 /dg is left-invariant since for any t2G Z Z 1 1 .t/f .g / G .g /dg D f .t 1 g 1 / G .g 1 /dg G

Z

G

f ..gt/1 / G ..gt/1 / G .t/dg

D G

D G .t 1 /

Z

f .g 1 / G .g 1 / G .t/dg

G

Z

f .g 1 / G .g 1 /dg:

D G

Hence, by the uniqueness of Haar measure, there exists c > 0, such that Z

f .g 1 / G .g 1 /dg D c

G

Z f .g/dg; f 2 Cc .G/: G

Let us show that c D 1. If we take f 2 Cc .G/; f  0, such that f .g/ D f .g 1 / for all g 2 G (for instance taking h  0 in Cc .G/ and putting f .g/ D h.g/ C h.g 1 /), then Z

1

Z

1

f .g / G .g /dg D c G

f .g/dg Z

G

f .g 1 /dg D c

Dc G

f .g 1 / G .g/ G .g 1 /dg G

Z Dc

Z

1

f .g/ G .g /dg D c

2 G

Z

f .g 1 / G .g 1 /dg:

2 G

Therefore c 2 D 1.



Example 2.1.10. 1. Let G be any group. We give it the discrete topology. This means that every subset of G is open and closed. The Haar measure is the counting measure: Z f .g/dg D G

X g2G

f .g/; f 2 Cc .G/:

2.1 The Haar Measure of a Locally Compact Group

35

2. Let G D Gln .R/; n 2 N . Then G is an open subset of the vector space Mn .R/. The Haar measure is given by: Z

Z

f .v/jdet.v/jn d v:

f .g/dg D G

Mn .R/

Indeed, for any u 2 G, we have that Z

Z

f .uv/jdet.v/jn d v

f .ug/dg D G

Z

Mn .R/

j det.u/jn f .uv/j det.uv/jn d v

D Z

Mn .R/

f .v/j det.v/jn d v D

D

Z

Mn .R/

f .g/dg; f 2 Cc .G/: G

3. Let G D exp.g/ be a connected Lie group for which the exponential mapping is a diffeomorphism. We can then transfer the multiplication in the group G to the vector space g and we obtain what we can call a vector group, i.e. a finite-dimensional real vector space V space endowed with an analytic group multiplication  such that .sX /  .tX / D .s C t/X; 8s; t 2 R; X 2 V: In particular 0 is the identity element in V and for every X 2 V its inverse X 1 is the opposite vector X . Furthermore, we can identify the Lie algebra g of V with the vector space V and then the exponential mapping is just the identity mapping from V to V , since for any X 2 V the curve t 7! tX is a homomorphism from the group .R; C/ to the group .V; /. Of course in our case we can take V D g and X  Y D log.exp X  exp Y /; X; Y 2 g. Let us denote as before by LX W V ! V the left multiplication in V . Then for any X; Y 2 V we have that LX ı LY D LX Y and so for the differentials dLX we have that dLX Y .U / D dLX .LY .U // ı dLY .U /; U 2 V:

(2.1.1)

Let for X 2 V j.X / WD j det.dLX .0//j1 be the inverse of the absolute value of the determinant of the differential dLX in 0.

36

2 Haar Measure and Group Algebra

The Haar measure on the group .V; / is then given by the integral Z f .X /j.X /dX; f 2 Cc .V /;

(2.1.2)

V

dX denoting Lebesgue measure on V . Indeed for f 2 Cc .V /, X 2 V : Z Z f .X  Y /j.Y /d u D f .X  Y /j.X 1  .X  Y //d Y Z

V

V

f .Y /j.X 1  Y /d Y j det.dLX 1 .Y //jd Y

D V

.change of variables Y ! LX 1 Y / Z D f .Y /jdet.dLX 1 Y ..0//j1 j det.dLX 1 .Y //jd Y Z

V

f .Y /jdet.dLX 1 .Y //j1 j det.dLY .0//j1 j det.dLX 1 .Y //jd Y

D V

.by(2.1.1)/ Z D f .Y /j.Y /d Y: V

Remark 2.1.11. For a Lie group G we have that G .g/ D j det Ad.g/j1 .g 2 G/: Let us show this identity for vector groups .V; /. First we remark that the function j W V ! RC is conjugation invariant. This is so because the integral Z f 7! f .U /j.X  U  X 1 /d U V

Z D

f .U /j.Ad.X /U /d U D j det.Ad.X //j1

V

Z

f .Ad.X 1 .U ///j.U /d U V

is also left-invariant. Hence there exists for X 2 V a positive number c.X / such that Z Z c.X / f .U /j.U /d U D f .X /j.X  U  X 1 /dX; 8f 2 Cc .V /: V

G

Whence c.X /j.U / D j.X  U  X 1 / for all U 2 V . In particular for U D 0 we have that j.0/ D 1 and so we get c.X / D c.X /j.0/ D j.X  0  X 1 / D j.0/:

2.2 The Group Algebra

37

This means that c.X / D 1 for all X 2 V . For f 2 Cc .V / we then have that Z

Z

f .U  X 1 /j.U /d u

f .U /j.U /d U D

G .X / V

Z

V

Z

V

Z

V

f .X  U  X 1 /j.U /d u

D D

f .Ad.X /.U //j.U /d u

D

f .Ad.X /.U //j.Ad.X /U /d u V

D j det.Ad.X //j1

Z f .U /j.U /d U; V

i.e. G .X / D j det.Ad.X //j1 .

2.2 The Group Algebra Definition 2.2.1. Let G be a locally compact group. For 1  p < 1, we denote by Lp .G/ the space of all measurable (with respect to Haar measure) complex-valued functions f W G ! C such that Z kf kpp WD

jf .g/jp dg < 1: G

Let f; g 2 L1 .G/. Since Z Z G

 Z jf .t/g.t 1 s/jds dt D G

Z

jf .t/g.s/jdsdt GG

D

Z

jf .t/jdt G

jg.s/jdt < 1; G

Fubini’s theorem implies that Z Z G

  Z Z jf .t/g.t 1 s/jdt ds D jf .t/g.t 1 s/jds dt < 1 G

G

G

and so again by Fubini’s theorem, there exists a measurable subset F .f; g/ in G whose complement has Haar measure 0, such that the integral Z

f .t/g.t 1 s/dt G

38

2 Haar Measure and Group Algebra

exists for every s 2 F .f; g/ and the function f g is defined by this integral, namely  0; if s is not in F .f; g/I R : f g.s/ D 1 s/ds; if s is in F .f; g/ G f .t/g.t It follows from the above that kf gk1  kf k1 kgk1 : It is easy to check that for f; g; h 2 Cc .G/, we have that f .g h/ D .f g/ h: This implies that .L1 .G/; / is an associative Banach algebra which has also an isometric involution f 7! f  , namely f  .s/ D G .s 1 /f .s 1 /; s 2 G: The mapping f 7! f  has the following properties: it is anti-linear, .f  / D f and .f g/ D g  f  for every f; g 2 L1 .G/. Definition 2.2.2. Let s 2 G and f 2 L1 .G/. Then we define the right translation .s/ on f by .s/f .t/ WD G .s/f .ts/; t 2 G: It follows from the definition that k.s/f k1 D kf k1 : Proposition 2.2.3. Let f; g 2 L1 .G/; s 2 G. We then have that f ..s/g/ D ..s 1 /f / g; .s/.f g/ D ..s/f / g: Proof. Indeed, for t 2 G, we have that Z

1

f ..s/g/.t/ D

Z

f .u/g.s 1 u1 t/d u

f .u/.s/g.u t/d u D G

D G .s 1 /

Z

f .us 1 /g.u1 t/d u D ..s 1 /f / g.t/: G

Z

f .s 1 u/g.u1 t/d u

..s/f / g.t/ D Z

G

G

f .u/g.u1 s 1 t/d u D .s/.f g/.t/:

D



G

We denote by V the set of all compact neighbourhoods of the neutral element of G.

2.2 The Group Algebra

39

Proposition 2.2.4. Let f 2 Lp .G/; 1  p < 1. Then limx!y .x/f D .y/f . Proof. Let " > 0 and let f 2 Lp .G/, 1  p < 1. We choose h 2 Cc .G/ such that kf  hkp < ". Since h is uniformly continuous, there exists a neighbourhood V 2 V such that k.y/h  hk1 <

"p jV supp.h/j

for all y 2 V . Here Z jV supp.h/j D

dg: V supp.h/

Hence for v 2 V Z k.yv/h 

.y/hkpp

D

j.yv/h.t/  .y/h.t/jp dt G

Z

 k.yv/h  .y/hk1

dt yV supp.h/

D k.v/h  hk1 jV supp.h/j  "p : Therefore k.yv/f  .y/f kp  k.yv/h  .y/hkp C k.yv/f  .yv/hkp C k.y/f  .y/hkp  3"; v 2 V:



Corollary 2.2.5. For every f; g 2 L1 .G/, we have that Z f g D

f .t/.t/gdt G

R Proof. Indeed, if f; g 2 Cc .G/ then the integral G f .t/.t/gdt converges in Cc .G/ and so if we evaluate this integral at a point s 2 G, it follows that Z

Z

Z

f .t/.t/gdt.s/ D G

f .t/g.t 1 s/dt D f g.s/:

f .t/.t/g.s/dt D G

G

R

1 Hence G f .t/.t/gdt D f g. Since Cc .G/ R is dense in L .G/ and since 1 translation is continuous in L .G/, it follows that G f .t/.t/gdt D f g for every f; g 2 L1 .G/. 

Corollary 2.2.6. P Let f; g 2 L1 .G/. Then f g can be approximated in L1 .G/ by sums of the form m j D1 cj .tj /g, where the constants cj are in C and the tj are certain elements in G.

40

2 Haar Measure and Group Algebra

Definition 2.2.7. Let A be a Banach algebra. We say that A possesses a (left) bounded approximate identity, if there exists a net .ei /i 2I in A such that limi !1 ei a D a for every a 2 A and such that supi 2I kei k < 1. Proposition 2.2.8. Let G be a locally compact group. Then L1 .G/ has a bounded approximate identity. Proof. As before, let V be the set of all compact neighbourhoods of the neutral element. We order V by inclusion, i.e. U  V , V  U for U; V 2 V. Choose for R V 2 V a non-negative function fV 2 Cc .G/ with support in V , such that G fV .t/dt D 1. We obtain in this fashion a net .fV /V 2V which is bounded by 1. Now let f 2 L1 .G/. Then for V 2 V we have that Z

ˇ ˇ

kfV f  f k1 D Z

G

Z

G

ˇ ˇ

D

ˇ ˇ

D G

Z

ˇ fV .s/f .s 1 t/ds  f .t/ˇdt

Z

G

Z

G

Z Z

fV .s/f .s 1 t/ds 

Z D

ˇ fV .s/dsf .t/ˇdt G

ˇ fV .s/.f .s 1 t/  f .t//ds ˇdt G

fV .s/jf .s 1 t/  f .t/jdtds

 G

Z

G

fV .s/k.s/f  f k1 ds G

by Fubini’s theorem. Choose now a neighbourhood V0 of e such that k.s/ f  f k1  " for any s in V0 . Then, for V  V0 ; V 2 V, we have that Z

Z

kfV f  f k1 

fV .s/k.s/f  f k1 ds  " V

fV .s/ds D ":



V

2.3 Representations Definition 2.3.1. Let G be a locally compact group. A representation of G on a Banach space X is a strongly continuous homomorphism .; X / of the group G into the group Gl.X / of the bounded invertible linear operators on X . Strongly continuous means that the mappings G ! X;

s 7! .s/x .8x 2 X /

are continuous. The representation .; X / is called bounded, if C WD supx2G k.x/kop < 1.

2.3 Representations

41

The representation .; H/ of G is called unitary, if H is a Hilbert space and for if all g 2 G, .g/ is a unitary operator, i.e. .g/1 D .g/ . Here .g/ denotes the adjoint of .g/. Similarly, let A be an involutive Banach algebra. A representation  of A on a Banach space X is a bounded algebra homomorphism of A into the algebra B.X / of all bounded linear operators of X . The representation .; X / is called non-degenerate if the subspace generated by the subset f.f /xI f 2 L1 .G/; x 2 X g is dense in X . The representation .; H/ is called unitary, if H is a Hilbert space and if .a/ D .a / for all a 2 A. Remark 2.3.2. Let .; X / be a bounded representation of a locally compact group G. For f 2 Cc .G/; x 2 X , the mappingR g ! f .g/.g/x is continuous with compact support, and so R the integral G f .g/.g/xdg converges in X . Furthermore, the mapping x ! G f .g/.g/xdg is obviously linear and Z  Z    f .g/.g/xdg   jf .g/jk.g/xkX ds   G G X Z jf .s/jds D C kxkX kf k1 :  C kxkX G

R Hence the operator .f / WD G f .g/.g/dg is bounded by C kf k1 . 1 .f / WD R Since Cc .G/ is dense in L .G/, it follows that the integral 1 G f .g/.g/dg converges in the strong topology for every f in L .G/ and k.f /kop  C kf k1 ; 8f 2 L1 .G/: Hence, for f 2 L1 .G/; s 2 G, we have that Z

Z

f .s 1 t/.t/dt

.s/f .t/.t/dt D

..s/f / D G

Z D

G

Z f .t/.st/dt D

G

f .t/.s/ ı .t/dt D .s/ ı .f /: G

Furthermore, for f; g 2 L1 .G/, we have Z



.f g/ D 

f .s/.s/gds G

Z

Z D

f .s/..s/g/ds G

f .s/.s/ ı .g/ds D .f / ı .g/:

D G

42

2 Haar Measure and Group Algebra

Theorem 2.3.3. Let G be a locally compact group. 1. Let .; X / be a bounded representation of G. There exists a unique bounded and non-degenerate representation .; X / of the algebra L1 .G/ such that ..s/f / D .s/ ı .f / for all f 2 L1 .G/; s 2 G. 2. Let .; X / be a non-degenerate representation of L1 .G/ and let C be the bound of . Then thereR exists a unique bounded representation .; X / of G on X such that .f / D G f .s/.s/ds for all f 2 L1 .G/ and which is furthermore bounded by C . Proof. 1. The existence of the representation  of L1 .G/ has been shown in Remark 2.3.2. Suppose that we have a second non-degenerate representation . 0 ; X / of L1 .G/, such that .s/ ı  0 .f / D  0 ..s/f / for all s and f . Then for a bounded approximate identity .ei /i of L1 .G/, we have  Z Z  0 .f ei/ D  0 f .s/.s/ei ds D f .s/ 0 ..s/ei /ds G

Z

G

f .s/.s/ ı  0 .ei /ds D

D G

Z

f .s/.s/ds ı  0 .ei /D.f / ı  0 .ei /: G

Hence, since  0 is non-degenerate, limi  0 .ei / D IX strongly and so  0 .f / D lim  0 .f ei / D lim .f / ı  0 .ei / D .f / ı lim  0 .ei / D .f /: i

i

i

2. Let .; X / be a non-degenerate representation of the group algebra L1 .G/. Given s 2 G and a finite family . i /i of vectors in X resp. of elements .fi /i of L1 .G/ such that X .fi / i D 0; i

let us show that also X

..s/fi / i D 0:

i

Indeed, it follows from the above that for every f 2 L1 .G/ 0 D .f /

X

! .fi / i

i

D

X

.f fi / i :

i

Hence for our approximate identity .ej /j , we obtain the relation 0D

X i

...s/ej / fi / i D

X i

j !1

..s/.ej fi // i !

X i

..s/.fi // i :

2.3 Representations

43

This identity allows us to define .s/, s 2 G, on the vector space X 1 WD .Cc .G//X by the formula .s/

X

 .fi / i

WD

X

i

...s/fi / i /:

i

Let us show that k.s/ kX  C k kX for every 2 X 1 . Let  2 X 1 and let " > 0. We take an element f in our bounded approximate identity defined in Proposition 2.2.8 such that k.f /..s//  .s/kX < ": Then for s 2 G, by Proposition 2.2.3 k.s/./kX  k.f /..s//kX C k.f /..s//  .s/kX D k..s 1 /f /./kX C k.f /..s//  .s/kX  C k.s 1 /f k1 kkX C "  C kkX C ": Hence k.s/kX  C kkX . We can now extend the linear operator .s/ to the whole space X , since  is non-degenerate. Since .s/..f / / D ..s/f / for every 2 X; s 2 G, the mappings s 7! .s/ . 2 X / are all continuous and we obtain in this way a representation .; X / of G on the Banach space X which is bounded by the constant C . Furthermore, for f; f 0 2 Cc .G/;  2 X , we have that 0

0

Z

 f .u/.u/f d u  0

.f /..f // D .f f / D  G

Z

f .u/..u/f 0 /d u D

D G

Z

f .u/.u/.f 0 /d u: G

This shows that the representation  of G is unique and that the representation  of L1 .G/ is the integrated version of the representation  of the group G.  Remark 2.3.4. For simplicity of notation, we shallRalways write in the following  instead of  for the integrated representation G f .x/.x/dx of a bounded representation .; X / of G. Definition 2.3.5. Let .; X / be a representation of a locally compact group G. Let Y  X be a subspace of X . We say that Y is invariant or G-invariant, if .s/ 2 Y; for all s 2 G and 2 Y: We say that a representation .; X / of the group G on the Banach space is irreducible, if f0g and X are the only closed invariant subspaces of X .

44

2 Haar Measure and Group Algebra

An element of X is said to be cyclic, if the subspace X spanned by the vectors .g/ ; g 2 G is dense in X . Corollary 2.3.6. 1. Let .; X / be a representation of a locally compact group G. Then a closed subspace Y of X is invariant, if and only if Y is L1 .G/-invariant, i.e. if and only if for every f 2 L1 .G/; 2 Y we have that .f / 2 Y . 2. A representation .; X / of G on a Banach space X is irreducible, if and only if it is irreducible for the representation  of the algebra L1 .G/, i.e. if and only if the only closed L1 .G/-invariant subspaces of X are the trivial ones, f0g and X . Proof. 1. Let Y be a closed G-invariant subspace of X and let ' 2 X 0 be a bounded linear functional which annihilates Y . Then, for every 2 Y; f 2 L1 .G/ ˝ h'; .f / i D ';

Z

˛ f .s/.s/ ds D G

Z

Z f .s/h'; .s/ ids D G

f .s/0ds D 0: G

Hence by the Hahn–Banach theorem, .f / 2 Y . Suppose that Y is L1 .G/invariant. Then for every  2 Y; f 2 L1 .G/ and s 2 G, we have that .f /..s// D ..s 1 /f / 2 Y . Hence .s/ D limV !feg .fV /..s// 2 Y , since Y is closed. 2. This is an immediate consequence of 1.  Theorem 2.3.7 (Schur’s Lemma). Let .; H/ be a unitary representation of a locally compact group G. The following conditions are equivalent: 1.  is irreducible. 2. Every nonzero vector in H is cyclic. 3. Every bounded linear endomorphism of H, which commutes with all the operators .g/; g 2 G, is a scalar multiple of the identity. Proof. 1 ” 2. Suppose that  is irreducible. Let 2 H be nonzero. Then the subspace H spanned by the vectors .g/ ; g 2 G, is G-invariant and hence dense in H, since  is irreducible. Hence is a cyclic vector. If every nonzero vector in H is cyclic, then any nonzero G-invariant closed subspace F of H equals H, since it contains cyclic vectors. 1 ” 3 Suppose that  is irreducible. Let u be a bounded endomorphism which commutes with the operators .g/; g 2 G. Then its adjoint u also commutes with them and so does v WD 12 .u C u / and w WD 2i1 .u  u /. Hence if we show that every self-adjoint linear operator commuting with all the .g/’s is a scalar multiple of the identity, then so is our u D v C i w. We can now assume that u is self-adjoint. By the spectral theorem, we have the spectral decomposition of u: Z uD

dP spectrum.u/

and we know that all the projections P commute with the operators .g/; g 2 G. Hence the kernels and the images of the P ’ s are closed G-invariant subspaces,

2.3 Representations

45

hence either trivial or the whole space and so P is either the identity operator or the zero operator. It then follows that u D 0 Id for some real number 0 . If every bounded endomorphism of H, which commutes with the .g/’s, is a multiple of the identity, then, given a closed G-invariant subspace F of H, the orthogonal projection P onto F commutes with the operators .g/; g 2 G, and so P D Id for some  2 R. But then  D 1 or 0, since P 2 D P and so F D H or F D f0g. Hence  is irreducible.  Definition 2.3.8. Let .; H/ and . 0 ; H0 / be two representations of a locally compact group G. We say that a bounded linear operator U W H ! H0 is an intertwining operator for  and  0 or intertwines  and  0 , if U ı .g/ D  0 .g/ıU for all g 2 G. We denote by BLG .H; H0 / the set of all intertwining operators for  and  0 . Definition 2.3.9. Two unitary representations .; H/ and . 0 ; H0 / are said to be equivalent if there exists in BLG .H; H0 / an invertible isometry. Proposition 2.3.10. Let .; H/ and . 0 ; H0 / be two irreducible unitary representations of a locally compact group G. Then BLG .H; H0 / is one-dimensional if  and  0 are equivalent and equal to f0g if they are not. Proof. Let a; b 2 BLG .H; H0 /. Then b  ıa commutes with  and so b  ıa is a multiple of the identity by Schur’s lemma (Theorem 2.3.7). In particular a ıa D cIdH for some c > 0 and similarly aıa D dIdH0 for some d > 0. This shows that a is a scalar multiple of a unitary operator from H to H0 , and that every other b 2 BLG .H; H0 / is then a scalar multiple of a if a ¤ 0.  Definition 2.3.11. Let G be a locally compact group. We define the dual space or spectrum GO of G to be the space of equivalence classes of irreducible unitary representations of G. Similarly for any involutive Banach algebra A we define the dual space or spectrum AO to be the space of equivalence classes of irreducible unitary representations of A. Definition 2.3.12. Let .; H/ be a unitary representation of a locally compact group G. Let 2 H. The function c  .g/ WD h.g/ ; i; g 2 G; is called a coefficient of the representation . We denote by E./ the set of the coefficients fc  I 2 H; k k D 1g. Proposition 2.3.13. Let .; H/ and . 0 ; H0 / be two irreducible unitary representations of a locally compact group G. Then the two representations  and  0 are equivalent if and only if E./ \ E. 0 / ¤ ;.

46

2 Haar Measure and Group Algebra

Proof. If  and  0 are equivalent, let U W H ! H0 be a unitary intertwining operator. Choose 2 H of norm 1. Let 0 WD U. /. Then 0

c 0 .g/ D h 0 .g/ 0 ; 0 i D h 0 .g/U. /; U. /i D hU  ı  0 .g/ıU. /; i D h.g/ ; i D c  .g/; g 2 G:

(2.3.1)

0

If now c  D c 0 , then we have that k.f / k2H D h.f / ; .f / i D h.f / ı.f / ; i D h.f  f / ; i Z Z D f  .u/f .u1 v/h.v/ ; id ud v Z Z

G

G

G

Z Z

G

G

G

D

f  .u/f .u1 v/c  .v/d vd u 0

D

f  .u/f .u1 v/c 0 .v/d vd u D k 0 .f / 0 k2H0 ; f 2 L1 .G/:

We can now obtain a linear operator U W .L1 .G/ / ! H0 by first defining U..f // WD  0 .f / 0 for f 2 L1 .G/, then using the preceding identity, we can extend this linear mapping to an isometry from H into H0 , since is a cyclic vector. This linear mapping intertwines obviously the representations  and  0 and therefore has dense image (as  0 is irreducible), i.e. is unitary.  Definition 2.3.14. Let A be an algebra. A two-sided ideal K of A is called prime if for any two-sided ideals I; J of A, for which IJ  K, we have that either I  K or J  K. Proposition 2.3.15. Let A be a Banach algebra and .; V / be a bounded topologically irreducible representation of A on a Banach space V . Then the kernel K WD ker of  is a closed prime ideal of A. Proof. Let I; J be two two-sided ideals of A such that IJ  K. Suppose that neither I nor J is contained in K. Then there exists a vector v in V , such that W WD .J /v ¤ f0g. Since W is an A-invariant subspace of V (as J is a two-sided ideal) it follows that W is dense in V . Hence .I /W ¤ f0g, because I is also not in the kernel of . But then .IJ /v D .I /W ¤ f0g and so IJ is not contained in the kernel of .  Definition 2.3.16. Let .; X / be a bounded representation of an algebra A on a Banach space X . We denote by F the two-sided ideal of A consisting of all the P elements a 2 A, for which .a/ is of finite rank. Let Xf i n D a2F Im..a//. The elements of Xf i n are then of the form

D

m X j D1

.aj / j ; a1 ; : : : ; am 2 F ; 1 ; : : : ; m 2 X:

2.3 Representations

47

Theorem 2.3.17. Let .; X / be an irreducible bounded representation of an algebra A on Banach space X . Suppose that the ideal F is different from f0g. Then the sub-module .; Xf i n / of A is simple. Proof. We must show that every element 2 Xf i n ; ¤ 0 is cyclic. Let  D P .a j /j 2 Xf i n . Since .aj /X D .aj /.A/ as .A/ is dense in H ( j being irreducible) and as the image of .aj / is finite-dimensional for every j , it follows Pthat .aj /j D .aj /.bj / for some bj 2 A and for every j . Hence  D . j .aj bj // 2 .A/ and is therefore a cyclic vector.  Definition 2.3.18. We say that an involutive Banach algebra .A; k  kA / is a C  algebra, if ku ukA D kuk2A ; u 2 A: We can associate to every locally compact group its C  -algebra C  .G/ which is defined to be the completion of L1 .G/ with respect to the norm kf kC  WD sup k.f /kop ; f 2 L1 .G/: 2GO

We now define the Fourier transform on a C  -algebra A in the following way. We need first to define the algebra of bounded operator fields defined over the spectrum AO of A. Choose for every 2 AO a representative . ; H / of the O of all bounded operator fields equivalence class and consider the algebra l 1 .A/ .. / 2 B.H / 2AO /; kk WD sup k. /kop < 1:

2AO

O is now defined by The Fourier transform F W A ! l 1 .A/ F .a/. / D a.

O / WD  .a/; a 2 A: We obtain in this way for every a 2 A a field of operators .a.

O // 2AO . This field is bounded: sup 2AO ka.

O /kop D kakA . Proposition 2.3.19. Let A be a commutative C  -algebra with unit. For every a D a 2 A, the spectrum .a/ is contained in R. In particular for every character  W A ! C, we have that .a / D .a/; a 2 A. Proof. Since for every t 2 R we have that 1 D k1A kA D ke i t a e i t a kA D ke i t a k2A ; we see that ke i t a kA D 1 for all t 2 R. Let  be a character of A. Then the relation je i t.a/ j D j.e i t a /j  ke i t a kA D 1

48

2 Haar Measure and Group Algebra

tells us that .a/ must be a real number. Hence for every a 2 A, .a C a / is in R and also .i.a  a // is in R. This tells us that .a / D .a/. We know from the general theory of Banach algebras that the spectrum of an element a 2 A is also the set of the values of all the characters of A on a. Hence .a/  R.  Proposition 2.3.20. Let A be an abelian C  -algebra. O of the algebra of 1. A is isomorphic as C  -algebra with the algebra C0 .A/ continuous functions vanishing at infinity defined on the dual space AO of A. 2. Let A D C0 .X / for some locally compact space X . Then AO is homeomorphic with X , the homeomorphism being given by X 3 x 7! ıx , where ıx denotes point evaluation at x. O i.e. S D f 2 AI O .I / D 3. Let I be a closed ideal of A. Let S be the hull of I in A, f0gg. Then I D fa 2 AI aO D 0 on S g. Proof. 1. The dual space AO of A consists of the unitary characters  W A ! C by Schur’s lemma, i.e. AO is the set of all characters of A by Proposition 2.3.19. Hence AO is contained in the unit ball of the dual space A0 of the bounded linear functionals on A. We equip AO with the weak-star topology .A0 ; A/. If A has a unit, then AO is a compact topological space, if not AO [ f0g is a compact subset of O is A0 . Hence AO is locally compact. The mapping a ! aI O a./ O D h; ai;  2 A, O The image of A in C0 .A/ O is an involutive then a homomorphism of A into C0 .A/. O Hence this subalgebra is dense in subalgebra which separates the points of A. O by the Stone–Weierstrass theorem. Let us show that kak C0 .A/ O 1 D kakA for every a 2 A. For a D a 2 A we have by the spectral radius theorem, that 1=2

k

1=4

1=2k

kakA D ka2 kA D ka4 kA D    D ka2 kA

! .a/ (as k ! 1/;

where .a/ D sup2spect rum.a/ jj denotes the spectral radius of a. Hence kakA D sup j.a/j D kak O 1: 2AO

If a is any element of A, we have that

b

O 21 : kak2A D ka akA D ka ak1 D kak Hence the Fourier transform a 7! aO is an isometry and so A is isometrically O isomorphic to C0 .A/. 2. The mapping X 3 x ! ıx 2 AO is clearly injective. It is also continuous. If a net .xi /i converges in X to some x1 , we have for any ' 2 C0 .X / that limhıxi ; 'i D lim '.xi / D '.x1 / D hıx1 ; 'i: i

O Hence limi ıxi D ıx1 in A.

i

2.3 Representations

49

Let us show that this mapping is also surjective. Suppose that there exists a homomorphism  W A ! C, which is not of the form ıx ; x 2 X . Then ker 6 kerıx for any x 2 X . Hence for every x 2 X , there exists 'x 2 ker such that 'x .x/ ¤ 0. If we replace 'x by 'x 'x , then we can even suppose that 'x is non-negative and multiplying it with a convenient scalar, we can even assume that 'x .x/ D 1; x 2 X . Now let 2 Cc .X / and let K WD supp. /. For every x 2 K, there exists a neighbourhood Ux of x, such that 'x .y/ > 12 for every y 2 Ux . Since K is P compact, we find a finite subset F of K such that S 1 U  K. Let h WD x x2F x2F 'x . Then h 2 ker and h.y/ > 2 for every y 2 K. Choose a continuous function q with compact support on X such that 1 q.y/ D h.y/ ; y 2 K. Then qh D 1 on K and so D qh 2 ker. This tells us that  annihilates every 2 Cc .X /. This contradiction implies that AO D fıx ; x 2 X g. To finish the proof, we have to show that our mapping is open. Let .xi /i be a net which converges in AO to some ıx1 . Let U be a compact neighbourhood of x1 . Suppose that there exists a sub-net .yj WD xij /, such that yj 62 U for every j . Choose a function ' 2 Cc .X / such that '.x1 / D 1; ' D 0 outside U . Then limhıxi ; 'i D hıx1 ; 'i D 1; i

but limj hıyj ; 'i D limj 0 D 0. This contradiction tells us that eventually xi 2 U . Hence our mapping is a homeomorphism. 3. We can assume that A D C0 .X / for some locally compact space X and that S D fx 2 X I f .x/ D 0; for all f 2 I g. Let j.S / D ff 2 AI supp.f / is compact and disjoint from S g: Let us show that j.S /  I . Since j.S / is dense in kerS WD ff 2 AI f .S / D f0gg, by the Stone–Weierstrass theorem, it will follow that kerS D j.S /  I  kerS ) I D kerS: Suppose that there exists f 2 j.S /; f 62 I . Then we can find an element g 2 kerS such that gf D f . For that, it suffices to take g 2 kerS , such that g D 1 on supp.f /. Let J WD fkg  k C i; i 2 I; k 2 Ag: We see that J is a modular ideal of A with modular unit g. If g 2 J , then g D kg  k C i for some k 2 A and some i 2 I . But then f D gf D f .kg  k C i / D k.fg  f / C f i D f i 2 I:

50

2 Haar Measure and Group Algebra

This contradiction tells us that J is a proper modular ideal and so there exists a maximal modular ideal M , which does not contain g (since g is the modular unit) and which contains J . There exists hence a character  of A, i.e. a point  2 X , with .g/ ¤ 0 and .M / D f0g. Since I  M , we see that  2 S . But then g./ D 0. This contradiction tells us that j.S /  I .  Definition 2.3.21. Let A be an abelian locally compact group. Let .; H / be a unitary representation of G. The subset of AO consisting of all characters annihilating O is called the support of  in A. O the kernel of  in C  .A/, i.e. the hull of ker in A, Let C be a closed normal subgroup of the locally compact group G. For every function f on C and g 2 G, let g f be the function on C defined by: g f .c/ WD f .g 1 cg/; c 2 C . For a representation .; X / of C and g 2 G, let .g  ; X / be the representation of C defined by g  .c/ WD .g 1 cg/; g 2 G: Proposition 2.3.22. Let G be a locally compact group and let .; H / be an irreducible unitary representation of G. Let C be a closed abelian normal subgroup of G. Suppose that there exists in CO an open G-invariant subset U , whose complement consists of fixed points for the action of G and which has the following separation property: two distinct G-orbits in U have disjoint open G-invariant neighbourhoods. Then the support of the representation  WD jC is the closure of a G-orbit. Proof. The support S of  is of course not empty, since if it is empty the whole algebra C  .C / would be contained in ker. It is also clear that S is G-invariant, since ker is G-invariant. Suppose that S \ U contains two G-orbits O and O 0 . Take an open G-invariant neighbourhood W of O and an open G-invariant neighbourhood W 0 of O 0 such that W \ W 0 D ;. We choose functions f; f 0 in L1 .C / such that their Fourier transforms fO resp. fO 0 do not vanish identically on O, resp. on O 0 and such that the support of fO, resp. of fb0 is contained in W , resp. in W 0 . This implies that f; f 0 62 ker and that

1

fO  .g f 0 / D 0 ) f .g f 0 / D 0 for all g 2 G: Here g f is the function defined by g

f .u/ WD f .g 1 ug/; u 2 C; g 2 G:

(2.3.2)

2.3 Representations

51

Consider the two ideals J WD L1 .G/ f L1 .G/; J 0 D L1 .G/ f 0 L1 .G/ in L .G/. Then J; J 0 are not contained in ker and J J 0 D f0g  ker, since for '; ; 0 2 L1 .G/, we have by Corollary 2.2.6 and the implication (2.3.2) that 1

' f f 0

0

D ' f .lim D lim

X

X

cj .tj /.f 0

0

//

j

cj ' .f .tj f 0 / G .tj // ..tj /

0

/

j

D lim

X

cj ' .0/ ..tj /

0

/ D 0:

j

Hence J J 0 D f0g  ker. Since ker is a prime ideal, it follows that J or J 0 is contained in ker. This contradiction tells us that S \ U can contain only one G-orbit. Hence S \ U is a G-orbit. Denote the closure of S \ U by SU . If now S ¤ SU then there exists an open G-invariant neighbourhood V of a point s0 2 S such that V \ S \ U D ;. Choose again as above f; f 0 in L1 .C / such that their Fourier transforms fO resp. fb0 do not vanish identically in s0 , resp. on S \ U and such that the support of fO, resp. of fb0 , is contained in V , resp. in U . Then for every g 2 G and s 2 S \ U we have that f g f 0 .s/ D fO.s/fb0 .g  s/ D 0 since fO D 0 on S \ U and if s 2 S n U :

2

2

f g f 0 .s/ D fO.s/fb0 .g  s/ D fO.s/fb0 .s/ D 0; since fb0 D 0 on S n U: and so f2 g f2 2 ker. But then we can conclude that the product J J 0 of the ideals J WD L1 .G/ f L1 .G/; J 0 D L1 .G/ f 0 L1 .G/ is contained in ker, whence f or f 0 2 ker. This contradiction tells us that S D SU and so S is the closure of a G-orbit.  Proposition 2.3.23. Let A be a closed normal subgroup of a locally compact group G. Let .; H/ be an irreducible unitary representation of G. Let  WD jA . If I  L1 .A/ is a G-invariant closed two-sided ideal, which is not contained in ker, then the subspace .I /H is dense in H. Proof. Let K be the closure of .I /H. Since .I / ¤ f0g, we have that K ¤ f0g. For every g 2 G, f 2 I and 2 H, we have that .g/.f / D c.g1 /.g f /.g/ , where for the Haar measure da on A we write d.g1 ag/ D c.g 1 /da; g 2 G. This shows that K is a non-trivial closed G-invariant subspace of H. Hence K D H, since  is irreducible. 

Chapter 3

Induced Representations

3.1 Measures on Quotient Spaces Let G be a locally compact group and H a closed subgroup of G. We are going to show that there exists a G-invariant Borel measure on the quotient space G=H , if and only if for the modular functions H and G we have that H D G jH

(3.1.1)

If this condition is not satisfied, then there exists a unique G-invariant positive linear form on a certain left translation invariant space of continuous complexvalued functions denoted by E.G=H /, which reduces to Cc .G=H / if relation (3.1.1) is satisfied. Definition 3.1.1. Let H be a closed subgroup of a locally compact group G. We define the functions G;H and H;G on H by G;H .h/ WD

G .h/ ; H .h/

H;G .h/ WD

H .h/ ; G .h/

h 2 H:

Remark 3.1.2. When G is a Lie group and H is a subgroup with Lie algebra h, then H;G .exp X / D e Tradg=h X .X 2 h/: This follows from Remark 2.1.11.

© Springer Japan 2015 H. Fujiwara, J. Ludwig, Harmonic Analysis on Exponential Solvable Lie Groups, Springer Monographs in Mathematics, DOI 10.1007/978-4-431-55288-8__3

53

54

3 Induced Representations

Definition 3.1.3. Let H be a closed subgroup of a locally compact group G. Let E.G=H / be defined by E.G=H / D f W G ! CI .gh/ D H;G .h/ .g/;

8g 2 G; h 2 H;

is continuous with compact support modulo H g We remark that the space E.G=H / is left translation and complex conjugation invariant and that the non-negative functions contained in E.G=H / span E.G=H / as a vector space. Proposition 3.1.4. Let H be a closed subgroup of a locally compact group G and p W G ! G=H the canonical projection. The linear mappings ˚ RG=H W Cc .G/ ! E.G=H /; f 7! G 3 x 7!

Z

f .xh/ G;H .h/dh ; H

˚ QG=H W Cc .G/ ! Cc .G=H /; f 7! p.x/ 7!

Z f .xh/dh; x 2 G



(3.1.2)

H

are positive, G-equivariant and surjective. Proof. Let us show first that the functions r WD RG=H f and q WD QG=H f (which are well defined since f 2 Cc .G/) are continuous with compact support modulo H . Indeed, let K be a compact neighbourhood of the support of f . Then for x 2 G the relation r.x/ ¤ 0 implies that f .xh/ ¤ 0 for at least one h 2 H and so xh 2 supp.f /, hence x 2 supp.f /H and h 2 x 1 supp.f /. Therefore the support of r is contained in the compact subset KH of G=H ; similarly for the function q. Furthermore, since for y close enough to x (i.e. for y in a small neighbourhood V x of x, for which V 1 supp.f /  K) we have that H \ y 1 supp.f /  H \ x 1 K and so Z Z RG=H f .y/ D f .yh/ G;H .h/dh D f .yh/ G;H .h/dh: H \y 1 supp.f /

H \x 1 K

It follows that for these y’s Z jRG=H f .y/  RG=H f .x/j 

H \x 1 K

jf .yh/  f .xh/j G;H .h/dh

 k.y 1 /f  .x 1 /f k1

Z H \x 1 K

G;H .h/dh:

This shows that RG=H f is continuous; similarly for QG=H . Let us now show that QG=H f is surjective. Let 2 Cc .G=H /. Choose a compact subset L  G such that supp. /  LH . We take a non-negative ' 2 Cc .G/ such that '.x/ D 1 for all x 2 L. Then the function 'Q WD QG=H '

3.1 Measures on Quotient Spaces

55

is strictly positive on the support of and so the function  WD 'Q is well-defined and continuous on G=H . Let a D .ıp/' 2 Cc .G/, where p W G ! G=H denotes the canonical projection. Then Z

Z

QG;H a.p.x// D

.ıp/.xh/'.xh/dh D .p.x//

'.xh/dh

H

H

D .p.x//'.p.x// Q D

.p.x// '.p.x// Q D '.p.x// Q

.p.x//;

x 2 G:

Hence QG=H is surjective. Now let 2 E.G=H /. Then as before, we choose a compact subset L  G such that supp. /  LH . We take a non-negative ' 2 Cc .G/ such that '.x/ D 1 for all x 2 L. Then the function 'Q WD QG=H ' is strictly positive on the support of and so the function  WD 'Q is well-defined and continuous on G=H . Let a D ' 2 Cc .G/. Then Z

Z

RG;H a.x/ D

.xh/'.xh/ G;H .h/dh D .x/

'.xh/dh

H

H

.x/ '.x/ Q D '.x/ Q

D .x/'.x/ Q D

.x/;

x 2 G: 

Hence RG=H is also surjective.

Corollary 3.1.5. Let A be a closed normal subgroup of a locally compact group G. Then for every ' 2 Cc .G/ we have that Z

Z

Z

'.g/dg D G

'.ga/dad g: P G=A

A

Proof. By the definition (3.1.2) the mapping ˚ ' 7! gP 7! '. Q g/ P WD

Z '.gb/db; g 2 G



A

from CRc .G/ to Cc .G=A/ is surjective and G-equivariant and the linear functional ' 7! G=A '. Q g/d P gP on Cc .G/ is positive and left-invariant. Hence it describes a R  Haar measure G 'dg on G. Proposition 3.1.6. Let H be a closed subgroup of a locally compact group G. There exists a unique (up to multiplication by a positive constant) G-invariant positive linear functional, denoted by I

I

k 7! G;H .k/ D

k.x/dG;H .x/ D G=H

k.x/d x; P G=H

56

3 Induced Representations

on the space E.G; H /. We have that Z

I

Z

 k.th/ G;H .h/dh d tP;

k.t/dt D G

G=H

8k 2 Cc .G/:

(3.1.3)

H

Proof. (a) Let us first prove the following relation. Let f 2 Cc .G/ such that Z RG=H .f /.x/ D

f .xh/ G;H .h/dh D 0 H

for all x 2 G. Then Z f .x/dx D 0: G

Indeed, we have for any

2 Cc .G/, that

Z

Z

 G;H .h/f .xh/dh dx D 0:

.x/ G

H

Hence, Z

Z 0D

.x/ G

 G;H .h/f .xh/dh dx

H

Z Z D



.x/ G;H .h/f .xh/dx dh .by Fubini) H

G

H

G

Z Z D Z D

 1 1 .h/ .xh / .h/f .x/dx dh .x 7! xh1 / G;H G Z

f .x/ Z

G

D

Z

H

Z

H

f .x/ Z

G

D

f .x/ G

1 G .h/ G;H .h/

 .xh /dh dx 1

1 G .h/ 1 H .h/ G;H .h /

.by Fubini)

 .xh/dh dx

.h 7! h1 /

 Z .xh/dh dx D f .x/QG=H . /.x/dx: P

H

G

Since by Proposition 3.1.4 the mapping QG=H W Cc .G/ ! Cc .G=H / is surjective, we can choose a 2 Cc .G/, such that QG=H . /.x/ P D 1 for all x 2 supp.f /H . Hence Z f .x/dx D 0: G

3.1 Measures on Quotient Spaces

57

(b) Let us construct now the positive definite linear functional E.G=H / is of the form

H G=H

k.x/d x. P If k 2

RG=H .f / D k D RG=H .f 0 / for two elements f; f 0 2 Cc .G/, then by (a) Z Z f .x/dx D f 0 .x/dx: G

G

Hence the function Z k 7! f .x/dx

.whenever QG=H .f / D k/

(3.1.4)

G

is well defined on E.G=H /H and gives us a linear functional on this space which we shall denote by G=H d x. P Since RG=H is positive and commutes H with left translation, it follows that G=H d xP is itself positive. Furthermore, for k D RG=H .f / 2 E.G=H / and s 2 G we have that I I .s/k.x/d xP D .s/RG=H .f /.x/d xP G=H

I

G=H

Z

G=H

D

RG=H ..s/f /.x/d xP Z

f .s 1 x/dx D

D

I f .x/dx D

G

G

k.x/d x: P G=H

H Hence G=H d xP is a positive, translation-invariant linear functional on E.G=H /. The uniqueness follows from the fact that any positive, left-invariant linear functional  on E.G=H / defines also a left Haar-measure d x on G through the relation Z f .x/d x WD .RG=H .f //; f 2 Cc .G/ G

By the uniqueness of the Haar measure on G, it follows that d x D cdx for some c > 0. Therefore, since the mapping RG=H is surjective, this implies that for every k D RG=H .f / 2 E.G=H /: Z .k/ D .RG=H .f // D G

Hence  D c tion (3.1.4).

H G=H

Z f .x/d x D c

I f .x/dx D c

G

k.x/d x: P G=H

d x. P Finally, formula (3.1.3) is a consequence of the defini

58

3 Induced Representations

Here is a very useful property for us, the transitivity of the form G;H (cf. [14, Chap. V]) . We consider a closed subgroup K of H . Let us show that I

I G=H

 I G;H .h/'.gh/d hP d gP D H=K

'.g/d g; P ' 2 E.G=K/:

(3.1.5)

G=K

The function 'Q defined by I

P g 2 G .' 2 E.G=K//; G;H .h/'.gh/d h;

'.g/ Q WD H=K

is contained in E.G=H /, since it is obviously continuous with compact support modulo H and since, for h0 2 H , I I 1 0 0 P /D G;H .h/'.gh h/d h D G;H .h0 h/'.gh/d hP '.gh Q H=K

D H;G .h0 /

H=K

I

G;H .h/'.gh/d hP D H;G .h0 /'.g/: Q

H=K

H It follows that the linear functional ' 7! G=H '.g/d Q g; P ' 2 E.G=K/ is positive and obviously G-invariant and so it must be equal to the positive invariant linear H functional ' 7! G=K '.g/d g. P Theorem 3.1.7. Let H be closed subgroup of a locally compact group G. Then the quotient space G=H has a G-invariant Borel measure if and only if G;H D 1: Proof. Let p W G ! G=H denote the canonical projection. If G;HH D 1, then the space E.G=H / is just Cc .G=H / and the positive linear functional G=H d sP is a G-invariant measure on G=H . Suppose now R that there exists a G-invariant measure on G=H . Then the functional f 7! G=H QG=H .f /.x/d P xP defines a G-invariant Borel measure on G. Hence Z  Z Z f .s/ds D f .xh/dh d x; P f 2 Cc .G/: G

G=H

H

2 Cc .G/. Then we easily see that

Now let f;

QG=H . /QG=H .f / D QG=H . QG=H .f /ıp/: Therefore Z

Z f .x/QG=H .x/dx D G

QG=H . /.x/Q P G=H .f /.x/d P xP G=H

(3.1.6)

3.2 Definition of an Induced Representation

59

Z

Z

Z

D

.x/QG=H .f /.x/dx D

.x/

G

G

Z Z G

1 1 f .x/ 1 G .h/ .xh /dx dh .x 7! xh /

Z

Z D

 .xh/ H .h / G .h/dh dx 1

f .x/ G

 .xh/ G;H .h/dh dx

f .x/ Z

G

D

.h ! h1 /

H

Z

Z D

H



D H

 f .xh/dh dx

H

f .x/RG=H . /.x/dx: G

Hence Z

  f .x/ QG=H . /.x/ P dx D

G

Z f .x/RG=H . /.x/dx;

2 Cc .G/;

f;

G

and so Z

Z .xh/dh D H

2 Cc .G/; x 2 G:

.xh/ G;H .h/dh; H

This implies that G;H .h/ D 1 for all h 2 H .



Proposition 3.1.8. Let A be a closed normal subgroup of a locally compact group G. Then G jA D A . Proof. We have by Proposition 3.1.5 that Z

Z

'.ya1 /dy D

'.y/dy D

G .a/ G

Z

G

D

Z G=A

Z

'.gba1 /dbd gP A

Z

'.gb/dbd gP D A .a/

A .a/ G=A

Z

A

for every a 2 A and ' 2 Cc .G/. Hence A D G jA .

'.y/dy; G



3.2 Definition of an Induced Representation Definition 3.2.1. Let H be a closed subgroup of a locally compact group G. Let .; H / be a unitary representation of H . Define the space of mappings E.G=H; /

60

3 Induced Representations

by E.G=H; / WD f W G ! H I .gh/ D H;G .h/.h/ . .g//; 1=2

8g 2 G; h 2 H;

is continuous with compact support modulo H g:

(3.2.1)

We remark that the space E.G=H; / is left translation invariant and that for

;  2 E.G=H; / the function x ! h .x/; .x/i DW q ; .x/; x 2 G is continuous with compact support modulo H and satisfies the relation q ; .xh/ D h .xh/; .xh/i D H;G .h/h.h/ . .x//; .h/ ..xh//i D H;G .h/q ; .x/; x 2 G; h 2 H; and so q ; 2 E.G=H /. We can thus define a scalar product on E.G=H; / by I h ; iind WD

h .x/; .x/id xP G=H

and a norm k kind WD

q

h ; iind :

The left invariance of the linear functional

H G=H

d xP tells us that

h.x/ ; .x/iind D h ; iind ; for all x 2 G;

;  2 E.G=H; /:

We can construct explicit elements in E.G=H; / through the mapping RG=H; W Cc .G/  H ! E.G=H; /; ˚ .f; u/ ! x !

Z H

1=2 f .xh/ G;H .h/.h/udh; x 2 G :

It is easy to verify that RG=H; .f; u/ is effectively in the space E.G=H; / whenever f 2 Cc .G/ and u 2 H and that the mapping G 3 t ! RG=H; .f; u/.t/ 2 H is continuous. We set:

3.2 Definition of an Induced Representation

61

Definition 3.2.2. L2 .G=H; / WD E.G=H; /

kkind

:

Since the left translation is isometric on E.G=H; /, we obtain an isometric action of G on the Hilbert space L2 .G=H; /. We denote this action by ;H WD  D indG H , where ;H .t/ .s/ WD .t 1 s/; 2 L2 .G=H; /; s; t 2 G:

(3.2.2)

Since obviously for 2 E.G=H; / lim k.t/  k1 D 0;

t !e

it follows also that lim k.t/  kind D 0:

t !e

Hence the mapping t !  .t/ from G into L2 .G=H; / is continuous and so  is a unitary representation of G. We say that  is an induced representation of G. In order to construct representations of a group from those of subgroups, this method is utilized very often. Especially representations induced from unitary characters, i.e. one-dimensional unitary representation, are called monomial representations. When every irreducible representation is equivalent to a monomial representation, the group G is said to be monomial. Exponential solvable Lie groups which we introduce later are monomial [10, 75], but in general solvable Lie groups are not necessarily monomial. Proposition 3.2.3. Let .; H/ be a unitary representation of a closed subgroup H of a locally compact group G. Let f 2 Cc .G/. Then  .f / is a kernel operator with a kernel function f W G  G ! B.H/ given by the formula f .s; t/ D G .t

1

Z

f .sht 1 / G;H .h/.h/dh 2 B.H/; s; t 2 G: 1=2

/ H

Proof. Let 2 L2 .G=H; / such that the function g 7! jf .g/ .g 1 x/j is in L1 .G/ for every x 2 G (for instance if f 2 Cc .G/ and 2 E.G=H; /) and let s 2 G. Then Z Z 1  .f / .s/ D f .t/ .t s/dt D G .t 1 /f .st 1 / .t/dt I

G

G

Z

G ..th/1 /f .s.th/1 / .th/ G;H .h/dhd tP

D G=H

H

.by Eq. (3.1.3)/

62

3 Induced Representations

I

Z

G .t 1 /f .s.th/1 / .th/ H .h1 /dhd tP

D I

G=H

Z

H

I

G=H

G .t 1 /f .sh1 t 1 /.h1 / H;G .h/ .t/ H .h1 /dhd tP 1=2

D H

Z

D I

G=H

H

 1=2 G .t 1 /f .sht 1 / G;H .h/.h/dh .t/d tP

f .s; t/ .t/d tP:

D G=H

Lebesgue’s theorem of dominated convergence tells us that the mapping .s; t/ ! f .s; t/ is continuous on G  G if f 2 Cc .G/.  Definition 3.2.4. Let us denote by CB.G=H / the space of all complex-valued continuous and bounded functions on G=H . We equip this space with the infinity norm k  k1 and we obtain a commutative C  -algebra which contains the space C0 .G=H / of the continuous functions on G=H which vanish at infinity as C  subalgebra. Let  2 CB.G=H /. Then for any 2 E.G=H; / we have that the mapping  , where  .t/ WD .t/ .t/;

t 2 G;

is also an element of E.G=H; / and so E.G=H; / is an CB.G=H / module. Furthermore we have the following estimate k kind  kk1 k kind : Hence we can extend the module action of CB.G=H / to L2 .G=H; /. Let us put M ./ WD  ;

 2 CB.G=H /; 2 L2 .G=H; /:

We obtain in this way a -representation M of CB.G=H / on the Hilbert space L2 .G=H; /. Let us see how the representations M of CB.G=H / and  are linked together. For 2 E.G=H; /;  2 CB.G=H / and s; t 2 G we have that  .s/.M ./ /.t/ D .s 1 t/ .s 1 t/ D ..s//.t/. .s/ /.t/ D M ..s//. .s/ /.t/: This shows that M ..s// D  .s/ıM ./ ı  .s/1 ; This is Mackey’s imprimitivity relation.

s 2 G;  2 CB.G=H /:

(3.2.3)

3.2 Definition of an Induced Representation

63

Proposition 3.2.5. Let  be a unitary representation of a closed subgroup H of a locally compact group G. Let f 2 Cc .G/;  2 L2 .G=H; / and let WD  .f /. Then the mapping G 3 t 7! .t/ 2 H is continuous. Proof. We have for s; t 2 G that Z

f .u/.u1 s/d u 

.s/  .t/ D Z

G

Z

G

Z

f .u/.u1 t/d u G

.f .su/  f .tu//.u1 /d u

D

G .u1 /.f .su1 /  f .tu1 //.u/d u:

D G

Hence  k .s/  .t/k2 D  Z

Z G

2 G .u1 /.f .su1 /  f .tu1 //.u/d u

G .u1 /jf .su1 /  f .tu1 /jk.u/k d u

 G



Z

G

2G .u1 /jf .su1 /  f .tu1 /jd u

2



 Z jf .su1 /  f .tu1 /jk.u/k2 d u .by Cauchy–Schwartz/  G

Z  G

2G .u1 /jf .su1 /  f .tu1 /jd u Z

I

1

1

jf .s.uh/ /  f .t.uh/ /j G;H .h/k.h/

 G=H

H

1

 ..u//k2 dh

.by Eq. (3.1.3)/ Z D 2G .u1 /jf .su1 /  f .tu1 /jd u G

Z

I

 G=H

k..u//k2

 jf .s.uh/1 /  f .t.uh/1 /j G;H .h/dh d uP : H

Since the functions Z

jf .s.uh/1 /  f .t.uh/1 /j G;H .h/dh

.s; t; u/ ! H

d uP

64

3 Induced Representations

and Z .s; t/ ! G

2G .u1 /jf .su1 /  f .tu1 /jd u

are continuous, it follows that lim k .s/  .t/k D 0:



s!t

Proposition 3.2.6. Let  be a unitary representation of a closed subgroup H of a locally compact group G. Let K be a closed subgroup of G containing H . Let f 2 Cc .G/;  2 L2 .G=H; / and let WD  .f /. Then the mapping 1=2 Q 2 L2 .K=H; / G 3 t 7! .K 3 k 7! G;K .k/ .tk// DW .t/

is continuous. Proof. Let  D indK H . We have for s; t 2 G; k 2 K that Z

1

.sk/  .tk/ D

Z

f .u/.u1 tk/d u

f .u/.u sk/d u  Z

G

Z

G

G

.f .su/  f .tu//.u1 k/d u

D D G

1 1 1 G .u/.f .su /  f .tu //.uk/d u:

Hence Q  .t Q /k2 D k .s/  Z

I  K=H

I 

G

Z

K=H

Z

 G

Z  G

G

I

 

K=H

Z G

2 1=2 1 1 /  f .t u1 // G;K .k/.uk/d u  d kP G .u/.f .su

2 1=2 1 1 1 .u/jf .su /  f .t u /j .k/k.uk/k d u d kP  G G;K  1 1 /  f .t u1 /jd u G .u/jf .su

1 1 /  f .t v1 /j G;K .k/k.vk/k2 d vd kP .by Cauchy-Schwartz/ G .v/jf .sv

j 1 G .u/g.u/jd u

I

I

Z

 K=H

G=H

H

2 1 P kP G ..uh//jg.uh/j G;H .h/ G;K .k/k.uhk/k G;H .h/dhd ud

3.2 Definition of an Induced Representation

65

where g.u/ WD gs;t .u/ WD f .t u1 /  f .su1 /; u; s; t 2 G .by Eq. (3.1.3)/ Z D j 1 G .u/g.u/jd u G

I

Z

I

 Z  G

G=H

K=H

G;K .k/k..uk//k2

H

 1 ..uh// .h/jg.uh/jdh d ud P kP K;H G

jg.u1 /jd ukgk Q 1 kk22 ;

where Z

K;H .h/ G .h1 u1 /jg.uh/jdh; u 2 G:

g.u/ Q WD K

Since the function Z .s; t/ !

jf .su/  f .tu/jd u G

is continuous, f being in Cc .G/, it follows that Q  .t/k Q lim k .s/  D 0:

s!t



Proposition 3.2.7. The image of the mapping RG=H; is total in L2 .G=H; /. Proof. Suppose that  2 L2 .G=H; / is orthogonal to the closed subspace L0 generated by the image of the mapping RG=H; . Since this mapping is G-equivariant in the first variable, its image is G-invariant, hence for every f 2 L1 .G/, the vector 1  .f / is also in L? 0 . Let .hi /i 2I be a bounded approximate identity in L .G/ consisting of continuous functions with compact support. Let i WD  .hi / 2 L? 0 ; i 2 I: Then limi  .hi / D  and for every i , the mapping G 3 t 7!  .t/i is continuous by Proposition 3.2.5. Therefore we can assume that  is continuous. If now  W G=H ! C is a continuous bounded function, we have by Definition 3.2.4 and the definition of RG=H; that for every 2 L0 , the mapping  is also in L0 and so I 0 D h;  iind D .t/h.t/; .t/i d tP: G=H

66

3 Induced Representations

Hence we must have that h.t/; .t/i D 0 for all t 2 G: In particular, taking D RG=H; .f; u/ (where f 2 Cc .G/ and u 2 H ), this gives us Z 1=2 0D f .th/ G;H .h/h.t/; .h/uidh; 8t 2 G: H

Hence h.t/; .h/ui D 0 for all u 2 H and all t 2 G. Finally, .t/ D 0 for all t 2 G.  Theorem 3.2.8 (Induction in Stages). Let K  H  G be two closed subgroups of a locally compact group G. Let .; H / be a unitary representation of K. Then G H the representations indG K  and indH .indK / are unitarily equivalent. Proof. Let E.G=H; E.H=K; // be the space E.G=H; E.H=K; // D f W G ! E.H=K; /;  is continuous with compact support modulo H; .gh/ D H;G .h/.h1 /..g//; 8g 2 G; h 2 H; and there exists 1=2

a compact subset L  G; such that supp .g/  g 1 L K \ H 8g 2 Gg; where  WD indH K . The mapping U W E.G=K; / ! E.G=H; E.H=K; //; 1=2

U.'/.g/.h/ WD G;H .h/'.gh/; g 2 G; h 2 H; is then well defined, i.e. U.'/ is in the space E.G=H; E.H=K; //, if ' is in E.G=K; /, since ' is continuous with compact support modulo K, hence U.'/ is continuous with compact support modulo H and U.'/.g/ has its support in LH for some compact subset L of G. Furthermore U.'/.g/.hk/ D G;H .hk/'.ghk/ D G;H .hk/ K;G .k/.k 1 /.'.gh// 1=2

1=2

1=2

D K;H .k/.k 1 /U.'/.g/.h/; 8g 2 G; h 2 H; k 2 K: 1=2

Hence U.'/.g/ 2 E.H=K; / for all g 2 G: Also, for h; h0 2 H; g 2 G, we have that U.'/.gh/.h0 / D G;H .h0 /'.ghh0 / D G;H .hh0 / H;G .h/'.ghh0 / 1=2

1=2

1=2

D H;G .h/U.'/.g/.hh0 / D H;G .h/.h1 /U.'/.g/.h0 /: 1=2

1=2

3.2 Definition of an Induced Representation

67

Hence U.'/ is contained in E.G=H; E.H=K; //. The inverse mapping V W E.G=H; E.H=K; // ! E.G=K; / is given by V ./.g/ WD .g/.e/; g 2 G: Indeed, for g 2 G, k 2 K, we have that V .gk/ D .gk/.e/ D H;G .k/.k 1 /.g/.e/ 1=2

D H;G .k/.g/.k/ D K;G .k/.k 1 /.g/ 1=2

1=2

and V ./ is obviously continuous and its support is contained in the set L K. This shows that V ./ is indeed in E.G=K; /. Furthermore, for  2 E.G=H; E.H=K; //; g 2 G; h 2 H , we have that 1=2

1=2

U ıV ./.g/.h/ D G;H .h/V ./.gh/ D G;H .h/.gh/.e/ 1=2

1=2

D G;H .h/ H;G .h/.g/.h/ D .g/.h/: Similarly for ' 2 E.G=K; /; g 2 G, we have that 1=2

V ıU.'/.g/ D U.'/.g/.e/ D G;H .e/'.ge/ D '.g/: Hence U ıV D Id and V ıU D Id . The mapping U is obviously G-equivariant and by Eq. (3.1.5) it is also isometric, since I 2 kU.'/kind.ind/ D kU.'/.g/k2ind d gP I

G=H

I

D I

G=H

D G=K

H=K



G;H .h/k'.gh/k2H d hP

d gP

k'.g/k2H d gP D k'k2ind :

Therefore U is a unitary intertwining operator.



Proposition 3.2.9. Let G be a locally compact group, let H be a closed subgroup of G and let A be a closed normal subgroup of G contained in H . Let GQ D G=A, HQ D H=A and let .; V / be a unitary representation of H , which is trivial on A.

68

3 Induced Representations

Let  W G ! G=A be the canonical projection. Denote by Q the representation of HQ which is defined through the relation ı Q D . Then the unitary representations GQ  and  Q WD ind ı Q of G are equivalent.  WD indG H HQ Q HQ ; / Proof. The two spaces E.G=H; / and E.G= Q are isomorphic, an isomorphism Q HQ ; / U W E.G=H; / ! E.G= Q being given by U .gA/ WD .g/; g 2 G; 2 E.G=H; /;

(3.2.4)

since G;H .a/ D G;H .a/ D A;H .a/ D 1, by Proposition 3.1.8 and since A is a normal subgroup of G. By the covariance relation we have that

.ga/ D H;G .a/.a1 / .g/ D .g/; g 2 G; a 2 H: 1=2

The spaces E.G=H / and E..G=A/=.H=A// are for the same reason G-isomorphic too through a map denoted also by U and therefore I

I

PQ ' 2 E.G=H /: U'.g/d Q g;

'.g/d gP D G=H

.G=A/=.H=A/

This shows that the left representation of G on the spaces L2 .G=H; / and Q Q A; Q / Q i.e. indG ı Q and indG  L2 .G= H  are equivalent. HQ Proposition 3.2.10. Let  D indG H  be an induced representation of a locally compact group G. If  is irreducible, then  must be irreducible too. Proof. If  is not irreducible, then the Hilbert space H of  can be written as the direct orthogonal sum of two non-trivial closed H -invariant subspaces H1 and H2 of H and  D 1 ˚ 2 , where i D jHi ; i D 1; 2. It follows that E.G=H; / is the orthogonal direct sum of the G-invariant subspaces Ei D E.G=H; i /; i D 1; 2. This implies that  is not irreducible. 

3.3 Conjugation of Induced Representations Definition 3.3.1. Let .; H/ be a unitary representation of a locally compact group G. For an automorphism a of G, we write . a ; H/ for the representation  a .s/ WD .a1 .s//; s 2 G: If a D It is the inner automorphism a.s/ D tst 1 ; t 2 G, then we write  t WD  It :

3.3 Conjugation of Induced Representations

69

Let H be a closed subgroup of the locally compact group G and let .; H / be a unitary representation of H . Then .a ; H / is the representation of the group H a WD a.H / defined by a .h/ WD .a1 .h//; h 2 H a : Proposition 3.3.2. Let H be a closed subgroup of a locally compact group G and let .; H / be a unitary representation of H . For every automorphism a of G we have that .;H /a is equivalent to a ;H a . Proof. The Haar measure of the group H a can be written in the following way: Z

Z

'.a.h0 //dh0 ; ' 2 Cc .H a /;

'.h/dh D Ha

H

since the mapping Cc .H a / ! Cc .H /, ' 7! 'ıa DW U.'/ is a positive linear 1 a bijection and since R U..s/'/ D .a .s//U.'/a for every s 2 H and so the linear functional ' 7! H '.a.h//dh is positive and H -invariant, hence is a Haar measure on H a . In particular, we have that H a .a.h// D H .h/ for every h 2 H;

(3.3.1)

because Z Z '.hs 1 /dh D '.a.h0 /s 1 /dh0 Ha

Z

H 0

D

1

'.a.h /a.a .s H

D H .a1 .s//

Z

1

0

Z

'.a.h0 a1 .s 1 ///dh0

///dh D H

'.a.h0 //dh0 D H .a1 .s//

Z '.h/dh: Ha

H

In a similar way, we have that the invariant “measure” on G=.H a / is given by I

I '.g/d gP D G=H a

'.a.g//d g: P

(3.3.2)

G=H

This follows from the fact that the mapping ' ! 'ıa is a positive linear bijection of E.G=H a / onto E.G=H /. Let us show that the mapping U W E.G=H a ; a / ! E.G=H; /I ' 7! 'ıa defines a unitary operator which intertwines both representations. First we observe that U jE.G=H a ;a / is a linear bijection onto E.G=H; /, since for ' 2 E.G=H a ; a /, we have that

70

3 Induced Representations

U'.gh/ D '.a.g/a.h// D H a ;G .a.h//a .a.h//1 '.a.s// 1=2

D H;G .h/.h/1 '.a.s// 1=2

.by the definition of a and (3.3.1)/ D H;G .h/.h/1 U'.g/: 1=2

This shows that U.'/ satisfies the covariance condition of the definition (3.2.1). Furthermore, since obviously U' is continuous and its support is compact modulo H , hence U' is an element of E.G=H; /. Furthermore, for ' 2 E.G=H a ; a /, we have by (3.3.2) that I kU.'/k2ind D

G=H

k'.a.t//k2H d tP D

I G=H a

k'.t/k2Ha d tP D k'k2inda

and for t; s 2 G; ' 2 E.G=H a ; a /: U.a .t/'/.s/ D a .t/'.a.s// D '.t 1 a.s// D '.a..a1 .t//1 s// D . /a .t/U'.s/: Hence a is equivalent to the representation . /a .



Let us give a criterion for the irreducibility of an induced representation. This criterion will be used in Sect. 5.3. Theorem 3.3.3. Let H be a closed subgroup of a connected locally compact group G. Let .; H / be an irreducible representation of H . Let C be a closed normal subgroup of G contained in the centre of H . Let  W C ! T be the unitary character of C defined through , i.e. .c/ D .c/IdH ; c 2 C . Suppose that the G-orbit of  in CO is locally closed and that the stabilizer G D fg 2 GI g D g of  is equal to H . Then the unitary representation  WD indG H  is irreducible. Proof. Since the G-orbit O of  is locally closed, we can write O D A \ U for some open subset U and some closed subset A of CO . Hence O  A and so O  O \ U  A \ U D O ) O D O \ U: Since G is connected, it is the union of an increasing sequence of compact sets and so we know from the general theory of locally compact groups, since O is locally closed, hence locally compact, that the quotient space G=G D G=H is homeomorphic to O. Such a homeomorphism is given by the mapping G=H 3 gH 7! g ; g 2 G:

3.3 Conjugation of Induced Representations

71

Now let 2 L1 .C / and let us compute jC . /. Let  2 E.G=H; / and g 2 G. We then have that Z Z 1 jC . /.g/ D .c/.c g/dc D .c/.g.g 1 c 1 g//dc Z

C

C

.c/.g 1 cg/.g/dc D O .g /.g/:

D C

Hence jC . / is the multiplication operator with the continuous bounded function m .gH / WD O .g /; gH 2 G=H . If we choose such that O has a compact support which is contained in U , then the function m has compact support in G=H and the collection I of all these functions m is a subalgebra of C0 .G=H / which separates the points in G=H and which is complex conjugation invariant. Hence I is dense in C0 .G=H / in the uniform norm. Let us show now that  is irreducible. Let 0 ¤ 0 be an element in L2 .G=H; / and let H0 WD .Cc .G//0 . We must show that H0? D f0g. Let ' 0 2 H0? . Let f 2 Cc .G/ run through an approximate identity and let ' WD .f /' 0 ;  WD .f /0 . The elements  and ' of H D L2 .G=H; / have the property that the mappings G ! H I g 7! .g/, g ! '.g/ are continuous (see Proposition 3.2.6). We shall show that ' D 0. This will imply then that ' 0 D 0. We have for all u; v 2 G, 2 L1 .C / that I 0 D hjC . /.u/; .v/'i D G=H

m .g/h.u1 g/; '.v1 g/iH d g: P

It follows that 0 D h.u1 g/; '.v1 g/iH 8u; v; g 2 G: Hence, in particular, if we replace u by gu1 and v by gv1 , then we obtain the relation 0 D h.h/..u//; '.v/iH 8u; v 2 G; h 2 H: This shows that, using the fact that  is irreducible, that '.v/ D 0 for all v 2 G,  since the element .u/ 2 H is not 0 for all u 2 G. Proposition 3.3.4. Let H be a closed subgroup of a connected locally compact group G. Let .; H / and . 0 ; H 0 / be two unitary representations of H and let G 0 0  WD indG H ;  WD indH  . 1. Let u W H ! H 0 be a bounded intertwining operator for  and  0 . Then the mapping Uu W H ! H 0 defined by Uu ./.g/ WD u..g//; g 2 G;  2 E.G=H; /; is a bounded intertwining operator for  and  0 .

72

3 Induced Representations

2. Let U W H ! H 0 be a bounded intertwining operator for  and  0 . Suppose that U intertwines also the representations M resp. M 0 of C0 .G=H / on H resp. on H 0 . Then there exists a bounded intertwining operator u for  and  0 such that U D Uu : Proof. 1. Take 2 E.G=H; /. Then the mapping g 7! u. .g// 2 H 0 is continuous with compact support modulo H and for g 2 G; h 2 H , we have that u. .gh// D u. H;G .h/.h1 /. .g/// D H;G .h/ 0 .h1 /u. .g//; 1=2

1=2

since u 2 BLH .H ; H 0 /. Hence Uu . / is an element of H 0 . Furthermore I I kUu . /k2 0 D kUu . /.g/k20 d gP D ku. .g//k20 d gP G=H

G=H

I

 kuk2op

G=H

k .g//k2 d gP D kuk2op k k2 :

This shows that Uu is bounded by kukop . For t; g 2 G we have that Uu ..t/ /.g/ D u..t/ .g// D u. .t 1 g// D  0 .t/u ı .g/ D  0 .t/.Uu /.g/: Hence Uu is an element of BLG .H ; H 0 /. 2. In order to find the operator u 2 BLH .H ; H 0 /, let H0 , resp. H 00 be the span of the vectors .f / ; 2 H ; f 2 Cc .G/, resp.  0 .f / ; 2 H 0 ; f 2 Cc .G/. Then every element  of H0 0 has the property that the mapping G 3 g 7!  .g/ 2 H 0 is continuous. Since U maps H0 into H0 0 , we can evaluate U./,  2 H0 , at every g 2 G. We observe first that kU. /.g/k 0  kU kop k .g/k for every 2 H0 ; g 2 G. Indeed, we have that for every ' 2 Cc .G=H / I j'.t/j2 kU. /.t/k20 d tP D kM .'/.U. //k2 0 D kU.M .'/ /k2 0 G=H



I

kU k2op kM .'/. /k2

D

kU k2op

G=H

j'.t/k2 k .t/k2 d tP:

Hence, for every g 2 G, we must have that kU. /.g/k 0  kU kop k .g/k :

(3.3.3)

In particular, whenever for two elements  and 0 in H0 , we have that .g/ D 0 .g/, then necessarily U./.g/ D U.0 /.g/. This allows us to define a linear operator u from the subspace H0 D f .e/I 2 H0 g into H 0 by letting u.v/ WD U. /.e/; whenever v D .e/ for some 2 H0 :

(3.3.4)

3.4 The Imprimitivity Theorem

73

It follows from (3.3.3) that u is bounded by kU kop . We extend u to a bounded operator on the whole space H . It follows from its definition that U. /.g/ D  0 .g 1 /U. /.e/ D U..g 1 / /.e/ D u..g 1 / .e// D u. .g//; for every 2 H0 and g 2 G. Hence U D Uu . We still have to show that u intertwines  and  0 . For v D .e/ 2 H0 , h 2 H , we have that u..h/v/ D u..h/ .e// D u. H;G .h/ .h1 // 1=2

D H;G .h/U .h1 / 1=2

D  0 .h/.U .e// D  0 .h/.u.v//: This shows that u 2 BLH .H ; H 0 /.



3.4 The Imprimitivity Theorem Definition 3.4.1. Let H be a closed subgroup of a locally compact group G. Let .; H / be a unitary representation of G and let .M; H / be a non-degenerate unitary representation of C0 .G=H / on the same Hilbert space H . We say that the pair .; M / is a system of imprimitivity, if the following relation holds: M..s// D .s/ıM./ ı .s/1 ;

s 2 G;  2 C0 .G=H /:

Relation (3.2.3) tells us that for every unitary representation  of H , the pair . ; M / is a system of imprimitivity. Theorem 3.4.2 (Mackey’s Imprimitivity Theorem). Let .; H / be a unitary representation of a locally compact group G. If there exists a closed subgroup H of G and a non-degenerate unitary representation M of C0 .G=H / on the space H of  such that the pair .; M / is a system of imprimitivity, then  is unitarily equivalent to a representation which is induced from a unitary representation  of H and the representation M is equivalent to M under the same equivalence. Proof. Let H1 be the Garding space of , i.e. the span of all the vectors of the form ..f / / with f 2 Cc .G/; 2 H . It is immediate that H1 is invariant under G and under Cc .G/. For f 2 Cc .G/, write also fQ D QG=H f . For ;  2 H1 , let ; be the linear functional on Cc .G/ defined by ; .f / D hM.fQ/ ; i ;

f 2 Cc .G/:

74

3 Induced Representations

We now make the following assumption: (A) The subset Hc2 of H  H consisting of all the ’s and ’s for which there exists a continuous function q ; W G ! C, such that Z ; .f / D

f .x/q ; .x/dx;

8f 2 Cc .G/

G

contains H1  H1 . Furthermore, for . ; / 2 H1  H1 the function G  G  G 3 .z; y; t/ ! q.z/ ;.y/ .t/ is continuous. We shall prove assertion (A) at the end of the present proof. The following properties of the function q ; ; . ; / 2 Hc2 , follow immediately from its definition. 1. q ; is linear in and anti-linear in  and q ; D q; . 2. q ; .t/  0 for every t in G, since for Cc .G/ 3 f  0 we have that fQ  0 and so R M.fQ/ is a non-negative operator, whence G f .t/q ; .t/dt D hM.fQ/ ; i  0. 3. q ; .th/ D H;G .h/q ; .t/ for all t 2 G and h 2 H . Indeed, for any f 2 Cc .G/, we have for the right translated function .h1 /f; h 2 H , that ..h1 /f /Q.tP/ D

Z

f .tkh1 /d k D H .h/

Z

H

f .tk/d k D H .h/fQ.tP/; t 2 G:

H

Hence Z G

f .t/q ; .th/dt D 1 G .h/

Z

f .th1 /q ; .t/dt

G

1 D 1 G .h/hM.QG=H ..h /f // ; i

D H .h/ 1 G .h/hM.QG=H .f // ; i Z D f .t/ H;G .h/q ; .t/dt: G

4. qM./ ; .t/ D .tP/q ; .t/; t 2 G;  2 Cc .G=H /. Indeed, for f 2 Cc .G/, we have Z G

f .t/.tP/q ; .t/dt D hM..f /Q/ ; i D hM.fQ/ ; i D hM.fQ/M./ ; i D

Z f .t/qM./ ; .t/dt: G

5. q ; .z1 t/ D q.z/ ;.z/ .t/; 8z; t 2 G.

3.4 The Imprimitivity Theorem

75

Indeed, for f 2 Cc .G/ we have Z

f .t/q ; .z1 t/dt D G

Z

f .zt/q ; .t/dt D hM...z1 /f /Q/ ; i

G

D h.z1 /.M.fQ//.z/. /; i D

Z f .t/q.z/ ;.z/ .t/dt: G

(by the imprimitivity relation) We need now to define a unitary representation of the group H . Let us therefore consider on the vector space H1 the bilinear form h ; i WD q ; .e/;

;  2 H1 :

It follows from properties 1 and 2 of the functions q ; that h; i is a scalar product. We denote by H0 the subspace of H1 of all for which h ; i D 0. By Property 3, the subspace H0 is H -invariant. Let H WD .H1 =H0 /

h;i

:

be the closure of H1 =H0 with respect to the norm obtained from h; i . We let the group H act on H1 =H0 in the following way: .h/. C H0 / WD H;G .h/.h/ C H0 ; h 2 H; 2 H1 : 1=2

We see that k.h/. C H0 /k2 D H;G .h/q.h/ ;.h/ .e/ D H;G .h/q ; .h1 / (by property 5) D H;G .h/ G;H .h/q ; .e/ D k k2 (by property 3): Hence this action is isometric. Furthermore, by the continuity of the functions q ; we have that lim k.h/. C H0 /  .h0 /. C H0 /k2

h!h0

D lim q. 1=2 h!h0

1=2 1=2 1=2 H;G .h/.h/ H;G .h0 /.h0 // ;. H;G .h/.h/ H;G .h0 /.h0 //

1=2

1=2

.e/

D lim .2k C H0 k2  H;G .h/ H;G .h0 /.q.h/ ;.h0 / .e/ C q.h0 / ;.h/ .e/// h!h0

D 2k C H0 k2  2 H;G .h0 /.q.h0 / ;.h0 / .e// D 0: This shows that .; H / is a unitary representation of H .

76

3 Induced Representations

Let us show that  is unitarily equivalent to the induced representation  . We define a mapping U from H1 to the space H in the following way: U. /.t/ WD .t 1 / C H0 ; t 2 G; 2 H1 : We must of course verify that the mapping U W G 3 t ! .t 1 / C H0 is really an element of H . Since .t 1 / 2 H1 for any t 2 G and 2 H1 we have that U .t/ 2 H for every t 2 G and 2 H1 . Furthermore U is continuous. Indeed, for s; t 2 G, we have that kU .t/  U .s/k2 D k.t 1 /  .s 1 / C H0 k2 D q.t 1 / .s 1 / ;.t 1 / .s 1 / .e/ ! 0; if t ! s, by the bilinearity in ;  and the continuity of the function G  G  G 3 .z; y; t/ 7! q.z/ ;.y/ .t/. Let us show that U satisfies the covariance condition. Let t 2 G and h 2 H . Then U .th/ D ..th/1 / C H0 D .h1 /..t 1 / / C H0 D .h1 /.U .t// D H;G .h/.h/1 .U .t//: 1=2

The last line follows from the definition of the representation . Let us show that U is an isometry. Choose 0  f 2 Cc .G/ and 2 H1 . Then we have I I fQ.tP/kU .t/k2 d tP D fQ.tP/q.t 1 / ;.t 1 / .e/d tP I

G=H

I

G=H

Z

G=H

G=H

fQ.tP/q ; .t/d tP .by property 5/

D

Z

D

 f .th/q ; .th/ G;H .h/dh d tP (by property 3)

H

f .t/q ; .t/dt D hM.fQ/ ; i

D G

by Eq. (3.1.3). We now take functions f 2 Cc .G/, such that the fQ’s form a bounded approximate identity in C0 .G=H /. It then follows that M.fHQ/ tends to in H , since the representation M is not degenerate. Therefore also G=H fQ.t/kU .t/k2 d tP H goes to G=H kU .t/k2 d tP; this yields that

3.4 The Imprimitivity Theorem

77

I G=H

kU .t/k2 d tP D k k2

and so U is a linear isometry, which can be extended to an isometry of H into H . Let us show that U intertwines  and  . For in H1 , s; t 2 G we have that  .s/.U. //.t/ D U .s 1 t/ D .t 1 s/ C H0 D .t 1 /..s/ / C H0 D U..s/ /.t/: Hence U ı.s/ D  .s/ıU; for all s 2 G: Let us now show that U intertwines also the representations M and M . Indeed for  2 Cc .G=H /; 2 H1 and t 2 G, we have that U.M./ /.t/ D .t 1 /.M./ / C H0 D .t 1 /ıM./ ı .t/ ı .t 1 / C H0 D M..t 1 // ı .t 1 / C H0 : Now by property 4 for every ;  2 H1 we have that qM..t 1 // ; .e/ D .tP/q ; .e/ D q.Pt / ; .e/; which shows that M..t 1 //.t 1 / C H0 D .tP/.t 1 / C H0 ; namely U ıM./ D M ./ıU;

 2 Cc .G=H /:

(3.4.1)

Let us show that the image of U is dense, which tells us then that U is a unitary operator. Let  2 H , such that  is orthogonal to the image of U . By Proposition 3.2.5, we can assume that the mapping G 3 t ! .t/ 2 H is continuous. Then we have by (3.4.1) for any 2 H1 and  2 C0 .G=H / that I 0 D h; U.M./ /iind D

.tP/h.t/; .t/1 . / C H0 i d tP; G=H

whence we have that 0 D h.t/; .t/1 . / C H0 i for all t 2 G and 2 H1 :

78

3 Induced Representations

Replacing by .t/ we get 0 D h.t/; C H0 i for all t 2 G and 2 H1 : Since H1 mod H0 is dense in H it follows that .t/ D 0 for all t and so  D 0. This shows that U W H ! H is a unitary intertwining operator for  and  . In order to finish the proof of the imprimitivity theorem we must give the following. Proof of Assumption (A): Let f 2 Cc .G/ and a; b 2 Cc .G/; 0 ; 0 2 H and D .a/ 0 ;  D .b/ 0 2 H1 . Then we have Z Z hM.fQ/ ; i D a.t/b.s/hM.fQ/.t/ 0 ; .s/0 i dsdt Z Z

G

G

G

G

a.t/b.s/h.t/M..t 1 /fQ/ 0 ; .s/0 i dsdt

D

(by the imprimitivity relation) Z Z b.s/h.s 1 t/M.a.t/.t 1 /fQ/ 0 ; 0 i dsdt D Z Z

G

G

G

G

D

h.s 1 /M.b.ts/a.t/.t 1 /fQ/ 0 ; 0 i dsdt

Z  Z D

M.b.ts/a.t/.t G

1

 Q /f /dt 0 ; .s/0 ds:

G



For s 2 G, the function Z Z Z b.ts/a.t/.t 1 /fQ.Pu/dt D b.ts/a.t/f .tuh/dhdt; s .u/ WD G

G

u 2 G;

H

can be rewritten as Z Z 1 1 s .u/ D 1 G .uh/b.t.uh/ s/a.t.uh/ /f .t/dhdt Z

G

D

H

Z

f .t/ G

H

 1 s/a.t.uh/1 /dh dt: 1 .st/. b/.t.uh/ G G

Let us put 1 1 cs;t .u/ W D 1 G .st/. G b/.tu s/a.tu / 1 1 1 1 D 1 / /; s; t; u 2 G: G .st/. G b/..ut / s/a..ut

(3.4.2)

Then the functions cs;t .s; t 2 R/ are in Cc .G/ and the mapping G  G 3 .s; t/ 7! cs;t 2 .Cc .G/; kk1 / is continuous. We have still to verify that the mapping GG 3

3.4 The Imprimitivity Theorem

79

.s; t/ 7! cs;t Q 2 Cc .G=H / is continuous with compact support in the variables s and u for fixed t. We can pick a compact subset K of G such that the support of the function G  G 3 .s; u/ 7! b.u1 s/a.u1 / is contained in K  K. Then it follows from (3.4.2) that for fixed t, the function .s; u/ 7! cs;t .u/ has its support in K  Kt. Hence, for s; u; t 2 G the function G  H 3 .s; h/ 7! cs;t .uh/ is supported by K  .u1 Kt \ H / and so Z cs;t Q .u/ D

1 1 1 G .st/. G b/.t.uh/ s/a.t.uh/ /dh;

H \u1 Kt

which shows that the function G  G  G 3 .s; t; u/ 7! cs;t Q .u/ is continuous and its support in .s; u/ (for fixed t) is contained in K  .KtH /. Hence the integral Z s D

f .t/cs;t Q dt G

converges in Cc .G=H / and so Z M.s / D

f .t/M.cs;t Q /dt; G

which yields that hM.fQ/ ; i D

Z

Z f .t/hM.cs;t Q / 0 ; .s/0 i dsdt D G

f .t/q ; .t/dt; G

where Z q ; .t/ WD

hM.cs;t Q / 0 ; .s/0 i ds; t 2 G; G

is continuous in t. A similar argument works for the continuity of the mapping G  G  G 3 .z; y; t/ 7! q.z/ ;.y/ .t/. Indeed, since Z .z/.a/ 0 D

Z

Z

a.s/.z/.s/ 0 ds D G

a.z1 s/.s/ 0 ds;

a.s/.zs/ 0 ds D G

G

we compute as above that for z; y 2 G, f; a; b 2 Cc .G/; 0 ; 0 2 H and D .a/

0 ;  D .b/0 we have as above that hM.fQ/.z/. /; .y/i D

G

Z  Z

a.z1 t/b.y 1 s/hM.fQ/.t/ 0 ; .s/0 i dsdt G

M.b.y 1 ts/a.z1 t/.t 1 /fQ/dt

D G

Z Z

G



0 ; .s/0 ds: 

80

3 Induced Representations

Hence, proceeding as before, we see that Z q.z/ ;.y/ .t/ WD G

A

hM.cz;y;s;t / 0 ; .s/0 i ds; t 2 G;

where

A

cz;y;s;t .u/ D

Z H \u1 Kz1 t

1 1 1 1 1 G .st/. G b/.y t.uh/ s/a.z t.uh/ /dh:



Corollary 3.4.3. Let G be a connected locally compact group and let .; H / be an irreducible unitary representation of G. Let C be a closed abelian normal subgroup of G. Suppose that the spectrum CO of the group C contains an open G-invariant subset U of CO , whose complement in CO consists of G-fixed points, which has the following separation property. Two distinct G-orbits in U have disjoint open Ginvariant neighbourhoods. Then there exist a character  2 S and an irreducible representation 0 of the stabilizer G0 of  in G such that  D indG H 0 and such that 0 jC D IdH0 . Proof. We know from Proposition 2.3.22 that the support S of the representation  WD jC is the closure of a G-orbit O WD G   for a certain unitary character  of C . In particular S \ U D O and so the G-orbit O is locally closed. Let G0 be the stabilizer of  in G. In order to construct a system of imprimitivity, we use the identification of G=G0 with G  . Let C0;0 .S / be the subspace of C0 .S /, consisting of all functions which vanish on the boundary S n G   of S . The mapping C0 .G=G0 / ! C0 .S /I ' 7! '; Q

(3.4.3)

where '.g Q  / WD '.g/; g 2 G, is an isomorphism of C  algebras. Since the kernel of jC in C  .C / is ff 2 C  .C /I fO 0 on S g, we can define a representation  of C0 .S / on H by letting . / WD jC .f /; .fO/jS D :

(3.4.4)

This allows us to define the representation M of C0 .G=G0 / on H through M.'/ WD .'/; Q ' 2 Cc .G=G0 /:

(3.4.5)

Since C0;0 .S / is not contained in the kernel of , it follows from Proposition 2.3.23 that M is not degenerate. The imprimitivity relation also holds. Let u 2 G; ' 2 Cc .G=G0 /. Then there exists f 2 C  .C /, such that fO D 'Q on S and so, letting d.u1 xu/ D ıC .u/dx .x 2 C; u 2 G/, .u/ıM.'/.u1 / D .u/ ı jC .f /.u1 / D ıC .u/jC .u f / D .u fOjS /

3.4 The Imprimitivity Theorem

81

.where u .r/ WD

1 .u r/; r 2 CO ;

2 C0 .CO //

D .u '/ Q D ...u/'/Q/ D M..u/'/: The imprimitivity theorem tells us that  ' indG G0 , for some irreducible representation  of G0 . Let U W H ! L2 .G=G0 ; / be an intertwining operator for the two imprimitivity systems. Then for f 2 Cc .C /, we have that jC .f / D M.'/, where as above 'Q D fOjS . Then for 2 H we get fO./U. /.e/ D '.e/U. /.e/ D U.M.'/ /.e/ D U.jC .f / /.e/ D .indG G0 /.f /.U /.e/ D jC .f /.U /.e/: We deduce from this that jC .f / D fO./IdH . Hence C is in the projective kernel of  and .c/ D .c/IdH ; c 2 C .  Mackey Theory: résumé It is well known that a connected and simply connected solvable Lie group G is obtained starting from R and repeating a semi-direct product by R. Therefore, if we would like to study the unitary representation of G, the Mackey theory we have seen is fundamental. We resume it in such a way that we shall need it later and apply it to typical exponential solvable Lie groups in the next chapter. It will be the objective of Chap. 5 to generalize these examples. Readers are recommended to refer to the lecture note [56]. Let G be a locally compact group satisfying the second countability condition and A a closed commutative normal subgroup of G on which G acts smoothly (for instance if A D C with the properties of Theorem 3.3.3 and Corollary 3.4.3). O Then G acts on the space AO of all unitary characters of A: for g 2 G;  2 A, 1 .g  /.a/ D .g ag/ .a 2 A/ in a nice way and we have the following Theorem 3.4.4 (1). Let G./ be the stabilizer of  2 AO in G. 1. Let  be an irreducible unitary representation of G./ such that its restriction jA to A is a multiple of , i.e. there exists a certain m 2 N [ f1g such that jA D m. Then the induced representation indG G./  is irreducible. 2. Let 1 ; 2 be two irreducible representations of G./ such that 1 jA ; 2 jA G are multiples of . Then indG G./ 1 ' indG./ 2 if and only if 1 ' 2 . O (2) Suppose that the G-orbit G   is for every  2 AO a locally closed set in A. Then any irreducible unitary representation  of G, which is not trivial on A, O is obtained as  ' indG G./  with a certain  2 A and an irreducible unitary representation  of G./ such that jA is a multiple of . (See Proposition 3.3.4, Theorem 3.3.3 and Corollary 3.4.3.)

Chapter 4

Four Exponential Solvable Lie Groups

4.1 The Group Rn The group .Rn ; C/; n 2 N , is the only connected and simply connected abelian Lie group of dimension n. Its irreducible unitary representations are one-dimensional by Schur’s lemma (2.3.7), and define in this way unitary characters  W Rn ! C of Rn . Such a character  is of the form ` for some real-valued linear functional ` on Rn , i.e. .x/ D ` .x/ D e 2 i `.x/ ; x 2 Rn : For ' 2 L1 .Rn /, we obtain the Fourier transform 'O of ', where Z

'.x/e 2 i `.x/ dx; ` 2 .Rn / :

'.`/ O WD Rn

The functions 'O are continuous, tend to 0 at infinity and k'k O 1  k'k1 . Furthermore for '; 2 L1 .Rn /, we have that

1 D 'O  O :

'

Also,

fc D fO; f 2 L1 .Rn /:

© Springer Japan 2015 H. Fujiwara, J. Ludwig, Harmonic Analysis on Exponential Solvable Lie Groups, Springer Monographs in Mathematics, DOI 10.1007/978-4-431-55288-8__4

83

84

4 Four Exponential Solvable Lie Groups

If f 2 L2 .Rn / \ L1 .Rn /, then fO is in L2 .Rn / too and we have Plancherel’s formula kf k2 D kfOk2 : This allows us to define the Fourier transform also for general L2 -functions and this generalized Fourier transform becomes a unitary operator from L2 .Rn / onto L2 .Rn /. Fourier’s inversion formula tells us that for any ' 2 L1 .Rn /, such that 'O is also integrable, we have that Z '.x/ D Rn

OO 2 i `.x/ L '.`/e O d `; x 2 Rn ; i.e. ' D ';

where 'L is defined by '.x/ L D '.x/; x 2 Rn . A particularly interesting subalgebra of L1 .Rn / is the Schwartz algebra Sn consisting of the rapidly decreasing C 1 -functions on Rn , i.e. Sn WD f' W Rn ! C; P @˛ ' 2 L2 .Rn / for every polynomial function P and ˛ 2 Nn g: Here @˛ is the partial differential operator @˛ D

@˛1 @˛n ˛1    @x1 @xn˛n

for ˛ D .˛1    ˛n / 2 Nn . The space Sn is mapped under the Fourier transform onto itself and the Fourier transform is thus an isomorphism of Fréchet spaces. Let us remark that the C  -algebra of L1 .Rn / is the algebra C0 .Rn / of the continuous complex-valued functions on Rn which go to 0 at infinity. The Fourier transform maps L1 .Rn / into C0 .Rn / and the C  -norm kf kC  of f 2 L1 .Rn / is given by kf kC  D sup jfO.y/j D kfOk1 : y2Rn

If now  is a unitary representation of Rn , let S be the support of . Then S is a closed subset of Rn and ker D ff 2 C  .Rn /I fO D 0 on S g. In particular C  .Rn /=ker is isomorphic with C0 .S /, an isomorphism being given by f 7! fOjS . This tells us that for f 2 C  .Rn / k.f /kop D kfOjS k1  kfOk1 :

(4.1.1)

4.2 The Heisenberg Group

85

4.2 The Heisenberg Group Let H1 D exp g, where g D hX; Y; ZiR I ŒX; Y  D Z. The Heisenberg group H1 of dimension 3 is a typical nilpotent Lie group. In matrix form, 0 1 0 1 0 1 010 000 001 X D @0 0 0A ; Y D @0 0 1A ; Z D @0 0 0A ; 000 000 000 80 9 1 < 1x z = H1 D @0 1 y A I x; y; z 2 R : : ; 00 1 More generally, let Hn WD Rn  Rn  R: We equip Hn with the following multiplication:    1 0 .x; y; t/.x 0 ; y 0 ; t 0 / WD x C x 0 ; y C y 0 ; t C t 0 C xy  x 0 y ; 2 where for two elements x D .x1 ; : : : ; xn /; y 0 D .y10 ; : : : ; yn0 / 2 Rn the symbol xy 0 means xy 0 WD

n X

xj yj0 :

j D1

We verify that this multiplication is associative, that the element 0 D .0n ; 0n ; 0/ of Hn is the neutral element and that the inverse of h D .x; y; t/ of Hn is the vector h1 D .x; y; t/ D h. The centre Cn of the group Hn is the subgroup Cn D f.0; 0; t/I t 2 Rg D f0g  f0g  R: We see that for every .x; y; t/ and .u; v; s/ in Hn , we have that Œ.x; y; t/; .u; v; s/ D .x; y; t/.u; v; s/.x; y; t/.u; v; s/ D .0n ; 0n ; xv  yu/: Hence ŒHn ; Hn  D Cn :

86

4 Four Exponential Solvable Lie Groups

The Lie algebra hn of Hn is the real vector space hn D Rn  Rn  R with the bracket Œ.x; y; t/; .x 0 ; y 0 ; t 0 / WD .0n ; 0n ; xy 0  x 0 y/: This follows easily from the fact that 

 ˇ  ˇ d .tx; ty; tz/ sx 0 ; sy 0 ; sz0 ˇt D0 ˇsD0 dt     ˇ  ˇ d d st  0 0 0 0 0 ˇ ˇ xy  x y tx C sx ; ty C sy ; tz C sz C D t D0 sD0 ds dt 2 d ds

1 D .0n ; 0n ; .xy 0  x 0 y//: 2 Hence we can also write the multiplication in Hn as 1 U V D U C V C ŒU; V ; U; V 2 Hn : 2 It follows that U V U 1 D V C ŒU; V ; U; V 2 Hn : The mapping R 3 t ! tU for a fixed element U of Hn is then a one-parameter subgroup of Hn and every such a one-parameter subgroup is of this form. Indeed, for s; t 2 R, U 2 Hn , we have that .sU /.tU / D sU C tU C

st ŒU; U  D sU C tU D .s C t/U: 2

Hence the exponential mapping exp W hn ! Hn is the identity. We take the following basis Z of the Lie algebra hn : Z WD fX1 ; : : : ; Xn ; Y1 ; : : : ; Yn ; Zg; where Xj D ..ıi;j /i ; 0n ; 0/; Yj D .0n ; .ıi;j /i ; 0/; Z D .0n ; 0n ; 1/: This gives us the non-trivial brackets ŒXi ; Yj  D ıi;j Z: We see from the definition of the multiplication in Hn that Lebesgue measure on Hn is left- and right-invariant, hence it is our Haar measure, and Hn is unimodular.

4.2 The Heisenberg Group

87

4.2.1 Induced Representations Let P WD f0g  Rn  R  Hn . Obviously P is a closed, connected and abelian subgroup of Hn . We can consider P also as an abelian subalgebra p of hn and write formally P D exp p. Since P is abelian, every linear functional f of hn defines a unitary character f of P through the rule f .exp U / WD e 2 if .U / ; U 2 p: We can now define the induced representation f WD f of G as in the definition (3.2.2). Let us show that the Hilbert space Hf of f is naturally isomorphic to the Hilbert space L2 .Rn /. Indeed, since Hn and P are unimodular, the homogeneous space Hn =P Rhas a (unique) left invariant Borel measure and it is easy to check that this measure Hn =P d xP is given by Z ' 7! '.U; 0; 0/d U; ' 2 Cc .Hn =P /: Rn

Indeed, for ' 2 Cc .Hn =P /, g D .x; y; t/ 2 Hn we have that   Z Z 1 du '.g 1 .u; 0; 0//d u D ' u  x; y; t C yu 2 Rn Rn    Z 1 1 D du ' .u  x; 0n ; 0/ 0n ; y; t C yu C .u  x/y 2 2 Rn Z Z D '.u  x; 0n ; 0/d u D '.u; 0n; 0/d u: Rn

Rn

In particular the norm of ' is given by Z k'kf D

1=2 j'.u; 0n ; 0/j d u 2

Rn

:

For 2 E.Hn =P; f /, the function R defined on Rn by R .U / WD .U; 0n; 0/; U 2 Rn ; is in Cc .Rn / and Z kR k22

Z

D

j .U; 0n; 0/j d U D 2

Rn

Furthermore, for every

Hn =P

j .x/j2 dx D k k2indf :

2 Cc .Rn /, if we define the function on G by

..U; 0n ; 0/.0; V; t// WD e 2 if .0;V;t / .U /I U; V 2 Rn ; t 2 R;

88

4 Four Exponential Solvable Lie Groups

then is an element of E.Hn =P; f / and R D . This shows that the mapping R is a linear bijection between E.Hn =P; f / and Cc .Rn / and extends to a unitary operator, which we shall also denote by R, between Hf and L2 .Rn /. By transferring the representation f to L2 .Rn /, we obtain a representation f;P D f of Hn on L2 .Rn /. For D R. / 2 L2 .Rn /, g D .X; Y; t/ 2 Hn n and U 2 R we have f .g/ .U / D R.f .g/ /.U / D ..X; Y; t/.U; 0n ; 0//   1 D U  X; Y; t C Y  U 2    1 1 D .U  X; 0n ; 0/ 0n ; Y; t C Y  U C .U  X /Y 2 2    1 D .U  X; 0n ; 0/ 0n ; Y; t C Y  U  X  Y 2 1

D e 2 if .t C 2 X Y Y U / .U  X /:

(4.2.1)

Let us now write  WD hf; Zi. We write f for the linear functional which is zero on all the Xi ’s and all the Yj ’s and which takes the value  on Z. We remark that for .0n ; 0n ; t/ 2 Cn ; f .0n ; 0n ; t/ D e 2 it IdL2 .Rn / :

(4.2.2)

Remark 4.2.1. Let .; H/ be an irreducible unitary representation of Hn . By Schur’s lemma (2.3.7), there exists a unitary character  of the centre Cn , such that .z/ D  .z/IH ; z 2 Cn for some  2 R. Since Cn is isomorphic to the real line, it follows that .tZ/ D e 2 i t ; t 2 R; for some  2 R: We shall write  for this character of Cn . Theorem 4.2.2. Let  2 R . The representation  WD f ;P is irreducible. Every irreducible unitary representation .; H/ of Hn , which restricts to the centre Cn of Hn to a multiple of the character  is equivalent to  . Proof. We shall use Theorem 3.3.3 and Proposition 3.4.3. We describe first the Hn -orbits of a character f ; f 2 hn ; of the normal abelian closed and connected subgroup P of Hn . For g D .u; v; 0/ 2 Hn and p D .0; y; t/ 2 P;  D f .Zn / 2 R; we have that f .p/ D g  f .p/ D f .g 1 pg/ D e 2 if ..u;v;0/.0;y;t /.u;v;0// g

D e 2 iuy e 2 it D Ad  .g/f .p/;

4.2 The Heisenberg Group

89

where Ad  .g/f; f 2 hn is the linear form defined on hn by Ad  .g/f .x; y; t/ D f .g 1 .x; y; t/g/ D f .x; y; t  uy C vx/: Hence identifying the spectrum PO of the abelian group P with the linear dual space of p and then with Rn  R, we see that g

f ' .u; / 2 Rn  R and G  f ' Rn  fg DW O if  ¤ 0; G  f D ff g; if  D 0: Now let U WD ff 2 p ; f .Zn / ¤ 0g. Then U is an open Hn -invariant subset of PO and its complement is the Hn -invariant subset Zn? . Two distinct Hn -orbits O and O0 have disjoint open neighbourhoods V D ff 2 p ; jf .Zn /  j < "g resp. V D ff 2 p ; jf .Zn /  0 j < "g, where 2" WD j  0 j. Furthermore, the stabilizer in Hn of a character f with  D f .Zn / ¤ 0 is obviously the group P itself. n Hence every induced representation f WD indH P f with f .Zn / ¤ 0 is irreducible. Furthermore if .; H/ is an irreducible unitary representation, then by Schur’s lemma it restricts on the centre to a multiple of a character  , where  .0n ; 0n ; t/ D e 2 it ; t 2 R. By Proposition 3.4.3,  restricts to P to the Hn -orbit O and  is equivalent to any representation f ; f 2 p , with f .Zn / D .  Definition 4.2.3. Let SB.L2 .Rn // be the space of bounded linear operators on the Hilbert space L2 .Rn / with Schwartz kernels, which means that every element b of SB.L2 .Rn // is a kernel operator with a kernel function Fb which is a Schwartz function on Rn  Rn . We denote by S.Hn / the space of the rapidly decreasing C 1 -functions on Hn , i.e. S.Hn / WD ff W Hn ! CI f 2 C 1 .Hn /; p@˛ f 2 L2 .Hn /; p is a polynomial function; ˛ 2 Nn g: (It is easy to see that S.Hn / is an involutive Fréchet subalgebra of L1 .Hn /.) Proposition 4.2.4. Let  2 R . For every f 2 S.Hn / the operator  .f / is a smooth bounded linear operator on L2 .Rn / and the mapping S.Hn / 7! SB.L2 .Rn // W f !  .f / is surjective. Proof. For f 2 S.Hn /, the operator  .f / is a Hilbert–Schmidt operator, whose kernel function f is a Schwartz function. Indeed, for ' 2 S.Rn /; u 2 Rn , we have by (4.2.1) that

90

4 Four Exponential Solvable Lie Groups

Z

1

f .x; y; t/e 2 i.t C 2 xyyu/ '.u  x/dxdydt

 .f /'.u/ D Z

Hn 1

f .u  x; y; t/e 2 i.t C 2 .ux/y/ '.x/dxdydt

D Hn

  1 2;3 O D f u  x;  .u C x/;  '.x/dx: 2 Hn Z

where fO2;3 .u; ; / WD

Z Rn R

f .u; y; t/e 2 i. yCt / dydt; . ; t/ 2 Rn  R;

is the partial Fourier transform in the variables y and t. This tells us that the operator  .f / is a kernel operator with kernel function f given by    f .u; x/ D fO2;3 u  x;  .u C x/;  ; x; u 2 Rn ; 2

(4.2.3)

which is a Schwartz function on Rn  Rn . Hence  .f / 2 B.L2 .Rn //1 . Now let b 2 B.L2 .Rn //1 and let Fb 2 S.Rn  Rn / be its kernel function. Define the Schwartz function 2 S.Rn  Rn / by   v u v u  ;  ; u; v 2 Rn : .u; v/ WD Fb  2  2  Then  .s  t;  .s C t// D Fb .s; t/; s; t 2 Rn : 2 O Take also a Schwartz function ˇ W R ! C, such that ˇ./ D 1. We can now define the Schwartz function f on Hn by O .x; y; t/ 2 Hn : f .x; y; t/ WD O 2;3 .x; y/ˇ.t/; Since  O D fO2;3 .s  t;  .s C t/; /ˇ./ 2 it follows that  .f / D b.

 .s  t;  .s C t// D Fb .s; t/; s; t 2 Rn ; 2 

We now give another proof of the imprimitivity theorem for the Heisenberg groups Hn . This result was first obtained by Stone and Von Neumann independently in 1929.

4.2 The Heisenberg Group

91

Theorem 4.2.5. Let .; H/ be an irreducible unitary representation of Hn , such that  D  21 i d .Z/ ¤ 0: Then  is equivalent to the representation  of Proposition 4.2.4. Proof. Let us show first that the kernel of  in L1 .Hn / is equal to the kernel of  . By Formula (4.2.3) it follows that ker D ff 2 L1 .Hn /; fO2;3 .s; u; / D 0 for all u; s 2 Rn g Z D ff 2 L1 .Hn /I f .a  .tZ//e 2 it dt D 0; a 2 Hn almost everywhereg: R

If now f 2 ker , then Z .f / D

f .g/.g/dg Z

Hn

Z .g  tZ/f .a  tZ/dtd gP

D Z

G=Cn

D

R

Z

.g/ Z

 f .a  tZ/dt d gP

R

G=Cn

D

e

2 it

.g/.0/d gP G=Cn

D 0: Hence ker  ker. Since .l.a/ ı r.a//f 2 ker for every f 2 ker; a D u C v; u 2 X; v 2 Y , it follows that 0 D .l.a/ ı r.a/.f // Z f .a1 ga/.g/dg D Z

Hn

f .g/.aga1 /dg

D Z

Hn

D

f .g/.g C Œa; g/dg Z

Hn

f .g/.g/e 2 i.uyvx / dg

D Z

Hn

Z

D

f .x C y C tZ/e X CY

R

2 it



dt .x C y/e 2 i.uyvx/ dxdy:

92

4 Four Exponential Solvable Lie Groups

for all a 2 Hn . Choosing any ;  2 H it then follows that Z  Z  0D f .x C y C tZ/e 2 it dt c ; .x C y/e 2 i.uyvx/ dxdy X CY

R

 for u 2 X; v 2 Y (where c ; .x C y/ WD h.x C y/ ; i). Hence the functions

Z  .x; y/ 7! c ; .x C y/

f .x C y C tZ/e 2 it dt



R

R are 0 for any ;  2 H . This means that R f .x C y C tZ/e 2 it dt D 0 for every x 2 X; y 2 Y . Hence ker  ker . We take now a Schwartz function 2 S..Rn //. There exists a self-adjoint f 2 S.Hn /, such that  .f / is the self-adjoint projection P onto the one-dimensional subspace C of L2 .Rn /. But then .f / is also a self-adjoint projection in B.H / different from 0, since ker D ker and .f f  f / D 0: Take now  in the image of .f / of length one. Then for any f 2 L1 .Hn / we have that  .f f f / D P ı  .f / ı P D h .f / ; iP D  .h .f / ; if /: Hence f f f D h .f / ; if modulo ker and so finally .f f f / D h .f / ; i.f /: Therefore h.f /; i D h.f /..f //; .f /./i D h.f f f /./; i D h .f / ; ih.f .//; i D h .f / ; i:  Hence E./ \ E. / ¤ ; and therefore  and  are equivalent. S cn is parametrized by R R2n . For every  2 Theorem 4.2.6. The unitary dual H  R , the representation  acts on L2 .Rn / and for every h 2 R2n , we have the unitary character h of Hn defined by h .x; y; t/ WD e 2 ih.x;y/ ; .x; y; t/ 2 Hn . Proof. We have seen that every irreducible representation of Hn defines a character  on the centre Cn . If  is not trivial then  is equivalent to  for some  ¤ 0 by Proposition 4.2.5. If  is trivial, then  is also trivial on Cn D ŒHn ; Hn , hence .g/ commutes with .g 0 / for every g; g 0 2 Hn . This shows that  must be a character by Schur’s lemma. But every character of Hn is of the form h ; h 2 R2n . 

4.2 The Heisenberg Group

93

Definition 4.2.7. Let cn / WD f˛ W R ! SB.L2 .Rn //I˛ measurable; L2 .H Z 2 k˛k2 WD ka./k2HS jjn d < 1g: R

Here kak2HS D tr.a a/ denotes the square of the Hilbert–Schmidt norm of a cn // of Hn by bounded operator a. We define a representation . ; L2 .H . .g/˛/./ WD  .g/ ı ˛./;  2 R : One has to check that the mappings g 7! .g/a are continuous. This is easy for a finite rank operator b and a mapping ˛./ WD './b;  2 R ; for some ' 2 cn /. Cc .R /. But these mappings are total in the Hilbert space L2 .H Proposition 4.2.8. For every F 2 S.Hn / and  2 R , the operator  .F / is Hilbert–Schmidt and Z 1 2 k .F /kHS D jFO .l/j2 d l: jjn Z ? Proof. Since the kernel function of  .F / is the Schwartz function F .s; x/ D FO 2;3 .s  x;  2 .s C x/; / it is clear that  .F / is Hilbert–Schmidt. Z k .F /k2HS D

X Y 

Z D

X Y 

Z D

X Y 

jF .x; u/j2 dxd u 1 jFO 2;3 .x  u;  .x C u/; /j2 dxd u 2 jFO 2;3 .x; u; /j2 dxd u

ˇZ ˇ2 ˇ ˇ 2 i.t Cyu/ ˇ D F .x C y C tZ/e dtdy ˇˇ d udx ˇ  X Y Y CRZ Z 1 D jFO .v C u C Z  /j2 d ud v jjn X  Y  Z 1 jFO .l/j2 d l:  D jjn Z ? Z

Theorem 4.2.9. The mapping F .F /./ WD  .F /;  2 R ; F 2 S.Hn /;

94

4 Four Exponential Solvable Lie Groups

cn / and extends to an isometry from L2 .Hn / onto the sends S.Hn / into L2 .H cn / and intertwines the left regular representation l and the Hilbert space L2 .H representation . Proof. For F 2 S.Hn / we have that Z kF k22 D Z D Z D

Hn

R

R

jF .l/j2 d l 1 jjn

Z Z?

 jFO .l C Z  /j2 d l jjn d

k .F /k2HS jjn d:

This shows that F is an isometry. Let us show that F is onto. Let ˛ be in the orthogonal of the image of F . If we take two C 1 mappings R ! S.Rn /; and ; and a function ' 2 Cc1 .R /, and if we write a'; ; ./ WD './P ./;./ ;  2 R ; where for two elements a; b 2 L2 .Rn / we have the linear operator Pa;b .ı/ WD hı; bia; ı 2 L2 .Rn /, then the function F'; ; W G ! C defined by Z F'; ; .x C y C tZ/ D Z

Z D

R

Rn

R

tr. .x  y  z/ ı a.//jjn d

.s/.s  x/e 2 i t e 2 iys dse  iyx jjn d; x 2 X; y 2 Y; t 2 R;

is contained in S.Hn / and F .F'; ; / D a'; ; , as can be easily verified. Hence 0 D h˛; a'; ; i Z D './tr.˛./ ı P; /jjn d; ' 2 Cc1 .R /; ;  2 S.Rn /: R

Hence 0 D tr.˛./ ı P; /; ;  2 S.Rn /;  2 R and finally ˛ D 0. This shows that the mapping F is onto. The intertwining property is easy. 

4.3 The “ax C b” Group

95

4.3 The “ax C b” Group Let G D R2 be equipped with the multiplication 0

.a; b/  .a0 ; b 0 / D .a C a0 ; e a b C b 0 /; a; a0 ; b; b 0 2 R: It is easy to verify that .G; / is a Lie group, that .0; 0/ DW e is the neutral element in G and that .a; b/1 D .a; e a b/; .a; b/ 2 G: The Lie algebra g of G is the vector space g D R2 with the basis fA D .1; 0/; B D .0; 1/g and the bracket ŒA; B D B: Indeed, it suffices to observe that for f 2 C 1 .G/ we have that d f ..t; 0/.a; b//jt D0 dt d @ f ..a  t; b//jt D0 D  .f /.a; b/: D dt @a

A f .a; b/ D

and d f ..0; t/.a; b//jt D0 dt d f ..a; e a t C b//jt D0 D dt @ D e a .f /.a; b/: @b

B f .a; b/ D

Hence  ŒA; B D



 @ @ @ ; e a D B: D e a @a @b @b

As we shall see later, G is a typical completely solvable Lie group. In matrix form,     01 10 ; ; BD AD 00 00   ab GD I a > 0; b 2 R : 01

96

4 Four Exponential Solvable Lie Groups

The exponential mapping exp W g ! G is given by   s e 1 t D E.s; t/; s; t 2 R: exp.s; t/ D s; s Indeed, we have for u; v 2 R; .s; t/ 2 G that E.u.s; t//E.v.s; t// D E.us; ut/E.vs; vt/    e us  1 e vs  1 D us; ut vs; vt us vs   us e vs  1 .e  1/ D us C vs; e vs ut C vt us vs   .uCv/s e 1 t D .u C v/s; s   e .uCv/s  1 .u C v/t D .u C v/s; .u C v/s D E..u C v/.s; t//: This shows that the mapping u 7! E.u.s; t// is a homomorphism. Furthermore d E.u.s; t//juD0 D .s; t/: du Hence E is indeed the exponential mapping. The left Haar measure dx on G is given by Z

Z

Z

'.x/dx D

'.E.sA/E.tB//dsdt D R2

G

R2

'.s; t/dsdt; ' 2 Cc .G/;

a since for any .a; b/ 2 G and ' 2 Cc .G/ we have that Z

Z .a; b/'.x/dx D G

'..a; e a b/.s; 0/.0; t//dsdt Z

G

Z

G

D

'.a C s; e as b C t/dsdt Z '.s; t/dsdt D

D R2

'.x/dx: G

4.3 The “ax C b” Group

97

We see immediately that G is not unimodular. For .a; b/ 2 G and ' 2 Cc .G/ we have that Z Z '.g.a; b/1 /dg D '.s  a; e a t  e a b/dsdt R2

G

De

a

Z '.s; t/dsdt D e

a

Z

R2

'.g/dg: G

Hence G .a; b/ D e a ; .a; b/ 2 G:

4.3.1 Induced Representations Let P D f0g  R. It is easy to see that P is a closed abelian and normal subgroup of G. Choose any  ¤ 0 2 R and let  be the character of P defined by  .0; p/ WD e ip ; p 2 R: Define the representation . ; H / of G by  WD indG P  : As can easily be checked, the invariant measure on G=P is given by Z

Z '.g/d gP D

R

G=P

'.E.s; 0//ds; ' 2 Cc .G=P /:

(4.3.1)

Let us realize the representation  on L2 .R/. The mapping U W L2 .R/ ! H I U. /.s; t/ D .s/ .0; t/1 D e it .s/; .s; t/ 2 G; is an isometry by (4.3.1) and so for the representation  obtained in this way on L2 .R/ we have that  .s; u/ .a/ D U 1 ı  .s; u/ıU. /.a/ D U. /.a  s; e aCs .u// D e i.e for .s; u/ 2 G and a 2 R.

aCs u/

.a  s/;

(4.3.2)

98

4 Four Exponential Solvable Lie Groups

Let us compute  .f / for f 2 L1 .G/. We have Z  .f / .u/ D Z

R2

f .s; t/e it e

D Z

su

.u  s/dsdt

R2

fO2 .s; e su / .u  s/ds

D Z

f .s; t/ .s; t/ .u/dsdt

R

fO2 .u  s; e s / .s/ds:

D R

Hence the operator  .f / is a kernel operator with kernel function f .u; s/ D fO2 .u  s; e s /; s; u 2 R;

(4.3.3)

where fO2 .s; u/ D

Z R

f .s; t/e i t u dt; f 2 L1 .G/; s; u 2 R:

Hence the kernel K of the representation  in the algebra L1 .G/ is given by the functions K D ff 2 L1 .G/I fO2 .R  f.sign /RC g/ D f0gg: Definition 4.3.1. Denote by p WD RB the Lie algebra of the group P . We identify the dual space p with R ' RB  . Proposition 4.3.2. The subset R ' R B  of p is open and G-invariant, consists of two open orbits and its complement f0g is the G-fixed point set of p . Proof. For  2 R B  and g D .a; b/ 2 G, we have that Ad  .g/.B  /.B/ D e a and so we have the two open G-orbits OC WD RC B  and O WD R B  . The complement f0g is obviously made of G-fixed points.  Theorem 4.3.3. For every  2 R the representation  is irreducible. If .; H/ is an irreducible representation of G, which is not trivial on the subgroup P D ŒG; G, then  is equivalent to C WD 1 or to  WD 1 . The representations C and  are not equivalent. Proof. Since the stabilizer of any B  ;  2 R ; is the group P D exp.RB/, it follows from Theorem 3.3.3 and Proposition 3.4.3 that for every  2 R

4.3 The “ax C b” Group

99

the representation  is irreducible. Furthermore, if .; H/ is an irreducible representation of G, which is not trivial on the subgroup P D ŒG; G, then the support of restriction to P of  is the closure of a non-trivial G-orbit, i.e. is either RC B  or R B  and so  is either equivalent to C or to  . In order to show that C and  are not equivalent, choose two functions f; g in L1 .G/, such that fO2 .s; u/ D 0 for all u > 0; fO2 .s; 1/ ¤ 0; s 2 R; and gO 2 .s; u/ D 0 for all u < 0; gO 2 .s; 1/ ¤ 0; s 2 R: Then relation (4.3.3) implies that  .f / ¤ 0; C .f / D 0  .g/ D 0; C .g/ ¤ 0: Hence C and  cannot be equivalent, since otherwise their kernels in L1 .G/ would be the same.  Remark 4.3.4. Let us give a direct proof that the representations  ;  2 R , are irreducible. Let V be a closed G-invariant subspace of L2 .R/. Then its orthogonal complement V ? is also G-invariant. Let 0 2 V and 0 2 V ? . We take any ' 2 Cc .R/ and we let Z Z

D '.s/ .s/ 0 ds 2 V;  D '.s/ .s/0 ds 2 V ? : R

R

Since Z

.a/ D R

'.s/ 0 .a  s/ds; a 2 R;

it follows that is a continuous function on R and so is the function . We have that h .0; b/ ; i D 0; b 2 R: Hence Z Z a 0D e ibe .a/.a/da D R

1 e ibt . log.t//. log.t// dt; b 2 R: t RC

100

4 Four Exponential Solvable Lie Groups

This means that the Fourier transform of the L1 -function t 7! 10;1Π. log.t//. log.t//

1 t

vanishes for every b 2 R. Hence this continuous function is identically zero on 0; 1Πand so is the function R 3 a 7! .a/.a/. The same argument works with the function  .s; 0/ ; s 2 R and therefore we have that

.a  s/.a/ D 0; s; a 2 R: Hence, if is not identically zero, we must have that .a/ D 0 for all a 2 R. Finally, the function 0 itself must be 0, because 0 can be approximated in L2 .R/ by functions  of the type above. This shows that if V ¤ f0g then V ? must be f0g, hence V D L2 .R/.  Proposition 4.3.5. The representations  are equivalent to C WD 1 , if  > 0 (resp. to  WD 1 if  < 0). Proof. Let a 2 R and Ua W L2 .R/ ! L2 .R/ be defined by Ua . /.u/ WD .u  a/; 2 L2 .R/; u 2 R: Let for  2 R a D Ua ı  ıUa : For 2 L2 .R/ we have by (4.3.2) that a .s; t/Ua .u/ D  .s; t/Ua .u  a/ D e i.e D e ie

a e uCs t

auCs t /

Ua .u  a  s/

.u  s/ D ea .s; t/ .u/:

Hence again by (4.3.2) a D ea :

4.3.2 The Plancherel Theorem Let ı be the unbounded self-adjoint operator defined on L2 .R/ by 1

ı .u/ D e  2 u .u/; u 2 R; 2 L2 .R/: For every f 2 L2 .G/ \ L1 .G/ we have that Z  .f / ı ı .u/ D

R

1 fO2 .u  s; e s /e  2 s .s/ds; 2 L2 .G/; u 2 R;



4.3 The “ax C b” Group

101

and so the kernel function f0 of the operator  .f / ı ı is given by 1 f0 .u; s/ D fO2 .u  s; e s /e  2 s ; s; u 2 R:

We compute the Hilbert–Schmidt norm k .f /kHS of the operator  .f /: Z k .f /k2HS D jf0 .u; s/j2 duds R2

Z

jfO2 .u  s; e s /j2 e s duds

D R2

Z Z

1

D R

jfO2 .s; t/j2 dudt:

0

Hence for f 2 L1 .G/ \ L2 .G/ we have that Z Z 1 Z Z jfO2 .u; t/j2 dudt C kC .f /k2HS C k .f /k2HS D R

Z D

R2

R

0

1

jfO2 .u; t/j2 dudt

0

jfO2 .s; t/j2 dtds Z

D 2 R2

jf .s; u/j2 dsd u D 2kf k22 :

This is Plancherel’s formula for the group G.

4.3.3 d for the Enveloping Algebra Let us compute d  for the elements A; B of g. We have for any C 1 -function with compact support that d  .A/ .u/ D

d

.u  t/jt D0 D  0 .u/: dt

and d  .B/ .u/ D

d i.eu t / e

.u/jt D0 D ie u .u/; u 2 R; dt

This shows that the C 1 -vectors of the representations  ;  ¤ 0; are not the Schwartz functions, but the C 1 functions f such that e mu f .k/ .u/ 2 L2 .G/ for all k; m 2 N.

102

4 Four Exponential Solvable Lie Groups

Remark 4.3.6. Let us give a direct proof of the imprimitivity theorem for the ax Cb group. Let .; H/ be an irreducible unitary representation of G. Let us construct an imprimitivity system for . Proposition 4.3.7. Suppose that ker./ contains ker. / C ker.C /. Then  is one-dimensional. Proof. Indeed, let f 2 L1 .R/, such that fO.0/ D 0. We show that Z  .f / WD P

f .t/.0; t/dt R

is zero. It suffices to choose a sequence of elements .fk / in Cc1 .R/ such that fOk is zero in a neighbourhood of 0 and such that fOk converges in the uniform norm to fO. Then  P .fk / converges in operator norm to  P .f /. But for any  ¤ 0 and for

2 L2 .R/; u 2 R, we have that Z P .fk / .u/ D

R

fk .t/e ie

u t

.u/dt D fOk .e u / .u/:

Since we can write fk D hC C h , where h˙ is in C0 .R/, where hO C vanishes identically on RC and hO  on R , it follows that for every ' 2 L1 .G/,  .' hC / D  .'/ ı P .hC / D 0; if  < 0 and in the same way  .' h / D 0 if  > 0. This tells us that ' f f 2 ker. P /, i.e. 0 D  P .f / D R k 2 ker.C / C ker. /  ker./. Finally, 1 f .t/.0; t/dt. But then for any f 2 L .R/, we have that for any s 2 R, R  P .f /   P .0; s/ ı  P .f / D  P .f  .s/f / D 0; since .f  .s/f /b.0/ D 0. Hence ..0; s/  IdH / ı  P .f / D 0 and so for every s 2 R we have that .0; s/ D IdH . But then  is trivial on P D ŒG; G and so by Schur’s lemma,  is one-dimensional.  Theorem 4.3.8. Let .; H/ be an infinite-dimensional representation. Then  is equivalent to C or to  . Proof. We have that ker.C / ker. /  ker.C / \ ker. / D f0g; since any f 2 ker.C / \ ker. / has the property that fO2 .s; u/ D 0 for all s; u 2 R

4.3 The “ax C b” Group

103

and so f D 0. Since ker./ is a prime ideal in the algebra L1 .G/, it follows from Proposition 2.3.15 that either ker.C / or ker. / is contained in ker./. Suppose now that ker.C / is contained in ker./. Then by Proposition 4.3.7, .ker. // ¤ f0g. We identify G=P with R via the mapping R 3 t ! exp.tA/P . For ' 2 Cc1 .R/, let  '. log.t// if t > 0; '.t/ Q WD 0 if t  0: Then the mapping ' ! 'Q is a -isomorphism from the algebra Cc1 .G=P / to the algebra Cc1 .R/ and we define the operator .'/ for ' 2 Cc .G=P / on the Hilbert space H by Z

b '.b/.0; Q b/db:

.'/ D R

We then have that Q 1  k'k1 k .'/kop  k'k and that .Cc1 .G=P //H is dense in H, since the functions 'Q are not 0 on RC if L ' ¤ 0 and so  P .b 'Q / ¤ 0. Furthermore for ';  2 Cc .G=P /, we have that C

.' / D  P



_

b

   Q Q ..'Q // D .2/ P _ ..c '// Q _ ..c //     '// Q ı  P _ ..c '// Q D .2/ .'/ ı ./: D .2/ P _ ..c

Hence can be extended as a -homomorphism to the whole C  -algebra C0 .G=P /. Let us verify the imprimitivity relation. For any a 2 R, ' 2 Cc1 .G=P /, we have that Z .a; 0/ ı .'/ ı .a; 0/ D .a; 0/ ı b '.b/.0; Q b/db ı .a; 0/ R

Z

b '.b/.a; Q 0/ ı .0; b/ ı .a; 0/db

D Z

R

b '.b/.0; Q e a b/db

D R

D e a

Z

b '.e Q a b/.0; b/db: R

104

4 Four Exponential Solvable Lie Groups

Since b '.e Q a b/ D

Z '. log.t//e R

i ea bt

Z dt D e

'. log.e a t//e i bt dt

a R

Z

'.a  log.t//e i bt dt D e a

D ea R

Z Z R

4 D

..a/'/. log.t//e i bt dt R

D e a ..a; 0/'/.b/; it follows that .a; 0/ ı .'/ ı .a; 0/ D ..a; 0/'/: Similarly, for c 2 R,

Z

b '.b/.0; Q b/db ı .0; c/

.0; c/ ı .'/ ı .0; c/ D .0; c/ ı R

Z

b '.b/.0; Q c/ ı .0; b/ ı .0; c/db

D Z

R

b '.b/.0; Q b/db D .'/ D ..0; c/'/;

D R

because .0; c/' D ', P being a normal subgroup of G. This shows that .; / is a system of imprimitivity. By the imprimitivity theorem,  is equivalent to a representation which is induced from an irreducible representation, hence from a unitary character  of P . Since  is of infinite dimension,  cannot be 0. Hence  is equivalent to C . The arguments are similar if ker./ contains ker. /:  We have thus found the irreducible unitary representations of “ax C b” group. Theorem 4.3.9. The unitary dual of the group ax C b contains two infinite dimensional irreducible unitary representations C and  and the unitary characters  ..a; b// WD e i a ; .a; b/ 2 G . 2 R/. Proof. Every character must vanish on P D ŒG; G. Hence it is of the form  for some  2 R. 

4.4 Grélaud’s Group Definition 4.4.1. Let  2 R and let     1  0  A D A D D I2 C J 2 M2 .R/ where J D :  1  0

4.4 Grélaud’s Group

105

We equip R  R2 with the multiplication 0

.s; u/  .s 0 ; u0 / WD .s; u/ .s 0 ; u0 / WD .s C s 0 ; e s A u C u0 /; s; s 0 2 R; u; u0 2 R2 ; and we write G for the group .R  R2 ;  /. Here  e

tA

D

e t cos. t/ e t sin. t/ e t sin. t/ e t cos. t/



 De

t

cos. t/ sin. t/  sin. t/ cos. t/

 D e t e tJ

and for u D .u1 ; u2 / 2 R2 e tA u D .e t cos. t/u1 C e t sin. t/u2 ; e t sin. t/u1 C e t cos. t/u2 /: It is easy to verify that G is a Lie group, that .0; 0/ DW e is the neutral element in G and that .s; u/1 D .s; e sA u/; .s; u/ 2 G : The Lie algebra g of G is the vector space g D R  R2 with the basis fT D .1; 0; 0/; X D .0; 1; 0/; Y D .0; 0; 1/g and the brackets ŒT; X  D X  Y; ŒT; Y  D Y C X: Indeed, it suffices to observe that for f 2 C 1 .G / we have that d f ..t; 0; 0/.s; u//jt D0 dt d @ f ..s  t; u//jt D0 D  .f /.s; u/; D dt @s

T f .s; u/ D

that d f ..0; t; 0/.s; u//jt D0 dt d f ..s; e sA .t; 0/ C u//jt D0 D dt d f ..s; .e s cos.s/t; e s sin.s/t/ C u// jt D0 D dt D .e s cos.s/@2  e s sin.s/@3 /f .s; u/

X f .s; u/ D

106

4 Four Exponential Solvable Lie Groups

and that d f ..0; 0; t/.s; u//jt D0 dt d f ..s; e sA ..0; t// C u//jt D0 D dt d f ...s; .e s sin.s/t; e s cos.s/t/ C u/ jt D0 D dt D .e s sin.s/@2  e s cos.s/@3 /f .s; u/:

Y f .s; u/ D

Hence  ŒT; X  D

@  ; e s cos.s/@2  e s sin.s/@3 @s



D e s cos.s/@2  e s sin.s/@2  e s sin.s/@3 C e s cos.s/@3 D X  Y: and  ŒT; Y  D



@ s ; e sin.s/@2  e s cos.s/@3 @s



D e s sin.s/@2  e s cos.s/@2  e s cos.s/@3  e s sin.s/@3 D Y C X: As we shall see later, G is a typical exponential solvable Lie group. In matrix form, 1 0 1 0 1 000 001 1 0 T D @ 1 0A ; X D @0 0 0A ; Y D @0 0 1A ; 000 000 0 00 80 t 9 1 < e cos. t/ e t sin. t/ x = G D @e t sin. t/ e t cos. t/ y A I t; x; y 2 R : : ; 0 0 1 0

The exponential mapping exp W g ! G is given by   sA 1 e t ; .s; t/ 2 G : E.s; t/ WD exp.s; t/ D s; sA

4.4 Grélaud’s Group

107

Indeed, we have for u; v 2 R; .s; t/ 2 G that E.u.s; t//E.v.s; t// D E.us; ut/E.vs; vt/    e usA  1 e vsA  1 D us; ut vs; vt usA vsA   usA  1/ e vsA  1 vsA .e D us C vs; e ut C vt usA vsA   e .uCv/sA  1 t D .u C v/s; sA   e .uCv/sA  1 D .u C v/s; .u C v/t .u C v/sA D E..u C v/.s; t//: This shows that the mapping u 7! E.u.s; t// is a homomorphism. Furthermore d E.u.s; t//juD0 D .s; t/: du Hence E is indeed the exponential mapping. The left Haar measure dx on G D G is given by Z

Z

Z

'.x/dx D

'.E.s; 0/E.0; u//dsd u D RR2

G

RR2

'.s; u/dsd u; ' 2 Cc .G/;

since for any .a; b/ 2 G and ' 2 Cc .G/ we have that Z

Z .a; b/'.x/dx D G

'..a; e aA b/.s; 0/.0; u//dsd u Z

G

Z

G

'..a C s; e .as/A b C u/dsd u

D

Z

D

'.s; u/dsd u D RR2

'.x/dx: G

We see immediately that G is not unimodular. For .a; b/ 2 G and ' 2 Cc .G/ we have that Z Z '.g.a; b/1 /dg D '.s  a; e aA u  e aA b/dsd u G

RR2

D e 2a

Z

'.s; u/dsd u D e 2a RR2

Z '.g/dg: G

108

4 Four Exponential Solvable Lie Groups

Hence G .a; b/ D e 2a ; .a; b/ 2 G:

4.4.1 Induced Representations Let P D f0g R2 and p D R2 . It is easy to see that P is a closed abelian and normal subgroup of G. Choose any  ¤ 0 2 R2 and let  be the character of P defined by  .0; p/ WD e ip ; p 2 R2 : Define the representation . ; H / of G by  WD indG P  : As can easily be checked, the invariant measure on G=P is given by Z

Z '.g/d gP D

R

G=P

'.E.s; 0//ds; ' 2 Cc .G=P /:

(4.4.1)

Let us realize the representation  on L2 .R/. The mapping U W L2 .R/ ! H I U. /.s; t/ D .s/ .0; t/1 D e it .s/; .s; t/ 2 G ; is an isometry by (4.4.1) and so for the representation  , we obtain in this way on L2 .R/:  .s; u/ .a/ D U 1 ı  .s; u/ıU. /.a/ D U. /.a  s; e .aCs/A .u// D e i.e

.aCs/A u/

.a  s/;

(4.4.2)

for .s; u/ 2 G and a 2 R. Definition 4.4.2. Let S D f 2 R2 ; kk2 D 1g be the unit circle in R2 . We denote by d the Haar measure on S with total measure 1, i.e. Z

1 f ./d D 2 S

Z

2

f .e i t /dt; f 2 Cc .S /: 0

Proposition 4.4.3. Let ; 2 p nf0g. The representations  and  are equivalent t if and only if D e sA  for some s 2 R.

4.4 Grélaud’s Group

109

Proof. Let a 2 R and Ua W L2 .R/ ! L2 .R/ be defined by Ua . /.u/ WD .u  a/; 2 L2 .R/; u 2 R: Let for  2 p a D Ua ı  ıUa : For 2 L2 .R/ we have by (4.4.2) that a .s; u/ .r/ D  .s; u/Ua .r  a/ D e i.e D e i.e

aAt

/.e.rCs/A u/

.arCs/A u/

Ua .r  a  s/

.r  s/ D eaAt  .s; u/ .r/:

t

Hence if D e aA , then  is equivalent to  . t t Suppose now that e RA  \ e RA D ;. By Sect. 4.4.3 we can find two Schwartz functions ' and in S.P /, such that their Fourier transforms have compact support, t t t such that 'O vanishes on e RA  but not on e RA and such that O vanishes on e RA t but not on e RA . We then have that Z  .'/ .s/ D

'.0; u/e i.e

sA u/

.s/d u D '.e O sA / .s/ t

R

for 2 L2 .R/ and s 2 R. Hence  .'/ D 0. Similarly,  . / D 0,  . / ¤ 0 and  .'/ ¤ 0. This shows that  and  are not equivalent. 

4.4.2 d for the Enveloping Algebra Let us compute d  for the elements T; X; Y of g. We have for any C 1 -function with compact support that d  .T / .r/ D

d

.r  t/jt D0 D  0 .r/: dt

and for u 2 p, d  .0; u/ .r/ D

d i.erA /t u e

.r/jt D0 dt

D i.e rA .u// .r/; r 2 R;

110

4 Four Exponential Solvable Lie Groups

Since  A.X C iY / D

1   1

 .1; i / D .1 C i/.1; i /;

it follows that d  .X C iY / .r/ D i e r.1Ci / .1; i / .r/; r 2 R; 2 L2 .R/: This shows that the C 1 -vectors of the representations  ;  ¤ 0 are not the Schwartz functions, but the C 1 functions f such that e mr f .k/ .r/ 2 L2 .R/ for all k; m 2 N:

4.4.3 The Imprimitivity Theorem for Grélaud’s Group We need to apply Theorem 3.3.3 and Proposition 3.4.3. Proposition 4.4.4. Let S be the unit circle in R2 . The mapping t

˚ W R  S ! R2 n f0g; ˚.r; s/ D e rA s; is a diffeomorphism. Proof. It suffices to show that ˚ has an inverse of class C 1 , since ˚ is obviously smooth itself. For u 2 R2 ; u ¤ 0, we write  .u/ WD .log.kuk2 /; e  log.kuk2 /A .u// 2 R  S: t

Since t

ke rA sk2 D e r ; it follows that  .˚.r; s// D .r; e rA .e rA s// D .r; s/ t

t

and ˚. .u// D ˚.log.kuk2 /; e  log.kuk2 /A .u// D e log.kuk2 /A e  log.kuk2 /A .u/ D u: t

Hence  is the inverse mapping of ˚.

t

t



4.4 Grélaud’s Group

111

Proposition 4.4.5. Identifying the character space PO of the normal closed subgroup P of G with R2 , we have that the open subset U WD R2 n f0g has the property that two distinct G-orbits in U are separated by two open G-invariant subsets in U and that the complement f0g of U in PO is made out of a single fixed point. Proof. Indeed any G-orbit O in U is determined by its intersection with the unit circle, i.e. there exists a unique sO 2 S , such that O D G  sO . So for two distinct orbits O; O 0 we take the subsets U WD G  I , resp. U 0 WD G  I 0 , where I D ft 2 S; jt  sO j < "g, resp. I 0 D ft 2 S; jt  sO 0 j < "g, where 2" D js0  sO 0 j (jt  t 0 j denoting the Euclidean distance of the two elements t; t 0 2 S .) By Proposition 4.4.4, the two sets U; U 0 are open and G-invariant and their intersections are empty.  Proposition 4.4.6. The representation  is irreducible for every  2 R . Every unitary irreducible representation of infinite dimension of Grélaud’s group is equivalent to  for some  2 R . Proof. This follows again immediately from Theorem 3.3.3 and Proposition 3.4.3.  Let us give concrete proofs of these results in the following propositions. Proposition 4.4.7. The representation  is irreducible for every  2 R . Proof. Let V be a closed G -invariant subspace of L2 .R/. Then its orthogonal complement V ? is also G -invariant. Let 0 2 V and 0 2 V ? . We take any ' 2 Cc .R/ and we let Z Z 0

D '.s/ .s; 0/ ds 2 V;  D '.s/ .s; 0/0 ds 2 V ? : R

R

Since

Z

.a/ D R

'.s/ 0 .a  s/ds; a 2 R;

it follows that is a continuous function on R and so is the function . We have that h .0; b/ ; i D 0; b 2 R2 : Hence

Z

e i.e

0D

aA b/

R

.a/.a/da; b 2 R:

(4.4.3)

Let f be any radial function in the Schwartz space S.P /. It follows from (4.4.3) that Z Z aA f .b/ e i.e b/ .a/.a/dadb 0D Z

R

D

R

Z

.a/.a/ R

R

f .b/e i.e

aA b/

 db da

112

4 Four Exponential Solvable Lie Groups

and so

Z

t fO.e aA / .a/.a/da

0D R

for all these functions f . Since the functions t 7! fO.e t /, f 2 S.P /, give us among others all the C 1 -functions with compact support on R, it follows that

.a/.a/ D 0 for all a 2 R: Replacing by .s; 0/ ; s 2 R, we see that

.a  s/.a/ D 0 for all s; a 2 R and so  D 0, if ¤ 0. This shows that the orthogonal complement of V is reduced to f0g if V ¤ f0g.  Proposition 4.4.8. Let .; H/ be an infinite-dimensional irreducible representation of G and let  WD jP . Then .f / is not 0 for all the functions f 2 Cc .P /, for which fO.0/ D 0 Proof. Indeed, suppose the contrary. Then for any u 2 P and f 2 Cc .P /, we have that ..u/f  f /b.0/ D fO.0/  fO.0/ D 0 and so .f / D ..u/.f // D .u/ ı .f / which implies that ..0; u/ D IdH for all u 2 R2 : Since P D ŒG ; G , this implies that  is one-dimensional.  Theorem 4.4.9. Let .; H/ be an infinite-dimensional irreducible representation of the group G . Then  is equivalent to  for some  2 S . Proof. As before, let  D jP . Let N be the hull in R2 of the ideal I WD ker./C  .P / . We have seen in Proposition 4.4.8 that N is not equal to f0g. Denote for any f 2 Cc .P / and s 2 R the function R2 3 .0; u/ 7! f ..s; 0/.0; u/.s; 0// by i.s/f . Then Z

e 2s f .e sA u/.0; u/d u

.s; 0/ ı .f / ı .s; 0/ D R2

and so for f 2 C  .P / and s 2 R, we have that t fO.e sA / D 0 for all  2 N ” .s; 0/ ı .f / ı .s; 0/ D 0

” .f / D 0 ” fO./ D 0 for all  2 N:

4.4 Grélaud’s Group

113

This shows that N is invariant under the R-action defined by the matrices t e sA ; s 2 R. Suppose that N contains more than two R-orbits. This means that N contains two distinct points  and of S . If we take two open disjoint R-invariant subsets U and V of R2 , U containing  and V containing , then the two ideals IU and IV of C  .P / defined by IU D ff 2 C  .P /I fO has a compact support contained in U g and similarly for IV , then IU IV D f0g and both ideals are invariant under conjugation by elements of R  f0g. Hence .IU IV / D f0g and so, since  is irreducible, either IU or IV is contained in ker./. But there exist elements f 2 IU , for which fO. / ¤ 0 and so this f is not contained in ker./; similarly for IV . t Hence the hull of the ideal ker./C  .P / is the closure N WD fe sA ; s 2 Rg [ f0g of a unique R-orbit. We can define a representation .; H/ of C1 .N / WD f'Q 2 C0 .N /; '.0/ Q D 0g by letting .'/ WD .f /; f 2 C  .P / with fO D ' on N: It is easy to check that  is a bounded -representation of the C  -algebra C1 .N /, since this algebra is isomorphic to C  .P /=kerC  .P / . We now identify G =P with N0 WD N nf0g via the mapping R 3 s 7! exp.sAt / for ' 2 C0 .R/, and let t

'.e Q sA / WD '.s/; s 2 R: Then the mapping ' ! 'Q is a -isomorphism from the algebra C0 .G =P / onto the algebra C0 .N0 /. We obtain a representation . ; H/ of C0 .G =P / on the Hilbert space H by .'/ D .'/; Q ' 2 Cc .G =P /: We then have for any s 2 R and 'Q 2 C0 .N0 /, for which there exists f 2 S.P / with fOjN 0 D ', Q that 

Z

 f .b/.0; b/db ı .s; 0/

.s; 0/ ı .'/ ı .s; 0/ D .s; 0/ ı R2

Z D

f .b/.s; 0/ ı .0; b/ ı .s; 0/db Z

R2

D

f .b/.0; e sA b/db R2

De

2s

Z R2

f .e sA b/.0; b/db:

114

Since

4 Four Exponential Solvable Lie Groups

Z

e 2s f .e sA b/e i .b/db D fO.e sA /; 2 p ; t

R2

and since for s; r 2 R,

A

t t t fO.e .sCr/A / D '.e Q .sCr/A / D ..s/'/.e rA /;

we have that .s; 0/ ı .'/ ı .s; 0/ D ..s; 0/'/ for all s 2 R and ' 2 C0 .G =P /. Similarly, for .0; c/ 2 P and 'Q as above, Z .0; c/ ı .'/ ı .0; c/ D .0; c/ ı

f .b/.0; b/db ı .0; c/

R2

Z

f .b/.0; b/db D .'/ D ..0; c/'/;

D R2

because .0; c/' D ', P being a normal subgroup of G . This shows that .; / is a system of imprimitivity. By the imprimitivity theorem,  is equivalent to a representation which is induced from an irreducible representation, hence from a unitary character  of P . Since  is of infinite dimension,  cannot be 0. Therefore  '  .  We have thus found the irreducible unitary representations of Grélaud’s group. Theorem 4.4.10. Let G be Grélaud’s group. The unitary dual of G consists of the infinite-dimensional representations  ,  2 p of length 1, and the unitary characters  ..a; u// WD e i a ; .a; u/ 2 G . 2 R/. Proof. Every character must vanish on P D ŒG ; G . Hence it is of the form  for some  2 R . 

4.4.4 The Plancherel Theorem Let us compute  .f / for f 2 L1 .G /. We have Z f .s; u/ .s; u/ .r/dsd u  .f / .r/ D Z

RR2

f .s; u/e ie

D Z

.r  s/dsdt

t fO2 .s; e .sr/A / .r  s/ds

D Z

.sr/A u

RR2

R t fO2 .r  s; e sA / .s/ds:

D R

4.4 Grélaud’s Group

115

Hence the operator  .f / is a kernel operator with kernel function f .r; s/ WD fO2 .r  s; e sA /; r; s 2 R; t

where fO2 .s; u/ D

Z R

f .s; t/e i t u dt; f 2 L1 .G /; .s; u/ 2 G :

Therefore the kernel K of the representation  in the algebra L1 .G / is given by the functions K D ff 2 L1 .G /I fO2 .r; e sA / D 0 for all r; s 2 Rg: t

Let ı be the unbounded self-adjoint operator defined on L2 .R/ by ı .u/ D e u .u/; u 2 R; 2 L2 .R/: For every f 2 L2 .G / \ L1 .G / we have that Z  .f / ı ı .u/ D

R

t fO2 .u  s; e sA /e s .s/ds; 2 L2 .R/; u 2 R;

and so the kernel function f0 of the operator  .f / ı ı is given by t f0 .u; s/ D fO2 .u  s; e sA /e s ; s; u 2 R:

We compute the Hilbert–Schmidt norm k .f / ı ıkHS of the operator  .f / ı ı: Z k .f / ı ık2HS D Z

jf0 .u; s/j2 duds

R2

jfO2 .u  s; e sA /j2 e 2s duds: t

D R2

Hence for f 2 L1 .G / \ L2 .G / we have that Z Z

Z S

k .f / ı ık2HS d D Z

S

jfO2 .r  s; e sA /j2 e 2s drdsd t

R2

jfO2 .s; u/j2 duds

D RR2

D .2/

Z

2 RR2

jf .s; u/j2 dsd u D .2/2 kf k22 :

This is Plancherel’s formula for the group G .

Chapter 5

Orbit Method

5.1 Auslander–Kostant Theory Auslander and Kostant [3] extended the orbit method to solvable Lie groups of type I. To explain their theory, we first prepare the ingredients. Let G be a solvable Lie group with Lie algebra g and g the dual vector space of g. G acts on g by the adjoint action and on g by its contragradient representation:   .gf /.X / D Ad .g/f .X / D f .Ad.g 1 /X / .g 2 G; f 2 g ; X 2 g/: The action of G obtained in this way is called the coadjoint representation of G. Let G.f / be the stabilizer of f 2 g in G. Hence the Lie algebra of G.f / is g.f / D fX 2 gI f .ŒX; Y / D 0; 8 Y 2 gg: We define the anti-symmetric bilinear form Bf on g  g by Bf .X; Y / D f .ŒX; Y /. For a vector subspace a of g, we denote by f ja the restriction of f to a and put 

a?;g D ff 2 g I f ja D 0g; af D fX 2 gI Bf .X; Y / D 0; 8 Y 2 ag: 

When there is no risk of confusion, we simply write a? instead of a?;g . When a  af , we say that a is isotropic (with respect to Bf ). Then,

© Springer Japan 2015 H. Fujiwara, J. Ludwig, Harmonic Analysis on Exponential Solvable Lie Groups, Springer Monographs in Mathematics, DOI 10.1007/978-4-431-55288-8__5

117

118

5 Orbit Method

a is a maximal isotropic subspace ,a D af ,a  af and dim a D

1 .dim g C dim.g.f ///: 2

We designate the set of Lie subalgebras h of g satisfying h  hf (resp. h D hf ) by S.f; g/ (resp. M.f; g/). Let gC be the complexification of g, and we linearly extend f; Bf to gC . Definition 5.1.1. Let p be a complex Lie subalgebra of gC . When p satisfies the following conditions, it is called a polarization of G at f 2 g : (1) p is a maximal isotropic subspace with respect to Bf ; (2) p C p is a complex Lie subalgebra of gC ; (3) p is stable by Ad.G.f //. p is a polarization of g, which means that p is a polarization of the connected and simply connected Lie group with Lie algebra g. We denote by P .f; G/ the set of polarizations of G at f 2 g . Definition 5.1.2. p 2 P .f; G/ being given, we define real Lie subalgebras d; e of g by d D p \ g; e D .p C pN / \ g: As is easily seen, dC D p \ pN ; eC D p C pN and d D ef . Hence Bf induces a non-degenerate bilinear form BO f on the quotient vector space e=d. Moreover, .e=d/C ' eC =dC D .p C pN / =p \ pN D p=dC ˚ pN =dC : Definition 5.1.3. We p define a linear operator J on .e=d/C by the following: J.X / D iX; i D 1 for X 2 p=dC , J.X / D iX for X 2 pN =dC . Then J defines a real linear transformation on e=d with square equal to 1, namely a natural complex structure, and for u 2 e=d, u C iJ u 2 p=dC ; u  iJ u 2 pN =dC : Definition 5.1.4. We define a bilinear form Sf on e=d by Sf .u; v/ D BO f .u; J v/; u; v 2 e=d: Proposition 5.1.5. Sf is a non-degenerate symmetric bilinear form on e=d and BO f ; Sf are J -invariant: i.e. BO f .J u; J v/ D BO f .u; v/; Sf .J u; J v/ D Sf .u; v/:

5.1 Auslander–Kostant Theory

119

Proof. p=dC is clearly isotropic relative to BOf and, for u; v 2 e=d, 0 D BO f .u C iJ u; v C iJ v/

  D BO f .u; v/  BO f .J u; J v/ C i BOf .J u; v/ C BO f .u; J v/ :

Taking the imaginary part of this equality, BOf .u; J v/ D BOf .J u; v/ D BO f .v; J u/:

(5.1.1)

Hence Sf .u; v/ D Sf .v; u/, i.e. Sf is symmetric. Since BO f ; J are non-degenerate, Sf too. From relation (5.1.1) and J 2 D 1, BO f .J u; J v/ D BO f .u; J.J v// D BOf .u; v/; Sf .J u; J v/ D BO f .J u; J.J v// D BOf .J u; v/ D BO f .v; J u/ D Sf .v; u/ D Sf .u; v/:  Definition 5.1.6. When the symmetric form Sf is positive definite or e=d D f0g, we say that p 2 P .f; G/ is positive. In particular, when p D pN , we say that p is real. On real polarizations of connected simply connected type I solvable Lie groups, please refer to [29], for example. We denote by P C .f; G/ the set of all positive polarizations of G at f . Taking p 2 P .f; G/, we define the Lie subalgebras d; e as before and denote by D0 (resp. E0 ) the connected Lie subgroup of G corresponding to d (resp. e). Since p is stable by the action of Ad.G.f //, D D G.f /D0 ; E D G.f /E0 are two subgroups of G. Proposition 5.1.7. D, D0 are closed subgroups of G. Moreover D0 is the connected component of the unit in D and d is the Lie algebra of D. Proof. As d and e are mutually the orthogonal complement of each other relative to Bf , for X 2 g: X 2 d , X f .Y / D Bf .Y; X / D 0 .8 Y 2 e/: Taking the image of the exponential map, for a 2 D0 : .af  f /.Y / D 0 .8 Y 2 e/:

(5.1.2)

120

5 Orbit Method

Thus Eq. (5.1.2) holds for any a 2 D0 and any element X of the Lie algebra of D0 satisfies X f .Y / D 0 .8 Y 2 e/: Hence X 2 d and D0 D D0 . Let us repeat a similar argument to prove the rest of the proposition. Let D1 denote the connected component of the unit in D D G.f /D0 . For a 2 D1 , .af  f /.Y / D 0 .8 Y 2 e/: We know from this that the Lie algebra d1 of D1 is contained in d. On the other hand, D0  D implies d  d1 . Hence d D d1 and D0 D D1 , while D0  D  D. In this way D is a closed subgroup and D0 is the connected component of the unit in D.  Here we consider the D-orbit Df in g . Proposition 5.1.8. Df is an open set of the affine space f C e? . Clearly Df D D0 f . Proof. First let us see that f C e? is stable by the action of D. Since e is D-stable, e? is too. As D D D0 G.f /, Df D D0 G.f /f D D0 f . Therefore, for d 2 D and ` 2 e? , there exists a 2 D0 such that d .f C `/  f D af  f C d `: Then, taking into account relation (5.1.2), d .f C `/  f 2 e? . Thus f C e? is D-stable and df  e? . On the other hand, df Š d=g.f / and e D df give dim d C dim e D dim g C dim.g.f //: Therefore dim.df / D dim .d=g.f // D dim d  dim.g.f // D dim g  dim e D dim.e? /: Consequently df D e? and, df being the tangent space of D0 f  f C e? at f , the implicit function theorem concludes that D0 f D Df is an open set of f C e? .  Now we investigate E-orbit Ef D E0 f in g . Definition 5.1.9. When Ef is a closed set in g , we say that p 2 P .f; G/ satisfies the Pukanszky condition.

5.1 Auslander–Kostant Theory

121

Remark 5.1.10. If p is real, D D E and Proposition 5.1.8 means: p satisfies the Pukanszky condition , Df D f C e? : Lemma 5.1.11. If p 2 P .f; G/ satisfies the Pukanszky condition, E0 ; E are closed subgroups of G and E0 is the connected component of the unit in E. Proof. Let W G ! g be the mapping defined by .g/ D gf . Evidently 1 E D .Ef / D 1 .E0 f / and consequently E is a closed set in G. Let E1 be the connected component of the unit in E. Clearly E0  E1 . On the other hand, .E0 / D .E1 / and, G.f /0 denoting the connected component of the unit in G.f /, G.f /0  E0 . Comparing their dimension E0 D E1 .  Proposition 5.1.12. If p 2 P .f; G/ satisfies the Pukanszky condition, then Df D f C e? . Proof. We put K D fa 2 E0 I af 2 f C e? g. Evidently K is a closed set in E0 and consequently a closed set in G by Lemma 5.1.11. Furthermore, e? being stable by E0 , K is a subgroup of E0 and f C e? is K-invariant. As we immediately see, D0  K and Proposition 5.1.8 means that D0 f is an open set in f C e? . Dividing K into cosets by D0 , Kf turns out to be an open set in f C e? . On the other hand, the Pukanszky condition implies that Kf D .E0 /f \ .f C e? / is a closed set in f C e? . Thus Kf D f C e? . Next let k be the Lie algebra of K. From what we have seen by now, Bf .k; e/ D f0g. Therefore k  d. Since the inclusion d  k is trivial, k D d and D0 is nothing but the connected component of the unit in K. In particular, D0 is a normal subgroup of K. For ` 2 e? , we write f C ` D kf with k 2 K. Then, D0 .f C `/ D D0 .kf / D k.D0 f /: Proposition 5.1.8 means that D0 f is an open set in f C e? and that each orbit D0 .f C `/ is an open set in f C e? . Hence D0 f D f C e? , and we finally conclude by D D D0 G.f / that Df D f C e? .  Lemma 5.1.13. We assume that p 2 P .f; G/ satisfies the Pukanszky condition. If we denote by G.f /0 the connected component of the unit in G.f /, D0 \ G.f / D G.f /0 . Let D1 be the simply connected covering group of D0 and  W D1 ! D0 the canonical projection. Then  1 .G.f /0 / D G.f /1 is connected. Proof. As Df D D0 f , Df ' D0 =.D0 \G.f //. Since G.f /0  D0 , G.f /0 is the connected component of the unit in D0 \ G.f /. From the proof of Proposition 5.1.12 D0 f is simply connected and D0 \ G.f / turns out to be connected. Hence D0 \ G.f / D G.f /0 and D1 =G.f /1 ' D0 =G.f /0 D D0 =D0 \G.f / D Df D D0 f: D0 f being simply connected, G.f /1 D  1 .G.f /0 / is connected.



122

5 Orbit Method

By the definition of g.f /, the restriction f jg.f / of f to g.f / gives a homomorphism of the Lie algebra g.f / into R. Definition 5.1.14. We say that f 2 g is integral if there is a homomorphism f W G.f / ! T satisfying df D if jg.f / . Hereafter we assume that f 2 g is integral and denote by f the corresponding character of G.f /. From the relation f .Œd; e/ D f0g, f jd gives a homomorphism d ! R. Proposition 5.1.15. When p 2 P .f; G/ satisfies the Pukanszky condition, f is uniquely extended to a character f W D ! T satisfying df D if jd . Proof. Let us adopt the notations of Lemma 5.1.13. By f .Œd; d/ D f0g and D1 being simply connected, there uniquely exists a character 1f W D1 ! T satisfying d1f D if jd . When p 2 P .f; G/ satisfies the Pukanszky condition, G.f /1 is connected from Lemma 5.1.13 and   1f jG.f /1 D f jG.f /0 ı : The kernel K of the homomorphism  W D1 ! D0 is contained in G.f /1 D  1 .G.f /0 / and 1f jK is trivial. Therefore there is a homomorphism 0f W D0 ! T such that 1f D 0f ı . Evidently, d0f D if jd . Now, G.f / acts on D0 and so on the group of its unitary characters. By the way, the unitary character of a connected Lie group is determined by its differential. On account of G.f /f D f , 0f .ud u1 / D 0f .d / .u 2 G.f /; d 2 D0 /: Now letting A D D0 Ì G.f / be the semi-direct product of D0 by G.f /, we define the mapping f W A ! T by f .d; u/ D 0f .d /f .u/ .d 2 D0 ; u 2 G.f //: Then f is a unitary character of A. Next we consider a homomorphism  of A onto D defined by .d; u/ D d u. By Lemma 5.1.13, ker D f.u; u1 /I u 2 G.f / \ D0 D G.f /0 g: Since 0f coincides on G.f /0 with f , the homomorphism f becomes trivial on ker and induces a unitary character f of D. f is clearly provided with the required property. Since f coincides on G.f / with f and is determined on D0 by its differential, its uniqueness is evident by D D D0 G.f /.  We start from p 2 P .f; G/ satisfying the Pukanszky condition and construct a unitary representation of G. From E D E0 D, X D E=D is connected. On the other hand, the alternating bilinear form BOf on e=d is non-degenerate and D-invariant.

5.1 Auslander–Kostant Theory

123

1 m O Let dim.e=d/ D 2m. Extending E-invariantly mŠ ^ Bf , we induce a measure X on X which is invariant by the action of E. We denote by M.E; f / the set of all measurable functions  on E which satisfy the covariance relation

.ab/ D f .b/1 .a/ .a 2 E; b 2 D/; consider the space of  2 M.E; f / such that Z jj2 dX < 1 X

and identify functions which coincide almost everywhere. In this fashion, we get O O a Hilbert space H.E; f /. Namely, H.E; f / is the Hilbert space of the induced E representation indD f . Let C 1 .E/ be the space of all C 1 functions on E. For z D x C iy .x; y 2 e/ and 2 C 1 .E/, we put z D x C i y. Here . x/.a/ D

d .aexp.tx//jt D0 .a 2 E/: dt

Moreover we set C 1 .E; f; p/ D f

2 C 1 .E/I z D if .z/ ; z 2 pg;

O L D C 1 .E; f; p/ \ M.E; f /; H.f; f ; p; E/ D L \ H.E; f /: Proposition 5.1.16 ([3])). The space H.f; f ; p; E/ is a closed subspace of the O Hilbert space H.E; f /. The space H.f; f ; p; E/ is regarded as the space of all complex holomorphic functions belonging to the L2 -space with a certain weight function, stable by the E action of indE D f and gives its subrepresentation. We designate this by indD .f ; p/. Finally, setting  E  .f; f ; p; G/ D indG E indD .f ; p/ ; we get a subrepresentation of indG D f and call it a holomorphically induced O representation. H.f; p; G/ denotes the Hilbert space of indG D f and H.f; f ; p; G/ its closed subspace corresponding to .f; f ; p; G/. Let us consider an exact sequence of Lie groups p

1 ! N ! G ! GQ ! 1: Q and dp W g ! gQ the differential Let n (resp. g; gQ ) be the Lie algebra of N (resp. G; G) mapping of p. We use the same notation for the linear extension of dp to gC .

124

5 Orbit Method

Proposition 5.1.17. We assume that fQ 2 gQ  is integral and denote by fQ the Q fQ/. We suppose that pQ 2 P .fQ; G/ Q satisfies corresponding unitary character of G. Qıdp 2 g ; p D .dp/1 .pQ /. Then f the Pukanszky condition and put f D f   Q fQ/ and the character f of G.f / defined by is integral, G.f / D p 1 G. f D fQıp is what corresponds to f . Furthermore, p is a polarization of G at f satisfying the Pukanszky condition. In addition, Q .f; f ; p; G/ ' .fQ; fQ; pQ ; G/ıp: Proof. It suffices to check the various definitions, paying attention to the following points. First, gQ  is identified with n?  g and p satisfies the Pukanszky condition. Q f D  Qıp; E D p 1 .E/ Q and p induces an isomorphism Next, D D p 1 .D/; f Q DQ (resp. G= Q D; Q G= Q E). Q Finally, between E=D (resp. G=D; G=E) and E= Q H.fQ; fQ; pQ ; G/ıp D H.f; f ; p; G/:



Let G be a connected and simply connected solvable Lie group with Lie algebra g. Let n be a nilpotent ideal of g containing Œg; g and N the corresponding analytic subgroup of G. We take f 2 g and put f0 D f jn . Since n is stable by Ad.G/, G acts on n . We write as G.f0 / the stabilizer of f0 in G. Definition 5.1.18. Let p 2 P .f; G/. We say that p is n-admissible, if p \ nC 2 P .f0 ; N /. In addition, if p \ nC is stable by Ad .G.f0 //, we say that p is strongly n-admissible. Remark 5.1.19. If p \ nC is a maximal isotropic subspace relative to Bf0 , p is n-admissible. In these circumstances we write down two important theorems obtained in Auslander and Kostant [3]. Though we cannot go into details, the proof of the second theorem is based on the Mackey theory. Being a connected and simply connected nilpotent Lie group, N D exp n is of type I and its unitary dual NO is realized by the space n =N of coadjoint orbits of N , as we shall see later. Keeping this fact in mind, they consider the G-action on NO and apply the Mackey theory. Theorem 5.1.20. At any f 2 g , there exists an n-admissible p 2 P C .f; G/. Moreover, if p 2 P .f; G/ is n-admissible, p satisfies the Pukanszky condition. Theorem 5.1.21. We assume that f 2 g is integral and that p 2 P C .f; G/ is strongly n-admissible. Then H.f; f ; p; G/ ¤ f0g; .f; f ; p; G/ gives an irreducible unitary representation of G and its equivalence class does not depend on either p or n.

5.2 Exponential Solvable Lie Groups

125

At the end of this section we note one result which we shall need later. Definition 5.1.22. Let g be a real Lie algebra. A complex Lie subalgebra m of gC N D gC holds. is said to be totally complex, if m C m Theorem 5.1.23. ([11]) Let G be a connected Lie group with Lie algebra g and f 2 g . We suppose that p 2 P .f; G/ is totally complex and satisfies the Pukanszky condition. Then, if H.f; f ; p; G/ ¤ f0g, .f; f ; p; G/ is irreducible.

5.2 Exponential Solvable Lie Groups In this section we give the definition of the exponential solvable Lie groups and explain the orbit method for these groups.

5.2.1 Co-exponential Sequence Definition 5.2.1. Let g be a real solvable Lie algebra. We consider a series of ideals g D g0  g1      gm D f0g of g such that gi =gi C1 is abelian for every 0  i  m  1. Take for every j a subspace vj of gj such that gj D vj ˚ gj C1 . Hence g D ˚jm1 D0 vj : We say that .vj /j is a co-exponential sequence. As an example we can take a composition series .gi /i , since for any i , the algebra gi =gi C1 must be abelian, since otherwise its commutator is a non-trivial g-submodule. Definition 5.2.2. Let G be a Lie group with Lie algebra g. We write e X for the element exp X; X 2 g, of G. Applying .m  1/ times Proposition 1.1.42 we obtain: Proposition 5.2.3. Let g be a solvable Lie algebra and let G be a connected and simply connected solvable Lie group with Lie algebra g. Let v D .v0 ;    ; vm1 / be a co-exponential sequence. Then the mapping Ev W v ! GI Ev .v1 ;    ; vm1 / D e v1   e vm1 is a diffeomorphism.

126

5 Orbit Method

Proposition 5.2.4. Let G be a simply connected solvable Lie group and let B D .v0 ;    ; vm1 / be a co-exponential sequence. Let g WD ˚im1 D0 vi be the Lie algebra of G. We define on g a group multiplication vB v0 D .

0 .v; v

0

/;    ;

m1 .v; v

0

  // WD EB1 EB .v/EB .v0 / ; v; v0 2 g:

Then the functions i W g  g ! vi ; i D 0;    ; m  1 are smooth and i .v; v0 / depends only on the coordinates v0 ;    ; vi of v and on the coordinates v00 ;    ; v0i of v0 . In particular j ..0;    ; 0; vj /; .0;    ; 0; v0j // D vj C v0j for every j . Proof. It is clear that the coordinate functions i of the multiplication B are smooth, since the mapping EB is smooth. Let Bi WD .vj /ij D0 . Then Bi is a co-exponential sequence for the algebra g=gi C1 and the group G=Gi C1 , where Gi C1 D exp.gi C1 /. This shows that i does not depend on the coordinates vk ; k > i of v and also 0 not on the coordinates v0k ; k > i of v0 , since e vk and e vk are contained in Gi C1 if k > i .  Definition 5.2.5. Again let g be a solvable Lie algebra, let h be a subalgebra of g. We take a sequence of co-abelian ideals g D g0  g1      gm D f0g and the corresponding co-exponential sequence .vi /i . We consider the index set I g=h  f0;    ; m  1g consisting of all the i ’s for which h C gi 6 h C gi C1 : For every i 2 I g=h , we choose a subset wi  vi such that gi C h D wi ˚ gi C1 C h: Let w WD .wj /j 2I g=h . We say that the sequence w WD .wj /j 2I g=h is a co-exponential sequence with respect to h. In the following, if we use a co-exponential sequence B D .wj /j relatively to a subalgebra h, we always are given also the co-abelian sequence .gi /i of ideals in g, which passes through the nilradical, the sequence of subspaces .vi /i , such that gi D gi C1 ˚ vi for every 0  i  m  1 and the index set I g=h . Proposition 5.2.6. Let G be a connected and simply connected solvable Lie group with Lie algebra g and let h be a subalgebra of g. Then there exists a closed connected subgroup H of G with Lie algebra h. Let w D .wj /j 2I g=h be a coexponential sequence with respect to h. Then the mapping

5.2 Exponential Solvable Lie Groups

127

w  H ! GI ..wj /; h/ 7! EB ..wj //h D

Y

e wj h;

j 2I g=h

is a diffeomorphism. Proof. We proceed by induction everything is clear. P on dim g. If g is abelian, theng=h Suppose first that h  g1 D jm1 v . Then 0 is contained in I and w0 D v0 . By j D1 Proposition 5.2.3, there exists a closed normal subgroup G1 of G with Lie algebra g1 , such that the mapping w0  g1 ! GI .w; u/ 7! e w u is a diffeomorphism. Hence, using the induction hypothesis for g1 and h, we obtain the result for G and h. Suppose now that h is not contained in g1 . Choose a vector X 2 h n g1 and a subspace v0 of v0 , such that g D RX ˚ v0 ˚ g1 : Let g1 WD g1 C v0 ; h1 WD h \ g1 and apply the induction hypothesis to g1 ; h1 and to the closed connected normal subgroup G 1 of G with Lie algebra g1 , which exists by Proposition 1.1.42. We then know that there exists a closed connected subgroup 0 H1 in G 1 , such that the mapping Ew from w  H1 onto G 1 is a diffeomorphism. Applying again Proposition 1.1.42, we can conclude that the mapping E W R  w  H1 ! GI .s; .Vj /; h1 / 7! e sX

Y

e Vj h1 ;

j

is a diffeomorphism. The subset H D fe sX h1 ; s 2 R; h1 2 H1 g is then closed in G and it is also a subgroup, since h1 is an ideal in h. The Lie algebra of H is evidently isomorphic to h. Since the mapping R  G 1 ! R  G 1 I .s; u/ 7! .s; e sX ue sX / is a diffeomorphism, it follows that the mapping R  w  H1 ! GI .s; .Vj /; h1 / 7!

Y e Vj h1 e sX ; j

is a diffeomorphism too. The mapping R  H1 ! H I .s; h1 / 7! h1 e sX being also a diffeomorphism, we see that the proposition is true.  Corollary 5.2.7. Let G D exp g be a simply connected solvable Lie group, let H D exp h be a closed subgroup of G and let A D exp a be a closed normal subgroup of G. Then HA is a closed subgroup too. Proof. We take a composition sequence .gj / of g, which passes through a, i.e. such that gj D a for some j . Then if we put wj WD ˚k2I g=h ;kj wk ;

128

5 Orbit Method

we have that AH D EB .wj /H 

and so AH is a closed subgroup of G.

Proposition 5.2.8. Let G D exp g be a simply connected solvable Lie group and let H D exp h be a closed subgroup of G. Choose a co-exponential sequence B D .wj /j of g relatively to h associated to a co-abelian sequence .gi /i such that gk D Œg; g for some k. For any X 2 h and w D .wj /j 2 w D ˚j wj , we have that Y e exp.j .X //wj Cpj .X;w0 ; ;wj 1 / qj .X; w0 ;    ; wj / mod H; e X EB .w/e X D j

(5.2.1)

where qj .X; w0 ;    ; wj / 2 Gj C1 and pj .X; w0 ;    ; wj 1 / is in wj if j  k, and pj D 0; qj 2 gk if j < k and where j .X / is the endomorphism defined by ad.X / on the space wj Š gj C h=gj C1 C h. Proof. We have for g D exp X 2 G, that ge wj g 1 D e exp.j .X //wj modulo Gj C1 \ ŒG; G. Hence, using Proposition 5.2.4, we see that ge wj g 1 mod.Gj C1 H / D e exp.j .X //wj

mod.hCgj C1 /

mod.Gj C1 H /: 

Let H be a closed connected subgroup of the simply connected solvable Lie group G. We can now explicitly describe the invariant positive linear functional H which exists on the functional space E.G=H / by Proposition 3.1.6. G=H Proposition 5.2.9. Let G D exp g be a simply connected solvable Lie group and let H D exp h be a closed connected subgroup of G. Let B D .wj /j be a co-exponential sequence relative to h and w D ˚j wj . Then the positive linear functional defined on E.G=H / by I

Z

'.t/d tP; ' 2 E.G=H /;

'.EB .w//d w DW

' 7!

G=H

w

is G-invariant. Proof. We proceed by induction on dim g. If G is abelian, the proposition is evident. Suppose first that h  g1 . Then 0 2 I g=h . Let w1 WD

X j 2I g=h ;j >0

wj :

5.2 Exponential Solvable Lie Groups

129

We apply the induction hypothesis to G1 D exp.g1 /, to H and to B1 WD .wj /m1 j D1 . Since G1 is a normal closed subgroup, for every g 2 G, we have that G1 ;H D G;H jG1 and so the function 'g .u/ WD '.gu/; u 2 G1 , is in E.G1 =H /. Therefore for g 2 G1 , by the induction hypothesis, Z

Z

Z '.gEB .w//d w D

w

Z

w0

Z

w0

Z

w1

D Z

w1

D

'.ge w0 EB1 .w1 //d w0 d w1 '.e w0 .e w0 ge w0 /EB1 .w1 //d w1 d w0 Z '.e EB1 .w //d w d w0 D w0

w1

w0

1

1

'.EB .w//d w: w

0

0

0

For g D e w0 with w00 2 w0 D v0 , we have that e w0 e w0 D e w0 Cw0 q.w0 ; w00 / where q.w0 ; w00 / is an element of G1 . Hence, Z

Z

Z '.gEB .w//d w D

w

Z

w0

Z

w1

0

D Z

w0

Z

w1

D w0

'.ge w0 EB1 .w1 //d w0 d w1

w1

'.e w0 Cw0 .q.w0 ; w00 //EB1 .w1 //d w0 d w1 Z '.e w0 EB1 .w1 //d w0 d w1 D

'.EB .w//d w: w

Suppose now that h 6 g1 . Choose a vector X 2 h n g1 and a subspace v0 of v0 , such that g D RX ˚ v0 ˚ g1 . Let g1 WD g1 C v0 ; h1 WD h \ g1 and apply the induction hypothesis to g1 ; h1 and to the closed connected normal subgroup G 1 D exp.g1 / of G. We have now that E.G=H / is isomorphic to E.G 1 =H1 / with H1 D exp.h1 /, an example of an isomorphism being the restriction mapping ' 7! 'jG 1 . Hence, it suffices to see that our integral is invariant under exp.sX / (recall that G D exp.RX /G 1 /. Now by (5.2.1) Z '.e X EB .w//d w w

Z D

' w

 Y j

  e exp.j .X //wj Cpj .X;w0 ; ;wj 1 / qj .X; w1 ;    ; wj / e X d w

Z

D H;G .e X /

' w

Y j

 e exp.j .X //wj d w

130

5 Orbit Method

Z

e

D e Tr.ad.X /g=h /

P j

Tr.j .X /.vj =vj \h/ /

'

Y

w

j

Z D

 e wj d w

'.EB .w//d w; w

since

P j

Tr.j .X /.vj =vj \h/ / D Tr.ad.X /g=h /.



Corollary 5.2.10. Let G be a simply connected solvable Lie Group with Lie algebra g. Let B D .vj /j be an co-exponential sequence for g. Then the linear functional Z ' 7!

'.EB .v//d v g

is a left Haar measure on G.

5.2.2 The Exponential Mapping for Solvable Lie Groups Definition 5.2.11. Let G be a simply connected solvable Lie group with Lie algebra g and let B D .vi /im1 D0 be a co-exponential sequence of g associated to a composition series .gj /j , such that gk D Œg; g for some k. We identify g with v D ˚im1 D0 vi and we write exp v D .˛0 .v/;    ; ˛m1 .v//; v 2 v: Then for any X 2 vj ; t 2 R, we have that exp..0;    ; 0; tX; 0;    ; 0// D .0;    ; 0; tX; 0;    ; 0/: We say that a solvable Lie group G with Lie algebra g is exponential, if the exponential mapping exp W g ! G is a diffeomorphism. We remark that an exponential group is automatically connected and simply connected. We say that a solvable Lie algebra is exponential if for every X 2 g, the spectrum of the endomorphism ad.X / of gC does not contain a complex number of the form i b; b 2 R . Example 5.2.12. 1. A connected and simply connected nilpotent Lie group is an exponential solvable Lie group. 2. Let g be a Lie algebra such that dim g D n. When there is a sequence of ideals f0g D g0  g1      gn1  gn D g; dim.gj / D j .0  j  n/; we say that g is completely solvable. A completely solvable Lie algebra is an exponential solvable Lie algebra.

5.2 Exponential Solvable Lie Groups

131

3. Any subalgebra and any quotient algebra of an exponential solvable Lie algebra is exponential. Proposition 5.2.13. A solvable Lie algebra g is exponential if and only if every root  of the g-module gC is of the form  D .1 C i/, where  W g ! R is real character of g and  is some real number. Proof. Every composition series ..gC /j /j of the g-module gC has the property that the simple g-module .gC /j =.gC /j C1 is one-dimensional and gives us a complex character j of g for j D 0;    ; dim.gC /  1. Choosing a Jordan–Hölder basis Z D .Zj 2 .gC /j n .gC /j C1 /; we have that for every X 2 g, the matrix of the endomorphism ad.X / is upper triangular and has as diagonal entries the numbers j .X /. Hence the spectrum of ad.X / is the set of numbers fj .X /I j D 0;    ; dim g  1g. This proves the assertion of the proposition.  Example 5.2.14. The group E.2/. Let  J WD

 0 1 : 1 0

and let g.2/ be the Lie algebra, whose underlying vector space is R  R2 with the bracket relation Œ.s; .a; b//; .s 0 ; .a0 ; b 0 // D .0; s 0 .a; b/J t s.a0 ; b 0 /J t / D .0; .s 0 bsb 0 ; sa0 s 0 a//: Namely, g.2/ D hX; P; QiRI ŒX; P  D Q; ŒX; Q D P with X D .1; .0; 0//; P D .0; .0; 1//; Q D .0; .1; 0//: In matrix form, 0 1 00 0 10 X D @1 0 0A ; P D @0 0 00 0 00 0

1 0 1 000 1 0A ; Q D @0 0 1A ; 000 0

The corresponding simply connected Lie group G.2/ is the space R  R2 equipped with the multiplication 0

.s; .a; b//.s 0 ; .a0 ; b 0 // D .s C s 0 ; .a; b/e s J C .a0 ; b 0 // D .s C s 0 ; .a cos s 0  b sin s 0 C a0 ; a sin s 0 C b cos s 0 C b 0 //:

132

5 Orbit Method

The exponential mapping can be explicitly computed. We have for s 2 R and u 2 R2 that   e sJ  I2 : exp.s; u/ D s; u  sJ Indeed, for t; t 0 2 R, we have that   0 e s.t Ct /J  I2 0 exp..t C t /.s; u// D .t C t /s; u  sJ 0

and    0 e t sJ  I2 e t sJ  I2 0 t s; u  exp.ts; tu/exp.t s; t u/ D ts; u  sJ sJ   0  e t sJ  I2  t 0 sJ e t sJ  I2 0 D .t C t /s; u  e Cu sJ sJ   0 0 0 e s.t Ct /J  e t sJ C e t sJ  I2 0 D .t C t /s; u  sJ   s.t Ct 0 /J e  I2 0 : D .t C t /s; u  sJ 0

0

Hence the mapping t 7! .t/ WD exp.ts; tu/ is a homomorphism of R into G.2/ and if we differentiate in t we see that d D .s; u/: This shows that we have found the exponential mapping. But if s D 2k with k 2 Z , then we see that     I2  I2 e 2kJ  I2 D 2k; u D .2k; 0/ exp.2k; u/ D 2k; u  2kJ 2kJ for every u 2 R2 . Hence the exponential mapping is not injective and not surjective. Furthermore the Lie algebra g.2/ is not exponential since the spectrum of the vector T D .1; 0R2 / is the set f0; i; i g. Remark 5.2.15. Let G be a Lie group, A D exp a be a closed normal subgroup of G. Then exp.t.X C Y // 2 exp.tX /A for every t 2 R; Y 2 a. Indeed, the mappings t 7! exp.tX /A and t 7! exp.t.X C Y //A are continuous homomorphisms from R into G=A, whose differentials are equal to X mod a resp. to X C Y moda. Hence these two homomorphisms coincide. Theorem 5.2.16 (cf. [18]). A connected and simply connected solvable Lie group G with Lie algebra g is exponential , exp is injective , exp is surjective , the Lie algebra g is exponential.

5.2 Exponential Solvable Lie Groups

133

Proof. We proceed by induction on the dimension of G. Every simply connected abelian Lie group and every abelian Lie algebra is exponential. As before, let .gj /m j D0 be a composition series of g. Let a WD gm1 D vm1 and consider m2 Q gQ D ˚jm2 D0 vj ' g=a, G D ˚j D0 vj ' G=A, where A D exp a. If we write Q the group G as G D G  a, then the multiplication in G has the following form. For Q a; a0 2 a, we have that u; u0 2 G; 0

.u; a/G .u0 ; a0 / D .uGQ u0 ; e .u / a C a0 C p.u; u0 //; Q GQ ! a is a smooth function. We can express the exponential mapping where p W G exp in terms of the root  D m1 associated to vm1 and of the exponential Q For u 2 gQ , we have mapping exp W gQ ! G.

e

  exp.u; 0a / D exp.u/; q.u/ ;

e

for some smooth mapping q W gQ ! a, by Remark 5.2.15, and we obtain the following expression for u 2 gQ ; a 2 a   e .u/  Ida .a/ C q.u/ ; exp.u; a/ D exp.u/; .u/

e

(5.2.2)

for some C 1 mapping q W gQ ! a. It follows from the property of the exponential mapping that q..t Ct 0 /u/ D e t

0 .u/

e

e

.q.tu//Cq.t 0 u/Cp.exp.tu/; exp.t 0 u//; t; t 0 2 R;

(5.2.3)

d q.tu/jt D0 D 0. Indeed, if we check the properties of these mappings, we and that dt see that for t; t 0 2 R,

exp.tu; ta/exp.t 0 u; t 0 a/ 0    e t .u/  Ida e t .u/  Ida 0 D exp.tu/; .a/ C q.tu/ exp.t u/; .a/ C q.t 0 u/ .u/ .u/  e t .u/  Ida 0 .a/ D exp..t C t 0 /u/; e t .u/  .u/ 0  e t .u/  Ida t 0 .u/ Ce q.tu/ C .a/ C q.t 0 u/ C p.exp.tu/; exp.t 0 u// .u/ 0  e .t Ct /.u/  Ida 0 .a/ D exp..t C t /u/; .u/  0 C e t .u/ q.tu/ C q.t 0 u/ C p.exp.tu/; exp.t 0 u//

e

e

e

e

e

e

e

e

134

5 Orbit Method

0   e .t Ct /.u/  Ida .a/ C q..t C t 0 /u/ : D exp..t C t 0 /u/; .u/

e

Together with (5.2.3), this shows that (5.2.2) describes the exponential mapping of G. Suppose first that the spectrum of  does not contain a number of the form i a with a 2 R . Then the linear mapping e .u/  Ida Wa!a .u/ is invertible for every u 2 gQ . This implies that exp is injective resp. surjective resp. a diffeomorphism, if and only if exp Q is injective, resp. surjective, resp. a diffeomorphism. Furthermore, if exp Q is not injective, resp. not surjective, then so is exp. Suppose now that the spectrum of  contains a number of the form i a with a 2 R , which means that the mapping e .u/ w is a rotation for every u 2 g. This .2ku0 / a is 0 for every gives us an element u0 2 g such that the linear map e .2kuId 0/  k 2 Z . Hence

e

exp.2ku0 ; a/ D exp.2ku0 ; 0/ D .exp.2ku0 /; q.2ku0 //

e

for every k 2 Z and .exp.2ku0 /; a C q.2ku0 // is not contained in the image of exp for every a 2 a; a ¤ 0.  Remark 5.2.17. Let G be an exponential group. Then we can define a group multiplication on the Lie algebra g of G by using the exponential mapping. For X; Y 2 g, let 1 X g Y WD exp1 .exp X G exp Y / D X C Y C ŒX; Y  C    2 according to the Campbell–Baker–Hausdorff formula. It follows that the exponential mapping .g; Œ; g / ! .g; g / is the identity and so every subalgebra of g is a closed connected subgroup of .g; g / and every closed connected subgroup of .g; g / is also a subalgebra of g. For X; Y 2 g we have that X g Y g .X / D e ad.X / .Y /: Lemma 5.2.18. Let H be a closed proper subgroup of the group .R; C/. Then there exists a 0 < h0 2 H such that H D Zh0 . Proof. Let h0 WD inff0 < h 2 H g. If h0 D 0, then for every " > 0, there exists h" 2 H such that 0 < h"  ". Let x 2 R. There exists k 2 Z, such that x 2 Œkh" ; .k C1/h" Œ. Since kh" 2 H , we have that the distance of x to H is less or equal to ". Hence H D R. Since H is proper, h0 must be strictly positive. Let x 2 H ,

5.3 Kirillov–Bernat Mapping

135

such that x 62 Zh0 . Then there exists k 2 Z, such that x 2kh0 ; .k C 1/h0 Πand then 0 < x  kh0 < h0 and x  kh0 2 H . This relation contradicts the minimality of h0 . Hence H D Zh0 . 

5.3 Kirillov–Bernat Mapping From Sect. 5.2.1 an exponential solvable Lie group G D exp g is equipped with the following useful property: being given a Lie subalgebra h of g, there is a basis fX1 ; : : : ; Xp g of a subspace of g complementary to h such that the mapping .t1 ; : : : ; tp ; X / 7! g1 .t1 /  gp .tp /exp X with gj .t/ D exp.tXj / .t 2 R/ is a diffeomorphism from Rp  h onto G. Such a basis is said to be coexponential in g to h and is constructed as follows. First of all, k being a Lie subalgebra containing h, we may put together a coexponential basis in g to k and that in k to h to make a coexponential basis in g to h. Thus it suffices to consider the following cases: (1) h is an ideal of codimension 1 in g; (2) h is not an ideal and g=h is an irreducible h-module. In case (1), any element of g not belonging to h gives a coexponential basis. Hence, if g is nilpotent, every Lie subalgebra of codimension 1 is an ideal so that we can construct by iteration of case (1) a coexponential basis to any Lie subalgebra. Next we consider case (2). Let n be the maximal nilpotent ideal of g. Because of n 6 h, n=.h\n/ is identified with a non-trivial h-submodule of g=h, hence with g=h. Since every subspace containing n is an ideal of g, we can construct by the iteration of case (1) a coexponential basis in g to n, all vectors of which are supposed by hypothesis taken in h. Applying case (1), we can construct a coexponential basis fX g or fX1 ; X2 g, depending on the dimension, in n to h \ n. It is obvious that this is also coexponential in g to h. Definition 5.3.1. Let G be a Lie group with Lie algebra g and V an G-module or g-module. We say that V is of exponential type if every weight of g in V is written X 7! .X /.1 C i ˛/ with ˛ 2 R;  2 g . Theorem 5.3.2. Let G D exp g be an exponential solvable Lie group (with Lie algebra g) and V an G-module of exponential type. Then the stabilizer in G of any point of V is connected. Proof. We denote by  the action of G on V . For X 2 g; v 2 V suppose .exp X /v D v. The set of t 2 R satisfying .exp.tX //v D v is a closed

136

5 Orbit Method

subgroup of R. If this is discrete (¤ f0g), we take its minimal positive element t0 . Then:             t0  t0  t0   exp X v  v D   exp X v  v ¤ 0:  exp X 2 2 2 From this d

 t0 2

 X has an eigenvalue i n for some nonzero integer n.



Remark 5.3.3. This theorem implies that the stabilizer G.`/ in G under the coadjoint representation of an element ` 2 g is connected and hence G.`/ D exp.g.`/ /. We can parametrize G-orbits in this situation and get the following lemma. Lemma 5.3.4. Let G be an exponential solvable Lie group and V an G-module of exponential type. We denote by G.v/ the stabilizer in G of a point v 2 V and introduce on the orbit Gv the relative topology from V . Then Gv is homeomorphic to the homogeneous space G=G.v/. These spaces are homeomorphic to Rd with some non-negative integer d . Hereafter in this section G D exp g denotes an exponential solvable Lie group with Lie algebra g. Let f 2 g . We define an alternating bilinear form Bf on g  g by the formula Bf .X; Y / D f .ŒX; Y /. We denote by S.f; g/ (resp. M.f; g/) the set of Lie subalgebras which are isotropic, i.e. self-orthogonal (resp. maximal isotropic) subspaces for Bf . We call elements of M.f; g/ (real) polarizations of g at f . Now h 2 S.f; g/ being given, we define a unitary character f of H D exp h by the formula f .exp X / D e if .X / .8 X 2 h/; and construct the induced representation .f; O h; G/ D indG H f : O We denote by H.f; h; G/ the Hilbert space of .f; O h; G/. Finally, we denote by I.f; g/ the set of h 2 S.f; g/ such that .f; O h; G/ is irreducible. Definition 5.3.5. Let G be a locally compact group and let .; H/ be an irreducible unitary representation of G. The projective kernel of , denoted by the symbol pker./ is the subgroup pker./ WD fs 2 GI .s/ D IdH ; for some  2 Cg: Proposition 5.3.6. Let G be a locally compact group and let .; H / be an irreducible representation of G. Then K WD pker./ is a closed normal subgroup of G and there exists a unitary character  of K, such that .k/ D .k/IdH for

5.3 Kirillov–Bernat Mapping

137

any k 2 K. If G is a Lie group, then there exists a G-invariant linear functional ` on the Lie algebra k of K, such that .e U / D e i `.U / IdH for every U 2 k. Proof. It is clear that K is closed, since  is strongly continuous. If k is in K and g 2 G, then .k/ D .k/IdH for some .k/ 2 T and so .gkg 1 / D .g/ ı .k/ ı .g/1 D .g/ ı .k/IdH ı .g/1 D .k/IdH : This shows that K is normal. Since for k; k 0 2 K, .k/.k 0 /IdH D .k/ ı .k 0 / D .kk 0 / D .kk 0 /IdH ; we see that the function k 7! .k/ is a continuous homomorphism and hence a continuous unitary character of K. From what we saw above this character is Ginvariant. If G is a Lie group, then the restriction of the G-invariant unitary character  of K to its connected component defines the G-invariant linear functional ` on k.  Lemma 5.3.7. The projective kernel of .`; O h; G/; h 2 I.`; g/; is the largest closed normal subgroup A` of G, which is contained in the stabilizer G.`/ of ` under the coadjoint representation of G. Its Lie algebra is the largest ideal a` contained in g.`/ and for every s 2 pker..`; O h; G//, .`; O h; G/.s/ D ` .s/IdH.`;h;G/ . O Proof. Let s 2 A` . Then for ' 2 E.G=H; ` / and g 2 G, we have that .`; O h; G/.s/'.g/ D '.s 1 g/ D '.g.g 1 s 1 g// D ` .g 1 sg/'.g/; since A` is a normal subgroup contained in G.`/  H by (2) of the next theorem. Let us write s D e S , with S 2 a` . Since a` is an ideal of g, we see that h`; ad.X /k .S /i D h`; ŒX; ad.X /k1 .S /i D 0 .X 2 g/ for all Z 3 k > 0, since then ad.X /k .S / 2 g.`/ . Hence for g D e X 2 G, we see that ` .g 1 sg/ D e i h`;Ad.g

1 /S i

D ei

P1

1 j j D0 j Š h`;ad .X /S i

D e i `.S / D ` .s/:

This shows that .`; O h; G/.s/ D ` .s/IdH.`;h;G/ and so A`  pker..`; O h; G//. O S O h; G//, then If conversely s D e 2 pker..`; '.s 1 g/ D .s/'.g/ for all ' 2 E.G=H; ` / and all g 2 G and so if S 62 h, we can use Proposition 5.2.6 and choose ' in E.G=H; ` / with '.e/ D 1 and we see that for k 2 N big enough, 0 D '.e kS / D '.s k / D .s/k '.e/ D .s k /:

138

5 Orbit Method

This contradiction implies that S 2 h. Since for any t 2 pker..`; O h; G// .`; O h; G/.g/ ı .`; O h; G/.t/ ı .`; O h; G/.g/1 D .`; O h; G/.g/ ı .t/IdH.`;h;G/ ı .`; O h; G/.g/1 D .t/IdH.`;h;G/ ; O O we have that pker..`; O h; G// is a closed normal subgroup, hence g 1 sg 2 H for any g 2 G and therefore .`; O h; G/.s/'.g/ D '.g.g1 s 1 g// D ` .g 1 sg/'.g/ D .s/'.g/; for ' 2 E.G=H; ` /; g 2 G. Hence e i h`;Ad.exp.tX //S i D ` .e tX se tX / D .s/ for any X 2 g and t 2 R. Hence differentiating this equation at t D 0, we find that i h`; ŒX; S ie i `.S / D 0; X 2 g: This relation tells us that S 2 g.`/. Hence O h; G//; A` D pker..`; T

since A` D

g2G gG.`/g

1

.



Theorem 5.3.8 ([10], Chap.VI). Let G D exp g be an exponential solvable Lie group with Lie algebra g and let f 2 g . Then: (1) I.f; g/ ¤ ;; (2) I.f; g/  M.f; g/; (3) for h1 ; h2 2 I.f; g/, .f; O h1 ; G/ ' .f; O h2 ; G/. From now on we aim to prove this theorem following [10]. When m is a linear subspace of g, we put mf D fX 2 gI f .ŒX; m/ D f0gg: For X 2 m, X f .mf / D f .Œmf ; X / D f0g so that mf  .mf /? . Further, dim.mf / D dim m  dim.m \ g.f // D dim g  dim.mf / so that mf D .mf /? . We denote by z the centre of g.

5.3 Kirillov–Bernat Mapping

139

Lemma 5.3.9. Let g be a solvable Lie algebra, .V; / an irreducible g-module of exponential type and h a Lie subalgebra of g. Then dim V  2. The representation jh is either trivial or irreducible and the codimension of its kernel in h is smaller than or equal to 1. Proof. By Lie’s theorem d D dim V  2. If d D 1, the lemma is clear. If d D 2,  is defined by two weights of the form .1 ˙ i ˛/;  2 g ; 0 ¤ ˛ 2 R. If jh D 0, jh is trivial. If not, jh is irreducible and its kernel is h \ ker.  Lemma 5.3.10. Let g be an exponential solvable Lie algebra and a its minimal non-central ideal, i.e. minimal among non-central ideals. Then a is commutative and dim.a=.a \ z//  2: Proof. By definition the g-module a=.a \ z/ is irreducible. By the previous lemma d D dim.a=.a \ z//  2 and Œa; a  a \ z. If d D 1, the lemma is clear. Assume d D 2 in what follows. aC is spanned by two vectors A; AN linearly independent over .a \ z/C , and we may assume that these vectors are eigenvectors in aC of the g-module aC =.a \ z/C . Then there exist X 2 g and 0 ¤ ˛ 2 R such that N D .1  i ˛/AN ŒX; A D .1 C i ˛/A; ŒX; A modulo .a \ z/C . Hence, N D .1 C i ˛/ŒA; A N C .1  i ˛/ŒA; A N D 2ŒA; A: N 0 D ŒX; ŒA; A



We denote by m.f; g/ the dimension of the elements of M.f; g/, namely g C dim.g.f ///.

1 2 .dim

Lemma 5.3.11. Let g be an exponential solvable Lie algebra and a its minimal non-central ideal. Then m.f; g/ D m.f; af / and f C .af /?  Gf . Proof. We set f 0 D f jaf and let p  af be a maximal isotropic subspace with respect to Bf 0 . Then Bf 0 .p C a; p C a/ D f .Œp C a; p C a/  f .Œaf ; a/ D f0g: 0

Hence p C a D p  a and pf  af . Therefore p D pf D pf , and we get the first assertion. Since a is commutative, .exp X /f D f C X f for any X 2 a. Setting A D exp a, f C .af /? D f C af D Af  Gf:



Lemma 5.3.12. Let g be an exponential solvable Lie algebra, a a minimal noncentral ideal of g and f 2 g ; h 2 S.f; g/. We set h0 D h \ af ; h0 D h0 C a; k D h C a.

140

5 Orbit Method

(1) dim.h=h0 /  dim.a=.h \ a//. (2) h0 2 S.f; g/ and dim.h0 /  dim h. Hence, if h 2 M.f; g/, then the equality holds. (3) dim h D dim.h0 / if and only if dim.h=h0 / D dim.a=.h \ a//. (4) If dim h D dim.h0 /, passing to the quotient spaces, Bf induces a bilinear form B W h=h0  a=.h \ a/ ! R. B gives the duality between h=h0 and a=.h \ a/ and induces an isomorphism as h0 -modules from h=h0 onto .a=.h \ a// . (5) If dim h D dim.h0 /, Tr adh0 =h0 X D

 1 Tr adk=h X  Tr adk=h0 X 2

for all X 2 h0 . Proof. As f .Œh; h \ a/ D f0g D f .Œh0 ; a/, Bf defines by passing to the quotient spaces the bilinear form B W h=h0  a=.h \ a/ ! R and hence a linear mapping BQ from h=h0 to .a=.h \ a// . XQ 2 h=h0 being the image of X 2 h, Q XQ / D 0 ” f .ŒX; a/ D f0g ” X 2 h \ af D h0 ” XQ D 0: B. Thus BQ is injective and the assertion (1) follows. Since a  af , h0 \ a D h \ af \ a D h \ a. Hence dim.h0 / D dim.h0 / C dim.a=.h0 \ a//  dim.h0 / C dim.h=h0 / D dim h: Since Œh0 ; h0   Œh; h C Œaf ; a, h0 2 S.f; g/. In this fashion we get the claims (2) and (3). In what follows we suppose that dim h D dim.h0 /. As we have seen above, BQ becomes bijective and B furnishes a duality between h=h0 and a=.h \ a/. Since Œh0 ; h \ a  h \ a and Œh0 ; h0   h0 , h=h0 and a=.h \ a/ are h0 -modules. Let X0 2 h0 ; X 2 h; Y 2 a and let XQ 2 h=h0 ; YQ 2 a=.h \ a/ be respectively the images of X; Y . Then B.X0 XQ ; YQ / C B.XQ ; X0 YQ / D f .ŒŒX0 ; X ; Y / C f .ŒX; ŒX0 ; Y / D f .ŒX0 ; ŒX; Y / 2 f .Œaf ; a/ D f0g and the assertion (4) follows. Furthermore, since h0 =h0 D .h0 C a/=h0 Š a=.h \ a/ D a=.h0 \ a/ as h0 -modules, taking the assertion (4) into account Tr adh0 =h0 X0 D Tr ada=.h\a/ X0 D Tr adh=h0 X0 :

5.3 Kirillov–Bernat Mapping

141

By the way, since Tr adk=h0 X0 D Tr adk=h X0 C Tr adh=h0 X0 D Tr adk=h0 X0 C Tr adh0 =h0 X0 ; we have  1 Tr adk=h X0  Tr adk=h0 X0 2  1 D Tr adh0 =h0 X0  Tr adh=h0 X0 D Tr adh0 =h0 X0 : 2



Lemma 5.3.13. Let a be a minimal non-central ideal and h a Lie subalgebra containing z. Then dim.a=.h \ a//  2. If dim.a=.h \ a// D 2, h \ a D z \ a and the action of g on a=.h \ a/ is irreducible. Proof. By hypothesis h  h \ a  z \ a. By the way, since the action of g on a=.z \ a/ is irreducible, dim.a=.h \ a//  2 and the equality holds only when h \ a D z \ a.  Lemma 5.3.14. We keep the notations in Lemma 5.3.12. Let a be a minimal noncentral ideal and suppose h  z. Let  be the action of h on a=.h \ a/ and denote its kernel by j. (1) The action  is either trivial or irreducible. (2) j and j C h0 are ideals of h. (3) j C h0 coincides with either h or h0 and the first possibility occurs if dim .h=h0 / D 2. (4) Œj; j  h0 \ j. Proof. By the previous lemma d D dim.a=.h \ a//  2. Concerning the assertion (1), it is clear if d D 1 and comes from the previous lemma if d D 2. In any way the codimension of j in h is smaller than or equal to 1, j  Œh; h and .h/ is commutative. These give (2). We show (3). If h0 6 j, h D h0 C j. If h0  j and if dim.h=h0 / is either 0 or 1, j coincides with either h0 or h. It remains the case where h0  j and dim.h=h0 / D 2. In this case j ¤ h0 , so it suffices to show j D h. Now suppose j ¤ h. By (1)  is irreducible. By Lemma 5.3.12 2 D dim.h=h0 /  dim.a=.h \ a//  2 and B provides a duality between h=h0 and a=.h \ a/. The orthogonal space of j=h0 with respect to B is the image in a=.h \ a/ of the subspace a.j/ D fY 2 aI f .ŒY; j/ D f0gg of a. Hence by duality h0 ¨ j ¨ h implies h \ a ¨ a.j/ ¨ a. Now, in order to arrive at a contradiction, it is enough to see that a.j/ is an h-module. Indeed, since Œj; h  j and Œj; a  h \ a, f .Œj; Œh; a.j// D f .ŒŒj; h ; a.j// C f .Œh; Œj; a.j//  f .Œj; a.j// C f .Œh; h \ a/ D f0g: Thus Œh; a.j/  a.j/.



142

5 Orbit Method

Lemma 5.3.15. Let G D exp g be an exponential solvable Lie group with Lie algebra g. For X1 ; X2 2 g, exp.X1 /exp.X2 / D exp.X1 C X2 /exp Y D exp.Y 0 /exp.X1 C X2 / with some Y; Y 0 2 Œg; g. Proof. Since the group G is exponential, the commutator subgroup ŒG; G coincides with exp.Œg; g/. Let p W G ! G=ŒG; G; q D dp W g ! g=Œg; g be canonical projections. Since these projections commute with the exponential maps and the group G=ŒG; G is commutative, p.exp.X1 C X2 // D exp.q.X1 / C q.X2 // D exp.q.X1 //exp.q.X2 // D p.exp.X1 /exp.X2 //: That is, exp.X1 /exp.X2 /exp..X1 C X2 // 2 ŒG; G D exp.Œg; g/.



Lemma 5.3.16. We use the notations of Lemma 5.3.12. Let w be a linear subspace of a complementary to h \ a. Then, k D h ˚ w and the mapping .X; Y / 7! .exp X /.exp Y / is a diffeomorphism from h  w onto K. Proof. Put H D exp h; K D exp k. If we write as ' the mapping in question, ' is analytic. Let us see that ' is bijective. Since a is a commutative ideal of k, K D HA and A D exp.h \ a/exp w. Hence K D H exp w so that ' is surjective. Next if exp.X1 /exp.Y1 / D exp.X2 /exp.Y2 / for .Xj ; Yj / 2 h  w .j D 1; 2/, since a is commutative and h is exponential, there is X 2 h such that exp.Y1  Y2 / D exp.Y1 /exp.Y2 / D exp.X1 /exp.X2 / D exp X: Hence Y1  Y2 2 h \ w D f0g and .X1 ; Y1 / D .X2 ; Y2 /. Finally, we show that ' 1 is analytic. For k 2 K we put ' 1 .k/ D . .k/; .k//;  D ı exp W k ! h;  D  ı exp W k ! w. By definition exp X D exp. .X //exp. .X // for X 2 k. Now, H being a closed subgroup of K, the canonical projection p W K ! H nK is analytic and, if we identify H nK with expw, p.exp X / D exp . .X //. Thus, since the exponential map is a diffeomorphism, the mapping exp ı  W X ! .exp X /p.exp X /1 and ; ; ' 1 are analytic.  Lemma 5.3.17. We use the notations of Lemma 5.3.14. (1) Suppose that h D h0 C j. Let t be a linear subspace of h complementary to h0 and contained in j. Then the mapping .T; X / ! .exp T /.exp X / is a diffeomorphism from t  h0 onto H D exp h.

5.3 Kirillov–Bernat Mapping

143

(2) Suppose that j D h0 ¤ h, and let X1 2 hnh0 : Then the mapping .t; X0 / 7! exp.tX1 /exp.X0 / is a diffeomorphism from R  h0 onto H . Proof. If j D h0 ¤ h, j is an ideal of codimension 1 and the assertion (2) is well known. So, assume h D h0 C j and denote by ' the mapping in the statement of (1). ' is analytic and, if it is shown to be bijective, ' 1 turns out to be analytic as seen in the latter part of the proof of the previous lemma. Let exp.T1 /exp.X1 / D exp.T2 /exp.X2 / for .Tj ; Xj / 2 t  h0 .j D 1; 2/. From Lemma 5.3.14(4) and Lemma 5.3.15 there is X0 2 h0 so that exp.X2 /exp.X1 / D exp.T2 /exp.T1 / D exp.T1  T2 /exp.X0 /: Hence T1  T2 2 h0 \ t D f0g and ' is injective. In h, h0 is a Lie subalgebra, j is an ideal and h D h0 C j, hence H D .exp j/exp.h0 /. To show from this that ' is surjective, it suffices to show that .exp t/exp.h0 \ j/, which generates exp j, is a subgroup. In fact H D .exp t/exp.h0 \ j/exp.h0 / D .exp t/exp.h0 /: Now we take .Tj ; Xj / 2 t  .h0 \ j/ .j D 1; 2/. Again from Lemma 5.3.14(4) and Lemma 5.3.15 exp.T1 /exp.X1 /exp.X2 /exp.T2 / D exp.T1 /exp.X3 /exp.T2 / D exp.T1 /exp.T2 /exp .exp.T2 /X3 / D exp.T1  T2 /exp.X4 /exp.X5 / D exp.T1  T2 /exp.X6 /; here Xj 2 h0 \ j .3  j  6/.

 

Proposition 5.3.18. Let a be a minimal non-central ideal and f 2 g ; h 2 S.f; g/, and assume h  z. Set h0 D h \ af ; h0 D h0 C a; k D h C a:

144

5 Orbit Method

Further, we denote by j the kernel of the action of h on a=.h \ a/. We suppose that dim.h=h0 / D dim.a=.h \ a// and also h D j C h0 . We denote by t a linear subspace of h complementary to h0 and contained in j, and by w a linear subspace of a complementary to h \ a. Then, (1) the restriction of Bf to t  w is non-degenerate, (2) the mapping .T; X; Y / 7! .exp T /.exp X /.exp Y / is a diffeomorphism from t  h0  w onto K D exp k, (3) .f; O h; K/ ' .f; O h0 ; K/. Proof. The subspaces t; w are respectively identified with h=h0 ; a=.h \ a/ and the claim (1) follows from Lemma 5.3.12. Besides, the claim (2) follows from the last two lemmas. Also from this the mapping .X; Y / 7! .exp X /.exp Y / is a diffeomorphism from h0  w onto H 0 D exp.h0 /. To simplify the notations, we set O h0 ; K/;  D .f; O h; K/; 0 D .f; O O h0 ; K/; H D H.f; h; K/; H0 D H.f; ı D H;K ; ı 0 D H 0 ;K : We construct an intertwining operator R W H ! H0 between  and 0 . Let us first consider it on the space C of continuous functions 2 H with compact support modulo H . Fixing 2 C and k 2 K, we define the complex-valued function ˚k; on H 0 by h0 7!

.kh0 /f .h0 /.ı 0 /1=2 .h0 /:

(5.3.1)

Then, let us see that ˚k; belongs to the space of functions E.H 0 =H0 /; H0 D exp.h0 / defined in Sect. 2.1. In fact, if we put ı0 D H0 ;H 0 ,  1=2 .h0 2 H0 / ı0 .h0 / D ı.h0 /=ı 0 .h0 / by Lemma 5.3.12(5). Hence, with h0 2 H 0 , ˚k; .h0 h0 / D D

.kh0 h0 /f .h0 h0 /.ı 0 /1=2 .h0 h0 / .kh0 /ı 1=2 .h0 /f .h0 /1 f .h0 /f .h0 /.ı 0 /1=2 .h0 /.ı 0 /1=2 .h0 /

D ˚k; .h0 /ı0 .h0 /:

5.3 Kirillov–Bernat Mapping

145

On the other hand, by K D AH D H 0 H , the canonical projection K ! K=H induces a surjection H 0 ! K=H and K=H becomes a homogeneous space of H 0 so that we get a diffeomorphism  W H 0 =H0 ! K=H . Let S be the support of ˚k; modulo H0 . Since the image of S by  is compact, S itself is compact. So, as in Sect. 2.1, making use of the H 0 -invariant positive linear form H 0 ;H0 on E.H 0 =H0 /, we define the complex-valued function R. / on K by the formula   R. /.k/ D H 0 ;H0 ˚k; .k 2 K/: The operator R defined in this manner on C obviously intertwines the actions ; 0 , that is to say, is commutative with the left translation of K. Let us verify that R extends to an isometry from H onto H0 . From what we have seen, K=H and H 0 =H0 are identified with w, besides K=H 0 with t. Namely, H is identified with L2 .w/ by the mapping ! Q ; Q .Y / D .exp Y /, C and 0 Q Q E.H =H0 / are with Cc .w/ by ˚ ! ˚; .˚/.Y / D ˚.exp Y /, H0 is with L2 .t/ by ' ! '; Q '.T Q / D '.exp T /. Besides, H 0 ;H0 is identified with the Lebesgue measure d Y on w. For ˚ 2 E.H 0 =H0 /; h0 2 H0 ; Y0 2 w, Z

Z ˚.exp.Y0 /h0 exp Y /d Y D w

Z ˚.exp.Y0 Ch0 Y //ı0 .h0 /d Y D

w

˚.exp Y /d Y: w

Moreover, since w  a, ı 0 jW 1; W D exp w. Therefore, for Y 2 w

e

˚k; .Y / D ˚k; .exp Y / D

.kexp Y /e if .Y /

and Z R. /.k/ D

.kexp Y /e if .Y / d Y: w

Now by Definition 5.3.1 and the left invariance of H 0 ;H0 we have R. /.kh0 / D f .h0 /1 ı 0 .h0 /1=2 R. /.k/ .k 2 K; h0 2 H 0 /

A

so that R. / is identified with the function R. / on t, Z

A

R. /.T / D R. /.exp T / D

..exp T /exp Y /e if .Y / d Y w

Z

.exp..exp T /Y /exp T /e if .Y / d Y .T 2 t/:

D w

By the way, t  j and, since Œj; w  h \ a, ı.exp T / D 1 and .exp T /Y D Y C ŒT; Y  C Y 0 ; ŒT; Y  2 h \ a; Y 0 2 Œh; h \ a:

146

5 Orbit Method

Hence ..exp T /exp Y / D

.exp..exp T /Y //ı.exp T /1=2 e if .T /

D

..exp Y /exp.ŒT; Y /exp.Y 0 //e if .T /

D

.exp Y /e if .ŒT;Y / e if .T /

and consequently

A R . /.T / D e

if .T /

Z

Q .Y /e iBf .Y;T / e if .Y / d Y: w

Since the restriction of Bf to tw is non-degenerate, Bf defines a Fourier transform FB W L2 .w/ ! L2 .t/ and

A A . / 2 H D L .t/ is shown, and we get, the Fourier transform being In this way R   R. /.T / D e if .T / FB O .T /; O .Y / D Q .Y /e if .Y / : 0

2

isometric, the desired isometry by multiplying R with some positive constant.



Proposition 5.3.19. As before, we define h; h0 ; h0 ; k; j and assume h  z; dim.h=h0 / D dim.a=.h \ a//; h ¤ h0 C j. (1) h0 D j; dim.h=h0 / D 1 and there are X 2 h n h0 ; Y 2 a n .h \ a/ such that k D RX ˚ h0 ˚ RY; ŒX; Y  D Y; f .Y / D 1 and the mapping .˛; X0 ; ˇ/ 7! exp.˛X /exp.X0 /exp.ˇY / provides a diffeomorphism from R  h0  R ' k onto K. (2) h0 ; h0 are ideals of k and Œh0 ; a  h \ a D z \ a. (3) For v 2 R we give fv 2 k by fv 2 f C h? ; fv .Y / D v. Then exp.˛X /fv D fve˛ for any ˛ 2 R. Moreover, if v ¤ 0, h0 2 M.fv ; k/. (4) If vC and v denote respectively a positive and negative real number, .f; O h; K/ ' .f O vC ; h0 ; K/ ˚ .f O v ; h0 ; K/: Proof. Since h0 C j ¤ h, from Lemma 5.3.14 and another one just before it, h0 D j and dim.h=h0 / D 1. Because h ¤ j, the action of h on a=.h\a/, and hence of course on a=.z \ a/, is not trivial, so the latter is irreducible. Since h \ a is a submodule of the h-module a different from a, h \ a D z \ a. While, by Lemma 5.3.14, j D h0 is an ideal of h, by definition Œj; a  h \ a  h0 and h0 is an ideal of k, likewise for h0 D h0 C a. Now the last claim of (1) follows.

5.3 Kirillov–Bernat Mapping

147

Next let X1 2 h n h0 ; Y1 2 a n .a \ h/. Then there are 0 ¤  2 R and Y2 2 h \ a such that ŒX1 ; Y1  D Y1 C Y2 . So, if we put X D 1 X1 ; Y D .Y1 C Y2 / .0 ¤

2 R/, we get ŒX; Y  D Y . As f .ŒX; Y / ¤ 0, we can choose in such a fashion that f .Y / D 1. Finally, since Œh0 ; a D Œj; a  h \ a, the statements (1), (2) follow. We show the assertion (3). If X 0 2 h, exp.˛X /X 0 2 X 0 C Œh; h. Since h 2 S.f; g/, exp.˛X /fv 2 f C h? . Hence we can write exp.˛X /fv D fv0 with v0 D .exp.˛X /fv / .Y / D fv .exp.˛X /Y / D e ˛ v: Since Œh0 ; h0  D Œh0 ; h0  C Œh0 ; a  h, Lemma 5.3.12 gives fv .Œh0 ; h0 / D f .Œh0 ; h0 / D f0g so that h0 2 S.fv ; k/, while, as fv .ŒX; Y / D f .Y / D v, k 62 S.fv ; k/ if v ¤ 0 so that h0 2 M.fv ; k/. O The representation  D .f; O h; K/ is realized on the space H D H.f; h; K/ of all complex-valued functions on K satisfying the relation .kh/ D f .h/1 ı 1=2 .h/ .k/; k 2 K; h 2 H; ı D H;K : Here note that ıjH0 D 1 by Œh0 ; a  h and ı.exp X / D e by ŒX; Y  D Y . Likewise, sine h0 is an ideal of k, the representation v0 D .f O v ; h0 ; K/ is realized on the space 0 0 O v ; h ; K/ of all complex-valued functions ' on K satisfying the relation Hv D H.f '.kh0 / D fv .h0 /1 '.k/; k 2 K; h0 2 H 0 :

(5.3.2)

As in the proof of the previous proposition, we argue on the space C of continuous functions 2 H with compact support modulo H . Fixing 2 C and k 2 K, we consider the function ˚k; W h0 7!

.kh0 /fv .h0 /; h0 2 H 0

(5.3.3)

on H 0 . The spaces K=H and H 0 =H0 being identified, ˚k; has a compact support modulo H0 . Moreover, since ıjH0 D 1 and f jh0 D fv jh0 , ˚k; .h0 h0 / D ˚k; .h0 /; h0 2 H 0 ; h0 2 H0 ; namely ˚k; 2 E.H 0 =H0 /. Now we set .Rv /.k/ D H 0 ;H0 .˚k; /: By Definition 5.3.3 of ˚k; , ˚kh0 ; .h00 / D fv .h0 /1 ˚k; .h0 h00 /; k 2 K; h0 ; h00 2 H 0 and, by the invariance of H 0 ;H0 , Rv

satisfies relation (5.3.2).

148

5 Orbit Method

Let us see Rv 2 Hv0 and that the operator Rv defined on C extends to a bounded linear operator from H to Hv0 . If this is done, since Rv is by definition commutative with the left translation by the elements of K, Rv gives an intertwining operator between  and v0 . Since .ˇ; X0 / 7! exp.ˇY /exp.X0 / is a homeomorphism from R  h0 onto H 0 , under the mapping 7! Q ; Q .ˇ/ D .exp.ˇY // we identify E.H 0 =H0 / with Cc .R/ and the measure H 0 ;H0 with the Lebesgue measure dˇ. Thus

e

˚k; .ˇ/ D ˚k; .exp.ˇY // D

.kexp.ˇY //e iˇv

and Z .Rv /.k/ D

.kexp.ˇY //e i vˇ dˇ: R

Similarly, since .ˇ; X1 / 7! exp.ˇY /exp.X1 / is a homeomorphism from R  h onto K, using the mapping 7! Q ; Q .ˇ/ D .exp.ˇY // we can identify H with L2 .R/. Likewise, by the mapping ' 7! '; Q '.˛/ Q D '.exp.˛X //, Hv0 is identified with L2 .R/ and Rv with the function Rv W ˛ 7! Rv .exp.˛X //. Under these identifications, F denoting the Fourier transformation in L2 .R/,

e

Z   Rv .˛/ D .Rv /.exp.˛X // D

e

Z

.exp.˛X /exp.ˇY //e i vˇ dˇ R

.exp .e ˛ ˇY // ı.exp.˛X //1=2 e i ˛f .X / e i vˇ dˇ

D R

D e i ˛f .X / D e i ˛f .X /

Z Z

e ˛=2 Q .e ˛ ˇ/ e i vˇ dˇ R ˛ e ˛=2 Q .ˇ/e i vˇe dˇ

R

p   D 2e ˛.if .X /C1=2/ F Q .ve ˛ / :

e

In this way, if v ¤ 0, by d.ve ˛ / D ve ˛ d˛, Rv 2 L2 .R/; Rv 2 Hv0 1=2 p Rv is extended to an intertwining operator and kRv k2  2 k k2 . Hence jvj jvj 2

between  and v0 . Let L2C .R/ and L2 .R/ be the space of all functions in L2 .R/ whose Fourier transforms vanish almost everywhere in the outside of RC and R respectively. From the above computations, if v > 0 (resp. v < 0/, ve ˛ 2 R˙ 1=2 p Rv provides an isometry from L2 .R/ (resp. L2 .R/) onto L2 .R/ and the and jvj  C 2 assertion (4) follows since L2 .R/ D L2C .R/ ˚ L2 .R/.



Corollary 5.3.20. We use the notations in the previous two propositions. Suppose h 2 I.f; g/; dim.h=h0 / D dim.a=.h \ a//. Then .f; O h; G/ ' .f; O h0 ; G/. Let us do concrete computations for groups of low dimension.

5.3 Kirillov–Bernat Mapping

149

Lemma 5.3.21. (1) Let G2 D exp.g2 /; g2 D hX; Y iR I ŒX; Y  D Y . Namely, G2 is the “ax C b” group of Chap. 4, the unique connected and simply connected noncommutative Lie group of dimension 2 and a typical completely solvable Lie group. For any f 2 g , the monomial representation .f; O RX; G2 / is reducible. (2) Let G3 .˛/ D exp.g3 .˛//; g3 .˛/ D hT; X; Y iR .0 ¤ ˛ 2 R/I ŒT; X  D X  ˛Y; ŒT; Y  D ˛X C Y . Namely, G3 .˛/ is the Grélaud group G˛ of Chap. 4, a typical exponential solvable Lie group which is not completely solvable. For any f 2 g , the monomial representation .f; O RT; G3 .˛// is reducible. Proof. The reducibility of .f; O RX; G2 / follows from the preceding proposition. We show the reducibility of  D .f; O RT; G3 .˛//. Put a D RX C RY . If we identify a with C by the mapping ˇX C Y 7! ˇ C i , we have ŒT; z D .1  i ˛/z for any z 2 C. The group G3 .˛/ is the semi-direct product of H D exp.RT / by A D exp C and   exp.T /exp z D exp e .1i ˛/ z exp.T /;  2 R; z 2 C: Let f 2 g ;  D f .T /. Since H;G3 .˛/ .exp.T // D e 2 ; f .exp.T // D e i  , O H D H.f; RT; G3 .˛// is the space of all complex-valued functions ' on G3 .˛/ satisfying the relation '.gexp.T // D e .1i / '.g/; .; g/ 2 R  G3 .˛/: When we identify H with L2 .C/ D L2 .R2 / through the correspondence ' 7! '; Q '.z/ Q D '.exp z/, with  2 R; z0 ; z 2 C; ' 2 H, Q .z/ D '.exp.z0 /exp.T /exp z/ ..exp.T /exp.z0 //'/   .1i ˛/     D ' exp ze  z0 exp.T / D e .i 1/ 'Q ze .1i ˛/  z0 : We denote by F the Fourier transformation in L2 .C/ and put 1 D F ı  ı F 1 . We denote by .j/ the inner product in R2 D C and by d z the Lebesgue measure on R2 . With g D exp.T /exp.z0 / 2 G3 .˛/; 2 Cc .C/; v 2 C, Z 2 .F ..g// / .v/ D

..g/ /.z/e i.vjz/ d z Z

C

D

e .i 1/ Z

C

  .1i ˛/ ze  z0 e i.vjz/d z

e .i C1/ .z0 /e i .vje

D

.1i ˛/ .z0 Cz

C

D 2e .i C1/Ci .vje

.1i ˛/ z

0

0/

/ d z0

/ .F / e .1Ci ˛/v  :

150

5 Orbit Method

We deduce from this the reducibility of 1 and hence of . Because R acts on z 2 C by the formula z D e .1Ci ˛/ z and there exists a non-negligible measurable set M of C, which is invariant by the action of R and whose complement is also nonnegligible, the closed subspace formed by all ˚ 2 L2 .C/ which vanish almost everywhere in the outside of M is invariant by 1 .  Henceforth we fix an exponential solvable Lie group G D exp g and f 2 g . The sum of all minimal ideals of g is called the pedestal of g, which we denote by .g/. Proposition 5.3.22. Every h 2 I.f; g/ contains .g/. Proof. Let a be a minimal ideal of g. Then k D h C a is a Lie subalgebra of g and h 2 I.f; k/. If Œh; a D f0g, h  a. In fact, if this is not true, K D exp k is a direct product of H D exp h with a non-trivial subgroup, and 1 D .f; O h; K/ is reducible. So, we suppose Œh; a ¤ f0g in what follows. Then the action of h on a is irreducible so that h \ a is equal to either f0g or a. Therefore, it is enough to show that 1 is reducible if Œh; a ¤ f0g and if h \ a D f0g. In this case, k D h ˚ a and O K is a semi-direct product of H by A D exp a. Now H1 D H.f; h; K/ is identified with L2 .a/, a continuous function ' 2 H1 corresponding to 'Q W Y 7! '.exp Y /. With X 2 h; Y 2 a, .1 .exp X /'/ Q .Y / D '.exp.X /exp Y / D '.exp.exp.X /Y /exp.X // 1=2

D '.exp.X Q /Y / H;K .exp X /e if .X / : If j denotes the kernel of the action of h on a, as we saw before, j is a hyperplane of h. Let us write h D j ˚ RX1 . Since exp.X /Y D Y; H;K .exp X / D 1 for X 2 j, the above computations say that 1 .exp X / is a scalar operator e if .X / . Hence, putting k1 D RX1 ˚ a; K1 D exp.k1 /, it suffices to see that 2 D 1 jK1 is reducible, while 2 D .f; O RX1 ; K1 / and k1 is isomorphic to either g2 or g3 .˛/ according to dim a D 1; 2. Now the desired result comes from the previous lemma.  Proposition 5.3.23. I.f; g/  M.f; g/. Proof. Take h 2 I.f; g/. We show dim h D m.f; g/ by induction on dim g. Since h  .g/ by the last proposition, h contains the centre z of g. We first assume that kerf contains a non-trivial ideal of g, and let a be a minimal ideal such that f ja D 0. The last proposition assures that h  a. We set gQ D g=a. Let hQ .resp: fQ/ be the canonical image of h .resp: f / in gQ .resp: gQ  / and p W G ! GQ D G=A; A D exp a; the canonical projection. Then, since Q G/ıp, Q .f; O h; G/ D . O fQ; h; hQ 2 I.fQ; gQ /. Thus, hQ 2 M.fQ; gQ / by the induction hypothesis. The bilinear form BfQ being obtained from Bf by passing to the quotient space, m.fQ; gQ / C dim a D m.f; g/. Therefore h 2 M.f; g/. In what follows suppose that kerf does not contain a non-trivial ideal of g. Then obviously dim z  1. Take a minimal non-central ideal a of g. Then af ¤ g. Indeed, if af D g, f vanishes on the ideal Œg; a ¤ f0g, contradictory to the hypothesis.

5.3 Kirillov–Bernat Mapping

151

We set G0 D exp.af /. If h  af , because of .f; O h; G/ D indG O h; G0 /, G0 .f; f h 2 I.f; a /. Hence, by the induction hypothesis and Lemma 5.3.11, we get h 2 M.f; g/. Suppose hereafter that h 6 af . Let h0 D h \ af ; h0 D h0 C a; k D h C a. By Lemma 5.3.12, 1  dim.h=h0 /  dim.a=.h \ a//  2: If dim.h=h0 / D dim.a=.h \ a//, dim h D dim.h0 / by Lemma 5.3.12 and .f; O h; G/ ' .f; O h0 ; G/ by Corollary 5.3.20. Since h0  af , h0 2 M.f; g/ as we have seen above. Thus, h 2 M.f; g/. Now remains the case where h 6 af and dim.h=h0 / < dim.a=.h \ a//. We will see there is no such possibility. In fact, if we assume such a situation, dim.h=h0 / D 1; dim.a=.h\a// D 2 and h\a D z\a by Lemma 5.3.13. Furthermore, z\a ¤ f0g. If not, a turns out to be a minimal ideal and h  a by the last proposition. Therefore h  hf  af , which is contradictory to the hypothesis. As dim z  1, h \ a D z \ a D z; dim z D 1: If we write z D RZ, we may take f .Z/ D 1 since f jz ¤ 0. Lemma 5.3.24. We denote by j the kernel of the action of h on a=.h \ a/ and by c.a/ the centralizer of a in g, namely c.a/ D fX 2 gI ŒX; Y  D 0; 8Y 2 ag: Then h \ c.a/ D h0  j. Q Since Proof. If we set BQ D Bf jaj , j \ h0 is the orthogonal space of a relative to B. ŒX; a  h \ a D RZ for any X 2 j, there is X 2 a such that ŒX; Y  D X .Y /Z for all Y 2 a and so f .ŒX; Y / D X .Y /. Here the subspace a.j/ D fY 2 aI ŒY; j D Q Since dim.h=h0 / D 1, f0gg is the orthogonal space of j relative to B. dim.a=a.j// D dim.j=.j \ h0 //  1: Consequently z \ a ¨ a.j/  a. By Lemma 5.3.14, j being an ideal of h and a.j/ becomes an h-module by Jacobi identity. If h D j, of course h0  j. Otherwise, the action of h on a=.h \ a/ is irreducible. Hence a.j/ D a; j D j \ h0 ; j  h0 ¨ h. Thus j D h0 since dim.h=j/  1. From what we have seen, h0  j in any case. So, if X 2 h0  j \ af , X D 0 and X 2 h \ c.a/. Thus h0  h \ c.a/  h \ af D h0 :



We continue the proof of the proposition. Take X 2 h n h0 so that h D h0 ˚ RX as vector space. First suppose h D j. Since dim.a=.h \ a// > dim.h=h0 / and Bf

152

5 Orbit Method

induces the bilinear mapping a=.z \ a/  h=h0 ! R, there is Y 2 .a n .h \ a// \ hf such that ŒX; Y  2 Œj; a  z; f .ŒX; Y / D 0. Hence ŒX; Y  D 0. As we saw in the last lemma, h0  c.a/ so that Œh; Y  D f0g. In consequence, Y belongs to the centre of k, but this contradicts h 2 I.f; k/. We next suppose h ¤ j so that the action of h on a=.z \ a/ is irreducible. Take Y D Y1 C iY2 2 aC which represents an eigenvector of the action of h on .a=.z \ a//C . So there are ˛;  2 R n f0g and 2 C such that ŒX; Y  D .1 C i ˛/Y C Z. Replacing if necessary Y by ŒX; Y  and X by 1 X , we may assume ŒX; Y  D .1 C i ˛/Y . Since h0  c.a/ by the last lemma, a0 D RY1 ˚ RY2 is a minimal ideal of k, and we are led to a contradiction as in the case described above.  Lemma 5.3.25. Let, as before, f 2 g and let a be a commutative ideal of g. Then I.f; af /  I.f; g/. We set G0 D exp.af /; A D exp a. If h 2 I.f; af /, h  a and the restriction of .f; O h; G0 / to A is f Id . Proof. The stabilizer of f with respect to the action of G on AO is nothing but G0 . Taking Theorem 3.4.4 into account, it is enough to verify that the restriction of  D .f; O h; G0 / to A is f Id . By Proposition 5.3.23 h 2 M.f; af / and, since h C a 2 S.f; af /, h  a, while  is realized by the left translation in the space O H.f; h; G0 / of complex-valued functions ' on G0 satisfying the relation '.gh/ D f .h/1 H;G0 .h/'.g/ g 2 G0 ; h 2 H; 1=2

and H;G0 jA D 1. Since a is commutative, with X 2 af ; Y 2 a, ..exp Y /'/.exp X / D '.exp.Y /exp X / D ' ..exp X /exp.exp.X /.Y /// : Here we can write exp.X /  .Y / D Y C Y 0 ; Y 0 2 Œaf ; a  a. As f .Y 0 / D 0, ..exp Y /'/.exp X / D '..exp X /exp.Y /exp.Y 0 // D f .exp Y /'.exp X /:  Proposition 5.3.26. At any f 2 g , I.f; g/ ¤ ;. Proof. We proceed by induction on g. If there is an ideal a ¤ f0g of g such that f ja D 0, let A D exp a; GQ D G=A, p W G ! GQ the canonical projection, dp W g ! gQ D g=a its differential and fQ 2 gQ  the image of f . By the induction Q h 2 S.f; g/ and hypothesis there exists hQ 2 I.fQ; gQ /. So, if we put h D .dp/1 .h/, Q Q Q .f; O h; G/ D . O f ; h; G/ıp. Hence h 2 I.f; g/. In the case where there is no such ideal as above, if a is a minimal non-central ideal of g, af ¨ g and I.f; af / ¤ ; by the induction hypothesis. Thus the result follows from the last proposition. 

5.3 Kirillov–Bernat Mapping

153

Proposition 5.3.27. Let f 2 g and h1 ; h2 be two elements of I.f; g/. Then the representations .f; O h1 ; G/ and .f; O h2 ; G/ are equivalent. Proof. We reason in the same scheme as in the proof of the previous proposition. If there is a non-trivial ideal a of g where f vanishes, we may assume that a is a minimal ideal. Then Proposition 5.3.22 assures that h1 ; h2 contain a. Using the notations introduced in the proof of the previous proposition, we put hQ j D Q Q and dp.hj / for j D 1; 2. Then . O fQ; hQ j ; G/ıp D .f; O hj ; G/, while . O fQ; hQ 1 ; G/ Q Q Q are equivalent by the induction hypothesis. So we have the conclusion. . O f ; h2 ; G/ In the case where there is no such ideal as the above, if a is a minimal non-central ideal of g, af ¨ g. When h1 ; h2 are both contained in af , the induction hypothesis says that j D .f; O hj ; G0 / .j D 1; 2/ with G0 D exp.af / are mutually equivalent, and likewise for representations .f; O hj ; G/ D indG G0 j .j D 1; 2/. After these considerations, the proof will be complete if we show the following. For h 2 I.f; g/ not contained in af , there is h0 2 S.f; g/ contained in af so that .f; O h; G/ ' .f; O h0 ; G/. Indeed, if we set h0 D h \ af ; h0 D h0 C a by modifying h, Proposition 5.3.23 and Lemma 5.3.12(2) imply dim h D dim.h0 /. Similarly Lemma 5.3.12(3) tells us that dim.h=h0 / D dim.a=.h \ a//. Now we are ready to apply Corollary 5.3.20 to get .f; O h; G/ ' .f; O h0 ; G/.  Theorem 5.3.8 is now proved. If we agree to identify unitary representations with their equivalence classes, this theorem allows us to simplify the notation .f; O h; G/ as .f O / and f 7! .f O / provides a mapping from g to the unitary dual GO of G. Moreover, for h 2 S.f; g/; g 2 G, the simple transfers of structures give gh 2 S.gf; g/ and .gf; O gh; G/ ' .f; O h; G/. Therefore, the above mapping induces a O denoted again by mapping from the space g =G of all coadjoint orbits of G to G, O D OG . Theorem 5.3.28 (Bernat [10]). Let G D exp g be an exponential solvable Lie O group with Lie algebra g. The mapping O is a bijection of g =G onto G. Proof. To begin with, we show following Takenouchi [75] the surjectivity of the O We have to see the existence of a closed connected mapping . O Take  2 G. subgroup H of G and its unitary character  so that  ' indG H , because, if this is checked, then h being the Lie algebra of H , there exists some f1 2 h such that .exp X / D e if1 .X / ; 8 X 2 h. So, choosing f 2 g in such a manner that f jh D f1 , we get  ' .f; O h; G/. As usual we show the claim by induction on the dimension of G. Let a be a minimal non-central ideal of g and b D Œg; a ¤ f0g; B D exp b. If jB D 1, we can write  D ıp Q with p W G ! GQ D G=B, where Q is an irreducible Q By the induction hypothesis there are a connected unitary representation of G. Q Q Q . Q Hence closed subgroup H of G and its unitary character Q so that Q ' indG HQ G 1 Q putting H D p .H /;  D ıp, Q  ' indH . Assume in what follows that jB ¤ 1. We denote as before by z the centre of g and put C D exp z. Schur’s Lemma means that jC is a unitary character of C and our hypothesis implies dim z  1. In order to apply Theorem 3.4.4 of

154

5 Orbit Method

Chap. 3, we verify that the commutative closed normal subgroup A D exp a is regularly embedded in G. From now on we identify AO with a and calculate case O dim.a=.a \ z// being 1 or 2, accordingly to these by case the action of G on A. possibilities we write the root coming from the action of G on a=.a \ z/ as  or .1 C i ˛/ .0 ¤ ˛ 2 R/ with  2 g . (1) a \ z D f0g; dim a D 1. If we put a D RY , ŒX; Y  D .X /Y for all X 2 g. Hence .exp X /  ` D e .X / ` .` 2 a / and the G-orbit in a is either one point or a half line. (2) a \ z D f0g; dim a D 2. There exist Y1 ; Y2 in a so that ŒX; Y1  D .X /.Y1  ˛Y2 /; ŒX; Y2  D .X /.˛Y1 C Y2 / for any X 2 g. Let fY1 ; Y2 g be the dual basis of fY1 ; Y2 g, and we calculate exp X  ` D a0 Y1 C b 0 Y2 with ` D aY1 C bY2 2 a .a; b 2 R/: a0 D e .X / .a cos .˛.X // C b sin .˛.X /// b 0 D e .X / .a sin .˛.X // C b cos .˛.X /// : Thus the G-orbit in a is either one point or a spiral. (3) a \ z D z ¤ f0g; dim a D 2. Let z D RZ and a D RY ˚ z, and take the dual basis Y  ; Z  of Y; Z. Now there is 2 g so that we can write ŒX; Y  D .X /Y C .X /Z for any X 2 g. Separating again various cases, we compute the G-orbit of ` D aY  C bZ  2 a .a; b 2 R/. (i)  D 0. If we put .exp X /  ` D a0 Y  C b 0 Z  , a0 D a  .X /b; b 0 D b: Hence the G-orbit in a is either a point or a line. (ii) The case where  ¤ 0 and ; 2 g are linearly dependent. Assume D c .c 2 R/ and put Y 0 D Y C cZ. Then ŒX; Y 0  D .X /Y 0 for any X 2 g and we are reduced to case (1). (iii) The case where ; 2 g are linearly independent. In this case, we can take X1 ; X2 2 g so that .X1 / D 1;

.X1 / D 0

.X2 / D 0; .X2 / D 1:

5.3 Kirillov–Bernat Mapping

155

If we write exp.tX1 /  ` .t 2 R/ as a0 Y  C b 0 Z  , a0 D e t a; b 0 D b: Likewise for exp.tX2 /  `, a0 D a  tb; b 0 D b: Hence the G-orbit in a is either one point, a half-line or a line. (4) a \ z D z ¤ f0g; dim a D 3. Let z D RZ. If we choose conveniently a basis fY1 ; Y2 g of a modulo z, there are ; ı 2 g so that ŒX; Y1  D .X /.Y1  ˛Y2 / C .X /Z; ŒX; Y2  D .X /.˛Y1 C Y2 / C ı.X /Z for any X 2 g. Besides  ¤ 0 by hypothesis. Here once again we examine various cases separately. Replacing Y1 ; Y2 and trying a modification similar to (3)(ii), it is enough to treat the following three possibilities. (i) D ı D 0. This case is clearly reduced to case (2). (ii) The case where ; ; ı are linear independent. There are X1 ; X2 ; X3 in g so that .X1 / D 1; .X2 / D 0; .X3 / D 0;

.X1 / D 0;

.X2 / D 1;

.X3 / D 0;

ı.X1 / D 0;

ı.X2 / D 0;

ı.X3 / D 1:

Using the dual basis fY1 ; Y2 ; Z  g of fY1 ; Y2 ; Zg, we compute the Gorbit passing ` D aY1 C bY2 C cZ  2 a .a; b; c 2 R/. If we write exp.tX1 /  ` .t 2 R/ as a0 Y1 C b 0 Y2 C c 0 Z  , a0 D e t .a cos.˛t/ C b sin.˛t// ; b 0 D e t .a sin.˛t/ C b cos.˛t// ; c 0 D c:

Likewise for exp.tX2 /  `, a0 D a  tc; b 0 D b; c 0 D c: For exp.tX3 /  `, a0 D a; b 0 D b  tc; c 0 D c:

156

5 Orbit Method

Consequently, the G-orbit G  ` is one point if a D b D c D 0, a spiral if jaj C jbj ¤ 0; c D 0 and a plane if c ¤ 0. (iii) The case where ; are linearly independent and ı D d .d 2 R/. Let us see that in reality there is no such a possibility like this. At first, if we put Y10 D Y1 C d Y2 ; Y20 D d Y1 C Y2 , ŒX; Y10  D .X /.Y10  ˛Y20 / C .1 C d 2 / .X /Z; ŒX; Y20  D .X /.˛Y10 C Y20 / for any X 2 g. Hence we may assume d D 0 from the beginning. Now if we take X1 ; X2 in g so that .X1 / D .X2 / D 1; .X2 / D .X1 / D 0; namely that ŒX1 ; Y1  D Y1  ˛Y2 ;

ŒX2 ; Y1  D Z;

ŒX1 ; Y2  D ˛Y1 C Y2 ;

ŒX2 ; Y2  D 0;

on the one hand, ŒX2 ; ŒX1 ; Y2  D ŒX2 ; ˛Y1 C Y2  D ˛Z: On the other hand, ŒX2 ; ŒX1 ; Y2  D ŒŒX2 ; X1 ; Y2  C ŒX1 ; ŒX2 ; Y2  D ŒŒX2 ; X1 ; Y2  from Jacobi identity. But this last term never coincides with ˛Z. Neither does it for the possibility where ; ı are linearly independent and D d ı .d 2 R/. In each of the above cases, it is easy to confirm that the G-orbits in AO are countably separated for the Borel structure of AO and hence the commutative closed normal subgroup A is regularly embedded in G. Theorem 3.4.4(2) tells us that there are some  2 AO and an irreducible unitary representation  satisfying jA D Id of the stabilizer G./ of  in G so that  ' indG G./ . Besides, Theorem 5.3.2 says that G./ is a connected closed subgroup of G. Now, if we take f 2 g such that  D f , G./ D exp.af /. Hence if G./ D G, af D g, namely f .Œg; a/ D f .b/ D f0g. So, it follows that jB D jB D 1. But this is contradictory to the assumption. Therefore G./ ¨ G and by the induction hypothesis there are a connected closed subgroup H of G./ and its G./ unitary character  so that  ' indH . In this way,  D indG H . Next we show the mapping O is injective.

5.3 Kirillov–Bernat Mapping

157

Lemma 5.3.29. Let a be a minimal non-central ideal of g, b D Œg; a; B D exp b and f 2 g . Then f 2 b? if and only if .f O /jB D 1. Proof. Using Lemma 5.3.25 we take h 2 I.f; af /  I.f; g/. Since hCa 2 S.f; g/, Proposition 5.3.23 gives h  a. Now, if we put  D .f; O h; G/, for X 2 g; Y 2 b O and a continuous function ' 2 H.f; h; G/, ..exp Y /'/ .exp X / D '.exp.Y /exp X / D' ..exp X /exp.exp.X /Y // D '.exp X /e if .exp.X /Y / : Hence if f jb D 0, we have jB D 1. Conversely, if jB D 1, f .Gb/ D f .b/ D 0 follows.  Let a be a minimal non-central ideal of g, b D Œg; a ¤ f0g and B D exp b. If we designate the images in g =G of b? and of g n b? by C and by D, the space of orbits g =G is decomposed into the disjoint union of C and D. By Lemma 5.3.29, .C/\ O .D/ O D ;. Therefore if we see that 1 D j O C and 2 D j O D are both injective, then the mapping O turns out to be injective. We suppose that .f O 1 / D .f O 2 / for f1 ; f2 2 b? . Put gQ D g=b, and denote by Q Q f1 ; f2 the canonical images in gQ  of f1 ; f2 respectively. Let hQ j 2 I.fQj ; gQ / .j D 1; 2/, be p the canonical projection from G onto GQ D G=B and its differential dp, then for j D 1; 2 Q O j ; hj ; G/ D . O fQj ; hQ j ; G/ıp: hj D .dp/1 .hQ j /; .f Hence . O fQ1 / ' . O fQ2 / and by the induction hypothesis there is g 2 G so that fQ2 D p.g/fQ1 , namely f2 D gf1 . This means that the mapping 1 is injective. We next assume that .f O 1 / ' .f O 2 / for f1 ; f2 2 g n b? . Since fj .Œg; a/ ¤ f0g fj with j D 1; 2, a ¨ g. If we set Gj D exp.afj /, there exist hj 2 I.fj ; afj / so that .f O j / D .f O j ; hj ; G/ ' indG O j ; hj ; Gj /: Gj .f As hj  a, the restriction of .f O j ; hj ; Gj / to A is a scalar operator corresponding to the unitary character fj of A. Hence, Mackey’s theory asserts that .f O j /jA O is defined by a measure j supported on the G-orbit ˝j D G  fj  A. The representations .f O 1 / and .f O 2 / being equivalent, the measures 1 and 2 are also equivalent. Consequently ˝1 D ˝2 . Therefore there is g 2 G so that f2 D g  f1 D gf1 . Hence replacing f1 by gf1 , we may assume f1 D f2 . Then, f2  f1 2 a? and af1 D af2 . We denote this common Lie subalgebra by g0 and let G0 D exp.g0 /; f0 D fj ja . The representations .f O j ; hj ; G0 / .j D 1; 2/ are irreducible representations of G0 , whose restrictions to A are the scalar operators corresponding to the same unitary character f0 W exp Y 7! e if0 .Y / . Since these two representations induce

158

5 Orbit Method

equivalent representations of G, they are mutually equivalent by Theorem  3.4.4(1).  Therefore by the induction hypothesis there is g 2 G such that g  f1 jg0 D f2 jg0 . That is, gf1 2 f2 C g0 ? . However, Lemma 5.3.11 asserts that f2 ? f2 C g? 0 D f2 C .a /  Gf2 :

In consequence, Gf1 D Gf2 . In this way it has been shown that the mapping O is injective too.  Definition 5.3.30. The mapping O from g or g =G to GO is called a Kirillov– Bernat mapping. We give a more precise statement of this result. We provide GO with Fell topology [24, 26, 41]. Let  2 GO with its Hilbert space H . We define a neighbourhood of O For finite vectors v1 ; : : : ; vk of H , a compact set C of G and a positive  in G. number  > 0, the neighbourhood U.v1 ; : : : ; vk I C I / of  is constituted by such  2 GO that there exist vectors w1 ; : : : ; wk in H satisfying j..g/vi ; vj /  ..g/wi ; wj /j < ; g 2 C; 1  i; j  k: Then, the unitary dual of an exponential solvable Lie group is realized including its topology by the space of the coadjoint orbits. This fact has been open for a long time under the name of Kirillov’s conjecture before being proved by the following theorem. However, its proof is too long to be included here. Theorem 5.3.31 ([51]). We equip the orbit space g =G with the quotient topology and the unitary dual GO with Fell topology. Then the Kirillov–Bernat mapping O is a homeomorphism. Let .; H / be a unitary representation of G. We denote simply by dg a left Haar measure on G and consider the space L1 .G/ with respect to this measure. With ' 2 L1 .G/ we define an operator .'/ in H by the formula Z .'/ D

'.g/.g/dg: G

When .'/ is a compact operator for arbitrary ' 2 L1 .G/,  is called CCR. G is called CCR when its all irreducible unitary representation are CCR [4, 20]. It is known that  2 GO is CCR if and only if fg  GO is a closed set for Fell topology [2, 25]. Then, all coadjoint orbits of a connected and simply connected nilpotent Lie group are closed sets so that these groups are CCR by Theorem 5.3.31. In the exponential solvable case, the coadjoint orbits are locally closed sets but not necessarily closed. Theorem 5.3.31 establishes also Moore’s conjecture [57] which asserts the correspondence between closed coadjoint orbits and CCR representations. Likewise, Theorem 5.3.31 which translates various problems concerning Fell topology to those concerning the quotient topology is a

5.4 Pukanszky Condition

159

significant result. What information on Fell topology offers the construction of the unitary dual due to Auslander and Kostant of connected and simply connected type I solvable Lie groups?

5.4 Pukanszky Condition Different from the nilpotent case, it is possible that I.f; g/ ¤ M.f; g/ in the case of exponential solvable Lie groups. The set I.f; g/ is characterized by the following theorem. Theorem 5.4.1. ([10]) Let G D exp g be an exponential solvable Lie group with Lie algebra g, f 2 g ; h 2 S.f; g/ and H D exp h. Then the following statements are equivalent: (1) (2) (3) (4)

H f D f C h? ; f C h?  Gf and h 2 M.f; g/; for any  2 h? , h 2 M.f C ; g/; h 2 I.f; g/.

Proof. (1) H) (2): By assumption the mapping h 7! hf is a surjection from H onto f C h? . Hence Sard’s theorem tells us that there is at least one point h0 2 H at which the differential of this mapping becomes a surjection. Thus the mapping X 7! X .h0 f / is a surjection from h onto h? . But h.h0 f / D  h f ? h0 . Therefore hh0 f D h and hf D h. (2) H) (3): If  2 h? , then clearly h 2 S.f C ; g/. So, it suffices to see dim.g.f C // D dim.g.f //. From the assumption there exists g 2 G such that f C  D gf . Hence g.f C / D g .g.f // and dim.g.f C // D dim.g.f //. (3) H) (1): The image of h by the differential map X 7! X f at the origin of  ? D h? . Hence H f the mapping h 7! hf from H to f C h? is hf D hf ? contains a neighbourhood of f in f C h . This argument may be applied to each point of f C h? and we get (1). (4) H) (3): For any  2 h? , .f; O h; G/ D .f O C ; h; G/. Hence Proposition 5.3.23 implies h 2 M.f C ; g/. (2) H) (4): This follows from Theorem 11.2.1 described later in Chap. 11.  Definition 5.4.2. When h 2 S.f; g/ satisfies the assertion (1) of Theorem 5.4.1, h is said to satisfy the Pukanszky condition. Remark 5.4.3. h satisfies the Pukanszky condition if and only if hC 2 P C .f; G/ satisfies the Pukanszky condition. Concerning the construction of the elements of I.f; g/, a standard method due to M. Vergne is well known. Let us consider a strong Malcev sequence of Lie subalgebras of g. Remember that this means a sequence of Lie subalgebras

160

5 Orbit Method

f0g D g0  g1      gn1  gn D g; dim.gj =gj 1 / D 1 .1  j  n/ with the following properties: if gj is not ideal of g, gj 1 and gj C1 are both ideals of g and the action of g on gj C1 =gj 1 is irreducible. By interpolating if necessary a composition series of ideals of g, we recognize that such a sequence of Lie subalgebras always exists. Put fj D f jgj for 1  j  n. P Theorem 5.4.4 ([10], Chap. IV). h D nj D1 gj .fj / is an element of I.f; g/. Proof. We first see that gn1 is an ideal of g. Otherwise, the action of g on g=gn2 being irreducible, Œg; g  gn2 and gn1 becomes an ideal, contrary to the assumption.   We next notice that, for 0  p  n, gj 0j p is a strong Malcev sequence of gp . Indeed, if gk .1  k  p  1/ is not an ideal of gp , of course it is not an ideal of g. Consequently gk˙1 are ideals of g and the action of g on gkC1 =gk1 is irreducible with root g ! .X /.1 ˙ i ˛/ . 2 g ; ˛ 2 R n f0g/. Hence by assumption jgp ¤ 0 and the action of gp on gkC1 =gk1 is irreducible. Let us verify by induction on n D dim g that h is a maximal isotropic subspace regarding Bf . If g.f /  gn1 , g.f /  gn1 .fn1 / and dim.g.f // D P dim.gn1 .fn1 //1. So, by the induction hypothesis, h D jn1 D1 gj .fj / is isotropic relative to Bf and dim h D

1 1 .n  1 C dim.gn1 .fn1 /// D .n C dim.g.f ///: 2 2

If g.f / 6 gn1 , gn1 .fn1 /  g.f / and dim.g.f // D dim.gn1 .fn1 // C 1.  Pn1 By the induction hypothesis, h D j D1 gj .fj / C g.f / is isotropic and 0 dim h D dim @

n1 X

1 gj .fj /A C 1

j D0

D

1 1 .n  1 C dim.g.f //  1/ C 1 D .n C dim.g.f ///: 2 2

In any case h \ gn1 D

n1 X

gj .fj /:

j D1

Let us show that h is a Lie subalgebra. Suppose that there Pp1 Pp exists a positive integer p such that j D1 gj .fj / is a Lie subalgebra but that j D1 gj .fj / is no longer a Lie subalgebra. Thus there are X 2 gp .fp / and Yk 2 gk .fk /; k  p  1 so that YkC1 D ŒX; Yk  62

p X j D1

gj .fj /:

5.4 Pukanszky Condition

161

Then, in the first place gk is not an ideal of g. Otherwise, YkC1 2 gk and f .ŒYkC1 ; Y / D f .ŒŒX; Yk ; Y / D f .ŒŒX; Y ; Yk /  f .ŒX; ŒY; Yk / D 0 for any Y 2 gk . Hence YkC1 2 gk .fk / follows, which contradicts the assumption. From the preceding, gk˙1 are ideals of g and the action of g on gkC1 =gk1 is irreducible. In particular, ŒgkC1 ; gkC1   gk1 . Now, we can compute as above to get f .ŒYkC1 ; Y / D f .ŒŒX; Yk ; Y / D 0 for any Y 2 gk1 . Therefore Yk 62 gk1 and gk D gk1 ˚RYk , because, if Yk 2 gk1 , we would have YkC1 2 gk1 .fk1 / from the above equation. But this is absurd. Moreover, YkC1 62 gk . Otherwise, f .ŒYkC1 ; Yk / D 0 since Yk 2 gk .fk /. Together with the above result of computations YkC1 2 gk .fk /. Finally, gkC1 D gk1 ˚ RYk ˚ RYkC1 ; while f .ŒYkC1 ; Yk / ¤ 0 follows from YkC1 62 gkC1 .fkC1 /. Next, since ŒYk ; YkC1  2 gk1 and f .ŒŒYk ; YkC1 ; Y / D 0; 8 Y 2 gk1 , we have ŒYk ; YkC1  2 gk1 .fk1 /: Here we put ŒX; YkC1  D ˇYk C YkC1 C Yk1 ; ˇ; 2 R; Yk1 2 gk1 : Then, from what we have seen until now, Yk1 2 gk1 .fk1 /. On the other hand, since X 2 gp .fp /, f .ŒŒX; YkC1 ; Yk / D f .ŒX; ŒYkC1 ; Yk / C f .ŒYkC1 ; YkC1 / D 0: From this and taking f .ŒYkC1 ; Yk / ¤ 0 into account, D 0. Hence the eigenvalue of the action of adX on gkC1 =gk1 is a root of the equation x 2  ˇ D 0. Consequently .X /.1 C i ˛/ D .X /.1  i ˛/ and .X / D 0. But this is absurd and thus h is a Lie subalgebra. In this manner, we have arrived at h 2 M.f; g/. Finally, we show that h satisfies the Pukanszky condition. Again we use the induction on n D dim g. Let us see f C  2 Gf for any  2 h? . If we put h0 D h \ gn1 and H 0 D exp.h0 /, the induction hypothesis means fn1 C .h0 /? D H 0 .fn1 /

162

5 Orbit Method

in gn1 , while, since gn1 is an ideal of g and the group H 0 leaves f C h? stable, we may assume that  vanishes on gn1 C h, by moving f C  by an element of H 0 . By the way, gn1 is an ideal of codimension 1 in g, Œg; g  gn1 and g.f /  h. In consequence, putting m D Œg; g C g.f /,  2 m? . Then G.f jm /f D f C m? : Indeed, G.f jm /f  f C m? by definition. On the other hand, if X 2 g.f jm /, X f 2 m? and, since m contains Œg; g, .exp X /f D f C X f . Hence f C g.f jm /f  G.f jm /f and f C m?  G.f jm /f since ?  g.f jm /f D g.f jm /f D m? :



An element of I.f; g/ constructed by this method is called a Vergne polarization. Concerning Theorem 5.3.8(3), for h1 ; h2 2 I.f; g/, it is natural to ask how to construct explicitly an intertwining operator between two monomial representations .f; O h1 ; G/ and .f; O h2 ; G/. First of all, we can prove Tr adh1 =.h1 \h2 / X C Tr adh2 =.h1 \h2 / X D 0 for all X 2 h1 \ h2 and consequently H1 ;G .h/ D H2 ;G .h/ 2H1 \H2 ;H2 .h/ .h 2 H1 \ H2 /: O Hence if we define,  2 H.f; h1 ; G/ and g 2 G being given, the function ˚g on H2 by 1=2

˚g .h/ D .gh/f .h/ H2 ;G .h/; the relation ˚g .hx/ D H1 \H2 ;H2 .x/˚g .h/ .h 2 H2 ; x 2 H1 \ H2 / holds. Therefore, with D H2 ;H1 \H2 , we can formally consider the integral I .Th2 h1 /.g/ D

1=2

H2 =H1 \H2

.gh/f .h/ H2 ;G .h/d .h/ .g 2 G/:

(5.4.1)

At least at formal level, the function Th2 h1  satisfies the covariance relation O required to the elements of the space H.f; h2 ; G/ and the operator Th2 h1 obviously

5.4 Pukanszky Condition

163

commutes with the left translation of G. Besides, if the integral (5.4.1) is convergent on the space of C 1 -vectors, we could prove that Th2 h1 gives a true intertwining operator. Therefore, it is no exaggeration to say that the main problem would be the convergence of (5.4.1). The simple product set H2 H1 turns out locally closed in G so that the homogeneous space H2 =H2 \H1 is homeomorphic to the quotient space H2 H1 =H1 . Thus, if H2 H1 is closed in G, which is true for example when G is nilpotent, the integral (5.4.1) is convergent for continuous functions  with compact support modulo H1 . To our regret, we do not know whether H2 H1 is always closed or not. Question. Let G be an exponential solvable Lie group, f 2 g , hj 2 I.f; g/ and Hj D exp.hj / .j D 1; 2/. Is the product set H2 H1 a closed set in G? If h1 or h2 is a Vergne polarization, we can show by choosing appropriately a coexponential basis to h1 in g that H2 H1 is closed in G, and by utilizing the transitivity of the form ; that the operator Th2 h1 furnishes a true intertwining operator. Now as always let f 2 g and consider three maximal isotropic subspaces Wj .1  j  3/ of g for the bilinear form Bf . Following M. Kashiwara we define a quadratic form Q on W1 ˚ W2 ˚ W3 by the formula Q.x1 ; x2 ; x3 / D f .Œx1 ; x2 / C f .Œx2 ; x3 / C f .Œx3 ; x1 /: We define the Maslov index of the triplet .W1 ; W2 ; W3 / as the index of the quadratic form Q and denote it by .W1 ; W2 ; W3 /. That is, regarding a proper basis of the direct sum space W1 ˚W2 ˚W3 the matrix of Q is diagonalized with diagonal entries ˙1; 0. Let p be the number of C1 and q the number of 1. Then .W1 ; W2 ; W3 / D p  q. We list the main properties of this index. Lemma 5.4.5 ([52, 53]). We write ij k instead of .Wi ; Wj ; Wk /. (a) 123 D 213 D 132 . (b) 234  134 C 124  123 D 0. (c) If p is an isotropic subspace of g for Bf containing g.f / and if W is a maximal isotropic subspace of g, W p D .W \ pf / C p is also a maximal isotropic subspace. Besides, if p is contained in .W1 \ W2 / C .W2 \ W3 / C .W3 \ W1 /, p p p 123 D .W1 ; W2 ; W3 /. Proof. Put V D g=g.f / and denote by B the image of Bf to V . Then, it is enough to discuss the symplectic space .V; B/. The property (a) is clear from the definition. Let us show the property (b). Let L; M be two transversal, i.e. L \ M D f0g, maximal isotropic subspaces for B. We write V D L ˚ M and denote by pL;M the projection from V to L along this decomposition. Then, clearly B .pL;M .x/; y/ D B .x; pM;L .y// D B .pL;M .x/; pM;L .y// ; 8 .x; y/ 2 V  V:

164

5 Orbit Method

In the case where Wi ; Wj are transversal, the projection pWi ;Wj will be simply written as pi;j . Lemma 5.4.6. If W1 ; W3 are transversal, 123 is equal to the index of the quadratic form W2 3 x ! B .p1;3 .x/; p3;1 .x// on W2 . Proof. Since we can deform Q.x1 ; x2 ; x3 / D B.x1 ; x2 / C B.x2 ; x3 / C B.x3 ; x1 / D B .x1 ; p3;1 .x2 // C B .p1;3 .x2 /; x3 /  B.x1 ; x3 / D B .x1  p1;3 .x2 /; x3 C p3;1 .x2 // C B .p1;3 .x2 /; p3;1 .x2 // ; the change of variables y1 D x1  p1;3 .x2 /; y3 D x3 C p3;1 .x2 / gives Q.x1 ; x2 ; x3 / D B.y1 ; y3 / C B .p1;3 .x2 /; p3;1 .x2 // : So, indicating the index by , 123 D .Q/ D  .B.y1 ; y3 // C  .B .p1;3 .x2 /; p3;1 .x2 /// : Here  .B.y1 ; y3 // D 0 and the claim follows.



Lemma 5.4.7. If M is a fourth maximal isotropic subspace transversal to Wj .1  j  3/, 123 D .W1 ; W2 ; M / C .W2 ; W3 ; M / C .W3 ; W1 ; M /: Proof. By the preceding lemma, the right member of the desired equality is the index of the quadratic form Q0 .y1 ; y2 ; y3 / DB .p1;M .y2 /; pM;1 .y2 // C B .p2;M .y3 /; pM;2 .y3 // C B .p3;M .y1 /; pM;3 .y1 // DB .p1;M .y2 /; y2 / C B .p2;M .y3 /; y3 / C B .p3;M .y1 /; y1 /

5.4 Pukanszky Condition

165

on W1 ˚ W2 ˚ W3 . Now, let us prove that the changes of variables, which are reciprocal to each other, x1 D y1 C p1;M .y2 /; x2 D y2 C p2;M .y3 /; x3 D y3 C p3;M .y1 / and 1 .x1  p1;M .x2 / C p1;M .x3 // ; 2 1 y2 D .x2  p2;M .x3 / C p2;M .x1 // ; 2 1 y3 D .x3  p3;M .x1 / C p3;M .x2 // 2 y1 D

make Q.x1 ; x2 ; x3 / and Q0 .y1 ; y2 ; y3 / equivalent. In fact, B.x1 ; x2 / DB.p1;M .y2 /; y2 / C B.y1 ; y2 / C B.y1 ; p2;M .y3 // C B.p1;M .y2 /; p2;M .y3 //: Interchanging the variables by cyclic permutations, it is enough to show B.y1 ; y2 / C B.y2 ; p3;M .y1 // C B.p3;M .y1 /; p1;M .y2 // D 0: Now, writing y1 D pM;3 .y1 / C p3;M .y1 /, B.y1 ; y2 / C B.y2 ; p3;M .y1 // C B.p3;M .y1 /; p1;M .y2 // D B.pM;3 .y1 /; y2 / C B.p3;M .y1 /; p1;M .y2 //: However, as B.y1 ; p1;M .y2 // D 0, the above value is equal to B.pM;3 .y1 /; y2 /  B.pM;3 .y1 /; y2  pM;1 .y2 // D 0: In this way Q.x1 ; x2 ; x3 / D Q0 .y1 ; y2 ; y3 /.



In order to deduce Lemma 5.4.5(b), take a maximal isotropic subspace M transversal to Wj .1  j  4/, describe each term .Wi ; Wj ; Wk / by .Wp ; Wq ; M / and use the property (a), then the desired property is immediately found. Let us finally show the property (c).

166

5 Orbit Method

Lemma 5.4.8. Let L; M be two maximal isotropic subspaces of V , p a isotropic subspace of V and p? be the orthogonal subspace of p with respect to B. Then, if     p  L \ p? C M \ p? ; .L; Lp ; M / D 0.

    Proof. Since Lp D L \ p? C p, the assumption implies Lp  L \ p? C   M \ p? . If p  L, the assertion is clear. Now assume p 6 L and take a linear complement m of L \ p? in Lp contained in M \ p? . For an element  y D .y1 ; u C v; y2 / of the direct sum space L ˚ .L \ p? / ˚ m ˚ M , the quadratic form Q whose index must be computed becomes Q.y/ D B.y1 ; v/ C B.u; y2 / C B.y2 ; y1 / D B.y2  v; y1  u/ because B.u; v/ D 0. Hence the index of Q is equal to the index of the quadratic form .w1 ; w2 / 7! B.w1 ; w2 / on M ˚ L, and evidently to 0.  Let us consider the situation of Lemma 5.4.5(c). Since p is contained in the isotropic subspace .W1 \W2 /C.W2 \W3 /C.W3 \W1 /, p? contains W1 \W2 ; W2 \W3 and W3 \ W1 . Since .W1 \ W2 / C .W3 \ W1 /  W1 \ p? and W2 \ W3  W2 \ p? , we get in particular that p  .W1 \ p? / C .W2 \ p? /. Applying the last lemma to p the pairs .W1 ; W2 / and .W1 ; W2 /, .W1 ; W1p ; W2 / D .W1 ; W1p ; W2p / D 0: If we establish similar equalities for all indices .i; j /; 1  i; j  3 and apply repeatedly the property (b), we obtain .W1 ; W2 ; W3 / D .W2 ; W3 ; W1p / D .W3 ; W1p ; W2p / D .W1p ; W2p ; W3p /: 

Through a Vergne polarization h0 at f 2 g , we set Th0 2 h1 D e

i 4  .h1 ;h0 ;h2 /

Th2 h0 ıTh0 h1 :

We mention the following result without proof. Theorem 5.4.9 ([1], Theorem 5.1). The intertwining operator Th0 2 h1 does not depend on the choice of h0 and satisfies the composition formula Th0 1 h3 ıTh0 3 h2 ıTh0 2 h1 D e

i 4  .h3 ;h2 ;h1 /

:

Furthermore, if at least one of h1 , h2 is of Vergne, Th0 2 h1 coincides with Th2 h1 .

Chapter 6

Kirillov Theory for Nilpotent Lie Groups

6.1 Jordan–Hölder and Malcev Sequences In this chapter, we examine in detail the Kirillov theory for nilpotent Lie groups, which are always assumed to be connected and simply connected.

6.1.1 Unipotent Representations Definition 6.1.1. Let G be a locally compact group and let .T; V / be a finitedimensional representation on a real vector space V . We say that the representation T is unipotent if for every x 2 G, we have that T .x/ D IdV C n.x/; where n.x/ is a nilpotent endomorphism of V . Proposition 6.1.2. Let G be a nilpotent Lie group and let .T; V / be a unipotent representation of G. Then there exists for every G-invariant subspace W of V a sequence V D V0  V1      Vm D f0g of G-invariant subspaces, such that dim.Vi =Vi C1 / D 1 for every i D 0; : : : ; m  1 and such that W D Vk for some k 2 f0; : : : ; mg.

© Springer Japan 2015 H. Fujiwara, J. Ludwig, Harmonic Analysis on Exponential Solvable Lie Groups, Springer Monographs in Mathematics, DOI 10.1007/978-4-431-55288-8__6

167

168

6 Kirillov Theory for Nilpotent Lie Groups

Proof. Let X 2 g. Since T .e X / D e d T .X / D IdV C n.e X /, it follows that d T .X / D log.I C n.e X // D

dim XV j D1

1 n.e X /j j

is itself nilpotent. Hence we can apply the theorem of Engel to the modules V =W and W .  Definition 6.1.3. We say that the sequence .Vj /m j D0 is a Jordan–Hölder sequence which passes through W . Corollary 6.1.4. Let g be a nilpotent Lie algebra. There exists for every ideal a of g a Jordan–Hölder sequence which passes through a. Definition 6.1.5. We say that a basis Z D fZ1 ; : : : ; Zn g of the nilpotent Lie algebra g is a Jordan–Hölder basis, if the subspaces gj WD spanfZj ; : : : ; Zn g; j D 1; : : : ; n; form a Jordan–Hölder sequence in g. We say that a basis Y D fY1 ; : : : ; Yn g of the nilpotent Lie algebra is a Malcev basis, if the subspaces gj WD spanfYj ; : : : ; Yn g; j D 1; : : : ; n, form a Malcev sequence in g. Let h  g be a subalgebra of g. A family of vectors Y D fY1 ; : : : ; Yd g is called a Malcev basis of g relative to h, if g D ˚dj D1 RYj ˚h and if for every j D 1; : : : ; d , P the subspace hj WD diDj RYi C h is a subalgebra of g. Proposition 6.1.6. Let g be a nilpotent Lie algebra. Let h be a subalgebra of g. There exists a Malcev sequence g D g1  g2      gnC1 D f0g such that gj D h for some j 2 f1; : : : ; n C 1g. Proof. We consider the restriction of the representation adh to g=h. There exists by Engel’s theorem a vector D X1 mod h, such that ŒX1 ; h  h. Let h01 D RX1 C h. Then h01 is a subalgebra of g containing h and dim.h01 =h/ D 1. We continue in this fashion and we find a sequence of subalgebras g D h0k      h01  h such that dim.h0i =h0i 1 / D 1 for all i . We choose also a Jordan–Hölder sequence in h. This gives us our Malcev sequence in g which passes through h.  Remark 6.1.7. Construction of a Malcev basis relative to a subalgebra. Let Z D fZ1 ; : : : ; Zn g be a Jordan–Hölder basis of g and let h be a subalgebra of g. We denote by I h the index set

6.1 Jordan–Hölder and Malcev Sequences

169

I h WD fi I h C gi D h C gi C1 g; where gi WD spanfZi ; : : : ; Zn g. For i 2 I h , there exists an element Hi 2 h \ .gi n gi C1 /. Hence we can assume that Zi D Hi 2 h for those indices i . Let I g=h be the index set I g=h WD fi I gi C h ¡ gi C1 C hg: P Let us write I g=h D fi1 ; : : : ; id g. If now i 2I g=h xi Zi CH D 0 for some scalars xi and some vector H 2 h, then for the smallest index i D ij 2 I g=h , such that xi ¤ 0, we have that Zi 2 gij C1 C h. This contradiction tells us that the sum P i 2I g=h RZi C h is direct. Furthermore X X X X gD RZi D RZi C RZi D RZi C h: i 2I h

i 2I g=h

i

i 2I g=h

Finally, the subspace hj WD

X

RZi C h

i 2I g=h ;i j

is a subalgebra since it is equal to h C gj . Definition 6.1.8. Let h be a Lie subalgebra of a nilpotent Lie algebra g. Let Y D fY1 ; : : : ; Yd g be a Malcev basis of g relative to h. We denote by EY the mapping EY W Rd ! GI EY .t/ WD exp.t1 Y1 /    exp.td Yd /: Proposition 6.1.9. The mapping EY;H W Rd  H ! G D exp gI EY;H .t; h/ WD EY .t/h is a bi-polynomial diffeomorphism. Proof. We proceed by induction P on the dimension of G. If G is abelian, there is nothing to prove. Let g1 D . dj D2 RYj / C h and G1 D exp.g1 /. Then g1 is an ideal of g, since it is a subalgebra of codimension 1. The mapping E W R  G1 ! GI .t; g1 / ! exp.tY1 /g1 is a bi-polynomial diffeomorphism by Proposition 1.1.42. Hence by the induction hypothesis applied to EY1 ;H (where Y1 D fY2 ; : : : ; Yd g), since EY;H .t1 ; t2 ; : : : ; td ; h/ D E.t1 ; EY1 ;H .t2 ; : : : ; td ; h//; we have that EY;H is a bi-polynomial diffeomorphism.



170

6 Kirillov Theory for Nilpotent Lie Groups

Remark 6.1.10. We can use the mapping EG=H W Rd ! G=H I EG=H .t/ WD EY .t/H; t 2 Rd ; to describe the invariant measure on G=H . Indeed, by Proposition 5.2.9 the measure Z Cc .G=H / 3 ' 7!

Rd

'.EG=H .t//dt

is G invariant and therefore it is our G-invariant measure on G=H .

6.1.2 Polynomial Vector Groups Definition 6.1.11. A group .V; / is called a polynomial vector group, if V is a real finite-dimensional vector space, if the multiplication  is polynomial, which means that for every basis B D fX1 ; : : : ; Xn g of V , there exists polynomial functions pj W Rn ! R, such that X Y D

n X

pj ..x1 ; : : : ; xn /; .y1 ; : : : ; yn //Xj ;

j D1

XD

n X j D1

xj Xj ; Y D

n X

yk Xk 2 V;

kD1

and if for every X 2 V; r; s 2 R, we have that .sX /  .tX / D .s C t/X:

(6.1.1)

Remark 6.1.12. It follows from the definition that 0 is the identity in V and that for any X 2 V , its inverse X 1 is the vector X . Furthermore it is clear that a polynomial vector group is a Lie group and we can identify the Lie algebra v of V with the vector space V . Since for every X 2 V , the mapping R 7! V; t ! tX is a group homomorphism, it follows that the exponential mapping exp W V ! V is the identity. Let us give now a direct proof of the existence of the Campbell–Baker–Hausdorf multiplication on a nilpotent Lie algebra. Theorem 6.1.13. (a) Every polynomial vector group is a (simply connected) nilpotent Lie group. (b) Let g be a real nilpotent Lie algebra. Then there exists on g a unique group multiplication g such that the group .g; g / is a polynomial vector group, which admits the Lie algebra g as its Lie algebra.

6.1 Jordan–Hölder and Malcev Sequences

171

Furthermore, for every Jordan–Hölder basis Z D fZ1 ; : : : ; Zn g of g, there exist polynomial functions n; defined on Rj 1  Rj 1 , such that for any Pqn j ; j D 2;0 : : : ;P 0 n t; t 2 R , for g D i D1 ti Zi ; g D niD1 ti Zi , we have that gg g 0 D

n X

.tj C tj0 C qj .t1 ; : : : ; tj 1 ; t10 ; : : : ; tj0 1 //Zj

(6.1.2)

j D1

with q1 0. Proof. (a) Since the exponential mapping is the identity, we have for every X; Y 2 V; s 2 R, that .sX /  Y  .sX / D Ad.sX /.Y /: P k Hence the mapping s ! Ad.sX /.Y / D kD0 skŠ ad .X /k .Y / is polynomial in s and therefore ad .X /k D 0 for k large enough. This shows that the Lie algebra .V; Œ; / is nilpotent. (b) We proceed by induction on dim g. The result is trivial for an abelian algebra. PnChoose any Jordan–Hölder basis Z D fZ1 ; : : : ; Zn g of g. Let g1 WD j D2 RZj . We apply the induction hypothesis to g1 . Let us define a group multiplication s on g by 0

.t1 Z1 C U /s .t10 Z1 C U 0 / WD .t1 C t10 /Z1 C .e t1 ad.Z1 / /.U /g1 U 0 ; for U; U 0 2 g1 ; t1 ; t10 2 R. Then .g; s / is a Lie group, whose Lie algebra is equal to .g; Œ; /, since by induction the Lie algebra of the normal subgroup g1 of .g; s / is isomorphic to .g1 ; Œ; / and furthermore for any U 2 g1 and t 2 R, we have that .tZ1 /s U s .tZ1 / D e t adZ1 U and so ads .Z1 / D ad.Z1 /. Here ads .Z1 / denotes the adjoint action coming from the new multiplication s . It follows from the induction hypothesis that the multiplication s has property (6.1.2) of the theorem. Let us show that the exponential mapping exps W g ! .g; s / is a bipolynomial diffeomorphism of the form exps

X n j D1

 tj Zj

D

n X

.tj C rj .t1 ; : : : ; tj 1 //Zj ;

j D1

where the functions rj are polynomial. Indeed, it suffices to remark that for X D P n j 2 g, the left-invariant vector field DX determined by X is given at the j D1 xj ZP point Y D nj D1 yj Yj by

172

6 Kirillov Theory for Nilpotent Lie Groups

DX .Y / D

n X d @ .xj C pj .y1 ; : : : ; yj 1 ; tx1 ; : : : ; txj 1 /jt D0 /Zj ; Y s .tX/jt D0 D dt @t j D2

for some polynomial functions pj ; j D 2; : : : ; n. Hence the integral curve A.t/ D

n X

aj .t/Zj

j D1

determined by X satisfies the equations aP 1 .t/ D x1 aP 2 .t/ D x2 C p2 .a1 .t/; x1 / :: :: :D: aP n .t/ D xn C pn .a1 .t/; : : : ; an1 .t/; x1 ; : : : ; xn1 /: Hence a1 .t/ D tx1 ; : : : ; an .t/ D txn C rn0 .t; x1 ; : : : ; xn1 /; where the rj0 s are polynomial functions. Therefore the exponential mapping exps W g ! .g; s /, given by exps X D .x1 ; : : : ; xn Crn0 .1; x1 ; : : : ; xn1 // is a bi-polynomial diffeomorphism. We define now a new product g on g in the following way: X g Y WD logs .exp X s exp Y /; X; Y 2 g: Here log s denotes the inverse of the mapping exps . Then the new Lie group .g; g / is isomorphic to .g; s / and it satisfies the condition (6.1.2), since log s has the same property. Furthermore, for s; t 2 R, X 2 g, we have that .sX /g .tX / D log s .exp.sX /s exp.tX // D log s .exps .s C t/X / D .s C t/X: Hence expg W g ! .g; g / is the identity of g. The uniqueness follows from the following theorem.  Theorem 6.1.14. Let g be a nilpotent Lie algebra and let K D expk k be a Lie group. Let ' be a homomorphism of g into a Lie algebra k. Then there exists a unique homomorphism ˚ of .g; g / into the group K associated to k such that ˚.X / D expk .'.X //; X 2 g. Proof. We proceed by induction on the dimension of g. If g is abelian, then it suffices to write ˚.X / WD exp.'.X //; X 2 g. Then ˚ is a homomorphism. We use now the notations and procedure of the proof of Theorem 6.1.13. We define ˚ in the

6.1 Jordan–Hölder and Malcev Sequences

173

following way. By the induction hypothesis ˚1 .U / D exp.'.U //; U 2 g1 , defines a homomorphism of .g1 ; g1 /. We remark that .tZ1 / s U D tZ1 C U; U 2 g1 ; t 2 R; and use the fact that the mapping hs WD exps W g 7! g is an isomorphism of the group .g; g / onto the group .g; s /. We define: ˚s .tZ1 C U1 / WD expk .t'.Z1 //  ˚1 .U /; t 2 R; U1 2 g1 : Then ˚s is a group homomorphism: for t; t 0 2 R; U; U 0 2 g1 we have that ˚.tZ C U /˚.t 0 Z C U 0 / D expk .t'.Z1 //  ˚1 .U /expk .t 0 '.Z1 //  ˚1 .U 0 / D expk ..t C t 0 /'.Z1 //expk .t 0 '.Z1 //expk .'.U //expk .t 0 '.Z1 //expk .'.U 0 // D expk ..t C t 0 /'.Z1 //expk .e t

0 ad.'.Z // 1

D expk ..t C t 0 /'.Z1 //expk .'.e t D expk ..t C t 0 /'.Z1 //˚1 .e t

0 ad.Z / 1

0 ad.Z / 1

'.U //expk .'.U 0 // .U ///expk .'.U 0 //

.U /g1 U 0 /

D ˚s ..tZ1 C U /g .t 0 Z1 C U 0 //: Hence the mapping ˚ W .g; g / 7! K; ˚.X / WD ˚s .hs .A//; X 2 g; is also a homomorphism. Now by Theorem 1.1.15: ˚.X / D ˚.expg X / D ˚s .exps X / D expk .'.X //; X 2 g:



6.1.3 Unipotent Orbits Following Pukanszky [64] we describe now an orbit of a unipotent representation. Let for a while G D exp g be a connected and simply connected nilpotent Lie group with Lie algebra g. Let  be a unipotent representation of G on a finite-dimensional real vector space V . Applying as above Engel’s theorem to the nilpotent Lie algebra fd.X / W X 2 gg, there exists in V a Jordan–Hölder basis E D fej W 1  j  ng, n D dim V with respect to . Namely, .g/.en / D en ; .g/.ej / ej mod

n X

Rek .1  k  n  1/

kDj C1

for any g 2 G. Now we fix an G-orbit ˝ in V . Then ˝ is parametrized as follows. In particular, ˝ is an algebraic subset of V . Theorem 6.1.15. Let .; V / be a unipotent representation of the nilpotent Lie group G D exp g. Let E D fe1 ; : : : ; en g be a Jordan–Hölder basis of V . There exist

174

6 Kirillov Theory for Nilpotent Lie Groups

n polynomials fPj W 1  j  ng of d D dim ˝, real variables fxk W 1  k  d g and d indices 1  j1 <    < jd < n such that: P 1. ˝ D f nj D1 Pj .x/ej ; .x1 ; : : : ; xd / 2 Rn g; 2. Pjk .x/ D xk .1  k  d /; 3. Pj .1  j  n/ depends only on fx1 ; : : : ; xk g, where k is the maximal natural number such that jk  j . Proof. To avoid the complexity of notations, we simply write gv; X vP instead of .g/.v/; d.X /.v/ .g 2 G; X 2 g; v 2 V /. Setting VnC1 D f0g; Vj D nkDj Rek .1  j  n/, we obtain a composition series f0g D VnC1  Vn      V1 D V of the G-module V . For j D n; : : : ; 1; v 2 V let gj .v/ WD fX 2 g; X  v 2 Vj g: Then gj .v/  gj 1 .v/; j D 1; : : : ; n and the stabilizer g.v/ of v in g is the subalgebra gnC1 .v/. Let I v D f1 < j1 < j2 <    < jd  ng be the index set of v defined to be the collection of all j 2 f1; : : : ; ng such that gj .v/ ¤ gj C1 .v/. If j 2 I v , then there exists Xj 2 gj .v/ such that Xj  v 2 Vj n Vj C1 . We can then replace Xj by a scalar multiple and assume that Xj  v D ej mod Vj C1 . We see that dim.gj .v/=gj C1 .v// D 1 and so the family of vectors Bj .v/ D fXj ; j 2 I v g is a Malcev basis relative to g.v/. Now, we put gk .t/ D exp.tXjk /.t 2 R; jk 2 I v / and g.T / D g1 .t1 /g2 .t2 /  gd .td / for T D .t1 ; t2 ; : : : ; td / 2 Rd . Then by Proposition 6.1.9, we have that d ˝ D .G/.v/ D f..˘kD1 exp.RXj //vg

D f.g.T //v; T 2 Rd g: We prove the other statements by induction on dim V . Let us first show that g.T /  v D

n X

Qj .t1 ; : : : ; td /ej ; T 2 Rd ;

j D1

where the Qj ’s are polynomial functions such that 2: Qjk ..t1 ; : : : ; td // D xk C Qj0 .t1 ; : : : ; tj 1 / .1  k  d /I

6.1 Jordan–Hölder and Malcev Sequences

175

3: Qj .1  j  n/ depends only on ft1 ; : : : ; tk g; where k is the maximal natural number such that jk  j: If dim V D 1, then .g/ D IV and the conclusion is obvious. Suppose that everything is clear for dimension n  1 and take V of dimension 1 with its Jordan–Hölder basis B D fe1 ; : : : ; en g. Then V =Ren defines a unipotent G module of dimension n  1 and according to the induction hypothesis we have the polynomial functions Q1 ; : : : ; Qn1 with the properties 2 and 3 above. If nowPn is not an index, then gn .v/ D gnC1 .v/ D g.v/ and writing g.T /v D nj D1 Qj .t1 ; : : : ; td /ej ; T 2 Rd , we obtain the polynomial function Qn in the variables T D .t1 ; : : : ; td /. Suppose now that n D jd is an index for v. Then there exists Xd 2 g such that Xd  v D en . Therefore g.t1 ; : : : ; td /v D exp.t1 X1 /    exp.td 1 Xd 1 /.v C td en / D exp.t1 X1 /    exp.td 1 Xd 1 /.v/ C td en D

n1 X

Qj .t1 ; : : : ; td 1 /ej C .td C Qn0 .t1 ; : : : ; td 1 //en ;

j D1

for some polynomial function Qn0 in d  1 variables. Hence putting Qn .t1 ; : : : ; td / WD Qn0 .t1 ; : : : ; td 1 / C td and using the induction hypothesis for the polynomials Qj ; j D 1; : : : ; n  1 we can conclude. It suffices now to let xi D ti C Qi0 .t1 ; : : : ; ti 1 /; i D 1; : : : ; d to finish the proof of the theorem.  Corollary 6.1.16. Every orbit of a unipotent representation of a connected and simply connected nilpotent Lie group is a Zariski closed subset of V . Corollary 6.1.17. If G D exp g is nilpotent, then I.f; g/ D M.f; g/ at any f 2 g . Proof. Let h 2 M.f; g/ and H D exp h. As we have already seen, H f is an open set of f C h? and at the same time a closed set by Corollary 6.1.16. Thus H f D f C h? , since it is connected.  Remark 6.1.18. Let G D exp g be a connected and simply connected nilpotent Lie group. We fix a Jordan–Hölder basis Z D fZn ; : : : ; Z1 g of g. i.e. Z is a basis of g and the sequence gj WD spanfZ1 ; : : : ; Zj g is an ideal of g for every j D 1; : : : ; n. Its dual basis Z  D fZ1 ; : : : ; Zn g is then a Jordan–Hölder basis of the G-module g and the subspaces gj WD spanfZj ; : : : ; Zn g are then G-invariant for every j D 1; : : : ; n. Let ` 2 g and O D O.`/ its coadjoint orbit. The subalgebra

176

6 Kirillov Theory for Nilpotent Lie Groups

gj .`/ WD fU 2 gI ad .U /` 2 gj g D fU 2 gI ad .U /` 2 spanfZj ; : : : ; Zn gg D fU 2 gI h`; ŒU; gj 1 i D f0gg: is then just the stabilizer in g of the restriction of ` to the ideal gj 1 . Hence ` gj .`/ D gBj 1 :

Here B` denotes the skew symmetric bilinear form B` .U; V / WD h`; ŒU; V i; U; V 2 g; of g and for a subspace v of g, vB` WD fU 2 gI B` .U; v/ D f0gg denotes as before the orthogonal of v with respect to B` . It is easy to see that .vB` /B` D v C g.`/: Therefore, since for any j in f1; : : : ; ng, j is in the index set I ` D IZ`  WD f1  j  nI gj .`/ ¤ gj 1 .`/g of ` if and only if ` gBj 1 ¤ gBj `

or equivalently gj 1 C g.`/ ¤ gj C g.`/: Hence IZ`  D I ` D fj 2 f1; : : : ; ngI gj C g.`/ ¤ gj 1 C g.`/g: In particular, we see that an index j is not contained in I ` , if and only if there exists an element of g of the form Tj0 .`/ D Tj0 D Zj C

X

ai .`/Zi 2 g.`/:

1i <j

If we replace the vectors Zi ; i 62 I ` ; 1  i  j  1; by Ti0 , then we get a vector Tj .`/ D Tj D Zj C

X 1i <j;i 2I `

ai .`/Zi 2 g.`/:

(6.1.3)

6.1 Jordan–Hölder and Malcev Sequences

177

Proposition 6.1.19. Let G D exp g be a connected and simply connected nilpotent Lie group, let ` 2 g and let as in Theorem 6.1.15: X n

O D O.`/ D

pjZ .z/Zj I z

2R

d

j D1

be its coadjoint orbit. There exists, up to a scalar multiplication, a unique G-invariant Borel measure dO on O. This measure can be described in the following way. Let Z  be a Jordan–Hölder basis of the G-module g . Then we have that ! Z Z n X Z  '.q/dO .q/ D ' pi .z/Zi d z; ' 2 Cc .g /: (6.1.4) Rd

O

i D1

Proof. Let d WD dim O D dim.g=g.`//. As in the proof of the Theorem 6.1.15, we choose for every j 2 I ` D fj1 ; : : : ; jd g an element Xj 2 g, such that ad .Xj /` D Zj modulo gj 1 . This gives us a Malcev basis X WD fXj1 ; : : : ; Xjd g of g relative to g.`/. Since O is closed in g , we see that the mapping EQ X W Rd ! OI EQ X .t1 ; : : : ; td / D Ad 

d Y

! exp.ti Xji / `

i D1

is a homeomorphism and so we can define a measure dO by Z

Z '.q/dO .q/ WD

Rd

O

'.EQ X .t//dt; ' 2 Cc .g /:

To see the invariance of this measure, let g 2 G and ' 2 Cc .g /. Then Z

Z



'.Ad .g/q/dO .q/ D O

Z

'.Ad .g/.EQ X .t///dt

! Z d   Y ' Ad g .exp.ti Xji // ` dt D

'.Ad .gh/`/d hP



D Z

Rd

Rd

G=G.`/

i D1

'.Ad .h/`/d hP D 

D

Z Rd

G=G.`/

'.EQ X .t//dt D

Z '.q/dO .q/; O

Q where h D diD1 exp.ti Xji /. In order to prove Eq. (6.1.2), it suffices to observe that by Theorem 6.1.15 and the notations of its proof: EQ X .t/ D

n X

.qj .t1 ; : : : ; td //Zj

j D1

(6.1.5)

178

6 Kirillov Theory for Nilpotent Lie Groups

and so, writing qj .t1 ; : : : ; td / D pjZ .z1 ; : : : ; zd /, Z

Z '.q/dO .q/ D

' Rd

O

' Rd

 qj .t/Zj dt

j D1

Z D

n X

n X

 pjZ .z/Zj d z

j D1

.by the change of variables zi WD ti C qj0 i .t1 ; : : : ; ti 1 //:  Proposition 6.1.20. Let Z D fZ1 ; : : : ; Zn g be a Jordan–Hölder basis of the nilpotent Lie algebra g. Then there exist polynomial functions pj W Rj 1  g ! R; j D P2; : : : ; n, which are linear in X 2 g, such that the vector field X at the point Y D niD1 yi Zi 2 g is given by X.Y / D

n X

.xj C pj .y1 ; : : : ; yj 1 ; X //Zj :

j D1

Proof. Indeed, we know from (6.1.4), that for X; Y 2 g, Y g X D

n X

.yj C xj C qj .y1 ; : : : ; yj 1 ; x1 ; : : : ; xj 1 //Zj :

j D1

Hence @ Y g .tX /jt D0 @t n X @ D .xj C qj .y1 ; : : : ; yj 1 ; tx1 ; : : : ; txj 1 /jt D0 /Zj @t j D1

X.Y / D

D

n X

.xj C pj .y1 ; : : : ; yj 1 ; X //Zj ;

j D1

where pj .y1 ; : : : ; yj 1 ; X / D

@ qj .y1 ; : : : ; yj 1 ; tx1 ; : : : ; txj 1 /jt D0 : @t



6.2 Schwartz Spaces

179

6.2 Schwartz Spaces Definition 6.2.1. We denote for a finite-dimensional real vector space V the algebra of all partial differential operators with complex polynomial coefficients on V by PD.V /. Let B D fB1 ; : : : ; Bn g be a basis of V . We define for ˛ 2 Nn the monomial function XD

n X

xj Bj 7! X ˛ WD x1˛1    xn ˛n

j D1

and the partial differential operator D˛ D

 @ ˛1  @ ˛n  : @x1 @xn

The Schwartz space S.V / is by definition the space of all smooth functions f W V ! C, such that for every  2 PD.V / the function f is in L2 .V / with respect to Lebesgue measure on V . We put on S.V / the family of norms kf

k2k

WD

XZ j˛jk

.1 C kX k2 /k kD ˛ f .t/k2 dt; f 2 S.V /:

V

(Here kX k2 denotes a Euclidean norm on V .) The space S.V / becomes in this way a Fréchet space. Definition 6.2.2. Let G D .g; g / a nilpotent Lie group. We choose again a Jordan– Hölder basis Z D fZ1 ; : : : ; Zn g of g and we define for ˛ 2 Nn , the element Z ˛ WD Z1˛1   Zn˛n of the enveloping algebra U.g/ of g. These elements Z ˛ ; ˛ 2 Nn ; form a basis of U.g/, by the Birkhoff–Poincare–Witt theorem. Let PU.g/ be the algebra of partial differential operators generated by the elements of U.g/ and the algebra P.g/ of complex-valued polynomial functions on g. It follows from Proposition 6.1.20 that PD.g/ D PU.g/: An element X of g acts on the left on a smooth function ' W G ! C by X '.g/ WD

d '.exp.tX /g/jt D0 ; g 2 G: dt

We extend this action to an action of U.g/ on the space C 1 .G/. Definition 6.2.3. Let G be a connected locally compact group and let U D U 1 be a compact symmetric neighbourhood of the identity. Then we know that

180

6 Kirillov Theory for Nilpotent Lie Groups

S G D n2N U n , where U n denotes the compact subset U n WD fu1   un I ui 2 U , i D 1; : : : ; ng and U 0 D ;. We define a function wU D w W G ! N by w.x/ D n; if x 2 U n nU n1 : In particular w.e/ D 1. Let !.x/ D 1 C w.x/; x 2 G. Proposition 6.2.4. For x; y 2 G, we have that w.xy/  w.x/Cw.y/. Furthermore !.xy/  !.x/!.y/. Proof. Let n D w.x/, m D w.y/. Then xy 2 U nCm and so w.xy/  n C m D w.x/ C w.y/. Therefore !.xy/ D 1 C w.xy/  1 C w.x/ C w.y/  .1 C w.x//.1 C w.y// D !.x/!.y/:  Proposition 6.2.5. Let k  k be a Euclidean norm on the nilpotent Lie algebra g and let G D .g; g /. Let U be the unit ball of centre 0 in g. There exists a constant C > 0 and an integer d such that wU .X /  1 C kX k  C.1 C wU .X //d ; X 2 g: Proof. Let X 2 g and choose n 2 N, such that n  kX k < n C 1. Let Y WD 1 nC1 2 U nC1 . This shows that nC1 X . Then Y 2 U and X D .n C 1/Y D Y w.X /  n C 1  kX k C 1. In order to find the constant C and the integer d , we show by induction on the dimension of g that there exists a polynomial function with non-negative coefficients p, such that for any m 2 N, X 2 U m , we have that kX k  p.m/: If g is abelian, then U m D mU and we can take p.s/ D s; s 2 R. Let g1 be an ideal of g of codimension 1. We write g D RY ˚ g1 for some Y 2 g n g1 . We take a neighbourhood V of 0 of the form V D exp.  1; 1ŒY /W , where W is the exponential of the unit ball of centre 0 in g1 . Any element X of V m can be written as XD

m Y

exp.sj Y /exp.Uj /

j D1

D exp..s1 C    C sm /Y /

m Y

exp.e .sj C1 CCsm /adY Uj /exp.Um /;

j D1

where s1 ; : : : ; sm 2  1; 1Πand Uj 2 W for j D 1; : : : ; m.

6.2 Schwartz Spaces

181

Since ad.Y / is a nilpotent endomorphism of g, it follows that ke sadY .U /k  q.jsj/; s 2 R; U 2 W; for some polynomial q with integer coefficients and therefore exp.e sadY / U  W q.Œs/ , taking now q.s C 1/ as the new q.s/. Hence X 2 exp.  m; mŒY /W q.m1/CCq.1/C1  exp.  m; mŒY /W mq.m1/ and so kX k  p 0 .m/ for some polynomial p 0 with non-negative coefficients, since there exists by the induction hypothesis a polynomial p1 with non-negative coefficients such that for any U 2 W k ; kU k  p1 .k/; k 2 N. Since the unit ball there exists an integer r, such that U  V r , since S1U inr g is compact, m G D rD1 V . Hence, U  V rm and we see that we can take as p the polynomial p.s/ WD p 0 .rs/; s 2 R. It follows that for X 2 g with wU .X / D m, X is contained in U m , therefore kX k  p.m/ D p.wU .X //  C.1 C kX k/d for some constant C > 0 and d 2 N being the degree of the polynomial p.  Definition 6.2.6. Let g be a nilpotent Lie algebra and let G be its connected and simply connected Lie group. We denote by S.G/ the Fréchet space of all smooth functions ' W G ! C, such that k'kN WD

X Z

!.x/N jZ ˛ '.x/jdx < 1; 8N 2 N: G

j˛jN

It follows from Proposition 6.2.5 and from Proposition 6.1.20 that the Fréchet space S.G/ is equal to the Fréchet space S.g/. It is easy to see that S.G/ is also a Fréchet algebra. Indeed, for f; g 2 S.G/ we have that kf gkN D D D  

X Z

!.x/N jZ ˛ f g.x/jdx

j˛jN

G

j˛jN

G

j˛jN

G

j˛jN

GG

j˛jN

GG

X Z

X Z

!.x/N j.Z ˛ f / g.x/jdx ˇZ ˇ ˇ ˇ !.x/N ˇˇ .Z ˛ f /.y/g.y 1 x/dy ˇˇdx

X Z X Z

G

!.yx/N j.Z ˛ f /.y/jjg.x/jdydx !.y/N !.x/N j.Z ˛ f /.y/jjg.x/jdydx

182

6 Kirillov Theory for Nilpotent Lie Groups



 X Z j˛jN

Z !.x/ jZ f jdx

G

N

!.y/N jg.y/jdy

˛

G

 kf kN kgkN : Here we have used the fact that ! N is submultiplicative. Definition 6.2.7. Let H D exp h be a closed connected subgroup of the connected and simply connected nilpotent Lie group G D exp g. Let  W H ! T be a unitary character of H . Let Y D fY1 ; : : : ; Yd g be a Malcev basis of g relative to h. We define the Schwartz space S.G=H; / of the representation  WD indG H  as the subspace of all smooth functions ' W G ! C contained in the Hilbert space H D L2 .G=H; / of  for which the function 'ıEY W Rd ! C, is in S.Rd /. It follows from Proposition 6.1.9, that the definition of the space S.G=H; / does not depend on the choice of the Malcev basis Y. Definition 6.2.8. Let .; H/ be a unitary representation of the Lie group G. We denote by H1 the space of the C 1 -vectors of H. This means that a vector is in H1 , if and only if the mapping G ! HI g 7! .g/ is C 1 . We can give the vector space H1 the structure of a Fréchet space. It suffices to take on H1 the family of norms k kN WD sup kd .Z ˛ / kH ; 2 H1 .N 2 N/: j˛jN



We denote by K.H/ (resp. by B.H/), the space of compact linear operators (resp. of bounded linear mappings), on H. Let B.H/1 be the space of all smooth linear operators on H. Here a linear operator u on H is called smooth or C 1 , if the mapping G  G ! K.H/I .g; g 0 / 7! .g/ıu ı .g 0 /1 is smooth in the strong operator topology. This means that we use the representation R of GG on Banach space K.H/ defined by R .g; g 0 /u WD .g/ıuı.g 0 /1 ; g; g 0 2 G; u 2 B.H/: If we take the subspace L2 .H/ of B.H/ consisting of all Hilbert–Schmidt operators on H, the representation R of G restricts to a unitary representation, since L2 .H/ is a Hilbert space and the mapping .g; g 0 / ! R .g; g 0 /u; u 2 L2 .H/, is continuous for the Hilbert–Schmidt norm. Definition 6.2.9. Let G be a Lie group. We denote by L1 .G/1 the smooth vectors in L1 .G/ for the regular representation  of G  G on L1 .G/: .g; g 0 /f .x/ D f .g 1 xg 0 /; g; g 0 ; x 2 G; f 2 L1 .G/: If .; V/ is a bounded representation of G on a Banach space V, then .f /V  V 1 for any f 2 L1 .G/1 . If G is a connected and simply connected nilpotent Lie group, then the Schwartz algebra S.G/ is contained in L1 .G/1 .

6.2 Schwartz Spaces

183

We want now to study the image of the enveloping algebra under an irreducible representation of a nilpotent Lie group. Let G D exp g be a connected and simply connected nilpotent Lie group with Lie algebra g. We study the differential O We know that representation d  of  2 G. .exp.tX // D exp .t.d /.X // .t 2 R/ for any X 2 g. In this situation, we can do the following. Remark 6.2.10. Let G D exp g be a nilpotent Lie group. Let ` 2 g . Choose a polarization p at ` let `;p D indG P ` be the corresponding irreducible representation. We choose a Malcev basis X D fX1 ; : : : ; Xd g relative to p, for instance as in Remark 6.1.7. We can then identify as in Definition 6.2.7 the Hilbert space L2 .G=P; ` / with the Hilbert space L2 .Rk /, using the unitary operator UX : UX . /.x/ WD .EX .x//; x 2 Rk : X X ; L2 .Rk // the representation `;p WD UX ı `;p ı UX . We denote by .`;p

Definition 6.2.11. We denote as before by PD.Rn / the algebra of differential operators with polynomial coefficients on Rn (n 2 N ). Lemma 6.2.12. Let g be a real nilpotent Lie algebra and let 0 ¤ ` 2 g . Let (as in Lemma 5.3.7) a` be the largest ideal of g contained in the subalgebra g.`/. Then `ja` ¤ 0. Let Y 2 g n a` , such Œg; Y   a` . Then the subspace g1 WD fU 2 gI h`; ŒU; Y i D 0g D ker.ad  .Y /`/ of g is an ideal of codimension 1. For any g 2 G and a 2 A` WD exp.a` / we have that h`; log.gag 1 /i D h`; log ai. Proof. Suppose that ` is zero on a` . Then for our Y we have that h`; Œg; Y i D f0g and so Y 2 g.`/ and the subspace RY C a` is an ideal of g contained in g.`/, a contradiction. The same argument tells us that there must exist X 2 g, such that h`; ŒX; Y i D 1. Since for any U; V 2 g we have that h`; ŒŒU; V ; Y i D h`; ŒŒU; ŒV; Y i  h`; ŒŒV; ŒU; Y i D 0 C 0 D 0; (as Œg; Y   g.`// it follows that Œg; g  g1 . So we have that g1 D ker.ad  .Y /`/ is an ideal of g of co-dimension 1. Furthermore, by the definition of a` , we know that for any S 2 a` ad k .g/.Y /  g.`/ for any k 2 N .



184

6 Kirillov Theory for Nilpotent Lie Groups

Theorem 6.2.13 (Kirillov). Let G D exp g be a simply connected nilpotent Lie group. Let  D `;p be an irreducible unitary representation of G. Let X D fX1 ; : : : ; Xk g be a Malcev basis of g relative to p. Identify the Hilbert space L2 .G=P; ` / with L2 .Rd / via the unitary operator UX W L2 .G=P; ` / ! L2 .Rk / W 7! ı EX : X of U.g/ is equal to the algebra PD.Rn /. Then the image under d `;p

Proof. It is easy to see that if we have proved the theorem for one Malcev basis X of g relative to p, the theorem is also proved for every other Malcev basis X 0 relative to p, since the passage from X to X 0 is given by bi-polynomial bijections, which preserve the spaces S.Rn / and PD.Rd /: X Let us first show that d `;p .U.g//  PD.Rd /. Choose a Jordan–Hölder basis Z D fZ1 ; : : : ; Zn g of g to introduce coordinates of the second kind on G. For g 2 G and t 2 Rd , we write the product g 1 EX .t/ in the coordinates of Rd and p. We have a polynomial mapping Q W G  Rd ! Rd and a polynomial mapping R W G  Rd ! p, such that g 1 EX .t/ D EX .Q.g; t//exp.R.g; t//; t 2 Rd ; g 2 G: Hence X `;p .g/ .EX .t// D e i h`;R.g;t /i .EX .Q.g; t///; t 2 Rd ; g 2 G; 2 L2 .G=P; ` /:

This implies that X `;p .g/.t/ D e i h`;R.g;t /i .Q.g; t//; t 2 Rd ; g 2 G;  2 L2 .Rd /: X .X / 2 PD.Rd / and Hence by differentiation, we see that for every X 2 g, d `;p X d therefore d `;p .U.g//  PD.R /. X In order to show that PD.Rd /  d `;p .U.g//, we proceed by induction on the dimension of g. If G is abelian, then d D 0 and there is nothing to prove. We can assume now that ` does not vanish on a` and so we have X 2 g; Y 2 g n a` , such that Œg; Y   g` , such that h`; ŒX; Y i D 1 and such that ker.ad  .Y /`/ D g1 is a co-one-dimensional ideal in g. Let G1 D exp.g1 /. Then G1 is a closed normal subgroup of G. Suppose first that p  g1 . Choose our Malcev basis X relative to p, such that X D fX g [ X1 , where X1 a Malvev basis of g1 relative to p. Then p is a polarization X1 at `1 D `jg1 and we have that `;p ' indG G1 `1 ;p . To simplify notations, let 1 1 WD `X1 ;p . X with We can identify the Hilbert space L2 .Rd / of the representation `;p

L2 .R; L2 .Rd 1 //

6.2 Schwartz Spaces

185

through the mapping U. /.t/.T1 / WD .t; T1 /; t 2 R; T1 2 Rd 1 : We define a new representation  of G on the space H D L2 .R; H1 / by X .g/ WD U ı `;p .g/ ı U  ; g 2 G:

(6.2.1)

X Then of course  is equivalent `;p . Let us compute the operators .g/ for g 2 G. Let  2 L2 .R; H1 /; t 2 R. Then we have

.exp.xX //.t/ D .t  x/

(6.2.2)

.exp.yY //.t/ D e iyt .t/ .g1 /.t/ D 1 .g1t /..t//; where g1 2 G1 ; t; x; y 2 R and where g1t WD exp.tX /g1 exp.tX /. This gives us for the elements X; Y of g and d1 2 U.g1 /, for a smooth vector  that d .t/ dt d .Y /.t/ D i t.t/

d .X /.t/ D 

d .d1 /.t/ D d 1 .d1t /..t//; where d1t D

P1

j 1 j j D0 j Š .t/ ad .X /d1 :

(6.2.3)

The mapping d1 7! d1t is an automorphism

of U.g/ for every t 2 R and we have that .adk .A/.d1 //t D adk .At /d1t for any A 2 g. Since g is nilpotent, we see that the sum above is finite and so d t is contained in U.g1 /. Let for d 2 U.g1 / dQ D

1 X

1 Y j adj .X /d: j j Š.i / j D0

Again the sum above is finite and so dQ is contained in U.g/. Let us compute d .dQ1 / where d1 is contained in U.g1 /. We have for a smooth vector  2 L2 .R; H1 / and t 2 R, that d .dQ1 /.t/ D d 1 ..dQ1 /t /..t// X  1 1 j j t D d 1 .Y ad .X /d1 / ..t// j Š.i /j j D0

186

6 Kirillov Theory for Nilpotent Lie Groups

D d 1

D d 1

X 1

 1 j t j t .Y / .ad .X /d / ..t// 1 j Š.i /j j D0

X 1

 1 j t j t .Y / .ad .X /.d1 // ..t//: j Š.i /j j D0

Since d 1 .Y t / D d 1 .Y  tZ/ D i tIH1 , we get therefore X 1

tj .adj .X /.d1t ///..t// j Š j D0  D d 1 ..d1t /t ..t// D d 1 .d1 /..t//:

d .dQ1 /..t// D d 1

Using the induction hypothesis, this computation shows that the algebra of X d `;p .U.g// contains the operators @t@1 ; Mt1 (i.e. the multiplication operator with the variable t1 ) and every partial differential operator with polynomial coefficients X in the variables t2 ; : : : ; td . Hence d `;p .U.g// D PD.Rd /. If p 6 g1 , then we can take our X in p, and even assume that h`; X i D 0, and we consider the polarization p0 WD p \ g1 C RY . The intertwining operator U for  D `;p and  0 D `;p0 from L2 .G=P; ` / to L2 .G=P 0 ; ` / is given by Z U. /.g1 exp.xX // D

R

.g1 exp.xX /exp.yY //dy

Z D

R

Z D

R

.g1 exp.yY /exp.yxZ/exp.xX //dy e iyx .g1 exp.yY //dy;

for x 2 R and g1 2 G1 . This shows that if we take a Malcev basis X D fX1 ; : : : ; Xd 1 ; Y g of g relative to p with fX1 ; : : : ; Xd 1 g  g1 , which gives us a Malcev basis X 0 D fX1 ; : : : ; Xd 1 ; X g of g relative to p0 , then the X0 X intertwining operator for `;p 0 and `;p is just a partial Fourier transform in the variable y1 . This intertwining operator then maps PD.Rd / into itself.  Corollary 6.2.14. Let `;p be an irreducible representation of the simply connected nilpotent Lie group G. Then H1X D S.Rdim.G=P / /: `;

Proof. It is well known that the Schwartz space S.Rn / is also the space of all L2 functions on Rn , such that for every D 2 PD.Rn /, the distribution D is also in L2 .Rn /. The corollary then follows from Theorem 6.2.13. 

6.2 Schwartz Spaces

187

Proposition 6.2.15. Let `;p be an irreducible unitary representation of the nilpotent Lie group G D exp g. Let u 2 B.L2 .Rd // .d D dim.G=P // be a X smooth operator for the representation `;p . Then u is trace class and its kernel d d function ku W R  R ! C is contained in S.Rd  Rd /. Proof. For any D 2 PD.Rn / the operator uıD, which is a priory defined on the Schwartz space S.Rd /, extends to a bounded operator on L2 .Rd /. There exist elements D 2 PD.Rd /, which can be inverted on S.Rd /, and whose inverse D 1 extends to a trace class operator T on L2 .Rd /. Hence u D .uıD 1 /ıT is a trace class operator. Let us construct such an operator D, using Hermite functions. Let 2 2 d m s 2 / .e /; s 2 R: Then m .s/ D e s =2 Qm .s/, where Qm is a

m .s/ WD e s =2 . ds polynomial of degree m; m 2 N/. Let hm D

1

m ; m 2 N: k m k2

Then the functions .hm /m2N form an orthonormal basis of L2 .R/. Let d2 2 D1 WD  ds 2 C s . It follows inductively on m, that D1 hm D .2m C 1/hm ; m 2 N:

(6.2.4)

We define now the element D 2 PD.Rd / by D WD D12 ˝    ˝ D12 „ ƒ‚ … d times

and for  D .m1 ; : : : ; md / 2 Nd , let h WD hm1 ˝    ˝ hmd : Then the family .h /2Nd constitutes an orthonormal basis of L2 .Rd /. Furthermore, by (6.2.1), we have that Dh D .2m1 C 1/2   .2md C 1/2 h ;  2 Nd : Hence D is invertible on S.Rd / and its inverse T acts on the vectors h by T h D

1 h ;  2 Nd : .2m1 C 1/2   .2md C 1/2

This shows that T is a positive trace class operator, since Tr.T / D

X 2Nd

1 D .2m1 C 1/2   .2md C 1/2

X m2N



1 2 2m C 1

d < 1:

188

6 Kirillov Theory for Nilpotent Lie Groups

Since now u is a trace-class and hence a Hilbert–Schmidt operator, its kernel function is in L2 .Rd  Rd /. But u is a smooth operator, hence for every X X D D d `;p .A/; D 0 D d `;p ..AL 0 // 2 PD.Rn /, the derivative in the distribution sense of the kernel function ku by D in the first variable and D 0 in the second X X .A/ ı u ı d `;p ..AL 0 //, variable gives us the kernel function of the operator d `;p 2 which is again an L -function. This shows that ku is a smooth function and also an element of S.Rn  Rn /.  Corollary 6.2.16. Let G be a connected and simply connected nilpotent Lie group and let `;p be an irreducible unitary representation of G. Then for every element f 2 L1 .G/, the operator `;p .f / is compact. Proof. The Schwartz algebra S.G/ is dense in L1 .G/ and for every f 2 S.G/ the operator `;p .f / is trace class, hence compact.  Remark 6.2.17. Let us compute for f 2 S.G/ the kernel function of the operator `;p for an element ` 2 g and a polarization p at `. Let g 2 G and

2 L2 .G=P; ` /. Then Z `;p .f / .g/ D

f .t/`;p .t/ .g/dt Z

G

Z

G

D

f .t/ .t Z

Z D

Z g/dt D

f .gt 1 / .t/dt

G

f .gp 1 t 1 /` .p 1 /dp .t/dt

D G=P

1

P

f`;p .g; t/ .t/dt; G=P

where Z

f .gpt 1 /` .p/dp; g; t 2 G:

f`;p .g; t/ WD P

This function f`;p is smooth, and it satisfies the covariance condition f`;p .gp; tp 0 / D ` .p 1 /` .p 0 /f`;p .g; t/; p; p 0 2 P; g; t 2 G: Let X be a Malcev basis of g relative to p. Then we know from Proposition 6.2.15, that the function f`;p ı .EY ˝EY / is contained in S.Rd Rd /, where d D dim.G=P /, since it is the kernel function of the operator `;p .f /. This allows us to define the space of the Schwartz kernels.

6.2 Schwartz Spaces

189

Definition 6.2.18. ˚ SK.G=P; ` / W D K W G  G ! CI K.gp; g 0 p 0 / D ` .p 1 /` .p 0 /K.g; g 0 /; g; g 0 2 G; p; p 0 2 P

and K ı .EY ˝EY / 2 S.Rd  Rd / : These Schwartz kernels are just the kernel functions of smooth operators on L2 .G=P; ` /. We can also define the Schwartz space of functions with covariance conditions Let H D exp h be a closed connected subgroup of the nilpotent Lie group G D exp g and let  be a unitary character of H . Let Y be a Malcev basis of g relative to h. Then S.G=H; / WD f 2 L2 .G=H; /; ı EY 2 S.Rdim.G=H / /g: Theorem 6.2.19. Let G D exp g be a simply connected nilpotent Lie group. Let ` 2 g and let p be a polarization at `. Let d WD dim.g=p/. Let X be a Malcev basis of g relative to p. There exists for every N 2 N a continuous linear mapping, which we call a retract, R D R`;p;X W S.RN ; S.Rd  Rd // 7! S.RN ; S.G// such that for every s 2 RN and F 2 S.RN ; S.Rd  Rd // the (smooth operator) X `;p .R.F .s///

admits the function F .s/ as kernel function. Proof. It is easy to see that if the theorem is true for one Malcev basis relative to p, it is also true for any such Malcev basis. For simplicity of notation, let S.N; d / WD S.RN ; S.Rd  Rd //. We proceed by induction on the dimension of G. If G is abelian, then d D 0. We choose a Schwartz function ' 2 S.G/, such that '.`/ O D 1 and it suffices to put R.F .s// WD F .s/'; s 2 Rd . We can assume now that ` does not vanish on a` and that a` ¤ g. So we have X 2 g; Y 2 g n a` , such that Œg; Y   g` , such that h`; ŒX; Y i D 1, h`; Y i D h`; X i D 0 and such that ker.ad  .Y /`/ D g1 is a co-one-dimensional ideal in g. Let G1 D exp.g1 /. Then G1 is a closed normal subgroup of G. Let `1 WD `jg1 . We take first p inside g1 and p is then a polarization at `1 . Choose our Malcev basis X relative to p, such that X D fX g [ X1 , where X1 a Malvev basis X1 of g1 relative to p. Then we have that `;p ' indG G1 `1 ;p . To simplify notation, let 1 1 WD `X1 ;p .

190

6 Kirillov Theory for Nilpotent Lie Groups

X We can identify the Hilbert space L2 .Rd / of the representation `;p with

L2 .R; L2 .Rd 1 // through the mapping U. /.t/.T1 / WD .t; T1 /; t 2 R; T1 2 Rd 1 : We define a new representation  of G on the space H D L2 .R; H1 / by X .g/ WD U ı `;p .g/ ı U  ; g 2 G:

(6.2.5)

X Then of course  is equivalent `;p . As before, we compute .f / on a vector 2 H . Let t 2 R. Then

Z Z .f / .t/ D

f .exp.xX /g1 /.exp.xX /g1 / .t/dxdg1

R

Z Z

G1

R

G1

R

G1

D Z Z D Z D R

f .exp.xX /g1 /1 .g1t x / .t  x/dxdg1 f .exp..t  x/X /g1x /1 .g1 / .x/dxdg1

1 .f x .t  x//. .x//dx;

(6.2.6)

where f x .u/ 2 S.G1 / denotes the Schwartz function f x .u/.g1 / WD f .exp.uX /g1x / D f .exp.uX /exp.xX /g1 exp.xX //; x; u 2 R; g1 2 G1 : Hence the operator .f / is given by the operator-valued kernel function k.t; x/ D 1 .f x .t  x// 2 B.H1 /; t; x 2 R:

(6.2.7)

Define the (normal) subgroup A of G by A WD exp a where a WD RY C a` . Now let S1 W S.RN ; S.Rd  Rd // ! S.RN C2 ; S.Rd 1  Rd 1 //; F ! F1 be defined by the rule: F1 ..s; t; x/; t ; x/ WD F .s; .t C x; t /; .x; x//; where t; x 2 Rd 1 ; s 2 RN ; t; x 2 R: The function F1 is obviously in S.RN C2 ; S.Rd 1  Rd 1 // and the mapping F 7! F1 , S.N; d / ! S.N C 2; d  1/, is continuous.

6.2 Schwartz Spaces

191

We apply the induction hypothesis and we get the retract R1 W S.N C2; d 1/ ! S.RN C2 ; S.G1 //. We define a new function hF W RN C2  G1 ! C by Z hF ..s; t; x/; g1 / WD

R1 F1 ..s; t; x//.g1 a/` .a/da: A

By definition, the function hF satisfies the following covariance relation: hF ..s; t; x/; g1 a/ D ` .a/1 hF ..s; t; x/; g1 /; s 2 RN ; t; x 2 R; g1 2 G1 ; a 2 a: Choose a Jordan–Hölder basis Z D fZ1 ; : : : ; Zm g of g relative to a` , such that Z1 D X; Zm D Y; g1 D spanfZ2 ; : : : ; ZM ; a` g: Let v WD spanfZ2 ; : : : ; Zm1 g. Then we have g1 D v ˚ RY ˚ a` . Let V WD exp v. We obtain the decomposition V  R2  A` ! G1 ; .v; x; y; a/ 7! vexp.yY /exp.zZ/exp a of G1 . It follows again from the definition of hF that the function RN  R2  V 3 .s; t; x; v/ 7! hF ..s; t; x/; v/ is a Schwartz function and that the linear mapping S.N; d / ! S.RN  R2 V / is continuous. Let ' 2 S.A` / such that '.0/ O D 1. We obtain now our retract R in the following way: for F 2 S.N; d / let Z RF .s/.exp.xX /g1 a/ WD '.a/

R

hF ..s; x; u/; g1u /d u;

for s 2 RN ; x 2 R; g1 2 V exp.RY /; a 2 A` . Here, as before, for g1 2 G1 we use the notation: g1u WD exp.uX /g1 exp.uX /; u 2 R: We observe that RF .s/.exp.xX /vexp.yY /a/ Z D '.a/ hF ..s; x; u/; .vexp.yY /a/u /d u R

D '.a/` .a/

Z R

e i uy hF ..s; x; u/; vu /d u:

for s 2 RN ; x; y; z 2 R; v 2 V . From this formula, we easily deduce that the function RF is in S.RN G/ and that the linear mapping F 7! RF is

192

6 Kirillov Theory for Nilpotent Lie Groups

continuous. We have to check that F .s/ D k.RF .s// for every F 2 S.N; d /. Let f .s/ WD RF .s/ 2 S.G/; s 2 RN . We use formula (6.2.6): let t; x 2 R. Then Z 1 .f .s/x .t  x// D Z

Z D

1 .v/ V

e

e R

Z

R

Z



iyx

iyu

G1

.g1 /f .s/.exp..t  x/X /.g1x //dg1

` .a/` .a/'.a/1 .v/ A

hF ..s; t  x; u/; .v

x u





/ /d u dady d v

Z D '.0/ O

1 .v/hF ..s; t  x; x/; v/d v V

Z Z D

1 .vexp.yY /a/R1 F1 .s; t  x; x/.vexp.yY /a/dydad v V

A

D 1 .R1 F1 .s; t  x; x//; X by the property of the retract R1 . Hence the kernel of the operator `;p .RF .s// is the function

F .s; t  x C x; t ; x; x/ D F .s; t; t ; x; x/: If now p 6 g1 , then we replace p by the polarization p0 D p \ g1 C RY . Since the spaces of the Schwartz kernels SK.G=P; ` / and SK.G=P 0 ; ` / are isomorphic and since this isomorphism is given by the composition with the intertwining operator U between `;p and `;p0 , i.e. UF .g; g0 / D

Z

F .gexp.yY /; g 0 exp.y 0 Y //dydy 0 ; R2

it suffices to use the retract R0 constructed for p0 to compose R with U . This finishes the proof of the theorem.  Corollary 6.2.20. Let G be a connected and simply connected nilpotent Lie group. Let .; H/ be an irreducible unitary representation of G. Then the ideal F D ff 2 L1 .G/I rank ..f // < 1g P of L1 .G/ is different from f0g and the sub-module H0 D f 2F Im..f // of 1 .L .G// is simple. The same statement is true if we replace L1 .G/ by the involutive algebra S.G/. X for some ` 2 g , some polarization p at ` and some Malcev Proof. Let  D `;p basis relative to p. Let d WD dim.g=p/. We choose any Schwartz function 2 S.Rd / of L2 norm 1 and let F WD ˝ 2 S.Rd  Rd /. By Theorem 6.2.19, there exists

6.3 Intertwining Operator for Irreducible Representations

193

a Schwartz function f 2 S.G/, such that `;p .f / admits F as kernel function. Hence .f / is the orthogonal projection onto C and f 2 F . It suffices then to use Theorem 2.3.17. 

6.3 Intertwining Operator for Irreducible Representations Let G D exp g be a nilpotent Lie group and let ` 2 g . Choose two polarizations p; p0 at `. We know from Proposition 5.3.27 that there exists a unitary intertwining operator U W L2 .G=P; ` / ! L2 .G=P 0 ; ` /, which is unique up to a constant of modulus 1. We want to give now an explicit expression for this operator U . We remark first that U maps the C 1 -vectors for `;p onto the C 1 -vectors for `;p0 . This means by Corollary 6.2.14 that U maps S.G=P; ` / onto S.G=P 0 ; ` /. We have another linear operator T , which could be an intertwining operator. Namely Z T .g/ WD

P 0 =P \P 0

.gp 0 /` .p 0 /d pP 0 ; 2 S.G=P; ` /; g 2 G:

(6.3.1)

In order to show that the operators U and T coincide on Schwartz functions, we need some preparations. Lemma 6.3.1. Let H D exp h and H 0 D exp.h0 / be two closed connected subgroups of the nilpotent Lie group G. Then the product H 0 H is closed in G. Proof. If G is abelian, then H 0 H D exp.h0 C h/ is the exponential of the sum of the two subspaces h0 and h. We can also assume that both subgroups are proper. There then exists an ideal g0 of codimension 1, which contains the subalgebra h. If h0 is also contained in g0 , then we can apply the induction hypothesis and H 0 H is closed in G0 D exp.g0 / and hence in G. If h0 6 g0 , we take a vector X 2 h0 n g0 . Then g D RX ˚ g0 , h0 D RX ˚ h00 where h00 WD h0 \ g0 and H 0 D exp.RX /H00 with H00 D exp.h00 /. Take a sequence gn D exp.tn X /h0n hn 2 H 0 H; h0n 2 H00 ; hn 2 H ; tn 2 R; n 2 N, which converges to some u D exp.tX /b 2 G D exp.RX /G0 . Since the function G 3 g D exp.tX /a 7! t is continuous, it follows that limn!1 tn D t and so h0n hn converges to b, which by the induction hypothesis is contained in H00 H  G0 . Hence u 2 H 0 H .  Now let Z D fZ1 ; : : : ; Zn g be a Jordan–Hölder basis of g and as before, we let gj WD spanfZi I i  j g for j D 1; : : : ; n. For a subspace q of g let I q be the index set I q WD f1  j  nI gj C q D gj C1 C qg D f1  j  nI gj \ q ¤ gj C1 \ qg: 0

0

0

With this definition it is obvious that I p\p  I p  I p Cp . 0 0 Now let i 2 I p \ I p n I p \p . Then given Xi1 2 gi \ p n gi C1 , there exists 2 0 Xi 2 gi \ p n gi C1 , such that Zj WD Xi1 C Xi2 2 gj n gj C1 for some j > i . We

194

6 Kirillov Theory for Nilpotent Lie Groups

choose now Xi1 and Xi2 , such that j is maximal. This index j is then unique and we 0 put j.i / WD j . Then it follows that j 2 I p Cp and the maximality of j implies that 0 j 62 I p [ I p . Let us verify that the mapping 0

0

0

0

I p \ I p n I p\p ! I p Cp n .I p [ I p /I i 7! j D j.i /; which had been defined above, is a bijection. If for i < i 0 we have that j.i / D j.i 0 /, then there exists Xi1 2 p \gi ngi C1 ; Xi10 2 p \gi 0 ngi 0 C1 , Xi2 2 p0 \gi ngi C1 ; Xi20 2 p0 \gi 0 ngi 0 C1 , such that Zj WD Xi1 CXi2 2 gj ngj C1 and Zj0 WD Xi10 CXi20 2 gj ngj C1 . For a certain scalar  2 R , we then have that Zj  Zj0 2 gj C1 and so Xi1  Xi10 2 gi \ p n gi C1 ; Xi2  Xi20 2 gi 0 \ p0 n gi 0 C1 and Xi1  Xi10 C Xi2  Xi20 2 gj C1 , contradicting the maximality of j . 0 0 For the indices j 2 I pCp n .I p [ I p /, we have an index i  j , and elements Xi1 2 p \ gi n gi C1 ; Xi2 2 p0 \ gi n gi C1 , such that Xi1 C Xi2 DW Zj 2 gj n gj C1 . 0 Since j 62 I p \ I p , the index i must be strictly smaller than the index j . We choose 0 i maximal with these properties. Then i 2 I p \ I p . Furthermore j D j.i /, since letting j 0 WD j.i /, we have that j 0  j and in the case where j 0 > j , there exists Ui1 2 gi \ p n gi C1 ; Ui2 2 gi \ p0 n gi C1 , such that Zj 0 WD Ui1 C Ui2 2 gj 0 n gj 0 C1 . But we can then take a scalar  2 R , such that Xi1  Ui1 2 gi C1 . This implies that Xi2  Ui2 2 gi C1 too and so .Xi1  Ui1 / C .Xi2  Ui2 / D Zj  Zj 0 2 gj n gj C1 , contradicting the maximality of i . This shows that our mapping is also surjective. We now order the indices i.j / and we write J WD fi1 <    < ir g WD fi.j /; 0 0 j 2 I p \ I p n I p \p g. 0 0 We can write for j 2 I p \ I p n I p\p Zj D Bj C cj Zi.j / C Qi.j /C1; where cj 2 R is not zero, Zj 2 p0 , Bj 2 p and where Qi.j /C1 2 gi.j /C1. Then for tj 2 R, we have exp.tj Zj / D exp.tj cj Zi.j / C tj .Bj C Qi.j /C1// D exp.tj cj Zi.j //exp.Rj .tj // mod P;

(6.3.2)

for some element Rj .tj / 2 gi.j /C1, which varies polynomially in tj . Furthermore, we always can write for two indices j < j 0 exp.sZj 0 /exp.tZj / D exp.tZj /exp.sZj 0 / mod gj 0 C1 ; s; t 2 R; 0

0

(6.3.3)

Write I p =p\p D fj1 <    ; js g and define for 2 S.G=P; ` /; g 2 G; the function .g/ W Rs 7! C by

.g/.t1 ; : : : ; ts / WD .gexp.t1 Zj1 /    exp.ts Zjs //:

6.3 Intertwining Operator for Irreducible Representations

195

We observe that the function .g/ is contained in S.Rs / for every g 2 G and that its Schwartz norms are bounded uniformly in g. Indeed, let us take our Jordan– Hölder basis Z D fZ1 ; : : : ; Zn g of G, such that the vectors Zj are in p for j 2 I p 0 0 0 and such that Zj 2 p0 , for j 2 I p n I p . Let Y 0 D fZj I j 2 I p n I p\p g. Then for 0 g D EZ .x1 ; : : : ; xn / and p D EY 0 ..tj /j 2I p0 =p\p0 / it follows from relations (6.3.2) and (6.3.3) that gp 0 D

Y

exp.qk ..xi /niD1 ; .tj /j 2I p0 =p\p0 /Zk /;

k2I g=p 0

where for k 62 I pCp

qk ..xi /niD1 ; .tj /j 2I p0 =p\p0 / D xk C qk0 ...xi /niD1 ; i < k/; .tj /j 2I p0 =p\p0 ;j
where for k 2 I p n I p qk ..xi /niD1 ; .tj /j 2I p0 =p\p0 / D xk C tk C qk0 ...xi /niD1 ; i < k/; .tj /j 2I p0 =p\p0 ;j
P 0 =P \P 0

.gp 0 /` .p 0 /d pP0 ; 2 S.G=P; ` /; g 2 G;

is contained in S.G=P 0 ; ` / and T .`;p .f / / D `;p0 .f /.T /; f 2 S.G/: Proof. We remark first that the function P 0 3 p 0 7! .gp 0 /` .p 0 / is contained in S.P 0 =P 0 \P /, since first we observe that for p 0 2 P; p0 2 P \ P 0 ,

.gp 0 p0 /` .p 0 p0 / D .gp 0 /` .p 0 /` .p01 /` .p0 / D .gp 0 /` .p 0 /: R Hence it is constant modulo P \ P 0 and so the integral P 0 =P \P 0 .gp 0 /` .p 0 /d pP 0 converges for and all its derivatives D ; D 2 U.g/. Furthermore T . / is a Schwartz function in the variable p 0 2 P 0 =.P \P 0 / and uniformly bounded in g by the observation above. It follows from the definition of the operator T that T .gp 0 / D ` .p 0

1

/T .g/; g 2 G; p 0 2 P 0 ; 2 S.G=P; ` /:

196

6 Kirillov Theory for Nilpotent Lie Groups

It is clear from the formula for the operator T that T commutes with left translation and also with derivations on the left and so for any element D 2 U.g/ we have the relation D T . / D T .D

/; D 2 U.g/: Since T . / satisfies the covariance condition with respect to P 0 and ` , it follows that d `;p0 .D/T . / D T .d `;p .D/ /; D 2 U.g/: This shows that d `;p0 .D/.T / is a bounded function for every D 2 U.g/ and hence it must be in S.G=P 0 ; ` / by Theorem 6.2.13. The last relation follows immediately from Fubini’s theorem, since Z

Z T .`;p .f / /.g/ D

P 0 =.P 0 \P /

Z D

Z

f .u/ G

f .u/ .u1 gp 0 /` .p 0 /d ud pP 0 G

P 0 =.P v\P /

.u1 gp 0 /` .p 0 /d pP 0 d u

D `;p0 .f /.T /.g/; g 2 G; 2 S.G=P; ` /:



Theorem 6.3.3 (cf. [52]). Let G D exp g be a nilpotent Lie group and let ` 2 g . Choose two polarizations p and p0 at `. The operator T W S.G=P; ` / ! S.G=P 0 ; ` / defined in (6.3.1) is isometric and defines a unitary intertwining operator for the representations `;p and `;p0 . Proof. We know that the algebra S.G/ acts irreducibly and simply on the space of the C 1 -vectors for the representation `;p (see Corollary 6.2.20). Furthermore, if we take our official unitary intertwining operator U W L2 .G=P; ` / ! L2 .G=P 0 ; ` /; then this operator also maps the C 1 -vectors onto C 1 -vectors. Hence if we compose T with U 1 , then we get an intertwining operator S D U 1 ıT W S.G=P; ` / ! S.G=P; ` / for `;p . But such an operator must be a multiple c of the identity by Schur’s lemma for simple modules. Hence T D cU . 

6.4 Traces and Plancherel Theorem Theorem 6.4.1 (Kirillov’s Character Formula). Let .`;p ; L2 .G=P; ` // be an irreducible representation of a nilpotent Lie group G D exp g. There exists a G-invariant measure d O on the coadjoint orbit O of `, such that

6.4 Traces and Plancherel Theorem

197

Z Tr.`;p .f // D

2

f ı exp.q/d O .q/; f 2 S.G/: O

Proof. We know from Proposition 6.2.15 that for f 2 S.G/ the operator `;p .f / is trace class and that its kernel function is Schwartz class. Therefore Z Z Tr.`;p .f // D k`;p .f / .t; t/dt D k`;p .f / .g; g/d gP Z

Rd

G=P

Z

f .gpg 1 /` .p/dpd gP

D Z

G=P

P

Z

f ı exp.Ad.g/U /e i `.U / d Ud g: P

D G=P

p

Using the Fourier inversion formula, we see that 1 Tr.`;p .f // D .2/d

Z

Z

G=P

p?

1

f ıexp.Ad .g/.` C q//dqd g; P

where d D dim.G=P /. If we choose a Jordan–Hölder basis fX1 ; : : : ; Xn g for the action of p on g , such that X WD fX1 ; : : : ; Xd g is a Jordan–Hölder basis of g relative to p , then we see that the index set I ` for the action of P must be the set I ` D f1; : : : ; d g, since Ad  .P /` D ` C p? , and therefore by formula (6.1.2) Z

Z p?

'.` C q/dq D

'.` C Rd

D

zj Xj /d z

j D1

Hence Tr.`;p .f // D

d X

1 .2/d 1 .2/d

Z Z

Z G=P

Z

'.Ad .p/`/d p; P ' 2 S.g /:

D P =G.`/

2

f ı exp.Ad .g/Ad .p/`/d pd P gP P =G.`/

2

f ı exp.q/d O .q/: O



Remark 6.4.2. We can use Theorem 6.4.1 to have another proof of the fact that the Kirillov mapping for nilpotent Lie groups is injective. Suppose that we have two different orbits O and O 0 . Choose a nonnegative Schwartz function on g , such that is nonzero on O, but vanishes on O 0 . Let f 2 S.G/ be defined by R f ı exp D . Then by Theorem 6.4.1 Tr..O/.f O // D O .q/d O .q/ > 0 R and Tr..O O 0 /.f // D O 0 .q/d O 0 .q/ D 0. Hence .O/ O and .O O 0 / cannot be equivalent.

2

198

6 Kirillov Theory for Nilpotent Lie Groups

6.5 Parametrization of All Orbits Let G D exp g be a nilpotent Lie group. Let Z D fZn ; : : : ; Z1 g be a Jordan–Hölder basis of g and for j 2 f1; : : : ; ng let, as before, gj D spanfZ1 ; : : : ; Zj g; g0 WD f0g. We had defined for ` 2 g : IZ` D f1  i  nI g.`/ C gi ¤ g.`/ C gi 1 g: Definition 6.5.1. For a subset I of f1; : : : ; ng, let gI D f` 2 g I IZ` D I g: We denote by IZ D I the family of the subsets I of f1; : : : ; ng, for which gI ¤ ; and we define the polynomial function QI on g by QI .`/ D det.h`; ŒZi ; Zj ii;j 2I /; I 2 IZ : Theorem 6.5.2. Let G D exp g be a nilpotent Lie group. There exists an ordering on the family IZ such that for every I 2 IZ : 1. The algebraic subset FI WD f` 2 g I QJ .`/ D 0; QI .`/ ¤ 0; J < I g is G-invariant and FI D gI . S  2. g D I 2IZ gI . 3. The largest I D ; corresponds to the set of all characters of g. Proof. We define an ordering on I in the following way. Let I D fid >    > i1 g, J D fje >    > j1 g 2 I. We say that I < J if either jI j > jJ j or otherwise (i.e. jI j D d D jJ j/, if i1 D j1 ; : : : ; ir1 D jr1 but ir < jr . Let us show that gI D f` 2 g I QI .`/ ¤ 0; QJ .`/ D 0; J < I g DW FI : We use the following criterion for linear independence. Lemma 6.5.3. Let ! be a non-degenerate skew symmetric bilinear form on a finitedimensional vector space V . Then a family of vectors B D fB1 ; : : : ; Bd g is linearly independent, if and only if the determinant of the matrix   M D !.Bi ; Bj / 1i;j d is different from 0.

6.5 Parametrization of All Orbits

199

Let I D fid >    > i1 g and now take ` 2 gI . Let J D fje >    > j1 g < I . If jJ j > jI j, then the family of vectors Zj ; j 2 J; cannot be linearly independent modulo g.`/, since dim.g=g.`// D d D jI j. Hence QJ .`/ D 0 by the above lemma. If d D jJ j, then there exists an index jr D j 2 J which is not in I , but the indices js ; s < r; are in I \ J . Hence there exists a vector Tr D Zjr C

X

ak .`/Zk 2 g.`/:

jr 1k1 k2I

But then the vectors fZj I j 2 J g cannot be linearly independent modulo g.`/ and therefore QJ .`/ D 0: Of course, by definition QI .`/ ¤ 0. Conversely, let ` 2 g such that QI .`/ ¤ 0, but QJ .`/ D 0 for all J < I . The fact that QI .`/ ¤ 0 tells us that I  I ` , since otherwise there exists a smallest index i which is contained in I but not in I ` , which implies again that Zi 2 g.`/ modulo gi 1 so that QI .`/ D 0. Suppose that I ` ¤ I . Then jI j < jI ` j and so I ` < I , whence QI ` .`/ D 0, which is a contradiction. Hence I D I ` and so gI D FI .  Theorem 6.5.4. Let G D exp g be a nilpotent Lie group, Z D fZn ; : : : ; Z1 g be a Jordan–Hölder basis of g and let Z  be its dual basis. For every index set I I D fjd >    > j1 g 2 IZ there exist rational functions Rj;Z D RjI W Rd  g ! R, j D 1; : : : ; n, which have no singularities on gI , such that: 1. RjI .z; `/ is polynomial in z 2 Rd for j D 1; : : : ; n. 2. RjIi .z; `/ D zi ; z 2 Rd ; ` 2 gI , for every ji 2 I . 3. For j 2 6 I , ji C1 > j > ji , the function RjI .z; `/ does only depend on .z1 ; : : : ; zi ; `/. 4. For all ` 2 gI O.`/ WD Ad .G/` D

n nX

o RjI .z; `/Zj I z 2 Rd :

j D1

5. RjI .z; Ad .g/`/ D RjI .z; `/ for all z 2 Rd ; ` 2 gI . Proof. We know that the matrix MI .`/ D .mij /i;j 2I WD .h`; ŒZi ; Zj i/i;j 2I is invertible for every ` 2 gI , since its determinant QI .`/ ¤ 0. Let NI .`/ D .ni;j .`// be its inverse. Since the coefficients of MI .`/ are linear functions in `, it follows that the coefficients ni;j .`/ of its inverse NI .`/ are rational functions without any singularities in ` 2 gI and the denominator is a power of the polynomial QI . Let for any j 2 I Xj .`/ WD 

X i 2J

nj;i .`/Zi :

200

6 Kirillov Theory for Nilpotent Lie Groups

Then we have for k 2 I had .Xj .`//`; Zk i D h`; ŒXj .`/; Zk i X D nj;i .`/mi;k .`/ D ıj;k : i 2I

For the indices 1  k  j; k 62 I , we have that Zk 2 spanfZi I 1  i < k; i 2 I g modulo g.`/ and so had .Xj .`//`; Zk i D 0 too. This means that Xj .`/ 2 gj 1 .`/ n gj .`/ and ad .Xj .`//.`/ D Zj modulo g? j 1 in the notations of Remark 6.1.18. Hence o nY O.`/ D Ad .exp.tj Xj .`//.`/; t 2 Rd j 2I

D

n X

.li C qi .t1 ; : : : ; td ; `//Zi ;

(6.5.1)

i D1

where for j D ji 2 I , we have that qji .t1 ; : : : ; td ; `/ D ti C qj0 i ;` .t1 ; : : : ; ti 1 / for some polynomial function qj0 i ;` in t 2 Rd , which varies rationally in ` 2 gI . If j 62 I , then ji > j > ji C1 for some 1  i  d and therefore the polynomial function qj;` depends only on t1 ; : : : ; ti . Now let inductively z1 D lj1 C t1 ; : : : ; zi D lji C ti C qj0 i ;` .t1 ; : : : ; ti 1 /; i D 2; : : : d: It then follows that conversely ti D zi C ri .z1 ; : : : ; zi 1 ; `/, for some polynomial function ri , for i D 1; : : : ; d . Hence, replacing the ti ’s by the zi ’s in (6.5.1), we obtain a canonical description of the orbit of ` (once the basis Z is fixed). There exist polynomial functions RjI .z1 ; : : : ; zd ; `/; j D 1; : : : ; n; on Rd which vary rationally in ` 2 gI , such that RjIi .z1 ; : : : ; zd ; `/ D zi for all ji 2 I , such that for j 62 I and ji C1 > j > ji for some i , RjI .z1 ; : : : ; zd ; `/ D Rj .z1 ; : : : ; zi ; `/ and such that O.`/ D

8 n <X :

j D1

RjI .z; `/Zj I z 2 Rd

9 = ;

:

We must prove in ` of the functions RjI ; j 62 I . For any g 2 G P the invariance   and any q D j qj Zj in the orbit O D O.`/ D O.Ad .g/`/, we have for g 2 G; j 62 I , ji > j > ji C1 , RjI .qj1 ; : : : ; qji ; Ad .g/`/ D qj D RjI .qj1 ; : : : ; qji ; `/:

6.5 Parametrization of All Orbits

201

This shows that RjI .z1 ; : : : ; zd ; Ad .g/`/ D RjI .z1 ; : : : ; zd ; `/; z D .z1 ; : : : ; zd / 2 Rd :



Definition 6.5.5. Let G D exp g be a nilpotent Lie group and let Z be a Jordan– Hölder basis of g. We denote by Imin the smallest element in IZ and by ggen the Zariski open subset gImin of g . We say that the elements of ggen are in general position. We consider the rational functions j ; j 62 Imin , defined by j .`/ WD RjImin .0; `/; ` 2 ggen : These rational functions j are regular in the points QImin , since j D

Pj QIk

min

for some k 2 N by the proof of Theorem 6.5.4 and they are also G-invariant by the same theorem, hence j 2 R.g /G , the algebra of the G-invariant rational P functions on g , for every j 62 Imin . Furthermore, since for ` D j lj Zj we have by (6.1.3) that j .`/ D lj C j0 .l1 ; : : : ; lj 1 /, we see that the functions j ; j 62 Imin ; are algebraically independent. Let dmax be the maximum of the dimensions of all coadjoint orbits. Then dmax D jImin j. We shall now parametrize the orbits in general position in the following way. For every orbit O D G  ` in ggen , there exists exactly one element `O , which is P contained in v introduced below. Indeed, if `O D nj D1 RjImin .0; `/Zj , `O is the only element of O, for which the components lj ; j 2 Imin ; are all 0. Definition 6.5.6. Let v D vZ WD

8 < :

`D

n X j D1

9 =

lj Zj 2 g I lj D 0; for all j 2 Imin ; ;

and we let vgen D ggen \ v D f` 2 v I QImin .`/ ¤ 0g: Then v is a .dim g  dmax /-dimensional subspace of g and vgen is Zariski open in v . Definition 6.5.7. Let G D exp g be a nilpotent Lie group. Let f D be a rational function on g . Let us write gf for the function gf .`/ D f .Ad .g 1 /`/ D

P Q

W g ! C

P .Ad .g 1 /`/ ; ` 2 g : Q.Ad .g 1 /`/

We say that f is G-invariant if gf D f for all g 2 G. This is equivalent to saying that

202

6 Kirillov Theory for Nilpotent Lie Groups

X f WD

d .X P /Q  P .X Q/ exp.tX /f jt D0 D D 0 for all X 2 g: dt Q2

Let R.g /G be the algebra of all G-invariant rational functions on g . Proposition 6.5.8. Let G D exp g be a nilpotent Lie group. 1. For any f 2 R.g /G there exist two G-invariant polynomial functions P; Q on P g such that f D Q . 2. Any polynomial P 2 R.g /G is a polynomial expression in the functions j ; j 62 Imin . Proof. Let g D gn      g1  g0 D f0g be a composition series of g. Let us show by induction on j that there exist P two polynomial functions Pj ; Qj , which are gj invariant, such that f D Qjj . Suppose that we have found gj -invariant polynomial functions Pj and Qj , such that P f D Qjj , then we are looking for such gj C1 -invariant ones. Choose X 2 gj C1 n gj . If a polynomial function R is gj -invariant, then the function X R is also gj -invariant since U  .X  R/ D X  .U  R/ C ŒU; X   R D 0 C 0 D 0: Now g acts on g by derivations. Hence there exists m 2 N , such that X m Qj D 0, X m1 Qj ¤ 0. If m > 1, then the equation X f D 0 tells us that .X Pj / X P P Qj  Pj .X Qj / D 0, i.e. X Qjj D Qjj D f . Continuing in this way we see that

X m1 Pj X m1 Qj

D f . Let Pj C1 WD X m1 Pj ; Qj C1 WD X m1 Qj . Then both are P

gj C1 -invariant and f D QjjC1 . C1 Suppose now that P is a G-invariant polynomial functionPon g . Then for ` 2 ggen , there exists an element g 2 G, such that Ad .g/.`/ D j RjImin .0; `/Zj . Hence  X  j .`/Zj : P .`/ D P .Ad .g/`/ D P j 62Imin

Hence P is a polynomial in the j ; j 62 Imin .



Remark 6.5.9. Let us recall that we have two precise measures on a coadjoint orbit O, the first one given by the trace formula (see Theorem 6.4.1) Z Tr.`;p .f // D

1

f ıexp.q/d O .q/; f 2 S.G/; ` 2 O; O

(6.5.2)

6.5 Parametrization of All Orbits

203

the second being given by the canonical description of the coadjoint orbit O using a Jordan–Hölder basis Z of g. Let I D IZ the index set of O. Then we have the second invariant measure Z Z X  '.q/dO .q/ WD ' RjI .t; O/ dt; ' 2 Cc .O/: Rd

O

j

Theorem 6.5.10. We have that d O D denotes the dimension of O.

1 dO , .2/d=2 QI .O/1=2

where O 2 gI and d

Proof. We recall the computations of Theorem 6.4.1. Let ` 2 O and let p be a polarization at `. Let E D fE1 ; : : : ; Ek g be a Malcev basis of g.`/, C D fC1 ; : : : ; Cm g be a Malcev basis of p relative to g.`/ and let D D fD1 ; : : : ; Dm g be a Malcev basis of g relative to p. We denote by Y WD fD1 ; : : : ; Dm ; C1 ; : : : ; Cm ; E1 ; : : : ; Ek g DW fY1 ; : : : ; Yn g the corresponding Malcev basis of g and we use this basis to describe the Haar measure of G. We have seen in the proof of Theorem 6.4.1 that 1 Tr.`;p .f // D .2/m

Z

Z G=P

p?

2

f ı exp.Ad .g/.` C q//dqd g; P f 2 S.G/: (6.5.3)

This formula can be given the form Tr.`;p .f // D

˛ .2/m

Z

Z G=P

2

f ı exp.Ad .g/Ad .p/  `/d pd P g; P f 2 S.G/: P =G.`/

Here we use the invariant measure on P =G.`/ given by the basis C. We obtain the constant ˛, which relates the invariant measures on P =G.`/ and on p? . In order to compute this constant, it suffices to write Ad .EC .t//` D ` C

m X

Qj .t/Dj ; t 2 Rm ;

j D1

P  and to make in the integral (6.5.3) the change of variables p? 3 q D m j D1 Qj .t/Dj . The Jacobian of this change of variables is this constant ˛. We compute the Jacobian for t D 0. This gives us ˇ ˇ ˇ @Qj .t/ ˇˇ ˛ D ˇˇ det. /ˇ @t i

: t D0

204

6 Kirillov Theory for Nilpotent Lie Groups

Since  ˇ ˇ @ Ad .exp.ti Ci /`/.Dj // ˇ @Qj .t/ ˇ ˇ D ˇ ˇ ˇ @ti t D0 @ti

D h`; ŒDj ; Ci i; t D0

we see that ˇ   ˇ ˇ ˇ ˛ D ˇ det h`; ŒDj ; Ci i ˇ:

(6.5.4)

i;j

Hence for f 2 S.G/ Tr.`;p .f // D Z

Z

 Rm

Rm

˛ .2/m

f2 ı exp

m Y

m Y



Ad .exp.ti Di //

i D1

! 

Ad .exp.si Ci //`

i D1

m Y i D1

dsi

m Y

dti :

i D1

We now use a new change of variables to pass from the basis Y to the basis Z. We write n X

Qj .t1 ; : : : ; tm ; s1 ; : : : ; sm /Zj

j D1

D

m Y



Ad .exp.ti Di //

i D1

m Y

Ad .exp.si Ci //`

i D1

and we have the change of variables zi D Qji .t1 ; : : : ; tm ; s1 ; : : : ; sm /; j1 ; : : : ; j2m 2 I ` : We obtain in this way a constant ˇ such that for ' 2 Cc .g / 0

Z

'@ R2m

n X

1 RjI .z; `/Zj A d z D ˇ

j D1

Z

'.Ad .g/`/d g: P

(6.5.5)

G=G.`/

Again the change of variables above gives us the value of ˇ as ˇ ˇ ˇ D ˇ det

@Qji .t;s/ @tp @Qji .t;s/ @sq

!

ˇ ˇ ˇ

t DsD0

:

Since @.Ad .exp.tp Dp /`/.Zji // ˇˇ @Qji .t; s/ ˇˇ D D h`; ŒDp ; Zji i; ˇ ˇ t DsD0 t DsD0 @tp @tp

6.5 Parametrization of All Orbits

205

and @.Ad .exp.sq Cq /`/.Zji // ˇˇ @Qji .t; s/ ˇˇ D D h`; ŒCq ; Zji i; ˇ ˇ t DsD0 t DsD0 @sq @sq we see that  ˇ ˇ h`; ŒDp ; Zji i ˇ ˇ ˇ D ˇ det ˇ: h`; ŒCq ; Zji i We develop now the vectors Dp ; Cq in the basis Z modulo g.`/, i.e. Dp D

2m X

ai;p Zji ; Cq D

i D1

2m X

ai;q Zji mod g.`/; p D 1; : : : ; m; q D 1; : : : ; m:

i D1

This change of basis gives us the 2m by 2m matrix  AD

ai;p ai;q

 :

We then see that ˇ D j det AjjQI .`/j:

(6.5.6)

Furthermore, since h`; ŒCi ; Ci 0 i D 0 for all i; i 0 , we observe that   ˇ ˇ ˇ ˇ h`; ŒDj ; Ci i h`; ŒDj ; Dj 0 i ˇ ˇ ˇ ˇ1=2 ˛ D ˇ det.h`; ŒDj ; Ci ii;j /ˇ D ˇ det ˇ h`; ŒCi ; Ci 0 i h`; ŒCi ; Dj i D j det AjjQI .`/j1=2 :

(6.5.7)

Therefore, by Eqs. (6.5.4)–(6.5.7) j det AjjQI .`/j1=2 1 ˛ D D : ˇ j det AjjQI .`/j jQI .`/j1=2



Definition 6.5.11. We define the function ı D ıZ on g by ı.`/ WD .2/dmax =2 jQImin .`/j1=2 ; ` 2 g : The function ı is of course G-invariant, since so is QImin , it is homogeneous of degree dmax , since QImin is homogeneous of degree dmax and ı is also a polynomial. 2 In order to see that, let A2m .R/ denote the space of all skew symmetric real matrices of order 2m.

206

6 Kirillov Theory for Nilpotent Lie Groups

Lemma 6.5.12. Let m 2 N. There exists a polynomial mapping Q W A2m ! R in the variables .ai;j /i;j (called the Pfaffian of A) such that for every element A D .ai;j / 2 A2m .R/ we have det.ai;j / D .Q.ai;j //2 : Proof. We define for A 2 A2m .R/ the skew symmetric bilinear form wA .x; y/ WD

X

ai;j xi yj :

i;j

Let X D fX1 ; : : : ; X2m g be the canonical basis of R2m . Starting with X1 .A/ WD

1 X1 ; X2 .A/ D X2 a1;2

and considering the subspace V1 .A/ WD fU 2 R2m ; wA .U; Xj / D 0; j D 1; 2g and the basis X1 .A/ D fX3 C wA .X3 ; X1 .A//X2 .A/  wA .X3 ; X2 .A//X1 .A/; : : : ; X2m C wA .X2m ; X1 .A//X2 .A/  wA .X2m ; X2 .A//X1 .A/g we construct inductively a symplectic basis X .A/ D fX1 .A/; : : : ; X2m .A/g for the skew symmetric bilinear form wA for all A in a Zariski open subset of A2m .R/. The vectors Xj .A/ of X .A/ vary rationally in the coefficients of A. Let U.A/ be the matrix which gives the change of the basis X to the basis X .A/. Then  0m Im Im 0m   P1 .A/2 P1 .A/2 0m Im D D det 2 Im 0m P2 .A/ P2 .A/2

  det A D det U.A/2 det



for two polynomial functions P1 ; P2 in the coefficients ai;j of A. Hence P2 .A/2 det.A/ D P1 .A/2 and therefore P2 divides P1 . This shows that det.A/ D Q.A/2 for some polynomial function Q.A/ in the variables ai;j .  We define now a measure d on the subspace v in the following way. Since v is a real finite-dimensional vector space we can define the measure d by Z v

Here dq D

Q j 62Imin

'.q/ı.q/dq; ' 2 Cc .g /:

d lj denotes the Lebesgue measure on v .

6.5 Parametrization of All Orbits

207

Theorem 6.5.13. Let G D exp g be a nilpotent Lie group. Let nG D dimG  dmax . For every Schwartz function f 2 S.G/, we have that f .g/ D

1 .2/nG

Z v

Tr.v .f / ı v .g 1 //d.v/; g 2 G:

Proof. By the ordinary Fourier inversion formula we have for f 2 S.G/, that 1 f .e/ D .2/n

Z g

2

f ı exp.`/d `:

(6.5.8)

Choose again a Jordan–Hölder basis Z D fZn ; : : : ; Z1 g of g, which gives us the Haar measure of the group G and the functions Rj .z; `/ WD RjImin .z; `/; ` 2 ggen , z 2 Rdmax . Let us recall that Rj .z; `/ D lj C Rj0 .l1 ; : : : ; lj 1 /; ` D

n X

lk Zk ; j 62 Imin ;

kD1

where Rj0 is a rational function in the variables l1 ; : : : ; lj 1 which is regular on ggen (see Theorem 6.1.15). Hence we can transform Eq. (6.5.8) in the following way: 1 f .e/ D .2/n D D

1 .2/n 1 .2/n

Z g

2

 X f ı exp lj Zj d ` j

Z 

Rdmax v

Z

Z v

2

 X f ı exp Rj .z; v/Zj d zd v

2

j

f ı exp.q/dO .q/d v O.v/

Z Z .2/dmax =2 1=2 Q .v/ f ı exp.q/d O.v/ .q/d v .by Theorem 6.5.10/ Imin .2/n v O.v/ Z 1 Tr.` .f //ı.v/d v .by formula (6.5.2)/: D .2/nG v D

2

Now for g 2 G we get f .g/ D .g 1 /f .e/ Z 1 Tr.` ..g 1 /f //ı.v/d v D .2/nG v Z 1 D Tr.v .g 1 / ı v .f //d.v/: .2/nG v



208

6 Kirillov Theory for Nilpotent Lie Groups

Corollary 6.5.14 (Plancherel Formula). Let f 2 S.G/. Then kf k22 D

1 .2/nG

Z v

kv .f //k2HS d.v/:

Proof. We have Z kf k22 D

f .t/f .t/dt D f f  .e/ G

1 D .2/nG D

1 .2/nG

Z Z

v

v

Tr.v .f / ı v .f / /d.v/ kv .f /k2HS d.v/:



Chapter 7

Holomorphically Induced Representations .f; h; G ) for Exponential Solvable Lie Groups

7.1 First Trial Let us consider holomorphically induced representations for an exponential solvable Lie group G D exp g. Since the stabilizer G.f / in G of any f 2 g is connected (Theorem 5.3.2), there uniquely exists the homomorphism f W G.f / ! T such that df D if jg.f / . So, we write the holomorphically induced representation .f; f ; h; G/ constructed from a polarization h 2 P .f; G/ satisfying the Pukanszky condition and its representation space H.f; f ; h; G/ simply as .f; h; G/ and H.f; h; G/. We show the following theorem. Theorem 7.1.1 ([27]). Let G D exp g be an exponential solvable Lie group with Lie algebra g and f 2 g . Suppose that h 2 P C .f; G/ satisfies the Pukanszky condition. Provided that H.f; h; G/ ¤ f0g, .f; h; G/ is irreducible and equivalent to the irreducible unitary representation of G corresponding to the coadjoint orbit Gf . Proof. When dim G D 1, the theorem is clear. Now we show the theorem by induction on dim G. Suppose that dim G D n and that the theorem holds for exponential solvable Lie groups whose dimension is smaller than n. Case 1.

There is an ideal a ¤ f0g of g such that f ja D 0.

Let A D exp a; GQ D G=A D exp.Qg/; gQ D g=a and p W G ! GQ the canonical projection. We denote by dp the differential of p and extend it linearly on gC . Then we consider the exact sequence p

1 ! A ! G ! GQ ! 1 of exponential solvable Lie groups and take fQ 2 gQ  such that fQıdp D f .

© Springer Japan 2015 H. Fujiwara, J. Ludwig, Harmonic Analysis on Exponential Solvable Lie Groups, Springer Monographs in Mathematics, DOI 10.1007/978-4-431-55288-8__7

209

210

7 Holomorphically Induced Representations...

Q As in Since a  g.f /  h, hQ D dp.h/ is an element of P C .fQ; G/. Proposition 5.1.17, gQ  is naturally identified with a?  g and hQ satisfies the Pukanszky condition. By Proposition 5.1.17 Q G/ıp Q .fQ; h; ' .f; h; G/:

(7.1.1)

Q G/ Q ¤ f0g. Since dim GQ < dim G, there It follows from the assumption that H.fQ; h; exists hQ 0 2 I.fQ; gQ / so that Q G/ Q ' . Q .fQ; h; O fQ; hQ 0 ; G/:

(7.1.2)

Q . O fQ; hQ 0 ; G/ıp ' .f; O h0 ; G/

(7.1.3)

Putting h0 D .dp/1 .hQ 0 /,

and h0 2 I.f; g/. Now Eqs. (7.1.1)–(7.1.3) give the desired result. Case 2.

There is no ideal a ¤ f0g such that f ja D 0.

N \ g ¤ g: (i) e D .h C h/ We choose and fix a complementary subspace m of e in g. If we identify e with m? , e  g . Letting p W g ! e be the restriction mapping, pje is the identity map. For ` 2 g , we put `0 D p.`/ D `je . As is known immediately, h 2 P C .f 0 ; E/. Let us verify that h satisfies the Pukanszky condition as polarization of e. First, since e is a Lie subalgebra of g, p.Ef / D .Ef /0 D Ef 0  e :

(7.1.4)

Next, let us see p 1 .Ef 0 / D Ef . Evidently a` 2 e? for all a 2 E; ` 2 e? . Let us decompose af .a 2 E/ as af D .af /0 C fO; fO 2 e? : Thus, for any ` 2 e? ,   a f  a1 .fO  `/ D .af /0 C `; where a1 .fO  `/ 2 e? . Since h satisfies the Pukanszky condition as polarization of g at f 2 g , there is by Proposition 5.1.12 b 2 D  E such that bf D f  a1 .f  `/. Hence .ab/f D .af /0 C `:

(7.1.5)

7.1 First Trial

211

Since a 2 E; ` 2 e? are arbitrary, the relations (7.1.4), (7.1.5) indicate p 1 .Ef 0 / D Ef:

(7.1.6)

Therefore Ef 0 D Ef \ e is a closed set of e and h satisfies the Pukanszky condition as polarization of e at f 0 . Meanwhile we have 0 .f; h; G/ D indG E .f ; h; E/:

Hence, provided H.f; h; G/ ¤ f0g, H.f 0 ; h; E/ ¤ f0g: As dim E < dim G, there is by the induction hypothesis h0 2 I.f 0 ; e/ so that O 0 ; h0 ; E/: .f 0 ; h; E/ ' .f

(7.1.7)

Theorem 5.3.8 means h0 2 M.f 0 ; e/. As h 2 P .f 0 ; E/, dimR .h0 / D dimC h D

1 .dimR g C dimR .g.f ///: 2

Consequently h0 2 M.f; g/. Let us see that h0 satisfies the Pukanszky condition. To begin with, 





f C h0 ?;g D f 0 C h0 ?;e C e?;g :

(7.1.8)

As h0 2 I.f 0 ; e/, Theorem 5.4.1 assures that h0 satisfies the Pukanszky condition at the stage of E, i.e. 

f 0 C h0 ?;e  Ef 0 :

(7.1.9)

From properties (7.1.6), (7.1.8), (7.1.9), 

f C h0 ?;g  Gf; That is, h0 2 M.f; g/ satisfies the Pukanszky condition and h0 2 I.f; g/. Consequently indG O 0 ; h0 ; E/ D .f; O h0 ; G/ is irreducible. Hence by (7.1.7), E .f 0 G .f; h; G/ D indG O 0 ; h0 ; E/ D OG .f /: E .f ; h; E/ ' indE .f

(ii) e D g (i.e. h is totally complex). In this case, Theorem 5.1.23 tells us that .f; h; G/ is irreducible. Then we assume .f; h; G/ D OG .f0 / and show f0 2 Gf .

212

7 Holomorphically Induced Representations...

Lemma 7.1.2. When G is an exponential solvable Lie group, d is an ideal of e. Proof. Since d and e are mutually orthogonal regarding Bf , 0 D f .Œx; Œy; z/ D f .ŒŒx; y; z/ C f .Œy; Œx; z/ .x 2 d; y; z 2 e/: Therefore, Bf .ad.x/y; z/ D Bf .y; ad.x/z/:

(7.1.10)

We designate by A.x/ the linear transformation on e=d induced by ad.x/. As x 2 N the linear transformation ad.x/ extended on eC keeps h; hN stable. Hence A.x/ h \ h, N C stable and consequently is commutative with J . Then, as y; z 2 keeps h=dC ; h=d e=d, Eq. (7.1.10) gives BO f .A.x/y; J z/ D BO f .y; A.x/J z/ D BO f .y; JA.x/z/; in other words, Sf .A.x/y; z/ D Sf .y; A.x/z/: Namely, A.x/ is anti-symmetric with respect to the positive definite symmetric bilinear form Sf and its eigenvalues are purely imaginary, whereas, as d-module through A, e=d is of exponential type. Finally, A.x/ D 0 and Œd; e  d.  Now let us return to the sequel of the proof of the theorem. Corollary 7.1.3. d is an ideal of g and dim d  1. Besides, let z be the centre of g. Then, d D g.f / D z. Proof. As e D g, Lemma 7.1.2 means that d is an ideal of g. We put b D d \ kerf . Since Œg; d  d and f .Œg; d/ D f .Œe; d/ D f0g, Œg; d  b. Hence b is an ideal of g and f .b/ D f0g. From the assumption of our case, b D f0g. In the sequel, dim d  1 and d  z. On the other hand, z  g.f /  d is obvious and finally d D g.f / D z.  To continue the proof of the theorem, we need the following fact. Theorem 7.1.4 (Invariance of Domain [72]). Let U1 ; U2 be the subsets of Rm homeomorphic to each other. Provided that U1 is an open set of Rm , so is U2 . Let us continue the proof of the theorem. If dim d D 0, g.f / D f0g and dim.Gf / D dim.g / D n. Taking Lemma 5.3.4 into account, the orbit Gf is homeomorphic to Rn and hence is an open set of g by Theorem 7.1.4, whereas, because h satisfies the Pukanszky condition, Gf D Ef is a closed set of g . Thus Gf D g but this is impossible. Hence dim d D 1.

7.1 First Trial

213

From the preceding, d D g.f / D z D RZ; f .Z/ ¤ 0 and dim.Gf / D dim g  dim.g.f // D n  1:

(7.1.11)

Put V D f` 2 g I `.Z/ D f .Z/g. Since Z 2 z, .af /.Z/ D f .Z/ for any a 2 G, i.e. Gf  V . By Lemma 5.3.4 and Eq. (7.1.11), the orbit Gf is homeomorphic to Rn1 and by Theorem 7.1.4 Gf is an open set of V . However, since h satisfies the Pukanszky condition, Gf is a closed set of V . Finally, Gf D V:

(7.1.12)

Now if we take h0 2 I.f0 ; g/, z  h0 . Let O 0 ; h0 ; G/ ! H.f; h; G/ R W H.f be the intertwining operator between LO D .f O 0 ; h0 ; G/ D OG .f0 / and L D O 0 ; h0 ; G/ and g 2 G, .f; h; G/. Thus, for  2 H.f 

   O RıL.g/ ./ D L.g/ıR ./:

Fix t0 2 R and put g0 D exp.t0 Z/. Since g0 is an central element of G,   O 0 / .g/ D .exp.t0 Z/g/ D .gexp.t0 Z// D e i t0 f0 .Z/ .g/ L.g for g 2 G. Hence,   O 0 / ./ D e i t0 f0 .Z/ R./: RıL.g On the other hand, ..L.g0 /ıR/.// .g/ D .R/.exp.t0 Z/g/ D .R/.gexp.t0 Z// D e i t0 f .Z/ .R/.g/: That is, .L.g0 /ıR/./ D e i t0 f .Z/ R./: Taking all into consideration, e i t0 f0 .Z/ D e i t0 f .Z/ for any t0 2 R. From this, f0 .Z/ D f .Z/:

(7.1.13)

Finally, (7.1.12) and (7.1.13) conclude f0 2 Gf .  Now, following [28], we proceed to more detailed study of holomorphically induced representations for exponential solvable Lie groups.

214

7 Holomorphically Induced Representations...

7.2 Exponential j-Algebras Definition 7.2.1. The triplet .g; j; ˇ/ consisting of an exponential solvable Lie algebra g, a linear operator j and an alternating bilinear form ˇ on g is called an exponential Kähler algebra if it has the following properties: (1) (2) (3) (4) (5)

j 2 D 1, ŒjX; jY  D j ŒjX; Y  C j ŒX; jY  C ŒX; Y , ˇ.jX; jY / D ˇ.X; Y /, ˇ.jX; X / > 0 for X ¤ 0, ˇ.ŒX; Y ; Z/ C ˇ.ŒY; Z; X / C ˇ.ŒZ; X ; Y / D 0. If, in addition to these properties, there is a linear form ! 2 g such that ˇ.X; Y / D !.ŒX; Y / for any X; Y 2 g

the triplet .g; j; !/ is called an exponential j -algebra. By abuse of language, we shall often say that g is an exponential j -algebra or Kähler algebra. Let J be a hermitian vector space of finite dimension, j the complex structure in J and ˇ the imaginary part of the hermitian scalar product on J . Definition 7.2.2. A representation  of an exponential Kähler algebra g by real linear transformations of a hermitian vector space J is called symplectic if it satisfies the following two conditions: Œ.jX /  j.X /; j  D 0;

(7.2.1)

ˇ..X /x; y/ C ˇ.x; .X /y/ D 0

(7.2.2)

for all X 2 g; x; y 2 J . For the next lemma, the proof of Lemma 3 in Part II of Gindikin, PjatetskiiShapiro and Vinberg [40] remains valid. Lemma 7.2.3. Every symplectic representation  of exponential type of a commutative Kähler algebra g is trivial, i.e. .X / D 0 for X 2 g. Lemma 7.2.4. Let g be a two-dimensional exponential Kähler algebra with basis fjs; sg satisfying the commutation law Œjs; s D s. If  is a symplectic representation of exponential type of g on a hermitian space J , then the operator L D .js/ is semi-simple and there is a direct sum decomposition J D J ˚ J0 ˚ JC which has the following properties: (1) The subspaces J ; J0 and JC are stable under L.

7.2 Exponential j-Algebras

215

(2) The real parts of the eigenvalues of L on J ; J0 and JC are equal to  12 ; 0 and 1 respectively. 2 (3) j.J / D JC ; j.J0 / D ( J0 . j on J ; (4) LjJ0 D 0 and .s/ D 0 on J0 C JC : For later use we prove this by the same process employed in Proposition 5.14 of Rossi [67] for a normal symplectic representation. Proof of Lemma 7.2.4. We proceed by induction on dim J . Let M D .s/. Since ŒL; M  D LM  ML D M , we conclude that M is singular, and L leaves kerM invariant. The condition (7.2.1) implies that M C jMj C jL  Lj D 0:

(7.2.3)

Suppose first that dim J D 2. Case 1. Assume that L has a real eigenvalue. Let e be a common eigenvector for L and M such that Le D ˛e; M e D 0; ˛ 2 R. Now relative to the basis fe; jeg, we can write       ˛ a 0 b 0 1 LD ; ; M D j D 0 a0 0 b0 1 0 but ˇ.Le; je/ C ˇ.e; Lje/ D 0, so a0 D ˛ and similarly b 0 D 0. By (7.2.3),  a 2˛ C b D 0: 2˛ C b a



Thus a D 0; b D 2˛ and ŒL; M  D M implies that 2˛b D b. Therefore if b D 0, then ˛ D 0 and we have J D J0 . If b ¤ 0, then ˛ D 12 , so b D 1, and we have J D Rje ˚ Re D J ˚ JC . Case 2. Assume that L has no real eigenvalue. In this case, M D 0 and Lj D jL. Let e be a nonzero element. Then relative to the basis fe; jeg,  LD

1 ˛ ˛ 1

 and j D

  0 1 ; 1 0

where ˛;  2 R; ˛ ¤ 0. Since ˇ.Le; je/ C ˇ.e; Lje/ D 0, we have 2ˇ.e; je/ D 0 which is a contradiction. Next we assume that Lemma 7.2.4 is valid for symplectic representations of exponential type of g on hermitian spaces of lower dimension than J .

216

7 Holomorphically Induced Representations...

Case 1. Suppose that LjkerM has a real eigenvalue. Choose an eigenvector e of LjkerM such that Le D ˛e .˛ 2 R/ and ˇ.je; e/ D 1. Let W be the annihilator of Rje ˚ Re with respect to ˇ. Then W is invariant under j and, since ˇ.Lx; e/ D ˇ.x; Le/ D 0 D ˇ.x; M e/ D ˇ.M x; e/ for x 2 W , W ˚ Re is invariant under both L and M . For x 2 W , write Lx D LW x C l.x/e and M x D MW x C m.x/e with LW x; MW x 2 W and l.x/; m.x/ 2 R. Then relation (7.2.3) becomes MW C jMW j C jLW  LW j D 0; and ŒL; M  D M implies ŒLW ; MW  D MW . Further, since LjW ˚Re D

    LW 0 MW 0 and M jW ˚Re D ; l ˛ m 0

both LW and MW have no nonzero purely imaginary eigenvalue. So the representation W of g on W defined by W .js/ D LW and W .s/ D MW is of exponential type. Furthermore, W is clearly symplectic. Thus the induction hypothesis gives the splitting W D W ˚ W0 ˚ WC : Therefore, if one can show that L and M leave Rje ˚ Re invariant, we shall have one of the two possibilities: J D W ; J0 D W0 ˚ Rje ˚ Re; JC D WC or J D W ˚ Rje; J0 D W0 ; JC D WC ˚ Re: Since ˇ.e; Lje/ D ˇ.Le; je/ D ˛, we can write Lje D w  ˛je C ae and Mje D w0 C a0 e with w; w0 2 W; a; a0 2 R. Now 0 D .M C jMj C jL  Lj /e D j w0 C a0 je C ˛je C ˛je  w  ae;

7.2 Exponential j-Algebras

217

so a D 0; a0 D 2˛ and w D j w0 . Since ˇ.Lx; je/ C ˇ.x; Lje/ D 0 for x 2 W , l.x/ D ˇ.x; w/ and similarly m.x/ D ˇ.x; w0 /. From the equality ŒL; M je D Mje, we deduce that 2˛.2˛  1/ D ˇ.w0 ; w/  ˇ.w; w0 / D 2ˇ.w0 ; j w0 /  0; i.e. 0  ˛  1=2 and .1  ˛/w0 D LW w0  MW w. But we can write w0 D w0 C w00 C w0C , so j w0 D j w0 C j w00 C j w0C , and thus we get .1  ˛/w0 D LW w0 ; .1  ˛/w00 D 0; ˛w0C D LW w0C : These equalities together with the inequality 0  ˛  1=2 imply that w0 D w00 D w0C D 0, i.e. w D w0 D 0. Case 2. Suppose that LjkerM has no real eigenvalue. Let e1 ; e2 be two linearly independent vectors in kerM such that the subspace V D Re1 ˚ Re2 is L-invariant, ˇ.je1 ; e1 / D 1 and that 

1 ˛ LjV D  ˛ 1



with ; ˛ 2 R and ˛ ¤ 0, relative to the basis fe1 ; e2 g. Since ˇ.Le1 ; e2 / C ˇ.e1 ; Le2 / D 0; ˇ.e1 ; e2 / D ˇ.je1 ; je2 / D 0. We set ˇ.je2 ; e2 / D k.> 0/ and ˇ.e1 ; je2 / D ˇ.e2 ; je1 / D , then 0  2 < k:

(7.2.4)

Since ˇ.e1 ; e2 / D 0, je1 62 V . Let W be the annihilator of Rje1 ˚Rje2 ˚Re1 ˚Re2 with respect to ˇ. Then W is invariant under j and, since ˇ.Lx; e1 / D ˇ.Lx; e2 / D ˇ.M x; e1 / D ˇ.M x; e2 / D 0 for x 2 W , W ˚ Re1 ˚ Re2 is invariant under both L and M . For x 2 W , we write Lx D LW x C l1 .x/e1 C l2 .x/e2 and M x D MW x C m1 .x/e1 C m2 .x/e2 with LW x; MW x 2 W and li .x/; mi .x/ 2 R.i D 1; 2/. By (7.2.3), we have MW C jMW j C jLW  LW j D 0; and ŒL; M  D M implies ŒLW ; MW  D MW . Further, since

LjW ˚Re1 ˚Re2

0 1 0 LW   D@ 1 ˛ A

 ˛ 1

218

7 Holomorphically Induced Representations...

and M jW ˚Re1 ˚Re2

  MW 0 D ;

0

we can define a symplectic representation W of exponential type of g just as in Case 1 above. Thus the induction hypothesis gives the splitting W D W ˚W0 ˚WC and LW is semi-simple. Write Lje1 D w1 C aje1 C bje2 C pe1 C qe2 ; Lje2 D w2 C cje1 C dje2 C re1 C se2 and Mje1 D w01 C e1 C e2 ; Mje2 D w02 C ıe1 C e2 with wi ; w0i 2 W .i D 1; 2/ and a; b; c; d; p; q; r; s; ; ; ı;  2 R. Now, ˇ.e1 ; Lje1 / D ˇ.Le1 ; je1 / implies a C b D .1 C ˛/. Similarly, we have a  kb D .˛  /; c C d D . C ˛k/; c  kd D .k  ˛/: Thus,    1Ck ˛.1 C k/  1 ; b D ˛ 1 C 2 ; aD 2  k  k     k.1 C k/ ˛.1 C k/ c D ˛ 1 C 2 ; d D  1 C :  k 2  k 

From the relation 0 D .M C jMj C jL  Lj /e1 , we deduce that j w01 D w1 ; C  D a;  ˛ D b; p D q D 0 and j w02 D w2 ; ı C ˛ D c;  C  D d; r D s D 0: Therefore,    1Ck ˛.1 C k/  2 ; D ˛ 2 C 2 ;

D 2  k  k     k.1 C k/ ˛.1 C k/ ı D ˛ 2 C 2 ;  D  2 C :  k 2  k 

7.2 Exponential j-Algebras

219

Since ˇ.Lje1 ; x/ C ˇ.je1 ; Lx/ D 0 .x 2 W /, we have ˇ.x; w1 /  l1 .x/ C l2 .x/ D 0: Similarly, ˇ.x; w2 / C l1 .x/  kl2 .x/ D 0; ˇ.x; w01 /  m1 .x/ C m2 .x/ D 0; ˇ.x; w02 / C m1 .x/  km2 .x/ D 0: From the equation ŒL; M je1 D Mje1 , LW w01  MW w1 D .1 C ˛/w01 C bw02 ;

(7.2.5)

l1 .w01 /  m1 .w1 / D .1 C a  / C bı  ˛; l2 .w01 /  m2 .w1 / D .1 C a  / C b C ˛:

(7.2.6)

LW w02  MW w2 D cw01 C .1 C d /w02 ;

(7.2.7)

Likewise,

l1 .w02 /  m1 .w2 / D .1 C d  /ı C c  ˛; l2 .w02 /  m2 .w2 / D .1 C d  / C c C ı˛:

(7.2.8)

From these relations, we obtain .   /.1 C a  / C b.ı  /  ˛. C  / D 2ˇ.w01 ; j w01 /  0; .k  ı/.1 C d  / C c.k   / C ˛.kı C / D 2ˇ.w02 ; j w02 /  0: Therefore, 2.21/C2˛.42˛/2˛ 22 .1Ck/C

2˛ 2 2 .1 C k/2  0; k  2

2k.2  1/ C 2˛.4 C 2k˛  /  2˛ 2 2 .1 C k/ C

2˛ 2 2 k.1 C k/2  0: k  2 (7.2.10)

Summing up these inequalities, we get 2.2  1/.1 C k/  8˛ 2 2 .1 C k/ C

(7.2.9)

2˛ 2 2 .1 C k/2  0; k  2

220

7 Holomorphically Induced Representations...

so   ˛ 2 2 .1  k/2 C 42 .2  1/ C  0: k  2

(7.2.11)

On the other hand, we can write C C 0 0  0 w01 D w 1 C w1 C w1 ; w2 D w2 C w2 C w2 ;

so the relations (7.2.5), (7.2.7) and j w01 D w1 ; j w02 D w2 imply that C C C   LW w 1 D .1 C a/w1 C bw2 ; LW w1 D aw1 C bw2 C C C   LW w 2 D cw1 C .1 C d /w2 ; LW w2 D cw1 C d w2 : C  C  We set P  D Rw D RwC and P C are both 1 C Rw2 , then P 1 C Rw2 and P invariant under LW . We first suppose that dim.P  / D 2 (resp. dim.P C / D 2) and let x be an eigenvalue of LW jP  (resp. LW jP C ). Then x satisfies the equation ˇ ˇ ˇ ˇ1  x C a c ˇ D .1  x  /2 C 2 ˛ 2 D 0 ˇ ˇ b 1  x C dˇ ˇ ˇ   ˇx C a c ˇˇ 2 2 2 ˇ D .x C / C  ˛ D 0 : resp. ˇ b x C d ˇ

Thus x D 1   ˙ i ˛ (resp. x D  ˙ i ˛). This equality together with the inequality 0    1=2, which comes from the inequalities (7.2.4) and (7.2.11), implies that the real part of x is not equal to 1=2 (resp. 1=2). Thus we have a contradiction. C C  Next we suppose that w 2 D tw1 or w2 D tw1 with 0 ¤ t 2 R. Then, .1 C 2 /t 2 C 2.1 C k/t C k 2 C 2 D 0: Since 2 .1 C k/2  .1 C 2 /.k 2 C 2 / D 2k2  k 2  4 D .k  2 /2 < 0; we conclude that t 62 R. This is a contradiction. Therefore P  D P C D f0g. Since ˇ ˇ ˇ1 C a b ˇ 2 2 2 ˇ ˇ ˇ c 1 C d ˇ D .1  / C  ˛ ; we have w01 D w02 D 0 by (7.2.4), (7.2.11) and ˛ ¤ 0. In consequence, w01 D w02 D 0 so w1 D j w01 D 0 and w2 D j w02 D 0. Therefore the inequalities (7.2.9)–(7.2.11) are replaced by corresponding equalities. By (7.2.9) and (7.2.10),

7.2 Exponential j-Algebras

221

2˛ .4.k C 1/  .k C 1//  2˛ 2 2 .1 C k/.k  1/ D 0; hence we have k1D

 .4  1/: ˛

(7.2.12)

On the other hand, by (7.2.6), .1 C k/.1  4 C 2˛/ C 2  8 D 0: 2  k By (7.2.8), .1 C k/.4k  k C 2˛/ C 8  2 D 0; 2  k which is added to the above equality to get .4  1/.k  1/ C 4˛ D 0:

(7.2.13)

  By (7.2.12) and (7.2.13),  .4  1/2 C 4˛ 2 2 D 0. Thus  D 0. By (7.2.11) and (7.2.12), k D 1 and  D 1=2. Therefore li .x/ D mi .x/ D 0 .i D 1; 2/ for all x 2 W , and W is invariant under both L and M . Further, 1 1 Lje1 D  .je1 C ˛je2 /; Lje2 D  .je2  ˛je1 / 2 2 and Mje1 D e1 ; Mje2 D e2 . Thus we have the splitting J D J ˚ J0 ˚ JC ; where J D W ˚jV D W ˚Rje1 ˚Rje2 ; J0 D W0 ; JC D WC ˚V D WC ˚Re1 ˚Re2 :  We generalize I. I. Pjatetskii-Shapiro’s structure theorem for a normal j -algebra (cf. Theorem 2 at Section 3 in Chapter 2 of [60], Theorem 5.13 of [67]) to an exponential j -algebra. Theorem 7.2.5. Let .g; j; !/ be an exponential j -algebra. We define an inner product S on g by S.X; Y / D !.ŒjX; Y / for X; Y 2 g. Let a be the orthogonal complement of  D Œg; g with respect to the form S . Then a is a commutative Lie subalgebra of g, g D a C , and the adjoint representation of a on  is complex diagonalizable. For ˛ 2 a , we set

222

7 Holomorphically Induced Representations...

˛ D fX 2 I ŒA; X  D ˛.A/X 8 A 2 ag and let ˛i ; 1  i  r; be those root spaces ˛ for which j.˛ /  a. Then dim.˛i / D 1 .1  i  r/ and r D dim a .r is called the rank of g/. If we order ˛1 ; : : : ; ˛r in an appropriate way, then all other roots are of the form 1 1 .˛m  ˛k /; .˛m C ˛k / 2 2 1 ˛k .1  k  r/ 2

.1  k < m  r/;

.not all possibilities need occur/, and  can be decomposed as follows: X

D

1

 2 .˛m ˛k / ˚ g 1 ˚

X

1

 2 .˛m C˛k / ;

2

k<m

km

P 1 1 where g 1 D k Q 2 ˛k and Q 2 ˛k is an .ad a/-invariant subspace, the complexification 2 of which is the sum of the root spaces of ada with the roots of the form A 7!

1 ˛k .A/.1 C iˇk;p / .A 2 a/ 2

with ˇk;p 2 R. Let g0 D a ˚

X

1

 2 .˛m ˛k / and g1 D

k<m

X

1

 2 .˛m C˛k / :

km

Then g D g0 ˚ g 1 ˚ g1 2

and Œgi ; gk   gi Ck with the convention that gi Ck D f0g if i C k ¤ 0; 1=2 or 1. We have  1    1 1 1 j  2 .˛m ˛k / D  2 .˛m C˛k / .1  k < m  r/; j Q 2 ˛k D Q 2 ˛k .1  k  r/: Let Ui be the nonzero element of ˛i such that Œj Ui ; Ui  D Ui and let s D Then

Pr

i D1 Ui .

˛k .j Ui / D ıi k ; adjsjg0 D 0; adjsjg1 D Id and adjsjg 1 is semi-simple and its eigenvalues have the real part 1=2. Finally, 2

jX D Œs; X  for X 2 g0 .

7.2 Exponential j-Algebras

223

Proof. We proceed by induction on dim g. Let RU be an ideal of dimension 1, which certainly exists by Lemma 1 in Sect. 2 of [61]. Then, since !.Œj U; U / > 0, we have Œj U; U  ¤ 0, so we can choose U in such a way that Œj U; U  D U . This completes the initial step of our induction, where dim g D 2. Let dim g > 2 and W the orthogonal complement of Rj U ˚ RU with respect to S . Then W is invariant under both j and adj U and L D adW j U D .adj U /jW commutes with j . We regard L as a complex linear transformation on the complex space W . Then the transformation L is semi-simple and the eigenvalues have the form either 12 C i 1 or i 2 , where 1 ; 2 are real numbers. We denote by W 1 the 2

subspace spanned by the eigenvectors for the eigenvalues of the form 12 C i 1 .1 2 R/ and by W0 for the eigenvalues of the form i 2 .2 2 R). Then W0 ; W 1 are clearly 2 j -invariant and W D W0 ˚ W 1 ; ŒW0 ; W0   W0 ; ŒW0 ; W 1   W 1 ; ŒW 1 ; W 1   RU 2

2

2

2

2

(cf. Theorem 1 of [61]). In our case, LjW0 D 0 by the assumption that g is an exponential algebra. Since .W0 ; j jW0 ; !jW0 / is an exponential j -algebra, the induction hypothesis allows us to write W0 as W0 D aW0 ˚

X

1

0 2 .˛Qm ˛Qk / ˚

1k<mr1

X

1

Q 02

˛Qk

X

˚

1kr1

1

0 2 .˛Qm ˚˛Qk /

1kmr1

with ˛Q k 2 aW0 for 1  k  r  1 and 0 D ŒW0 ; W0 . Let X 2 W0 and let Y 2 W 1 be an eigenvector of L corresponding to the eigenvalue  Œj U; Y  D

1 2

C i  . 2 R/. Then

 1 1 C i  Y; i.e. Œj U; Y  D Y C jY: 2 2

Hence Œj U; jY  D 12 jY  Y and 1 Œj U; Y   Œj U; jY  D 2



 1 2 C  Y: 4

Thus Y D Œj U; Y0  with Y0 D

4 .1 C 42 /



1 Y  jY 2

 2 W1 ; 2

so S.X; Y / D !.ŒjX; Y / D !.ŒjX; Œj U; Y0 / D !.Œj U; ŒjX; Y0 / D S.U; ŒjX; Y0 /:

2

224

7 Holomorphically Induced Representations...

Since ŒjX; Y0  2 W 1 , S.X; Y / D 0. Thus, the orthogonal complement a of 2

 D Œg; g D 0 ˚ W 1 ˚ RU 2

with respect to S is aW0 ˚ Rj U , which clearly is a commutative subalgebra of g. Define ˛k 2 a by ˛k jaW0 D ˛Q k and ˛k .j U / D 0 for 1  k  r  1, and let ˛r 2 a be such that ˛r jaW0 D 0 and ˛r .j U / D 1. Then it is easy to see that the spaces 1

1

˛Q

1

0 2 .˛Qm ˛Qk / ; Q 02 k ; 0 ˛Qk ; 0 2 .˛Qm C˛Qk / are the same as 1

1

1

 2 .˛m ˛k / ; Q 2 ˛k ; ˛k ;  2 .˛m C˛k / respectively, so X

gDa˚

1

 2 .˛m ˛k / ˚

1k<mr1

˚

X

X

1

Q 2 ˛k

1kr1

1

 2 .˛m C˛k / ˚ W 1 ˚ RU: 2

1kmr1

Now on the finite-dimensional vector space W 1 over R with the complex 2 structure j jW 1 , the bilinear form ˇ is determined by ˇ.X; Y / D !.ŒX; Y / .X; Y 2 2

W 1 /, and the symplectic representation k .1  k  r  1/ of the exponential 2 Kähler algebra gk D Rj Uk C RUk defined by k .j Uk / D Lk D adW 1 j Uk and k .Uk / D Mk D adW 1 Uk is of exponential type. Therefore we can write

2

2

W 1 D Wk ˚ W0k ˚ WCk 2

with the properties stated in Lemma 7.2.4. For k ¤ l, we have Lk jWl ˚W l D 0 D Mk jWl ˚W l : C

C

In fact, since Lk and Ll are both semi-simple and commute mutually, Wl is invariant under Lk , and the transformations Lk j.Wl /C and Ll j.Wl /C are simultaneously diagonalizable. Thus, if we suppose Lk jWl ¤ 0, we shall have one of the following two possibilities: (1) There is a nonzero vector X 2 Wl such that Ll X D  12 X and Lk X D ˙ 12 X . But we can apply Lemma 7.2.4 to the symplectic representation k;l of the exponential Kähler algebra

7.2 Exponential j-Algebras

225

gk;l D R.j Uk C j Ul / C R.Uk C Ul / defined by k;l .j Uk C j Ul / D Lk C Ll and k;l .Uk C Ul / D Mk C Ml : Hence if Lk X D  12 X , we should have 1 as an eigenvalue for Lk CLl , which is impossible. If Lk X D 12 X , then Mk X D 0 so that .Lk C Ll /X D 0 with .Mk C Ml /X D jX , which is also impossible. (2) There are two nonzero vectors X1 ; X2 2 Wl such that 1 1 Ll X1 D  .X1  ˛X2 /; Ll X2 D  .X2 C ˛X1 / 2 2 and Lk X1 D .X1  ˛X2 /; Lk X2 D .X2 C ˛X1 / with  D ˙ 12 ; 0 ¤ ˛ 2 R. As in case (1), we have a contradiction. Similarly we can show Lk jW l D 0. Thus Mk jWl ˚W l D 0. Next, the spaces W˙k C C are L-invariant for 1  k  r  1. We show LjWk ˚W k D C

1 1 1 Id; Lk jWk D  Id; Lk jW k D Id: C 2 2 2

It suffices to show that LjW k D 12 Id and Lk jW k D 12 Id . In fact, that j ıL D Lıj C

C

on W implies LjWk D 12 Id , and that ŒLk ; Mk X D Mk X for X 2 Wk implies jLk X D  12 jX , so Lk jWk D  12 Id . We suppose that LjW k ¤ 12 Id or Lk jW k ¤ 1 Id . 2

C

Then there are nonzero vectors X1 ; X2 2 WCk such that LX1 D Lk X1 D

1 1 .X1  ˛X2 /; LX2 D Lk X2 D .X2 C ˛X1 / 2 2

with 0 ¤ ˛ 2 R. Then, since Lıj D j ıL, on W , LjX1 D

1 1 .jX1  ˛jX2 /; LjX2 D .jX2 C ˛jX1 /: 2 2

But by the proof of Lemma 7.2.4, 1 1 Lk jX1 D  .jX1 C ˛jX2 /; Lk jX2 D  .jX2  ˛jX1 /: 2 2

C

226

7 Holomorphically Induced Representations...

If we set ŒjX1 ; jX2  D tU with t 2 R, then Œj Uk ; ŒjX1 ; jX2  D tŒj Uk ; U  and hence ŒjX1 ; jX2  D 0. Furthermore, we have Œj.U C Uk /; jX1  D ˛jX2 ; Œj.U C Uk /; jX2  D ˛jX1 ; which contradicts the assumption that g is an exponential algebra (cf. Theorem 1 of  12 ˛Qk

[69], I). Therefore it is clear that Wk D W 1 1

˛Q

is the  12 ˛Q k -weight space of W 1 2

2

with respect to adaW0 . Also WCk D W 12 k , so that 2

X

W1 D

 12 ˛Qk

W1

2

2

1kr1

X

˚

1

W 12

˛Qk

˚ W 10 ; 2

2

1kr1

where W 10 D fX 2 W 1 I ŒA; X  D 0 8 A 2 aW0 g: 2

2

Taking LjWk ˚W k D C f˛1 ; : : : ; ˛r g of roots, W1 D

1 2 Id

into account, we have, relative to the new system

X

2

1

 2 .˛r ˛k / ˚

1kr1

X

1

1

 2 .˛r C˛k / C Q 2 ˛r :

1kr1

These facts with the observation that j.WCk / D Wk ; j.W0k / D W0k by Lemma 7.2.4 prove Theorem 7.2.5 except for the last assertion with RU D ˛r ; U D Ur . Since 1 1  2 .˛r ˛k /  W , the relation Xr;k 2  2 .˛r ˛k / implies ŒUr ; Xr;k  D 0; ŒUk ; Xr;k  D ŒMk ; Xr;k  D jXr;k : So, if X 2 g0 is written in the form X D X 0 C j Ur C

X

Xr;k

1kr1 1

with X 0 2 W0 ;  2 R and Xr;k 2  2 .˛r ˛k / , then 2 r1 X X 4Uk C Ur ; X 0 C j Ur C Œs; X  D kD1

2

D jX 0 C 4

3 Xr;k 5

1kr1

r1 X kD1

Uk ;

X 1kr1

3

Xr;k 5  Ur D jX:



7.3 Exponential Kähler Algebras

227

7.3 Exponential Kähler Algebras We generalize the fundamental theorem for a normal Kähler algebra in Part III of [40]. Let .g; j; ˇ/ be an exponential Kähler algebra. A j -invariant Lie subalgebra (resp. ideal) of g is called a Kähler subalgebra (resp. ideal). Lemma 7.3.1. Let g be an exponential Kähler algebra and let a be a minimal ideal of g such that dim a D 2. Then k D j a C a is a commutative Kähler algebra and dim k D 4. Proof. Since g is an exponential solvable Lie algebra, there is a basis fe1 ; e2 g of a such that Œx; e1  D .x/.e1  ˛e2 /; Œx; e2  D .x/.e2 C ˛e1 / .8 x 2 g/ with  2 g and 0 ¤ ˛ 2 R. This implies that Œe1 ; e2  D 0. Since a is a minimal ideal, there is an element x0 2 g such that .x0 / D 1. Then, 0 D ˇ.x0 ; Œe1 ; e2 / D ˇ.Œx0 ; e1 ; e2 / C ˇ.e1 ; Œx0 ; e2 / D 2ˇ.e1 ; e2 /: It follows that dim k D 4. Let 1 D .je1 / and 2 D .je2 /. Then Œje1 ; je2  D j Œje1 ; e2  C j Œe1 ; je2  D .1 ˛  2 /je1 C .1 C ˛2 /je2 : We apply ade1 to both sides of this equation to get 0 D ˛.21 C 22 /.e1  ˛e2 /: Therefore 1 D 2 D 0; Œje1 ; je2  D 0, which implies that k is a commutative Kähler algebra.  Theorem 7.3.2. Let g be an exponential Kähler algebra, then g can be decomposed into a semi-direct sum g D J C H; where J is a commutative Kähler ideal and H is an exponential j -subalgebra. This section will be fully devoted to the proof of this theorem. The proof will follow one given in [40] for the case of a normal Kähler algebra with slight modifications. As it is rather complicated, we sketch here the outline of its logical structure. The proof will proceed by induction on dim g. We first present some consequences of the theorem as Lemmas 7.3.3–7.3.8, which will be included in the induction hypothesis. The theorem for dim g D 2 is trivial. The essential part of

228

7 Holomorphically Induced Representations...

the proof is that of key Proposition 7.3.10, which will be divided into two cases and each case will be proved in Proposition 7.3.13 and Proposition 7.3.14 respectively. (The induction hypothesis will be used only in the proof of Proposition 7.3.14.) Assume that the theorem is true and let H D H0 ˚H 1 ˚H1 be the decomposition 2 of H given in Theorem 7.2.5 and let s 2 H be the element used there. The element s is called the principal idempotent of the Kähler algebra g. Lemma 7.3.3. The transformation adJ js is semi-simple and its eigenvalue is either 0 or ˙ 12 C i . 2 R/. We denote by J0 the eigenspace corresponding to the eigenvalue 0 and by Je 1 .resp. J e1 / the sum of the eigenspaces of JC corresponding  2

2

C i (resp.  12 C i). Then clearly Je 1 D Je 1 ; J e1 D J e1 . 2 2 2 2 We set J 1 D Je 1 \ J .resp. J 1 D J e1 \ J /, then 

to the eigenvalues 2

1 2

2

2

2

 1  J D J 1 ˚ J0 ˚ J 1 ; j J D J  D ˙ ; 0 2 2 2 and Œs; x D jx for x 2 J 1 . 2

Proof. We consider the representation of the two-dimensional Kähler algebra generated by s and js on the ideal J induced by the adjoint representation of g. This is easily seen to be a symplectic representation of exponential type. Lemma 7.3.3 is an immediate consequence of Lemma 7.2.4.  We define an inner product S on g by S.x; y/ D ˇ.jx; y/ for x; y 2 g. Lemma 7.3.4. The subspaces H and J are mutually orthogonal with respect to S . Proof. We extend ˇ to gC by linearity and denote it by the same letter ˇ. Let x; y 2 gC be eigenvectors of adjs corresponding to eigenvalues ˛; 2 C respectively. Then ˇ.js; Œx; y/ D .˛C /ˇ.x; y/ and xN (resp. y) N is an eigenvector corresponding to the eigenvalue ˛N (resp. N ). Thus, if we denote by V the space J or the space H , then ˇ.js; ŒV ; V / D 0 and  C  ¤ 0 will imply ˇ.V ; V / D 0. (a) S.H ; H / D f0g for  < . In fact, j H D H1 and, since 1   C  > 1, ŒH1 ; H  D f0g. Therefore, S.H ; H / D ˇ.H1 ; H / D f0g. (b) S.J ; J / D f0g for  < . In fact, j J D J ; C > 0 and ŒJ ; J  D f0g since J is commutative. Thus, S.J ; J / D ˇ.J ; J / D f0g. (c) It remains to show that S.J ; H/ D f0g. We first prove that ˇ.js; J 1 / D f0g. 2 We can write each element of J 1 in the form j w; w 2 J 1 , 2

2

ˇ.s; w/ D ˇ.js; j w/ D ˇ.js; Œs; w/ D ˇ.Œjs; s; w/ C ˇ.s; Œjs; w/ D ˇ.s; w/ C ˇ.s; Œjs; w/: This shows that ˇ.s; J 1 / D ˇ.s; Œjs; J 1 / D f0g, so ˇ.js; J 1 / D f0g. We show 2 2 2 that S.J ; H / D f0g for  < . In fact, j J D J and  C  > 0, thus

7.3 Exponential Kähler Algebras

229

ŒJ ; H   J 1 and ˇ.js; ŒJ ; H /  ˇ.js; J 1 / D f0g. Hence S.J ; H / D 2 2 ˇ.J ; H / D f0g. If   , then  < 1   and S.J ; H / D S.j J ; j H / D S.J ; H1 / D f0g:



Lemma 7.3.5. ŒH0 ; J0  D f0g. Proof. We consider the operator A D adJ0 h with h 2 H0 . Since ŒH1 ; J0  D f0g, using the integrability condition, it follows that A commutes with j . Since ˇ is closed and since J is commutative, ˇ.Ax; y/ C ˇ.x; Ay/ D 0 .x; y 2 J0 /: This shows that the operator A is skew-symmetric with respect to the scalar product S . Since its eigenvalues must be purely imaginary, we have A D 0.  Lemma 7.3.6. Assume that  C  C D 0. If an element h 2 H commutes with J , then it commutes with J . Proof. Let a 2 J . The form ˇ being closed, the assumption implies ˇ.Œh; a; x/ D 0 for all x 2 J . Since j Œh; a 2 J by Lemma 7.3.3, we have ˇ.Œh; a; j Œh; a/ D 0 and therefore Œh; a D 0.  Lemma 7.3.7. J D fa 2 gI Œja; a D 0g. Proof. It is sufficient to prove that a 2 J if Œja; a D 0. Write a D x C h with x 2 J ; h 2 H. Then 0 D Œja; a D Œj h; h .mod J /: Since H is an exponential j -algebra, we have h D 0 as desired.



Lemma 7.3.8. The centre z of g is contained in J0 and j -invariant. Proof. The last lemma shows that z  J . Moreover, it is clear that z  J0 . From Lemma 7.3.3 and the fact that ŒH1 ; J0  D f0g, it follows that z coincides with the centralizer of H 1 in J0 . Let a 2 z. From the integrability condition, we find that 2 Œja; j h D j Œja; h for all h 2 H 1 so that 2

ˇ.j Œja; h; Œja; h/ D ˇ.Œja; j h; Œja; h/: The form ˇ being closed, the equalities Œja; Œja; h D 0 and Œj h; Œja; h D 0 imply ˇ.j Œja; h; Œja; h/ D 0. This proves that Œja; h D 0 and ja 2 z.  We now begin the proof of Theorem 7.3.2 by induction on dim g. If dim g D 2, the theorem is trivially valid. We assume dim g D n and that Theorem 7.3.2 and therefore Lemmas 7.3.3–7.3.8 are valid for exponential Kähler algebras of dimension less than n.

230

7 Holomorphically Induced Representations...

Definition 7.3.9. An exponential j -algebra of rank 1 is said to be elementary. Proposition 7.3.10. In g, there is either an elementary or commutative nonzero Kähler algebra. In what follows, we aim at the proof of Proposition 7.3.10. We choose and fix a minimal ideal a of g. Then one of the following three cases will occur (cf. Lemma 7.3.1). (i) dim a D 1; Œj a; a D a. Let r 2 a be the nonzero element such that Œjr; r D r. (ii) dim a D 1; Œj a; a D f0g. Let r 2 a be a nonzero element. (iii) dim a D 2; k D j a ˚ a is a four-dimensional commutative Kähler subalgebra. Let fr1 ; r2 g be a basis of a such that Œa; r1  D l.a/.r1  r2 /; Œa; r2  D l.a/.r2 C r1 / for all a 2 g with l 2 g ;  2 R. In each case, we set k D j a ˚ a. Lemma 7.3.11. We put P D fa 2 gI Œa; a D Œja; a D f0gg. Then P is invariant under both j and adjx .x 2 a/. Moreover, the operator adP jx .x 2 a/ commutes with j . Proof. The invariance of P under j is immediate from the definitions. From the integrability condition, we have Œjx; jp D j Œjx; p

(7.3.1)

for all p 2 P . Using the Jacobi identity and (7.3.1), we find ŒŒjx; p; y D 0 and Œj Œjx; p; y D ŒŒjx; jp; y D 0 for y 2 a, so that Œjx; p 2 P . The commutativity of j and adP jx follows from (7.3.1).  Lemma 7.3.12. For all u; v 2 g and x 2 a,   d  t .adjx/ t .adjx/  ˇ e u; e v D ˇ jx; e t .adjx/ Œu; v : dt

(7.3.2)

Proof. d  t .adjx/ t .adjx/  ˇ e u; e v dt     Dˇ Œjx; e t .adjx/ u; e t .adjx/ v C ˇ e t .adjx/ u; Œjx; e t .adjx/ v     Dˇ jx; Œe t .adjx/ u; e t .adjx/ v D ˇ jx; e t .adjx/ Œu; v :



Now, we prove Proposition 7.3.13, from which Proposition 7.3.10 will follow in case (i). Note that we shall not use the induction hypothesis for the proof of Proposition 7.3.13.

7.3 Exponential Kähler Algebras

231

Proposition 7.3.13. If the minimal ideal a is of type (i), then g can be decomposed into a semi-direct sum g D N C g0 which satisfies the following conditions. (1) N is a Kähler ideal and is an elementary j -algebra, and is decomposed into a direct sum of subspaces: N D Rjr C Rr C U , where U D N 1 in conformity 2 with the notation of Theorem 7.2.5. 0 0 (2) g is a Kähler subalgebra orthogonal to N and Œk; g  D f0g. Proof. Let P be the subspace constructed in Lemma 7.3.11. By the relation Œjr; r D r one can find in a unique way, for any u 2 g, numbers  and  so that u  jr  r 2 P . This means that g decomposes into the direct sum of subspaces g D Rjr ˚ Rr ˚ P:

(7.3.3)

Using the last lemma, we study the eigenvalues of adjr on P . (a) If in (7.3.2) we set u D r and v 2 P , then the right-hand side is 0. Using the equality e t .adjr/ r D e t r;

(7.3.4)

  ˇ r; e t .adjr/ v D ˛e t ;

(7.3.5)

we obtain for v 2 P

where ˛ is some real number. According to Lemma 7.3.11, the operator adjr commutes with j on P . Therefore, for v 2 P ,   ˇ jr; e t .adjr/ v D ˛e t : (7.3.6) By (7.3.3) and (7.3.4), we obtain, for x 2 g,   ˇ jr; e t .adjr/ x D ˛e t C e t

(7.3.7)

with some real numbers ˛; . Now for u; v 2 g, formula (7.3.2) gives   ˇ e t .adjr/ u; e t .adjr/ v D ˛e t C e t C ı

(7.3.8)

with some real numbers ˛; ; ı. Since adP jrıj D j ıadP jr, formula (7.3.8) implies, for u 2 P; v 2 g,   S e t .adjr/ u; e t .adjr/ v D ˛e t C e t C ı

(7.3.9)

with some real numbers ˛; ; ı. (b) We regard the operator adjrjP as a complex linear transformation on P . Let p 2 P be its eigenvector corresponding to an eigenvalue  C i .;  2 R/, then e t .adjr/ p D e .Ci/t p. So by (7.3.9),

232

7 Holomorphically Induced Representations...

e 2t S..cos t C i sin t/p; .cos t C i sin t/p/ D ˛e t C e t C ı: The left-hand side equals e 2t S.p cos t C jp sin t; p cos t C jp sin t/ D e 2t .cos2 t C sin2 t/S.p; p/ D e 2t S.p; p/: Thus we have e 2t S.p; p/ D ˛e t C e t C ı: Since S.p; p/ ¤ 0,  can have only the values 0; ˙ 12 . We now check that the operator adP jr is semi-simple (and therefore adjr is also semi-simple on g). Let p; q be vectors in P such that Œjr; p D . C i/p; Œjr; q D . C i/q C p .;  2 R/: Then e t .adjr/ q D e .Ci/t q C te .Ci/t p and by formula (7.3.9)   S e t .adjr/ p; e t .adjr/ q D e 2t S..cos t C i sin t/p; .cos t C i sin t/q/ C te 2t S..cos t C i sin t/p; .cos t C i sin t/p/ D e 2t S.p; q/ C te 2t S.p; p/ D ˛e t C e t C ı: S.p; p/ being nonzero, this equality is impossible. (c) Since the operator adP jr has no eigenvalue with real part 1, we have ˛ D 0 in (7.3.5). Looking at the formulas deduced from (7.3.5), we see that in each of them ˛ D 0, and therefore  12 C i . 2 R/ is not an eigenvalue of adP jr. We denote by P0 the eigenspace of adP jr corresponding to the eigenvalue 0, and by P 1 the sum of its eigenspaces corresponding to the eigenvalues of the form 1 2

2

C i . 2 R/. The algebra g decomposes into a direct sum of subspaces: g D Rjr C Rr C P 1 C P0 : 2

The subspaces P 1 ; P0 are clearly j -invariant. The formulas (7.3.5), (7.3.6) 2 and (7.3.9) show that P0 is orthogonal to N D Rjr C Rr C P 1 . As in Sect. 2 2 of [61], we can show that N is an ideal of g. Now the relations (7.3.5) and (7.3.6)

7.3 Exponential Kähler Algebras

233

show that P 1 and k D Rjr ˚ Rr are mutually orthogonal with respect to S . Let u 2

(resp. v) be an eigenvector of adP jr corresponding to an eigenvalue 1 C i ; 2 R). By (7.3.9), 2

1 2

C i (resp.

e t .S.u; v/ cos..  /t/ C S.j u; v/ sin..  /t// D e t C ı: Hence if  ¤ , it follows that S.u; v/ D S.j u; v/ D 0. Therefore ˇ.u; v/ D 0 and ˇ.jr; Œu; v/ D 0. If  D , ˇ.jr; Œu; v/ D

1 1 ˇ.u; v/ C ˇ.j u; v/ C ˇ.u; v/ C ˇ.u; j v/ D ˇ.u; v/: 2 2

In either case we have ˇ.jr; Œu; v/ D ˇ.u; v/:

(7.3.10)

Let ! 2 N  be defined by !.x/ D ˇ.jr; x/ .x 2 N /. Then it can be shown that !.Œx; y/ D ˇ.x; y/ for all x; y 2 N . In fact, let x D ajr C u C br; y D cjr C v C dr .a; b; c; d 2 R/ with u; v 2 P 1 . Then 2

Œx; y D .ad  bc/r C w C Œu; v with w 2 P 1 . Thus 2

!.Œx; y/ D ˇ.jr; .ad  bc/r C w C Œu; v/ D .ad  bc/ˇ.jr; r/ C ˇ.jr; Œu; v/: On the other hand, ˇ.x; y/ D .ad  bc/ˇ.jr; r/ C ˇ.u; v/ and Eq. (7.3.10) means !.Œx; y/ D ˇ.x; y/. In consequence, N is an exponential j -algebra. If we set g0 D P0 , then g0 is the orthogonal complement of the Kähler ideal N so that g0 is a Kähler subalgebra of g (cf. Lemma 2 at Sect. 6 in Part II of [40]).  From now on, we always assume that the minimal ideal a is of type (ii) or (iii). Proposition 7.3.14. There is a commutative Kähler ideal N which contains a.

234

7 Holomorphically Induced Representations...

We prepare several lemmas for the proof of this proposition. We denote by P the space constructed in Lemma 7.3.11. Lemma 7.3.15. Œj a; g  P . Proof. The Jacobi identity and the integrability condition imply that ŒŒj a; g; a D Œj a; Œg; a D f0g; Œj Œj a; g; a D ŒŒj a; g; a D f0g: That is, Œj a; g  P .



Lemma 7.3.16. ad.jx/ ı ad.jy/ D 0 for any x; y 2 a. Proof. Since ad.jx/ ı ad.jy/ D ad.jy/ ı ad.jx/, it suffices to show that .ad.jx//2 D 0 for all x 2 a. (a) We show that the set Œj a; P  is orthogonal to k. Let p 2 P and x; y 2 a. Since ˇ is closed, we have ˇ.y; Œjx; p/ D ˇ.jx; Œy; p/ D 0. Using Lemma 7.3.11, we also obtain ˇ.jy; Œjx; p/ D ˇ.y; j Œjx; p/ D ˇ.y; Œjx; jp/ D 0: (b) From Lemma 7.3.15 and (a), it follows that ˇ.jy; .adjx/2 g/ D f0g. Then, formula (7.3.2) shows that, for u; v 2 g,  d 3  t .adjx/ t .adjx/  d2  ˇ e u; e v D ˇ jx; e t .adjx/ Œu; v 3 2 dt dt   D ˇ jx; Œjx; Œjx; e t .adjx/ Œu; v D 0; i.e.   ˇ e t .adjx/ u; e t .adjx/ v D ˛t 2 C t C ı .˛; ; ı 2 R/: Whence for p 2 P; q 2 g,   S e t .adjx/ p; e t .adjx/ q D ˛t 2 C t C ı:

(7.3.11)

Setting q D p, we obtain that the operator adjx has no nonzero eigenvalue on P (and therefore on g). (c) We assume that .adjx/2 ¤ 0. Then, there exist elements u; v; w 2 g such that Œjx; u D 0; Œjx; v D u; Œjx; w D v: It is clear that e t .adjx/ u D u; e t .adjx/ v D v C tu; e t .adjx/ w D w C tv C

t2 u: 2

7.3 Exponential Kähler Algebras

235

We have v 2 P by Lemma 7.3.15 and we can put p D v; q D w in (7.3.11) to get   t2 S v C tu; w C tv C u D ˛t 2 C t C ı: 2 Since S.u; u/ ¤ 0, this is impossible.



Let L be a Lie algebra over R. A decreasing sequence of Lie subalgebras fL.k/ gk2Z ,     L.2/  L.1/  L.0/  L.1/  L2     with the following properties is called a filtration of L: (1) [k L.k/ D L; (2) \k L.k/ D f0g; (3) ŒL.k/ ; L.l/   L.kCl/ . With the aid of the operator adjx .x 2 a/, we construct a j -invariant filtration of g. Let U D g=k and  the natural projection of g onto U . Case 1.

a is of type (ii). Let A0 be the operator on U induced by adjr. We set g.k/ D g.k < 0/; g.0/ D  1 .kerA0 /; g.1/ D  1 . Im A0 /; g.2/ D k; g.k/ D f0g.k > 2/:

Case 2. a is of type (iii). Let Ai .i D 1; 2/ be the operators on U induced by adjri . We set g.k/ D g.k < 0/; g.0/ D  1 .kerA1 \ kerA2 /; g.1/ D  1 . Im A1 C Im A2 /; g.2/ D k; g.k/ D f0g.k > 2/ From the integrability condition, it follows that Ai .i D 0; 1; 2/ commute with j on U . Lemma 7.3.17. The subspaces g.k/ form a j -invariant filtration of the Lie algebra g. Furthermore Œg.1/ ; g.1/  D f0g: Proof. (a) It follows from the preceding lemma that A2i D A1 A2 D A2 A1 D 0 .i D 0; 1; 2/:

(7.3.12)

236

7 Holomorphically Induced Representations...

Therefore kerA0  Im A0 ; kerA1 \ kerA2  Im A1 C Im A2 ; so we have the inclusions g.1/  g.0/  g.1/  g.2/ : The invariance of the subspaces g.k/ with respect to j follows from the commutativity of Ai with j . (b) We prove that Œg.1/ ; g.1/  D f0g. In Case 1, let g1 ; g2 2 g and put u1 D Œjr; g1 ; u2 D Œjr; g2 . Since .adjr/2 D 0, 0 D .adjr/2 .Œg1 ; g2 / D 2Œu1 ; u2 . Therefore ŒŒjr; g; Œjr; g D f0g. Since g.1/ D Œjr; g C k, it remains to prove that Œk; Œjr; g D f0g, which follows from the last two lemmas. In Case 2, we first show that g.1/ is a Lie subalgebra. In fact, we have ŒŒjr1 ; g; Œjr1 ; g D ŒŒjr2 ; g; Œjr2 ; g D f0g as above and ŒŒjr1 ; g; Œjr2 ; g  Œjr1 ; g \ Œjr2 ; g: Since g.1/ D k C Œjr1 ; g C Œjr2 ; g; it follows that g.1/ is a Lie subalgebra. Moreover Œg.1/ ; Œg.1/ ; g.1/  D f0g. Thus g.1/ is a nilpotent Kähler algebra and the minimality of the ideal a proves that the dimension of g.1/ is less than n. Hence, by the induction hypothesis concerning Theorem 7.3.2, Œg.1/ ; g.1/  D f0g. (c) It follows immediately from the definitions that Œg.1/ ; g.2/   g.1/ ; Œg.0/ ; g.2/   g.2/ : (d) We show that Œg.0/ ; g.1/   g.1/ . Let l 2 g.0/ ; u 2 g.1/ . Since Œg.0/ ; g.2/   g.1/ , it is sufficient to consider the case where u D Œjx; g with x 2 a and g 2 g. We have Œjx; Œl; g D Œl; u C ŒŒjx; l; g. Since Œjx; Œl; g; ŒŒjx; l; g 2 g.1/ , we also have Œl; u 2 g.1/ . (e) We prove that Œg.1/ ; g.1/   g.0/ . Let g 2 g and u 2 g.1/ . Since Œjx; u D 0 and Œjx; g 2 g.1/ for x 2 a, the Jacobi identity and the already proved commutativity of g.1/ show that Œjx; Œg; u D 0. Therefore Œg; u 2 g.0/ . (f) It remains to prove that g.0/ is a Lie subalgebra. Let g1 ; g2 2 g.0/ and let x 2 a. Since Œg.2/ ; g.0/   g.2/ , we have Œjx; Œg1 ; g2  D ŒŒjx; g1 ; g2  C Œg1 ; Œjx; g2  2 g.2/ :

7.3 Exponential Kähler Algebras

237

Consequently, Œg1 ; g2  2 g.0/ .



If g.0/ D g in the filtration constructed above, then N D g.2/ is a commutative Kähler ideal in g and Proposition 7.3.14 is proved. Therefore in the following, we shall assume that g.0/ ¤ g. The induction hypothesis may be applied to the Kähler algebra g.0/ . Let g.0/ D J C H

(7.3.13)

be the decomposition corresponding to Theorem 7.3.2. Lemma 7.3.18. g.1/  J . Proof. Let g 2 g.1/ . Then jg 2 g.1/ . It follows from (7.3.12) that Œjg; g D 0. By Lemma 7.3.7, this means that g 2 J .  Lemma 7.3.19. Œg; J   g.0/ . Proof. Let g 2 g and u 2 J . Then Œjx; g 2 g.1/ for x 2 a and ŒŒjx; g; u D f0g by the previous lemma. Therefore Œjx; Œg; u D Œg; Œjx; u D 0, which proves the lemma.  From this lemma, it follows in particular that if the subalgebra g.0/ is commutative, then it is an ideal of g, so that Proposition 7.3.14 is proved in this case. In what follows, we assume that g.0/ is not commutative, i.e. H ¤ f0g. We denote by s the principal idempotent of the Kähler algebra g.0/ . For ˛ 2 C;  2 R, we set .gC /˛ D fx 2 gC I .adjs  ˛/m x D 0; 0 < 9m 2 Ng; gQ D

P

2R .gC /Ci

and set g D gQ \ g. Then it is clear that

gQ D gQ ; g D

X

.0/

g ; Œg ; g   gC ; g \ g.0/ D g :



Lemma 7.3.20. If  C  > 0 or  D  D 0, then ŒŒg; J ; J  D f0g. Proof. Let g 2 g and x 2 J . Then by the last lemma .0/

Œg; x 2 gC  HC C J : According to Lemma 7.3.6 it suffices to prove that ŒŒg; x; J.CC /  D f0g. Let y 2 J.CC / . The commutativity of J implies ŒŒg; x; y D ŒŒg; y; x. We have .0/

Œg; y 2 g.C/  H.C/ C J : If  C  > 0, then Œg; y 2 J since the operator adH js has only eigenvalues with non-negative real part. Consequently, ŒŒg; y; x D 0 in this case. If  D  D 0, then Œg; y 2 H0 C J , x 2 J0 and ŒŒg; y; x D 0 by Lemma 7.3.5. 

238

7 Holomorphically Induced Representations...

We consider the graded Lie algebra gN D gN .1/ C gN .0/ C gN .1/ C gN .2/ ; which is associated to the filtered Lie algebra g. For every element g 2 g.k/ , we denote by gN the corresponding element of gN .k/ . If we regard the same element g 2 g.k/ as an element of g.k1/ , then gN D 0. In the following, however, it will always be clear which of the subspaces g.k/ we have in mind. It follows from (7.3.12) that Œg.1/ ; g.1/  D f0g:

(7.3.14)

For a nonzero element x 2 a, we define on gN .1/ a trilinear operation .a; b; c/ ! .abc/x by .abc/x D ŒŒŒjx; a; b; c:

(7.3.15)

We establish some properties of this operation. Lemma 7.3.21. (1) The operation (7.3.15) is commutative. (2) For y 2 a, Œjy; .abc/x  D ŒŒŒjx; a; b; Œjy; c. Proof. (1) By the Jacobi identity ŒŒŒjx; a; b; c  ŒŒŒjx; b; a; c D ŒŒjx; Œa; b; c D 0 since ŒNg.1/ ; gN .1/  D f0g. This shows that .abc/x D .bac/x . It is proved analogously that .abc/x D .acb/x . (2) We take the commutator of both sides of Eq. (7.3.15) with jy and use the Jacobi identity. From the properties of the graduation and relation (7.3.14), it follows that Œjy; Œjx; a D 0 and ŒŒjx; a; Œjy; b D 0, so that there remains only ŒŒŒjx; a; b; Œjy; c of the three terms on the right-hand side.  Lemma 7.3.22. .abc/x D 0 for all a; b; c 2 g.1/ and x 2 a. Proof. To the decomposition (7.3.13) of the algebra g.0/ , there corresponds the decomposition gN .0/ D J C H

(7.3.16)

of the algebra gN .0/ . By Lemma 7.3.18, ŒNg.1/ C gN .2/ ; J  D f0g

(7.3.17)

7.3 Exponential Kähler Algebras

239

and from Lemma 7.3.19 it follows that ŒNg.1/ ; J  D f0g:

(7.3.18)

Lemma 7.3.18 implies that x 2 J . In Case 1, Œjs; r D ˛r .˛ 2 R/ since a D Rr is an ideal. By Lemma 7.3.3, ˛ D 0 or ˙ 12 , and if ˛ D  12 , then Œs; r D jr 62 a, so that this case is impossible. Furthermore, if r 2 J˛ , then jr 2 J˛ and Œjs; jr D ˛jr (cf. the proof of Lemma 7.2.4). Going over to the algebra gN , we obtain the following relation Œjs; jr  D ˛jr .˛ D 0 or ˛ D

1 /: 2

(7.3.19)

By the definition of the subspaces gN .k/ , it is clear that the operator adjr maps gN .1/ isomorphically onto gN .1/ . The operator adjs is semi-simple on gN .1/ and has eigenvalues 0; ˙ 12 C i . 2 R/ on it. Relation (7.3.19) shows that adjs is semisimple also on gN .1/ and has eigenvalues ˛; ˙ 12 C i C ˛ there. We define the .1/ space gN  . 2 R/ just as the space g by making use of the operator adjs. Let .1/ .1/ .1/ a 2 gN  b 2 gN  ; c 2 gN such that .abc/r D ŒŒŒjr ; a; b; c ¤ 0:

(7.3.20)

Œjr ; .abc/r  D ŒŒŒjr ; a; b; Œjr ; c ¤ 0:

(7.3.21)

Then also

From what has been mentioned above, it is clear that ; ; D ˛ or ˙

1 C ˛: 2

(7.3.22)

Moreover, .0/

ŒŒjr ; a; b 2 gN C˛ D HC˛ C J C˛ : If  C   ˛ ¤ 0; 1 or 12 , then ŒŒjr ; a; b 2 J , which is impossible in view of (7.3.18) and (7.3.20). Using the symmetry of .abc/r , we know that the possible values of  C ;  C ; C  are one of ˛; 1 C ˛; 12 C ˛:  C ;  C ; C  D ˛; 1 C ˛; or

1 C ˛: 2

(7.3.23) .1/

.1/

Lemma 7.3.20 shows that if  C ı > 0 or  D ı D 0, then ŒŒNg.1/ ; gN  ; gN ı  D f0g. .1/

.1/

Since Œjr ; a 2 gN ˛ and Œjr ; c 2 gN ˛ , the condition (7.3.21) can be satisfied only

240

7 Holomorphically Induced Representations...

in the case where  C  2˛  0 and   ˛;  ˛ are not simultaneously zero. Using the symmetry of .abc/r , we obtain  C ;  C ; C   2˛;

(7.3.24)

where at most one of the three numbers ; ; is equal to ˛. This condition and (7.3.22)–(7.3.24) can be simultaneously satisfied when neither ˛ D 0 nor ˛ D 12 . In Case 2, we have Œjs; r1  D ˛.r1  r2 /; Œjs; r2  D ˛.r2 C r1 / . 2 R/ with ˛ D 0 or ˛ D ˙ 12 (cf. Lemma 7.3.3). If ˛ D  12 , then Œs; r1  D jr1 62 a, which is impossible. Furthermore, the proof of Lemma 7.2.4 shows that Œjs; jr1  D ˛.jr1 C jr2 /; Œjs; jr2  D ˛.jr2  jr1 /: Let uN 1 ; uN 2 be two elements of gN .1/ not simultaneously zero such that Œjs; uN 1  D .Nu1  uN 2 /; Œjs; uN 2  D .Nu2 C uN 1 / .; 2 R/: We set x1 D Œjr1 ; uN 1   Œjr 2 ; uN 2 ; y1 D Œjr1 ; uN 2  C Œjr 2 ; uN 1 ; x2 D Œjr1 ; uN 1  C Œjr 2 ; uN 2 ; y2 D Œjr1 ; uN 2   Œjr 2 ; uN 1 : Then, Œjs; x1 C iy1  D f.  ˛/ C i. C ˛/g.x1 C iy1 /; Œjs; x2 C iy2  D f.  ˛/ C i.  ˛/g.x2 C iy2 /: By definition of the subspaces g.k/ , at least one of the elements x1 C iy1 ; x2 C iy2 is different from zero. The real part of the eigenvalues of the operator adjs on gN .1/ is .1/ .1/ .1/ 0 or ˙ 12 . Thus  D ˛ or  D ˙ 12 C ˛. Let a 2 gN  ; b 2 gN  ; c 2 gN be such that .abc/x D ŒŒŒjx; a; b; c ¤ 0:

(7.3.25)

Then there is a nonzero element y 2 a such that Œjy; .abc/x  D ŒŒŒjx; a; b; Œjy; c ¤ 0:

(7.3.26)

7.3 Exponential Kähler Algebras

241

From what has been mentioned above, the conditions (7.3.25), (7.3.26) give rise to the same restrictions on the numbers ; ; as in Case 1.  The statement of this last lemma can be rephrased as follows: ŒŒg.1/ ; g; g  g.0/ :

(7.3.27)

From the Jacobi identity and the commutativity of g.1/ , it follows that Œg.2/ ; Œg.1/ ; g D f0g:

(7.3.28)

The condition (7.3.27) means Œjx; ŒŒg.1/ ; g; g  g.2/ for x 2 a. By (7.3.28) this is equivalent to ŒŒg.1/ ; g; g.1/   g.2/ :

(7.3.29)

The form ˇ being closed, (7.3.28) implies ˇ.ŒŒg.1/ ; g; g.1/ ; g.2/ / D f0g: Comparing this with (7.3.29), we finally obtain the relation ŒŒg.1/ ; g; g.1/  D f0g:

(7.3.30)

Lemma 7.3.23. The centralizer z.g.1/ / of the subalgebra g.1/ in g is a Kähler ideal. Proof. Note that z.g.1/ /  g.0/

(7.3.31)

since j a  g.1/ and z.j a/  g.0/ . Equation (7.3.30) means Œg.1/ ; g  z.g.1/ /:

(7.3.32)

From the Jacobi identity, we have Œjx; Œz.g.1/ /; g  Œz.g.1/ /; g.1/  D f0g for any x 2 a. Consequently Œz.g.1/ /; g  g.0/ and ŒŒz.g.1/ /; g; g.1/   g.1/ : Furthermore from (7.3.32), it follows that ŒŒz.g.1/ /; g; g.1/   Œz.g.1/ /; z.g.1/ /

(7.3.33)

242

7 Holomorphically Induced Representations...

and ˇ.ŒŒz.g.1/ /; g; g.1/ ; g.1/ /  ˇ.Œz.g.1/ /; z.g.1/ /; g.1/ / D f0g:

(7.3.34)

Combining (7.3.33) with (7.3.34), we find ŒŒz.g.1/ /; g; g.1/  D f0g; i:e: Œz.g.1/ /; g  z.g.1/ /: Hence z.g.1/ / is an ideal of g. Now we show that the ideal z.g.1/ / is invariant under j . If z 2 z.g.1/ /, then j z 2 g.0/ . From the integrability condition, it follows that the operator A D adg.1/ j z commutes with j . Since ˇ is closed and that g.1/ is commutative, it follows that A is skew-symmetric with respect to ˇ. Therefore the operator A is skew-symmetric with respect to the scalar product S and since it has no nonzero purely imaginary eigenvalue, we have A D 0. This means that j z 2 z.g.1/ /, and the lemma is proved.  Now we can prove Proposition 7.3.14. We denote by N the centre of z.g.1/ /. This is a commutative ideal of g and is j -invariant by Lemma 7.3.8 applied to the Kähler algebra z.g.1/ /. Since N  g.1/ , Proposition 7.3.14 has been proved under the induction hypothesis of Theorem 7.3.2. Combining Propositions 7.3.13 and 7.3.14, Proposition 7.3.10 has been proved. Proof of Theorem 7.3.2. Let N be a Kähler ideal of g satisfying the conditions of either Proposition 7.3.13 or Proposition 7.3.14. Let g0 be the orthogonal complement of N . Then g0 is a Kähler subalgebra of g. By the induction hypothesis, g0 can be decomposed into a semi-direct sum g0 D J 0 C H 0 ;

(7.3.35)

where J 0 is a commutative Kähler ideal and H0 is an exponential j -subalgebra. Applying Lemma 7.2.3 on symplectic representations to the Kähler algebra N CJ 0 , we see that ŒN ; J 0  D f0g:

(7.3.36)

We consider separately the two cases corresponding the two possible types of the ideal N . (a) N is an elementary Kähler ideal. We set J D J 0 ; H D N C H0 and show that H is an exponential j -algebra. Since N and H0 are j -algebras, there are !1 2 N  and !2 2 .H0 / such that ˇ.x; y/ D !1 .Œx; y/ for all x; y 2 N ; ˇ.u; v/ D !2 .Œu; v/ for all u; v 2 H0 :

(7.3.37) (7.3.38)

7.4 Structure of Positive Polarizations

243

If we set ! D !1 C !2 2 H , then ˇ.p; q/ D !.Œp; q/ for all p; q 2 H: In fact, we write p D x C u; q D y C v with x; y 2 N and u; v 2 H0 . Note that N D Rjr C Rr C U; ŒRjr C Rr; g0  D f0g; ŒU; g0   U (cf. Proposition 7.3.13). Moreover, we have !1 jU D 0 by the proof of Proposition 7.3.13. Thus ˇ.p; q/ D ˇ.x; y/ C ˇ.u; v/ and !.Œp; q/ D !.Œx C u; y C v/ D !1 .Œx; y/ C !1 .Œx; v C Œu; y/ C !2 .Œu; v/ D !1 .Œx; y/ C !2 .Œu; v/: These equalities together with the formulas (7.3.37), (7.3.38) show that ˇ.p; q/ D !.Œp; q/. From (7.3.35) and (7.3.36), it follows that J is an ideal of g. This finishes the proof in case (a). (b) N is a commutative Kähler ideal. We set J D N C J 0 and H D H0 . From (7.3.35) and (7.3.36), it follows that the ideal J is commutative, so that the decomposition g D J C H satisfies the requirements of the theorem. 

7.4 Structure of Positive Polarizations Let G be an exponential solvable Lie group with Lie algebra g, f 2 g and h 2 P C .f; G/. Further let d; e be Lie subalgebras of g defined in Definition 5.1.2 and let b D d \ kerf . Then d; b are ideals of e by Lemma 7.1.2 and z D d=b is the centre of eQ D e=b of dimension at most 1. Let  W e ! eQ be the natural projection, f0 D f je 2 e ; hQ D .h/ and let fQ 2 eQ  such that fQı D f0 . Let k be a Lie algebra over R. Recall that k is called a Heisenberg algebra if its centre c is of dimension 1 and k=c is commutative. Theorem 7.4.1. eQ can be decomposed into a semi-direct sum eQ D n C m, m being a Lie subalgebra and n being an ideal, satisfying the following conditions. Note that n or m may be f0g. Let h1 D hQ \ nC ; h2 D hQ \ mC ; fQ1 D fQjn 2 n and fQ2 D fQjm 2 m .

244

7 Holomorphically Induced Representations...

(a) n is a Heisenberg algebra with centre z and h1 2 P C .fQ1 ; N / where N D exp n. (b) h2 2 P C .fQ2 ; M / with M D exp m, h2 C h2 D mC and h2 \ m D f0g. We define the linear operator j on m by j.x/ D ix if x 2 h2 and j.x/ D ix if x 2 h2 . Then .m; j; fQ2 / is an exponential j -algebra. (c) m  fQ1 D f0g, where the operation is the coadjoint action. Proof. We know that d is the orthogonal subspace to e relative to Bf , so that Bf induces a non-degenerate alternating form BO f on eO D e=d. eOC may be identified N C where the sum is direct. If we consider eO as the with eC =dC and eOC D h=dC C h=d N C D h=dC . We define j 2 EndOeC by j D i on h=dC real form of eOC , then h=d N C . Then, j maps eO to itself. If we define an alternating form ˇ by and j D i on h=d ˇ.X; Y / D BOf .X; Y / for X; Y 2 eO , .Oe; j; ˇ/ is an exponential Kähler algebra (cf. Section 4 in Chapter I of [3]). Q with Case 1. We assume f jd D 0. Since eQ D e=b D e=d D eO, hQ 2 P C .fQ; E/ EQ D exp.Qe/. hQ C hQ D eQC and hQ \ eQ D f0g, it is easy to see that .Qe; j; fQ/ is an exponential j -algebra. Case 2. Assume f jd ¤ 0. Let Z 2 z be such that fQ.Z/ D 1 and let O W eQ ! eO be the natural projection. By Theorem 7.3.2, eO can be decomposed into a semi-direct sum eO D J CH, where J is a commutative Kähler ideal and H is an exponential j -algebra. For X 2 eO, there uniquely exists an element X 0 2 eQ \ kerfQ such that .X O 0 / D X . We identify eO with kerfQ as vector space by the correspondence X $ X 0 . Now by Theorem 7.2.5, H D H0 ˚ H 1 ˚ H1 ; H0 D a ˚

X

2

H1 D

X

2

Q

1 2 ˛k

; H1 D

k

X

1

 2 .˛m ˛k / ;

m>k



1 2 .˛m C˛k /

mk

dim.˛k / D 1 .1  k; m  r/; where r denotes the rank of H. We take the nonzero element Uk 2 ˛k introduced in Theorem 7.2.5. Then, aD

r X

Rj Uk ; Œj Uk ; Ul  D ıkl Ul ; i.e. ˛l .j Uk / D ıkl :

kD1

Since H is an exponential j -algebra, there exists ! 2 H such that ˇ.X; Y / D fQ.ŒX 0 ; Y 0 / D !.ŒX; Y / for all X; Y 2 H. This shows that !.Uk / D !.Œj Uk ; Uk / D fQ.ŒUk0 ; .j Uk /0 / > 0:

(7.4.1)

7.4 Structure of Positive Polarizations

245

Q h/ Q D f0g, we have Since fQ.Œh; fQ.ŒA0 C i.jA/0 ; X 0 C i.jX /0 / D 0 for A 2 a and X 2 H0 C H 1 . Hence by (7.4.1), 2

!.ŒA; X / D 0 D !.ŒjA; X  C ŒA; jX / D !.j ŒA; X / and ! vanishes on X

 1 1  2 .˛m ˛k / C  2 .˛m C˛k / ˚ H 1 : 2

m>k

Thus, if we set 0 D !jH1 , 0 can be written in the form 0 D ak > 0 .1  k  r/. Let Œ.j Uk /0 ; Ul0  D ıkl Ul0 C bk;l Z. Then

Pr kD1

ak Uk with

bk;l D f .Œ.j Uk /0 ; Ul0 / D !.Œj Uk ; Ul / D ıkl ak .1  k; l  r/:

(7.4.2)

We set r  0 X X 1 m D .H0 / ˚ H 1 ˚ R.Uk0  ak Z/ ˚  2 .˛m C˛k / 0

2

kD1

!0 ;

m>k

where V 0 D fX 0 I X 2 V g for subspaces V of eO. We show that m is a Lie subalgebra of eQ. First, the subspaces H . D 0; 1=2; 1/ are mutually orthogonal with respect to the scalar product S , which is defined on eO by S.X; Y / D ˇ.jX; Y / for X; Y 2 eO (cf. Lemma 7.3.4). So, if we set H100

D

r X

R.Uk0

 ak Z/ ˚

kD1

X

!0 

1 2 .˛m C˛k /

;

m>k

then .H0 /0 ; H100 are Lie subalgebras of eQ and 

 0   0  0 00  .H0 /0 ; H 1  H 1 ; H 1 ; H1 D f0g: 2

2

2

1

Since ! vanishes on  2 .˛m C˛k / .m > k/, 

0   1 0  1 a0 ;  2 .˛m C˛k /   2 .˛m C˛k / ;  1 .˛ ˛ / 0  1 .˛ C˛ / 0   1 .˛ C˛ / 0  2 m p 2 m k ; 2 p k

for m > k; p > k; m ¤ p and

246

7 Holomorphically Induced Representations...

 1 .˛m ˛ / 0  1 .˛ C˛p / 0   1 .˛m C˛p / 0 k ; 2 k 2  2 1

1

for m > k > p. For X 2  2 .˛m ˛k / and Y 2  2 .˛m C˛k / .m > k/, we write ŒX; Y  D

Um0 C ıZ. Similarly to (7.4.2), we get ı D am . Thus ŒX 0 ; Y 0  D .Um0  am Z/. These considerations and relation (7.4.2) show that Œ.H0 /0 ; H100   H100 . Finally, we  0  0  have H 1 ; H 1  H100 . In fact, we see 2

2

 1 ˛m 0  1 ˛ 0   1 .˛m C˛ / 0 k ; Q 2 k   2 .m ¤ k/ Q 2 1

as above. For X; Y 2 Q 2 ˛k , we write ŒX 0 ; Y 0  D ˛Uk0 C Z. Then just like (7.4.2),  0  0   H100 . This finishes the it follows that  D ak ˛, which proves H 1 ; H 1 2 2 verification that m is a Lie subalgebra. Next we set n D J 0 ˚ z. It is clear that n is an ideal of eQ . Since ˇ.jx; x/ > 0 for any nonzero element x 2 J , n has the one-dimensional centre z. Since n=z is commutative, n is a Heisenberg algebra. Finally, the orthogonality of the spaces J and H with respect to the scalar product S (cf. Lemma 7.3.4) means that mf1 D f0g. 

7.5 Non-vanishing of H.f; h; G / A. Siegel domains of type II. Definition 7.5.1. Let ˝ be a convex cone in Rn . Its dual cone ˝  is defined by ˝  D f 2 Rn I h ; yi > 0; 8 y 2 ˝nf0gg; P where h ; yi D niD1 i yi for D . 1 ; 2 ; : : : ; n /; y D .y1 ; y2 ; : : : ; yn / 2 Rn . A convex cone ˝ in Rn such that ˝  ¤ ; is called proper. Definition 7.5.2. Let ˝ be an open proper convex cone in Rn . A real-bilinear form Q W Cq  Cq ! Cn is called a ˝-hermitian form if it has the following properties: (1) (2) (3) (4)

Q is complex-linear in the first variable. Q.u; u0 / D Q.u0 ; u/. Q.u; u/ 2 ˝ for all u 2 Cq . Q.u; u/ D 0 implies u D 0.

Definition 7.5.3. Let ˝ be an open proper convex cone in Rn and Q an ˝hermitian form. The domain D.˝; Q/ D f.x C iy; u/ 2 Cn  Cq I y  Q.u; u/ 2 ˝g

7.5 Non-vanishing of H.f; h; G/

247

is called the Siegel domain of type II associated to .˝; Q/. We shall often write simply D for D.˝; Q/. Let O.D/ represent the space of holomorphic functions on the Siegel domain D.˝; Q/. Let  (resp. ˚) be a positive continuous function on ˝ (resp. Cq /. We set ˚ H.D; ; ˚/ D F 2 O.D/I Z

2 kF k;˚ D jF .z; u/j2  .y  Q.u; u//˚.u/dxdyd u < 1 : D

The following two lemmas can be proved just as in Chapter 2 of [68]. Lemma 7.5.4. H.D; ; ˚/ is a Hilbert space. Lemma 7.5.5. For F 2 H.D; ; ˚/, let Z NF2 .y; u/ D

jF .x C iy; u/j2 dx: Rn

For almost all u, NF2 .y; u/ is finite for all y. Further, for such u, FO . ; u/ D

Z

F .x C iy; u/e i h ;zi dx Rn

is independent of y and Z 1 FO . ; u/e i h ;zi d ; .2/n Rn Z 1 2 NF .y; u/ D jFO . ; u/j2 e 2h ;yi d ; .2/n Rn  Z Z 1 2 2h ;Q.u;u/i O D jF . ; u/j e ˚.u/d u I . /d ; .2/n Rn Cq F .z; u/ D

kF k2;˚ where

Z

e 2h ;t i  .t/dt:

I . / D ˝ 2

Let ˚m .u/ D e mjuj D e mhu;ui .u 2 Cq /, where m is some positive real number. Let ˝O  be the set of 2 Rn such that Z I . / D ˝

e 2h ;t i  .t/dt < 1;

248

7 Holomorphically Induced Representations...

Q and let ˝O ˚m be the set of 2 Rn such that 2h ; Q.u; u/i mjuj2 is a positive definite form on Cq . Q Lemma 7.5.6. If ˝O  \ ˝O ˚m has interior points, then H.D; ; ˚m / ¤ f0g. Q Proof. Let B be a closed ball contained in ˝O  \ ˝O ˚m , and let K D maxfI . /I 2 Bg whose existence follows from the continuity of I on ˝O  (cf. Chapter 2 of [68]). For 2 B, 2h ; Q.u; u/i  mjuj2 is a positive definite form on Cq and the mapping

7! 2h ; Q.u; u/i  mjuj2 is continuous. Thus, there is a positive number k such that 2h ; Q.u; u/i  mjuj2  kjuj2 for all u 2 Cq and 2 B. Now

F .z; u/ D

Z

1 .2/n=2

is an entire function on Cn  Cq and Z Z kF k2;˚m D B

Cq

e i h ;zi d B

2 e 2h ;Q.u;u/iCmjuj d u I . /d Z

 Kvol.B/

e kjuj d u < 1: 2

Cq



B. The non-vanishing of H.f; h; G/. Let G be an exponential solvable Lie group with Lie algebra g, f 2 g and h 2 P C .f; G/. We keep the notations used in the previous sections. Let E D exp e; EQ D exp.Qe/ and  0 W E ! EQ the natural projection. Then Q E/ Q H.f0 ; h; E/ D H.fQ; h; Q ı 0 h 2 P C .f0 ; E/; hQ 2 P C .fQ; E/; and .f; h; G/ D indG E .f0 ; h; E/, so that H.f; h; G/ ¤ f0g if and only if Q E/ Q ¤ f0g. Therefore, as for the non-vanishing of H.f; h; G/, we can H.fQ; h; confine ourselves to the consideration of the following two cases by Theorem 7.4.1. The triplet .g; h; f / satisfies: Case I. e D g and d D f0g. Case II. e D g, d is equal to the one-dimensional centre z of g and f jz ¤ 0. We separately examine these two cases. B1 : Case I. Theorem 7.2.5 enables us to generalize to exponential groups, without any modification, the method and the results of [68] for completely solvable Lie groups. For details of the following observations, see Section 5 in Chapter 2 of [60] and Chapter 4 of [68]. We employ the notations of Theorem 7.2.5 by which we can realize G as a transitive group of affine automorphisms on a Siegel domain of type II as follows. Let Gk D exp.gk / .k D 0; 1=2; 1/ and ˝ the orbit of s in g1 under the adjoint

7.5 Non-vanishing of H.f; h; G/

249

representation of G0 . Then ˝ is an open proper convex cone in g1 . Next, we consider the space g 1 with complex structure j jg 1 and define the form Q W g 1  g 1 ! 2 2 2 2 g1 C i g1 by Q.U; V / D

1 .ŒjV; U   i ŒV; U /: 4

Then Q is an ˝-hermitian form. Thus we can construct the Siegel domain of type II: D.˝; Q/ D f.X C iY; U /I X; Y 2 g1 ; U 2 g 1 ; Y  Q.U; U / 2 ˝g: 2

The group G D G0 G 1 G1 acts on the domain D.˝; Q/ by the following rules: 2

(i) R.h0 / W .z; u/ 7! .h0 z; h0 u/; h0 2 G0 , (ii) R.exp.u0 // W .z; u/ 7! .z C 2iQ.u; u0/ C iQ.u0; u0 /; u C u0 /; u0 2 g 1 , 2 (iii) R.exp.x1 // W .z; u/ 7! .z C x1 ; u/; x1 2 g1 . The point .i s; 0/ is in D.˝; Q/ and the map ˛ from G to D.˝; Q/ defined by ˛ W h0 exp uexp x 7! .h0 .x C i s C iQ.u; u//; h0u/ is a bijection of G onto D.˝; Q/. A function  2 C 1 .G/ is holomorphic on D.˝; Q/ if and only if .X C ijX / D 0 for all X 2 g. These facts imply that the non-vanishing of the space H.f; h; G/ is equivalent to the existence of a nonzero holomorphic function on D.˝; Q/ which belongs to the L2 -space with respect to some Radon measure. 1

1

Definition 7.5.7. Let ˛k ;  2 .˛m ˛k / and Q 2 ˛k be as defined in Theorem 7.2.5. Let Lk D

X

1

 2 .˛m ˛k / ; L0k D

m>k

X

1

 2 .˛k ˛i / ;

k>i 1

pk D dim.L0k /; qk D dim.Lk /; rk D dim.Q 2 ˛k /: The following two theorems can be proved in a way similar to that used in [68]. Theorem 7.5.8. H.f; h; G/ ¤ f0g if and only if   1 2fk  pk C 1 C .qk C rk / > 0 .1  k  r/; 2 where fk D f .Uk / and r denotes the rank of g. Theorem 7.5.9. Suppose H.f; h; G/ ¤ f0g. Then .f; h; G/ ' .Gf O /, the irreducible unitary representation of G corresponding to the coadjoint orbit Gf . B2 : Case II. In this case, we have from Theorem 7.4.1 the following situation. The Lie algebra g is decomposed into a semi-direct sum g D n C m, where n is an ideal and m is a Lie subalgebra. We set N D exp n and M D exp m. Thus G D N Ì M (semi-direct product). Let

250

7 Holomorphically Induced Representations...

fQ1 D f jn 2 n ; fQ2 D f jm 2 m ; h1 D h \ nC ; h2 D h \ mC and let d1 ; e1 (resp. d2 ; e2 ) be the Lie subalgebras of n (resp. m) defined in Definition 5.1.2. Then, h1 2 P C .fQ1 ; N /; h2 2 P C .fQ2 ; M /; d1 D z; e1 D n; d2 D f0g; e2 D m; h D h1 C h2 : Furthermore, n is a Heisenberg algebra with centre z and .m; j; fQ2 / is an exponential j -algebra with j defined in Theorem 7.4.1 and the triplet .m; h2 ; fQ2 / satisfies the condition of Case I. So the notions, the notations and the results in B1 can be transferred to m, which we shall do without explicit mentioning. For example, the numbers pk ; qk and rk are defined as in Definition 7.5.7. Let W D kerfQ1  n. Since m  fQ1 D 0, W is invariant under adn m. If we regard W as commutative Lie algebra, then the semi-direct sum W C m has the structure of an exponential Kähler algebra. Lemma 7.5.10. The adjoint representation adWC a of the commutative subalgebra a  m on WC is diagonalizable and WC can be decomposed into root spaces .WC /

with roots of the form .A/ D ˙ 12 ˛k .A/.1 C i k;l / .A 2 a/ with k;l 2 R or

D 0 .not all possibilities need occur/. We put, for 1  k  r, 1 2 ˛k

WQ C

0

X

D

.WC / @resp.

D 12 .1Ci k;l /˛k

WQ

1 2 ˛k

1

D WQ C2

˛k

1˛ WQ C 2 k

X

D

1 .WC / A ;

D 12 .1Ci k;l /˛k

  1 1˛ \ W resp. WQ  2 ˛k D WQ C 2 k \ W :

Then W can be decomposed into the direct sum W D

X

1 WQ  2 ˛k C W0 C

k

X

WQ

1 2 ˛k

;

k

where W0 D fX 2 W I ŒA; X  D 0; 8 A 2 ag. 1 Note that j WQ  2 ˛k D WQ

1 2 ˛k

and that W D W 1 ˚ W0 ˚ W 1 gives the 2 P P 21 1 decomposition in Lemma 7.3.3 with W 1 D k WQ  2 ˛k ; W 1 D k WQ 2 ˛k . 2

2

Proof. We assume that there exist two elements X; Y in W not simultaneously zero and two numbers k; l .1  k; l  r/ such that 1 1 Œj Uk ; X  D  .X  Y /; Œj Uk ; Y  D  .Y C X / 2 2

7.5 Non-vanishing of H.f; h; G/

Œj Ul ; X  D

251

1 1 .X  Y /; Œj Ul ; Y  D .Y C X /; 2 2

with  2 R. To prove the lemma it is enough to show that this assumption gives rise to a contradiction. In fact, we have ŒUk ; X  D jX; ŒUl ; X  D 0 by Lemma 7.2.4. We consider the adjoint representation of the elementary Kähler algebra Rj.Uk CUl /˚ R.Uk C Ul / of dimension 2. Then Œj.Uk C Ul /; X  D 0 so that ŒUk C Ul ; X  D 0 since the representation is also symplectic. Thus we have jX D 0 and X D 0. Similarly Y D 0, which is impossible.  Definition 7.5.11. We set 1 tk D dim.WQ  2 ˛k / D dim.WQ

1 2 ˛k

m D dim.W 1 / D dim.W 1 / D 2

/ .1  k  r/;

r X

2

tk

kD1

and n D 12 dim.W0 /. The space W possesses the inner product S : S.X; Y /Df .ŒX; jY / for 1 X; Y 2 W . As in Lemma 7.3.4 it is found that the subspaces W0 ; WQ ˙ 2 ˛k .1  k  r/ are mutually orthogonal with respect to S . This fact together with the proof of Theorem 7.2.4 enables us to take a basis fP1 ; P2 ; : : : ; PmCn ; Q1 ; Q2 ; : : : ; QmCng with the following properties. Let C be the nonzero element of z such that f .C / D 1. Then, jPk D Qk ; ŒPk ; Ql  D ıkl C; ŒPk ; Pl  D ŒQk ; Ql  D 0 for 1  k; l  m C n. 1 WQ  2 ˛k D hPik1 C1 ; Pik1 C2 ; : : : ; Pik iR

with i0 D 0; ir D m; ik1  ik .1  k  r/ and W0 D hPmC1 ; PmC2 ; : : : ; PmCn ; QmC1 ; QmC2 ; : : : QmCn iR : So WQ

1 2 ˛k

D hQik1 C1 ; Qik1 C2 ; : : : ; Qik iR

and tk D ik  ik1 .1  k  r/. Moreover,

252

7 Holomorphically Induced Representations...

1 1 Œj Uk ; P  D  P ; Œj Uk ; Q  D Q .ik1 C 1   ik0 /; 2 2 1 1 Œj Uk ; P  D  .P  P Cik00 /; Œj Uk ; P Cik00  D  .P Cik00 C P /; 2 2 1 Œj Uk ; Q  D .Q C Q Cik00 /; 2 1 Œj Uk ; Q Cik00  D .Q Cik00  Q / .ik0 C 1   ik0 C ik00 / 2 with ik1  ik0  ik0 C ik00 so that ik D ik0 C 2ik00 ; tk D ik0  ik1 C 2ik00 .1  k  r/: Finally, hD

mCn X

C.Pk C iQk / C CC C h2 :

kD1

We set Z D exp z. Proposition 7.5.12. Let G be an exponential solvable Lie group with Lie algebra g, f 2 g and h 2 P C .f; G/. We set N \g d D h \ g; b D d \ kerf; e D .h C h/ and E D exp e. If Ef is closed in g , eQ D e=b is a Heisenberg algebra with centre z D d=b so that H.f; h; G/ ¤ f0g. In this case, .f; h; G/ is irreducible and .f; h; G/ ' .Gf O /. In particular, The equivalence class of .f; h; G/ is independent of such h. Proof. By Theorem 7.1.1, it suffices to show that eQ is a Heisenberg algebra. Let B D exp b; EQ D E=B; eQ D n C m be the semi-direct decomposition given in Theorem 7.4.1, N D exp n and M D exp m. Then, EQ is decomposed into the semidirect product EQ D N Ì M . Let fQ 2 eQ be the linear form induced by f and O.fQ/ the coadjoint orbit of EQ through fQ. Then we have O.fQ/ D f` 2 eQ I `jz D f jz g:

(7.5.1)

Suppose that m ¤ f0g and denote by s the principal idempotent in m. Then, .hfQ/.s/ < 0 and .hfQ/jn D fQjn for h 2 M . For any a 2 N , we write a1 D exp X; X D X1 C X2 C X3 C X4

7.5 Non-vanishing of H.f; h; G/

253

with X1 2 W 1 ; X2 2 W0 ; X3 2 W 1 ; X4 2 z: 2

2

Then, .adX /.s/ D ŒX1 C X2 C X3 C X4 ; s D jX1 ; .adX /2 .s/ D ŒX1 ; jX1  2 z; .adX /k .s/ D 0 .k  3/: It follows that 1 ..ah/  fQ/.s/ D .h  fQ/.a1 s/ D .h  fQ/.s  jX1  ŒX1 ; jX1 / 2 1 D .h  fQ/.s/  f .ŒX1 ; jX1 / < 0: 2 Since h 2 M; a 2 N are arbitrary, this contradicts (7.5.1).



By the integrability of j , j ŒX; Y   j ŒjX; Y  D ŒjX; Y  C ŒX; jY  for any X 2 m; Y 2 W . In particular, j ŒX; Y  D ŒjX; Y 

(7.5.2)

for X 2  2 .˛k ˛l / ; Y 2 WQ  2 ˛k and for X 2 Q 2 ˛k ; Y 2 WQ  2 ˛k . Also we have 1

1

1

1

ŒjX; jY  D ŒX; Y 

(7.5.3)

1 1 1 for X 2  2 .˛k C˛l / .k > l/; Y 2 WQ  2 ˛l and for X 2 Q 2 ˛k ; Y 2 W0 . Let V be an 1 1 element in  2 .˛k ˛l / or in Q 2 ˛k such that ŒjV; V  D 2Uk . Then by (7.5.2) and (7.5.3), 1 we have for P 2 WQ  2 ˛k

ŒV; ŒjV; P  D ŒV; j ŒV; P  D ŒjV; ŒV; P  and 2jP D 2ŒUk ; P  D ŒŒjV; V ; P  D ŒjV; ŒV; P   ŒV; ŒjV; P : Thus ŒjV; ŒV; P  D ŒV; ŒjV; P  D jP

254

7 Holomorphically Induced Representations...

and Bf .ŒV; P ; j ŒV; P / D Bf .ŒV; P ; ŒjV; P / D Bf .ŒŒV; P ; jV ; P / D Bf .ŒŒV; jV ; P ; P / C Bf .ŒV; ŒP; jV ; P / D 2Bf .jP; P /  Bf .P; jP / D Bf .P; jP /:

(7.5.4)

1

The spaces Q 2 ˛k are mutually orthogonal with respect to S . By considering an 1 orthogonal basis in each space Q 2 ˛k , we can take a basis n

 o  1 rk0 D rk k 2

V1k ; V2k ; : : : ; Vrk0 ; jV1k ; jV2k ; : : : ; jVrk0 k

1

in Q 2 ˛k such that ŒjVlk ; Vlk  D ŒVlk ; jVlk  D 2Uk .1  l  rk0 / 1

and all the other brackets are equal to zero. Let fEk;l g be the basis of  2 .˛k ˛l / p

1

introduced in [68]: that is, we consider the scalar product Q0 on  2 .˛k ˛l / defined p by Q0 .L; L0 /Uk D ŒL; ŒL0 ; Ul  and choose fEk;l g as an orthonormal basis relative to 12 Q0 . We can write il X

p

ŒEk;l ; P  D

p;

ck;l; P .ik1 C 1   ik /;

(7.5.5)

 Dil1 C1

ŒVk ; P  D

mCn X

k;l k;l .d; Pl C e; Ql / .ik1 C 1   ik /:

(7.5.6)

lDmC1 p

p

Since Bf .ŒEk;l ; P ; Q / D Bf .ŒEk;l ; Q ; P / for ik1 C1   ik and il1 C1    il , we have p

ŒEk;l ; Q  D 

ik X

p;

ck;l; Q :

Dik1 C1

By (7.5.2) and (7.5.3), p

ŒjEk;l ; P  D

il X  Dil1 C1

p;

ck;l; Q .ik1 C 1   ik /;

7.5 Non-vanishing of H.f; h; G/

p

ŒjEk;l ; P  D

255

ik X

p;

ck;l; Q .il1 C 1    il /;

Dik1 C1

ŒjVk ; P 

D

mCn X

k;l k;l .e; Pl C d; Ql / .ik1 C 1   ik /:

lDmC1

For ik1 C 1   ik and m C 1  l  m C n, we have k;l Bf .P ; ŒVk ; Pl / D Bf .ŒP ; Vk ; Pl / D e; ; k;l Bf .P ; ŒVk ; Ql / D Bf .ŒP ; Vk ; Ql / D d; :

Hence ik X

ŒVk ; Pl  D

ik X

k;l e; Q ; ŒVk ; Ql  D 

Dik1 C1

k;l d; Q :

Dik1 C1

By (7.5.3), ŒjVk ; Pl  D

ik X

ik X

k;l d; Q ; ŒjVk ; Ql  D

Dik1 C1

k;l e; Q :

Dik1 C1

Recall that the space H.f; h; G/ consists of all C 1 -functions  on G which satisfy the following two conditions: X D if .X / .8 X 2 h/;

(7.5.7)

Z kk2 D

j.g/jd gP < C1;

(7.5.8)

G=Z

where d gP denotes an invariant measure on G=Z. Each element g 2 G D M N can be uniquely written in the form g D ha with h 2 M; a 2 N and a can be uniquely written in the form a D exp

mCn X

.xi Pi C yi Qi /exp.wC /:

i D1

Through this expression, we regard  2 C 1 .G/ as a C 1 -function relative to variables .h; x1 ; y1 ; x2 ; y2 ; : : : ; xmCn ; ymCn ; w/ 2 M  R2.mCn/C1:

256

7 Holomorphically Induced Representations...

We calculate the condition (7.5.7) for X 2 h2 D h \ mC . For X; Y 2 m and  2 C 1 .G/, we set .  X /.h; x1 ; y1 ; : : : ; xmCn ; ymCn ; w/ M

ˇ ˇ d D .hexp.tX /; x1 ; y1 ; : : : ; xmCn ; ymCn ; w/ˇˇ ; dt t D0   .X C iY / D   X C i   Y: M

M

M

We have the following relations:   .Uk C ij Uk / D   .Uk C ij Uk / M



ik0

X



x

Dik1 C1

"

ik0 Cik00



  i @ @ @ x   y @y 2 @x @y

X

Dik0 C1

 i @ @ @ 00 C x Cik  x .x C x Cik00 / @y @y Cik00 2 @x

C .x Cik00  x /

@ @x Cik00

@ @ .y  y Cik00 /  .y Cik00 C y / @y @y Cik00 p

p

p

)# :

(7.5.9)

p

.Ek;l C ijEk;l / D   .Ek;l C ijEk;l / il X



0

ik X

@

 Dil1 C1

i

M

ik X Dik1 CŠ

ck;l; x A p;

Dik1 C1

8 il < X :

 Dil1

1



@ @ Ci @x @y

9 = @ p; ck;l; .x C iy / : ; @y C1



(7.5.10)

  .Vk C ijVk / D   .Vk C ijVk / M



mCn X

ik X

lDmC1 Dik1

   @ @ @ k;l k;l : .e; C id; / .xl C iyl /  ix Ci @y @xl @yl C1 (7.5.11)

7.5 Non-vanishing of H.f; h; G/

257

We define 0 2 C 1 .G/ by 1

0 .g/ D e i w e  4

PmCn i D1

.xi2 Cyi2 /

P when g D hexp imCn D1 .xi Pi C yi Qi /exp.wC /. Then 0 satisfies the infinitesimal condition (7.5.7) and the following relations: 9 8 ik = < i X 0  .Uk C ij Uk / D  .x C iy /2 0 ; ; : 4 Di C1

(7.5.12)

k1

p

p

0  .Ek;l C ijEk;l / D

8 <1 :2

ik X

il X

Dik1 C1  Dil1

8 < i mCn X 0  .Vk C ijVk / D  : 2

ik X

p;

ck;l; .x C iy /.x C iy / 0 ; ;

k;l .e;

lDmC1 Dik1 C1

9 =

(7.5.13) 9 = k;l C id; /.x C iy /.xl C iyl / 0 : ; (7.5.14)

for any  2 H.f; h; G/, then O is independent of Therefore, if we set O D w and is a C 1 -function on M  CmCn with zk D xk C iyk .1  k  m C n/. Furthermore, O is holomorphic relative to the variables zk .1  k  m C n/ and the above relations (7.5.9)–(7.5.11) imply the following relations: 01

0

ik @O i X z O  .Uk C ij Uk /  M 2 Di C1 @x k1

ik0 Cik00

i X  2 0

Dik C1

(

@O @O .z  z Cik00 / C .z Cik00 C z / @x @x Cik00

)

0

1 ik X i O z2 A ; D i fQ2 .Uk C ij Uk /O C @ 4 Di C1

(7.5.15)

k1

ik X

p p O  .Ek;l C ijEk;l / C M

0

il X

Dik1 C1  Dil1 C1 ik X

1 D @ 2 Di

il X

k1 C1  Dil1 C1

p;

ck;l; z

@O @x

1

O ck;l; z z A ; p;

(7.5.16)

258

7 Holomorphically Induced Representations... mCn X

O  .Vk C ijVk /  i M

ik X

k;l k;l .e; C id; /zl

lDmC1 Dik1 C1

8 mCn i < X D 2:

ik X

k;l k;l .e; C id; /z zl

lDmC1 Dik1 C1

9 = ;

@O @x

O :

(7.5.17)

As for the infinitesimal conditions on the space H.fQ2 ; h2 ; M /, Rossi and Vergne [68] gave a particular solution: let  W m ! h2 be the linear map defined by 8 ˆ X 2 m0 ; ˆ <X C ijX; 1 .X / D 2 .X C ijX /; X 2 m 1 ; 2 ˆ ˆ :0 X 2 m1 ; and let fQ2 2 C 1 .M / be defined by Q

fQ2 .exp X / D e i f2 ..X // .X 2 m/: Then 1 satisfies the infinitesimal conditions which the space H.fQ2 ; h2 ; M / fQ2 requires. P Lemma 7.5.13. Let g D ha .h 2 M; a 2 N / with a D exp imCn D1 .xi Pi C yi Qi /exp.wC /. (1) The C 1 -function i w e 1 .g/ D f1 Q .h/e

  P 2  12 jmCn D1 .xj Cixj yj /

2

satisfies the infinitesimal condition (7.5.7). (2) If M 2 C 1 .M / satisfies the infinitesimal conditions imposed on the space H.fQ2 ; h2 ; M /; the function .g/ D

1

 i w

 12

PmCn j D1

.xj2 Cixj yj /

M .h/e e .2/.mCn/=2 Z P  P  mCn mCn B h1  j D1 j Pj ; j D1 zj Qj  e f d



(7.5.18)

B

satisfies the infinitesimal condition (7.5.7), where d denotes a Lebesgue measure on RmCn , B a compact set in RmCn and zj D xj C iyj .1  j  m C n/.

7.5 Non-vanishing of H.f; h; G/

259

Proof. (1) Simple calculations show that the function 

2 .g/ D f1 Q .h/e

 14

PmCn

z2j

j D1

2



;

where zj D xj C iyj .1  j  m C n/, satisfies the conditions (7.5.15)– (7.5.17). It follows that the function 1 D 0 2 satisfies the infinitesimal condition (7.5.7). (2) The function Z P  P  mCn mCn Bf h1  j D1 j Pj ; j D1 zj Qj e d 3 .g/ D B

is clearly holomorphic in zj .1  j  m C n/. Thus, it suffices to show that the function  2 C 1 .G/ given by 0

0

 .g/ D Bf @h1  @

mCn X

1

j Pj A ;

j D1

mCn X

1 zj Qj A

j D1

satisfies the condition  X D 0 .8 X 2 h2 /:

(7.5.19)

In fact, since  is holomorphic in zj .1  j  m C n/, the relations (7.5.9)– (7.5.11) imply at first that  X .X 2 h2 / is a homogeneous polynomial in z1 ; z2 ; : : : ; zmCn of degree at most 1 and then give us the desired equation (7.5.19) by direct computations.  Now we estimate kk2 for  given by (7.5.18). We denote by dh a left-invariant measure on M , then d gP D dhdxdy where dx; dy denote Lebesgue measures on Rm ; Rn . We set q D hQ1 ; Q2 ; : : : ; QmCn iR , which is invariant under the adjoint representation AdW M of M on W . For any h 2 M , we set AdW =q h D hO and 0 h1  @

mCn X

1

j Pj A

j D1

mCn X

j0 Pj mod q:

j D1

By Plancherel formula Z

Z

kk D

Z

j.g/j d gP D

2

jM .h/j

2

G=Z

Z  R

M

2

e RmCn

 P  2  jmCn D1 xj

ˇ ˇ2 Z PmCn  ˇ ˇ 0 1 j D1 j zj ˇ ˇ dhdxdy e d

ˇ ˇ .mCn/=2 mCn .2/ B

260

7 Holomorphically Induced Representations...

Z

 2 Z jM .h/j2 det hO

D M

e

 P  2  jmCn D1 xj

RmCn

ˇ ˇ2 Z P  mCn ˇ ˇ 1 j D1 j zj ˇ ˇ dydxdh  e d

ˇ ˇ .mCn/=2 O1 RmCn .2/ h B Z  P Z  P   2 Z 2  jmCn 2 jmCn D1 xj D1 xj yj D jM .h/j2 det hO e e dydxdh Z

M

RmCn

Z

 2 Z 2 O det h jM .h/j

D .mCn/=2 Z

P hO1 B

e

hO1 B 

mCn 2 j D1 yj

dydh

M

  Z PmCn 0 2  .y / 2 O jM .h/j det h e j D1 j dydh:

M

B

D .mCn/=2

(7.5.20)

If we write h D h0 exp U h1 with h0 2 M0 ; U 2 m 1 ; h1 2 M1 , then det hO D 2 detW 1 h0 and 2

0 h1  @

mCn X

1

0

@ yj Pj A exp.U /h1 0 

j D1

mCn X

1 yj Pj A

j D1

0 @ h1 0 exp.h0 U /

mCn X

1 yj Pj A mod q:

j D1

Using the expressions h0 D

r Y

! exp.ai j Ui / exp.L1 /exp.L2 /  exp.Lr1 /; Li D

i D1

XX k>i

we calculate yj0 and estimate

PmCn

p

p

 XX uk Vk C vk jVk ; h0 U D k

p

xk;i Ek;i ; (7.5.21)



0 2 j D1 .yj / .

For fixed k .1  k  r/, we have

y0 D e ak =2 y  Ik; when ik1    ik0 , and   a   a   k  k C y Cik00 sin  Ik; y0 D e ak =2 y cos 2 2   a   a   k  k  y sin  Ik; Cik00 y0 Ci 00 D e ak =2 y Cik00 cos k 2 2

7.5 Non-vanishing of H.f; h; G/

261

when ik0 C 1    ik0 C ik00 . Here, Ik; D

XX

2

i0

j X

4

Dij 1 C1

j >k p ij0 Cij00

X n

C

p;

cj;k; y

Dij0 C1

  a   a  j j p; cj;k; y cos C y Cij00 sin 2 2

 a   a oi  j j p; p  y sin e aj =2 xj;k : C cj;k; Ci 00 y Cij00 cos j 2 2 Further 0

yl0

D yl 

rk r X X

ik X

  k;l k;l y uk d;  vk e;

kD1 D1 Dik1 C1 1

when mC1  l  mCn. Since Ek;l 2  2 .˛k ˛l / satisfies the relation ŒjEk;l ; Ek;l  D 2Uk , we have p

ik X

p;

.cj;k; /2 D 1;

 Dik1 C1

mCn X

p

n

p

o k;l 2 k;l 2 .d; / C .e; / D1

lDmC1

by (7.5.4)–(7.5.6). For fixed k .1  k  r/ and  .ik1 C 1    ik /, we set IOk; D

XX

2

i0

j X

4

p;

.cj;k; /2 .y /2

Dij 1 C1

j >k p

  a   a 2 X  p; j j C y Cij00 sin .cj;k; /2 y cos 2 2 0

ij0 Cij00

C

Dij C1

  a   a 2  j j p; p  y sin e aj .xj;k /2 : C .cj;k; Ci 00 /2 y Cij00 cos j 2 2 Then, mCn X

.yj0 /2

j D1

2  c4

r X kD1

2 4

0

ik X  Dik1 C1

  e ak .y /2 C IOk;

262

7 Holomorphically Induced Representations...

X 

ik0 Cik00

C

e ak

 y cos

 Dik0 C1

 a   a 2  k  k C y Cik00 sin 2 2

   a   a 2  k  k O O  y sin C Ik; C Ik; Cik00 C y Cik00 cos 2 2 33 2 rk0 ik mCn r X   X X X k;l 2 k;l 2 55 4.yl /2 C C .y /2 .uk /2 .d; / C .vk /2 .e; / kD1 D1 Dik1 C1

lDmC1

2 0 1 2 i0 2 j ik r XX X X X 4e ak @ 4 .y /2 A C .y /2 c4  Dik1 C1

kD1

Dij 1 C1

j >k p

 a   a 2 X  j j C y Cij00 sin y cos 2 2 0

ij0 Cij00

C

Dij C1

   a   a 2  j j p  y sin e aj .xj;k /2 C y Cij00 cos 2 2 3 0 1 0 rk ik mCn r X   X X X @ C .yl /2 C .y /2 A .uk /2 C .vk /2 5 Dik1 C1

kD1 D1

lDmC1

8 0 2 1 0 ij ik r < XX X X X @ Dc4 .y /2 A C e ak @ :  Dik1 C1

kD1

C

mCn X

0

.yl /2 C

lDmC1

rk r X X kD1 D1

j >k p

0 @

ik X

1

1 .y /2 A e aj .xj;k /2 p

Dij 1 C1

3   .y /2 A .uk /2 C .vk /2 5 ;

Dik1 C1

where c denotes some positive constant. Therefore, if we take mCn X

˚ . j /2  c 1 ; B D 2 RmCn I j D1

we obtain by (7.5.20) Z



kk2  .mCn/=2 M



jM .h/j2 detW 1 h0 e 2

Pr

(

a kD1 e k 1C

)

P

p

2 j
9 = ;

7.5 Non-vanishing of H.f; h; G/

 e

Pr kD1

Prk0

D1

263

..uk /2 C.vk /2 / dh Z

Pr

e

kD1

P

edy

  jM .h/j2 detW 1 h0 2

M

eak 1C



B

D e .mCn/=2 vol.B/ (

Z

)

p 2 j
e

Pr kD1

Prk0

D1

..uk /2 C.vk /2 / dh:

(7.5.22)

As stated in subsection A, the subgroup M D exp m can be identified with the Siegel domain D.˝; Q/ which was constructed in A under the mapping ˛ W h 7! h.i s; 0/. Then M can be written in the form M .h/ D f1 Q .h/F .˛.h//; 2

(7.5.23)

where F is a holomorphic function on D.˝; Q/. Under the identification ˛, a Haar measure on M is given by   1 2 detm1 ˛ 1 .y  Q.u; u// dxdyd u; dh D detm 1 ˛ 1 .y  Q.u; u// 2

where detmi h .h 2 M0 ; i D 1=2; 1/ denotes the determinant of the adjoint representation. Let d D dim.m1 /; q D 12 dim.m 1 /. We define a positive continuous 2 function 0 on ˝ D M0 s by Pr kD1

( eak 1C

P

) p 2 j
0 .h0 s/ D e jfQ2 .h0 /j2  1   detW 1 h0 detm 1 h0 .detm1 h0 /2 2

2

for h0 2 M0 . Further we set ˚1 .u/ D e juj

2

for u 2 Cq . Then by (7.5.22) Z kk2 

jF .z; u/j2 0 .y  Q.u; u//˚1 .u/dxdyd u:

(7.5.24)

D

Therefore, if there exists a nonzero holomorphic function F0 which makes the righthand side of (7.5.24) finite, we can construct a nonzero element  2 H.f; h; G/ by means of (7.5.18) and (7.5.23). For the existence of such P F0 , it suffices by Q Lemma 7.5.6 that ˝O 0 \ ˝O ˚1 has interior points. Let 0 D rkD1 Uk 2 ˝  and let 1 be the positive continuous function on ˝ given by

264

7 Holomorphically Induced Representations...

  1 1 .h0 s/ D jfQ2 .h0 /j2 detW 1 h0 detm 1 h0 .detm1 h0 /2 : 2

(7.5.25)

2

Then, under expression (7.5.21) of h0 2 M0 , " Pr

(

ak 1C kD1 e

0 .h0 s/ D e

)#

P

p

2 j
1 .h0 s/ D e 0 .h0 s/ 1 .h0 s/

p

(cf. Theorem 4.10 in [68]). Hence Z I0 . / D

e

2 .t /

Z 0 .t/dt D

˝

e

2.  12 0 /.t /

˝

  1 1 .t/dt D I1  0 : 2 (7.5.26)

The calculations similar to those of [68] show that the integral I1 . 0 / is, up to a constant factor, the product of the r-integrals Z

C1

e 2t t Œ2fi fpi C1C 2 .qi Cri Cti /g1 dt; 1

0

where fi D fQ2 .Ui / .1  i  r/. Therefore, if 

1  2fi  pi C 1 C .qi C ri C ti / > 0 .1  i  r/; 2

(7.5.27)

then I1 . 0 / < C1 so that I1 . / < C1 throughout ˝  , since 1 is homogeneous in the sense that 1 .py/ D p  y .0 < p 2 R; y 2 ˝/ for some real number  (cf. Theorem 2.20 in [68]). Thus by (7.5.26), 1

0 C ˝   ˝O 0 ; 2



 1 Q Q 

0 C ˝ \ ˝O ˚1  ˝O 0 \ ˝O ˚1 : 2

Since the left-hand side of the last inclusion relation is an open set which contains Q

D ˛ 0 with ˛ > 1, it follows that ˝O 0 \ ˝O ˚1 has interior points. In conclusion, the inequalities (7.5.27) give a sufficient condition for the nonvanishing of the space H.f; h; G/. Now we prove that the inequalities (7.5.27) are also necessary for the nonvanishing of H.f; h; G/. By Theorem 7.4.1 and our assumption, f 2 g has the form f D C C

r X kD1

fk Uk

7.5 Non-vanishing of H.f; h; G/

265

with fk < 0 .1  k  r/. Let m 0 D m 0 C m 1 ; g0 D n C m 0 ; where the sums are semi-direct. We set f 0 D f jg0 2 .g0 / and h0 D h \ .g0 /C . Then m0 (resp. g0 ) is a Lie subalgebra of m (resp. g) and h0 is a positive polarization of g0 at f 0 , i.e. h0 2 P C .f 0 ; G 0 / with G 0 D exp.g0 /. We set M 0 D exp.m0 / and let  W G ! G=Z be the natural projection. If we write gP D .g/ 2 G=Z in the form gP D .hexp Y / with h 2 M and Y 2 W , then d gP D dhd Y with a Haar measure dh on M and a Lebesgue measure d Y on W . Moreover, h can be uniquely written in 1  d udh0 the form h D exp uh0 with u 2 m 1 and h0 2 M 0 so that dh D detm 1 h0 2

2

with a Lebesgue measure d u on m 1 and a Haar measure dh0 on M 0 . Thus, we have 2

1  d gP D detm 1 h0 d udh0 d Y; 2

when we write g 2 G uniquely in the form g D exp ug 0 ; g 0 D h0 expY exp.wC / with u 2 m 1 ; h0 2 M 0 ; Y 2 W; w 2 R. 2 We suppose H.f; h; G/ ¤ f0g. Then there is a nonzero C 1 -function  on G which satisfies the infinitesimal condition (7.5.7) and Z  1 j.exp ug 0 /j2 detm 1 h0 d ud gP 0 < C1 (7.5.28) 2

G=Z

where d gP 0 D dh0 d Y . By (7.5.28), we have Z  1 j.exp ug 0 /j2 detm 1 h0 d gP 0 < C1 G 0 =Z

(7.5.29)

2

for almost all u 2 m 1 . We take and fix such an element u0 2 m 1 and define a 2 2 nonzero C 1 -function u0 on G 0 by u0 .g 0 / D .exp.u0 /g 0 /. Then u0 satisfies the infinitesimal condition on the space H.f 0 ; h0 ; G 0 /. We write g 0 D h0 exp

mCn X

.xj Pj C yj Qj /exp.wC /

j D1

and define the function

on G 0 by 

0 i w .g 0 / D f1 e Q .h /e 2

Then by virtue of Lemma 7.5.13 H.f 0 ; h0 ; G 0 /. Thus O u0 D u0 

 12

PmCn j D1

.xj2 Cixj yj /



:

satisfies the infinitesimal condition on the space is independent of w and is a C 1 -function on

1

266

7 Holomorphically Induced Representations...

M 0  CmCn , and holomorphic in zj D xj C iyj .1  j  m C n/. Furthermore, by (7.5.15) and (7.5.16) 0

ik @Ou0 i X .O u0 /  .Uk C ij Uk /  z M 2 Di C1 @x k1

0

(

00

i Ci i kXk  2 0

@Ou0 @Ou0 .z  z Cik00 / C .z Cik00 C z / @x @x Cik00

Dik C1

.Ou0 /  .Ek;l C ijEk;l / C p

M

p

ik X

il X

p;

ck;l;

Dik1 C1  Dil1 C1

) D 0;

@Ou0 D 0; @x

(7.5.30)

(7.5.31)

and Z G 0 =Z

 1 jO u0 j2 j j2 detm 1 h0 d gP 0 < C1

(7.5.32)

2

by (7.5.29). We set z D .z1 ; z2 ; : : : ; zmCn / 2 CmCn and define the function Q u0 on M 0  CmCn by Qu0 .h0 ; z/ D O u0 .h0 ; zO/; P PmCn 0 Oj Qj D h1 where zO D .Oz1 ; zO2 ; : : : ; zOmCn / given by jmCn 0 . j D1 zj Qj / for h D D1 z h0 h1 with h0 2 M0 ; h1 2 M1 . Then by (7.5.30) and (7.5.31), it follows that p p .Qu0 /  .Uk C ij Uk / D 0; .Q u0 /  .Ek;l C ijEk;l / D 0: M

M

(7.5.33)

Moreover, it is obvious that Q u0 is holomorphic in zj .1  j  m C n/. Let ˝ D M0 s. Then the Siegel domain corresponding to M 0 is the tube domain D 0 D m1 C i ˝. We denote again by ˛ the identification between M 0 and D 0 . By (7.5.33), the function Fu0 on D 0  CmCn defined by Fu0 .˛.h0 /; z/ D Qu0 .h0 ; z/ is a nonzero holomorphic function. We denote the coordinates in D 0 by z0 D x 0 C iy 0 ; x 0 D .x10 ; x20 ; : : : ; xd0 /; y 0 D .y10 ; y20 ; : : : ; yd0 / with d D dim.m1 /. If we introduce the positive continuous function 2 on ˝ D M0 s by

7.5 Non-vanishing of H.f; h; G/

267

 1  2 2 .h0 s/ D jfQ2 .h0 /j2 detm 1 h0 .detm1 h0 /2 detW 1 h0 .h0 2 M0 /; 2

2

then we obtain by (7.5.32) Z D 0 CmCn

jFu0 .z0 ; z/j2 e

 P  O j /2  jmCn D1 .x

2 .y 0 /dx 0 dy 0 dxdy < C1:

(7.5.34)

By Theorem 2.8 in [68], a nonzero holomorphic function Fu0 which satisfies (7.5.34) exists if and only if the set of 2 RmCn such that Z I. / D

e

2 .t /

e

 P  O j /2  jmCn D1 .x

2 .y 00 /dt < C1

˝RmCn

PmCn has interior points, where t D .y 00 ; x/; y 00 D h0 s and O j Qj D j D1 x P mCn d mCn h1 . x Q /. We write

D . ;

/;

2 R ;

2 R and transform 1 2 1 2 j D1 j j 0 the integral into an integral over M0 : Z

e 2 1 .h0 s/ 2 .h0 s/ .detm1 h0 /

I. / D M0

Z

e 2 2 .x/ e



 P  O j /2  jmCn D1 .x

RmCn

2 .y 00 /dxdh0 :

And Z

e 2 2 .x/ e

 P  O j /2  jmCn D1 .x

RmCn

Z  D detW 1 h0 2

e

dx

  P 2 Q  jmCn D1 .xj / 2 2 .x/

dx;

(7.5.35)

RmCn

P P where xQ D .xQ 1 ; xQ 2 ; : : : ; xQ mCn / with jmCn Q j Qj D h0 . jmCn D1 x D1 xj Qj /, and the integral in the right-hand side of (7.5.35) has, up to a constant factor, the form e .h0 ; 2 / with .h0 ; 2 /  0 for all h0 2 M0 ; 2 2 RmCn . Therefore it follows that if the set f 2 Rd CmCn I I. / < C1g has interior points, then the set of 2 Rd such that Z   I1 . / D e 2 .h0 s/ 2 .h0 s/ .detm1 h0 / detW 1 h0 dh0 Z

M0

2

e 2 .t / 1 .t/dt D I1 . / < C1

D ˝

has interior points, where 1 is the function on ˝ defined by (7.5.25). Again by Theorem 2.20 in [68], ˝O 1 has interior points if and only if I1 . 0 / < C1 for

268

7 Holomorphically Induced Representations...

Pr   O

0 D kD1 Uk 2 ˝ , since 1 is homogeneous. Therefore if ˝1 has interior points, then the condition (7.5.29) is satisfied. B3 : General case. Let G be an exponential solvable Lie group with Lie algebra g, f 2 g and h 2 P C .f; G/. Using the notations introduced at the beginning of Sect. 7.4, we decompose eQ into a semi-direct sum eQ D n C m by Theorem 7.4.1. As the triplet .m; h2 ; fQ2 / satisfies the condition of Case I, Definition 7.5.7 applied to m gives us the numbers pi ; qi ; ri for m. Similarly, the number fi is defined as in Theorem 7.5.8. Define the numbers ti .1  i  r D rank m/ as follows. If n D f0g, put Q fQ/ satisfies the condition of Case ti D 0 .1  i  r/. If n ¤ f0g, the triplet .Qe; h; Q fQ/. II, so we apply Definition 7.5.11 of ti to .Qe; h; Then the considerations in B1 and B2 give us the next theorem which generalizes Theorem 7.5.8. Theorem 7.5.14. H.f; h; G/ ¤ f0g if and only if   1 2fi  pi C 1 C .qi C ri C ti / > 0 .1  i  r/: 2

7.6 Irreducibility and Equivalence of .f; h; G / We generalize the notion of the Pukanszky condition (cf. Definition 5.1.9). Definition 7.6.1 (cf. Proposition 5.1.12). We say that h 2 P C .f; G/ satisfies the N \ g and D D exp d Pukanszky condition if Df D f C e? , where e D .h C h/ with d D h \ g as before. Note that, if h is totally complex, i.e. e D g, then h satisfies the Pukanszky condition. We denote as before by ./ O the irreducible unitary representation of G associated with the coadjoint orbit  2 g =G in the Kirillov–Bernat sense (cf. Definition 5.3.30). Now we prove the following theorem which generalizes Theorems 7.1.1 and 7.5.9. Theorem 7.6.2. Let f 2 g and h 2 P C .f; G/. Suppose H.f; h; G/ ¤ f0g. Then .f; h; G/ is irreducible if and only if h satisfies the Pukanszky condition. In this case, .f; h; G/ ' .Gf O /. In particular, (the equivalence class of) .f; h; G/ is independent of h. Proof (cf. Proof of Theorem 7.1.1). (A) First we prove that .f; h; G/ ' .Gf O / for a positive polarization h satisfying the Pukanszky condition. This claim is trivial when dim G D 1, so we prove it by induction on dim G and assume dim G D n. Case 1. There is an ideal a ¤ f0g of g such that f ja D 0. Let A D exp a; GQ D G=A and  W G ! GQ the canonical projection. Let gQ be the Lie algebra of GQ and d  W gC ! gQ C the differential of . Now we consider the exact sequence of exponential solvable Lie groups:

7.6 Irreducibility and Equivalence of .f; h; G/

269

 1 ! A ! G ! GQ ! 1:

Let fQ 2 gQ  be such that fQıd  D f and let hQ D d .h/. Since h  g.f /  a, Q and gQ  is naturally isomorphic to a?;g , so that hQ it is clear that hQ 2 P C .fQ; G/ satisfies the Pukanszky condition. So by Proposition I.5.13 in [3], Q G/ı Q .fQ; h; ' .f; h; G/:

(7.6.1)

Q G/ Q ¤ f0g. The induction hypothesis implies Hence by our assumption, H.fQ; h; Q Q Q Q Q that .f ; h; G/ ' . O Gf /. That is, there exists hQ 0 2 I.fQ; gQ / such that Q G/ Q ' . Q .fQ; h; O fQ; hQ 0 ; G/:

(7.6.2)

Set h0 D .d /1 .hQ 0 /. Then it is obvious that Q . O fQ; hQ 0 ; G/ı ' .f; O h0 ; G/

(7.6.3)

and h0 2 I.f; g/. From (7.6.1)–(7.6.3), O /: .f; h; G/ ' .f; O h0 ; G/ ' .Gf Case 2. There is no ideal a ¤ f0g of g such that f ja D 0. This case is divided into two subcases. (1) Suppose e ¤ g. We choose and fix one complementary linear subspace c of e in g, and let  W e ! g be the injection such that .l/.x/ D l.y/ for l 2 e and x D y C z with y 2 e; z 2 c. From now on, we identify e with its image .e /, so that e  g . Let  W g ! e be the restriction mapping such that .l/ D l0 D lje for l 2 g . Then je D I (the identity mapping of e ) under the above identification. Now h is clearly a positive polarization of E D exp e at .f / D f0 D f je and satisfies the Pukanszky condition as a polarization of E at f0 since h 2 P C .f0 ; E/ is totally complex. Since .f; h; G/ D indG E .f0 ; h; E/, H.f0 ; h; E/ ¤ f0g provided H.f; h; G/ ¤ f0g. We can apply the induction hypothesis to have .f0 ; h; E/ ' .O.f O 0 //, where O.f0 / is the coadjoint orbit of E in e through f0 . That is to say, there exists h0 2 I.f0 ; e/  M.f0 ; e/ such that O 0 ; h0 ; E/ .f0 ; h; E/ ' .f Since h 2 P C .f0 ; E/, dimR .h0 / D dimC h D Thus h0 2 M.f; g/.

1 .dimR g C dimR .g.f ///: 2

(7.6.4)

270

7 Holomorphically Induced Representations...

We show that h0 2 I.f; g/, i.e. h0 satisfies the Pukanszky condition. For the proof of the next lemma, see Proposition 3.1.5 in [10]. Lemma 7.6.3 (cf. Lemma 5.1.11). Let l 2 g and k 2 S.l; gC / such that g.l/C  N \ g and set P D exp p. Then, the following k. We put p D k \ g; q D .k C k/ conditions are equivalent: (1) P l D l C q? ; (2) l C q?  Gl and k 2 M.l; gC /; (3) k 2 M.l C '; gC / for any ' 2 q? . Now we continue the proof of the theorem. Since e is a Lie subalgebra of g, .Ef / D .Ef /0 D O.f0 /:

(7.6.5)

Let us see that  1 .O.f0 // D Ef . If a 2 E and l 2 e? , we have al 2 e? . For a 2 E, we write af D .af /0 C fO with fO 2 e? . Then we have, for any ` 2 e? , a1 .fO  `/ 2 e? and a.f  a1 .fO  `// D .af /0 C `: Since h satisfies the Pukanszky condition as a polarization of g at f , f  a1 .fO  `/ D bf for some b 2 D  E. Hence .ab/f D .af /0 C `:

(7.6.6)

Since a 2 E and ` 2 e? are arbitrary, (7.6.5) and (7.6.6) imply that  1 .O.f0 // D Ef:

(7.6.7)

Now we have 



D f0 C h?;e C e? : f C h?;g 0 0

(7.6.8)

But since h0 2 I.f0 ; e/, Theorem 5.4.1 asserts that h0 satisfies the Pukanszky condition as an element of S.f0 ; e/ so that 

f0 C h?;e  O.f0 /: 0

(7.6.9)

7.6 Irreducibility and Equivalence of .f; h; G/

271

One knows from (7.6.7)–(7.6.9) that ?;g

f C h0

 Gf:

Thus h0 satisfies the Pukanszky condition as an element of S.f; g/. Hence Theorem 5.4.1 implies that h0 2 I.f; g/. That is, .f; O h0 ; G/ D indG O 0 ; h0 ; E/ is E .f irreducible. So by (7.6.4), G .f; h; G/ D indG O 0 ; h0 ; E/ ' .Gf O /: E .f0 ; h; E/ ' indE .f

(ii) Suppose e D g (i.e. h is totally complex). This case is divided into two subcases, that is, Case I and Case II in B of the previous section. Thus by Theorem 7.5.9 we can again assume Case II and consider the situation B2 of the previous section. Put f0 Df C

r X ti i D1

4

Ui ; fQ20 D f 0 jm D fQ2 C

r X ti i D1

4

Ui

and h0 D

mCn X

CQk ˚ CC ˚ h2

kD1

with h2 D h \ mC as before. Then e0 D .h0 C h0 / \ g D

mCn X

RQk ˚ RC ˚ m ¤ g

kD1

and d0 D h0 \ g D

mCn X

RQk ˚ RC:

kD1

Lemma 7.6.4. f 0 2 Gf; h0 2 P C .f 0 ; G/ and h0 also satisfies the Pukanszky condition. Proof. Note that f has the form f D C C

r X

fk Uk .fk < 0; 1  k  r/

kD1

with respect to the dual basis fPk ; Qk ; C  ; .j Uk / ; .Ek;l / ; .Vk / ; .jVk / ; .jEk;l / ; .Uk / g p

p

272

7 Holomorphically Induced Representations...

and that mfQ1 D 0. By these facts, it is easy to see that f 0 2 Gf since fk C

tk < 0 .1  k  r/ 4

by our assumption H.f; h; G/ ¤ f0g (cf. Theorem 7.5.14). Since h2 is a Lie subalgebra of mC , Œh2 ; h2   h2 \ ŒmC ; mC  

 X 1 1  2 .˛k ˛l / C  2 .˛k C˛l / ˚ .m 1 /C C

k>l

2

and f 0 .Œh0 ; h0 / D 0 by mfQ1 D 0. Further, dim.h0 / D dim h and f 0 .Uk / < 0 .1  k  r/. These facts imply that h0 2 P C .f 0 ; G/. Finally, h0 satisfies the Pukanszky condition since exp.tQk /f 0 D f 0 C tPk .t 2 R/ 1  k  m C n.

 0

By this lemma together with the fact e ¤ g, it suffices to show that .f; h; G/ ' .f 0 ; h0 ; G/. We write g 2 G D N M in the form g D exp

mCn X kD1

! xk Pk exp

mCn X

! yk Qk exp.wC /h

kD1

with h 2 M . Let E 0 D exp.e0 / and let  W G ! G=E 0 be the canonical projection. Under the correspondence .g/ $ x D .x1 ; x2 ; : : : ; xmCn / between G=E 0 and RmCn , the Lebesgue measure dx on RmCn gives a quasiinvariant measure on G=E 0 . A function ˚ of the form  ˚.g/ D e i w .x/ .h/  2 L2 .RmCn /;

2 H.fQ20 ; h2 ; M /



is clearly contained in H.f 0 ; h0 ; G/. We denote as before by Cc .RmCn / the space of complex-valued continuous functions on RmCn with compact support and set ( SD

X

ck ˚k I ck 2 C; ˚k .g/ D e i w k .x/

k .h/;

kWfinite

) k 2 Cc .RmCn /;

k

2 H.fQ20 ; h2 ; M / :

7.6 Irreducibility and Equivalence of .f; h; G/

273

The space S is invariant under .f 0 ; h0 ; G/. Now we show that S is dense in H.f 0 ; h0 ; G/. For  2 H.f 0 ; h0 ; G/, we set O .x; h/ D  exp

mCn X

! ! xk Pk h

kD1

with x D .x1 ; x2 ; : : : ; xmCn / 2 RmCn ; h 2 M . Suppose that  is orthogonal to S. Then Z Z Z O .h/O .x; h/dhdx D 0 (7.6.10) .x/ .h/ .x; h/dxdh D .x/ 2 H.fQ20 ; h2 ; M /. Here

for all  2 Cc .RmCn / and Z ˇZ ˇ ˇ ˇ

ˇ2  Z Z Z ˇ .h/O .x; h/dhˇˇ dx  j .h/j2 dh jO .x; h/j2 dh dx D k k2H.fQ0 ;h 2

2 ;M /

 kO k2 < C1:

By letting  run over Cc .RmCn /, this implies that Z

.h/O .x; h/dh D 0

(7.6.11)

for almost all x 2 RmCn . But, if the function x on M is given by x .h/ D O .x; h/ .h 2 M /, then we have x 2 H.fQ20 ; h2 ; M / for almost all x 2 RmCn . Since is arbitrary, this implies that x D 0 in H.fQ20 ; h2 ; M / for almost all x 2 RmCn and hence that  D 0. Next we write g 2 G D M N in the form g D hexp

mCn X

! xk Pk exp

kD1

mCn X

! yk Qk exp.wC / .h 2 M /:

kD1

Then, S is the set of all functions g 7!

X

 PmCn 0 00  x x i w 2i D1 ck e k0 .x/ k .h/;

kWfinite

where ck 2 C;

k

2 H.fQ20 ; h2 ; M /;

mCn X D1

x 0 P C

mCn X

x 00 Q D h

D1

0 and k0 .x/ D k .x10 ; x20 ; : : : ; xmCn / with k 2 Cc .RmCn /.

mCn X D1

! x P

274

7 Holomorphically Induced Representations...

For ˚.g/ D

X

ck e

 PmCn 0 00  x x i w 2i D1 k0 .x/ k .h/;

kWfinite

we set X .x P C y Q /exp.wC / .R˚/ hexp

!

P

1

De .i w 2



x 2  2i

P

x y /



  12 detW 1 h0 2

Z

P

e.



P

1 z  2

2

X

/ ˚ hexp

!!

P

d



D

X

1

ck e .i w 2

P

x 2  2i

P

x y /

 12  detW 1 h0

Z

P

e.

 D

X

z

P

/ e . 12 1

ck e .i w 2

P

i 2 C 2

x 2  2i

P

P

0 00

x y /

/  0 . /d k

 12  detW 1 h0

k .h/

2

kWfinite  P

Z 

k .h/

2

kWfinite

e

1 O z  2

P

. O /

2

C 2i

P

O

O



k . /d ;

(7.6.12)

where h D h0 h0 ; h0 2 M0 ; h0 2 M 1 M1 ; z D x C iy .1   m C n/; 2

and h

mCn X

!

Ok Pk

kD1

D

mCn X

k Pk C

kD1

mCn X

O

O k Qk :

(7.6.13)

kD1

Lemma 7.6.5. The operator 2.mCn/=2 3.mCn/=4 R gives an isometry from S into H.f; h; G/. Proof. Note that R˚ is a C 1 -function on G. Let ˚0 be the C 1 -function on G given by

7.6 Irreducibility and Equivalence of .f; h; G/

275

  12 ˚0 .g/ D detW 1 h 2

for g D hexp

mCn X

.x P C y Q /exp.wC / .h 2 M /;

(7.6.14)

D1

where h D h0 h0 ; h0 2 M0 ; h0 2 M 1 M1 . It is easy to see that 2

˚0 .Uk C ij Uk / D

i tk ˚0 .1  k  r/; 4

˚0 X D 0 .X 2 h1 ˚ .h2 \ Œm; mC // : This fact together with Lemma 7.5.13 implies that if the function ˚1 given by ˚1 .g/ D e

 12

P

. O /

2

C 2i

P

  O

O

satisfies ˚1 X D 0 .X 2 h2 /;

(7.6.15)

then R˚ .˚ 2 S/ satisfies the infinitesimal condition on the space H.f; h; G/ by virtue of formula (7.6.12). By (7.6.13),

Ok D Ok .h; / D

X

Bf .h1 P ; Qk /;

(7.6.16)

mCn X O O

O Bf .h1 Pk ; P /

O k D O k .h; / D D1

D

mCn X

 Bf .h1 P ; Q /Bf .h1 Pk ; P /

; D1

for 1  k  m C n. For X 2 m and t 2 R, we set     O O

Ok D Ok .hexp.tX /; /; O k D O k .hexp.tX /; /: tX

tX

(7.6.17)

276

7 Holomorphically Induced Representations...

In order to show (7.6.15), it is sufficient to verify that 0 mCn X kD1

B B Ok @

  d Ok

ˇ ˇ ˇ tX ˇ ˇ dt ˇ

  ˇ ˇ O ˇ d O k ˇ

k tX ˇ i ˇ 2 dt ˇ ˇ t D0

C i Ok

ˇ ˇ ˇ tjX ˇ ˇ ˇ dt ˇ

t D0

  d Ok

C

k 2

ˇ ˇ ˇ ˇ tjX ˇ ˇ ˇ dt ˇ ˇ

  O d O k

t D0

1 C C C A

(7.6.18)

t D0

is equal to zero for any X 2 m. Using (7.6.16) and (7.6.17), we make the following calculations:   X X

Ok D

Bf .h1 P ; exp.tUl /Qk / D

Bf .h1 P ; Qk / D Ok ; t Ul

  O

O k



D

t Ul

 

Ok

tj Ul



X

O

O Bf .h1 Pk ; exp.tUl /P / D O k C t

D

Bf .h1 P ; exp.tj Ul /Qk /



8 t =2 ˆ ˆe Ok ˆ ˆ     ˆ <e t =2 Ok cos k t C OkCi 00 sin k t 2   2 00 t  l  00 t  D kil kil t =2 O ˆ O 00 sin ˆ



e cos k ki ˆ 2 2 l ˆ ˆ :O

k   X O

O k D

O Bf .h1 Pk ; exp.tj Ul /P / tj Ul

O Bf .h1 Pk ; Q /:

Dil1 C1



X

il X

.il1 C 1  k  il0 /; .il0 C 1  k  il0 C il00 /; .il0 C il00 C 1  k  il /; .otherwise/

1 il1

i` C1 mCn 00

Ce

t =2

il X

O Bf .h1 Pk ; exp.tj Ul /P /

Dil1 C1

    X  O cos t C O Ci 00 sin t l 2 2 0

il0 Cil00

Ce

t =2

Dil C1

 Bf .h1 Pk ; exp.tj Ul /P /

7.6 Irreducibility and Equivalence of .f; h; G/

Ce

il X

t =2

Dil0 Cil00 C1

277

    

00 t

il00 t O O cos il  il00 sin 2 2

 Bf .h1 Pk ; exp.tj Ul /P /: But the sum of the last two terms is equal to     X 

t

t C O Cil00 sin

O cos 2 2 0

il0 Cil00

Dil C1

    

t

t 1 1  Bf .h Pk ; P / cos C Bf .h Pk ; P Cil00 / sin 2 2      i l X

il00 t

il00 t C  O il00 sin

O cos 2 2 0 00 Dil Cil C1

    

il00 t

il00 t  Bf .h1 Pk ; P il00 / sin  Bf .h1 Pk ; P / cos 2 2 il0 Cil00

X

D

il X

O Bf .h1 Pk ; P / C

Dil0 C1

O Bf .h1 Pk ; P /:

Dil0 Cil00 C1

Thus we have

  O

O k

O D O k :

tj Ul

Hence for X D Ul , the real part of (7.6.18) is equal to zero and its imaginary part is equal to 0

1 2

Cil    2 1 ilX O k  O k 1 Ok C k OkCi 00 l 2 2 2 0 0

il X kDil1 C1 il X

C

Ok



kDil0 Cil00 C1

D

l1

kDil C1

 il mCn X

kil00 1X 1O O 00

O Bf .h1 Pk ; Q /

k 



k 2 2 kil 2 Di C1 kD1

 2 1

Ok  2 Di C1

il X

1 2 Di

00

il X

l1

 2

Ok D 0

l1 C1

by (7.6.16). Using the same notations for the structure constants as in Sect. 7.5, B2 ,  

O

p

tEk;l

D

mCn X i D1

i Bf .h1 Pi ; exp.tEk;l /Q / p

278

7 Holomorphically Induced Representations...

D

mCn X

0

1

ik X

i @Bf .h1 Pi ; Q /  t

i D1

ck;l; Bf .h1 Pi ; Q /A p;

 Dik1 C1 ik X

D O  t

O

 Dik1 C1

for il1 C 1   il . Otherwise,  

O

D

p

tEk;l

mCn X

p

i Bf .h1 Pi ; exp.tEk;l /Q / D O :

i D1

Again,   O

O

p tEk;l

X

D

p

Oi Bf .h1 P ; exp.tEk;l /Pi /

1i il1

il C1i mCn il X

C

0 @ Oi  t

i Dil1 C1

X

D

1

ik X

ck;l; O A Bf .h1 P ; exp.tEk;l /Pi / p;i

 Dik1 C1

Oi Bf .h1 P ; Pi / C t

il X

0

p

tjEk;l

  O

O

p

tjEk;l

D

mCn X

il X

Oi

ck;l;i Bf .h1 P ; P /

 Dil1

O p;i ck;l; O A Bf .h1 P ; Pi / D O ;

 Dik1 C1

i Bf .h1 Pi ; exp.tjEk;l /Q / D O ; p

i D1

D

mCn X

p

Oi Bf .h1 P ; exp.tjEk;l /Pi /

i D1

O D O C t

8 ik < X :

i Dik1 C1

C

il X i Dil1 C1

Oi

il X

Oi

ck;l;i Bf .h1 P ; Q / p;

 Dil1 C1

ik X  Dik1 C1

ck;l; Bf .h1 P ; Q / p;i

p;

1

ik X

@ Oi  t

i Dil1 C1

 

O

ik X

i Dik1 C1

1i il1

il C1i mCn

C

p

9 = ;

7.6 Irreducibility and Equivalence of .f; h; G/

X

O D O C t

279

ck;l;i . Oi Bf .h1 P ; Q / C O Bf .h1 P ; Qi //: p;

ik1 C1i ik

il1 C1 il p

Therefore the imaginary part of (7.6.18) is equal to zero for X D Ek;l . The real part of (7.6.18) is equal to X



ck;l; O O C p;

il1 C1 il

ik1 C1 ik

mCn 1X

2 D1

 C O Bf .h1 P ; Qi / D 

 p; ck;l;i Oi Bf .h1 P ; Q /

X ik1 C1i ik

il1 C1 il

X

X

ck;l; O O C p;

il1 C1 il

ck;l;i Oi O D 0 p;

ik1 C1i ik

ik1 C1 ik

il1 C1 il

by (7.6.16). Further,  

O

tVk

D

mCn X

i Bf .h1 Pi ; exp.tVk /Q /

i D1

D O 

mCn X

ik X

i

i D1

k; d; Bf

ik X

.h Pi ; Q / D O  t 1

 Dik1 C1

k; O d;



 Dik1 C1

for m C 1   m C n. When 1   m,  

O

tVk

D

mCn X

i Bf .h1 Pi ; exp.tVk /Q / D O :

i D1

Again,   O

O

D

tVk

m X

Oi Bf .h1 P ; exp.tVk /Pi /

i D1 mCn X

C

i DmC1

D

m X

0 @ Oi  t

ik X

1 k;i O A

 Bf .h1 P ; exp.tVk /Pi / d;

 Dik1 C1 ik X

Oi Bf .h1 P ; Pi / C t

i D1

Oi

i Dik1 C1



k;l Bf .h1 P ; Ql / C C e;i

mCn X i DmC1

0

mCn X



k;l Bf .h1 P ; Pl / d;i

lDmC1

@ Oi  t

ik X  Dik1 C1

1 k;i O A d;



280

7 Holomorphically Induced Representations...

0  @Bf .h1 P ; Pi / C t

k;i e; Bf .h1 P ; Q /A

 Dik1 C1

0

ik X

O D O C t @

mCn X

Oi

i Dik1 C1 mCn X

C

1

ik X

i DmC1

lDmC1

ik X

Oi

k;l e;i Bf .h1 P ; Ql /

1

k;l e;i Bf

.h1 P ; Q /A C terms of degree  2 in t:

 Dik1 C1

Again,  

O

tjVk

D

mCn X

i Bf .h1 Pi ; exp.tjVk /Q /

i D1 mCn X

D O C t

ik X

i

i D1

ik X

k; e; Bf .h1 Pi ; Q / D O C t

 Dik1 C1

k; O

 e;

 Dik1 C1

for m C 1   m C n. When 1   m,  

O

tjVk

D

mCn X

i Bf .h1 Pi ; exp.tjVk /Q / D O :

i D1

Further,   O

O

D

tjVk

m X

Oi Bf .h1 P ; exp.tjVk /Pi /

i D1

C

mCn X

0 @ Oi C t

i D1

D

m X

1

ik X

k;i O A

 Bf .h1 P ; exp.tjVk Pi // e;

 Dik1 C1 ik X

Oi Bf .h1 P ; Pi /  t

i D1

Oi

i Dik1 C1



k;l  d;i Bf .h1 P ; Ql / C

0

mCn X

lDmC1

@ Oi C t

i DmC1

0  @Bf .h P ; Pi / C t 1

ik X  Dik1 C1

 k;l e;i Bf .h1 P ; Pl /

mCn X

k;i d; Bf

ik X  Dik1 C1

1 k;i O A e;



1

.h P ; Q /A 1

7.6 Irreducibility and Equivalence of .f; h; G/

0

ik X O D O C t @

mCn X

Oi

i Dik1 C1 mCn X

C

i DmC1

k;l d;i Bf .h1 P ; Ql /

lDmC1

ik X

Oi

281

1

k;i d; Bf .h1 P ; Q /A C terms of degree  2 in t:

 Dik1 C1

Thus by (7.6.16), the real part of (7.6.18) is equal to



mCn X DmC1

0 ik mCn mCn X X X k;l 1 k; O d;

@ d;i Bf .h1 P ; Ql /

 C

Oi 2 C1 D1 i Di C1

ik X

O

 Dik1

lDmC1

k1

C

mCn X

ik X

Oi

i DmC1

1

k;i d; Bf .h1 P ; Q /A D 0;

 Dik1 C1

and the imaginary part of (7.6.18) is equal to mCn X DmC1

O

0 ik mCn mCn X X X k;l 1 k; O

 

Oi e;

@ e;i Bf .h1 P ; Ql / 2 C1 D1 i Di C1

ik X  Dik1

k1

C

mCn X

ik X

Oi

i DmC1

lDmC1

1

k;i e; Bf .h1 P ; Q /A D 0:

 Dik1 C1

Through these calculations we can conclude that, if ˚ 2 S, R˚ satisfies the infinitesimal condition imposed on the elements of H.f; h; G/. Furthermore, by the Plancherel formula, Z kR˚k2H.f;h;G/

D

e

P

.x /

2



 detW 1 h0 2

ˇZ !! ˇ2 ˇ ˇ P P X 1 2 ˇ ˇ

z  . /  ˇ e e 2 ˚ hexp d ˇ dxdydh

P ˇ ˇ Z   P 2 D.2/mCn e  .x / detW 1 h0 0

2

ˇ !!ˇ2 1 Z ˇ ˇ P P X 2 ˇ ˇ  @ e 2 x y  .y / ˇ˚ hexp y P ˇ dy A dxdh ˇ ˇ

282

7 Holomorphically Induced Representations...

Z D.2/mCn

e

P

.x y /

2

Z  dx

 detW 1 h0 2

ˇ !!ˇ2 ˇ ˇ X ˇ ˇ y P ˇ dydh D 2mCn 3.mCn/=2 k˚k2H.f 0 ;h0 ;G/ :  ˇ˚ hexp ˇ ˇ



We put 1 D .f; h; G/ and 2 D .f 0 ; h0 ; G/. Lemma 7.6.6. We have .1 .g0 /ıR/.˚/ D .Rı2 .g0 //.˚/

(7.6.19)

for all g0 2 G; ˚ 2 S. Proof. We again use expression (7.6.14). (a) Let g0 D hQ D hQ 0 hQ 0 2 M; hQ 0 2 M0 ; hQ 0 2 M 1 M1 . 2

..1 .g0 /ıR/˚/.g/ D .R˚/.g01 g/   12 P 1 P 2 i D e i w 2 .x /  2 x y detW hQ 1 h0  21

Z 

P

e

z

e

 12

P

. /

2

0

X

˚ hQ 1 hexp

!!

P

d

1

D e i w 2

P

.x /

P

2 i 2



x y

  12 detW 1 hQ 0 2

Z 

P

e

z

e

 12

P

.

/2

Q .2 .h/˚/ hexp

X

!!

P

d



D..Rı2 .g0 //˚/.g/: (b) Let g0 D exp.w0 C /. Since ˚ 2 H.f 0 ; h0 ; G/ and f 0 .C / D 1, (7.6.19) is trivially satisfied. P 0  P 0  P 0 1 (c) Let g0 D exp y Q and let h  y Q D y Q . Then 1

..1 .g0 /ıR/˚/.g/ D e i w 2

P

.x /

2 i 2

P

x y



detW 1 h0 2 !! Z P P X 0 //  1 2

.x Ci.y y . / ˚ e 2  e hexp

P



D..Rı2 .g0 //˚/.g/

 12

7.6 Irreducibility and Equivalence of .f; h; G/

283

since X

.2 .g0 /˚/ hexp



X

D ˚ hexp

!!

P !

P exp 



D e i

P

0 y

X

!

X

y 0 Q exp



˚ hexp

X

!!

y 0 C



!! :

P



(d) Let g0 D exp

P

 x 0 P and let X

1

h 

! D

x 0 P

X



x 0 P C

X



y 0 Q :



Then, we have: ..1 .g0 /ıR/˚/.g/ D .R˚/ hexp

X ..x  x 0 /P C .y  y 0 /Q /

! !!

 exp

wC 1

P

1X .x y 0  x 0 y / C 2 i

P

1

P

0 2

P

0

P

0

i

D e i w 2 .x /  2 x y  2 .x / C x x Ci x y  2  12 Z P  1 P 0 0 2  detW 1 h0 e ..x x /Ci.y y // e  2 . / 2

2

 ˚ hexp

X

P

x 0 y 0

!!

P

d :

(7.6.20)



On the other hand, .2 .g0 /˚/ hexp

X

!!

P



! !! X X .x 0 P C y 0 Q / exp

P D˚ hexp 

De

i

P

. y 0  12 x 0 y 0 / ˚ hexp



X .  x 0 /P

!! :

284

7 Holomorphically Induced Representations...

Thus ..Rı2 .g0 //˚/.g/ D e

i w 12

P

.x /

2 i 2

P

x y

  12 Z P detW 1 h0 e z 2

e

 12

1

D e i w 2

P

2 i . /

P

e

P

.x /

2 i 2

P

0 1 . y  2

x y

P

0 0 x y /

 detW 1 h0

! X 0 .  x /P d ˚ hexp

 12 Z

P

e

0 . Cx /z

2

e

 12

P

0 2 . Cx / i

P

0 0 1 0 0 .. Cx /y  2 x y / ˚ hexp

X

!!

P

d



D ..1 .g0 /ıR/˚/.g/ 

by (7.6.20). In view of Lemmas 7.6.5, 7.6.6, the linear operator 2.mCn/=2 3.mCn/=4 R

defined on S can be extended to an isometry RO between H.f 0 ; h0 ; G/ and some closed subspace H0 of H.f; h; G/. It is obvious that RO satisfies the relation O 2 .g/ for any g 2 G, and hence the space H0 is invariant under 1 .g/ıRO D Rı 1 D .f; h; G/. But .f; h; G/ is irreducible since h is totally complex (cf. Theorem 5.1.23). Therefore H0 D H.f; h; G/, which finishes the proof of our assertion (A). (B) We prove the converse, namely, if H.f; h; G/ ¤ f0g and if .f; h; G/ is irreducible, then h satisfies the Pukanszky condition. Assume the hypotheses. Let E D exp e and f0 D f je 2 e . Then h 2 P C .f0 ; E/ and h satisfies the Pukanszky condition as an element of P C .f0 ; E/. Since H.f0 ; h; E/ ¤ f0g by our assumption, we can deduce from (A) that .f0 ; h; E/ ' .Ef O O 0 ; h0 ; E/ 0 /, i.e. .f0 ; h; E/ ' .f for any h0 2 I.f0 ; e/  S.f; g/. Then O 0 ; h0 ; E/ ' indG .f; O h0 ; G/ ' indG E .f E .f0 ; h; E/ D .f; h; G/ and hence .f; O h0 ; G/ is irreducible. Thus we obtain that h0 2 I.f; g/. By Theorem 5.4.1 and Lemma 5.1.11, it follows that f C h? 0  Gf and hence f C e?  Gf . This implies by Lemma 5.1.11 again that h satisfies the Pukanszky condition as an element of P C .f; G/. The proof of Theorem 7.6.2 is complete. 

7.7 Decomposition of .f; h; G/

285

7.7 Decomposition of .f; h; G / In this last section, G stands for an exponential solvable Lie group with Lie algebra g as before. Let f 2 g and h 2 M.f; gC /. Further, let e be the subspace of g defined N \ g, d a Lie subalgebra of g defined by d D h \ g and D D exp d. by e D .h C h/ We denote by U.f; h/ the set of all orbits  2 g =G such that  \ .f C e? / is a non-empty open set in f C e? . For  2 g =G, we denote by c.; f; h/ the number of the connected components in  \ .f C e? /. Lemma 7.7.1 (cf. Lemma 11.2.2 in Chap. 11). Let h 2 M.f; gC /. We denote by V .f; h/ the set of all orbits  2 g =G of the form  D G.f C /, where  2 e? and h 2 M.f C ; gC /. Then U.f; h/ D V .f; h/. Moreover, let  2 U.f; h/ and f C  . 2 e? / an element of  \ .f C e? /. Then the connected component of  \ .f C e? / which contains f C  is equal to D.f C /. Proof. (a) We prove that V .f; h/  U.f; h/. Let  2 e? such that h 2 M.f C ; gC /. Then f C  2 G.f C / \ .f C e? / and hence W D G.f C / \ .f C e? / is non-empty. We show that W is an open set in f C e? . Let f C  0 2 W , i.e.  0 2 e? and f C  0 2 G.f C /. It follows that h 2 S.f C  0 ; gC / and that dim.g.f C  0 // D dim.g.f C //. Thus h 2 M.f C  0 ; gC / and hence D.f C  0 /  W is an open set in f C e? (cf. Proposition 3.1.1 in Chapter IV of [10]). (b) We prove the inverse inclusion U.f; h/  V .f; h/. Let  2 U.f; h/ and U1 D f 2 e? I dim.g.f C // D dim.g.f //g: Since h 2 S.f C ; gC / for any  2 e? , dim.g.f // D min dim.g.f C //: 2e?

It follows that U1 is a non-empty Zariski open set in e? and thus dense in e? . Consequently  \ .f C e? / contains at least one point f C 1 of f C U1 and therefore  D G.f C 1 / 2 V .f; h/. (c) We prove the latter part of the lemma. Let  2 U.f; h/; f C  . 2 e? / an element of  \ .f C e? / and let C denote the connected component of  \ .f C e? / which contains f C . Then C is stable under the coadjoint action of D on g . For any f C  0 2 C , D.f C  0 / is a connected open set in  \ .f C e? / and hence in C . Thus D.f C / is an open set in C and its complement in C is a union of open D-orbits. Consequently D.f C / is not only open but also closed in C , and thus C D D.f C /. 

286

7 Holomorphically Induced Representations...

The following theorem generalizes the result of M. Vergne for real polarizations (cf. Theorem 11.2.1). Theorem 7.7.2. Let f 2 g and h 2 P C .f; G/. If H.f; h; G/ ¤ f0g, then: (a) U.f; h/ is a finite set, (b) c.; f; h/ < C1 P for  2 U.f; h/, O (c) .f; h; G/ '  2U.f;h/ c.; f; h/./. Proof. Let f0 D f je 2 e and E D exp e. Then h 2 P C .f0 ; E/ and .f; h; G/ D indG E .f0 ; h; E/: By our assumption, H.f0 ; h; E/ ¤ f0g and hence, by Theorem 7.6.2, .f0 ; h; E/ ' .Ef O O 0 ; h0 ; E/ 0 /; i:e:.f0 ; h; E/ ' .f for any h0 2 I.f0 ; e/. Since dimR .h0 / D dimC h, we have h0 2 M.f; g/. Therefore by Theorem 11.2.1, .f; h; G/ can be decomposed into irreducible components as follows: .f; h; G/ ' indG O 0 ; h0 ; E/ ' .f; O h0 ; G/ E .f X c.; f; .h0 /C /./: O '  2U.f0 ;.h0 /C /

Thus it suffices to establish the following two assertions: (A) U.f; .h0 /C / D U.f; h/; (B) c.; f; .h0 /C / D c.; f; h/ for  2 U.f; .h0 /C /. First we prove (A). Let  2 U.f; h/ D V .f; h/, i.e. let  be given by  D G.f C / where  2 e? and h 2 M.f C; gC /. Clearly  2 .h0 /? ; h0 2 M.f C; g/, and hence  2 U.f; .h0 /C /. Conversely let  2 U.f; .h0 /C / be given by  D G.f C/ where  2 .h0 /? and h0 2 M.f C ; g/. We write f C  in the form 

f C  D f0 C l1 C l2 ; l1 2 .h0 /?;e ; l2 2 e? :

(7.7.1)

Let  denote the coadjoint representation of E. Since h0 2 I.f0 ; e/, Theorem 5.4.1 E

asserts that there is some a 2 H0 D exp.h0 / such that a  f0 D f0 C l1 : E

If we set a1  .f0 C l1 / D a1  .f0 C l1 / C l3 .l3 2 e? /; E

(7.7.2)

7.7 Decomposition of .f; h; G/

287

then, since e is a Lie subalgebra, l4 D l3 C a1 l2 2 e? and we have a1  .f C / D f C l4 2 f C e? by (7.7.1), (7.7.2). Thus dim.g.f C l4 // D dim.g.f //; h 2 M.f C l4 ; gC / and  D G.f C l4 / 2 U.f; h/. Next we verify (B). Let  2 U.f; .h0 /C / and f C  2 f C .h0 /? such that h 2 M.f C ; gC /. We showed in the proof of (A) that there exists some a 2 H0 D exp.h0 / such that a.f C / 2 f C e? . This fact together with Lemma 7.7.1 implies that any connected component of  \ .f C .h0 /? / is of the form C D H0 .f C  0 / where  0 2 e? and f C  0 2  \ .f C e? /. We prove that C \ .f C e? / D D.f C  0 /: In fact, since d D ef0 , d  h0 and hence D.f C  0 /  C \ .f C e? /: Conversely let f C 2 C \ .f C e? / with 2 e? . Then there is a0 2 H0 such that a0 .f C  0 / D f C . But clearly a0  f0 D f0 and hence a0 2 D. Consequently f C

E

D a0 .f C  0 / with a0 2 D so that C \ .f C e? /  D.f C  0 /:

Hence by Lemma 7.7.1, the map C D H0 .f C  0 / 7! C \ .f C e? / D D.f C  0 / gives a one-to-one correspondence between the connected components of  \ .f C .h0 /? / and those of  \ .f C e? /. 

Chapter 8

Irreducible Decomposition

As usual let G D exp g be an exponential solvable Lie group with Lie algebra g. It is well known that there is a strong duality between the induction and the restriction of representations. In this chapter, we study the irreducible decomposition of the representation induced from a subgroup or the representation restricted to a subgroup.

8.1 Monomial Representations We begin with a simple lemma. Lemma 8.1.1. Let V be a finite-dimensional real vector space where G acts by a representation of exponential type. Let v be a nonzero G-invariant vector of V : i.e. gv D v for all g 2 G. We arbitrarily fix x 2 V and consider the line Lx D x C Rv. We then have the following two possibilities: Lx \ Gx D x or Lx \ Gx D Lx : In other words, the line passing x and having the direction of the invariant vector v either meets the orbit Gx at only one point or is completely contained there. Proof. Let V0 be the subspace of V spanned by v, VN the quotient space V =V0 and p W V ! VN the canonical projection. The representation of G obtained on VN by passing to the quotient is obviously of exponential type. Then, in order that gx 2 Lx .g 2 G/ it is necessary and sufficient that g belongs to G.p.x//. Whereas, if we put gx D x C .g/v, it is immediately known that  gives a homomorphism of G.p.x// to R. Theorem 5.3.2 assures that G.p.x// is connected so that either  0 or the image of  coincides with the whole R. Now it is enough to remark that the intersection of Lx and Gx is nothing but fgxI g 2 G.p.x//g.  © Springer Japan 2015 H. Fujiwara, J. Ludwig, Harmonic Analysis on Exponential Solvable Lie Groups, Springer Monographs in Mathematics, DOI 10.1007/978-4-431-55288-8__8

289

290

8 Irreducible Decomposition

Definition 8.1.2. In the situation of the above lemma, provided Lx  Gx, the orbit Gx is said to be saturated in the v-direction and otherwise Gx is said to be non-saturated. When there is an ideal g0 of g such that dim.g=g0 / D 1, a linear form ` 2 g satisfying `jg0 D 0 is an invariant vector relative to the coadjoint representation of G. We write g D RX C g0 . Let p be the projection from g onto g0 and G0 D exp.g0 /. The next lemma is immediately derived from the definition of the radical of an alternating bilinear form. Lemma 8.1.3. Let ` 2 g and put `0 D p.`/. If g.`/  g0 , then g.`/  g0 .`0 / and dim.g0 .`0 // D dim.g.`// C 1: If g.`/ 6 g0 , then g0 .`0 /  g.`/ and dim.g.`// D dim.g0 .`0 // C 1: Lemma 8.1.4. Let ` 2 g ; `0 D p.`/ and ˝ D G`. (1) If the orbit ˝ is saturated in the direction of g? 0 , there is a family f!s gs2R of G0 -orbits in g0 such that p.˝/ D [s2R !s and exp.tX /  !s D !sCt . Moreover, G.`0 /  G0 . (2) If the orbit ˝ is non-saturated in the direction of g? 0 , then p.˝/ D G0 `0 . Proof. (1) First G D exp.RX /G0 D G0 exp.RX /. Now if we set !0 D G0 `0 , !0  p.˝/ and, for any t 2 R, exp.tX /!0 is an G0 -orbit contained in p.˝/. Put !t D exp.tX /!0 . Since p.˝/ D p.G`/ D G`0 D exp.RX /!0 , The union of f!t gt 2R coincides with p.˝/. From the definition of !s , exp.tX /!s D !sCt . Furthermore, !s D !t when and only when s D t. In fact, if exp.sX /`0 2 !t , exp.sX /`0 D exp.tX /g0 `0 with g0 2 G0 . Hence .exp..t  s/X // g0 2 G.`0 /. So, if we show G.`0 /  G0 , then exp..t  s/X / 2 G0 and consequently s D t. For this aim it is enough to show g.`0 /  g0 . By Lemma 8.1.3 there is X1 2 g0 .`0 /ng.`/ so that  D X1 ` ¤ 0 belongs to g? 0 . Y being an arbitrary element of g.`0 /, Y ` 2 g? 0 so that Y ` D .tX1 /` with some t 2 R. From this Y  tX1 2 g.`/  g0 and Y 2 g0 since X1 2 g0 . (2) Since g.`/ 6 g0 , G D G0 G.`/. Hence the orbit G` is equal to G0 `. From this we immediately see p.G`/ D p.G0 `/ D G0 `0 .  We simply write O0 instead of OG0 . Proposition 8.1.5. Let 0 2 GO0 . We suppose 0 ' O0 .`0 / with `0 2 g0 . Let ` be an extension of `0 to g, and we put ˝ D G`. G O (1) If ˝ is saturated in the direction of g? 0 , indG0 0 ' .`/. R˚ G ? O /d . Here, X (2) If ˝ is non-saturated in the direction of g0 , indG0 0 ' R .`  being a fixed element of g.`/n .g.`/ \ g0 /, ` 2 g is defined by ` jg0 D `0 and ` .X / D 2 .

8.1 Monomial Representations

291

Proof. (1) Let h be the Vergne polarization at ` of g constructed from a strong Malcev sequence of Lie subalgebras of g passing g0 , h 2 I.`; g/ and h  g0 . Hence h 2 I.`0 ; g0 /. Therefore   G0 G O indG G0 0 ' indG0 indH `0 ' .`/ with H D exp h. The assertion follows from this. ? (2) Each G-orbit which meets ` C g? 0 is non-saturated in the direction of g0 . Let  X be a fixed element of g.`/n .g.`/ \ g0 /, and define ` 2 g . 2 R/ by ` jg0 D `0 ; ` .X / D 2 : If we construct as in (1) h 2 I.`; g/, h 2 I.` ; g/. We set h0 D h \ g0 and H0 D exp.h0 /. R˚ We show indH H0 `0 ' R ` d . We denote by C the space of all complex-valued continuous functions  on H which satisfy .hh0 / D ` .h0 /1 .h/ for any h 2 H and h0 2 H0 , with compact support modulo H0 . Since h0 is an ideal of h, there exists on H=H0 an H -invariant measure . We denote by  the representation indH H0 `0 . Hence its representation space H is the completion of C regarding the norm Z

1=2

kk D

j.h/j d .hH0 / 2

:

H=H0

Q .t/ Q The space C is identified with Cc .R/ by the mapping  7! , D .exp.tX //. 2 Then,

is identified with a Lebesgue measure and H with L .R/. Thus kk2 D  R 2 R j.exp.tX //j dt. R˚ The space of the representation  D R ` d becomes L2 .R/ and, for h 2 H and  2 L2 .R/, ..h//. / D ` .h/. /. F denotes the Fourier transformation of L2 .R/. We will see that the operator U from H to L2 .R/ defined by U D F Q furnishes an equivalence from  to . Since F and the mapping  7! Q are both unitary bijection, the same for U . Hence it is enough to show that U intertwines  and . For h 2 H; 2 R and  2 H , Z

1

..U ı .h/// . / D

e 2i  t .h1 exp.tX //dt;

1

...h/ıU /.// . / D ` .h/

Z

1

1

e 2i  t .exp.tX //dt:

292

8 Irreducible Decomposition

Since H D exp.RX /H0 D H0 exp.RX /, it suffices to show the equality of these two expressions for h D exp.sX / .s 2 R/ and h 2 H0 . By the way, with X0 2 h0 , Z ..U ı .exp.X0 //.// . / D

1

1 Z 1

D 1

e 2i  t .exp.X0 /exp.tX //dt    e 2i  t  exp.tX /exp e ad.tX / .X0 / dt:

  But with Y 2 Œh; h  ker` \ h0 , exp e ad.tX / .X0 / D exp.X0 C Y /. Therefore Z ..U ı .exp.X0 ///.// . / D

1 1

e 2i  t `0 .exp.X0 C Y //1 .exp.tX //dt:

Further, `0 .exp.X0 C Y // D e i `0 .X0 CY / D e i `0 .X0 / D `0 .exp.X0 //1 . Hence, Z ..U ı .exp.X0 ///.// . / D

1 1

e 2i  t `0 .exp.X0 //.exp.tX //dt

D ...exp.X0 //ıU /.// . /: On the other hand, Z

1

..U ı .exp.X ///.// . / D 1 Z 1

D

e 2i  t .exp..t  /X /dt e 2i  t e 2i   .exp.tX //dt

1

D ` .exp.X //

Z

1

e 2i  t .exp.tX //dt

1

D ...exp.X //ıU // . / with  2 R. Thus indH H0 ` ' From this,

R˚ R

` d .

Z    H  G0 G G G indG ind ' ind ind ' ind  ' ind   0 ` ` G0 G0 H H0 0 H H0 0 by use of Theorem 3.2.8 in the third chapter. Since indG H  R˚ G R indH ` d and h 2 I.` ; g/ for any 2 R,



˚ R

` d

R ˚ R

 ` d

'

8.1 Monomial Representations

293

Z indG G0 0 '

˚

.` O /d R



is obtained.

Now h 2 S.f; g/ being given, let us examine the monomial representation  D .f; O h; G/ D indG H f . In this examination of , our stage turns out to be the affine subspace  D f C h? of g . We begin to examine the intersections of this affine space with G-orbits. 

Theorem 8.1.6. Let k be an ideal of g, f 2 k and h 2 S.f; k/. We denote by h?;k the annihilator of h in k , by  a positive finite measure on k equivalent to the  Lebesgue measure in f C h?;k and by the image of  by the canonical projection   from k onto the space k =G of G-orbits. Then, the following holds for -almost all orbits ˝ 2 k =G.    (1) Each connected component C of f C h?;k \ ˝ is a manifold. 

(2) The tangent space of C at a point ` 2 C is equal to g  ` \ h?;k . R (3) When we decompose the measure  with respect to as  D ˝ d .˝/, the restriction of the fibre measure ˝ to any coordinate neighbourhood .U I x1 ; : : : ; xm / is equivalent to dx1   dxm . Proof. We proceed by induction on dim k. Let k0 be an ideal of g contained in k such that k=k0 is irreducible as g-module. We identify k0 with a subspace of k . If dim .k=k0 / D 1, for  2 g , k D R ˚ k0 ; jk0 D 0; X  D .X / .X 2 g/: And if dim .k=k0 / D 2, using ˛ 2 R n f0g and  2 g , k D R 1 ˚ R 2 ˚ k0 ; 1 jk0 D 2 jk0 D 0; X  1 D .X /. 1  ˛ 2 /; X  2 D .X /. 2 C ˛ 1 / .X 2 g/: Then, the restriction map k 3 ` 7! `0 D `jk0 2 k0 is nothing but the projection p W k ! k0 along this decomposition. For ` 2 k , if we denote by G.`0 / D exp.g.`0 // the stabilizer of `0 in G, 1 p .`0 / \ G` D G.`0 /`. Provided g.`0 / is contained in ker, G.`0 / keeps (or

1 ; 2 ) invariant. When dim .k=k0 / D 1, for g 2 G.`0 /, if we write g` D ` C a.g/ using a.g/ 2 R, the function a W G.`0 / ! R satisfies a.g1 g2 / D a.g1 / C a.g2 /. Since G.`0 / is connected, the image of the homomorphism a either degenerates to f0g or is the whole R. When dim .k=k0 / D 2, setting likewise g` D ` C a1 .g/ 1 C a2 .g/ 2

294

8 Irreducible Decomposition

for g 2 G.`0 /, we get the two homomorphisms a1 ; a2 W G.`0 / ! R and we recognize that p 1 .`0 / \ G` has three possibilities: the plane p 1 .`0 /, a line or one point f`g. Besides, these possibilities depend on `0 , not exactly on ` itself. We next suppose that g.`0 / is not contained in ker and take X 2 g.`0 / such that .X / D 1. If dim .k=k0 / D 1. The relation exp.tX /` D ` C a.t/ .t 2 R/ tells us that a.t/.e s  1/ D a.s/.e t  1/ holds for all s; t 2 R. From this, a.t/ D c.e t  1/: with some constant c 2 R. Hence p 1 .`0 / \ G` is either one point f`g, a half-line or the line p 1 .`0 /. If dim .k=k0 / D 2, Writing exp.tX /  ` D ` C a1 .t/ 1 C a2 .t/ 2 .t 2 R/ and computing exp.tX /exp .X=˛/  ` D exp .X=˛/ exp.tX /  `; we have a1 .t/ C a2 .=˛/ e t sin.˛t/ C a1 .=˛/ e t cos.˛t/ D a1 .=˛/  e =˛ a1 .t/; a2 .t/ C a2 .=˛/ e t cos.˛t/  a1 .=˛/ e t sin.˛t/ D a2 .=˛/  e =˛ a2 .t/: From this, with certain constants c1 ; c2 2 R, a1 .t/ D c1  e t .c1 cos.˛t/ C c2 sin.˛t// ; a2 .t/ D c2  e t .c2 cos.˛t/  c1 sin.˛t// hold for all t 2 R. Now, p 1 .`0 / \ G` is either one point f`g, a spiral or the plane p 1 .`0 /. To begin with, assume h  k0 . We denote by  the set of ` 2  such that the G-orbit G  `  k has the maximal dimension among the G-orbits meeting   D f Ch?;k . We set G 0 D exp.g0 /; g0 D ker; and, replacing G by its connected normal subgroup G 0 , introduce likewise the subset  0 of . Next, using 0 D  f0 C h?;k0 ; f0 D f jk0 ; instead of  we define the subsets 0 ; 00 of 0 . Now if we put E D  \  0 \ p 1 .0 / \ p 1 .00 /, then  \ GE  E and, since

8.1 Monomial Representations

295

the -measure of nE is 0, we may restrict our reasoning on E, where the same eventuality of the possibilities of G.`0 /  ` examined above occurs for all ` 2 E. Thus we obtain the claims (1)–(3) by applying the induction hypothesis to 0 . Hereafter we assume h 6 k0 and put h0 D h \ k0 . Suppose first that dim .k=k0 / D 1 and set h D RX ˚ h0 ; k D RX ˚ k0 . In this case, we identify k0 with the hyperplane f` 2 k I `.X / D f .X /g and apply the induction hypothesis ?;k

to 0 D f0 C h0 0 . As before, in order to construct a non-empty Zariski open  , we utilize h0 and 0 . When G.`0 /  ` is a line for ` 2 E, or set E of f C h?;k 0 when G.`0 /  ` is a half-line for ` 2 E and one point for ` 2 , there is nothing to prove. Besides, if G.`0 /  `; ` 2 E; is a half-line and  \ E ¤ ;, it suffices to remark that each connected component of  \ G  ` is an open submanifold of the connected component of 0 \ G  `0 containing it, which is a manifold by the induction hypothesis. Let us examine the essential case where G.`0 /  ` is one point for any ` 2 E. We fix ` 2  such that each connected component of M D 0 \ G  `0   is a manifold. Here, `0 signifies ` regarded as an element of k0 . Now we define a function  on M as follows. Provided  D g  `0 in k0 , namely if 0 D g  `0 , ./ D .g 1  /.X /; here the action of G is taken in k . Notice that  \ G  ` D  1 .`.X //: By the way,  being analytic, the set N of the critical points of  in a connected component of M is a null set with respect to the fibre measure induced on M as a result of the decomposition of the measure , namely a measure equivalent to the Lebesgue measure in a coordinate neighbourhood. Otherwise,  becomes a constant map on this connected component. Sard’s theorem [73] tells us that the set W of the critical values of  is a null set for the Lebesgue measure and each connected component of  \G ` D  1 .`.X // is a submanifold if `.X / 62 W . In a neighbourhood of a regular point of , we may take  as one of local coordinates so that (1) and (3) follows. Concerning (2), the function  considered above is defined on the manifold p 1 .G  `0 /  k and separates there the orbits. By restricting this to , it turns out that the tangent space of  \ G  ` D  \  1 .`.X // at a regular point ` is  h?;k \ g  `. Relating to the above things we evoke the following computations. For arbitrary Y 2 g, Y  ./ D Y  .g  `0 .gX // D .Y  0 /.gX / C  .ŒY; gX / D  0 .ŒY; .gX /0 / C  .ŒY; gX / D c.ŒY; X / .0 ¤ c 2 R/; Here, .gX /0 denotes the k0 -component of gX 2 k D RX ˚ k0 .

296

8 Irreducible Decomposition

This reasoning is applied to other cases. For example, Assume dim .k=k0 / D 2. We apply the induction hypothesis to the pair .f0 ; h0 / and as in the previous case fix ` 2  in a general position. Provided G.`0 /  ` coincides with the plane p 1 .`0 /, there is nothing left to prove. Assume dim .h=h0 / D 1. By induction hypothesis M D  \ p 1 .G  `0 / is a manifold. In the case where G.`0 /  ` is a line, as ` C R 1 for instance. Then, for any  2 M , G   meets the line ` C 1 C R 2 at only one point ` C 1 C ./ 2 , and we get the function  W M ! R. In the case where G.`0 /` is a spiral, its gushing point is in the plane p 1 .`0 /, that is to say, the unique point f`1 g such that G.`0 /  `1 D f`1 g. In the plane p 1 .`0 / we draw at the centre `1 a circle S with a positive radius arbitrarily chosen. Then for any  2 M , G   intersects S at only one point ./ and we obtain the mapping  W M ! S . In the case where G.`0 /  ` D f`g, our map  W M ! p 1 .`0 / associates  2 M with the common point p 1 .`0 / \ G  `. ?;k Assume finally dim .h=h0 / D 2. We identify f0 C h0 0  k0 with  D f C  h?;k  k , and fix ` 2  such that M D  \ G  `0 is a manifold by the induction hypothesis. Next, completely as above we construct the function  according to the cases which occur. Anyhow, we may apply Sard’s theorem cited above to the mapping  obtained in this manner.  If we take g as k in this theorem and continue the use of notations of the theorem: Corollary 8.1.7. Let f 2 g ; h 2 S.f; g/ and  D  D f C h? . (1) For almost all coadjoint orbits ˝ 2 g =G with respect to the measure , the support of the fibre measure ˝ is the whole  \ ˝. (2) For almost all coadjoint orbits ˝ 2 g =G with respect to the measure , each connected component C of  \ ˝ is a manifold, whose dimension is larger than or equal to 12 dim ˝. (3) We have dim C D 12 dim ˝ in the statement (2), if and only if H ` D C for every ` 2 C . Besides, in this case h C g.`/ is a maximal isotropic subspace for the bilinear form B` . Proof. The statement (1) is clear. Take a generic connected component C in question and let ` 2 C . By easy calculations:     dim C D dim g  ` \ h? D dim h` =g.`/ D dim g  dim h C dim .g.`/ \ h/  dim.g.`// D dim g  dim .h C g.`//  dim g  D

1 .dim g C dim.g.`/// 2

1 dim ˝: 2

Here the equality holds if and only if h C g.`/ is a maximal isotropic subspace for B` . In such a case,

8.1 Monomial Representations

297

dim.H  `/ D dim .h= .g.`/ \ h// D dim h  dim .g.`/ \ h/ D dim .h C g.`//  dim.g.`// D

1 dim ˝: 2 

From this, we get the statements (2) and (3).

Now, let us obtain the canonical irreducible decomposition of . We denote by z the centre of g and let a be a minimal non-central ideal of g. Of course, dim.a=.a \ z//  2. Everybody has probably noticed that, when it is a matter of exponential solvable Lie groups, the main tool of proofs is the induction. We use induction on dim g; dim h; dim.g=h/ and dim g C dim.g=h/. In many cases we are able to pass from h to h C z without problem and consequently we may consider only the case where h contains z. If f 2 g vanishes on a non-trivial ideal of g, we could go down to the quotient space by this ideal. Through this routine we arrive at the case where dim z  1; dim a  3. Provided g ¤ h C Œg; g, there is an ideal g0 of g containing h and such that dim.g=g0 / D 1. In this case, we may apply the induction hypothesis to G0 D exp.g0 / and combine with Proposition 8.1.5. If g D h C Œg; g, set k D h C a, K D exp k. Taking in mind the Theorem 3.2.8 in the third chapter concerning the induction by stage, what we have to do first is to analyse the monomial representation indK H f . Lemma 8.1.8. (1) Let G D exp.g2 /, g2 D hX; Y iR D RX C RY W ŒX; Y  D Y . Take f 2 g2 and put h D RX , H D exp h. Further, if we set H 0 D exp.h0 /; h0 D RY , G G indG H f ' indH 0 Y  ˚ indH 0 Y  :

(2) Take G D exp .g3 .˛//, g3 .˛/ D hT; Y1 ; Y2 iR W ŒT; Y1  D Y1  ˛Y2 ; ŒT; Y2  D Y2 C ˛Y1 .0 ¤ ˛ 2 R/, and let f 2 g3 .˛/ ; h D RT , H D exp h. Then, using H 0 D exp.RY1 C RY2 /; O D .cos /Y1 C .sin /Y2 2 g3 .˛/ , Z indG H f '

˚

Œ0;2/

indG H 0 O d:

(3) We take G D exp.g4 /, g4 D hT; X; Y; ZiR W ŒT; X  D X; ŒT; Y  D Y , ŒX; Y  D Z. Let f D ˛T  C ˇZ  2 g4 .ˇ ¤ 0/; h D hT; X; ZiR and H D exp h. Then, using H 0 D exp.h0 /; h0 D hT; Y; ZiR , G indG H f ' indH 0 f :

(4) Take G D exp.g6 /, g6 D hT; X1 ; X2 ; Y1 ; Y2 ; ZiR W ŒT; X1  D X1  ˛X2 ; ŒT; X2  D X2 C ˛X1 ; ŒT; Y1  D Y1  ˛Y2 ; ŒT; Y2  D Y2 C ˛Y1 ; ŒXi ; Yj  D ıij Z .0 ¤ ˛ 2 R/. Let f D ˇT  C Z  . ¤ 0/; h D hT; X1 ; X2 ; ZiR and H D exp h. Then, using H 0 D exp.h0 /; h0 D hT; Y1 ; Y2 ; ZiR , G indG H f ' indH 0 f :

298

8 Irreducible Decomposition

Proof. Probably these facts are well known except (2) [10, 77]. Here we show (2). For this aim, it is enough to continue a little more the observations on page 135 of [10]. We borrow the notations used there. We identify a D p RY1 C RY2 with the complex plane C by the bijection Y1 C Y2 7! C i; i D 1. Put  D f .T /. 2 2 2 The monomial representation  D indG H f is realized in the space L .C/ D L .R / 2 by the following formula: for 0 2 R; z0 ; z 2 C and  2 L .C/,   .exp. 0 T /exp.z0 //.z/ D e 0 .i 1/  ze  0 .1i ˛/  z0 : Let F be the Fourier transformation of L2 .C/, .j/ be the inner product in R2 D C and d z a Lebesgue measure of R2 . A direct computation gives the following: for gO D exp. 0 T /exp.z0 / 2 G; 2 L2 .C/ and v 2 C, F ..g/ O / .v/ D e 0 .i C1/Ci.vje

0 .1i ˛/ z / 0

F

 .1Ci ˛/  e0 v ;

while, using the dual basis fY1 ; Y2 g of a and the bijection Y1 C Y2 7! C i, a is identified with C. Then, if we compute 0 C i0 D exp.tT /  . C i/,

0 D e t . cos.˛t/ C  sin.˛t// ; 0 D e t . sin.˛t/ C  cos.˛t// : From this the coadjoint orbit ˝ passing the point e i is given by e t Ci. ˛t / with t running over R, and the Jacobian of the change of variables . 0 ; 0 / 7! .t; / becomes e 2t . We set 1 D F ı  ı F 1 . For gO D exp. 0 T /exp.z0 / 2 G,   t Ci. ˛t /   t Ci. ˛t / je 0 . ˛t / z / 0 D e 0 .1Ci /Ci.e O e  e .t C 0 /.1Ci ˛/Ci : 1 .g/ Hence 1 keeps L2 .˝ / stable and the formula . O g/ O .t/ D e 0 .1Ci /Ci.e

i je . 0 t /.1i ˛/ z

0/

 .t  0 /;

gives a representation O of G on the space of all functions  satisfying Z e 2t j .t/j2 dt < C1: R

However, O is nothing but the representation  D indG H 0 O . Indeed, as before for gO 2 G and  2 L2 .R/, O D e i.e  .g/.x/

i je . 0 x/.1i ˛/ z

0/

.x  0 / .x 2 R/:

Finally, by carrying  by the mapping which sends  2 L2 .R/ to ˚ 2 L2 .RI e 2x dx/; ˚.x/ D e .1Ci /x .x/; we get O and finish the proof.



8.1 Monomial Representations

299

The reasoning the way of which we mentioned above leads us to the following result. Take a finite measure Q on  equivalent to the Lebesgue measure and regard it a measure on g . Let  D O ./ Q be the image of Q by the Kirillov–Bernat O For  2 G, O we denote by ˝./ D ˝G ./ the coadjoint mapping O W g ! G. orbit of G corresponding to  and by m./ the number of the H -orbits contained in  \ ˝./. By Corollary 8.1.7, this m./ is also given as follows. When each connected component of  \ ˝./ is a manifold of dimension 12 dim ˝./, it is the number of the connected components. Otherwise, it is C1. Theorem 8.1.9 ([14, 32]). Let G D exp g be an exponential solvable Lie group with Lie algebra g, f 2 g , h 2 S.f; g/ and H D exp h. Then, the monomial representation  D .f; O h; G/ D indG H f is decomposed as Z '

˚ GO

m./d./:

(8.1.1)

Proof. Let us proceed by induction on dim G. We first suppose g ¤ h C Œg; g. Then there is an ideal g0 of codimension 1 in g containing h and we can apply the induction hypothesis to G0 D exp.g0 /. The coadjoint orbit ˝ of G is either saturated or non-saturated regarding the restriction mapping p W g ! g0 and the same eventuality occurs almost everywhere for Q [42]. Now, by Lemma 8.1.4 and Proposition 8.1.5, we know the following facts. If ˝ is saturated, p.˝/ consists of a one-parameter family f!t gt 2R of G0 -orbits and dim ˝ D dim.!t / C 2; OG .˝/ ' indG G0 OG0 .!t / for all t 2 R. On the contrary, if ˝ is non-saturated, ! D p.˝/ is a G0 -orbit, p gives a diffeomorphism from ˝ onto !, p 1 .!/ consists of a one-parameter family f˝t gt 2R of G-orbits and Z indG G0 OG0 .!/ '

˚ R

OG .˝t /dt:

Combining this with the induction hypothesis, we immediately obtain the assertion on the measure . Let us examine the multiplicity. In the case where almost all coadjoint G-orbits are non-saturated, this claim also would follow immediately from the above fact combined with the induction hypothesis. In the case where almost all coadjoint G-orbits are saturated, we set f0 D p.f / 2 g0 ; 0 D 0 ?;g 0 , and write the canonical irreducible decomposition indG H f0 ; 0 D f0 C h of 0 as Z 0 '

˚

b0 G

m0 .0 /0 d0 .0 /:

300

8 Irreducible Decomposition

c0 and GO is except for a null Since G0 is a normal subgroup of G, G acts as usual on G c0 =G. If we decompose the measure 0 set for  identified with the quotient set G c0 ! G c0 =G, with respect to , which is nothing but its image by the projection q W G Z  ' indG G0 0 ' Z '

  m0 .0 / indG G0 0 d0 .0 /

b0 G Z ˚

˚ GO

˚

d./

b0 G

   m0 .0 / indG G0 0 d0 .0 /;

where 0 is a measure on the fibre q 1 ./. As we have seen until now, the irreducible representation which appears in the representation Z

˚

b0 G

   m0 .0 / indG G0 0 d0 .0 /

is only  and Z m./ '

˚

b0 G

   m0 .0 / indG G0 0 d0 .0 /

for almost all  regarding . Therefore, m./ becomes finite for such  when and only when the measure 0 is supported by finite 0 such that m0 .0 / is finite. And 1 in that case, if we write the G0 -orbit OG .0 / as !.0 /, the multiplicity m0 .0 / is 0 given by the number of the connected components of 0 \ !.0 /. Combining these observations with Corollary 8.1.7, we know that q 1 ./ is a finite set and X m./ D m0 .0 /: 0 2q 1 ./

For each connected component C of  \ ˝./, p.C / coincides with a connected component of 0 \ !.0 / corresponding to a certain 0 2 q 1 ./ and, since the latter is a manifold of dimension 12 dim.!.0 //, C itself turns out to be a manifold of required dimension. Besides, m./ becomes equal to the number of the connected components of  \ ˝./. From now on we suppose g D h C Œg; g. Recall that we may assume that h contains the centre z of g. Moreover, if h \ kerf contains a non-trivial ideal a of g, we may apply the induction hypothesis to the quotient Lie group G=exp a. Thus, we may assume dim z  1. Let a be a minimal non-central ideal of g and A D exp a. By Lemma 5.3.10 of Chap. 5, a is commutative and dim a  3. The adjoint action of g on a=.a \ z/ produces a root  2 g and there is by our assumption T 2 h verifying .T / D 1, while g0 D ker is an ideal of g so that g D RT ˚ g0 . Let us examine the situations case by case.

8.1 Monomial Representations

301

(i) Suppose a \ z D f0g. Then h \ a D f0g. We set h0 D .h \ g0 / C a and H 0 D exp.h0 /. As far as we reason at the stage of the subgroup K D exp k; k D h C a; the situation is totally the same as (1) or (2) of Lemma 8.1.8 according to the dimension of a. Provided dim a D 1, we write a D RY and take f˙ 2  such that f˙ .Y / D ˙1. Clearly h0 2 S.f˙ ; g/ and the theorem is already established for the pair .f˙ ; h0 / because g ¤ h0 C Œg; g. So, on the understanding of our notations concerning the multiplicity and the measure, G  ' indG H 0 fC ˚ indH 0 f Z ˚ Z ' mC ./dC ./ C GO

˚

m ./d ./:

GO

(8.1.2)

Now put P˙ D f` 2 g I `.Y / ? 0g ; ˙0 D f˙ C h0 ? ; ˙0 D f˙ C k? and ˙ D  \ P˙ . Then ˙ D exp.RT /  ˙0 ; ˙0 D A  ˙0 and P˙ are G-stable. In this way formula (8.1.2) furnishes the desired result. When dim a D 2, we choose Y1 ; Y2 2 a so that the three elements T; Y1 ; Y2 satisfy the commutation relation of (2) in Lemma 8.1.8. Take O 2  such that O a D .cos /Y  C .sin /Y  : j 1 2 O g/. Similarly to (2) in Lemma 8.1.8, Evidently h0 2 S.; Z

Z

˚

' Œ0;2/

indG H 0 O d

Z

˚

'

d Œ0;2/

˚ GO

m ./d ./:

(8.1.3)

We set 0 D O C h0 ? and 0 D O C k? . Now `ja ¤ 0 .` 2 ˝/, hence we take ˝ 2 g =G such that dim .g.`/ \ a/ D 1. For ˝ like this, there uniquely exists  2 Œ0; 2/ so that      \ ˝ D exp.RT /  0 \ ˝ ; 0 \ ˝ D A  0 \ ˝ : Taking these observations into account, formula (8.1.3) gives exactly the desired result. (ii) Let us examine the case where a \ z D z ¤ f0g. Suppose first that a  h. If we introduce the proper Lie subalgebra g1 D af of g, then h  g1 . Applying the induction hypothesis to the subgroup G1 D exp.g1 / Z 1 D

1 indG H f

'

˚

b1 G

m1 ./d1 ./;

302

8 Irreducible Decomposition

and so Z  ' indG G1 1 '

˚

b1 G

  m1 ./ indG G1  d1 ./:

(8.1.4)

Here, since jA is a multiple of f jA , Theorem 3.4.4 of the third chapter assures that these indG G1  are irreducible and non-equivalent to each other, whereas, A  ` D ` C .af /? for any ` 2  . Finally, formula (8.1.4) provides exactly the desired result. Next we treat the case where h \ a D z. Suppose first that h0 D h \ g0  g1 . O g/ according Since Œh0 ; a D f0g, as we described before h0 2 S.f˙ ; g/ or h0 2 S.; to the dimension of a. Further by the above arguments, we are led to the irreducible decomposition (8.1.2) if dim a D 2 and (8.1.3) if dim a D 3. However, this time a more complicated situation occurs. Assume dim a D 2 and take Y 2 a such that ŒT; Y  D Y . With the previous notations the two sets P˙ are no longer G-invariant and the two direct integrals Z

Z

˚ GO

mC ./dC ./ and

˚ GO

m ./d ./

in formula (8.1.2) could have common irreducible components. Anyhow, passing to the quotient space regarding the action of G, we get the assertion on the measure. Now concerning the multiplicity, Lemma 8.1.8(1) gives m./ D mC ./ C m ./: Let ˝ 2 g =G be such that each connected component C 0 of ˙0 \˝ is an H 0 -orbit and that each connected component of  \ ˝ is a manifold, then   exp.RT /  ˙0 \ C 0 D H  `; ` 2 ˙0 \ C 0 ; is the connected component of  \ ˝ containing ˙0 \ C 0 . This means it is impossible that such an H  ` contained in C \ ˝ and another H  `0 contained in  \ ˝ could be included in the same connected component of  \ ˝. Indeed, this last connected component is by assumption a manifold and its dimension is necessarily 12 dim ˝. By Corollary 8.1.7(3), we obtain the result almost everywhere with respect to . This finally implies that for such an ˝ the connected components of  \ ˝ contained in the hyperplane f` 2 g I `.Y / D 0g do not contribute by Corollary 8.1.7(1). We have finished the proof in this case. Next assume dim a D 3 and take Y1 ; Y2 2 a so that the three elements fT; Y1 ; Y2 g satisfy the commutation relations in Lemma 8.1.8(2). We already know that

8.1 Monomial Representations

303

Z

Z

˚

'

d Œ0;2/

˚

m ./d ./

GO

but, for 1 ¤ 2 , the elements of 01 and those of 02 are mixed by the action of G. Nevertheless, the statement concerning the measure is established by passing to the quotient space by the equivalence relation. Regarding the multiplicity, if we denote by ./ the set of  2 Œ0; 2/ such that 0 \ ˝./ ¤ ;, Corollary 8.1.7(1) means m./ D

X

m ./

 2./

when ./ is a countable set, otherwise m./ D C1. Let us examine in more detail the case where ./ is a countable set. Let C be a connected component of  \ ˝./. For all ` 2 C , the restriction of ` to V D RY1 C RY2 belongs to [

exp.RT /  .jV /

 2./

unless `jV D 0. Furthermore, we suppose that each connected component C 0 of 0 \˝;  2 ./; is one H 0 -orbit, i.e. C 0 D H 0 `0 with `0 2 C 0 satisfying `0 .T / D f .T /, and that each connected component of  \ ˝./ is a manifold. Exactly as in the preceding case, we understand that H  `0 is the connected component of  \ ˝./ containing `0 and that we may neglect the connected components contained in the subspace f` 2 g I `jV D 0g. In this way we obtain the statement on the multiplicity. Let us finally consider the case where h0 6 g1 . The situation is essentially similar to that in Lemma 8.1.8(3) or (4). If dim a D 2, take Z 2 z; Y 2 a n z and X 2 h0 in such a fashion that the four elements fT; X; Y; Zg satisfy the commutation relations similar to those in Lemma 8.1.8(3), while we replace ŒT; X  D X by the relation ŒT; X  X modulo g0 \ g1 . Likewise, if dim a D 3, we choose Z 2 z; Y1 ; Y2 2 a n z and X1 ; X2 2 h0 so that these elements together with T satisfy the commutation relations similar to those in Lemma 8.1.8(4), while modulo g0 \ g1 , ŒT; X1  X1  ˛X2 ; ŒT; X2  X2 C ˛X1 .0 ¤ ˛ 2 R/. If we set h0 D .h \ g1 / C a; H 0 D exp.h0 /, clearly h0 2 S.f; g/. Arguing at the stage of K D exp k; k D h C a; Proposition 5.3.18 in Chap. 5 gives  ' indG H 0 f : Thus, this is reduced to the case already treated and, for .f; h0 /, Z ˚  ' m0 ./d0 ./  ' indG 0 H f GO

holds with the measure 0 and the multiplicity m0 ./ given by the theorem.

304

8 Irreducible Decomposition

To finish the proof of the theorem it is enough to notice the following. ˝ 2 g =G being arbitrarily given, each connected component C of  \ ˝ is written as C D .exp U /C0 with a connected component C0 of 0 \ ˝; 0 D f C k? , where U D RX or U D RX1 C RX2 . Similarly, a connected component C 0 of  0 \ ˝;  0 D f C h0 ? is written as C 0 D .exp V /C0 with V D RY or V D RY1 C RY2 .  Let us generalize this result a little bit. Starting from a subgroup K D exp k and  2 KO and examine the induced representation indG K . We denote by !./ 1 the coadjoint orbit OK ./  k of K corresponding to  and by p W g ! k the restriction mapping. A K-invariant measure on !./ and a Lebesgue measure on k? furnish the measure O on the submanifold p 1 .!.// of g . Let Q be a finite O measure on g equivalent to , O and put  D G D .OG / ./. Q Moreover, for  2 G, we denote by n ./ the number of the K-orbits contained in  .; / D ˝./ \ p 1 .!.//. Using the fact that  is a monomial representation of K, we generalize Theorem 8.1.9: Theorem 8.1.10 ([14, 34]). Let G D exp g be an exponential solvable Lie group with Lie algebra g, K D exp k an analytic subgroup of G and  an irreducible unitary representation of K. Then, Z indG K '

˚ GO

n ./d./:

8.2 Restriction of Unitary Representations to Subgroups To begin, we prepare Mackey’s subgroup theorem [55]. However, most of the situations where we shall employ this theorem are very simple and we will not really need this theorem. Let H1 ; H2 be closed subgroups of a locally compact topological group G. Definition 8.2.1. We denote by H2 nG=H1 the space of the .H2 ; H1 /-double cosets and by p W G ! H2 nG=H1 the canonical projection. We introduce in H2 nG=H1 the Borel structure induced by the map p from that of G. We take on G a finite Borel measure  equivalent to the Haar measure and define the Borel measure N on H2 nG=H1 by .E/ N D .p 1 .E//. When N is a standard Borel measure, i.e. up to a -null N set H2 nG=H1 is a standard Borel space, H1 and H2 are said to be regularly related. Theorem 8.2.2 (Subgroup Theorem). Suppose that H1 and H2 are regularly related. Let  be a unitary representation of H1 and  D indG H1 . Besides, g .g 2 G/ denotes the unitary representation of the closed subgroup gH1 g 1 defined by the formula g .h/ D .g 1 hg/. The unitary representation g of H2 induced from the restriction of g to H2 \ gH1 g 1 is up to equivalence determined

8.2 Restriction of Unitary Representations to Subgroups

305

by the double coset Œg D p.g/ of g, and we have Z jH2 '

˚

H2 nG=H1

g d .Œg/: N

We only sketch a proof of the theorem. For a Borel set E of H2 nG=H1 , we denote by 'E the characteristic function of the set p 1 .E/ and consider the linear transformation of H defined by .PE /.g/ D 'E .g/ .g/ .g 2 G/. Evidently, PE is a projection in H and its range is stable by the actions of the representation jH2 . Next let us compute the actions of jH2 in this range. Taking a Borel cross-section s W E ! p 1 .E/, we choose a representative element g 2 G of a double coset C contained in E in such a way that g 2 s.C /. Hence C D H2 gH1 . If we set '.h/ Q D '.hg/ .h 2 H2 / for ' belonging to the range of PE , then 'Q is a mapping from H2 to H . As is easily seen, with h 2 H2 ; x 2 H2 \ gH1 g 1 , 1

1

Q '.hx/ Q D '.hxg/ D '.hgg1 xg/ D .g 1 xg/ '.hg/ D .g 1 xg/ '.h/: Conversely, let be a mapping from H2 to H which satisfy this relation on each double coset C contained in E. Then, putting '.h2 gh1 / D .h1 /1 .h2 / .h1 2 H1 ; h2 2 H2 / on p 1 .C / and '.x/ D 0 .x 62 p 1 .E//, we get ' belonging to the range of PE such that 'Q D . The map ' ! 'Q is linear and if, for ' belonging to the range of PE , 'Q D 0 holds on each double coset contained in E, then ' D 0. Namely, the linear mapping ' ! 'Q gives a bijection from the range of PE to the representation space of the field of unitary representations Œg ! g on E. This gives us the desired equivalence relation. Returning to the situation mentioned at the beginning of the previous section and using the notations employed in Proposition 8.1.5, we would like to examine the restriction jG0 of  2 GO to G0 . Proposition 8.2.3 ([42]). Let G D exp g be an exponential solvable Lie group with Lie algebra g and G0 D exp.g0 / an analytic normal subgroup of codimension 1 in G. We denote by ; O O0 the Kirillov–Bernat mappings for G; G0 . Let  D .`/ O with ` 2 g and put `0 D `jg0 . (1) When the orbit G` is saturated in the direction of g? 0 , Z jG0 '

˚ R

O0 .`s /ds:

Here `s D exp.sX /`0 with X 2 gng0 . (2) When the orbit G` is non-saturated in the direction of g? 0 , jG0 ' O0 .`0 /.

306

8 Irreducible Decomposition

Proof. (1) Let us use Mackey’s subgroup theorem although we do not necessarily need it. Take a Vergne polarization h at ` of g constructed from a strong Malcev sequence of Lie subalgebras passing g0 and put H D exp h. Then h 2 I.`; g/ and h  g0 . We see that H and G0 are regularly related. Since H  G0 , the double coset HgG0 D HG0 g D G0 g is nothing but the coset modulo G0 . Therefore, the space of the double cosets becomes the group G=G0 and there exists a countable separating family of Borel sets. Thus the subgroups H and G0 are regularly related. The group G=G0 is identified with R by the mapping s 7! exp.sX /G0 and we can take a Haar measure on G=G0 , hence a Lebesgue measure on R, as a measure used in the direct integral decomposition [54] so that Z jG0 '

˚ R

Vt dt:

Here Vt is the representation of G0 induced from the representation t W g0 7! ` .exp.tX /g0 exp.tX // of the subgroup Ht D G0 \ .exp.tX /H exp.tX // D exp.tX /H exp.tX / of G0 . However, t is nothing but exp.tX /` D exp.tX /` . Hence,   G0 0 ' exp.tX /  O0 .`0 /: Vt ' indG  ' exp.tX /  ind  exp.tX /` ` Ht H In this way Vt is irreducible and the Lie algebra of Ht belongs to I.exp.tX /  `0 ; g0 /. Thus, Vt ' O0 .exp.tX /  `0 /. Putting `s D exp.sX /  `0 , we obtain the desired decomposition: Z jG0 '

Z

˚ R

O0 .`s /ds '

˚ R

O0 .`s /ds:

(2) If we construct a Vergne polarization h 2 I.`; g/ just as above, h0 D h \ g0 2 I.`0 ; g0 / and h D h0 C g.`/. Since h0 is an ideal of h, H D H0 exp.RX / with H0 D exp.h0 /. We again apply (although not strictly necessary) Mackey’s subgroup theorem to the pair .H; G0 /. Here, since HG0 D H0 .exp.RX /G0 / D G, there is only one double coset so that H and G0 are regularly related. Thus 0 jG0 ' indG G0 \H `0 ;

while G0 \ H D H0 and h0 2 I.`0 ; g0 /, so

8.2 Restriction of Unitary Representations to Subgroups

307

0 indG G0 \H `0 ' O0 .`0 /:

Hence jG0 ' O0 .`0 / and the assertion follows.



Now let K D exp k be a subgroup of G and p W g ! k the restriction map. Let  2 GO be given. We take in g a finite measure Q D Q  equivalent to the G-invariant  measure on the orbit ˝./ D ˝G ./ D O1 ./ and set D K D .OK ıp/ . /. Q Making use of the measure obtained in this manner on KO and the same multiplicity n ./ as in Theorem 8.1.10, we procure the canonical irreducible decomposition of the restriction jK of  to K. Theorem 8.2.4 ([15, 34]). Let G D exp g be an exponential solvable Lie group with Lie algebra g and K D exp k an analytic subgroup of G. We take an irreducible unitary representation  of G. The restriction jK of  to K is decomposed as Z jK '

˚

KO

n ./d ./:

Proof. As usual we employ the induction on the dimension of G. First of all, Proposition 8.2.3 offers the expected result when k is an ideal of codimension 1. Suppose that k is contained in an ideal g0 of codimension 1, and put G0 D exp.g0 /. By Proposition 8.2.3 and the induction hypothesis applied to G0 , jK D .jG0 / jK is equivalent to either Z

˚ R

 Z O0 .`s /ds jK '

Z

˚ R

.s jK / ds '

Z

˚

ds R

˚ KO

s ns ./d K ./

(8.2.1)

with s D O0 .`s / or Z

˚ KO



n ./d K ./

c0 . with  D jG0 2 G From this we get the statement about the measure . Besides, the claim concerning the multiplicity also follows immediately in the second case. Regarding the multiplicity in the first case, it is clear if the K-orbits are saturated in the direction of g? 0 almost everywhere on ˝./. Assume that the K-orbits are nonsaturated almost everywhere. Then for almost all s 2 R there exists X.s/ 2 g n g0 O we know that the number so that X.s/  `s 2 k? in g0 . Hence, for  D OK .`jk / 2 K, n ./ of the K-orbits contained in ˝./ \ p 1 .!.// and the multiplicity of  in expression (8.2.1) are both C1. Provided k is contained in a proper ideal m of g, pQ W g ! gQ D g=m being the canonical map, we take an ideal gQ 0 of gQ having the codimension 1 and containing ŒQg; gQ . If we put g0 D pQ 1 .Qg0 /, then g0 is an ideal of codimension 1 in g which contains k.

308

8 Irreducible Decomposition

From now on we suppose that K is not contained in any proper normal subgroup of G. Besides, it is easily seen that we may assume k contains the centre z of g. We assume this in what follows. Using a polarization b at f 2 ˝./, we realize  as  ' indG B f ; B D exp b. We further assume that kerf does not contain any non-trivial ideal of g, otherwise we may pass to the quotient Lie algebra by such an ideal and apply the induction hypothesis. Consequently, dim z  1 and f jz ¤ 0 if dim z D 1. Take a minimal non-central ideal a of g and put A D exp a; g1 D af ; G1 D exp.g1 /. Provided g1 D g, f would vanish on the ideal Œg; a ¤ f0g of g, which is a contradiction. Hence g1 is a proper Lie subalgebra of g. By Proposition 5.3.18 in Chap. 5, b can be taken in g1 and  ' indG G1 1 1 with 1 D indG  . Taking into account the existence of a coexponential basis and B f the proof of Proposition 5.3.18(2), we know that it is possible to choose a linear subspace t complementary to g1 in g in such a fashion that G D T G1 ; T D exp t. Hereafter, we will do case-by-case studies. (1) First of all we treat the case where k is coirreducible, i.e. g=k is irreducible as an k-module. Under this condition g D k C a or a  k. (a) Assume g D k C a. Clearly G D KG1 and by Mackey’s subgroup theorem [55] jK ' indK K1 .1 jK1 / with K1 D K \ G1 . We write the Lie algebra of K1 as k1 . In this case 1 jA is a multiple of f jA and G1 D K1 A so that 0 D 1 jK1 is irreducible and corresponds to the coadjoint orbit of K1 passing f jk1 2 k1 . From the decomposition theorem 8.1.10 of induced representations, Z jK '

˚ KO

m./dK0 ./

(8.2.2)

holds. Here we recall the formula which gives the multiplicity m./. If we denote by p.k; k1 / the restriction map from k to k1 and put  D p.k; k1 /1 .˝K1 .0 //  k , m./ is obtained as the number n .0 / of the K1 -orbits contained in ˝K ./ \ . Let us interpret formula (8.2.2). Taking t in k, k D t C k1 and K D TK1 . Then   ˝./ D T  K1 f C g? 1 ;   p.g; k/ .˝.// D T  K1  .f jk1 / C k? 1 D T : Hence the quotient space p.g; k/ .˝.// =K is identified with =K, the  . As for the multiplicity, fixing arbitrarily measure K0 is qualified as K the value on a, we know n .0 / D n ./ for  belonging to the support of K0 .

8.2 Restriction of Unitary Representations to Subgroups

309

(b) Next assume a  k and divide this case into two subcases. (i) If k 6 g1 D af for any choice of f 2 ˝./, the situation is similar to that examined above. As before, we can take t in k so that G D KG1 . Using Mackey’s subgroup theorem jK ' indK K1 .1 jK1 / with K1 D exp.k1 / D K\G1 . In general 1 jK1 is no longer irreducible. By the induction hypothesis Z 1 jK1 '

˚

b1 K

1 n1 ./d K ./: 1

Therefore Z jK '

indK K1

˚

b1 K

Z 1 n1 ./d K ./ 1

'

˚

b1 K

  1 n1 ./ indK K1  d K1 ./: (8.2.3)

Let us examine formula (8.2.3) in more detail. Since jA is a multiple of f jA and K1 coincides with the stabilizer of f jA in K, Theorem 3.4.4 in the third chapter tells us that all  D indK K1  are irreducible and non-equivalent to each other. Putting f1 D f jg1 2 g1 we embed g1 as the subspace f 2 g I jt D 0g of g . Let us designate the coadjoint action of G .resp: G1 / by  .resp:  /. Then G

G1

    D T  A  ˝./ D Gf D T  G1  f1 C g? G  f 1 1 1 G1

G

G1

and the quotient space p.g; k/ .˝G .// =K is identified with p.g1 ; k1 / .˝G1 .1 // =K1 : In these circumstances the action of T varies the value on a and we 1  qualified as K . Moreover, paying attention to the find the measure K 1 intersection of each orbit with the affine subspace p.k; a/1 .f ja / D .f jk / C a?  k ; we recognize that the multiplicity n1 ./ is equal to n ./.

310

8 Irreducible Decomposition

(ii) Let us pass to the case where k  g1 D af for some f 2 ˝./. We denote by g0 the centralizer of a in g and put k0 D k \ g0 . By our assumption f ja ¤ 0 and k is not contained in the ideal g0 . So, referring to the structural analysis concerning the minimal non-central ideal on the way of the proof of Theorem 5.3.8 in Chap. 5, this case occurs only when g0 is a proper Lie subalgebra of g1 and dim.k=k0 / D dim.a \ z/ D dim z D 1: Put K0 D exp.k0 / and notice that the theorem holds for jK0 because k0 is contained in a proper ideal of g. Now set the desired irreducible decomposition as Z jK '

˚ KO

m0 ./d 0 ./:

(8.2.4)

Restricting to K0 the both sides of this formula Z jK0 '

˚ KO

m0 ./ .jK0 / d 0 ./:

(8.2.5)

But this ought to give the expected formula for jK0 . Applying Proposi tion 8.2.3 to the pair .k; k0 /, we see that the measure 0 is equivalent to K . If an element of  D p.g; k/ .˝.//  k satisfies ja ¤ f ja , then A  D C k? 0 . So, case (2) of Proposition 8.2.3 occurs only when ˝K ./ is contained in the affine subspace .f ja / C a?  k . Considering p.k; a/./ D f 2 a I jz D f jzg; case (1) of Proposition 8.2.3 occurs almost everywhere in formula (8.2.5). R˚ Setting jK0 ' R t dt, formula (8.2.5) becomes Z

˚

b0 K

Z 0 n0 ./d K ./ 0

'

˚

KO

0

0

Z

˚

m ./d ./ R

t dt:

All these t being non-equivalent to each other, we get m0 ./ D n0 .t / and this last quantity is equal to n ./. In this way we establish the claim on the multiplicity m0 ./. Finally we understand that formula (8.2.4) has the desired form. Now let us leave the assumption that k is coirreducible. (2) Here we prove the assertion on the measure. Let h be a coirreducible proper Lie subalgebra containing k. Making an intervention of H D exp h, we have from the arguments of (1) mentioned above and the induction hypothesis

8.2 Restriction of Unitary Representations to Subgroups

Z jK ' .jH / jK ' Z '



˚

HO

Z  n ./d H ./

Z jK '

 n ./d H ./

˚

HO

311 ˚ HO

˚

 n ./.jK /d H ./



n ./d K ./:

KO

 Decomposing the measure p.g; h/ . Q  / with respect to H ,

Z p.g; h/ . Q  / D



HO

 Q H d H ./:

Since the equivalence class of the measure Q  is invariant by the action of H , the  fibre measures Q H are almost everywhere equivalent to the H -invariant measure  on ˝H ./. By extending Q H to h , we get the measure Q  . O For a Borel set E  K,   1   K .E/ D Q  p.g; k/1 OK .E/    1  .E/ D Q  p.g; h/1 p.h; k/1 OK  1   .E/ D .p.g; h/ . Q  // p.h; k/1 OK Z  1     Q H ˝H ./ \ p.h; k/1 OK .E/ d H ./ D Z D

HO

 HO

  1   p.h; k/ . Q  / OK .E/ d H ./ D

Z 

HO

 K .E/d H ./:

So, we obtain the result concerning the measure. (3) Let us search for the multiplicity formula when ˝./ is saturated with respect to a certain ideal g0 of codimension 1 in g. Let f 2 ˝./; f0 D p.f /; p D p.g; g0 /; G0 D exp.g0 / and 0 the irreducible unitary representation of G0 corresponding to the orbit G0 f0 . Thus  ' indG G0 0 . Because we assume k 6 g0 , we confirm that G D KG0 . Putting K0 D K \ G0 D exp.k0 /, we utilize once again Mackey’s subgroup theorem and apply the induction hypothesis to find Z K jK ' indK K0 .0 jK0 / ' indK0

Z '

˚

b0 K

˚

b0 K

0 n0 ./indK K0 d K0 ./:

0 n0 ./d K ./ 0

(8.2.6)

R˚ K By Proposition 8.1.5, either  D indK K0  is irreducible or indK0  ' R t dt. In the latter case, we easily verify that the multiplicity of t in formula (8.2.6) coincides with n .t /. Let us compute the multiplicity of  2 KO in the former  case. For  2 KO contained in the support of K , we denote by m./ the number

312

8 Irreducible Decomposition

of the K0 -orbits contained in G0 f \ p.g; k/1 .˝K .//. As Gf D KG0 f , n ./ is nothing but m./. Replacing f if necessary, we consider the situation where f jk 2 ˝K ./. For all  2 g such that p./ 2  .0 ; / D ˝G0 .0 / \ p.g0 ; k/1 .˝K .// ; if p 1 p./  K, m./ is equal to the sum of n0 ./ in formula (8.2.6) 1 regarding  such that  ' indK K0 . Provided p .f0 / 6 Kf , clearly n ./ D K C1. Now we assume  ' indK0 . We have   dim Gf0 \ p.g0 ; k0 /1 .˝K0 .// D dim.Gf0 /  dim.K0 f / C dim.˝K0 .//; dim. .0 ; // D dim.G0 f0 /  dim.K0 f0 / C dim.˝K0 .//;

while dim.K0 f / D dim.K0 f0 /; dim.Gf0 / D dim.G0 f0 / C 1 by our assumption, and   dim. .0 ; // < dim Gf0 \ p.g0 ; k0 /1 .˝K0 .// :

(8.2.7)

Set k D RX C k0 and k.s/ D exp.sX / for any s 2 R. Even if we replace f by k.s/f , the inequality (8.2.7) remains still valid. On the other hand, [  .k.s/˝G0 .0 // \ p.g0 ; k0 /1 .˝K0 .// : Gf0 \ p.g0 ; k0 /1 .˝K0 .// D s2R

In the sequel, .k.s/˝G0 .0 // \ p.g0 ; k0 /1 .˝K0 .// ¤ ;; That is, the set of all s 2 R satisfying ˝G0 .0 / \ p.g0 ; k0 /1 .k.s/˝K0 .// ¤ ; is not a null set for the Lebesgue measure. This means that  appears with multiplicity C1 in the irreducible decomposition of jK . Thus we procure the desired result in any case. (4) We finally suppose that ˝./ is non-saturated with respect to any ideal of codimension 1. Let us utilize again a coirreducible proper ideal h containing k. As usual, put p D p.g; h/ and H D exp h. Let us take up the former formula Z jK D .jH /jK ' Z '

˚ HO

 n ./.jK /d H ./

˚ HO

 n ./d H ./

Z

˚

KO



n ./d K ./:

(8.2.8)

8.2 Restriction of Unitary Representations to Subgroups

313

If we have   H f jh f  Kf for all f 2 ˝./ such that f jk 2 ˝K ./, we get the multiplicity formula in (8.2.8) by counting the K-orbits in  .; /. It remains only for us to check the following. Provided dim.Kf / < dim. .; // for f 2  .; /, the multiplicity m./ of  in formula (8.2.8) is C1. Here we examine the case where dim .g=h/ D 2. The case where dim .g=h/ D 1 can be treated in exactly the same way. Choose a coexponential basis fX1 ; X2 g to h in g. Namely, the mapping R2  H 3 ..s; t/; h/ 7! g.s; t/h; g.s; t/ D exp.sX1 /exp.tX2 /; is a diffeomorphism onto G. Put f 2  .; /; fs;t D g.s; t/f and fs;t0 D p.fs;t /, then [ H fs;t0 : p .˝.// D .s;t /2R2

Since g0 D fX 2 gI ŒX; g  hg is an ideal of codimension 1 and by assumption ˝./ is non-saturated with respect to g0 , there are two H -invariant possibilities for  2 p .˝.//: p 1 ./ \ ˝./ D fg or p 1 ./ \ ˝./ D p 1 ./: We designate by E .resp: F / the set of all .s; t/ 2 R2 such that fs;t0 verifies the first (resp. second) eventuality. Remark that E is an open set of R2 . Regarding .s; t/ 2 E, the condition H fs;t0 \ p.h; k/1 .˝K .// ¤ ; implies 00 1 1 [   dim Kfs;t0 < dim @@ H fs;t0 A \ p.h; k/1 .˝K .//A :

(8.2.9)

.s;t /2E

If there exists .s; t/ 2 E satisfying      dim Kfs;t0 < dim H fs;t0 \ p.h; k/1 .˝K .// ; it turns out that n ./ is infinite for  2 HO corresponding to H fs;t0 . Otherwise, by relation (8.2.9), the set of .s; t/ 2 E satisfying

314

8 Irreducible Decomposition



 H fs;t0 \ p.h; k/1 .˝K .// ¤ ;

is not a null set, and the desired result follows. Therefore let us assume 0 1 [ @ H fs;t0 A \ p.h; k/1 .˝K .// D ;: .s;t /2E

Then we know immediately that   dim p .˝.// \ p.h; k/1 .˝K .// D dim. .; //  2 D dim.˝.//  dim.Kfs;t / C dim.˝K .//  2;

(8.2.10)

while        dim H fs;t0 \ p.h; k/1 .˝K .// D dim H fs;t0  dim Kfs;t0 C dim.˝K .// (8.2.11) and

  dim .Kfs;t /  dim Kfs;t0 C 2:

(8.2.12)

  Since dim H fs;t0 D dim.˝.//  4, the formulas (8.2.10)–(8.2.12) give:    dim H fs;t0 \ p.h; k/1 .˝K .//  dim.˝.//  4  .dim .Kfs;t /  2/ C dim.˝K .// D dim.˝.//  dim .Kfs;t / C dim.˝K .//  2   D dim p .˝.// \ p.h; k/1 .˝K .// :

(8.2.13)

If the equality holds in the inequality (8.2.13), it does in the inequality (8.2.12) too. In such a case,      dim Kfs;t0 D dim .Kfs;t /  2 < dim H fs;t0 \ p.h; k/1 .˝K .// and from this n ./ D C1 for  corresponding to the orbit H fs;t0 . If the equality in the inequality (8.2.13) holds nowhere, the relation p .˝.// \ p.h; k/1 .˝K .// D

[   H fs;t0 \ p.h; k/1 .˝K .// .s;t /2F

assures that the set of all .s; t/ 2 R2 verifying

8.2 Restriction of Unitary Representations to Subgroups



315

 H fs;t0 \ p.h; k/1 .˝K .// ¤ ;

is not a null set for the Lebesgue measure. From this we immediately get m./ D C1 and finish the proof of the theorem.  We generally consider a closed subgroup K of a Lie group G. Let  be an irreducible unitary representation of K and  that of G. We call the Frobenius reciprocity the phenomenon that the multiplicity of  in the irreducible decomposition of the induced representation indG K  is equal to the multiplicity of  in the irreducible decomposition of the restriction jK . Of course, when these irreducible decompositions are given by direct integrals, there is no meaning in talking about the multiplicity of a separate irreducible representation. Therefore in such a case we say that the Frobenius reciprocity holds when the multiplicity functions of indG K and jK , the former being a function of  with parameter  and the latter being a function of  with parameter , have the same expression. Corollary 8.2.5. The Frobenius reciprocity holds in these circumstances. Let j .j D 1; 2/ be two irreducible unitary representations of G. The outer Kronecker product 1  2 of 1 and 2 corresponds to the orbit ˝G1 G2 .1  2 / D .˝.1 /; ˝.2 //  g ˚ g : We identify G with the subgroup of G G consisting of all diagonal elements. Since the (inner) Kronecker product 1 ˝ 2 of 1 and 2 is by definition the restriction of 1  2 to G: Corollary 8.2.6 ([34]). Let p W g ˚ g ! g be the restriction map. Then Z 1 ˝ 2 '

˚ GO

m./d ./;

Here D .OG ıp/ . Q 1 2 / and the multiplicity m./ is obtained as the number of all G-orbits contained in .˝.1 /; ˝.2 // \ p 1 .˝G .//.

Chapter 9

e-Central Elements

9.1 Fundamental Result of Corwin and Greenleaf In order to practise a more detailed analysis of monomial representations, we suppose in this chapter that G D exp g is a connected and simply connected nilpotent Lie group with Lie algebra g. Let us introduce e-central elements due to Corwin and Greenleaf [17]. Let f0g D g0  g1      gn1  gn D g; dim.gk / D k .0  k  n/

(9.1.1)

be a composition series of ideals of g. Let fXj g1j n be a Malcev basis of g according to this composition series, i.e. Xj 2 gj ngj 1 .1  j  n/ and fXj g1j n its dual basis in g . We denote the coordinates of ` 2 g by      ? .`1 ; : : : ; `n /; `j D `.Xj /. Then g? j D hXj C1 ; : : : ; Xn iR  g ; gj Š g =gj     and the projection pj W g ! gj intertwines the actions of G on g and gj .   For ` 2 g , we define ej .`/ D dim Gpj .`/ ; e.`/ D .e1 .`/; : : : ; en .`// and set E D fe.`/I ` 2 g g. We also recognize ej .`/ D dim.Gj `/ with Gj D exp.gj /. In fact,       dim Gpj .`/ D dim g=g`j D dim.g=g.`//  dim g`j =g.`/    D dim.g=g.`//  dim.g=g.`//  dim gj =gj .`/ D dim.Gj `/: For e 2 E we define the G-invariant layer Ue D f` 2 g I e.`/ D eg. With e0 D 0 we define the set of jump indices S.e/ D f1  j  nI ej D ej 1 C 1g and that of nonjump indices T .e/ D f1  j  nI ej D ej 1 g. U.g/ being the enveloping algebra of gC , A 2 U.g/ is called an e-central element if, with ` D OG .`/, ` .A/ D d ` .A/ is a scalar operator for any ` 2 Ue .

© Springer Japan 2015 H. Fujiwara, J. Ludwig, Harmonic Analysis on Exponential Solvable Lie Groups, Springer Monographs in Mathematics, DOI 10.1007/978-4-431-55288-8__9

317

9 e-Central Elements

318

Now we mention a fundamental result of Corwin and Greenleaf. There exists a Zariski open set Z of g which satisfies the following. Z \ Ue is a non-empty G-invariant set and for each j 2 T .e/ there is Aj 2 U.gj / with the following properties. (a) Each Aj is e-central on Z \ Ue , i.e. ` .Aj / is a scalar operator for ` 2 Z \ Ue . Furthermore, Aj has a form Aj D Pj Xj C Qj such that: 1. Pj is a polynomial of Ak .k 2 T .e/; k < j /, in particular Pj 2 U.gj 1 /; 2. Pj is e-central on Z \ Ue ; 3. Qj 2 U.gj 1 /, in particular P1 ; Q1 2 C1. (b) ` .Pj / ¤ 0 at 8 ` 2 Z \ Ue . (c) ` .Aj / D 'j .`/Id , where we can write 'j .`/ D pQj .`0 /`j C qQj .`0 / with two rational functions pQj ; qQj on Z \ Ue , which depend only on `0 D .`1 ; : : : ; `j 1 /. (d) pQj .`0 / is G-invariant and pQj .`0 / ¤ 0 for 8 ` 2 Z \ Ue . Remark 9.1.1. We can repeat again the construction of these elements Aj on Ue n.Z \ Ue /. Let us return to a monomial representation  D indG H f , where H D exp h with h 2 S.f; g/, and consider the algebra D .G=H / of G-invariant differential operators on the line bundle with base space G=H associated with data .H; f /. We fix a basis fYs g1sd of h and define the vector subspace a D

d X

C .Ys C if .Ys //

sD1

of U.g/. Let U.g/a be the left ideal of U.g/ spanned by a and set U.g; / D fA 2 U.g/I ŒA; Y  2 U.g/a ; 8 Y 2 hg: The elements of U.g/ act as left G-invariant differential operators, i.e. .R.X / / .g/ D

d .gexp.tX //jt D0 .8 g 2 G/ dt

for X 2 g and 2 C 1 .G/. Now the algebra D .G=H / is the image of the map R W U.g; / 3 A 7! R.A/ and the kernel of this map is U.g/a . Thus D .G=H / is isomorphic to U.g; /=U.g/a Š .U.g/=U.g/a /H ; where the last expression represents the set of all H -invariant elements. Corwin and Greenleaf [17] presented two conjectures about D .G=H /.

9.2 Monomial Representations and e-Central Elements

319

Commutativity Conjecture. The algebra D .G=H / is commutative if and only if  has finite multiplicities, i.e. m./ < 1 almost everywhere for  in Theorem 8.1.9. Polynomial Conjecture. When  has finite multiplicities, the algebra D .G=H / is isomorphic to the algebra CΠH of all H -invariant polynomial functions on  . Remark 9.1.2. The commutativity conjecture has been formerly conjectured by Duflo [22] in a more general context.

9.2 Monomial Representations and e-Central Elements There uniquely exists e 2 E such that  \Ue is a non-empty Zariski open set of  . Theorem 9.2.1 ([36]). Let j 2 T .e/ and take the e-central element Aj . Then there exists a polynomial function 'j .`/ on  such that ` .Aj / D 'j .`/Id at all ` 2  . Indeed, following [36] let us prove the next theorem. To simplify the notations we merely write  instead of  . Theorem 9.2.2. For an element  of U.g/, the following statements are mutually equivalent: (1) Regarding the ordinary topology there is a dense subset M of a certain non-empty open set in  such that ` ./ is a scalar operator on M. That is, there exists a complex-valued function '0 W ` 7! '0 .`/ on M so that ` ./ D '0 .`/Id holds at any ` in M. (2) ` ./ is a scalar operator everywhere on G . That is, there exists a complexvalued function ' W ` 7! '.`/ on G so that ` ./ D '.`/Id holds at any ` in G . This function is a polynomial function on  and G-invariant on G   . To prove the theorem let us utilize the theory [21, Chap. 6] on primitive ideals, i.e. kernels of algebraically irreducible representations, of U.g/ and elementary results (for instance, [47, Chap. 1]) of the algebraic geometry. A guideline of the proof is as follows. First we assume (1), then '0 will be extended to whole C as rational function. Since its graph is a connected closed set with Zariski topology, we shall conclude that it is a polynomial function. Some assertions in the following also hold for general Lie algebras or solvable Lie algebras, but we always assume that g is nilpotent. Let  2 gC and b 2 S.; gC /. Namely, b is a complex Lie subalgebra of gC , isotropic with respect to the bilinear form B . Then, we denote by b the complex linear subspace of U.g/ generated by fY  .Y /gY 2b . We set M.gC ; b; / D U.g/= .U.g/b /

9 e-Central Elements

320

and designate by .; b; gC / the representation of U.g/ induced naturally on the U.g/-module M.gC ; b; /. Towards a proof of Theorem 9.2.2, we first prepare the necessary materials. We give the natural structure of affine varieties to gC and C. For an integer r verifying 1  r  n D dim g, we denote by Gr.gC ; r/ the Grassmann manifold formed by the r-dimensional complex linear subspaces of gC and equip it with the structure of a natural projective algebraic variety. We designate by Gr.gC / their disjoint union t1rn Gr.gC ; r/ regarded as an algebraic variety. We provide the direct product set gC Gr.gC /C; gC Gr.gC /; gC C with the natural structure of the product algebraic variety and q1 ; q2 ; q3 denote the following natural projections: q1 W gC  Gr.gC / ! gC ; q2 W gC  Gr.gC /  C ! gC  C; q3 W gC  Gr.gC /  C ! gC  Gr.gC /: Let W 2 U.g/. We denote by FQW;r the subset of gC  Gr.gC ; r/ composed of the pair .; b/, where b 2 S.; gC / such that .; b; gC /.W / is a scalar operator. We write as FQW the disjoint union t1rn FQW;r in gC  Gr.gC /. Next we set FW D q1 .FQW / and FW;r D q1 .FQW;r /. These are subsets of gC , composed of  where exists b 2 S.; gC /, and of dimension r, such that .; b; gC /.W / is a scalar operator. Let us give the function QW on FQW by .; b; gC /.W / D QW .; b/Id: Proposition 9.2.3. (1) The subset FQW;r ; FQW of gC  Gr.gC / is a Zariski closed set. Besides, the function QW W FQW ! C is a rational function. (2) FW;r and FW D t1rn FW;r are Zariski closed sets of gC . Proof. (1) By the result of Conze and Duflo [21, Lemma 6.4.1], FQW;r .1  r  n/ is a Zariski closed set of gC  Gr.gC / and the restriction of the function QW to FQW;r is a rational function. (2) Gr.gC ; r/ is a projective variety and hence complete [47, Section 6.2]. Thus the projection q1 is a closed mapping, and the assertion follows.  For E  FW  gC , we set EQ D q11 .E/ \ FQW . We indicate by EQ C the subset Q satisfying the following three of gC  Gr.gC /  C composed of the triplet .; b; / conditions: (a)  2 E; (b) b 2 S.; gC /; Q . (c) .; b; gC /.W / D Id Set E C D q2 .EQ C /. This is the subset of gC  C composed of the pairs .; / such that there exists b 2 S.; gC / verifying .; b; gC /.W / D Id .

9.2 Monomial Representations and e-Central Elements

321

Proposition 9.2.4. (1) If E is a Zariski closed set of FW  gC , E C is also a Zariski closed set of gC  C. (2) Suppose that E is an irreducible closed set of gC . Let r be the minimal integer such that E \ FW;r ¤ ;, then E  FW;r . (3) Suppose that E is an irreducible closed set of gC and that the closed set E C of gC  C is a graph of some complex-valued function on E. Then, E C is an irreducible closed set. Proof. (1) As the inverse image of some closed set, EQ is a closed set of gC Gr.gC / and in particular an algebraic subvariety. Namely, it satisfies the separation condition [47, Section 2.5, Examples (2), (3)] as gC and Gr.gC /, while, from (1) of the preceding proposition, the function QW jEQ is a morphism of varieties Q C satisfy the separation and EQ C is its graph in EQ  C. Since the varieties E; condition, EQ C is a closed set of EQ  C and gC  Gr.gC /  C [47, Section 2.5, Proposition]. Since the variety Gr.gC / is complete, the projection q2 is a closed map and E C is a closed set. (2) By the previous proposition each FW;r is a closed set and E is irreducible, so the assertion follows immediately. (3) Let EQ D [1i q EQ i be the decomposition of EQ into the irreducible components. Then, E D [1i q q1 .EQ i / and fq1 .EQ i /g1i q gives a closed covering of E. Since E is irreducible, one of the irreducible components EQ i , for instance EQ 1 , satisfies q1 .EQ 1 / D E. Set EQ 1C D q31 .EQ 1 / \ EQ C . Since EQ 1C is the graph of the morphism QW jEQ1 from the variety EQ 1 to the variety C, both satisfying the separation condition, the projection q3 induces an isomorphism from EQ 1C onto EQ 1 [47, Section 9]. Especially, as EQ 1 , EQ 1C is an irreducible closed  2.5,CExercise  Q set. Since q1 q3 .E1 / D E, we get q2 .EQ 1C / D E C using the fact that E C is a  graph. In the sequel, E C is irreducible like EQ 1C . We denote by Prim.U.g// the set of all primitive ideals of U.g/. We mention the well-known orbit method only in the nilpotent case. Concerning the proofs, please refer to [21, Theorem 6.5.12 and Lemma 6.4.3]. Put GC D exp.gC /. Theorem 9.2.5. There exists a mapping  7! I./ from gC to Prim.U.g// which induces a bijection from gC =Ad.GC / to Prim.U.g//, and the following hold: (1) For any polarization p 2 M.; gC / ¤ ;, ker.; p; gC / D I./. (2) Provided b 2 S.; gC /, ker.; b; gC /  I./. We will show that the graph hypothesis in Proposition 9.2.4(3) is automatically satisfied. Let us begin with a simple lemma. Lemma 9.2.6. We fix W 2 U.g/. (1) When .; V / is a representation of U.g/, the following two properties are equivalent: (i) .W / is a scalar operator Id;  2 C; (ii) W 2 ker C C. In this case, W  modulo ker.

9 e-Central Elements

322

(2) Let .0 ; V 0 / be another representation of U.g/ such that ker0  ker. Then, if 0 .W / is a scalar operator, .W / is also the scalar operator Id . Proof. (1) This is clear from the following changes of expression: .W / D Id ” .W  / D 0 ” W  mod ker: (2) Since ker0  ker and C \ ker D f0g, W  mod ker0 implies W  mod ker.  Proposition 9.2.7. We fix W 2 U.g/. For  2 gC , the next two properties are equivalent: (i) W 2 I./ ˚ C; (ii)  2 FW . In other words, there exists b 2 S.; gC / so that .; b; gC /.W / is a scalar operator QW .; b/Id . Then, the scalar value QW .; b/ does not depend on the choice of b. Therefore, there is the function W W FW ! C such that QW .; b/ D W ./ and W W ./ mod I./ on FW . Namely, for E  FW , E C is the graph of W jE in E  C. Proof. We first assume (ii). From (1) of the last lemma, W 2 ker.; b; gC / ˚ C. From Theorem 9.2.5(2), ker.; b; gC /  I./. So, using (2) of the last lemma, W QW .; b/ modulo I./ and W 2 I./ ˚ C. Next, if we consider another b0 2 S.; gC / such that .; b0 ; gC /.W / is a scalar operator QW .; b0 /Id , W QW .; b/ mod I./ and W QW .; b0 / mod I./: Since C \ I./ D f0g, this means QW .; b/ D QW .; b0 /. Conversely we assume (i). By Theorem 9.2.5(1), ker.; p; gC / D I./ for any polarization p 2 M.; gC / at . Hence, .; p; gC /.W / is a scalar operator.  When G D exp g is nilpotent, let us clarify the relation between the kernels of irreducible unitary representations of G and the primitive ideals of U.g/. Theorem 9.2.8 ([19]). At every ` 2 g , ker` D I.i `C /. Here, `C denotes the element of gC which extends ` complex linearly. Proof. This result itself is referred to in the preface of [21] and is well known. However, it seems rather difficult to read the reference [19] which preserves the proof. It is why we prove it anew. We choose a real polarization p of g at ` and, using a coexponential basis fXk g1kp to p in g, realize ` in L2 .Rp /. Then, as we saw in Corollary 6.2.14 in Chap. 6, the space of C 1 -vectors is just the space S.Rp / of all rapidly decreasing functions of Schwartz and so its anti-dual space is the space S  .Rp / of the tempered distributions. We denote also by ` the actions of G and U.g/ extended to S  .Rp /.

9.2 Monomial Representations and e-Central Elements

323

For Y 2 p, let us consider the element YQ D Y  i `.Y / of U.p/. Let ı 2 S  .Rp / be the Dirac measure at the origin of Rp . We first denote by I.ı/ the left ideal of U.g/ composed by the annihilators of ı: I.ı/ D fU 2 U.g/I ` .U /  ı D 0g: Obviously, ker` is a two-sided ideal of U.g/ and contained in I.ı/. Put b D pC . By Theorem 9.2.5, the U.g/-module M D M .i `C ; b; gC / D U.g/= .U.g/bi `C / is a simple module. Consider the representation  D .i `C ; b; gC / of U.g/ on M . When we denote by I.1/ the annihilator of 1M in U.g/, I.1/ is a left ideal of U.g/ spanned by fYQ gY 2p and I.i `C / is a two-sided ideal of U.g/, which is maximal among those contained in I.1/. From now on, we show the equality I.ı/ D I.1/. If this is shown, since ker` is a two-sided ideal, we have ker`  I.i `C / at once. For this aim we take a basis fY1 ; : : : ; Ynp g of p. Hence fY1 ; : : : ; Ynp ; X1 ; : : : ; Xp g is a basis of g and gC . By the Poincaré–Birkhoff–Witt theorem in the first chapter:

˚ ˛np   YQ1˛1 I .˛1 ; : : : ; ˛np / 2 Nnp is a basis of U.p/; o (a) nYQnp ˇ ˛np ˇ (b) Xp p   X1 1 YQnp   YQ1˛1 I .˛1 ; : : : ; ˛np ; ˇ1 ; : : : ; ˇp / 2 Nn is a basis of U.g/. Finally, from (a) and (b) the family np Xp p   X1 1 YQnp   YQ1˛1 ;

ˇ

ˇ

˛

where .˛1 ; : : : ; ˛np ; ˇ1 ; : : : ; ˇp / 2 Nn ; ˛1 C    C ˛np ¤ 0; turns out to be a basis of I.1/. We immediately get ` .Y /ı D i `.Y /ı; Y 2 p; ` .Xk /ı D 

@ ı; 1  k  p; @xk

and   ˇ ˛np ˇ   YQ1˛1 ı ` Xp p   X1 1 YQnp 8 < 0; ˛1 C    C ˛np ¤ 0 D  @ ˇp  @ ˇ1 :      @x1 ı; ˛1 D    D ˛np D 0: @xp

(9.2.1)

9 e-Central Elements

324

 ˇp  ˇ1 @ @ p Now the family  @xp     @x1 ıI .ˇ1 ;    ; ˇp / 2 N forms in Rp a basis of the space of the distributions supported at the origin, so the family (9.2.1) forms a basis of I.ı/. Thus I.ı/ D I.1/ and ker`  I.i `C /. In order to show the inverse inclusion I.i `C /  ker` , let us make use of the next two classical facts: (a) since I.i `C / is a two-sided ideal, it is invariant by the adjoint representation of G; (b) the linear subspace spanned by ` .G/  ı is dense in S  .Rp /. In this case, ı is called a cyclic vector of the representation ` . In particular from (a), ` .I.i `C //  ` .G/ı D f0g: Since the action of U.g/ on S  .Rp / by the representation ` is continuous, ` .I.i `C //  S  .Rp / D f0g by (b). Thus I.i `C /  ker` , and the theorem is proved.





Proposition 9.2.9. Let W 2 U.g/ and ` 2 g . In order that ` .W / is a scalar operator, it is necessary and sufficient that i `C 2 FW . In this case, ` .W / D W .i `C /Id . Proof. From Proposition 9.2.7 and Theorem 9.2.8, W W .i `C / modulo ker` . Next by Lemma 9.2.6(1), ` .W / D W .i `C /Id .  Proof of Theorem 9.2.2. Now we are ready to prove Theorem 9.2.2. It is enough to see that in the theorem assumption (1) leads to assumption (2). We set C D f 2 gC I .  if /jh D 0g; choose W D ; E D C and put FQ D FQ ; F D F ; Q D Q ;  D  . First we show the following. Lemma 9.2.10. C  F . Proof. As we saw already, F \ C is a Zariski closed set of C and consists of the common zero points of an ideal of polynomial functions on C . Therefore, it suffices to verify that all elements P of this ideal vanish on whole C . From the assumption and Proposition 9.2.9, ` 2 M ) i `C 2 F \ C ) P .i `C / D 0:

9.2 Monomial Representations and e-Central Elements

325

N of M. Since Hence by continuity P .i `C / D 0 for all elements ` in the closure M N is not an empty set, P .`/ D 0 at all points ` of some open set of the interior of M i for the ordinary topology. Hence P .C / D f0g.  Let us show that jC is a polynomial function on C . Replacing E at Proposition 9.2.4(1), (3) by the irreducible closed set C of gC : Lemma 9.2.11. The graph CC of jC is an irreducible Zariski closed set of C C. The fact that jC is a polynomial function is derived from the following lemma. Lemma 9.2.12. Let ˛ be a function from Cp to C such that its graph Y in Cp C is a connected closed set with respect to the Zariski topology. Then, ˛ is a polynomial function on Cp . Remark 9.2.13. We need here the assumption of the connectedness. In fact, let p D 1 for instance and consider the non-connected Zariski closed set f.z1 ; z/I z1 .z1 z  1/ D 0 and z.z1 z  1/ D 0g of C  C, this is a graph of the function ˛ from C to C given by ˛.z1 / D

1 .z1 ¤ 0/ and ˛.0/ D 0: z1

Nevertheless, ˛ is not a polynomial function. Proof of the Lemma. To simplify the notations, the polynomial ring CŒX1 ; : : : ; Xp ; X  .resp. CŒX1 ; : : : ; Xp / is denoted by A .resp. A0 /. We denote by Z D .z1 ; : : : ; zp ; z/ .resp. Z0 D .z1 ; : : : ; zp // a variable of Cp  C .resp. Cp / and, for P 2 A, by dXı P the degree of P regarding the last variable X . For P 2 A .resp.P 2 A0 /, we designate by Z.P / .resp.Z0 .P // the variety formed by all zero points of P in Cp  C .resp. Cp /. Let C.X1 ; : : : ; Xp / be the field of the rational functions of CŒX1 ; : : : ; Xp  and P D C.X1 ; : : : ; Xp /ŒX  the polynomial ring of one variable X with coefficients in C.X1 ; : : : ; Xp /. Let I be the ideal of A composed of all polynomials which vanish on Y . Then let us see that all elements of I are divisible by a certain polynomial T1 of A. Let I 0 be the ideal generated by I in P. The elements of I 0 are those written in the form PA with relatively prime P 2 I and A 2 A0 . Since Y is a graph, dXı P  1 if P 2 I n f0g. Since P is a principal ideal ring, there exists a certain element of this ideal so that every element of I 0 is a multiple of this element. This element is 1 written in the form AT with A; B 2 A0 and T1 2 A such that dXı T1  1. Here, we B

9 e-Central Elements

326

may assume that T1 is not divisible in A0 and A; B are relatively prime. Besides, 1 with q > 0, using aq1 2 A0 n f0g and aq1 ; : : : ; a01 2 A0 , we can write T1 D aq1 X q C    C a01 : 1 . Here, P1 and In particular, any element P of I is written in the form P D PC1  AT B C are respectively in A and A0 . Since A and A0 are unique factorization rings, B divides P1 and C divides A. Finally, every element of I is a multiple of T1 . That is, for arbitrary P 2 I, there is Q 2 A so that P D QT1 . Therefore, Z.T1 /  Y and at arbitrary Z0 2 Cp the zero point of the function

z 7! aq1 .Z0 /zq C    C a01 .Z0 / on C is at most one. Moreover, it has just one zero point at Z0 62 Z0 .aq1 /. This zero point is also the zero point of the function 1 z 7! qaq1 .Z0 /z C aq1 .Z0 / 1 which is obtained by differentiating T1 for q  1 times. Put T0 D qaq1 X C aq1 and use the fact that A is a unique factorization ring to write down

T0 D bT; b 2 A0 ; T D a1 X C a0 2 A n A0 with an irreducible T . Since Z.b/  Z.aq1 /, Z.T / n Z.aq1 /  Y . Besides, aq1 is not divisible by T . By Hilbert’s zero point theorem, every element of I is a multiple of T which is a polynomial of X with degree one. Next let us show by the absurdity Z0 .a1 / \ Z0 .a0 / D ;. Indeed, if there exists Z0 2 Z0 .a1 /\Z0 .a0 /, .Z0 ; C/ is contained in the set of zero points of T and hence of all elements of I, what contradicts the fact that Y is a graph. Hence Z.T /  .Cp  C/ n Z.a1 /. From Z.T /  Y , Z.T / D Y n .Y \ Z.a1 // and since Y D .Y \ Z.a1 // [ .Y n .Y \ Z.a1 //; we get Z.T / [ .Y \ Z.a1 // D Y; Z.T / \ .Y \ Z.a1 // D ;:

(9.2.2)

This result is obtained whenever the graph Y is any Zariski closed set. Here we use the assumption that Y is connected and let us see by absurdity that the element a1 of A0 nf0g is in fact an element of C . Provided Z.a1 / ¤ ;, Y \ Z.a1 / ¤ ; and from Eq. (9.2.2), Y becomes a disjoint union of two closed sets, which is absurd. Summing up the above, I is the set of all multiples of the polynomial T with the form X  a0 ; a0 2 A0 and the lemma follows. 

9.2 Monomial Representations and e-Central Elements

327

Returning to the proof of the theorem, since '.`/ D .i `C / on  , ' becomes a polynomial function by the lemma. Thus the proof of Theorem 9.2.2 is complete.  Recall the sequence of ideals (9.1.1) of g. Set I D fi1 < i2 <    < id g D f1  i  nI h \ gi ¤ h \ gi 1 g and J D fj1 < j2 <    < jq g D f1; 2; : : : ; ngnI. Here d D dim h and q D dim.g=h/. On the one hand, setting k0 D h; kr D h C gjr .1  r  q/, we get the sequence of Lie subalgebras h D k0  k1      kq1  kq D g; dim.kr / D d C r:

(9.2.3)

On the other hand, setting h0 D f0g; hs D h \ gis .1  s  d /, we procure a sequence of ideals of h: f0g D h0  h1      hd 1  hd D h; dim.hs / D s:

(9.2.4)

We choose a basis fYs g1sd of h in such a fashion that Ys 2 hs n hs1 .1  s  d /. Next, we set as D

s X

C.Yj C if .Yj //

j D1

with 1  s  d . We denote by T .eH / the set of indices is 2 I such that hs  hs1 C g.`/ for almost all ` 2  with respect to . Q As T .eH /  T .e/, put U.e/ D T .e/nT .eH /. By ˙ we indicate the principal anti-automorphism of U.g/. Let is 2 T .eH / and T .e/ \ f1; 2; : : : ; is g D fm1 < m2 <    < mk D is g. For brevity, we write the Corwin–Greenleaf e-central element Amj .1  j  k/ as j . The next lemma will be a key lemma in Chap. 12 in order to prove the commutativity conjecture. To get the lemma we need much on e-central elements and this topic will be treated in Chap. 12. Lemma 9.2.14. ˙.k / is algebraic on f˙.1 /; : : : ; ˙.k1 /g modulo U.gis /as . Let us calculate in typical cases the Corwin–Greenleaf e-central elements and see what the lemma means. In these cases, those elements are nothing but elements in the centre of U.g/. Example 9.2.15. We take up g D hX1 ; : : : ; Xn iR : ŒXn ; Xk  D Xk1 .2  k  n  1/. The centre of g is z D RX1 and b D hX1 ; : : : ; Xn1 iR is a commutative ideal of codimension 1 in g. When n D 4, we have in matrix form

9 e-Central Elements

328

0

000 B0 0 0 X1 D B @0 0 0 000 0 000 B0 0 0 X3 D B @0 0 0 000

0 1 1 000 B C 0C 000 ; X2 D B @0 0 0 0A 0 000 0 1 0 010 B0 0 1 0C C ; X4 D B @0 0 0 1A 0 000

1 0 1C C; 0A 0 1 0 0C C: 0A 0

Thus, 80 1 ˆ ˆ
w w2 1 w 0 1 0 0

9 1 z > > = yC C I w; x; y; z 2 R : A > x > ; 1

The generic coadjoint orbits of G are two-dimensional and their elements admit b as a common polarization. Then, except for the case where h D z and f jz ¤ 0, our monomial representation  D indG H f has finite multiplicities even if dim h D 1. Setting gj D hX1 ; : : : ; Xj iR for 1  j  n, we get a Jordan–Hölder sequence of g: f0g D g0  g1      gn1  gn D g: Very simply, let us seek elements belonging to the centre of U.g/. For example, assuming n  5, 1 1 D X1 ; 2 D 2X1 X3  X2 2 ; 3 D X1 2 X4  X1 X2 X3 C X2 3 ; etc. 3 First let n D 5; h D hX3 ; X4 iR and f .X3 / D ; f .X4 / D . Then,  has finite multiplicities and the algebra D .G=H / coincides with the polynomial ring of X1 ; X2 . Hence 3 does not contribute to the formation of D .G=H /. We set a D C.X3 C if .X3 // C C.X4 C if .X4 // D C.X3 C i / C C.X4 C i / and consider a1 D C.X3 C if .X3 // D C.X3 C i /. We have 2 2i 1  X22 modulo U.g1 /a1 and   1 2 3 1 X4 C i 1 C X2 X2 3 2

9.2 Monomial Representations and e-Central Elements

329



1 1 X4 C i 1  .2 C 2i 1 / X2 3 2

1 D 1 2 X4  .2  i 1 /X2 mod U.g2 /a 3 From this, if we take h0 D RX3 and consider the induced representation  0 D 0 0 0 indG H 0 f with H D exp.h /, 3 contributes substantially to form D 0 .G=H /. However, 9.3  12 X4 /2 C .2 C 2i 1 /.2  i 1 /2 0 mod U.g2 /a and especially 9.3 C i 1 2 /2 C .2 C 2i 1 /.2  i 1 /2 0 mod U.g4 /a : We can continue likewise. Indeed, let n  6. 4 D 2X1 X5 2X2 X4 CX3 2 belongs to the centre of U.g/ and if we start from h D hX3 ; X4 ; X5 iR with f .X5 / D , 4 21 .i/  2X2 .i / C .i /2 D 2i1 C 2i X2  2 modulo U.g3 /a . Hence .4 C 2i1 C 2 /2 4 2 X2 2 4 2 .2 C 2i 1 / modulo U.g5 /a . Next let n  7. If we take 1 1 1 5 D X1 4 X6  X1 3 X2 X5 C X1 2 X2 2 X4  X1 X2 3 X3 C X2 5 ; 2 6 30 this again belongs to the centre of U.g/. At generic ` 2  , we take h D hX3 ; X4 ; X5 ; X6 iR in order that h \ g.`/ becomes sufficiently large. Put f .X6 / D . Then, 1 1 1 5 1 4 .i /  1 3 X2 .i/ C 1 2 X2 2 .i /  1 X2 3 .i / C X2 5 2 6 30 i  i  1 D i 1 4 C i1 3 X2  1 2 X2 2 C 1 X2 3 C X2 5 2 6 30 modulo U.g2 /a . Hence i 2 1 .2 C 2i 1 / 2 1 i  1 .2 C 2i 1 /X2 C .2 C 2i 1 /2 X2 6 30

5  i 1 4 C i1 3 X2 C

9 e-Central Elements

330

D  i 1 4  1 3 C

i 2 1 2 2

1 .2 2 C 4i 1 2  42 1 2  5i 1 2 C 102 1 2 C 30i1 3 /X2 30 1 D  .2i 1 4 C 21 3  i 1 2 2 / 2 1 C .62 1 2  i 1 2 C 2 2 C 30i1 3 /X2 30 C

modulo U.g2 /a . In consequence,

2 ˚ 305 C 15.2i 1 4 C 21 3  i 1 2 2 / C .62 1 2  i 1 2 C 2 2 C 30i1 3 /2 .2 C 2i 1 / 0 modulo U.g6 /a . These all show that 1 ; 2 , for example, generate algebraically the algebra of the  -central elements. Now take h D hX3 ; X5 iR and put f .X3 / D ; f .X5 / D  as before. Then, 4 21 .i/  2X2 X4 C .i /2 D 2i1  2X2 X4  2 modulo U.g3 /a . Hence  1 1 2 4 2i1 3  2X2 3 C .2  i 1 /X2  2 1 2 3 2 2i1 3  2X2 3 C .2  i 1 /.2 C 2i 1 /  2 1 2 3 1 2i  2 1 2 C 2 2  2X2 3 D 2i1 3 C 2 1 2 C 3 3 3 modulo U.g3 /a . After all, .31 2 4  22 2  2i 1 2 C 6i1 3  2 1 2 /2 C 36.2 C 2i 1 /3 2 0 modulo U.g5 /a . So, if we look at the case where n D 6, D .G=H / is identified with the polynomial ring of three variables X1 ; X2 ; X4 . Finally, let h D hX4 ; X5 iR and put f .X4 / D ; f .X5 / D  as before. This time 4 21 .i/  2X2 .i / C X3 2 D 2i1 C 2i X2 C X3 2

9.2 Monomial Representations and e-Central Elements

331

and 41 2 4 8i13 C 8i 1 2 X2 C .2 C X2 2 /2 D 8i13 C 8i 1 2 X2 C 2 2 C 22 X2 2 C X2 4 modulo U.g2 /a , while 1 1 3 1 2 .i /  2 X2  .2 C X22 / mod U.g2 /a : 3 6 Combining these two relations together, we get modulo U.g5 /a an algebraic relation among 1 ; : : : ; 4 . In this case also, if n D 6, the algebra D .G=H / becomes the polynomial ring of the three variables X1 ; X2 ; X3 . We denote by Z.g;  / the algebra composed by the function  on G such that there exists W 2 U.g/ which satisfies ` .W / D .`/Id at any ` 2  . Relating to Lemma 9.2.14, we will show the following theorem in Chap. 12: Theorem 9.2.16 ([36]). f'j I j 2 U.e/g forms a transcendental basis of Z.g;  /.

Chapter 10

Frobenius Reciprocity

10.1 Frobenius Reciprocity in Distribution Version Let G be a -compact Lie group with Lie algebra g, and we only consider unitary representations  whose Hilbert space H are separable. First we remember the C 1 -vectors. Let v 2 H . When the function G 3 g 7! .g/v 2 H is C 1 , v is called a C 1 -vector. We denote by H1 the space of the C 1 -vectors of . 1 f n g1 nD1 being the approximate identity of L .G/ introduced in Proposition 2.2.8 and chosen in D.G/, we see that k. n /w  wk ! 0 .n ! 1/ for any w 2 H . As . n /w 2 H1 , H1 is a dense subspace of H and g acts there by the differential representation d  of : d .X /v D

d .exp.tX //vjt D0 .X 2 g; v 2 H1 /: dt

The differential representation d  is uniquely extended to a representation of U.g/. fX1 ; : : : ; Xn g being a basis of g, H1 becomes a Fréchet space with semi-norms d .v/ D

X

kd .Xi1   Xid /vk .d 2 N/:

1ik n

We designate by H1 the anti-dual space of H1 , i.e. the space of the continuous anti-linear forms on H1 with values in C. We call elements of H1 generalized vectors of . We equip H1 with the strong dual topology of H1 . Then, the antidual space of H1 is identified with H1 . For a 2 H˙1 and b 2 H1 , we write ha; bi for the image of b by a. Hence ha; bi D hb; ai. The actions of G and g are continuously extended on H1 by duality. Notice that   .'/ H1  H1

© Springer Japan 2015 H. Fujiwara, J. Ludwig, Harmonic Analysis on Exponential Solvable Lie Groups, Springer Monographs in Mathematics, DOI 10.1007/978-4-431-55288-8__10

333

334

10 Frobenius Reciprocity

for ' 2 D.G/. When a closed subgroup K and its character  W K ! C are given, set 

H1

K;

D fa 2 H1 I .k/a D .k/a; 8 k 2 Kg:

Theorem 10.1.1 ([31, 46]). Let G D exp g be an exponential solvable Lie group with Lie algebra g, f 2 g and h 2 I.f; g/. As before, we define the character f O of H D exp h by f .exp X / D e if .X / .X 2 h/ and consider  D indG H f 2 G. O Then, for  2 G, dim



H;f H;G H1 1=2

( D

1;

 ' ;

0;

 6' :

Though we confirmed in Corollary 8.2.5 that Frobenius reciprocity holds, Theorem 10.1.1 also announces in a very special situation a kind of Frobenius reciprocity. Does this kind of reciprocity hold in the general situation? Question. Does the following hold in the irreducible decomposition (8.1.1) of monomial representations: H;f 1=2  H;G m./ D dim H1 O for -almost all  2 G?

10.2 Realization of the Reciprocity in Nilpotent Case Let us examine this problem in more detail for nilpotent Lie groups. Hereafter in this chapter, let G D exp g be a connected and simply connected nilpotent Lie group with Lie algebra g. Penney [59] showed the inequality  H;f 1=2 H;G m./  dim H1 O So, we are interested in the inverse inequality. for -almost all  2 G. Here we write down some comments on formula (8.1.1). (1) The following alternative occurs: -almost everywhere the multiplicities m./ are uniformly bounded or -almost everywhere m./ D C1. According to these two eventualities, we say that  has finite or infinite multiplicities. (2)  has finite multiplicities if and only if -almost Q everywhere h C g.`/ is a maximal isotropic subspace for the bilinear form B` . O each connected (3) When  has finite multiplicities, for -almost all  2 G, component of O1 ./ \  consists of only one H -orbit and its dimension

10.2 Realization of the Reciprocity in Nilpotent Case

335

is equal to 12 dim.O1 .//. Therefore, the multiplicity m./ is given by the number of the connected components of O1 ./ \  . O Suppose that  D indG H f has finite multiplicities. For  2 G, we write ˝./ 1 instead of O ./. In  , up to a null set for , Q every connected component Ck .1  k  m.// of ˝./ \  turns out to be a single H -orbit. We fix ` 2 ˝./ and b 2 M.`; g/. In other words, we realize  D indG B ` by means of B D exp b. For 1  k  m./, we take gk 2 G in such a manner that gk  ` 2 Ck and take an invariant measure d hP on the homogeneous space H=.H \gk Bgk1 /. Proposition 10.2.1 ([30]). We can construct linearly independent elements H;f  by the following formula: for all  2 H1 , ak .1  k  m.// in H1 Z hak ; i D

H=.H \gk Bgk1 /

P .hgk /f .h/d h:

(10.2.1)

Proof. First of all let us confirm that the integral in the right member is well-defined. In fact, with h0 2 H \ gk Bgk1 , .hh0 gk /f .hh0 / D .hgk gk1 h0 gk /f .h/f .h0 / D ` .gk1 h0

1

gk /.hgk /f .h/f .h0 /

1

D gk ` .h0 /f .h0 /.hgk /f .h/ D .hgk /f .h/: Secondly, there exists in h a coexponential basis to h \ gk  b which constitutes a part of a coexponential basis to gk  b in g. Taking Corollary 6.2.14 into account, the space obtained from H1 through right translation by gk is identified by means of this basis with the space S.Rm /; m D dim.G=B/ of the rapidly decreasing functions and the measure d hP is identified with the Lebesgue measure on Rp  Rm ; p D dim.H=.H \gk Bgk1 //, We derive from this the continuity of ak . Actually, this right translation by gk gives an intertwining operator between the two realizations, at ` and gk  `, of . The semi-invariance required on ak is easily verified by a direct computation. Finally, we choose a Haar measure db on B and define, for 2 D.G/, a function Q on G by Z Q .g/ D .gb/` .b/db: B

ak on Clearly Q 2 H1 and the generalized vector ak supplies a distribution f ak coincides with the G by the formula f ak . / D h Q ; ak i. Then the support of f k double coset Hgk B which is closed. Hence to show that fa g1km./ are linearly independent it is enough to check Hgj B ¤ Hgk B provided j ¤ k. Otherwise, the connected set gj .` C b? / \  of ˝./ \  would intersect Cj and Ck at the same time, which is absurd. 

336

10 Frobenius Reciprocity

Remark 10.2.2. Up to a scalar multiplication the generalized vector ak does not depend on the choice of either gk 2 G or b 2 M.`; g/. When G is nilpotent, we can answer affirmatively to the question of the previous section. Theorem 10.2.3 ([38]). Let G D exp g be a nilpotent Lie group with Lie algebra g, f 2 g ; h 2 S.f; g/ and  D indG H f . We consider the canonical irreducible decomposition Z '

˚ GO

m./d./

of  described in Theorem 8.1.9 of Chap. 8. In this situation, a kind of Frobenius O reciprocity holds: for -almost all  2 G,  H;f m./ D dim H1 : In particular, if  has finite multiplicities, 

H1

H;f

X

m./

D

Cak

kD1

O for -almost all  2 G. Proof. Let us give only an outline of a proof. We employ the induction on dim g C dim.g=h/. First we can assume that h contains the centre z of g and that f does not vanish on any non-trivial ideal. Thus we arrive at the situation where dim z D 1; f jz ¤ 0. As usual we take a Heisenberg triplet fX; Y; Zg such that z D RZ; f .Z/ D 1; ŒX; Y  D Z; g D g0 C RX . Here, g0 denotes the centralizer of Y in g. Let ` 2 ˝./ and realize  by means of a polarization b at ` of g contained in g0 . According to the decomposition G D exp.RX /G0 , G0 D exp.g0 /, the space m1 O H1 becomes S.Rm / Š S.R/˝S.R /. Here S.Rm1 / represents the space H10 0 c with 0 D indG B ` 2 G0 . Now every g 2 G is uniquely written as g D exp.xX /g0 with x 2 R; g0 2 G0 . We would like to go down to the subgroup G0 by eliminating the first coordinate x. H;f  Take a 2 H1 . If h 6 g0 , choose X in h \ kerf . Then from the semiH;f  invariance of a, there is a0 2 H1 so that 0 Z ha; .x/ .g0 /i D Here  2 S.R/; x 2 R; subgroup G0 .

R

 .x/dx ha0 ; .g0 /i:

2 H10 ; g0 2 G0 . In this way we may descend to the

10.3 Examples of Semi-invariant Distributions

337

Next assume h  g0 . It is sufficient for us to consider the case where  has finite multiplicities. So we assume that, by Lemma 9.2.14, there are Corwin-Greenleaf e-central elements f1 ; : : : ;  g in such a way that we have a polynomial relation   P .˙.1 //; : : : ; .˙. //; Y 0 modulo U.g/a . Here P is a polynomial of C1 variables and Y appears essentially. By applying this relation to our generalized vector a, we have F .x/a D 0 with some non-constant polynomial F .x/. Hence, f˛j g1j r being real roots of the algebraic equation F .x/ D 0, the support of the distribution aQ is contained in the disjoint union of the submanifolds Mj D exp.˛j X /G0 .1  j  r/ of G. Thus, a is written as u X @k Dk aD @x k kD0

in a neighbourhood of Mj . Here, Dk .1  k  u/ are distributions on G0 . It remains for us to show u D 0. This is done by choosing appropriately the polynomial P used above and by using the fact that  has finite multiplicities.  Remark 10.2.4. When G D exp g is an exponential solvable Lie group and the monomial representation  has finite multiplicities, is it possible to construct an element of  1 H;f 1=2 H;G H by a formula similar to (10.2.1): for  2 H1 , I hak ; i D

1=2

H=H \gk Bgk1

.hgk /f .h/ H;G .h/d .h/ .g 2 G/

(10.2.2)

with D H;H \gk Bg1 ? In this case, already at the step of taking this value we k encounter two problems: is this integral well defined? If so, as in the observation of intertwining operators, what can we say about its convergence? Finally, do the elements ak .1  k  m.// give generalized vectors of ? In the case of exponential solvable Lie groups, it is a difficult problem to determine the space H1 ; maybe we could say something by starting from a Vergne polarization b.

10.3 Examples of Semi-invariant Distributions Example 10.3.1. Let g be the completely solvable Lie algebra of dimension 4 with a basis .T; P; Q; E/ satisfying the commutation relations

338

10 Frobenius Reciprocity

ŒT; P  D P =2; ŒT; Q D Q=2; ŒT; E D E; ŒP; Q D E: In matrix form 1 0 1 0 010 1=2 0 0 T D @ 0 0 0 A ; P D @0 0 0A ; 000 0 0 1=2 1 0 1 0 001 000 Q D @0 0 1A ; E D @0 0 0A : 000 000 80 9 1 < wx z = G D exp g D @ 0 1 y A I w > 0; x; y; z 2 R : : ; 0 0 w1 Let f D E  2 g and h D RT C RQ 2 M.f; g/. We put as usual H D exp h;  D indG H f . Here again  ' C ˚  with square integrable irreducible unitary representations ˙ 2 GO corresponding to the two open orbits ˙GE  . H;f 1=2  H;G . When we realize ˙ by means of b D RQ C RE 2 Let a 2 H1 ˙ I.˙E  ; g/, their representation spaces H˙ are identified with L2 .R2 / by the mapping 7! Q .s; t/ D .exp.sT /exp.tP // ..s; t/ 2 R2 /. In this situation, the semi-invariance of a requires (

ha; Q .s C x; t/i D e x=4 ha; Q i; s=2 ha; .1  e ˙ixt e / Q i D 0

for any 2 H1˙ and x 2 R. From the second condition, the support of a is contained in f.s; t/ 2 R2 I t D 0g, then from the first condition ha; Q i D

Z

Q .s; 0/e s=4 ds R

up to a constant multiplication. Thus here also H;f 1=2  H;G D 1; dim H1 ˙ but of course this is no more than a very simple and special case obtained from the results of [9, 59] combined.

10.3 Examples of Semi-invariant Distributions

339

Next, for the point .˛; ˇ/ of R2 not equal to the origin, we consider ˛;ˇ 2 GO corresponding to the coadjoint orbit through `˛;ˇ D ˛P  C ˇQ 2 g . Let  H;f 1=2 H;G 1 a 2 H˛;ˇ . Using b D hP; Q; EiR 2 I.`˛;ˇ ; g/, we identify the space 2 H˛;ˇ with L .R/ by the mapping 7! Q .t/ D .exp.tT // .t 2 R/. The semiinvariance of a requires (

ha; Q .t C x/i D e 3x=4 ha; Q i; t =2 ha; .1  e iˇxe / Q .t/i D 0

for any 2 H1˛;ˇ and x 2 R. From the second condition, ˇ ¤ 0 means a D 0. When ˇ D 0, the first condition means ha; Q i D

Z

Q .s/e 3s=4 ds R

up to a constant multiplication. Indeed, we know as in the last example that this H;f 1=2  H;G 1 . In consequence, though formula furnishes a nonzero element of H˛;0 ˛;0 does not appear in the irreducible decomposition of , H;f 1=2  H;G dim H1 D 1: ˛;0 Finally, if we denote by c˛ the unitary character of G corresponding to ˛T  .˛ 2 R/,  1 H;f 1=2 H;G Hc˛ D f0g: As we have seen until now, we can say at least the following: regarding  2 GO H;f 1=2  H;G whose orbit does not encounter  , H1 D f0g. Let us add a simple example of non-exponential solvable Lie group. Example 10.3.2. Let G be the universal covering group of the motion group of a plane. Its Lie algebra g is given by a basis .T; X; Y / with the commutation relations ŒT; X  D Y; ŒT; Y  D X . An element of G is expressed by a triplet of real numbers .; a; b/ and the multiplication among them is described by .; a; b/. 0 ; a0 ; b 0 / D . C  0 ; a C a0 cos   b 0 sin ; b C a0 sin  C b 0 cos /: At a point `s D sY  2 g .0 ¤ s 2 R/, take a real polarization b D RX C RY . The stabilizer G.`s / is nothing but the extension of exp.RY / by the centre Z D exp.2ZT / of G and its unitary character ˛ ; ˛ 2 Œ0; 1/, is given by

340

10 Frobenius Reciprocity

˛ .exp.2mT // D e 2 i m˛ .m 2 Z/. Let B0 D exp b and B D ZB0 . We extend ˛ to a unitary character s;˛ of B by the formula s;˛ .zb/ D ˛ .z/`s .b/ .z 2 Z; b 2 B0 /: When we construct s;˛ D indG B s;˛ by right translation, it is well known that s;˛ ; s 2 RC D .0; C1/; ˛ 2 Œ0; 1/, are all irreducible and non-equivalent to each other. The action of s;˛ is described by the formula

1

.s;˛ ..; a; b///.t/ D e i s.a sin t Cb cos t / e i ˛Œt C  .t C / in the space L2 .T/; T D R=2Z endowed the measure dt=2. Here g D .; a; b/ 2 G and we use the writing t C  D Œt C  C t C  with Œt C  2 2Z and t C  2 Œ0; 2/. Now, let h D RX and H0 D exp h. Starting from the trivial representation 1H0 , we construct the monomial representation  D indG H0 1H0 of G. Benoist [8] asserts

1

1

Z

˚

 '2 RC Œ0;1/

s;˛ dsd˛:

H0 ;1H0  Besides, let ıx be the Dirac measure at x 2 T. Then H1 D Cı0 ˚ Cı , s;˛ while the intersection G`s \ h? consists of two connected components, each of which is a H0 -orbit of dimension 12 dim.G`s /. If we compute the subset S D fg 2 GI g1 .`s Cb? /\h? ¤ ;g of G, it is easily seen that S D fg D .; a; b/ 2 GI  2 Zg, and this coincides with the union of  H0 ;1H0 the supports as distribution of the elements of H1 . s;˛ Henceforth, we consider the extension H D exp.ZT /H0 and define its character  ; 2 Œ0; 1/, by  .exp.mT /h0 / D e 2 i m .m 2 Z; h0 2 H0 /: Now,  D indG H  realized by right translation is given by

A

. .g//.t; / D e 2 i ŒŒt C  .t C ; .1/ŒŒt C  . C a sin t C b cos t//: Here  2 L2 .Œ0; /  R/; g D .; a; b/ 2 G and we use the writing t C  D ŒŒt C  C t C  with ŒŒt C  2 Z and t C  2 Œ0; /.

A

A

10.3 Examples of Semi-invariant Distributions

341

By the Fourier transformation with respect to the second variable O / D p1 .t; 2

Z e iy .t; y/dy; R

another realization of  is written as

A

O O t C ; .1/ŒŒt C  /: / D e 2 i ŒŒt C  e i.a sin t Cb cos t / . . .g//.t; Next by the mapping which sends O 2 L2 .Œ0; /  R/ to Q 2 L2 .Œ0; 2/  RC / given by (

O Q / D .t; / .t; O  ; / e 2 i .t

t 2 Œ0; /; t 2 Œ; 2/;

we obtain the third realization in L2 .Œ0; 2/  RC / written as 

 Q C ; /:  .g/Q .t; / D e 2i Œt C  e i.a sin t Cb cos t / .t

Thus, Z  '

˚

RC

s;2 .mod Z/ ds

with uniform multiplicity 1 as a result of [9] asserts. From a point of view of the orbit method, this phenomenon seems to result from the fact that the two H0 -orbits contained in G`s \ h? are linked together by the action of exp.T / 2 H so that G`s \ h? becomes only one H -orbit in spite of being non-connected. H0 ;1H0  . ;  2 C/ belongs to Finally, in order that a D ı0 C ı 2 H1 s;˛  H;

H1 , it is necessary and sufficient that hs;˛ .exp.T //a; i D he 2 i a; i s;˛ for all 2 H1s;˛ . Hence, ( H;

 C.ı0 C e 2 i ı / 1 Hs;˛ D 0

˛ D 2 ; ˛ ¤ 2 :

So, we encounter again the above set S as the union of the supports as distribution of its elements.

Chapter 11

Plancherel Formula

11.1 Penney’s Plancherel Formula As before, let G D exp g be an exponential solvable Lie group with Lie algebra g, f 2 g ; h 2 S.f; g/; H D exp h; f .exp X / D e if .X / .X 2 h/ and consider  D .f; O h; G/ D indG H f . We denote by e the unit element of G. Since every H;f 1=2  1 1 H;G  2 H is an C -function [63] on G, ı 2 H1 is defined by ı ./ D hı ; i D .e/. In this chapter, we shall be interested in the abstract Plancherel formula due to Penney [59] and Bonnet [12] applied to the cyclic representation .; ı /. Provided h 2 I.f; g/,  is irreducible and, by Theorem 10.1.1 in Chap. 10, the H;f 1=2  H;G space H1 ;  2 GO is equal to Cı when  '  and f0g when  6' . Provided h 62 I.f; g/, let us decompose .; ı / into irreducible components: Z '

Z

˚ GO

m./d./; ı '

˚

GO

a d./:

H;f 1=2  H;G Then, -almost everywhere a belongs to m./H1 and at least when  has finite multiplicities H;f 1=2  H;G a D .ak /1km./ ; ak 2 H1 from the uniqueness of the irreducible decomposition [59]. So, h./ı ; ı i D

Z X m./ GO

kD1

h./ak ; ak id./

for  2 D.G/. © Springer Japan 2015 H. Fujiwara, J. Ludwig, Harmonic Analysis on Exponential Solvable Lie Groups, Springer Monographs in Mathematics, DOI 10.1007/978-4-431-55288-8__11

343

344

11 Plancherel Formula

Choosing a left Haar measure dh on H , we construct an element H of H1 by the formula Z f 1=2 H .g/ D .gh/f .h/ H;G .h/dh .g 2 G/ f

H

2 H1 ,

for  2 D.G/. With h./ı ; i D

˝

Z

˛

.g/.g/ı dg;

Z

G

˝

G

I

G

˛

1=2

.gh/ H;G .h/ .gh/dh

H

Z

1=2

.g/dG;H .g/ G=H

Z

dG;H .g/ G=H

I D

.g/.g 1 / dg

D ı ;

.g/ .g/dg D

D

Z

H

f

.gh/f .h/ H;G .h/dh D hH ; i:

Thus ./ı D H 2 H1 and f

Z

1=2

h./ı ; ı i D H

f

.h/f .h/ H;G .h/dh D H .e/

for arbitrary  2 D.G/. In consequence, the abstract Plancherel formula for the cyclic monomial representation .; ı / is described as follows. Theorem 11.1.1 ([31, 59]). Let G D exp g be an exponential solvable Lie group with Lie algebra g, f 2 g and h 2 S.f; g/. Let Z ˚ m./d./ ' GO

be the canonical irreducible decomposition of  D .f; O h; G/. Assume that  has O elements ak ; 1  k  finite multiplicities. There exist, for -almost all  2 G,  1 H;f 1=2 G;H with which the formula m./, of H f

H .e/ D

Z m./ X h./ak ; ak id./ GO kD1

(11.1.1)

holds for any  2 D.G/. Like the symmetric space of Benoist [8], we will practise the calculation of .ak /2G;1km./ O to procure concrete Plancherel formulas in certain cases.

11.1 Penney’s Plancherel Formula

345

Theorem 11.1.2 ([30]). When G D exp g is nilpotent and  has finite multiplicities, the generalized vectors constructed by formula (10.2.1) satisfy the abstract Plancherel formula (11.1.1), if we appropriately normalize the measures d hP utilized in their construction. Proof. We begin by preparing a lemma. We choose invariant measures dg; dh; d gP and d hP D dk hP respectively on G; H; G=B and H=.H \gk Bgk1 /. Next, we induce on G=H the quotient measure d g, R determine by transitivity the invariant measure dk gP on G=.H \ gk Bgk1 /, take its image by the canonical map on G=.gk1 Hgk \B/ and finally induce the invariant measure dk bP on B=.B \ gk1 Hgk /. Lemma 11.1.3. With  2 D.G/, Z

  ./ak .g/ D

H .gbgk1 /` .b/d bP .g 2 G/; f

B=.B\gk1 Hgk /

Hence Z h./ak ; ak i

Proof. With

D

2

H=.H \gk Bgk1 /

H1

f .h/d hP

Z B=.B\gk1 Hgk /

f P H .hgk bgk1 /` .b/d b:

(11.1.2)

let us compute Z

h./ak ; i D

G

.g/hak ; .g 1 / idg

Z D

Z .g/dg

G

Z D

G=H

Z d gR

G=H

Z

H=.H \gk Bgk1 /

H=.H \gk Bgk1 /

.ghgk /f .h/dk hP

f H .gh/ .ghgk /dk hP

f

D

G=.H \gk Bgk1 /

D G=B

˝

Z

H .g/ .ggk /dk gP

Z

Z

D

.ghgk /f .h/dk hP

Z

f H .g/d gR

Z D

H=.H \gk Bgk1 /

B=.B\gk1 Hgk /

B=.B\gk1 Hgk /

From this we derive the conclusion.

f H .gbgk1 / .gb/dk bP

f P H .gbgk1 /` .b/dk b;

˛

: 

Let us show the theorem rather by normalizing the measure  which appears in formula (8.1.1). Let g0 be an ideal of codimension 1 in g containing h. Assume

346

11 Plancherel Formula

first that almost all orbits are non-saturated in the direction g? 0 . For -almost all O  2 G, the restriction 0 of  to G0 D exp.g0 / is irreducible, m./ D m.0 / and, c0  R. By the except for a set of measure 0, the Borel space GO is identified with G c induction hypothesis, there exists a measure 0 on G0 so that formula (11.1.1) holds for G0 . Under our identification, putting ak D ak 0 for 1  k  m./, the measure  D 0  dx satisfies our request, here dx denotes a Lebesgue measure on R. In fact, let p W g ! g0 the restriction map. In order that the measure 0 exists, we can choose d gP 0 ; dk hP 0 and hence dk bP0 . Here the index 0 indicates corresponding objects at the stage of G0 . Let b0 2 M.`0 ; g0 / be a real polarization appropriately c0 . Let the mapping G c0 3 0 7! `0 2 g be a Borel chosen to give 0 D OG0 .`0 / 2 G 0   cross-section. For 0 -almost all `0 2 g0 , let ` 2 g be such that p.`/ D `0 . We can take an element T D T .`0 / of g depending on `0 so that g.`/ D RT C g0 .`0 / and that except for an -null set the Borel map  7! .jG0 ; `.T // would supply c0  R. Since the generalized vector ak does not an identification between GO and G depend up to a constant multiplication on the choice of the polarization, we can choose b0 so that the polarization b 2 M.`; g/ would be obtained as RT C b0 . Then if we express b 2 B D exp b as b D exp.tT /b0 with t 2 R and b0 2 B0 D exp.b0 /, we know that B=.B\gk1 Hgk / is isomorphic to RB0 =.B0 \gk1 Hgk /. We choose dk bP as dk bP D dtdk bP0 . In this situation, if we put  D `.T /, the measure  D 0  d2 meets the request: for any  2 D.G/, formula (8.1.2), the induction hypothesis on 0 and the Plancherel formula for R give Z m./ X h./ak ; ak id./ GO kD1

Z D

Z

b0 G

d0 .0 / Z

Z



e i t dt R

Z D

d0 .0 /

D

B0 =.B0 \gk Hgk1 /

f H .hgk exp.tT /b0 gk1 /`0 .b0 /dk bP0

X kD1

Z

Z

kD1

m.0 / Z

b0 G 

R

m.0 / Z d X f .h/dk hP 2 H=.H \gk Bgk1 /

H=.H \gk Bgk1 /

f .h/dk hP

H .hgk b0 gk1 /`0 .b0 /dk bP0 f

B0 =.B0 \gk Hgk1 /

X

m.0 /

b0 kD1 G

f

h0 ./ak 0 ; ak 0 id0 .0 / D H .e/:

Next, let us consider the case where -almost all orbits are saturated in the O direction g? 0 . Let  2 G be such that each connected component of ˝./ \  is

11.1 Penney’s Plancherel Formula

347

only one H -orbit of dimension 12 dim.˝.//. There exist finite irreducible unitary 1 s representations  0 ; : : : ;0 of G0 such that p .˝./ \  / is a disjoint union of 

˝0 .0 / \ f0 C h?;g0 ; 1  j  s whose connected components are presented j

j

j

j

as C0;1 ; : : : ; C0;ij with ij D m.0 /. The connected components of ˝./ \  are Ck D p 1 .C0;k / with 1  k  m.0 /; 1  j  s. C0;k associates with ak j 2 0  H;f   P H; j j 1 f H1 and Ck associates with ar 2 H1 , with r D i D1 m.0i /Ck. j j

j

j

j

0

By the induction hypothesis, we can appropriately choose d gP 0 ; dk hP 0 ; 1  k  c0 which satisfies the theorem at the stage m.0 / in order to get the measure 0 on G of G0 . Then, it is enough to take the image measure  of 0 by the induction of representations. In fact, if we consider the decomposition Z 0 D

GO

 d./ j

of 0 with respect to , the Psfibre over  consists of 0 with 1  j  s, and it is easily seen that  D j D1 cj ı0j with cj > 0. Here ı0j denotes the Dirac j c measure at the point 0 2 G0 . Let  2 D.G/. The expressions (11.1.2) applied to Pj 1 j h0 ./ak j ; ak j i and h./ar ; ar i with r D i D1 m.0i / C k, and the transfers by 0

0

intertwining operators permit us to choose d g; P dk hP so that j

h./ar ; ar i D cj h0 ./ak j ; ak j i 0

0

for all indices j; k and any  2 D.G/. With these choices we compute: for any  2 D.G/, Z

f

H .e/ D

X

m.0 /

b0 kD1 G

h0 ./ak 0 ; ak 0 id0 .0 /

Z D

D

D

Z GO

d./

b0 kD1 G

j Z X s m. X0 /

GO j D1 kD1

Z m./ X GO kD1

X

m.0 /

h0 ./ak 0 ; ak 0 id .0 /

j

cj h0 ./ak j ; ak j id./ 0

0

h./ak ; ak id./:

Thus, the theorem has been proved.



348

11 Plancherel Formula

Example 11.1.4. Let g D he1 ; e2 ; e3 ; e4 iR W Œe1 ; e2  D e3 ; Œe1 ; e3  D e4 . Put f D e4 relative to the dual basis of g and h D Re2 C Re4 2 S.f; g/. Considering f D e3 C e4 . 2 R/ in  D f C h? , g.f / D R.e2  e3 / ˚ Re4 . Hence, h C g.f / is a maximal isotropic subspace for Bf when  ¤ 0. Put G D exp g. For ` 2  such that `.e3 / ¤ 0, the intersection G  ` \  is composed of two lines each of which is a H -orbit with H D exp h, while, concerning ` 2  which vanishes at e3 , this intersection is a line which contains infinite H -orbits. Let p be the canonical map of g onto the orbit space g =G. Then, p. / D fGf I   0g and if we set  O  D .Gf O  / using the Kirillov map O W g =G ! G, Z  D .f; O h; G/ D indG H f ' 2

1

 d: 0

If we take a real polarization b D he2 ; e3 ; e4 i at the point f and realize  2 GO in L2 .R/ through the identification H 3  $ ˚ 2 L2 .R/ with ˚.t/ D .exp.te1 // .t 2 R/, the space H1 is realized as the space S.R/ of the rapidly decreasing functions and  1 H;f D Ca1 ˚ Ca2 ; H Here a1 W ˚ 7! ˚.0/ and a2 is given by ˚ 7! ˚.2/ when  ¤ 0 and by ˚ 7! d˚ dt .0/

when  D 0. In this way, H;f  dim H1 D 2; 

which is equal to the multiplicity of  in the irreducible decomposition of . Likewise, if  2 GO is not equivalent to any  ,  1 H;f H D f0g: To write down the concrete Plancherel formula, set Z f .g/ D .gh/f .h/dh .g 2 G/ H H

for 2 D.G/. Here dh D dx2 dx4 when h D exp.x2 e2 /exp.x4 e4 /. Now we put dg D ˘j4D1 dxj for g D ˘j4D1 exp.xj ej /. In this situation, the formula f H .e/

1 D 2

Z

1 0

fh . /a1 ; a1 i C h . /a2 ; a2 igd

holds. Indeed, put B D exp b and db D ˘j4D2 dxj for b D ˘j4D2 exp.xj ej /. Then, 1 D  . /a1 2 H1 D S.R/ is obtained by Z 1 .t/

D

.exp.te1 /b/f .b/db .t 2 R/: B

11.2 Vergne’s Decomposition Theorem

349

Therefore, Z /a1 ; a1 i

h .

D

.b/f .b/db: B

Similarly, 2 D  . /a2 2 S.R/ is obtained by Z 2 .t/

D

.exp..t  2/e1 /b/f .b/db .t 2 R/

B

and so Z /a2 ; a2 i

h .

D B

.b/f .b/db:

Thus, using the Plancherel formula for R, 1 2

Z

1 0

fh . /a1 ; a1 i C h . /a2 ; a2 igd 1 D 2 D

1 2 Z

Z

Z

1

d Z

0

R2

R

Z



dx3



.hexp.x3 e3 //f .h/dh

H

Z

e i x3 ddx3

.hexp.x3 e3 //f .h/dh H

.h/f .h/dh D

D

e i x3 C e i x3

H

f H .e/:

11.2 Vergne’s Decomposition Theorem If we go beyond the nilpotent case, we know few. We describe here a particular case. Let G D exp g be an exponential solvable Lie group, f 2 g ; h 2 M.f; g/ and H D exp h. The irreducible decomposition of  D indG H f was obtained by Vergne [77] and it was our starting point toward Theorem 8.1.9 in Chap. 8. We denote by U.f; h/ the set of the orbits ˝ 2 g =G whose intersections with  are non-empty open subsets of  . Theorem 11.2.1. When h 2 M.f; g/: (1) U.f; h/ is a finite set; (2) if ˝ 2 U.f; h/, the number c.˝/ of the connected components of ˝ \  is finite;P (3)  ' ˝2U.f;h/ c.˝/.˝/. O Proof. First of all we prepare a lemma.

350

11 Plancherel Formula

Lemma 11.2.2. Let h 2 M.f; g/. If we denote by V .f; h/ the set of all G-orbits ˝ D G.f C / such that h 2 M.f C ; g/;  2 h? , then U.f; h/ D V .f; h/. Besides, for ˝ 2 U.f; h/, the connected component containing a point f C  . 2 h? / of ˝ \ .f C h? / is equal to H .f C /. Proof. We repeat the same reasoning as in the complex case treated by Lemma 7.7.1 of Chap. 7. Let us show V .f; h/  U.f; h/. Take  2 h? such that h 2 M.f C; g/. Hence E D G.f C/\.f Ch? / is not empty. The question is to show that E is an open set of f C h? . Provided f C 2 E, 2 h? and f C 2 G.f C /. Hence h 2 S.f C ; g/ and dim.g.f C // D dim.g.f C// so that h 2 M.f C ; g/. Thus, as we saw in the proof of (3) H) (1) of Theorem 5.4.1 in Chap. 5, H .f C /  E becomes an open set of f C h? and the claim follows. Now let us show the inverse inclusion U.f; h/  V .f; h/. Take ˝ 2 U.f; h/ and set ˚

U1 D  2 h? W dim.g.f C // D dim.g.f // : Because h 2 S.f C ; g/ for arbitrary  2 h? , the dimension of g.f / is equal to the minimum of the dimensions of g.f C / when  moves through h? . Hence U1 is a non-empty Zariski open set of h? and dense in h? . Therefore, f C U1 meets ˝ \ .f C h? / at least at one point f C 1 and ˝ D G.f C 1 / belongs to V .f; h/. Lastly, take ˝ 2 U.f; h/. Let f C  . 2 h? / be a point of ˝ \ .f C h? / and C the connected component of ˝ \ .f C h? /, which is an open set of f C h? , containing f C . C is stable under the action of H and, for every point f C 0 of C , H .f C 0 / is a connected open set of ˝ \ .f C h? / and hence of C . Moreover, its complement set in C is a union of open H -orbits by the same reasoning. In consequence, H .f C / turns out to be open and closed in C and so coincides with C .  Let us proceed to prove Theorem 11.2.1 by induction on dim g as usual. Taking the last lemma into account, it is enough to show the following. (A) .f; O h; G/ is a finite sum of irreducible representations. (B) Every irreducible component of .f; O h; G/ is .f O C / corresponding to f C  such that h 2 M.f C ; g/ . 2 h? /. (C) Every irreducible representation corresponding to f C  such that h 2 M.f C ; g/ . 2 h? / appears as an irreducible component of .f; O h; G/ with multiplicity equal to the number c.˝; f; h; g/ of connected components of ˝ \ .f C h? / where ˝ D G.f C /. (I) Suppose that there is a commutative ideal a ¤ f0g such that f ja D 0. Then, since h C a 2 S.f; g/, h  a. Put A D exp a; GQ D G=A. Let p W G ! GQ be the canonical projection, dp W g ! gQ D g=a its differential, fQ the image of f Q G/ıp. Q in gQ  and hQ D dp.h/. Then, hQ 2 M.fQ; gQ / and .f; O h; G/ D . O fQ; h; By the induction hypothesis,

11.2 Vergne’s Decomposition Theorem

351

X

Q G/ Q ' . O fQ; h;

Q gQ /. Q fQ; h; Q c.˝; O ˝/

Q g/ Q ˝2V .fQ;h;Q

and we can write X

.f; O h; G/ '

Q gQ /. Q fQ; h; Q c.˝; O ˝/ıp:

Q g/ Q ˝2V .fQ;h;Q

Q Hence (A) is settled. Now let ˝Q be the G-orbit of fQ C Q .Q 2 hQ ? / and put Q Q  D ıdp. Then, . O ˝/ıp D .G.f O C // and h 2 M.f C ; g/ if hQ 2 Q Q M.f C ; gQ /. Hence (B) is settled. Finally, if h 2 M.f C ; g/ . 2 h? /, Q Q gQ /. ja D 0 by h  a. So, let Q 2 gQ  be such that  D ıp, then hQ 2 M.fQ C ; ?  Furthermore, the canonical bijection q W a ! gQ supplies a diffeomorphism Q \ .fQ C hQ ? /. Hence (C) is Q fQ C / between G.f C / \ .f C h? / and G. settled. (II) Suppose that there does not exist such an ideal a used in (I). If we take a minimal non-central ideal a of g, af ¨ g because f .Œg; a/ ¤ f0g. First assume  N h  a. Hence a  h  af . We  denote by ` 7! ` thef restriction map from g to f  f N .a / and put G0 D exp a . Clearly h 2 M.f ; a / and .f; O h; G/ ' indG O fN; h; G0 /: G0 . By the induction hypothesis, X

. O fN; h; G0 / '

N fN; h; af /. N c.˝; O ˝/

N ˝2V .fN;h;af /

and we can write X

.f; O h; G/ '

N fN; h; af /indG N c.˝; O ˝/: G0 .

N ˝2V .fN;h;af /

Now let ˝N be the G0 -orbit of fN C N with N 2 h? in .af / . Let  2 g be an N From j N a D 0, af C D af and by Lemma 5.3.25 in Chap. 5 extension of . N D .G.f indG O ˝/ O C //: G0 . Hence claim (A) follows, while, by Lemma 5.3.11, h 2 M.f C ; g/ if N af /. Claim (B) follows. h 2 M.fN C ; Next, if we take  2 h? such that h 2 M.f C ; g/, then N 2 h? and N af /. Hence, with ˝N D G0 .fN C /, N .G.f N h 2 M.fN C ; O C // D indG O ˝/ G0 . appears as irreducible component of .f; O h; G/. By Theorem 3.4.4 in the third N are all mutually non-equivalent. Therefore, chapter, the representations indG O ˝/ G . 0

352

11 Plancherel Formula

the multiplicity of the representation .G.f O C// in the irreducible decomposition N fN; h; af /. Now, r W g ! .af / denoting the of .f; O h; G/ is equal to c.˝; restriction map,   r G.f C / \ .f C h? / D ˝N \ .fN C h? / and G.f C / \ .f C h? / is equal to the whole inverse image of its projection. In fact, let f C 1 2 G.f C / \ .f C h? / and denote by ˝1 the G0 -orbit through fN C 1 . Then N .G.f O C 1 // ' .G.f O C // ' indG O 1 / ' indG O ˝/: G0 .˝ G0 . N Conversely, assume Hence, by Theorem 3.4.4 in the third chapter, fN C 1 2 ˝. ? ? N N N f C 1 2 ˝ \ .f C h / and let 1 2 h be an arbitrary extension of 1 . Then, N ' .G.f O ˝/ O C 1 // ' .G.f O C //: indG G0 . So, f C1 2 G.f C/\.f Ch? /. Thus, for any  2 h? such that h 2 M.f C; g/, N fN; h; af / c.G.f C /; f; h; g/ D c.˝; and claim (C) follows. Hereafter, we assume a 6 h, put h0 D h \ af ; h0 D h0 C a; k D h C a and denote by j the kernel of the action of h on a=.h \ a/. We separate the cases where h0 C j is equal to h or not. Suppose that h0 C j D h. Since h 2 M.f; g/, dim.h=h0 / D dim.a=.h \ a// by Lemma 5.3.12 in Chap. 5. Hence, if we define t; w as in Lemma 5.3.18, this lemma means h D t ˚ h0 ; h0 D h0 ˚ w; k D t ˚ h0 ˚ w; Œt; w  h \ a; O h; G/ and Bf gives a duality between t and w, and the representations .f; .f; O h0 ; G/ are equivalent. Since h0  a, if we set ˝ 0 D G.f C 0 / .0 2 .h0 /? /, the preceding arguments imply .f; O h; G/ '

X

c.˝ 0 ; f; h0 ; g/.˝ O 0/

˝ 0 2V .f;h0 /

and (A) is derived. To show assertion (B), it suffices to show that, for any 0 2 .h0 /? , there exists  2 h? such that f C  2 G.f C 0 /. In fact, since then dim.g.f C // D dim.g.f C 0 //, we have h 2 M.f C; g/ if h0 2 M.f C0 ; g/.

11.2 Vergne’s Decomposition Theorem

353

Since Bf gives a duality between t and w, there uniquely exists V .0 / 2 w so that 0 .T / D f .ŒV .0 /; T / for any T 2 t. Let us see that .exp.V .0 /// .f C 0 / belongs to f C k? . Since V .0 / 2 h0 , .exp.V .0 /// .f C 0 / belongs to f C .h0 /? . Hence it suffices to see that .exp.V .0 /// .f C 0 / belongs to f C t? . Because a  w is commutative, .exp.V .0 /// T D T  ŒV .0 /; T . Hence     exp.V .0 // .f C 0 /.T / D f .T / C 0 .T /  f ŒV .0 /; T  D f .T /: To show assertion (C), it suffices to show that, for  2 h? such that h 2 M.f C ; g/, there exists 0 2 .h0 /? so that f C 0 2 G.f C / and that c.G.f C 0 /; f; h0 ; g/ D c.G.f C /; f; h; g/: Let us proceed as in the above arguments. For  2 h? , there uniquely exists an element T ./ 2 t so that .Y / D f .ŒT ./; Y / for any Y 2 w. Then, .exp.T ./// .f C/ belongs to f Ck? . Indeed, as T ./ 2 h, .exp.T ./// .f C/ belongs to f C h? , while, because Œt; w  h, we can write .exp.T ./// .Y / D Y  ŒT ./; Y  C Y 0 for any Y 2 w. Here Y 0 2 Œh; h. Therefore, .exp.T ./// .f C /.Y / D f .Y / C .Y /  f .ŒT ./; Y / D f .Y /: Now we define the mapping R from f C h? to f C k? by R.f C / D .exp.T ./// .f C / and the mapping R0 from f C .h0 /? to f C k? by   R0 .f C 0 / D exp.V .0 // .f C 0 /: Now, if we show the following two claims (a), (b), then the desired assertion (C) will be settled. (a) Let  2 h? be such that h 2 M.f C ; g/ and C a connected component of G.f C/\.f Ch? /, then R.C / is a connected component of G.f C/\.f C k? / and the mapping C 7! R.C / supplies a bijection between the connected components of G.f C / \ .f C h? / and those of G.f C / \ .f C k? /. (b) Let 0 2 .h0 /? be such that h0 2 M.f C 0 ; g/ and C 0 a connected component of G.f C 0 / \ .f C .h0 /? /, then R0 .C 0 / is a connected component of G.f C 0 / \ .f C k? / and the mapping C 0 7! R0 .C 0 / supplies a bijection between the connected components of G.f C 0 / \ .f C .h0 /? / and those of G.f C 0 / \ .f C k? /.

354

11 Plancherel Formula

In fact, let  2 h? be such that h 2 M.f C ; g/. Then R.f C / 2 G.f C / \ .f C .h0 /? / and h0 2 M.R.f C /; g/. Hence the representation .G.f O C // appears as an irreducible component of .f; O h; g/ with multiplicity c.G.f C /; f; h0 ; g/, while, from (b) this value is equal to the number of connected components of G.f C / \ .f C k? /, and further from (a) also to c.G.f C /; f; h; g/. Thus we obtain the desired equality c.G.f C /; f; h; g/ D c.G.f C /; f; h0 ; g/: Because assertions (a), (b) are proved in the same way, let us show (a) for instance. Since T ./ D 0 if  2 k? , the mapping R is the identity mapping on f C k? . Now let C be a connected component of G.f C / \ .f C h? /, C is invariant by the group H . Thus R.C /  C \ .f C k? / and R.C / D C \ .f C k? /, while, G.f C / \ .f C k? / is the union of C \ .f C k? /, where C runs through the connected components of G.f C / \ .f C h? /. Hence (a) follows. Finally, assume h0 C j ¤ h. By Lemma 5.3.12(2) in Chap. 5, dim.h=h0 / D dim.a=.h\a// and the conditions of Proposition 5.3.19 in Chap. 5 are fulfilled. With the notations used there, j D h0 ; k D RX ˚ h0 ˚ RY , Here ŒX; Y  D Y; f .Y / D 1. Besides, vC and v denoting respectively a positive and a negative real number, we had .f; O h; K/ ' .f O vC ; h0 ; K/ ˚ .f O v ; h0 ; K/: Therefore, fQvC and fQv denoting respectively an extension of fvC and fv to g, .f; O h; G/ ' . O fQvC ; h0 ; G/ ˚ . O fQv ; h0 ; G/: h0  a and, if v ¤ 0, the orthogonal space of a with respect to BfQv is a proper Lie subalgebra of g. So, if we can choose vC ; v ; fQvC ; fQv so that h0 2 M.fQvC ; g/ \ M.fQv ; g/, the problem will be reduced to the case already treated. Let us see such a choice is indeed possible. Let p be a linear complement of k in g and we identify g with k ˚ p . Then the linear variety f C h? is identified with R  p by the mapping fQv 7! .v; fQv jp/. Since h0 2 S.fQv ; g/ for any v; fQv , h0 2 M.f; g/ implies that the set of the pairs .v; / 2 R  p such that h0 2 M.fv C ; g/ is a non-empty Zariski open set and hence dense. So, assume h0 2 M.fQvC ; g/ \ M.fQv ; g/. Then, from what we saw already, putting ˝C D G.fQvC C 0 / and ˝ D G.fQv C 0 / with 0 2 .h0 /? , .f; O h; G/ '

X

c.˝C ; fQvC ; h0 ; g/.˝ O C/

˝C 2V .fQvC C0 ;h0 /

˚

X ˝ 2V .fQv C0 ;h0 /

and assertion (A) holds.

c.˝ ; fQv ; h0 ; g/.˝ O /

11.2 Vergne’s Decomposition Theorem

355

To show assertion (B), it suffices to see that, for any 0 2 .h0 /? and v ¤ 0, there exists  2 h? so that f C  2 G.fQv C 0 /. Then, h 2 M.f C ; g/ if h0 2 M.fQv C 0 ; g/. Indeed, we choose a real number a.0 / so that fQv .Œa.0 /Y; X / D 0 .X /, i.e. 0 / a.0 / D   .X v . Then, exp.a.0 /Y /X D X  Œa.0 /Y; X  and ? exp.a.0 /Y /.fQv C 0 / 2 fQv C h0

since Y 2 h0 . Hence   exp.a.0 /Y /.fQv C 0 /.X / D f .X / C 0 .X /  fQv Œa.0 /Y; X  D f .X / and exp.a.0 /Y /.fQv C 0 / 2 fQv C k?  f C h? : We define the mapping Rv0 from fQv C .h0 /? to fQv C k? by Rv0 .fQv C 0 / D exp.a.0 /Y /.fQv C 0 /: Then as before, let 0 2 .h0 /? be such that h0 2 M.fQv C 0 / and C 0 a connected component of G.fQv C 0 / \ .fQv C .h0 /? /, then Rv0 .C 0 / D C 0 \ .fQv C k? / and the mapping Rv0 gives a bijection between the connected components of G.fQv C 0 / \ .fQv C .h0 /? / and those of G.fQv C 0 / \ .fQv C k? /. Therefore, we have c.G.fQv C 0 /; fQv ; h0 ; g/ D c.G.fQv C 0 /; fQv ; k; g/: Let us prove assertion (C). Let h 2 M.f C ; g/ and  2 h? . Then from the decomposition formula of .f; O h; G/, the multiplicity of the representation .G.f O C // is equal to c.G.f C /; fQvC ; h0 ; g/ C c.G.f C /; fQv ; h0 ; g/: Next, if we put ˚

.f C h? /C D f C ` 2 f C h? W .f C `/.Y / > 0 ; ˚

.f C h? / D f C ` 2 f C h? W .f C `/.Y / < 0 ;

356

11 Plancherel Formula

G.f C / \ .f C h? / is the union of G.f C / \ .f C h? /C and G.f C / \ .f C h? / . In fact, let f C ` 2 G.f C / \ .f C h? /, then h 2 M.f C `; g/ and Proposition 5.3.19(3) in Chap. 5 gives .f C `/.Y / ¤ 0. Hence each connected component of G.f C/\.f Ch? / is contained in either .f Ch? /C or .f Ch? / . We denote by c C .G.f C/; f; h/ and c  .G.f C/; f; h/ the number of connected components of G.f C / \ .f C h? /C and G.f C / \ .f C h? / respectively. From what we have seen until now, it is enough to show the equalities c C .G.f C /; f; h/ D c.G.f C /; fQvC ; h0 ; g/ D c.G.f C /; fQvC ; k; g/ and c  .G.f C /; f; h/ D c.G.f C /; fQv ; h0 ; g/ D c.G.f C /; fQv ; k; g/: Because it is the same thing, let us prove c C .G.f C /; f; h/ D c.G.f C /; fQvC ; k; g/: Let f C ` 2 .f C h? /C , then there uniquely exists a real number x.`/ satisfying exp.x.`/X /.f C `/ 2 fQvC C k? : In fact, since exp.x.`/X /.f C `/ 2 fQvC C h? , it suffices that exp.x.`/X /.f C `/.Y / D vC . Namely, e x.`/ D

vC : .f C `/.Y /

We define the mapping RC W .f C h? /C ! fQvC C k?  f C h? by RC .f C `/ D exp.x.`/X /.f C `/. Then, if C C is a connected component of G.f C / \ .f C h? /C , it turns out that RC .C C / D C C \ .fQvC C k? /. Thus, we find c C .G.f C /; f; h/ D c.G.f C /; fQvC ; k; g/ and assertion (C) holds. To examine the concrete Plancherel formula, we begin with a lemma:



Lemma 11.2.3. Take h 2 M.f; g/. Then there exists b 2 I.f; g/ provided   with  and denote by H1 0 the following properties. We put B D exp b;  D indG B f the subspace of H1 composed of all functions with compact supports modulo B. Then:

11.2 Vergne’s Decomposition Theorem

357

(1) H \B;H .h/ H \B;B .h/ D 1 for any h 2 H \ B; (2) HB is a closed set of G; (3) From (1) and (2), we are able to consider the anti-linear form   a W H1 0 3  7!

I

1=2

H=.H \B/

.h/f .h/ H;G .h/dH;H \B .h/ 2 C

H;f 1=2  H;G and this form is extended to an element of H1 . Proof. If there is a non-trivial ideal of g where f vanishes, we may immediately pass to the quotient by this ideal and apply the induction hypothesis. From now on we suppose that there is no such ideal. Let a be a minimal non-central ideal, g0 D af ¤ g; G0 D exp.g0 /; h0 D h \ g0 C a 2 M.f; g/ and H 0 D exp.h0 /. By the induction hypothesis, at the stage of the subgroup G0 there is b 2 I.f0 ; g0 / with three required properties. Here f0 D f jg0 2 g0 . It is a matter of the case where a 6 h. (1) Put h0 D h\g0 and H0 D exp.h0 /. For all h D exp X 2 H \B D H0 \B .X 2 h0 \ b/, Tr adh=h0 X C Tr ada=.h\a/ X D 0:

(11.2.1)

However, b is chosen so that H 0 \B;H 0 .h0 / H 0 \B;B .h0 / D 1 .h0 2 H 0 \ B/ might hold, and this means Tr adh0 =.h0 \b/ X C Tr adb=.h0 \b/ X D 0 for all X 2 h0 \ b D .h \ b/ C a. Again, this is the same as Tr adh0 =.h0 \b/ X C Tr adb=.h0 \b/ X  Tr ad.h0 \b/=.h0 \b/ X D 0

(11.2.2)

holds. From (11.2.1) and (11.2.2), identifying .h0 \ b/=.h0 \ b/ with a=.h \ a/, Tr adh0 =.h0 \b/ X C Tr adb=.h0 \b/ X C Tr adh=h0 X D 0 for any X 2 h \ b. Besides, Tr adh=.h\b/ X C Tr adb=.h\b/ X D 0: So, we obtain the desired equality. (2) We denote by z the centre of g and by j (resp. g1 ) the kernel of the adjoint representation of h (resp. g) on a=.h \ a/ (resp. a=.z \ a/). Then, there occurs

358

11 Plancherel Formula

two possibilities: either j C h0 D h or j C h0 D h0 . In the first case, let m be a linear complement to h0 in h contained in j. Then a basis of m constitutes a part of a coexponential basis to g0 in g and so HB is a closed set of G. If j C h0 D h0 , then j D h0 ; h \ a D z \ a and dim.h=h0 / D dim.a=.h \ a// D 1. Hence g.f /  g1 and by the induction hypothesis we can choose our polarization b in g1 . If we take the coexponential basis X 2 h; X 62 j to g1 in g, HB D exp.RX /H 0 B is a closed set of G because H 0 B is a closed set of G1 D exp.g1 /. (3) Taking (1) into account, with  2 H , we can apply the linear functional H;H \B to the function 1=2

H W H 3 h 7! .h/f .h/ H;G .h/:   Now, when  moves in H1 0 , we have by (2) that I H;H \B .H / D

1=2

H=.H \B/

.h/f .h/ H;G .h/dH;H \B .h/ < 1:

  Then the anti-linear form H1 0 3  7! H;H \B .H / could be uniquely H;f 1=2  H;G . This is easily derived from extended to a nonzero element a of H1 what we have seen until now, but we add some comments. If we can choose b 2 I.f; g/ so that an intertwining operator R between  D G indG B f and  D indH f is given by the formula I

1=2

.R/.g/ D H=H \B

.gh/f .h/ H;G .h/dH;H \B .h/ .g 2 G/;

the generalized vector a D ı ıR is what we are looking for. Now we think about R. 0 Put  0 D indG H 0 f . If an intertwining operator T between  and  is constructed by giving a sense to the formal operator defined by I

1=2

.T /.g/ D H=H0

.gh/f .h/ H;G .h/dH;H0 .h/ .g 2 G/

for 2 H 0 , the desired R is obtained from the induction hypothesis applied to G0 and the transitivity of ; . As for T , it is enough to consider the Lie subalgebra k D h C a. So, we assume g D k. For instance, let us examine the case where  splits into two parts by the modification h ! h0 . Let a D RY (or a D RY C z), f .Y / D 1; h D RX C h0 and ŒX; Y  D Y . Using the one-parameter subgroup exp.RY / (resp. exp.RX //, the space H (resp. H 0 ) is identified with L2 .R/ and

11.3 Examples of Penney’s Plancherel Formula

Z

359

.exp.sY /exp.tX //e t =2 dt

.T /.s/ D .T /.exp.sY // D R

Z

t

D

.exp.tX /exp.se Y //e

t =2

Z

R

for

t

.exp.tX //e i se e t =2 dt

dt D R

2 H 0 . Hence, by change of variables, Z

ˇ ds ˇ

kT k2 D Z

R

ˇ ds ˇ

D R

Z Z

ˇ2 t .exp.tX //e i se e t =2 dt ˇ R C1 0

Z

d u ˇ2 .exp.. log u/X //e i su p ˇ u

C1

D 2

j .exp.. log u/X //j2 Z

0

du u

j .exp.vX //j2 d v D 2k k2 :

D 2 R

When  does not split, T is similarly interpreted as a Fourier transformation.  In the above situation we have the following: Theorem 11.2.4 ([31]). When ˝ runs over U.f; h/, take `k˝ .1  k  c.˝// arbitrarily in each connected component C˝k of ˝ \  . At these points `k˝ 2 g .˝ 2 U.f; h/; 1  k  c.˝//, we choose polarizations bk˝ 2 I.`k˝ ; g/ of H;f 1=2  H;G k 1 Lemma 11.2.3 and construct a˝ 2 H.˝/ . Then by means of matrix O k elements concerning these a˝ appropriately normalized, the concrete Plancherel formula for  is written down: for any  2 D.G/,

f

H .e/ D

X

X

c.˝/ k k h.˝/a O ˝ ; a˝ i:

˝2U.f;h/ kD1

Examining various cases, we can prove this theorem by induction. However, we here abstain from mentioning its proof and give in what follows certain examples.

11.3 Examples of Penney’s Plancherel Formula Example 11.3.1. Let g be the ax C b Lie algebra taken up in Chap. 4: g D Re1 C Re2 ; Œe1 ; e2  D e2 . f D e2 and h D Re1 2 M.f; g/. The irreducible unitary representations of the completely solvable Lie group G D exp g, which are of

360

11 Plancherel Formula

infinite dimension, are only ˙ corresponding to two open orbits ˙Ge2 . Now the monomial representation  D indG H f ; H D exp h, is equivalent to the direct sum C ˚  . We realize ˙ by means of B D exp.Re2 / as indG B ˙e2 and identify H˙ with L2 .R/ by the mapping 7! Q .t/ D .exp.te1 // .t 2 R/. For 2 H1˙ , ..˙ .e2 //m / .t/ D .˙i /m e mt .t/ 2 L2 .R/ with any m 2 N. Thus, ˇ ˇ

Z e

t =2

ˇ Q .t/dt ˇ 

Z

0

je

t =2

Q .t/jdt C

Z

1

R

Z

0

je t Q .t/j2 dt

e t dt 1

Z

C1

C

t

C1

j Q .t/j2 dt

e dt

1=2

0 C1

D

1=2

1

1=2 Z

0

Z

je t =2 Q .t/jdt

0

1=2 Z

0



C1

j Q .t/j2 dt

1=2

Z

0

je t Q .t/j2 dt

C

1=2  k k C k˙ .e2 / k:

1

0

From this, if we set Z a˙ . / D

.exp.te1 //e t =2 dt

R

H;f 1=2  H;G for 2 H1˙ , then a˙ 2 H1 . Take  2 D.G/. Choosing dg on G, ˙ we compute Z



.C ./aC / .t/ D

R2

.exp.se1 /exp.ue2 //e i ue

st

e .st /=2 dsd uI

further, Z

Z

C1

hC ./aC ; aC i D

dv R2

0

.exp.se1 /exp.ue2 //e i uv e s=2 dsd u:

Likewise, Z h ./a ; a i D

Z

0

dv 1

R2

.exp.se1 /exp.ue2 //e i uv e s=2 dsd u:

In this way our concrete Plancherel formula for  reduces, normalizing appropriately various measures, to the obvious formula

11.3 Examples of Penney’s Plancherel Formula

361

hC ./aC ; aC i C h ./a ; a i Z f D2 .exp.se1 //e s=2 ds D H .e/ D h./ı ; ı i: R

If we consider the trivial representation 0 of G, clearly ˝.0 /\ D ˝.0 / D f0g, which is connected and one H -orbit. Besides, if we regard 0 as constructed by means of g 2 I.0; g/, the closure of the set fg 2 GI g.0 C g? / \  ¤ ;g is the  H;f 1=2 H;G whole G. In spite of these facts, H1 D f0g. 0 Example 11.3.2. Let G D G3 .˛/ D exp.g3 .˛//; g3 .˛/ D hT; X; Y iR taken up in Chap. 4: ŒT; X  D X  ˛Y; ŒT; Y  D Y C ˛X . Take f D X  2 g3 .˛/ and h D RX . Here we may assume that ˛ is negative. Let us parameterize using .x; y/coordinates the orbit passing ` D .1; / 2 a D RX  C RY  . Here, x.t/ D e t .cos.˛t/   sin.˛t//;

(11.3.1)

y.t/ D e t .sin.˛t/ C  cos.˛t//:

(11.3.2)

If the line x D 1 is tangent to the orbit of `, then .dx=dt/t D0 D 1  ˛ D 0 and  D 1=˛, We indicate this value by 0 . We denote by t  the first positive number t satisfying e t .cos.˛t/  .1=˛/ sin.˛t// D 1 by 1 the y-coordinate of the crossing point and choose ` D .1; / 2 a ; 0   < 1 as the representatives of the orbits which encounter the affine space  . All these orbits are saturated in the direction of RT  . Using the parametrization of the orbits and imposing e t .cos.˛t/   sin.˛t// D 1; let us look for the crossing point with  . With  satisfying sin  D .1 C /1=2 ; cos  D .1 C 2 /1=2 ; e t .1 C 2 /1=2 cos.˛t C / D 1: Hence, e t .sin.˛t/ C  cos.˛t// D .1 C 2 /1=2 e t sin.˛t C / D tan.˛t C /: Thus `n D .1; tan.˛tn C //; `0 D `. Choosing arbitrarily gn 2 G such that gn W ` 7! `n , we construct the map I an W  7!

1=2

H=.H \gn Bgn1 /

.hgn /f .h/ H;G .h/d .h/ . 2 H1 /:

(11.3.3)

362

11 Plancherel Formula

This last value is nothing but .gn /. We put B D exp b according to the polarization b D RX C RY and denote by H the representation space of  D ` D indG B ` .  1 H;f 1=2 H;G . So, if we compute for 2 D.G/, As is easily seen, an 2 H .. /an /.g/ D 1 G .gn /

Z B

.gbgn1 /` .b/db .g 2 G/:

Moreover, Z h. /an ; an i D

R

f H .exp.sY //exp.i s

tan.˛tn C //ds:

Writing simply y instead of y.t/, the formulas (11.3.1), (11.3.2) give .1  ˛y/.dt=d/ D e t sin.˛t/;

(11.3.4)

.dy=d/  .y C ˛/.dt=d/ D e t cos.˛t/:

(11.3.5)

On the other hand, Taking e 2t D .1 C y 2 /=.1 C 2 / into account, f.1 C 2 /.dt=d/ C g.1 C y 2 /=.1 C 2 / D y.dy=d/:

(11.3.6)

Equations (11.3.4), (11.3.5) imply .dy=d/ C .1  ˛y  y  ˛/.dt=d/ D y; .dy=d/ C .˛y    y  ˛/.dt=d/ D 1: Therefore, .  y/.dy=d/ C .1  ˛/.1 C y 2 /.dt=d/ D 0: Substituting (11.3.6), .  y/.dy=d/ C .1  ˛/fy.dy=d/  .1 C y 2 /=.1 C 2 /g D 0; Namely, .1  ˛y/.dy=d/ D .1 C ˛/.1 C y 2 /=.1 C 2 /: Finally, for  not equal to 0, .dy=d/ D .1  ˛/.1 C y 2 /=.1  ˛y/.1 C 2 /:

11.3 Examples of Penney’s Plancherel Formula

363

Now the formula which we must show becomes the following: because we may properly multiply an by a constant if necessary, yn D yn ./ denoting tan.˛tn C /, Z f H .e/

1

D

./d 0

1 X

Z

f H .exp.sY //exp.i s

.n/ R

nD0

D .2/1=2

!

Z

1

./ 0

1 X

.n/.

tan.˛tn C // ds

f H /b.yn /d;

nD0 f

here ./ denotes a certain measurable function, .n/  0 and . H /b the Fourier f transform of H ı exp. In fact, set ./ D .1  ˛/=2.1 C 2 / and .n/ D .1 C yn2 /j1  ˛yn j, that is, take f.1 C yn2 /=j1  ˛yn jg1=2 an . Then, .2/

1=2

Z

1 0

D.2/1=2 D.2/

1=2

f H /b.yn /.n/

./d

nD0

X1 Z nD0

Z . R

!

1 X .

1

. 0

f H /b.yn /.1

f H /b.s/ds

D.

f H

 ˛/.1 C yn2 /=j1  ˛yn j.1 C 2 /d

ı exp/.0/ D

f H .e/:

Example 11.3.3. Like the last example let G D G3 .˛/ D exp.g3 .˛//. Here take h D RT and let f 2 g be arbitrary. Then,  D f C h? D f .T /T  C RX  C RY  : A generic orbit has its representative O D .cos /X  C .sin /Y  and corresponds to the irreducible representation  D indG B O of G. Here B is the analytic subgroup having the polarization b D RX C RY as Lie algebra. In this case the usual formula (11.3.3) to obtain an H -semi-invariant generalized vector gives Z

1=2

a W  7! H

.h/f .h/ H;G .h/dh:

We know by similar reasoning as in the case of ax C b that a fulfils our request. For 2 D.G/, Z . . /a / .g/ D B

f H .gb/O .b/db

.g 2 G/;

364

11 Plancherel Formula

Z h . /a ; a i D

Z e 2t dt R

B

f H .b/` .b/db;

where ` D hO D e t cos.˛t C /X  C e t sin.˛t C /Y  ; h D exp.tT /. In this fashion, .2/

2

Z

2

h . /a ; a id 0

D .2/

2

Z

Z

2

Z 2t

d

e dt R

0

R2

f H .exp.b1 X

C b2 Y //

   exp i e t .b1 cos.˛t C / C b2 sin.˛t C // db1 db2 : If we perform the change of variables x D e t cos.˛t C /; y D e t sin.˛t C /; its Jacobian is @.x; y/[email protected]; / D e 2t I dxdy D e 2t dtd and so .2/2

Z

2

h . /a ; a id 0

D .2/2

Z

Z dxdy R2

D. where .

f  H / .0; 0/

f  H / .b1 ; b2 /

D

f  H / .b1 ; b2 /exp.i.xb1

. R2

D

C yb2 //db1 db2

f H .e/;

f H .exp.b1 X

C b2 Y //.

Example 11.3.4. Let G D exp.g4 /; g4 D hT; X; Y; ZiR I ŒT; X  D X; ŒT; Y  D Y; ŒX; Y  D Z (split oscillator group). In matrix form 1 0 1 010 100 T D @0 0 0 A ; X D @0 0 0 A ; 000 001 1 0 1 0 000 001 Y D @0 0 1A ; Z D @0 0 0A : 000 000 0

11.3 Examples of Penney’s Plancherel Formula

365

80 9 1 < wx z = G D exp g D @ 0 1 y A I w > 0; x; y; z 2 R : : ; 0 0w Set f .˛; ˇ/ D ˛T  C ˇZ  2 g4 with ˇ ¤ 0 and ˝.˛; ˇ/ D Gf .˛; ˇ/. Now take f D f .˛0 ; ˇ0 /. (i) First let h D RT C RX . Then, Z  D indG H f '

R

.˝.˛ O 0 ; ˇ//dˇ:

Using ` D f .˛0 ; ˇ/ 2  \ ˝.˛0 ; ˇ/ and a polarization b D hT; X; ZiR at `, we construct .˛0 ; ˇ/ D indG O 0 ; ˇ// with B D exp b. In B ` D .˝.˛ this situation, aˇ given as usual by haˇ ; i D .e/ clearly fulfils our request. b D h C RZ and Z is a central element. So, 

1 H.˛ 0 ;ˇ/

H;f 1=2 G;H

2 D.G/, .˛0 ; ˇ/. /aˇ D

Now, for

  .˛0 ; ˇ/. /aˇ .g/ D

Z

D Caˇ : ` B,

i.e. 1=2

.gb/` .b/ B;G .b/db .g 2 G/;

B

then Z

1=2

h.˛0 ; ˇ/. /aˇ ; aˇ i D B

.b/` .b/ H;G .b/db:

Therefore, .2/1

Z R

h.˛0 ; ˇ/. /aˇ ; aˇ idˇ

D .2/

1

Z

Z dˇ R

R

f H .exp.wZ//exp.iˇw/d w

D

f H .e/:

(ii) Secondly, let h D RT C RZ. If we take ` D f .˛; ˇ0 / D ˛T  C ˇ0 Z  , then  \ ˝.˛; ˇ0 / D ˛0 T  C xX  C yY  C ˇ0 Z  I xy D ˇ0 .˛  ˛0 /: If the value .exp.aX /exp.bY /`/ .T / D .exp.bY /`/.T C aX / D `.T C aX  bY C abZ/ D ˛ C abˇ0

366

11 Plancherel Formula

is equal to f .T /, ˛ C abˇ0 D ˛0 , i.e. abˇ0 D ˛0  ˛. Modifying the elements of the basis by appropriate constants, we may assume ˇ0 D 1. Then, the above equality becomes ab D ˛0  ˛ and, for f .˛; ˇ0 / such that ˛ ¤ ˛0 , gj f .˛; ˇ0 / 2  .j D 1; 2/ with g1 D exp X exp..˛0  ˛/Y /; g2 D exp.X /exp..˛  ˛0 /Y /. As in case (i), we realize the representation .˛; ˇ0 / D indG B .˛; ˇ0 / D 1 .˝.˛; O ˇ0 //. Take  2 H.˛;ˇ and recall the usual formula: 0/ I ha˛1 ;i D Z

1=2

H=.H \g1 Bg11 /

.hg1 /f .h/ H;G d .h/

.exp.tT /exp X /e i t ˛0 dt D

D

R Z 1

D

Z .exp.e t X //e i t .˛˛0 /Ct =2 dt R

.exp.sX //s i.˛˛0 /1=2 ds: 0

This last integrand is integrable and H;f 1=2  H;G 1 a˛1 2 H.˛;ˇ : / 0 Similarly the formula I ha˛2 ; i D Z

1=2

H=.H \g2 Bg21 /

.hg2 /f .h/ H;G .h/d .h/

1

D

.exp.sX //s i.˛˛0 /1=2 ds 0

defines the nonzero element  H;f 1=2 H;G 1 a˛2 2 H.˛;ˇ : 0/ Let a be any element of this space. By the semi-invariance of a relative to h D exp.tT /; t 2 R;, he i t .˛0 ˛/t =2 a; .exp.xX //i D ha; .exp.e t xX //i: Thus, if the support of  is contained in RC D fs 2 RI s > 0g, Z ha; i D c1

RC

.exp.xX //x i.˛˛0 /1=2 dx .c1 W constant/;

11.3 Examples of Penney’s Plancherel Formula

367

that is, a D c1 a˛1 (c1 : constant) on RC . Likewise, a D c2 a˛2 (c2 : constant) on R D fs 2 RI s < 0g. Now assuming that the support of a is contained in B, we write aD

m X

j /.0/: O j Dj ; hDj ; i D .d j =dx

j D0

O Here .x/ D .exp.xX //. By the semi-invariance of a relative to h D exp.tT /, e i ˛0 t

m X

j /.0/ D O .d j =dx

j D0

m X

j /.0/e .j C1=2/t e i ˛t : O j .d j =dx

j D0

j O Here if we take  satisfying .d j =dx /.0/ D ıj m,

m e i ˛0 t D m e .mC1=2/t e i ˛t .t 2 R/: Thus m D 0 and a D 0. In consequence,  H;f 1=2 H;G 1 H.˛;ˇ D Ca˛1 ˚ Ca˛2 : / 0 Let us consider the concrete Plancherel formula for the monomial representation  D indG H f ; f D f .˛0 ; ˇ0 /. To simplify the writing, we write  in place of .˛; ˇ0 / below. Let 2 D.G/ and assume that  2 H1 has compact support j modulo B. Writing simply aj instead of a˛ .j D 1; 2/, we compute I h. /a ; i D

I

1

H=.H \g1 Bg11 / 1=2

d .h/

Z dG;B .g/

G=B

B

.gbg11 h1 /

1=2

 .g/ B;G .h/` .b/f .h/ 1 G .h/ H;G .h/db: As we will see in what follows, we can change the order of the first two integrals in the right member. First G .h/ D H;G .h/ D 1. In the above expression, we write g as g D exp.xX / and make x move in some finite interval J . We denote the integrand by .h; g; b/ and write h D exp.tT /; b D exp.sT /exp.yY /exp.wZ/. Then, Z Z O .h; g; b/db D .x/ .exp.xX /exp.yY /exp.wZ/exp..˛  ˛0 /Y / B

R3

 exp.X /exp.tT //e s=2 e i.wC˛s˛0 / dsdyd w:

368

11 Plancherel Formula

Here, exp.xX /exp.sT /exp.yY /exp..˛  ˛0 /Y /exp.X /exp.tT /   Dexp .w C e s x.y C ˛  ˛0 //Z      exp e s .y C ˛  ˛0 /Y exp .x  e s /X exp..s  t/T /: Taking this into account, ˇ ˇ

Z

ˇ ˇ ˇ O ˇ .h; g; b/db ˇ  ˇ.x/

Z

  j .exp.wZ/exp e s yY R3

B

   exp .x  e s /X exp..s  t/T /je s=2 dsdyd w Z O D j.x/j j .exp.wZ/exp.yY / R3

    exp .x  e sCt /X exp.sT / je .sCt /=2 dsdyd w: Since x moves in the finite interval J , this expression is integrable with respect to dtdx and we can change the order of the first two integrals in the equality concerning h. /a1 ; i. Thus, we get I

I

h. /a ; i D 1

dG;B .g/ G=B

Z H=.H \g1 Bg11 /

d .h/ B

.gbg11 h1 /

1=2

1=2

 .g/ B;G .b/` .b/f .h/ 1 G .h/ H;G .h/db: Hence, for g 2 G, I .. /a1 /.g/ D

I D

Z H=.H \g1 Bg11 /

 H=.H \g1 Bg11 /

d .h/

1=2 B;G .b/` .b/f

I

d .h/ Z 

B

1=2

.h/ 1 G .h/ H;G .h/db 1=2

B=.B\g11 Hg1 /

B\g11 Hg1

.gbg11 h1 /

` .b/f .h/ 1 O G .h/ H;G .h/d .b/

.gbb01 g11 h1 /

1=2

 B;G .b0 /` .b0 / B\g1 Hg1 ;B .b0 / B\g1 Hg1 .b0 /db0 ; 1

1

where O D B;B\g1 Hg1 . Exactly the same arguments as above permit us to change 1 the order of the first two integrals. In consequence,

11.3 Examples of Penney’s Plancherel Formula

I .. /a1 /.g/ D

1=2

B=.B\g11 Hg1 /

` .b/ B;G .b/d O .b/

I



369

1=2

H=.H \g1 Bg11 /

f .h/ 1 G .h/ H;G .h/d .h/

Z



.gbb01 g11 h1 /

B\g11 Hg1

 ` .b0 / B;G .b0 / B\g1 Hg1 ;B .b0 / 1 .b0 /db0 : B\g 1 Hg 1=2

1

1

1

This last integral is equal to Z

.gbg11 b 01 h1 /g1 ` .b 0 /

H \g1 Bg11

1=2 .b 0 / g1 Bg1 \H;g1 Bg1 .b 0 / g1 Bg1 \H .b 0 /db 0 : 1 1 1 1 1 Bg1 ;G

 g

Although it is easily verified in our case, if the equality H .b 0 / g1 Bg1 \H .b 0 / D g1 Bg1 .b 0 / .b 0 2 g1 Bg11 \ H / 1=2

1

1

holds, we get 1=2 .b 0 / g1 Bg1 \H;g1 Bg1 .b 0 / g1 Bg1 \H .b 0 / 1 1 1 1 1 Bg1 ;G

g

0 0 0 D 1 H .b / H;G .b / g1 Bg 1 \H;H .b / 1=2

1

and so I .. /a1 /.g/ D Z

1=2

B=.B\g11 Hg1 /

1=2

 H

.gbg11 h/f .h/ H;G .h/dh

I D

B;G .b/` .b/d O .b/

B=.B\g11 Hg1 /

f 1=2 1 O .b/: H .gbg1 /` .b/ B;G .b/d

Likewise, I .. /a /.g/ D 2

B=.B\g21 Hg2 /

f 1 Q .b/; H .gbg2 /` .b/d

370

11 Plancherel Formula

where Q D B;B\g1 Hg2 . From what we have seen, 2

I h. /a ; a i D 1

1=2

1

H=.H \g1 Bg11 /

I 

` .h/ H;G .h/d .h/ f 1=2 1 O .b/ H .hg1 bg1 /` .b/ B;G .b/d

B=.B\g11 Hg1 /

Z

e i t .˛0 ˛/Ct =2 dt

D R

Z

 t   i ˛xx=2 f dxdy H exp e X exp.xT /exp.yY /exp.X / e

 R

Z

e i.t x/.˛0˛/C.t Cx/=2 dt

D

Z

R

R2

 t f H exp .e

  x  e x /X exp.yY / e iye dxdy:

Performing the change of variables t ! t C x; e x D s, we get Z

e i t .˛0 ˛/Ct =2 dt

h. /a1 ; a1 i D R

Z  RRC

 t f H exp s.e

   1/X exp.yY / e iys dyds:1

Similarly, I h. /a ; a i D 2

1=2

2

H=.H \g2 Bg21 /

I  Z D

e

B=.B\g21 Hg2 / i t .˛0 ˛/Ct =2

R

f .h/ H;G .h/d .h/ f 1=2 1 Q .b/ H .hg2 bg2 /` .b/ B;G .b/d

Z

dt RRC

  f H .exp s 1

   e t X exp.yY //e iys dyds:

f H .exp.s.1

 e t /X /exp.yY //e iys dyds:

Finally, h. /a1 ; a1 i C h. /a2 ; a2 i Z Z D e i t .˛0 ˛/Ct =2 dt R2

R

Now, if we set Z  .t/ D R2

 f H exp s.1

   e t /X exp.yY / e iys dyds;

11.3 Examples of Penney’s Plancherel Formula

371

this function is infinitely differentiable at t ¤ 0. In fact, in this case the integral is f taken on a compact set. Further, the function R2 3 .s; y/ ! H .exp.sX /exp.yY // 2 1 belongs to D.R /. If we make an L -approximation with a function of the form X

j .s/j .y/ . j ; j 2 D.R//;

j

for any positive number , we can choose j ; j in such a fashion that j .t/  .2/

1=2

XZ R

j

 

j s.1  e t / bj .s/dsj < 

holds for arbitrary t 2 R. Here bj denotes the Fourier inverse transform of j . When t tends to 0, the limit of Z  

j s.1  e t / bj .s/ds R

is equal to .2/1=2 j .0/j .0/. If we take this into consideration,  .t/ is continuous even at t D 0. Lastly, let us consider the function e t =2  .t/. When t ! 1, this function is rapidly decreasing because of e t =2 . Let us examine its movement when t tends to C1. By a change of variables, we have e t =2  D e t =2 .1  e t /1

Z R2

f iys.1et /1 dsdy H .exp.sX /exp.yY //e

and je t =2  .t/j  e t =2 j1  e t j1

Z f H .exp.sX /exp.yY //jdsdy:

j R2

Hence, e t =2  .t/ is rapidly decreasing when t tends to C1. Thus, the Fourier inversion formula supplies the concrete Plancherel formula for the monomial representation  D indG H f : .2/2

Z R

  h. /a˛1 ; a˛1 i C h. /a˛2 ; a˛2 i d˛ D .2/

1

Z R

f iys H .exp.yY //e dsdy

D

f H .e/:

372

11 Plancherel Formula

11.4 Bonnet’s Plancherel Formula Until now we have tried to write down explicitly Penney’s abstract Plancherel formula for monomial representations  D indG H f with finite multiplicities. At the end of this chapter, we will treat Bonnet’s Plancherel formula in the case of nilpotent Lie groups. These two Plancherel formulas seem to be essentially the same, however the latter does not put in front the canonical irreducible decomposition of . Let dg, dh be Haar measures on G D exp g; H D exp h respectively and  be a unitary character of H . According to Bonnet [12], as a distribution of positive type on G,  has its Fourier transform .; U /. Namely, there exist a positive measure  on GO and a field of nuclear operators U , U W H1 ! O so that the abstract Plancherel formula H1 . 2 G/ Z

Z .h/.h/dh D

GO

H

Tr ../U / d./

holds for all  2 D.G/. Here Tr denotes the trace of the operator. If we identify .; U / and .0 ; U 0 / when 0 D  ( > 0) and U 0 D 1 U , the pair .; U / satisfying the formula is unique. In order to describe this formula explicitly, we write  D f using f 2 g . Then the measure  in question is noting but the measure which appears in the canonical decomposition (8.1.1) of the monomial representation  D indG H f . Employing the notations used before, let us first look at the case where H is a normal subgroup. G acts on h and it turns out that G` \  D P `, P D G.f jh /, for all ` 2  . Moreover, it is well known [14] that the multiplicities m./ in formula (8.1.1) are uniformly equal to either 1 or 1. In this situation, the arguments developed in Grélaud [43] work well. We realize  2 GO belonging to the support of  by means of a polarization b at ` 2 O1 ./ \  :  D indG B ` with B D exp b. What simplifies the state of affairs in this case is that we can choose b in such a way that it contains h. Hence H  B  P . For ˛; ˇ 2 H1 , the function ˛ ˇN is B-invariant from the right and integrable with respect to P -invariant measure d gP on P =B. We define the nuclear operator U from H1 to H1 by the formula Z hU ˛; ˇi D

˛.g/ˇ.g/d g: P P =B

This operator is a positive self-adjoint operator, while, for  2 D.G/, ./ is a operator with integral kernel which is expressed by the formula Z

.xby 1 /` .b/db

K .x; y/ D B

with .x; y/ 2 G  G. Here db is a Haar measure on B. Exactly as in [43],

11.4 Bonnet’s Plancherel Formula

373

Z

Z

Tr ../U / D

Z

K .g; g/d gP D P =B

Z D

P =B

Z d gP

P =B

d gP

.gbg 1 /` .b/db

B

 g .exp X /e i `.X / dX; b

Here  g .x/ D .gxg 1 / .x 2 G/. As `jb ¤ 0 if ` ¤ 0, we take a linear complement t to b in g in such a manner that `jt D 0. Taking B` D ` C b? into account and denoting by F . / the Fourier transform of 2 D.G/, We derive from the Fourier inversion formula that Z Z Z Z Tr ../U / D d gP d dY  g .exp.X C Y //e i `.X /Ci .Y / dX Z

P =B

Z

D

d gP Z

P =B

Z

D

b?

b?

t

Z

 g .exp X /e i.`C /.X /dX

d g

d bP

d gP P =B

B=G.`/

D

b

Z

Z .exp X /e i b`.g

1 X /

dX

g

F . ı exp/.g  `/d gP Z

P =G.`/

D

F . ı exp/./d: G`\

In this way, ˝./ denoting the orbit O1 ./, Z Tr ../U / D

F . ı exp/./d: ˝./\

When H is trivial, this formula is evidently reduced to Kirillov’s character formula. From this immediately results the Plancherel formula: regarding all  2 D.G/, Z

Z

Z

.h/f .h/dh D H

Z

GO

dX D

F . ı exp/. /d 

h

Z D

.exp X /e

if .X /

F . ı exp/./d D

d./ ˝./\

Z GO

Tr ../U / d./:

Now we abandon the assumption that H is a normal subgroup, and show at first a lemma. Abusing a little the notation, we write  =G instead of .G /=G. Lemma 11.4.1. The Lebesgue measure on  is decomposed relative to the base measure  on  =G as

374

11 Plancherel Formula

Z D

˝ d.˝/;  =G

where ˝ is a measure supported on the G-orbit ˝. Proof. The proof is reduced to the case where h contains the centre z of g, and again by a classic reasoning to the case where dim z D 1; f jz ¤ 0. We take in g a Heisenberg triplet fX; Y; Zg; ŒX; Y  D Z; such that z D RZ; f .Y / D 0; f .Z/ D 1 and that a D RY C RZ is a commutative ideal of g. Then the centralizer g0 of a in g is an ideal of codimension 1. By the induction hypothesis, we may assume that the lemma holds for G0 D exp.g0 /. We indicate the objects regarding G0 by the index 0. Let p W g ! g0 be the restriction map and put 0 D p. /. For ˝ 2 g =G verifying ˝ \  ¤ ;, ˝ D p 1 .p.˝// and, by Lemma 8.1.4 in Chap. 8, p.˝/ is decomposed into one-parameter family !t , t 2 R, of G0 -orbits where !t D exp.tX /  !0 . To say more exactly, !t is, for instance Pt being the hyperplane f` 2 g0 I `.Y / D tg in g0 , given by !t D p.˝/ \ Pt . Let 0 the Lebesgue measure on the affine space 0 . (1) First assume h 6 g0 . In this case, is identified with 0 . Besides, choosing X in h, exp.tX /.!0 \0 / D !t \0 for arbitrary t 2 R. Set h1 D h\g0 CRY; f0 D p.f / 2 g0 and 1 D f`0 2 g0 I `0 jh1 D f0 jh1 g. Then, the Borel space  =G is identified with 1 =G0 . The induction hypothesis decomposes according to the action of G0 the Lebesgue measure 1 on 1 : Z 1 D ! d0 .!/; 1 =G0

where 0 is the base measure and ! is a measure on ! \ 1 . Hence, Z Z Z Š 0 D d0 .!/ exp.tX /  ! dt D ˝ d.˝/; 1 =G0

R

 =G

R

where .˝/ D 0 .!0 / D 0 .p.˝/ \ P0 / and ˝ D R exp.tX /  ! dt. (2) Secondly, assume h  g0 . In this case, using the Lebesgue measure dx on  g? 0  g , D 0  dx. According to [32], except for a null set ˝ \  (resp. ! \ 0 ) is a differential manifold and its dimension r (resp. r0 ) does not depend on ˝ 2 g =G (resp. ! 2 g0 =G0 ). There occur two possibilities: either r D r0 C 2 or r D r0 C 1. (a) Provided r D r0 C 2, let us argue as in the case (1). Let h1 D h C RY; ft 2  be such that ft .Y / D t and put t D f`0 2 g0 I `0 jh1 D ft jh1 g .t 2 R/. Then, Z 0 D

t dt; R

where t is the Lebesgue measure on the affine space t .

11.4 Bonnet’s Plancherel Formula

375

By the induction hypothesis, Z t D t =G0

!t dt .!/ .t 2 R/

with agreement on the notations. All the spaces t =G0 .t 2 R/ are identified with  =G, t are mutually equivalent. Retaking !t if necessary, we may think that all t coincide with the measure  on  =G. Consequently, Z D

˝ d.˝/;  =G

R  where ˝ D R !t t dt  dx with p.˝/ D tt 2R !t . (b) Provided r D r0 C1, the induction hypothesis supplies the decomposition of 0 : Z 0 D

! d0 .!/: 0 =G0

c0 introduces an equivalence relation in 0 =G0 and by it The action of G on G we get the projection q W 0 =G0 !  =G. As is immediately seen, there exists a constant k > 0 so that ]fq 1 .˝/g < k for almost all ˝ 2  =G. Here ].A/ denotes the cardinal of A. From this (cf. [13]), 0 is decomposed with respect to q: Z 0 D  =G

0 ˝ d.˝/;

0 where ˝ is the finite sum of ! regarding ! such that q.!/ D ˝. Finally,

Z D

˝ d.˝/  =G

0  dx. with ˝ D ˝



Remark 11.4.2. By a standard reasoning, the measure class of the measure  is the image of that of by the canonical projection from  to  =G and this decomposition of is essentially unique: if Z D  =G

0 ˝ d0 .˝/

376

11 Plancherel Formula

0 with other choices of the measures 0 and . ˝ /˝2 =G , there exists a measurable 0 0 function F with positive values so that  D F and ˝ D F ˝ for -almost all ˝. Besides, these ˝ are of course invariant relative to the action of H .

Consider a coadjoint orbit ˝ of G. Let ` 2 ˝. Let  denote the mapping G 3 g ! g  ` 2 ˝ and put G` D  1 . /. Let b 2 M.`; g/ and B D exp b. The set of all double classes H nG=B is a standard Borel space [23]. We write the canonical projection of G to H nG=B as p and a Borel section from H nG=B to G as . Put  D p.G` / and  D ./. For x 2 , let d hP be an invariant measure on H=.H \xBx 1 /, d a Lebesgue measure on the affine space bŒx D .`Cb? /\x 1   and Bx D  1 .bŒx/. We designate by Hx the image of the section constructed by means of a coexponential basis to h \ x  b in h. Then, each element g 2 G` is uniquely written as g D hxb; h 2 Hx ; x 2 ; b 2 Bx : f` D G` =G.`/. In the proof of Lemma 11.4.1, Let us transfer the measure ˝ on G we know the following by induction. ˝ decomposes with respect to a measure  on : Z ˝ D x d.x/ P .xP D p.x//; 

where the measure x on the fibre Hx  Bx is obtained by d hP  1 .d /. We realize  by means of b:  D indG B ` . We define a nuclear operator U from 1 H to H1 by the following formula: Z

Z

h; U i D

d.x/ P 

Z



H=.H \xBx 1 /

H=.H \xBx 1 /

.hx/f .h/d hP

.h0 x/f .h0 /d hP0

for arbitrary ; 2 H1 . With the help of a coexponential basis to b in g, H1 is identified with the space S.Rm /; m D dim.g=b/; of the Schwartz’s rapidly decreasing functions and hence H1 is identified with the space S 0 .Rm / of all tempered distributions. In this situation, the operator U is defined by the integral kernel   P K .hx; h0 x/ D f .h/dH=.H \xBx 1 / hP  f .h0 /dH=.H \xBx 1 / hP0 d.x/ and we know by induction (cf. the proof of Lemma 11.4.1) that this kernel belongs to S 0 .Rm  Rm /. We derive from this (cf. [76, p. 531]) that U is a nuclear operator and can compute Tr ../U / for  2 D.G/. Set

11.4 Bonnet’s Plancherel Formula

377

Z

f

H .g/ D

.gh/f .h/dh .g 2 G/: H

as before. In this situation we can compute Tr ../U /. O we have Theorem 11.4.3 (Character Formula [33]). For -almost all  2 G, with an invariant measure d bP Z

Z

Tr ../U / D

d.x/ P

H=.H \xBx 1 /



d hP

Z



B=.B\x 1 H x/

f H .hxbx 1 h1 /` .b/d bP

for all  2 D.G/. Further, up to a normalization of measures, the value of the right-hand side does not depend on the choice of either ` 2 ˝ or b 2 M.`; g/. Proof. The kernel of the operator ./ is given by the formula [66]: K .g; g 0 / D

Z

.gbg 01 /` .b/db; .g; g 0 / 2 G  G; B

and by [43] Tr ../U / D hK .g 0 ; g/; K i Z Z D d.x/ P xf .h/d hP x 1 H x=.x 1 H x\B/



Z

 Z D

x 1 H x=.x 1 H x\B/

Z

d.x/ P 

xf .h0 /d hP0

H=.H \xBx 1 /

d hP

Z

Z

.xhbh01 x 1 /` .b/db B

B=.B\x 1 H x/

f P H .hxbx 1 h1 /` .b/d b:

It is clear that the mapping D.G/ 3  7! Tr ../U / is, up to a normalization, independent of the choice of ` 2 ˝. Let us see that it does not depend on the choice of b 2 M.`; g/. Remark first we can assume that h contains the centre z of g. Next, if p D z \ kerf is not trivial, we can pass to the quotient algebra g=p to which the induction hypothesis is applied. Thus, we assume hereafter dim z D 1 and f jz ¤ 0. Let fX; Y; Zg be a Heisenberg triplet such that z D RZ; f .Z/ D 1; ŒX; Y  D Z and that a D RY ˚ RZ is a minimal non-central ideal of g. Then the centralizer g0 of a in g is an ideal of g and g D RX C g0 . Suppose that b 6 g0 . Knowing that

378

11 Plancherel Formula

b0 D b0 C a; b0 D b \ g0 , is an element of M.`; g/ contained in g0 , we show that this change from b to b0 does not influence the mapping D.G/ 3  7! Tr ../U /. Set G0 D exp.g0 /. First let h  g0 . We set E D fx 2 G` I h \ xb D h \ xb0 g; F D fx 2 G` I h \ xb ¤ h \ xb0 g: If x 2 E, the elements xexp.tY /; t 2 R, belong to different double classes H nG=B. In fact, hxb D xexp.tY / .h 2 H; b 2 B/ means b 2 B0 D exp.b0 / and xexp.tY /b 1 x 1 2 H \ xB 0 x 1 D H \ xBx 1 , where B 0 D exp.b0 /. This necessitates t D 0. Hence both sets H nE=B and H nE=B 0 are identified with R  .H nE0 =B0 /; E0 D E \ G0 ; the first with exp.RY /  .H nE0 =B/ and the second with .[t 2R bt /  .H nE0 =B0 /, bt 2 xBx 1 not belonging to xB0 x 1 except for only one value of t 2 R. This last affirmation is seen as follows. Because of x 2 E, Y 62 h C xb. For any t 2 R, we consider t 2 .h C xb/?  g satisfying t .Y / D t. The Pukanszky condition says that there exists bt satisfying bt x` D t , namely bt x 2 G` . Now let x 2 F , which means Y D V C xW with V 2 h; W 2 b0 . Then, taking X in b, H=.H \ xBx 1 / Š exp.RV /  .H=.H \ xB 0 x 1 //; B=.B \ x 1 H x/ Š exp.RX /  .B0 =.B0 \ x 1 H x//; B 0 =.B 0 \ x 1 H x/ Š B0 =.B0 \ x 1 H x/; while, if hxb 2 G` with h 2 H and b 2 B, then hY D hV C hxbb 1 W and f .V / D hxb`.hV / D `.b 1 x 1 Y  b 1 W / D `.b 1 x 1 Y  W /: Therefore b 2 B0 and H nF=B Š H nF=B0 . But for x 2 F , we have H xB 0 D H xexp.x 1 RV /B0 D H xB0 ; which allows us to identify H nF=B0 with H nF=B 0 . From these observations we immediately conclude that the formula of Tr ../U / does not change under the modification from b to b0 . Secondly, let h 6 g0 . X being taken in h, we set h D RX C h0 with h0 D h \ g0 and H0 D exp.h0 /. Let E D fx 2 G` I h \ xb ¤ h0 \ xbg; F D G` nE:

11.4 Bonnet’s Plancherel Formula

379

For x 2 E, we have h \ xb0 D h \ xb0 . Therefore, B 0 =.B 0 \ x 1 H x/ Š exp.RY /  .B0 =.B0 \ x 1 H x//; B=.B \ x 1 H x/ Š B0 =.B0 \ x 1 H x/; H=.H \ xBx 1 / Š H0 =.H0 \ xBx 1 /; H=.H \ xB 0 x 1 / Š exp.RX /  .H0 =.H0 \ xBx 1 //: On the other hand, for x 2 E0 D E \ G0 , the relation y 2 .H xB/ \ G0 leads to y 2 H xexp.RXQ /x 1 xB0 with XQ 2 bnb0 such that xXQ 2 h and y 2 H xB0 . This says that H nE=B Š H nE=B 0 . Hence on the set E it suffices for us to apply the inversion formula of Fourier in the expression of Tr ../U /. It remains for us to examine the set F , which is divided into F1 D fx 2 F I h \ xb ¤ h \ xb0 g and F2 D fx 2 F I h \ xb D h \ xb0 g: For x 2 F1 , Y is written as Y D x 1 V C W with certain V 2 h and W 2 b0 . Hence H=.H \ xBx 1 / Š exp.RV /  .H=.H \ xB 0 x 1 //; B=.B \ x 1 H x/ Š exp.RX /  .B0 =.B0 \ x 1 H x//; B 0 =.B 0 \ x 1 H x/ Š B0 =.B0 \ x 1 H x/: Moreover, if y 2 .H xB/ \ G` , then xb D hy with certain b 2 B and h 2 H . It follows that xb`.V / D hy`.V / D f .h1 V / D f .V / on one side and xb`.V / D b`.x 1 V / D b`.Y  W / on the other. It results that b 2 B0 and H nF1 =B D H nF1 =B 0 . We conclude in this way that also on F1 our claim comes from the inversion formula of Fourier. Finally, we divide F2 into two parts, F21 D fx 2 F2 I Y 2 h C xbg and F22 D fx 2 F2 I Y 62 h C xbg: For x 2 F21 , Y D V C xW with certain V 2 h n h0 and W 2 b n b0 . Here, B=.B \ x 1 H x/ Š exp.RW /  .B0 =.B0 \ x 1 H x//; B 0 =.B 0 \ x 1 H x/ Š exp.RY /  .B0 =.B0 \ x 1 H x//

380

11 Plancherel Formula

and H xB 0 D [t 2R H xexp.tx 1 V /exp.tW /B0 D H xB: It follows that Z Z d.x/ P Z

p.F21 /

D

Z

H=.H \xBx 1 /

Z

ds

R

d hP

Z

B0 =.B0 \x 1 H x/

d bP

H .hexp.sV /xexp.tW /x 1 xbexp.sx 1 V /x 1 h1 /` .exp.tW /b/dt Z

D

Z

d.x/ P

ds

p.F21 /

Z

H0 =.H0 \xBx 1 /

R

d hP

Z B0 =.B0 \x 1 H x/

d bP

H .hexp.sV /exp.t.Y  V //exp.sV /xbx 1 h1 /` .exp.tW /b/dt f

 R

Z

Z

Z

d.x/ P

D

ds

p.F21 /

Z

R

H0 =.H0 \xBx 1 /

d hP

Z B0 =.B0 \x 1 H x/

d bP

H .hexp.t.Y  V //xbx 1 h1 /` .exp.tW /b/e i st dt f

 Z

H0 =.H0 \xBx 1 /

R

H .hxbx 1 h1 /` .b/d bP f

B=.B\x 1 H x/

f

 Z

Z

Z

d.x/ P p.F21 /

d hP

R

Z d.x/ P

D p.F21 /

H0 =.H0 \xBx 1 /

d hP

Z B0 =.B0 \x 1 H x/

In the same way this last member is equal to Z Z Z d.x/ P d hP p.F21 /

H=.H \xB 0 x 1 /

B 0 =.B 0 \x 1 H x/

f P H .hxbx 1 h1 /` .b/d b:

P H .hxbx 1 h1 /` .b/d b; f

what finishes the computations on F21 . For x 2 F22 , B=.B \ x 1 H x/ Š exp.RX /  .B0 =.B0 \ x 1 H x// with X chosen in b n b0 and B 0 =.B 0 \ x 1 H x/ Š exp.RY /  .B0 =.B0 \ x 1 H x//: Besides, as Y 62 h C xb, we recall the reasoning already done once before. For any t 2 R, we consider t 2 .h C xb/? satisfying t .Y / D t. By the Pukanszky condition, there exists bt 2 xBx 1 outside of xB0 x 1 such that bt x` D t , namely bt x 2 G` . We get from this H xB D [t 2R H bt xB0 and hence H nF22 =B0 Š R  .H nF22 =B/:

11.4 Bonnet’s Plancherel Formula

381

On the other hand, as X 2 x 1 h C b0 , for any t 2 R we similarly define t 2 g such that t 2 .h C xb0 /? ; t .xX / D t. Hence there exists bt0 2 xB 0 x 1 not belonging to xB0 x 1 and satisfying bt0 x 2 G` . We now conclude that H xB 0 D [t 2R H bt xB0 and H nF22 =B0 Š R  .H nF22 =B 0 /: These observations find that there exists a subset F3 of F22 such that H nF22 =B Š R  .H nF3 =B0 / under the correspondence R 3 t 7! bt0 2 xB 0 x 1 ; bt0 62 xB0 x 1 and H nF22 =B 0 Š R  .H nF3 =B0 / under R 3 t 7! bt 2 xBx 1 ; bt 62 xB0 x 1 : Finally, a simple calculation confirms that the formula of Tr ../U / remains unchanged. Until now we proved that the formula in question does not change under the modification from b to b0 . In order to see its independence of the choice of polarizations it suffices therefore to confirm the claim when polarizations are contained in g0 . By the induction hypothesis we assume the result at the level of G0 . First, suppose h  g0 . We consider the slices ˝t ; t 2 R, of  \˝ by hyperplane f 2 g I .Y / D tg. On each ˝t , the induction hypothesis assures the invariance of the formula. Integrating them with respect to the variable t, we arrive at the awaited result. Secondly, suppose h 6 g0 . If we fix the element X in h and if we look at the independence of the formula at the level of G0 for 0 2 D.G0 / constructed from  2 D.G/ by Z 0 .g0 / D

R

.g0 exp.tX //dt .g0 2 G0 /

we notice immediately that there remains nothing to prove.



Now, the value of Tr ../U / being independent of the choice of polarizations, the next theorem can be proved just like the last part of the proof of the above theorem.

382

11 Plancherel Formula

Theorem 11.4.4 (Plancherel Formula [33]). Normalizing appropriately the measures, Z f H .e/ D Tr .˝ ./U˝ / d.˝/  =G

for all  2 D.G/. Here ˝ is the irreducible unitary representation of G corresponding to the coadjoint orbit ˝ 2 g =G.

Chapter 12

Commutativity Conjecture: Induction Case

12.1 Toward the Commutativity Conjecture Let G be a connected and simply connected nilpotent Lie group with Lie algebra g and H D exp h an analytic subgroup of G with Lie algebra h. Given a unitary character  of H , we construct the monomial representation  D indG H  and examine the algebra D .G=H / of the G-invariant differential operators on the line bundle GH C associated with . Our target is the commutativity conjecture due to Duflo [22] and Corwin and Greenleaf [17]. The latter proved one direction of implication: if  has finite multiplicities, then D .G=H / is commutative. Therefore, we are interested in the inverse direction of implication. p We take an f 2 g such that d D if jh ; i D 1 and set  D f C h? as before. Our first step is: Theorem 12.1.1 ([35]). Assume that  has infinite multiplicities. Let g0 be a Lie subalgebra of codimension 1 containing h and G0 D exp.g0 /. Suppose that 0 D 0 indG H  has finite multiplicities. If there exists W 2 U.g; / such that W 62 U.g0 / C U.g/a , then there is T 2 U.g0 ; 0 / such that ŒW; T  62 U.g/a . Proof. Let us employ the induction on dim g C dim.g=h/. Notice first that we may assume that h contains the centre z of g. In fact, provided z 6 h, we take 0 ¤ Z 2 z in the outside of h, and describe the representatives in U.g0 ; 0 / by means of an p adapted basis fZ; X1 ; X2 ; : : : ; Xp g to h in g0 . Here fXj gj D1 is an adapted basis to 0 h D h C RZ in g0 . At a generic ` 2  , we put ˛ D `.Z/. Then, if we denote by 0 ` the character of H 0 D exp.h0 / defined by d` D i `jh0 ,  ˛ D indG H ` has finite multiplicities. A subset S of an algebra A is said to be a system of rational generators if for arbitrary a 2 A there exist two polynomials P; Q of the elements of S such that p P a D Q. Using fXj gj D1 and a generic ˛, there is a system of rational generators of the algebra D ˛ .G0 =H / and its element in U.g0 ;  ˛ /=U.g0 /a ˛ is expressed, by means of an element © Springer Japan 2015 H. Fujiwara, J. Ludwig, Harmonic Analysis on Exponential Solvable Lie Groups, Springer Monographs in Mathematics, DOI 10.1007/978-4-431-55288-8__12

383

384

12 Commutativity Conjecture: Induction Case

T D

X

j

j

j

ck;J Z k X1 1 X2 2    Xpp

k;J

of U.g0 ; 0 /, as T .˛/ D

X

j

j

j

ck;J .i ˛/k X1 1 X2 2    Xpp

k;J

with the notation J D .j1 ; j2 ; : : : ; jp / for p-tuple of non-negative integers. We write g D g0 C RX and describe a representative of W by means of the basis ˛ fZ; X1 ; : : : ; Xp ; X g to construct W .˛/ 2 U.g; ˛ /. Here ˛ D indG G0  . From the above, if ŒW; T  2 U.g/a for arbitrary T 2 U.g0 ; 0 /, ŒW; T .˛/ D ŒW .˛/ C WQ .Z C i ˛/; T .˛/ 2 U.g/a˛ with a certain WQ 2 U.g/. Then, if we take ˛ so that W .˛/ 62 U.g0 / C U.g/a˛ , this contradicts the induction hypothesis applied to Rthe pair .h0 ; ` /. (All of these correspond to the direct integral decomposition  ' R  ˛ d˛.) Hereafter we assume z  h. Provided z \ kerf ¤ f0g, everything passes to the quotient space and the desired result is immediately obtained. Hence it remains for us to examine the case where dim z D 1 and f jz ¤ 0. Take as usual a Heisenberg triplet .XQ ; Y; Z/ such that z D RZ; ŒXQ ; Y  D Z; Y 2 g0 and that g D RXQ C k, where k denotes the centralizer of Y in g. Assume h  k. If Y 2 h,  0 D indK H  .K D exp k/ has infinite multiplicities. G0 \K 0 Since 0 D indH  for the given g0 has finite multiplicities, it is enough to apply the induction hypothesis to this  0 . If Y 62 h, T D Y 2 U.g0 ; 0 / fulfils the request of the theorem when g0 D k. So, let us assume g0 ¤ k. If W 62 U.k/ C U.g/a , we can always takeT D Y . Hence assume W 2 U.k/ C U.g/a . Since 0 D  G0 \K 0  has finite multiplicities,  0 must have infinite multiplicities. ind indG G0 \K H Otherwise W 2 U.k \ g0 / C U.g/a , which contradicts the assumption. Now it is enough to apply the induction hypothesis to k. Finally, assume h 6 k. We consider h0 D h \ k; H0 D exp.h0 / and f0 D f jk 2 k . Then, we know that 1 D indK H0 f0 has infinite multiplicities. We set m D g0 \ k, M D exp m and notice that 2 D indM H0 f0 has finite multiplicities. Since W is represented by an element of U.k/, the induction hypothesis applied to k assures that there exists T 2 U.m; 2 / such that ŒW; T  62 U.k/a1 . Here, we may think that T is an element of a system of rational generators of U.m; 2 / studied in Corwin and Greenleaf [17], while, utilizing a weak Malcev basis adapted to h in g0 , it is easily checked that fY; 2 ; : : : ; q g form a system of rational generators for q U.m; 2 /. Here f j gj D2 constitute rational generators for U.g0 ; 0 /. As ŒW; Y  D 0, we recognize the existence of such an element T in U.g0 ; 0 /.  Let us use here again the notations introduced at the beginning of Chap. 9. However, fXr g1rq denotes a coexponential basis to h in g adapted to the sequence

12.1 Toward the Commutativity Conjecture

385

of Lie subalgebras (9.2.3), i.e. Xr 2 kr nkr1 .1  r  q/. In what follows, if g ¤ h, g0 always denotes an ideal of codimension 1 in g which contains h. Moreover, we choose the composition series of ideals (9.1.1) in such a way that gn1 D g0 . Likewise, when dim h  1, h0 always denotes a Lie subalgebra of codimension 1 in h and the composition series (9.2.4) satisfies hd 1 D h0 . Then, when g0 ; h0 both exist, dim g  2 and g © g0  h © h0 . Further, we make the following convention. Let N D f0; 1; 2; : : : g. We consider the elements fXr g1rq , fYs g1sd introduced above and just before Lemma 9.2.14. For each q-tuple J D .j1 ; j2 ; : : : ; jq / 2 Nq , d -tuple K D .k1 ; k2 ; : : : ; kd / 2 Nd and .d  1/-tuple L D .`1 ; `2 ; : : : ; `d 1 / 2 Nd 1 , we denote by X J ; Y K and j j j Y 0L respectively the elements X J D Xq q   X2 2 X1 1 ; Y K D Ydkd   Y2k2 Y1k1 and d 1   Y2`2 Y1`1 of U.g/. As in the common usage, jJ j (resp. jKj; jLj) Y 0L D Yd`1 signifies the sum j1 C j2 C    C jq (resp. k1 C k2 C    C kd ; `1 C `2 C    C `d 1 ). Similarly, we consider the elements YOs D Ys C if .Ys /; 1  s  d of U.g/. Hence L d 1    YO2`2 YO1`1 . YO K D YOdkd    YO2k2 YO1k1 ; and YO 0 D YOd`1 By the Poincaré–Birkhoff–Witt theorem, the families fX J YO K I .J; K/ 2 Nq  Nd g; fX J YO K I .J; K/ 2 Nq  Nd ; jKj > 0g form respectively bases of U.g/ and U.g/a . The elements fX J I J 2 Nq g make a basis of a linear complement S of U.g/a in U.g/. Assuming h ¤ f0g, we set O K D YO k YO 0 L . Thus, the families H 0 D exp.h0 / and  0 D indG H 0 f . Put Y d L

fX J YOdk YO 0 I .J; k; L/ 2 Nq  N  Nd 1 ; jLj > 0g; fX J YOdk I .J; k/ 2 Nq  Ng constitute bases respectively of U.g/a 0 and a linear complement of U.g/a 0 in U.g/. We mention some properties of U.g; / obtained in [5]. Lemma 12.1.2. (i) gid 1 S  S ˚ U.g/a 0 . (ii) Œh; S   S ˚ U.g/a 0 . Proof. (i) For 0  r  q and k 2 N, we denote by Sr;k the subspace of S j j generated by the elements X J D Xr r   X1 1 ; J 2 Nr ; satisfying jJ j  k. In particular, S0;k D Sr;0 D C. In order to confirm that many monomials are well ordered, the index r is useful. Now, it suffices to show the following by induction on k. . / Y 0 X J 2 Sr;kC1 ˚ U.g/a 0 ; 8 Y 0 2 h0 ; 8 X J 2 Sr;k ; .

/ Xs X J 2 Smax.s;r/;kC1 ˚ U.g/a 0 for any s such that js  id  1 and 8 X J 2 Sr;k . This is clear when k D 0. Assume k > 0 and that the result is correct until the step k  1. If we take Xs such that s  r, then Xs X J 2 Ss;kC1 and the result is clear. If we take T D Y 0 2 h0 or T 2 gid 1 such that T D Xs with s < r,

386

12 Commutativity Conjecture: Induction Case 0

j 1

0

then we can write X J as X J D Xr X J using X J D Xr r use of the equality 0

j

  X1 1 and make

0

TX J D ŒT; Xr X J C Xr TX J : 0

ŒT; Xr  2 gjr 1 \ gid 1 and X J 2 Sr;k1 . Hence, 0

ŒT; Xr X J 2 Sr;k ˚ U.g/a 0 0

0

by induction. Similarly, TX J 2 Sr;k ˚ U.g/a 0 . Thus Xr TX J 2 Sr;kC1 ˚ U.g/a 0 . (ii) We show by induction on k that ŒY; X J  2 Sr;k ˚ U.g/a 0 for 8 Y 2 h; 8 X J 2 Sr;k . This is clear when k D 0. Assume that the claim is correct at the step k  1; k ¤ 0. First of all 0

0

ŒY; X J  D ŒY; Xr X J C Xr ŒY; X J : 0

Since ŒY; Xr  2 gjr 1 \ gid 1 and X J 2 Sr;k1 , we know from the claim (i) 0 0 ŒY; Xr X J 2 Sr;k ˚ U.g/a 0 . Finally, ŒY; X J  2 Sr;k1 ˚ U.g/a 0 by induction 0 and Xr ŒY; X J  2 Sr;k ˚ U.g/a 0 .  P Lemma 12.1.3. Let W be an element of U.g/ and assume that W kk 0 Ak YOdk modulo U.g/a 0 , where Ak are elements of S . Assume Y 2 h. By Lemma 12.1.2, we obtain elements .Bk /kk 0 of S so that ŒY; Ak  Bk modulo U.g/a 0 . Then, modulo U.g/a 0 , X X X Ak YOdk  ŒY; Ak YOdk Bk YOdk : ŒY; W  ŒY; kk 0

kk 0

kk 0

Proof. The first and the third equalities are obtained from the inclusion h  U.g;  0 / and the second from the relations ŒY; Ak YOdk  D ŒY; Ak YOdk CAk ŒY; YOdk  and ŒY; YOd  2 h0 \ kerf  a 0 .  Proposition 12.1.4. (i) Œh; U.g; / \ S   U.g/a 0 . Especially, U.g; / \ S  U.g;  0 /: (ii) We get the decomposition U.g; / D .U.g; / \ S /˚U.g/a and the restriction to U.g; /\S of the projection from U.g; / onto D .G=H / is a bijection. Thus, each element of D .G=H / Š U.g; /=U.g/a has a unique representative in S . This representative belongs to U.g;  0 /.

12.1 Toward the Commutativity Conjecture

387

Proof. (i) By (ii) of Lemma 12.1.2, Œh; U.g; / \ S   .S ˚ U.g/a 0 / \ U.g/a  U.g/a 0 : (ii) The result follows from the decomposition U.g/ D S ˚ U.g/a and the inclusion relation U.g/a  U.g; /.  Proposition 12.1.5. Let .Ak /0kk 0 be a family of elements of S , then X

Ak YOdk 2 U.g;  0 / ” Ak 2 U.g;  0 /; 8 k  k 0 :

kk 0

Proof. Let Y 0 2 h0 . We define for all k the element Bk of S so that ŒY 0 ; Ak  D Bk modulo U.g/a 0 . By Lemma 12.1.3, 

Y 0;

X

 Ak YOdk 2 U.g/a 0 ” Bk D 0; 8 k  k 0 :

kk 0 0

0



Proposition 12.1.6. Let g; g ; h and h be as before. (i) The following equivalence holds: U.g;  0 / 6 U.g0 / C U.g/a 0 ” U.g;  0 / 6 U.g0 / C U.g/a : (ii) Besides, if we assume U.g;  0 /  U.g; /, U.g;  0 / 6 U.g0 / C U.g/a 0 ” U.g; / 6 U.g0 / C U.g/a :  0/ n Proof. (i) ( is clear. We show ). Let W 0 be an element P of U.g; 0 0 0 k .U.g / C U.g/a 0 /. W is written modulo U.g/a 0 as W kk 0 Ak YOd . Here Ak are elements of S . Because W 0 does not belong to U.g0 / C U.g/a 0 , one of the Ak , say Ak0 , does not belong to U.g0 /. In other words, Xq appears in Ak0 . Then, by Proposition 12.1.5, Ak0 2 U.g;  0 /. Since .U.g0 / C U.g/a / \ S D U.g0 / \ S and Ak0 2 S n U.g0 /, Ak0 does not belong to U.g0 / C U.g/a . Thus we get ). (ii) ) is directly obtained from (i) and our assumption. We show (. From the decomposition   U.g0 / C U.g/a D U.g0 / \ S ˚ U.g/a ; U.g; / D .U.g; / \ S / ˚ U.g/a and our assumption, U.g; / \ S 6 U.g0 / \ S D .U.g0 / C U.g/a / \ S . The result follows from the inclusion relation U.g; / \ S  U.g;  0 / of Proposition 12.1.4. 

388

12 Commutativity Conjecture: Induction Case

Proposition 12.1.7. Assume U.g;  0 / \ S  U.g; / \ S . Then, Œh; U.g;  0 /  U.g/a 0 : Proof. We write an element W of U.g;  0 / as X Ak YOdk mod U.g/a 0 ; W kk 0

where Ak are elements of S . By Proposition 12.1.5, Ak 2 U.g;  0 / \ S and Ak 2 U.g; / \ S by assumption. Finally, from the first assertion of Proposition 12.1.4,  X Œh; Ak YOdk C Ak Œh; YOdk   U.g/a 0 : Œh; W   kk 0

 Proposition 12.1.8. Let g; g0 and h (also h0 in the third assertion) be as before and suppose that Xq exists. Pm k 0 (i) Let W D kD0 Xq Ak 2 U.g; /, where m  1 and Ak 2 U.g /. Then, Am 2 U.g; / and mXq Am C Am1 2 U.g; /. (ii) Let W D Xq U C V with U; V 2 U.g0 /. Then, W 62 U.g0 / C U.g/a ” U 62 U.g0 /a : (iii) Assume U.g; / 6 U.g0 / C U.g/a .resp. U.g;  0 / 6 U.g0 / C U.g/a /. Then, there exists an element   W D Xq U C V 2 U.g; / n U.g0 / C U.g/a   .resp. 2 U.g;  0 / n U.g0 / C U.g/a / with U 2 .U.g; / \ U.g0 //nU.g0 /a .resp. U 2 .U.g;  0 / \ U.g0 //nU.g0 /a / and V 2 U.g0 /. Proof.

(i) Immediately from the Poincaré–Birkhoff–Witt theorem, U.g/ D ˚j Xqj U.g0 /; U.g/a D ˚j Xqj U.g0 /a :

(12.1.1)

0 Set im2 D ˚jm2 D0 Xq U.g /. Then for arbitrary Y 2 h, j

ŒW; Y  Xqm ŒAm ; Y  C

m X

Xqj 1 ŒXq ; Y Xqmj Am

j D1

C Xqm1 ŒAm1 ; Y  mod im2 Xqm ŒAm ; Y  C Xqm1 .mŒXq ; Y Am C ŒAm1 ; Y / mod im2 belongs to U.g/a . Therefore, Am 2 U.g; / and mXq Am C Am1 2 U.g; /.

12.1 Toward the Commutativity Conjecture

389

(ii) Making use of expression (12.1.1), W 2 U.g0 / C U.g/a when and only when U 2 U.g0 /a . Because,     W 2 Xq U.g0 / ˚ U.g0 / \ ˚j Xqj U.g0 /a ˚ U.g0 / D Xq U.g0 /a ˚ U.g0 /: (iii) With Ak belonging to U.g0 /, we put W0 D

m X

  Xqk Ak 2 U.g; / n U.g0 / C U.g/a

kD0



0

resp. W D

m X

  Xqk Ak 2 U.g;  0 / n U.g0 / C U.g/a :

kD0

From assertion (i), Am 2 U.g; / and W D mXq Am C Am1 2 U.g; / (resp. Am 2 U.g;  0 / and W D mXq Am C Am1 2 U.g;  0 /). Without loss of generality, we may assume Am 62 U.g0 /a and, taking (ii) into account, W is provided with the required properties.  Theorem 12.1.9. With the notation and assumption of Proposition 12.1.8, assume U.g;  0 / 6 U.g0 / C U.g/a 0 and U.g;  0 / \ U.g0 / 6 U.g; / \ U.g0 /: Then U.g; / 6 U.g0 / C U.g/a . Proof. We first show that there exists an element W D Xq U C V 2 U.g;  0 / \ S , where U; V 2 U.g0 / \ S , so that: (a) W 62 U.g0 / C U.g/a , or equivalently for U ¤ 0, (b) .adYd /W 2 U.g0 / C U.g/a . This fact will be utilized later in various situations in order to construct an element of U.g; / not belonging to U.g0 /CU.g/a . From assertion (i) of Proposition 12.1.6, we know that U.g;  0 / 6 U.g0 / C U.g/a . Thus, from the third assertion of Proposition 12.1.8, there exists W 00 D Xq U 00 C V 00 2 U.g;  0 / n .U.g0 / C U.g/a / with U 00 2 .U.g;  0 / \ U.g0 // n U.g0 /a and V 00 2 U.g0 /. Next let m0 2 N be the 0 maximal integer such that W 0 D .ad Yd /m .W 00 / 62 U.g0 / C U.g/a . Using the fact that U.g/ D S ˚ U.g/a , let W be the unique element of S such that W W 0 modulo U.g/a . Then W satisfies the conditions (a) and (b). Now let us introduce the notation used only in this proof. For A; B 2 U.g/, set fA; Bg D AB C BA. If we write As D .ad Yd /s A with s 2 N, then A0 D A. Next, for r 2 N n f0g, we define Tr .A/ D fA0 ; A2r g  fA1 ; A2r1 g C    C .1/r1 fAr1 ; ArC1 g C .1/r A2r :

390

12 Commutativity Conjecture: Induction Case

Since Yd belongs to U.g;  0 /, if A belongs to U.g;  0 /, so does As and Tr .A/. Furthermore, if r is large enough to get A2rC1 2 U.g/a 0 , then Tr .A/ 2 U.g; /. In order to see this, it suffices to show that this new condition implies .ad Yd /Tr .A/ 2 U.g/a . In fact, .ad Yd /Tr .A/ D A2rC1 A C AA2rC1 2 U.g/a U.g;  0 / C U.g/a 0 D U.g/a 0 : Let m be the minimal integer such that Wm D .ad Yd /m .W / 2 U.g/a . By means of the induction on s and assertion (ii) of Lemma 12.1.2, we see that Ws belongs to S ˚ U.g/a 0 and hence Wm 2 U.g/a 0 . We separate the cases by the value of m. – When m D 1, from W 2 U.g; / n .U.g0 / C U.g/a / the result is obvious. – Let r  1 and m D 2r C 1. The above remark says that Tr .W / D fW0 ; W2q g  fW1 ; W2q1 g C    C .1/r1 fWr1 ; WrC1 g C .1/r Wr2 belongs to U.g; /. We want to show Tr .W / 62 U.g0 / C U.g/a . Modulo U.g0 /, Tr .W / 2W W2r 2Xq U W2r . Neither U nor W2r belongs to U.g0 /a and the quotient ring .U.g; / \ U.g0 // =U.g0 /a has no non-trivial zero divisor. So, the result follows. – Let r > 1 and m D 2r. For arbitrary c 2 C, we know .adYd /2rC1 .W .W2r2 C cW2r1 // 2 U.g/a 0 : Q If we mentioned before gives  put W .c/ D W .W2r2 C cW2r1 /, the remark  Tr WQ .c/ 2 U.g; /. We want to show Tr WQ .c/ 62 U.g0 / C U.g/a for some   c 2 C. For a while we regard WQ .c/ and T WQ .c/ as elements of the polynomial ring U.g/Œc of c with coefficients in U.g/. For any r  1, WQ .c/0 D W .W2r2 C cW2r1 / WQ .c/1 W1 .W2r2 C cW2r1 / C W W2r1 WQ .c/2 W2 .W2r2 C cW2r1 / C 2W1 W2r1 2 U.g0 /Œc :: :

:: :

:: :

:: :

WQ .c/2r1 W2r1 .W2r2 C cW2r1 / C .2r  1/W2r2 W2r1 2 U.g0 /Œc 2 2 U.g0 / WQ .c/2r 2rW2r1

modulo U.g/a Œc. Now, let us show   Tr WQ .c/ 2 cXq U.g0 / ˚ Xq U.g0 / ˚ U.g0 /Œc mod U.g/a Œc

12.1 Toward the Commutativity Conjecture

391

and draw from it the component relative to cXq U.g0 /. The following relations hold for r > 1 but not for r D 1:   Tr WQ .c/ fWQ .c/0 ; WQ .c/2r g  fWQ .c/1 ; WQ .c/2r1 g   mod U.g0 /Œc C U.g/a Œc

  2WQ .c/0 WQ .c/2r  2WQ .c/1 WQ .c/2r1 mod U.g0 /Œc C U.g/a Œc   3 mod Xq U.g0 / ˚ U.g0 /Œc C U.g/a Œc 2c.2r  1/W W2r1   3 mod Xq U.g0 / ˚ U.g0 /Œc C U.g/a Œc : 2c.2r  1/Xq U W2r1

  Now, if we regard Tr WQ .c/ as an element of U.g/, there exists from what we mentioned above UQ 2 U.g0 / so that       3 C UQ mod U.g0 / C U.g/a : Tr WQ .c/ Xq 2c.2r  1/U W2r1 3 Here, U W2r1 does not belong to U.g0 /a . Because, modulo U.g/a Œc neither U nor W2r1 belongs there and the ring .U.g; / \ U.g0 // =U.g0 /a has no non-trivial zero divisor. Paying attention to Xq U.g0 / \ .U.g0 / C U.g/a / D Xq U.g0 /a , for all c satisfying 3 2c.2r  1/U W2r1 C UQ 62 U.g0 /a ;

  Tr WQ .c/ 62 U.g0 / C U.g/a . Remark that this method produces no result for m D 2. Because, in this case   T1 WQ .c/ 2 Xq U.g0 / ˚ U.g0 /Œc C U.g/a Œc: – Let m D 2. Let us use for the first time the assumption U.g;  0 / \ U.g0 / 6 U.g; /\U.g0 /. Immediately from this, there exists T 2 .U.g;  0 / \ U.g0 / \ S /n .U.g; / \ U.g0 // so that .ad Yd /T 2 .U.g; / \ U.g0 // n U.g/a . Then, W T 2 U.g;  0 /; .ad Yd /3 .W T / 2 U.g/a 0 and T1 .W T / D f.W T /0 ; .W T /2 g  .W T /21 2 U.g; /: Therefore we want to show T1 .W T / 62 U.g0 / C U.g/a . To begin with, T1 .W T / fW T; W1 T1 C T1 W1 g  .W1 T C W T1 /2 mod U.g/a   W 2 T12 Xq2 .U T1 /2 mod Xq U.g0 / ˚ U.g0 / C U.g/a :   Remarking that Xq2 U.g0 / \ Xq U.g0 / ˚ U.g0 / C U.g/a D Xq2 U.g0 /a , the desired result comes from the fact that neither U nor T1 belongs to U.g0 /a . 

392

12 Commutativity Conjecture: Induction Case

12.2 Corwin–Greenleaf Functions Theorem 12.1.1 and Theorem 12.1.9 offer nice tools in many situations to attack the commutativity conjecture, nevertheless there is still a case out of their range. In terms of Theorem 12.1.9, it is the case where U.g;  0 / \ U.g0 /  U.g; / \ U.g0 /: In order to treat this case, we need still to dig into the Corwin–Greenleaf e-central elements introduced in Chap. 9 (cf. [36]). An element A 2 U.g/ is said to be  -central if ` .A/ is a scalar operator at any ` in a non-empty Zariski open set O of  . Namely, there is a function 'A on G  O so that ` .A/ D 'A .`/Id; 8 ` 2 G  O:

(12.2.1)

Theorem 9.2.1 of Chap. 9 is extended to this situation: ` .A/ are scalars at all ` 2  and the function 'A is an H -invariant polynomial function. We denote by U.g;  / the algebra of all  -central elements in U.g/ and by Z.g;  / D f'A I A 2 U.g;  /g the algebra of all functions satisfying (12.2.1). We designate by ˛ the mapping U.g;  / 3 A 7! 'A 2 Z.g;  / and by CD .G=H / the centre of the algebra D .G=H /; we set UC .g; / D R1 .CD .G=H // D fA 2 U.g; /I ŒA; U.g; /  U.g/a g: If we write as $ the restriction of the mapping R to UC .g; /, $ W UC .g; / 3 A 7! R.A/ 2 CD .G=H /: We indicate by L (resp. R as before) the left (resp. right) action of U.g/: for X 2 g and 2 C 1 .G/, .L.X / / .g/ D .R.X / / .g/ D

d .exp.tX /g/jt D0 I dt d .gexp.tX //jt D0 : dt

12.2 Corwin–Greenleaf Functions

393

The principal anti-automorphism ˙ sends U.g;  / to UC .g; / and the mappings ˛; $ are surjective. Besides, it turns out that there is an injection ı W Z.g;  / ! CD .G=H / so that ı.'A / D L.A/ D R .˙.A//. Let us recall the sequence of ideals f0g D g0  g1      gn1  gn D g; dim.gk / D k .0  k  n/:

(9:1:1)

Put gn1 D g0 and `0 D `jg0 for ` 2 g . For a Lie subalgebra m of g, we denote respectively by m.`/ and m.`0 / Lie subalgebras m \ g.`/ and m \ .g0 /` . In Chap. 9, we took the uniquely determined multi-indices e 2 E and introduced the index sets T .e/; T .eH / and U.e/ D T .e/nT .eH /. Besides, as T .e/ D fm1 < m2 <    < mt g; making use of a Malcev basis fXj g1j n associated with the sequence of ideals (9.1.1), we introduced the Corwin–Greenleaf e-central element k D Aj D Pj Xj C Qj ; Pj ; Qj 2 U.gj 1 / for j D mk 2 T .e/. There, Pj itself was e-central and we wrote simply as 'j .`/ the polynomial function 'Aj .`/ given by the above formula (12.2.1). We agree to call these 'j .`/ Corwin–Greenleaf functions. Furthermore, for 0  i  n, we set Ti .e/ D T .e/ \ f0; : : : ; i g; Ti .eH / D T .eH / \ f0; : : : ; i g and Ui .e/ D U.e/ \ f1; : : : ; i g. Finally, let us define the algebra Zi .g; / D f'A I A 2 U.gi / \ U.g;  /g; 0  i  n: Thus, f0g D Z0 .g; /  Z1 .g; /      Zn .g; / D Z.g;  /: The main results of [36] are as follows: ( ) For all integers 0  k  n, the family f'j I j 2 Tk .e/g makes a system of rational generators of Zk .g; /. (

) For all integers 0  k  n, the family f'j I j 2 Uk .e/g makes a transcendental basis of the algebra Zk .g; /. Toward these results, let us follow [36] for a while. We indicate the cardinal of a set A by ].A/. When a property holds for all elements of some non-empty Zariski open set of a vector space V , we shall say that this property holds generally on V or for general elements of V . Let us first show ( ).

394

12 Commutativity Conjecture: Induction Case

System of Rational Generators of Zk .g; / Theorem 12.2.1. Let 0  k  n. (1) For all integers 0  k  n, the family f'j I j 2 Tk .e/g makes a system of rational generators of Zk .g; /. (2) If k 2 S.e/, then Zk .g; / D Zk1 .g; /. #

#

We define the elements fXj g1j n in U.g/ by Xj D Xj if j 2 S.e/ and by #

Xj D Aj if j 2 T .e/. Here Aj D Pj Xj C Qj is the Corwin–Greenleaf e-central element. Lemma 12.2.2. Let W 2 U.gk /; 1  k  n;. Then there exist an element .˛j /j 2Tk .e/ of N].Tk .e// and a mapping .ˇ1 ; : : : ; ˇk / 7! ˇ1 :::ˇk from Nk to C, whose values are 0 except at finite points, so that we have 0 ` @

Y

0

1 ˛ Pj j W

X

A D ` @

j 2Tk .e/

1  ˇk  ˇ1 # # ˇ1 :::ˇk Xk    X1 A

.ˇ1 ;:::;ˇk /2Nk

for arbitrary ` 2  . Proof. Our proof is a little technical but its idea is simple. Let us show by induction on k 0 that the following holds for 1  k 0  k: there exist an element .˛j /j 2.Tk .e/nTkk0 .e// of N].Tk .e//].Tkk0 .e// and a mapping .ˇkk 0 C1 ; : : : ; ˇk / 7! Vˇkk0 C1 :::ˇk 0

from Nk to U.gkk 0 /, whose values are 0 except at finite points, so that we have `

Y

˛

Pj j W

j 2.Tk .e/nTkk0 .e//



X .ˇkk0 C1 ;:::;ˇk /2Nk

0

 ˇk ˇkk0 C1  # # Xk    Xkk 0 C1 Vˇkk0 C1 :::ˇk

! D0 (12.2.2)

on  . Notice that, when k 0 D k, this property is nothing but the formula of the lemma since U.gkk 0 / D U.g0 / D C. Each element of U.gkk 0 C1 / is written as a linear ˇ combination of elements having the form Xkk 0 C1 B; B 2 U.gkk 0 /. Hence, when 0 k D 1, we get a relation of the form

12.2 Corwin–Greenleaf Functions

0 ` @W 

395

X

1 Xk B A D 0; B 2 U.gk1 /:



2N

Even when k 0 > 1, the following analogous relation is obtained: Y

`

˛

Pj j W

j 2.Tk .e/nTkk0 .e//

  k  kk0 C2

 # # kk 0 C1 Xk    Xkk 0 C2 Xkk 0 C1 B

X



D. kk0 C1 ;:::; k

! D 0;

0 /2Nk

(12.2.3) 0

where B . 2 Nk / is an element of U.gkk 0 /. # If k  k 0 C 1 2 S.e/, then Xkk 0 C1 D Xkk 0 C1 and Tkk 0 .e/ D Tkk 0 C1 .e/. Thus, if we put

k D ˇk ; : : : ; kk 0 C1 D ˇkk 0 C1 ; B D Vˇkk0 C1 :::ˇk in relation (12.2.3), formula (12.2.2) is directly obtained. 0 If k  k 0 C 1 2 T .e/, we put ˛ D maxf kk 0 C1 I 2 Nk ; B ¤ 0g. Operating ` .Pkk 0 C1 /˛ on both sides of Eq. (12.2.3), it is enough to see that the product ˇ ` .Pkk 0 C1 /˛ Xkk 0 C1 .ˇ  ˛/ is written as a linear combination of elements a  # having the form Xkk 0 C1 R .a 2 N; R 2 U.gkk 0 //. To simplify the notation, we put Pkk 0 C1 D P; Xkk 0 C1 D X; Qkk 0 C1 D Q and X # D PX C Q. By ˛  ˇ,   ` .P ˛ X ˇ / D ` .P ˇ X ˇ P ˛ˇ / D ` .X #  Q/ˇ P ˛ˇ and, since P; Q belong to U.gkk 0 /, we get the desired result.



When we take the product of the family Q of elements .aj /jQ 2T , not necessarily commutative, indexed by an ordered set T , j 2T "aj .resp: j 2T #aj / denotes the product where j increases (resp. decreases). For S D .Sj /j 2Sk .e/ 2 N].Sk .e// , we Q S put X S D j 2Sk .e/ #Xj j . Then, the previous lemma is rewritten in the following form. Lemma 12.2.3. Let W 2 U.gk /; 1  k  n. Then, there exist an  element ˛ D .˛j /j 2Tk .e/ of N].Tk .e// and a mapping S 7! aS D aS .Aj /j 2Tk .e/ , whose values are 0 except at finite points, from N].Sk .e// to the subalgebra of U.gk / generated by fAj gj 2Tk .e/ so that

396

12 Commutativity Conjecture: Induction Case

0 ` .P ˛ W / D ` @

1

X

Y

aS X S A ; P ˛ D

˛

Pj j ;

j 2Tk .e/

S 2N].Sk .e//

holds for ` 2  . In this lemma, if we replace W by an  -central element  of U.gk /: Proposition 12.2.4. Let 1  k  n and  an  -central element of U.gk /. Then, ].Tk .e// there exist and a polynomial expression  ˛ D .˛j /j˛2Tk .e/ of N  an element a D a .Aj /j 2Tk .e/ so that ` .P / D ` .a/ holds for all ` 2  . Here, P ˛ D Q ˛j j 2Tk .e/ Pj . Proof. Using the notations of the last lemma, we have 0 1 X ` @a0  P ˛  C aS X S A D 0: S 2N].Sk .e// nf0g

Here a0 ; aS are  -central elements of U.gk / and, if we supply the functions ˇ0 .`/; ˇS .`/ on  by ` .a0  P ˛ / D ˇ0 .`/Id; ` .aS / D ˇS .`/Id; then

X

ˇS .`/` .X S / D 0:

S 2N].Sk .e//

According to Pedersen [58], the restriction of the representation ` at a general point ` of  to the subspace of U.g/ spanned by X S ’s is faithful, namely injective. Since X S ’s are mutually linearly independent by the Poincaré–Birkhoff– Witt theorem mentioned in the first chapter, ˇS .`/ D 0 for arbitrary S 2 N].Sk .e// . In particular, taking S D 0; a D a0 , we get the proposition.  Let us begin a proof of Theorem 12.2.1. We first show assertion (2). So, assume k 2 S.e/ and take an  -central element  in U.gk /. Then, let us see that there exists an  -central element 0 in U.gk1 / so that ` ./ D ` .0 / on  . Again by the Poincaré–Birkhoff–Witt theorem, we write  D 0 C Xk 1 using 0 2 U.gk1 / and 1 2 U.gk /. If we replace W by  in Lemma 12.2.3, we have 0 ` .P ˛ / D ` @

X

1 aS X S A

S 2N].Sk .e//

at a general point ` of  . On the other hand, ` .P ˛ / D ` .P ˛ 0 / C ` .P ˛ Xk 1 /:

12.2 Corwin–Greenleaf Functions

397

Hence, using again the above-mentioned Pedersen’s result, 0 ` .P ˛ Xk 1 / D ` @

1

X

aS X S A D 0

S 2N].Sk .e// ;Sk >0

and ` .Xk 1 / D 0 on  . Therefore ` ./ D ` .0 /. Let us next show (1) by induction on k. For this aim, an element  of Zk .g; / being given, we show that  is written as a fractional expression of f'j gj 2Tk .e/ . When k D 0, since g0 D f0g; U.g0 / D Z0 .g; / D C,  is a constant and the assertion is clear. When k > 0, we take an  -central element  2 U.gk / so that ` ./ D .`/Id  on  . Choosing a D a .Aj /j 2Tk .e/ and ˛ 2 N].Tk .e// as in the previous proposition, we have ` .P ˛ / D ` .a/. Hence, if we put     ˚0 D a .'j /j 2Tk .e/ 2 C .'j /j 2Tk .e/ ; then ` .a/ D ˚0 .`/Id on  . Let 0 2 Zk1 .g; / be such that ` .P ˛ / D 0 .`/Id on  , then 0  D ˚0 . Since Pj ; j 2 Tk .e/; is a polynomial of Aj ; j 2 Tk1 .e/ as we saw at the beginning of Chap. 9, the desired result follows.  Transcendental Basis of Zk .g; / Now we pass to (

). Theorem 12.2.5. Let 0  k  n. (1) The family f'j I j 2 Uk .e/g makes a transcendental basis of the algebra Zk .g; /. (2) At a general point ` of  , the transcendental degree of the algebra Zk .g; / is equal to dim .gk .`/=hk .`//. We need some preparations to prove this theorem. To simplify the notation, we simply write  instead of  . Related to the sequence of ideals (9.1.1), let pj W g ! gj and pj0 W .g0 / D gn1 ! gj be the restriction maps, and put j D pj . /. Let 0  k  j  n  1. We write Z.gk ; j / to indicate the algebra of the functions  on j given by ` ./ D .`/Id; ` 2 j with j -central element  of U.gk /. Assume in general that a nilpotent Lie group K D exp k acts on a finitedimensional real vector space V . Let L D .kj /j 2J be a sequence of ideals of k beginning with k0 D f0g, Kj D exp.kj / and P an affine subspace of V . In this situation,      O.P; V; K; L/ D ` 2 P I dim Kj ` D max dim Kj  ; 8 j 2 J 2P

398

12 Commutativity Conjecture: Induction Case

is a Zariski open set of P . We put S.e.P; V; K; L// D fj 2 J n f0gI kj .`/  kj 1 ; 8 ` 2 O.P; V; K; L/g; T .e.P; V; K; L// D fj 2 J n f0gI kj D kj 1 C kj .`/; 8 ` 2 O.P; V; K; L/g: By Z.gk ; k /G we denote the subring of Z.gk ; k / composed of the elements extensible to G-invariant functions on Gk . Proposition 12.2.6. Let 0  k  n; ` 2 G . When we put `k D `jgk , there exists an injection k from Zk .g; / to Z.gk ; k /G , which is characterized by k ./.`k / D .`/ for  2 Zk .g; /. Proof. Let us first show the next lemma. Lemma 12.2.7. Let b be the Vergne polarization at ` of g constructed from the composition series (9.1.1). Hence, bk D b \ gk is a polarization at `k of gk . Put Gk B D exp b; Bk D exp.bk / and ` D indG B ` ; `k D indBk `k . Then, the restriction 1 1 map ˚ ! ˚jGk is a surjection from H` to H` . k

Proof. With respect to the composition series (9.1.1), let S.`/ and T .`/ be respectively the set of the jump and the non-jump indices of the orbit G`. We set Sk .`/ D S.`/ \ f1; : : : ; kg; Tk .`/ D T .`/ \ f1; : : : ; kg. If fXj g1j n is a Malcev basis of g relative to the composition series (9.1.1), then fXj gj 2T .`/ is a Malcev basis of g.`/. Using the notations introduced before, we define by   ˇ fsj gj 2S.`/nSk .`/ ; gk ; ftj gj 2T .`/nTk .`/ Y D # exp.sj Xj /gk  j 2S.`/nSk .`/

Y

" exp.tj Xj /

j 2T .`/nTk .`/

a polynomial map ˇ W R].S.`/nSk .`//  Gk  R].T .`/nTk .`// ! G;   which is a bijection. We fix ' 2 S R].S.`/nSk .`// such that '.0/ D 1. For ˚k 2 1 H` , if we give the function ˚ on G by k    ˚ ˇ fsj gj 2S.`/nSk .`/ ; gk ; ftj gj 2T .`/nTk .`/  i P  j 2.T .`/nTk .`// `.tj Xj / D ' fsj gj 2.S.`/nSk .`// e ˚k .gk /; then ˚ 2 H1` and ˚jGk D ˚k .



12.2 Corwin–Greenleaf Functions

399

Let  2 Zk .g; / and take a  -central element  in U.gk / such that ` ./ D .`/Id for ` 2 G . Then, with the notations of the lemma,     `k ./ ˚jGk .gk / D ` ./.˚/.gk / D .`/ ˚jGk .gk / for arbitrary ˚ 2 H` ; gk 2 Gk . Namely, `k ./ D .`/Id for ` 2 G . Because the G-invariance of  in G induces the G-invariance of k ./ in Gk , the proposition holds.  We indicate by L the composition series (9.1.1) of g and, for a Lie subalgebra k of g, put kj D k \ gj .0  j  n/. We set IK D f0g [ f1  j  nI kj 1 ¤ kj g and designate by LK the composition series fkj gj 2IK of k. As before, let g0 D gn1 ; G 0 D exp.g0 /, k0 D kn1 ; K 0 D exp.k0 / and L0 D LG 0 ;  0 D n1 . Moreover, set p D pn1 ; `0 D p.`/; IH D I; I 0 D IH 0 . If we set   O D O.; g ; K; LK / \ p 1 O. 0 ; .g0 / ; K 0 ; LK 0 / ; O and its projection O0 D p.O/ are Zariski open sets respectively in  and  0 . To say more precisely, O is composed of all ` 2  satisfying dim.kj .`// D min dim.kj .`//; 8 j 2 IK ; `2

dim.kj .`0 // D min dim.kj .`0 // D min dim.kj .`0 //; 8 j 2 IK 0 : 0 0 `2

` 2

Taking into account whether the Kj -orbits are saturated or not relative to g0 , we get the following situation. Proposition 12.2.8. There occurs the following alternative of (1) or (2). (1) T .e . 0 ; .g0 / ; K 0 ; LK 0 // D Tn1 .e .; g ; K; LK //. (2) There exists an integer rK 2 IK \ f1; : : : ; n  1g D IK 0 n f0g so that       T e  0 ; .g0 / ; K 0 ; L0K 0 D Tn1 e ; g ; K; LK [ frK g: In order that eventuality (1) occurs it is necessary and sufficient that g D g0 C k` at any ` 2 O. In order that eventuality (2) occurs it is necessary and sufficient that k`  g0 at any ` 2 O. Proof. Let j 2 IK 0 n f0g. Taking into account whether the Kj -orbits are saturated or not relative to g0 , we have the alternative of the following situations. (a) kj .`/ D kj .`0 / at any ` 2 O, in other words g D g0 C k`j . In this case, if j 0 2 IK 0 ; j 0  j , then kj 0 .`/ D kj 0 .`0 / at any ` 2 O.

400

12 Commutativity Conjecture: Induction Case

(b) kj .`/ ¨ kj .`0 / at any ` 2 O, in other words dim.kj .`// D dim.kj .`0 //  1, or equivalently k`j  g0 . In this case, if j 0 2 IK 0 ; j 0  j , then dim.kj 0 .`// D dim.kj 0 .`0 //  1 at any ` 2 O. Therefore, taking j D n  1, we find the following two possibilities. (a) At any ` 2 O, k0 .`/ D kn1 .`/ D kn1 .`0 / D k0 .`0 /: Thus, k.`/ D k.`0 /. In fact, this is clear if k D k0 . Otherwise, g D g0 C k and k.`/ D k \ g` D k \ .g0 /` \ k` D k.`0 /: Hence necessarily we are in situation (1). (b) The case where dim.k0 .`// D dim.k0 .`0 //  1 at any ` 2 O. In this case, there exists the minimal element rK 2 IK 0 n f0g such that dim.krK .`// D dim.krk .`0 //  1; 8 ` 2 O: Then, if j < rK , dim.kj .`// D dim.kj .`0 // at any ` 2 O and therefore       TrK 1 e  0 ; .g0 / ; K 0 ; LK 0 D TrK 1 e ; g ; K; LK : Similarly, if j > rK , dim.kj .`// D dim.kj .`0 //  1 at any ` 2 O and therefore T .e . 0 ; .g0 / ; K 0 ; LK 0 // nTrK .e . 0 ; .g0 / ; K 0 ; LK 0 // D T .e .; g ; K; LK // nTrK .e .; g ; K; LK // : Besides, at any ` 2 O, dim.krK 1 .`// D dim.krK 1 .`0 // and dim.krK .`// D dim.krK .`0 //  1: Hence, dim.krK .`//  dim.krK 1 .`// D dim.krK .`0 //  dim.krK 1 .`0 //  1; while, since the difference of the two terms concerning the dimension in both sides must be equal to either 0 or 1, it is necessarily the case that dim.krK .`0 // D dim.krK .`0 // C 1; dim.krK .`// D dim.krK 1 .`//:

12.2 Corwin–Greenleaf Functions

401

Thus,       rK 2 T e  0 ; .g0 / ; K 0 ; LK 0 ; rK 62 T e ; g ; K; LK : Consequently, we are inevitably in situation (2) and the proposition is established.  Following our usage of the notations, we set     eH 0 D e  0 ; .g0 / ; H 0 ; LH 0 ; e 0 D e  0 ; .g0 / ; G 0 ; L0 : Thus, T .eH 0 / D fj 2 I 0 I hj D hj 1 C hj .`0 /g; T .e 0 / D f1  j  n  1I gj D gj 1 C gj .`0 /g for general elements `0 of  0 . When we apply the induction on n  1, it is often useful to compare T .e/ and T .e 0 /, likewise T .eH / and T .eH 0 /. From now on, we put r D rH ; q D rG . Begin by rewriting the previous proposition in the cases where k D h and k D g. Proposition 12.2.9. In  we put      O D O ; g ; H; LH \ p 1 O  0 ; .g0 / ; H 0 ; LH 0 : Then we have the following alternative situations: (1) T .eH 0 / D Tn1 .eH /. (2) There exists r 2 I 0 n f0g such that T .eH 0 / D Tn1 .eH / [ frg. In order that eventuality (1) occurs it is necessary and sufficient that g D g0 C h` at any ` 2 O, or equivalently h.`/ D h.`0 /; `0 D p.`/. In order that eventuality (2) occurs it is necessary and sufficient that h`  g0 at any ` 2 O, or equivalently h.`/ ¨ h.`0 /; `0 D p.`/. In this case, dim.h.`// D dim.h.`0 //  1. Again in this situation, r is the element of I 0 uniquely determined by the following condition. At any ` 2 O, if j < r, g D g0 C h`j , or equivalently hj .`/ D hj .`0 / holds, and if r  j  n  1, h`j  g0 , or equivalently hj .`/ ¨ hj .`0 /; dim.hj .`// D dim.hj .`0 //  1: Proposition 12.2.10. In  we put      O D O ; g ; G; L \ p 1 O  0 ; .g0 / ; G 0 ; L0 : Then we have the following alternative situations: (1) T .e 0 / D Tn1 .e/. (2) There exists 1  q  n  1 such that T .e 0 / D Tn1 .e/ [ fqg.

402

12 Commutativity Conjecture: Induction Case

In order that eventuality (1) occurs it is necessary and sufficient that g D g0 C g` at any ` 2 O, or equivalently g.`/ D g.`0 /; `0 D p.`/. In order that eventuality (2) occurs it is necessary and sufficient that g`  g0 at any ` 2 O, or equivalently g.`/ ¨ g.`0 /; `0 D p.`/. In this case, dim.g.`// D dim.g.`0 //  1. Again in this situation, q is the element of I 0 uniquely determined by the following condition. At any ` 2 O, if j < q, g D g0 C g`j , or equivalently gj .`/ D gj .`0 / holds, and if q  j  n  1, g`j  g0 , or equivalently gj .`/ ¨ gj .`0 /; dim.gj .`// D dim.gj .`0 //  1: As a result of these two propositions: Proposition 12.2.11. When T .eH 0 / D Tn1 .eH / [ frg, there exists an integer q such that T .e 0 / D T .e/ [ fqg; 1  q  r. Proof. Indeed, since g`r  h`r  g0 , we obtain q  r from the above result.



When n > 1, we put Uk .e 0 / D U.e 0 / \ f1; : : : ; kg D Tk .e 0 /nTk .eH 0 / for 0 < k  n  1. Let us compare Uk .e/ and Uk .e 0 /. Proposition 12.2.12. Let 0  k  n  1. Only one of the following situations occurs: (1) Uk .e 0 / D Uk .e/. (2) Uk .e 0 / D Uk .e/ [ fqg. (3) Uk .e 0 / [ frg D Uk .e/ [ fqg. In order that eventuality (1) occurs it is necessary and sufficient that one of .1a/ q does not existI .1b/ k < qI .1c/ q D r  k holds. In order that eventuality (2) occurs it is necessary and sufficient that either .2a/ q  k and rdoes not exist or .2b/ q  k < r: And as for (3) it is necessary and sufficient that .3a/ q < r  k holds. Now we prepare a kind of differentiable curve in k . Lemma 12.2.13. Let V be a differentiable manifold, ˝ and P submanifolds embedded in V and ` 2 ˝ \ P . Suppose that there exist an open set U of V and an open set V of P such that ` 2 V  U \ P , and further a differentiable map F from U to a differentiable manifold W satisfying the following conditions: putting ' D F jV ,

12.2 Corwin–Greenleaf Functions

403

(1) ' 1 .'.`// D ˝ \ V; (2) F 1 .'.`//  ˝ \ U; (3) the rank of .d'/`0 is constant independent of `0 2 V. Then, T` .˝/ \ T` .P / D T` .˝ \ P /. In other words, the intersection of ˝ and P is transversal at the point `. Proof. No matter what the conditions, it is clear that T` .˝ \ P /  T` .˝/ \ T` .P /. In order to show the inverse inclusion, let Y 2 T` .˝/ \ T` .P /. Since Y 2 T` .˝/, .dF /` .Y / D 0 by (2). Besides, as Y 2 T` .P /, .d'/` .Y / has a meaning and .d'/` .Y / D .dF /` .Y / D 0. Therefore, Y 2 ker.d'/` D T` .˝ \ P / follows from (1) and (3).  Returning to the former situation, assume that a nilpotent Lie group K D exp k of dimension d acts linearly on a finite-dimensional real vector space V , and let P be an affine subspace of V . In these circumstances, the following holds. Theorem 12.2.14 (Transversality of Orbits). The interior OT .P; V; K/ relative to the Zariski topology of the set of all ` 2 P satisfying T` .K`/ \ T` .P / D T` ..K`/ \ P / is not a empty set. Proof. Let us make use of the following lemma. Lemma 12.2.15 (Cf. [16, pp. 88–93]). There exist a subset U of V , a mapping F0 from U to a vector space W and a family fUj ; pj ; qj g1j p of a Zariski open set Uj of V and polynomial maps pj ; qj on Uj in such a manner that: (1) (2) (3) (4) (5)

U is K-invariant; U \ P is a Zariski open set of P ; F0 .`1S / D F0 .`2 / ” `1 and `2 belong to the same K-orbit; U D 1j p .U \ Uj /; p qj does not vanish on Uj and F0 jU \Uj D qjj .

We look for appropriate U; F; V and, putting ˝ D K  ` for any ` 2 V, let us apply Lemma 12.2.13. In Lemma 12.2.15, one of the Zariski open sets Uj of V , say U1 , satisfies P \ U1 ¤ ;. Take U D U1 and set F D pq11 on U. In fact, on the nonempty Zariski open set U \U \P of P , the restriction F jU \U \P is well defined and becomes a rational mapping. Besides, take V as the Zariski open set of P composed of the points where the differential of this restriction has the maximal rank. Then, if we put ˝ D K  ` for any ` 2 V, Lemma 12.2.13 holds and V  OT .P; V; K/. So, the theorem is settled.  Let us abuse a little bit the notations used until now. Let a  g and p a vector subspace of g such that Œg; a  p. For ` 2 p , we set a` D fX 2 gI `.ŒX; a/ D f0gg. Corollary 12.2.16. Let k D pk . /  gk ; 0  k < n, ` be an element of the Zakiski open set OT .k ; gk ; G/ of k and finally X 2 h`k n g.`/. Then there exists a differentiable curve t 7! g.t/ from ."; "/ to G satisfying the following conditions:

404

12 Commutativity Conjecture: Induction Case

(1) g 0 .0/ X mod g.`/; (2) g.t/  ` 2 k ; " < t < ". Proof. Obviously X  ` ¤ 0 and X  `jhk D 0. Since ` 2 OT .k ; gk ; G/, the last theorem gives X  ` 2 .T` .G  `/ \ T` .k // nf0g D T` ..G  `/ \ k /nf0g: Therefore, there exists a differentiable curve  W t 7! .t/ from ."; "/ to .G  `/ \ k so that .0/ D `; 0 .0/ D X  `. Then, let s be a differentiable section from G=G.`/ ' G  ` to G. The mapping t 7! .s ı /.t/ is provided with the desired properties.  With these preparations we start to prove Theorem 12.2.5. The proof of the theorem will be done for each case listed in the next proposition which is immediately seen. Proposition 12.2.17. Among the cases listed below, only one occurs: (1) 0 D k D n. (2) 0  k < n: (a) Uk .e/ D Uk .e 0 /; (b) Uk .e/ [ fqg D Uk .e 0 /; (c) Uk .e/ [ fqg D Uk .e 0 / [ frg. (3) 0 < k D n: (a) (b) (c) (d)

n 2 S.e/; n 62 I; n 2 T .e/; n 2 I; n 2 U.e/; n 2 I; n 2 T .eH /.

Here, the integers r; q are respectively introduced by Propositions 12.2.9 and 12.2.10. The different cases of situation (2) were mentioned in Proposition 12.2.11. In the proof of Theorem 12.2.5 which we begin now, the proof for (2) will be done by induction on n. n  2 being given, the proof for cases (3)(a), (b) presumes the theorem in situation (2) for the same n. Further, the proof for (3)(c), (d) presumes the theorem for (3)(a), (b). Therefore, n being given, we need to establish the theorem in this order for each case listed up in this proposition. Proof of Theorem 12.2.5. For the brevity of the notation, we simply write Zk 0 instead of Zk .g; /. Likewise, Zk0 for Z.gk ;  0 / and Z 0 for Zn1 . Further, we write 0 G  for n1 W Zn1 ! .Z / in Proposition 12.2.6. Finally, we introduce the injection 0 0k from Zk0 to .Z k /G given by the same proposition at the level of G 0 through 0 0 0 k ./.`k / D .` / for  2 Zk0 ; `0 2 G 0   0 ; `0k D `0 jgk , where Z k stands for Z.gk ; k /.

12.2 Corwin–Greenleaf Functions

405

(1) The case 0 D k D n. We have U.g0 / D C, while, since U0 .e/ D ;, the transcendental degree of Z0 and ].U0 .e// are both 0. The assertion holds clearly. (2) The case 0  k < n. We employ induction on n. In these cases, we make  2 Zk correspond to  0 D ./ 2 Zk0 . Especially, we make the family f'j gj 2Uk .e/ of Corwin–Greenleaf functions in Zk correspond to the family 'j0 D .'j / of those in Zk0 . (a) Let Uk .e/ D Uk .e 0 /. By the induction hypothesis, the functions f'j0 gj 2Uk .e0 / make a transcendental basis of Zk0 and belong to the subalgebra .Zk / of Zk0 . Hence these make a transcendental basis of .Zk / too. As Zk is isomorphic to .Zk /, f'j gj 2Uk .e/ is a transcendental basis of Zk . (b) Let Uk .e/ [ fqg D Uk .e 0 /. If we take in Zk0 the Corwin–Greenleaf function 'q corresponding to the index q, the induction hypothesis assures that f'j0 gj 2Uk .e/ [ f'q g makes a transcendental basis of Zk0 . Hence f'j0 gj 2Uk .e/ are algebraically independent in .Zk /  Zk0 and f'j gj 2Uk .e/ are algebraically independent in Zk . Let us see that those are algebraic generators. Let  2 Zk . By the induction hypothesis,  0 satisfies a non-trivial algebraic relation with coefficients in i h 0 C f'j gj 2Uk .e/ ; 'q . Arranging this relation according to the power of 'q , X

  Pˇ f'j .`/gj 2Uk .e/ ; .`/ 'q .`0 /ˇ D 0

0ˇ˛

  at any ` 2  . Here Pˇ 2 C f'j gj 2Uk .e/ ;  and `0 D p.`/. By the assumption, the power of  in at least one of the Pˇ , say P D P , is larger than or equal to 1. Therefore,  if we can show  that all Pˇ .0  ˇ  ˛/ are 0, we will have in particular P f'j gj 2Uk .e/ ;  D 0 and a desired algebraic dependence. Recall that f'j gj 2Uk .e/ depend only on `k D pk .`/, More precisely, 'j .`/ D k .'j /.`k /; 8 j 2 Uk .e/; 8 ` 2 G  ; 'q .`0 / D 0k .'q /.`k /; 8 ` 2 G 0  ; `0 D p.`/: Let us consider the following functions. For 0  ˇ  ˛, the functions    Qˇ D k Pˇ f'j gj 2Uk .e/ ;  on G  k . These are elements of .Z k /G . Besides, # D 0k .'q / and R D P ˇ 0 k G0 0ˇ˛ Qˇ # on G k are elements of .Z / . The restrictions of these functions to k D pk . / are polynomial functions.

406

12 Commutativity Conjecture: Induction Case

Using the notations used before in Theorem 12.2.14 and others, we define a nonempty Zariski open set of k by     O D pk O.; g ; G; L/ \ pk0 O. 0 ; .g0 / ; G; L/     \ pk O. ; g ; H; LH / \ pk0 O. 0 ; .g0 / ; H; LH / \ OT .k ; gk ; G/: Let us show that the polynomial x 7!

X

Qˇ .`/x ˇ

0ˇ˛

has infinite zero points for ` 2 O. If so, it follows that Qˇ and hence Pˇ is 0. The theorem will be settled in this situation. We make use of the form of the Corwin– Greenleaf function 'j and the following facts: (i) Qˇ is G-invariant; (ii) # is G 0 -invariant; (iii) R is identically 0 on G 0  k . For arbitrary ` 2 gk , we put `q1 D `jgq1 . From the properties of the Corwin– Greenleaf function described in Chap. 9, # is written as #.`/ D pQq .`q1 /`.Xq / C qQ q .`q1 /; 8 ` 2 k ; where Xq 2 gq ngq1 and pQq is an G-invariant polynomial function not identically 0. Let us utilize Corollary 12.2.16 and the notations employed there. Let ` 2 O. Since g D g0 C h`k , take an element X` of h`k n g0 . As q  k, g.`/  g0 and therefore X` 62 g.`/. Let us consider a differentiable curve t 7! g` .t/ from ."; "/ to G such that g`0 .0/ X` modulo g0 and that g` .t/  ` 2 k for all t. Next, since g D g0 C g`q1 , there exists an element Z` of g0 so that Y` D g`0 .0/ C Z` 2 g`q1 ng0 . Now if we give a differentiable curve t 7! h` .t/ from ."; "/ to G by h` .t/ D exp.tZ` /g` .t/, then Y` D h0` .0/. If we put here `t D h` .t/  `, `t 2 G 0  k for any t. Hence R.`t / D 0 by (iii). Secondly by (i), X 0ˇ˛

Qˇ .`t /#.`t /ˇ D

X

Qˇ .`/#.`t /ˇ D 0:

0ˇ˛

To show Qˇ D 0 from this, if we see that the value at the point 0 of the derivative of the differentiable function t 7! #.`t / is not equal to 0, this function will have infinite values on an open interval ."; "/ and we should have Qˇ D 0. Since Y` 2 g.`q1 / D g`q1 , Y`  `q1 D 0, while `.ŒXq ; Y` / ¤ 0. In fact, if `.ŒXq ; Y` / D 0, then `.Œgq ; Y` / D f0g, which contradicts g`q  g0 ; Y` 62 g0 . Therefore, on a non-empty Zariski open set of k ,

12.2 Corwin–Greenleaf Functions

407

  d d #.`t /jt D0 D pQq h` .t/  `q1 jt D0 `.Xq / dt dt d d C pQq .`q1 / .h` .t/  `/jt D0 .Xq / C qQ q .h` .t/  `q1 /jt D0 dt dt D.d pQq /`q1 .Y`  `q1 /`.Xq / C pQq .`q1 /`.ŒXq ; Y` / C .d qQq /`q1 .Y`  `q1 / DpQq .`q1 /`.ŒXq ; Y` / ¤ 0: In consequence this case has been settled. (c) The case where Uk .e/ [ fqg D Uk .e 0 / [ frg; q < r. In this case, q; r  k; r 2 Tk .e/ \ Tk .eH 0 / and r 62 Tk .eH /. If we denote again by 'q the Corwin– Greenleaf function in Zk0 corresponding to the index q, the treatment of this case is due to the following result. i h Lemma 12.2.18. In these circumstances, 'q is algebraic over C f'j0 gj 2Ur .e/ . Proof. 'r0 2 Zr0 and Ur .e 0 / D Ur1 .e/ [ fqg. By the induction hypothesis, f'j0 gj 2Ur1 .e/ [ f'q g makes a transcendental basis of Zr0 . Especially, 'r0 is algebraic on i h C f'j0 gj 2Ur1 .e/ ; 'q : i h Hence, using Q 2 C f'j0 gj 2Ur1 .e/ ; 'q ; 'r0 , there exists an algebraic relation Q D 0 and the coefficient of the term in Q having the maximal degree for 'r0 is i h

an element of C f'j0 gj 2Ur1 .e/ ; 'q not equal to 0. Arranging Q according to the power of 'q , two cases are possible: ˇ

(1) Among the coefficients of 'q in Q, there is one not equal to 0. Then 'q becomes algebraic on i h i h C f'j0 gj 2Ur1 .e/ ; 'r0 D C f'j0 gj 2Ur .e/ and the lemma follows. ˇ (2) The coefficients of 'q in Q are all 0. Then 'r0 is algebraic on i h C f'j0 gj 2Ur1 .e/ :   In other words, 'r is algebraic on C f'j gj 2Ur1 .e/ .

408

12 Commutativity Conjecture: Induction Case

In order to show the lemma, it is enough to see that this second possibility supplies a contradiction. The relation X   Pˇ 'rˇ D 0; Pˇ 2 C f'j gj 2Ur1 .e/ 0ˇ˛

being given, we show P˛ D 0. Then, we are led to a contradiction. The function 'r depends only on `r and the family of functions f'j gj 2Ur1 .e/ depend only on `r1 . More precisely, putting `r D pr .`/; `r1 D pr1 .`/ for arbitrary ` 2 G   , 'r .`/ D r .'r /.`r /; 'j .`/ D r1 .'j /.`r1 /; 8 j 2 Ur1 .e/: We take Xr 2 gr ngr1 in h and put D f .Xr /. Let  W ` 7! `jgr1 be the projection from gr to gr1 . Then, we can write r .'r /.`/ D pQr ..`//`.Xr / C qQ r ..`//; 8 ` 2 G  r ; as a Corwin–Greenleaf function. 1  From nowon, we identify  .r1 / and r1  R by the bijection ` 7! `jgr1 ; `.Xr / . Then, r ' r1  f g. The restriction of the mapping  gives a bijection from r onto r1 and .`; / D ` for ` 2 r1 . Utilizing the function # given by #.`; x/ D pQr .`/x C qQ r .`/ on  1 .r1 / D r1  R, 'r jr D #jr1 f g and 'r .`; / D pQr .`/x C #.`; x/ C pQr .`/

for .`; x/ 2 r1  R. If we substitute this right member into the expression X   Pˇ fr1 .'j /.`/gj 2Ur1 .e/ 'r .`; /ˇ ; ` 7! 0ˇ˛

which is by assumption identically 0 on r1 , we obtain the identity X

.1/ 

Qˇ

 fr1 .'j /.`/gj 2Ur1 .e/ ; pQr .`/; #.`; x/ x ˇ D 0

0ˇ˛

for all ` 2 r1 ; x 2 R. If we put .1/ 

Qˇ .`; x/ D Qˇ

 fr1 .'j /.`/gj 2Ur1 .e/ ; pQr .`/; #.`; x/ ;

  Q˛ .`; x/ D P˛ fr1 .'j /.`/gj 2Ur1 .e/ .pQr .`//˛ does not depend on x.

12.2 Corwin–Greenleaf Functions

409

Using the notations given before in Theorem 12.2.14 and others, we define a non-empty Zariski open set of r1 by     0 O D pr1 O.; g ; G; L/ \ pr1 O. 0 ; .g0 / ; G; L/     0 \ pr1 O.; g ; H; LH / \ pr1 O. 0 ; .g0 / ; H; LH / \ OT .r1 ; gr1 ; G/: Let ` 2 O and utilize Corollary 12.2.16 and the notations employed there. In our case, since g D g0 C .gr1 \ h/` , we take X` 2 .gr1 \ h/` in the outside of g0 . As q  r  1, g.`/  g0 and X` 62 g.`/. Take a differentiable curve t 7! g` .t/ from ."; "/ to G such that g`0 .0/ X` modulo g.`/ and `t D g` .t/  ` 2 r1 for any t. Taking into account that g` .t/ keeps  1 .r1 / stable, put .`t ; xt / D g` .t/  .`; /. Since the functions 'j ; j 2 Ur .e/; and pQr are invariant by the action of G, the function t 7! Qˇ .`t ; xt / is constant. Therefore, we know that t 7!

X

ˇ

Qˇ .`; /xt D 0; 8 t 2 ."; "/:

0ˇ˛

Now, let us see that the mapping t 7! xt admits infinite number of values. If so, it follows that Q˛ .`; / D 0 for ` 2 O. Since pQr is not identically 0 on r1 , we conclude that P˛ D 0 and the lemma holds. For this aim, it suffices to show xt0 .0/ ¤ 0. So, let us remark that `.ŒXr ; X` / ¤ 0. Indeed, if `.ŒXr ; X` / D 0, `.Œgr ; X` / D f0g, which contradicts g`r  g0 ; X` 62 g0 . Hence xt0 .0/ D .gt0 .0/  `/.Xr / D `.ŒXr ; X` / ¤ 0 and we get the lemma.  In order to finish the treatment of this situation in the proof of the theorem, we start from the presumed fact by induction that f'j gj 2Uk .e/nfrg [ f'q g is a transcendental basis of Zk0 . From this: (1) The transcendental degree of Zk0 is equal i to ].Uk .e//. h 0 0 (2) Zk is algebraic on C f'j gj 2Uk .e/ ; 'q . h i h i By the last lemma, 'q is algebraic on C f'j0 gj 2Uk .e/ and C f'j0 gj 2Uk .e/ ; 'q i h h is algebraic on C f'j0 gj 2Uk .e/ . Hence, Zk0 becomes algebraic on C f'j0 gj 2 Uk .e/ and f'j0 gj 2Uk .e/ makes a system of algebraic generators of Zk0 . Since the transcendental degree of Zk0 is equal to ].Uk .e//, f'j0 gj 2Uk .e/ is a transcendental basis of Zk0 contained in .Zk /. In consequence, f'j gj 2Uk .e/ becomes a transcendental basis of Zk and this case is settled.

410

12 Commutativity Conjecture: Induction Case

(3) The case where 0 < k D n. We use the just-established fact that f'j gj 2Un1 .e/ makes a transcendental basis of Zn1 . Besides, we will utilize twice the following lemma. Lemma 12.2.19. Let 0 < k D n and assume n 2 U.e/. Then, f'j gj 2U.e/ is a system of algebraic generators of Z. Proof. T .e/ D Tn1 .e/ [ fng and U.e/ D Un1 .e/ [ fng. By Theorem 12.2.1, Z is algebraic on     C f'j gj 2T .e/ D C f'j gj 2Tn1 .e/ Œ'n ;       on C f'j gj 2Un1 .e/ , C f'j gj 2T .e/ while, since C f'j gj 2Tn1  .e/ is algebraic  becomes algebraic on C f'j gj 2U.e/ and the system f'j gj 2U.e/ is a system of algebraic generators.  (a) The case where 0 < k D n; n 2 S.e/. In this case, U.e/ D Un1 .e/, while, by Theorem 12.2.1, Z D Zn1 and the theorem follows. (b) The case where 0 < k D n; n 62 I; n 2 T .e/. By Lemma 12.2.19, the system f'j gj 2U.e/ is a system of algebraic generators of Z. It remains to see that this system is algebraically free. So, let us see that the relation of the form X

  Pˇ 'nˇ D 0; Pˇ 2 C f'j gj 2Un1 .e/

0ˇ˛

means Pˇ D 0; 0  ˇ  ˛;. Then such a relation reduces to the trivial one and the system f'j gj 2U.e/ turns out to be algebraically free in Z. For this aim, it suffices to see that, for a general element ` of  , the polynomial x 7!

X

Pˇ .`/x ˇ

0ˇ˛

has infinite number of zero points. Let ` 2 ; Xn 2 g n g0 and `0 D p.`/. We consider the differentiable curve t 7! `t from R to  given by p.`t / D `0 ; `t .Xn / D t. Because Pˇ .`t / D .Pˇ /.p.`t // D Pˇ .`/ for all ˇ, X

Pˇ .`/'n .`t /ˇ D 0:

0ˇ˛

Finally, it suffices to see that, for a general element ` of  , the mapping t 7! 'n .`t / admits an infinite number of values. This is derived from the expression of the Corwin–Greenleaf function 'n . Indeed, 'n .`t / D pQn .`0 /t C qQn .`0 /

12.2 Corwin–Greenleaf Functions

411

and the function ` 7! pQn .`0 / does not vanish on a Zariski open set of  . Thus this situation has been treated. In order to treat the remaining cases, let us begin with a simple lemma which we utilized several times. Lemma 12.2.20. Provided h ¤ g, there is a composition series of ideals LQ D fQgj g0j n in g such that h  gQ n1 . Proof. Take an ideal gQ n1 of codimension 1 in g, which contains h. Then it suffices to take a composition series of the g-module gQ n1 .  Lemma 12.2.21. The transcendental degree of Z is equal to ].U.e//. Proof. This result has just been proved in all circumstances where h  g0 . So, let us assume n 2 I and use the fact that ].U.e// D min dim.g.`//  min dim.h.`//: `2

`2

(12.2.4)

The lemma is evident if h D g. Indeed, in this case,  D ff g; g.f / D h.f / D g. Hence the transcendental degree of Z and ].U.e// are both 0. Let h ¤ g. Because the right member of Eq. (12.2.4) does not depend on the choice of the composition series L, it suffices to replace L by LQ so that h  gQ n1 .  (c) The case where 0 < k D n; n 2 I; n 2 T .e/ and n 62 T .eH /. In this case, n 2 U.e/. Taking into account the algebraic dimension, it suffices that f'j gj 2U.e/ is a system of algebraic generators of Z. However, this is already shown in Lemma 12.2.19. (d) The case where 0 < k D n; n 2 I; n 2 T .e/ and n 2 T .eH /. In this case, n 62 U.e/. By the same reasoning as in the preceding situation, it suffices that f'j gj 2U.e/ is an algebraically free system in Z. Since U.e/ D Un1 .e/, this system is a transcendental basis of Zn1 and clearly satisfies the desired property. Theorem 12.2.5 concerning the transcendental basis of Z bas been proved. 

Key Lemmas Now we are ready to explain in detail Lemma 9.2.14 whose concrete examples were computed in Chap. 9. Lemma 12.2.22. Assume dim h  1 and let is 2 T .eH /. That is to say, at a general ` 2  , hs D hs1 C hs .`/ and there exists k .1  k  t/ such that mk D is . Then, the following assertions hold.

412

12 Commutativity Conjecture: Induction Case

(i) There exists a polynomial P satisfying P .˙.1 /; : : : ; ˙.k // 0 mod U.gmk /as ;

(12.2.5)

where the coefficient of the maximal power of ˙.k / does not belong to U.g/a . (ii) There exists a polynomial Q satisfying Q .˙.1 /; : : : ; ˙.k /; Ys / 0 mod U.gmk 1 /as1 ; where the coefficient of the maximal power of Ys does not belong to U.g/a . Proof. As mk 2 T .eH /, mk 62 U.e/. Hence by Theorem 12.2.5, the family f'j I j 2 Umk 1 .e/g makes a transcendental basis of Zmk .g; /. Especially, the element 'k of Zmk .g; / is algebraic on the ring generated by this family and hence naturally on the ring generated by the family f'j I j 2 Tmk 1 .e/g. In other words, there exists a polynomial P of k-variables so that P .'1 ; : : : ; 'k / D

m X

Pj .'1 ; : : : ; 'k1 /'jk D 0

j D0

with Pm .'1 ; : : : ; 'k1 / ¤ 0. From this, $ .P .˙.1 /; : : : ; ˙.k /// D ı.P .'1 ; : : : ; 'k // D 0 and $ .Pm .˙.1 /; : : : ; ˙.k1 /// ¤ 0. From this, P .˙.1 /; : : : ; ˙.k // 2 U.g/a \ U.gmk / D U.gmk /as and Pm .˙.1 /; : : : ; ˙.k1 // 62 U.g/a . (ii) First, remark that YOs 2 U.g/a  UC .g; /. We write k D k Ys C Rk , where

k ; Rk 2 U.gmk 1 /. Thus, ˙.k / D Ys ˙. k / C ˙.Rk /: In this manner Eq. (12.2.5) is rewritten as     p P ˙.1 /; : : : ; ˙.k / C Ys C 1f .Ys / ˙. k / 0 mod U.gmk 1 /as1 (12.2.6) and Ys disappears. By the way, let us notice that in relation (12.2.6) the elements ˙.r / .1  r  k/; k and Ys belong to UC .g; /. Developing P with respect to Ys ,

12.2 Corwin–Greenleaf Functions

413

 $ Pm .˙.1 /; : : : ; ˙.k1 //.˙. k /Ys /m C

m1 X

 QQ j .˙.1 /; : : : ; ˙.k /; ˙. k //Ysj D 0;

(12.2.7)

j D0

where QQ j are polynomials whose degree relative to ˙. k / is less than or equal to m. Now, by Theorem 12.2.1, there exist two polynomials S; T of k  1 variables so that $ .S .˙.1 /; : : : ; ˙.k1 // ˙. k // D $ .T .˙.1 /; : : : ; ˙.k1 /// and the both members of this equation are not 0. We multiply (12.2.7) by $ .S .˙.1 /; : : : ; ˙.k1 //m / to find that $

m  T .˙.1 /; : : : ; ˙.k1 // Pm .˙.1 /; : : : ; ˙.k1 // Ysm C

m1 X

! Qj .˙.1 /; : : : ; ˙.k // Ysj

D0

j D0

holds with some polynomials Qj and (ii) follows.



Lemma 12.2.23. Suppose that dim h  1 and h D h0 C h.`/ at a general ` 2  . Then, R.Yd / is algebraic on CD 0 .G=H /. In other words, provided id 2 T .eH /, there exists a polynomial Q of Yd with coefficients in UC .g;  0 / satisfying Q.Yd / 2 U.g/a 0 , and such that the coefficient of the maximal power of Yd does not belong to U.g/a 0 . Proof. Let e 0 2 E be the multi-index such that  0 \ Ue0 becomes a non-empty Zariski open set of  0 . Let us begin by showing Tid .e 0 / D Tid .e/. Introducing the restriction map p W g ! gid 1 , p is a continuous open map with respect to the Zariski topology. Let us remark that p. /  p. 0 /\gid 1 and p 1 .p. //   0 . Take 1  r  id . Since Œg; gid   gid 1 , the restriction of arbitrary ` 2 g to Œg; gid  is not changed when we replace ` by an arbitrary element of p 1 .p.`//. The same applies for gr .`/. Assume r 2 Tid .e/ (resp. r 62 Tid .e/). Namely, gr D gr1 C gr .`/ (resp. gr .`/  gr1 ) on a non-empty Zariski open set O of  . The same relation holds on a nonempty Zariski open set p 1 .p.O// of  0 . Thus, r 2 T .e 0 / (resp. r 62 T .e 0 /) and our assertion follows. In this way, there exists a sub-sequence .r /1rk of  0 -central elements. For arbitrary  0 -central element , ` ./ is a scalar at all ` 2  0 hence at all ` 2    0 . If so, .r /1rk is a sequence of  -central elements too. We take this sequence, choose s D d and can apply Lemma 12.2.22 (ii). The desired result comes from the fact that ˙.r / belongs to UC .g; / \ UC .g;  0 /. 

414

12 Commutativity Conjecture: Induction Case

Example 12.2.24. We take up once again g D hX1 ; : : : ; X5 iR W ŒX5 ; Xk  D Xk1 .2  k  4/; which was treated in Example 9.2.15 of Chap. 9. Let f D a3 X3 C a4 X4 with a3 ; a4 2 R and h D RX3 ˚ RX4 . Then, h 2 S.f; g/. We write our affine space as  D f` D f C `1 X1 C `2 X2 C `5 X5 I .`1 ; `2 ; `5 / 2 R3 g Š R3 : Setting gj D hX1 ; : : : ; Xj iR as before, we consider a composition series f0g D g0  g1  g2  g3  g4  g5 D g of g. If we introduce a Zariski open set O D f` 2  I `1 ¤ 0; `2 ¤ 0g of  , at arbitrary ` 2 O, e D e.`/ D .0; 1; 1; 1; 2/ and T .e/ D f1; 3; 4g; S.e/ D f2; 5g; T .eH / D f4g; U.e/ D f1; 3g. According to this, we obtained as Corwin– Greenleaf e-central elements A1 D 1 D X1 ; A3 D 2 D 2X1 X3  X22 ; A4 D 3 D X12 X4  X1 X2 X3 C

X23 : 3

They belong to the centre of U.g/. In order to make more precise the corresponding Corwin–Greenleaf functions f'j gj D1;3;4 and the operators f` .Xj /g1j 4 , let us realize the representation ` of U.g/ in S.R/ and S  .R/ by means of the common P4 polarization b D j D1 RXj at ` 2 O and the coexponential basis X5 to b in g. By a simple computation, for arbitrary 2 S.R/, i .` .X1 / / .x/ D `1 .x/ i .` .X2 / / .x/ D .`2  `1 x/ .x/   x2 .x/ i .` .X3 / / .x/ D a3  `2 x C `1 2   x2 x3  `1 .x/: i .` .X4 / / .x/ D a4  a3 x C `2 2 3Š So, if we set F1 .`1 ; `2 I x/ D `1 F2 .`1 ; `2 I x/ D `2  `1 x F3 .`1 ; `2 I x/ D a3  `2 x C `1

x2 2

F4 .`1 ; `2 I x/ D a4  a3 x C `2

x2 x3  `1 ; 2 3Š

12.2 Corwin–Greenleaf Functions

415

then each Zk ; 1  k  4; is a subring of the polynomial ring generated by fFj g1j k and composed of functions independent of x. Especially, i'1 .`1 ; `2 ; `5 / D `1 '3 .`1 ; `2 ; `5 / D 2a3 `1  `22 i'4 .`1 ; `2 ; `5 / D a4 `21  a3 `1 `2 C

`32 : 3

As we have seen until now, Z1 D Z2 D CŒ'1  and f'1 ; '3 g makes a system of rational generators of Z3 and at the same time a transcendental basis of Z4 , while f'1 ; '3 ; '4 g makes a system of rational generators of Z4 and Z D Z5 D Z4 . Proposition 12.2.25. In particular, let f D 0, i.e. a3 D a4 D 0. Then, (1) Z3 D CŒ`1 ; `22 ; (2) Z D Z4 D Z5 is isomorphic to the subalgebra of CŒ`1 ; `2  spanned linearly by p q the monomials `1 `2 verifying q ¤ 1. Proof. (1) The elements of Z3 are polynomials of `1 ; `2 and rational fractional functions of `1 ; `22 . Hence, by the parity of the degree, they are elements of CŒ`1 ; `22 . (2) Let Z 0 denote the subalgebra in question. Since i'1 D `1 ; '3 D `22 ; 3i'4 D `32 in our situation, evidently Z 0  Z. To show the inverse inclusion, we give ' 2 Z arbitrarily. Then, there exists a polynomial P such that ' D P .F1 ; F2 ; F3 ; F4 / is independent of x. When we develop P by means of concrete expressions of Fj ; 1  j  4, the monomials of Fj ; 1  j  4, which give monomials p pC1 p of `1 ; `2 ; x having the forms `1 `2 and `1 x, are only F1 F2 . Hence, in the p development of ' by `1 ; `2 , the coefficient of the monomial `1 `2 is 0, as is that pC1 0 of `1 x. Thus ' 2 Z .  Remark 12.2.26. Let k 2 T .eH /, then f'j gj 2Uk1 .e/ D f'j gj 2Uk .e/ and f'j gj 2Tk1 .e/ containing f'j gj 2Uk1 .e/ is a system of rational generators of Zk1 and also a system of algebraic generators of Zk . But in general it is not a system of rational generators of Zk . Further, Zk1 and Zk admit the same transcendental basis but in general they are not equal. Indeed, let k D 4 for instance in the previous proposition. Then, we saw that i'1 D `1 ; '3 D `22 ; i'4 D

`32 : 3

Thus 9'42 C '33 D 0 and '4 is algebraically dependent of '3 . However, if there exist polynomials S.'1 ; '3 /; T .'1 ; '3 / satisfying S.'1 ; '3 /'4 D T .'1 ; '3 /;

416

12 Commutativity Conjecture: Induction Case

the parities of the degree of `2 will be different in the both members of this equation. That is to say, there are no such polynomials S; T . Besides, since '4 2 Z4 nZ3 , it is clear that Z4 ¤ Z3 . Remark 12.2.27. In general, Z is not a polynomial ring. In fact, let us consider for example the situation of the last proposition. Let Z  be the ideal of Z composed of all functions which vanish at `1 D `2 D 0. Besides, we denote by I the ideal p q of Z spanned linearly by the elements of the form `1 `2 where p  2; q D 0 or p  1; q  2 or q  4. Then, since Z  D C`1 ˚ C`22 ˚ C`32 ˚ I and Z  Z  D I; f`1 ; `22 ; `32 g makes a basis of the vector space Z  =I and dim.Z  =I / D 3. Since the transcendental degree of Z is 2, assume that f˛1 D ˛1 .`1 ; `2 /; ˛2 D ˛2 .`1 ; `2 /g form Z as polynomial ring. Without loss of generality, we may suppose that ˛1 ; ˛2 2 Z  . Then, all elements of Z  are written as polynomials of ˛1 ; ˛2 and, since Z  Z  D I , it follows that f˛1 ; ˛2 g makes a basis of the vector space Z  =I . But this contradicts dim.Z  =I / D 3.

12.3 Proof of the Commutativity Conjecture We are now ready to establish the commutativity conjecture. Let us begin with a simple lemma which will be repeatedly utilized in the proof of the next theorem. Lemma 12.3.1. Let k; k0 be two Lie subalgebras of g such that h  k0  k. Let h00 be a Lie subalgebra of h. Then, the following properties are mutually equivalent: (i) For a general ` 2  , h00 \k` D h00 \k0 ` .resp: dim.h00 \k` / D dim.h00 \k0 ` /1/. (ii) At a general ` 2 k;h00 D f` 2 k I `jh00 D f jh00 g, h00 \ k` D h00 \ k0 ` .resp: dim.h00 \ k` / D dim.h00 \ k0 ` /  1/. Proof. Let k00 be the ideal of k generated by h00 . Let p W g ! k00  and q W k ! k00  be the restriction maps. Let us make use of the fact that these are both continuous open maps with respect to the Zariski topology. Using thenotations similar to k;h00 , p. /  k00 ;h00 ; q 1 .p. //  k;h00 and p 1 q.k;h00 /  g;h00 . Let ` 2 g and  2 q 1 .p.`//. Since Œk; h00   k00 , `jŒk;h00  D jŒk;h00  . From this, h00 \ k` D h00 \ k and h00 \ k0 ` D h00 \ k0  . Hence, if one of the two claims of (i) holds on a non-empty Zariski open set O of  , then it holds on the non-empty Zariski open set q 1 .p.O// of k;h00 too. In this way, (i) means (ii).

12.3 Proof of the Commutativity Conjecture

417

Conversely, let ` 2 k and  2 p 1 .q.`//  g . By the same arguments as above, if one of the two claims of (ii) holds on a non-empty Zariski open set O of k;h00 , it holds on the non-empty Zariski open set p 1 .q.O// \  of  . Thus, (ii) means (i).  The commutativity conjecture is a by-product of the following theorem. Theorem 12.3.2. Let G be a connected and simply connected nilpotent Lie group with Lie algebra g and assume dim g  1. Let H be a closed connected proper subgroup of G with Lie algebra h, f an element of the linear dual space g of g such that f .Œh; h/ D f0g and g0 an ideal of codimension 1 in g containing h. Then, the following properties are mutually equivalent: (i) U.g; /  U.g0 / C U.g/a . (ii) At a general ` 2  , the H -orbit H  ` is saturated in the direction of g0 ? . Proof. It is convenient to show the following equivalence: U.g; /  U.g0 / C U.g/a ” dim.h.`// D dim.h.`0 //  1 for a general ` 2  : Let us use the inductions on the dimension n of g and the dimension d of h. When d D 0, clearly at all ` 2 g , h.`/ D h.`0 / D f0g. Besides, U.g; / (resp. U.g0 / C U.g/a ) is equal to U.g/ (resp. U.g0 /). Hence, the theorem is trivial in this case. Let n > d  1. Assume that by the induction hypothesis the theorem is correct Q fQ/ with required properties, where dim gQ < n or dim gQ D n and for .Qg; gQ 0 ; h; dim hQ < d . Choosing h0 as before, h0 ¨ h  g0 ¨ g. There are various cases. At a general ` 2  , h.`/ D h.`0 / or dim.h.`// D dim.h.`0 //  1. At a general ` 2  0 , h0 .`/ D h0 .`0 / or dim.h0 .`// D dim.h0 .`0 //  1. If we replace in Lemma 12.3.1 k by g, k0 by g0 and h00 by h0 , we know that h0 .`/ D h0 .`0 / .resp. dim.h0 .`// D dim.h0 .`0 //  1/ at a general ` 2  0 when and only when the same property holds at a general ` 2  . Moreover, if h.`/ D h.`0 / at a general ` 2  , then evidently h0 .`/ D h0 .`0 / at a general ` 2  . These remarks lead us to the following three cases. Case 1.

At a general ` 2  , dim.h0 .`// D dim.h0 .`0 //  1.

As we already remarked, in this case dim.h.`// D dim.h.`0 //1 generally. First, by the induction hypothesis applied to .g; h0 /, we have U.g;  0 /  U.g0 / C U.g/a 0 : Here, U.g; /  U.g0 / C U.g/a :

418

12 Commutativity Conjecture: Induction Case

In fact, suppose that there exists an element W 2 U.g; / such that W 62 U.g0 / C U.g/a , then from the second assertion of Proposition 12.1.4, there exists W 0 2 U.g;  0 / n .U.g0 / C U.g/a 0 / satisfying W 0 W modulo U.g/a . This contradicts the induction hypothesis. Case 2.

At a general ` 2  , h.`/ D h.`0 /.

In this case, at a general ` 2  0 , h0 .`/ D h0 .`0 /. By the induction hypothesis applied to .g; h0 /, we have U.g;  0 / 6 U.g0 / C U.g/a 0 : We divide this case further into two subcases. (2a)

U.g;  0 / \ U.g0 / 6 U.g; / \ U.g0 /.

Taking what we have seen until now into account, we know immediately from Theorem 12.1.9 that U.g; / 6 U.g0 / C U.g/a : (2b)

U.g;  0 / \ U.g0 /  U.g; / \ U.g0 /.

Our first objective is to show that h D h0 C h.`/, i.e. id 2 T .eH /, at a general ` 2  and to make use of Lemma 12.2.23. First of all, we show by induction that, for 0  r  q  1, h D h0 C .h \ k`r / at a general ` 2  . This is evident for r D 0 because of h \ k`0 D h. Let r > 0 and assume that the property is true until the step of r  1. Recall our assumption U.g;  0 / \ U.kr /  U.g; / \ U.kr /. Then, replacing g by kr in assertion (ii) of Proposition 12.1.6, we have U.g; / \ U.kr / 6 U.kr1 / C U.kr /a ; or U.g; / \ U.kr /  U.kr1 / C U.kr /a and U.g;  0 / \ U.kr /  U.kr1 / C U.kr /a 0 : ` ` Hence, by the induction hypothesis on the dimension  of G, h \ kr D h \ kr1 at a   ` ` general ` 2  \ kr , or dim h \ kr D dim h \ kr1  1 at a general ` 2  \ kr     and dim h0 \ k`r D dim h0 \ k`r1  1 at a general ` 2  0 \ kr . Replacing k by kr , k0 by kr1 and h00 by h or h0 in Lemma 12.3.1, as saying that    this is the same h \ k`r D h \ k`r1 at a general ` 2  , or dim h \ k`r D dim h \ k`r1  1 and     dim h0 \ k`r D dim h0 \ k`r1  1 at a general ` 2  . In the first case, by induction on r D dim.kr =h/, we clearly have h D h0 C ` 0 .h \ k`r / at a general ` 2  , while, in the second case, the assumption  h \ `kr   h leads us to a contradiction. In fact, verifying by induction that dim h \ kr1 D   dim h0 \ k`r1 C 1, we deduce         dim h \ k`r D dim h0 \ k`r D dim h0 \ k`r1  1 D dim h \ k`r1  2

from h \ k`r D h0 \ k`r . But this is impossible since kr1 has the codimension 1 in kr .

12.3 Proof of the Commutativity Conjecture

419

Thus, in all cases, h D h0 C .h \ k`r / at a general ` 2  . Applying this result as r D q  1, h D h0 C h.`0 / at a general element ` 2  , we get h D h0 C h.`/ by h.`/ D h.`0 /. Now, let us show U.g; / 6 U.g0 / C U.g/a . From the first assertion of Proposition 12.1.6, U.g;  0 / 6 U.g0 / C U.g/a and there exists   W D Xq U C V 2 U.g;  0 / n U.g0 / C U.g/a from the third assertion of Proposition 12.1.8. Here U 2 .U.g;  0 / \ U.g0 // n U.g0 /a and V 2 U.g0 /. It remains to see ŒW; Yd  2 U.g/a . Indeed, let us show ŒW; Yd  2 U.g/a 0 . First, Yd 2 U.g;  0 / and replacing g by g0 in Proposition 12.1.7, we know that ŒYd ; U.g;  0 / \ U.g0 /  U.g0 /a 0 : Therefore,   ŒW; Yd  D ŒXq U C V; Yd  2 U.g/a 0 C U.g0 / \ U.g;  0 / D U.g/a 0 C .U.g;  0 / \ U.g0 //; and from this ŒŒW; Yd ; Yd  2 U.g/a 0 . Now, by Lemma 12.2.23, there exist m > 0 and Qj 2 UC .g;  0 /; 0  j  m; such that Qm 62 U.g/a 0 and m X

j

Qj Yd 0 mod U.g/a 0

j D0

holds. Among these equalities, we choose one where m becomes minimal. The adjoint action of W is written as 0 @

m X

1 j 1 A

jQj Yd

ŒW; Yd  0 mod U.g/a 0 :

j D1

P  j 1 m Here, 6 0 modulo U.g/a 0 . This comes from the minimality of jQ Y j j D1 d m when m > 1 and the fact that Q1 6 0 modulo U.g/a 0 when m D 1. Because the ring UC .g;  0 /=U.g/a 0 is an integral domain, we get ŒW; Yd  2 U.g/a 0 and finish the proof of this case. Case 3.

At a general ` 2  , dim.h.`// D dim.h.`0 //  1 and h0 .`/ D h0 .`0 /.

420

12 Commutativity Conjecture: Induction Case

Let us remark that the centre z of g is contained in g0 if dim.h.`// D dim.h.`0 // 1 at a certain point `. The assumption h0 .`/ D h0 .`0 / will be utilized only in situation (3c)(i). Let us show the inclusion U.g; /  U.g0 / C U.g/a . For this aim, it is enough by assertion (iii) of Proposition 12.1.8 to show that, if W D Xq U C V 2 U.g; / using U 2 U.g; / \ U.g0 / and V 2 U.g0 /, then inevitably U 2 U.g/a . According to z and zQ D z \ h \ kerf , let us divide the case into three subcases. (3a)

zQ ¤ f0g.

This case is easily settled by applying the induction hypothesis to .g=Qz; h=Qz/. (3b)

zQ D f0g and dim z  2.

In this case, dim.h \ z/ D 1 or h \ z D f0g. Because these two possibilities are treated in the same way, we treat here only the second possibility. So, let us assume h\z D f0g. Let us recall the elements fXr g1rq and fYs g1sd introduced, relating to the composition series (9.1.1), just before Lemma 9.2.14 in Chap. 9. We suppose that X1 ; X2 belong to z, and put hO D k2 . For each .q  3/-tuple J D jq1 j j .j3 ; j4 ; : : : ; jq1 / 2 Nq3 , set X J D Xq1   X4 4 X3 3 . Let us consider the vector subspace S1 of U.g0 / spanned by the family .X J YO K /J 2Nq3 ; K2Nd and the vector subspace S1 of U.g0 /a spanned by .X J YO K /J 2Nq3 ; K2Nd ; jKj>0 . In this proof, we denote by  ; Xq g fX1 ; X2 ; Y1 ; : : : ; Yd ; X3 ; : : : ; Xq1

the basis of g dual to the basis fX1 ; X2 ; Y1 ; : : : ; Yd ; X3 ; : : : ; Xq1 ; Xq g of g. O h/ O D f0g at arbitrary fO 2  , we set O D indG  O with HO D Because fO.Œh; HO f O Then, the family exp h. 

 j  k p p X J YO K X1 C 1fO.X1 / X2 C 1fO.X2 / I J 2 Nq3 ; K 2 Nd ; j; k 2 N; jKj C j C k > 0

o

makes a basis of U.g0 /aO . In particular, any element U of U.g0 /aO is uniquely written as U D

X j;k

.j;k/

U

 j  k p p X1 C 1fO.X1 / X2 C 1fO.X2 / ;

(12.3.1)

12.3 Proof of the Commutativity Conjecture

421

where U 2 S1 if j C k ¤ 0 and U 2 S1 . Now, W 2 U.g; / O and U 2 U.g; / O \ U.g0 /. There exists a non-empty Zariski open set O0 of  , whose elements ` verify that dim.h.`// and dim.h0 .`0 // are O f by fO and  by O . Then, we recognize minimal. Take fO in O0 and replace h by h, that the condition of case (3a) is satisfied,p and we deduce U 2 U.g0 /aO . O Now fix f in O0 and put XO j D Xj C 1fO.Xj / .j D 1; 2/. Replacing U by U in the expression (12.3.1), .j;k/

.0;0/

U D

X

j U .j;k/ XO 1 XO2k ;

(12.3.2)

j;k

where U .j;k/ 2 S1 if j C k ¤ 0 and U .0;0/ 2 S1 . As is easily seen, there exists a non-empty Zariski open set O of R2 , whose elements .u; v/ satisfy fOu;v D fO C uX1 C vX2 2 O0 . Replacing U by U and fO by fOu;v in expression (12.3.1), we get U D

X

j  k  p p .j;k/ Uu;v XO 1 C 1u XO 2 C 1v ;

(12.3.3)

j;k .j;k/

.0;0/

where Uu;v 2 S1 if j C k ¤ 0 and Uu;v From the equation U D

X

2 S1 .

 j  k p p p p XO2 C 1v  1v U .j;k/ XO 1 C 1u  1u

j;k

and formula (12.3.3), .0;0/ Uu;v D

X p p . 1u/j . 1v/k U .j;k/ 2 S1  U.g0 /a ; 8 .u; v/ 2 O: j;k

From this relation, for all .j; k/ 2 N2 and J 2 Nq3 , the coefficient of X J in U .j;k/ vanishes and U .j;k/ 2 S1  U.g0 /a . In particular, U 2 U.g0 /a and the theorem holds in this case. Now, if we cut the composition series (9.1.1) at the stage g0 D gn1 which contains h, f0g D g0  g1      gn1 D g0 : Let pj W .g0 / ! gj .1  j  n  1/ be the restriction map. As before, we    define ej .`/ D dim G 0 pj .`/ ; e  .`/ D .e1 .`/; : : : ; en1 .`// at ` 2 .g0 / and   0    set E D fe .`/I ` 2 .g / g. Let e 2 E and agree that e0 D 0. We define the G 0 -invariant layer Ue D f` 2 .g0 / I e  .`/ D e  g, the set of the jump indices

422

12 Commutativity Conjecture: Induction Case

S 0 .e  / D f1  j  n  1I ej D ej1 C 1g and the set of the non-jump indices T 0 .e  / D f1  j  n  1I ej D ej1 g. We take the multi-index e  2 E  such that the layer Ue encounters 0 D f`0 2 .g0 / I `0 jh D f jh g in a non-empty Zariski  open set of 0 . Finally, we denote by T 0 .eH / the set of the indices is 2 I such that 0 0 hs D hs1 C h.` / at general elements ` 2 0 . Before we continue the study of case 3, let us notice that from our assumptions h D h0 C h.`0 / and h.`/ D h0 .`/ at general elements ` 2  . In other words,  id 2 T 0 .eH / and id 62 T .eH /. In fact, from our assumptions dim.h.`//  dim.h0 .`// D dim.h.`0 //  dim.h0 .`0 //  1 at general elements ` 2  , while dim.h.`//  dim.h0 .`// and dim.h.`0 //  dim.h0 .`0 // are either 0 or 1. Hence, dim.h.`//  dim.h0 .`// D 0 and dim.h.`0 //  dim.h0 .`0 // D 1 at general elements ` 2  . Besides, by Proposition 12.2.9, the  difference between T 0 .eH / and Tn1 .eH / is at most one element. Thus, in the present situation,  T 0 .eH / D Tn1 .eH / [ fid g:

(3c)

(12.3.4)

dim z D 1 and zQ D f0g.

Let z0 be the centre of g0 . Take Y 2 g2 ng1 and Z 2 znf0g such that z D g1 D RZ and denote by gQ the centralizer of g2 in g. In what follows we shall examine four cases. (i) g0 D gQ . This is equivalent to saying g2  z0 . In this case, f .ŒXq ; Y / ¤ f0g and h  gQ . From the assumption, 2 2 T 0 .e  / and 2 62 T .e/. By Proposition 12.2.10, the difference between T 0 .e  / and Tn1 .e/ is at most one element. Hence, in this case, T 0 .e  / D Tn1 .e/ [ f2g:

(12.3.5)

Our first objective is to show that R.Y / is algebraic on the centre CD .G=H / of D .G=H /. In other words, let us see that there exists a polynomial P of Y so that P .Y / D

m X

Pj Y j 2 U.g/a ;

j D0

where the coefficients Pj belong to UC .g; / and Pm 62 U.g/a . This result is easily seen when 2 2 I. In fact, in this case there is a real number a such that Y C aZ 2 h. Obviously Y C aZ C if .Y C aZ/ 2 a and Z 2 UC .g; /. This supplies a desired polynomial relation. Next assume 2 62 I. In this case, id ¤ 2 hence id > 2. By the  facts (12.3.4), (12.3.5), id 2 T 0 .eH /  T 0 .e  / and id 2 T .e/nT .eH / D U.e/. This is also directly proved as follows. dim.h.`0 //  dim.h0 .`0 // D 1 holds generally

12.3 Proof of the Commutativity Conjecture

423

on  . From this, Y .`/ 2 h.`0 / n h0 , namely there is an Y .`/ 2 gid .`0 / n gid 1 . Then, from id 62 T .eH /, `.ŒXq ; Y .`// ¤ 0. Therefore,   ` Xq ; `.ŒXq ; Y /Y .`/  `.ŒXq ; Y .`//Y D 0: We know from this that `.ŒXq ; Y /Y .`/  `.ŒXq ; Y .`//Y 2 gid .`/ n gid 1 and that id 2 T .e/. Let mk D id . From the preceding results, we get a subsequence .r /1rk of Corwin–Greenleaf  -central elements. As we  already saw, the family  .$ .˙.r ///1rk generates algebraically the subring ı Zmk .g; / of CD .G=H /. Then, we know also that r is at the same time a 0 -central element. Furthermore, Y is a 0 -central element. In this way, the family fY g [ .r /1rk makes a 0 subsequence of Corwin–Greenleaf 0 -central elements and, with  D indG H f , 0  $.Y / [ .$ .˙.r ///1rk generate algebraically the subring ı Zmk .g ;  / of CD  .G 0 =H /.  As mk 2 T 0 .eH /, applying the first assertion of Lemma 12.2.22 to the stage 0 of g ˙.k / is algebraic on the family fY g [ .˙.k //1rk1 modulo U.g0 /a . Consequently, we obtain a polynomial P so that P .˙.1 /; : : : ; ˙.k1 /; Y; ˙.k // 0 mod U.gid /a ;

(12.3.6)

where the coefficient of the maximal power of ˙.k / does not belong to U.g/a . Equation (12.3.6) is rewritten with certain polynomials Pj D Pj .˙.1 /; : : : ; ˙.k // ; 0  j  m; as m X

Pj .˙.1 /; : : : ; ˙.k // Y j 0

(12.3.7)

j D0

modulo U.gid /a . These Pj are elements of UC .g; /. Since mk 2 T .e/nT .eH /, from Theorem 12.2.5 ˙.k / is algebraically independent of the family .˙.r //1rk1 modulo U.g/a . If so, we may assume without loss of generality that, in relation (12.3.7), Pm does not belong to U.g/a and m  1. Hence (12.3.7) is a non-trivial relation. Henceforth, we choose the polynomial P in such a fashion that the degree m  1 in (12.3.7) will be minimal. As ŒW; Y  D ZU D UZ, applying the adjoint action of W to formula (12.3.7), we obtain

424

12 Commutativity Conjecture: Induction Case

0 @

m X

1 jPj Y j 1 A UZ 0 mod U.g/a :

j D1

P  m j 1 Then, it turns out that jP Y 6 0 modulo U.g/a . This follows from j j D1 the minimality of m if m > 1 and, if m D 1 from the fact that P1 6 0 modulo U.g/a . Besides, clearly Z 62 U.g/a . Since the ring U.g; /=U.g/a has no nontrivial zero divisor, U 2 U.g/a and we finish the proof of this case. (ii) At general elements ` 2  , g0 ¤ gQ ; h  gQ and dim.h.`// D dim.h \ gQ ` /. First, we can choose Xq 2 gQ and X 2 g0 so that g0 D .g0 \ gQ / ˚ RX and gQ D .g0 \ gQ / ˚ RXq : Since g0 \ gQ has the codimension 2 in g, the value of    `  `      dim h \ g` D dim h \ g0 \ gQ  dim.h.`// dim h \ g0 \ gQ might be 0; 1 or 2. From the assumptions,   dim.h.`// D dim.h.`0 //  1 and dim.h.`// D dim h \ gQ ` at general elements ` 2  . By the way, let us pay attention also to   `   dim.h.`// dim h \ g0 \ gQ    `  dim.h.`0 // C dim.h.`0 //  dim.h.`// D dim h \ g0 \ gQ       `   dim h \ gQ ` C dim h \ gQ `  dim.h.`//: D dim h \ g0 \ gQ We know from this that   `  dim h \ g0 \ gQ  dim.h.`// D 1: Take W D Xq U C V as before. Let us see that W 62 U.g0 / C U.g/a means a Q contradiction. In fact, putting GQ D exp gQ and Q D indG H f , this condition implies Q with UQ 2 .U.Qg; / Q \ U.g0 //n the existence of an element WQ D Xq UQ C VQ 2 U.Qg; / 0 0 U.Qg \ g /a and VQ 2 U.Qg \ g /. However, since     `   dim h \ gQ ` D 1; dim h \ g0 \ gQ this contradicts the induction hypothesis concerning n D dim G.

12.3 Proof of the Commutativity Conjecture

425

For b 2 N, we define the subspace Sb of U.g0 / by Sb D b  1 and by S0 D f0g. It is easy to rewrite W as W D

a X i D0

X i Xq Ui C

b X i D0

X i Vi D

a X

Pb1 i D0

X i U.g0 \ gQ / if

X i X q U i C X b Vb C W b

(12.3.8)

i D0

with some integers a; b. Here Ui ; Vi 2 U.g0 \ gQ / and Wb 2 Sb . Without loss of generality, we may choose W in such a manner that Ui ; Vi 2 U.g0 \ gQ /nU.g0 \ gQ /a and b  a. In fact, assume b > a. In the first assertion of Proposition 12.1.8, if we replace g byg0 , g0 by g0 \gQ and Xq by X , it results that Vb 2 .U.Qg; / Q \ U.g0 //. Next,

taking dim h \ .g0 \ gQ /` D dim.h.`0 // into account, the induction hypothesis Q \ U.g0 // n U.Qg \ g0 /a gives an element XA C B 2 U.g0 ;   / using A 2 .U.Qg; / 0 and B 2 U.Qg \ g /. Then, 0

W 0 D WAb  .XA C B/b Vb D X a Xq Ua Ab C X b Vb 0 C Wb 0 is an element of U.g; / n .U.g0 / C U.g/a / and it turns out that b 0 < b; Vb 0 2 U.g0 \ gQ / and Wb 0 2 Sb 0 . In consequence, by repeating this process we may assume in (12.3.8) that b  a. Provided b D a (resp. b < a), applying again Proposition 12.1.8, we see that WQ D Xq Ua C Va 2 U.Qg; / Q (resp. WQ D Xq Ua 2 0 U.Qg; /) Q using Ua 62 U.g \ gQ /a . This contradicts the fact that    `   1 dim h \ gQ ` D dim h \ g0 \ gQ and the induction hypothesis.

  (iii) At general elements ` 2  , g0 ¤ gQ ; h  gQ and dim.h.`// D dim h \ gQ `  1. From the assumption h`  g0 ; h`  gQ and hence h`  g0 \ gQ . By the same reason, this means   `   dim.h.`0 // D 1 dim h \ g0 \ gQ and    `    dim h \ gQ ` D 1: dim h \ g0 \ gQ Take W D Xq U CV as before. Let us see that U 62 U.g/a gives a contradiction. P i We know U 2 U.g0 ;   /. Using Ui 2 U.g0 \ gQ /, we write U in the form m i D0 X Ui . 0 0 0 Replacing g by g and g by g \ gQ , it results that   `  dim h \ g0 \ gQ  dim.h.`0 // D 1

426

12 Commutativity Conjecture: Induction Case

at general elements ` 2   . Taking this into consideration and applying the induction hypothesis concerning dim g, we see that U.g0 ;   /  U.g0 \ gQ / C U.g0 /a : Finally, Ui 2 U.g/a for i ¤ 0. Thus, we may assume U D U0 2 U.g0 \ gQ /. Likewise, we may assume without loss of generality that, using Vi 2 U.g0 \ Pm i gQ / n U.g/a , V is written in the form V D i D0 X Vi . Here if m  1, then we are led to a contradiction. In fact, under this assumption, the first assertion of Proposition 12.1.8 gives   mX Vm C Vm1 2 U.g0 ;   / n U.g0 \ gQ / C U.g0 /a ; but this is impossible as we saw above. In consequence, we may choose W D Xq U0 C V0 using U0 ; V0 2 U.g0 \ gQ / with U0 62 U.g0 \ gQ /a . Finally, making use of the fact that   `    dim h \ g0 \ gQ  dim h \ gQ ` D 1 at general elements ` 2  , it results by induction that U.Qg; / Q  U.g0 \ gQ / C U.Qg/a : This is absurd and the theorem has been settled in this case. (iv) h 6 gQ . Q In this case, we set hQ D h\ gQ ; HQ D exp hQ and choose X 2 h so that h D h˚RX . ` Obviously, g D gQ C RX and h  .RX / at any ` 2  . From this,   dim.h.`// D dim.h.`0 //  1; dim.h.`// D dim h \ gQ ` at general elements ` 2  . If so, by arguments analogous to those developed in case (3)(ii) above it results that   `   dim.h.`// D 1 dim h \ g0 \ gQ at general elements ` 2  . g being any ideal of g containing Y , we see that h \ g` D hQ \ g` at general elements ` 2  . So,     `  dim hQ \ g0 \ gQ  dim hQ \ gQ ` D 1 generally in  .

(12.3.9)

12.3 Proof of the Commutativity Conjecture

427

Let us see that assuming W 62 U.g0 / C U.g/a we are P led to a contradiction. P Indeed, using Ui ; Vi 2 U.g0 \ gQ /, we can write W D Xq . i Ui X i / C i Vi X i in this case and W Xq

i i X p X p  1f .X / Ui C  1f .X / Vi mod U.g/a : i

i

Therefore, we may assume without loss of generality that W D Xq U C V with Q Q Q D indG  ; and W 62 U.g0 \ gQ / C U.Qg/aQ . U; V 2 U.g0 \ gQ /. Then, W 2 U.Qg; /; HQ f Q and using the induction hypothesis Replacing g by gQ , g0 by g0 \ gQ and h by h, concerning dim g, we see that this is incompatible with (12.3.9). Thus the proof of the theorem is complete.  Now we arrive at the commutativity conjecture. Corollary 12.3.3 ([37]). Let G D exp g be a nilpotent Lie group with Lie algebra g, f 2 g and h 2 S.f; g/. We keep the previous notations. The algebra D .G=H / is commutative if and only if  D indG H f has finite multiplicities. Proof. Corwin and Greenleaf [17] already showed one direction: if  has finite multiplicities, the algebra D .G=H / is commutative. So, it remains to show the inverse direction. Assuming that  has infinite multiplicities, let us show by induction on dim g that D .G=H / is non-commutative. Let us first recall the following [14]: generally on  ,  has finite multiplicities ” dim.H  `/ D

1 dim.G  `/ 2

” 2.dim h  dim.h.`/// D dim g  dim.g.`//: Therefore, it suffices to show that D .G=H / is non-commutative if 2.dim h  dim.h.`/// < dim g  dim.g.`// at general elements ` 2  . In this situation, clearly h ¤ g. Let g0 be an ideal of codimension 1 in g containing h. If D  .G 0 =H /  D .G=H / is already non-commutative, there is nothing to do. Hence, suppose that D  .G 0 =H / is commutative. This is the same as saying that 2.dim h  dim.h.`0 /// D dim.g0 /  dim.g0 .`0 // at general elements ` 2  . Hence, it results that 2.dim.h.`0 //  dim.h.`/// < 1 C dim.g0 .`0 //  dim.g.`//  2 and hence h.`0 / D h.`/ at general elements ` 2  . If so, by Theorem 12.3.2, there exists an element W 2 U.g; / such that W 62 U.g0 / C U.g/a . In consequence,

428

12 Commutativity Conjecture: Induction Case

by Theorem 12.1.1, there exists an element T 2 U.g0 ;   / so that ŒW; T  62 U.g/a . Thus we can conclude that the algebra D .G=H / is non-commutative.  Example 12.3.4. Let g be the nilpotent Lie algebra of dimension 7 with a basis fXi I 1  i  7g which satisfies the following nonzero commutation relations: ŒX6 ; X2  D X1 ; ŒX6 ; X4  D X2 ; ŒX6 ; X5  D X4 ; ŒX6 ; X7  D X3 ; ŒX4 ; X5  D X3 ; ŒX5 ; X3  D X1 ; ŒX4 ; X7  D X1 : The centre of g is clearly z D RX1 . We choose the composition series of ideals (9.1.1) as follows: f0g D g0  g1      g6  g7 D g; where gj D hX1 ; : : : ; Xj iR D

j X

RXk .1  j  7/:

kD1

We consider in g the basis fXj I 1  j  7g dual to the basis fXj I 1  j  7g of g and put f D X4 . Taking h D RX4 ,  D

8 7 <X :

j D1

9 =

j Xj I 4 D  : ;

Let us describe general H -orbits in  , namely H -orbits of maximal dimension. They are contained in the Zariski open set OD

8 7 <X :

j D1

9 =

j Xj 2  I 3 ¤ 0 : ;

By a simple direct computation, if `D

7 X

j Xj 2  ;

j D1



Ad .exp.tX4 // .`/ D

7 X j D1

j .t/Xj .8 t 2 R/;

12.3 Proof of the Commutativity Conjecture

429

where

1 .t/ D 1 ; 2 .t/ D 2 ; 3 .t/ D 3 ; 4 .t/ D ;

5 .t/ D 5 C t 3 ; 6 .t/ D 6  t 2 ; 7 .t/ D 7  t 1 :

(12.3.10)

The index sets I; J defined just after Theorem 9.2.1 become respectively I D f4g and J D f1; 2; 3; 5; 6; 7g. Besides, the sequence of Lie subalgebras (12.2.3) becomes k0 D h D RX4 ; k1 D RX1 ˚ RX4 ; k2 D RX1 ˚ RX2 ˚ RX4 ; kj D gj C1 .3  j  6/: K

Setting j D indHj f ; Kj D exp.kj /, we consider the associated sequence of subalgebras: D1 .K1 =H /  D2 .K2 =H /      D5 .K5 =H /  D6 .K6 =H / D D .G=H /: (12.3.11) Theorem 12.3.2 tells us which of these inclusions are proper and which are equalities. Using the computations (12.3.10) of H -orbits and putting `j D `jkj .1  j  6/ for any ` 2 g , dim.H  `j / D 0; 8 ` 2 O; 0  j  3; dim.H  `j / D 1; 8 ` 2 O; 4  j  6:

(12.3.12)

According to the sequence k0  k1      k6 , the jump of the dimensions of general H -orbits in  happens at only one place, the passage from k3 to k4 . Hence, by Theorem 12.3.2, Dj .Kj =H / is properly contained in Dj C1 .Kj C1 =H / except j D 3 and the equality D3 .K3 =H / D D4 .K4 =H / holds when j D 3. Thus, if we describe (12.3.11) in detail, D1 .K1 =H / ¨ D2 .K2 =H / ¨ D3 .K3 =H / D D4 .K4 =H / ¨ D5 .K5 =H / ¨ D .G=H /:

(12.3.13)

In other words, except j D 3 there exist nonzero elements of Dj C1 .Kj C1 =H / not belonging to Dj .Kj =H /. To confirm it, let us construct concretely these new comers. In the computations (12.3.10), if we put uPD 5 C t 3 , we can parameterize general H -orbits in  as follows: provided ` D 7j D1 j Xj 2 O, Ad .exp.tX4 // .`/ D

7 X j D1

rj .u/Xj .8 t 2 R/;

430

12 Commutativity Conjecture: Induction Case

where r1 .u/ D 1 ; r2 .u/ D 2 ; r3 .u/ D 3 ; r4 .u/ D ; r5 .u/ D u; r6 .u/ D r7 .u/ D

6 3 C 2 5  u 2 ;

3

7 3 C 1 5  u 1 :

3

This gives H -invariant polynomials

1 ; 2 ; 3 ; 6 3 C 2 5 ; 7 3 C 1 5 on g . Applying to these polynomials the symmetrization map S.g/ ! U.g/, we procure elements X6 X3 C X2 X5 ; X1 ; X2 ; X3 ; X7 X3 C X1 X5 of U.g; /. Precisely, we get X1 at the passage from C D U.k0 ; f / to U.k1 ; 1 /, X2 from U.k1 ; 1 / to U.k2 ; 2 /, X3 from U.k2 ; 2 / to U.k3 ; 3 /, X6 X3 C X2 X5 from U.k4 ; 4 / to U.k5 ; 5 / and X7 X3 C X1 X5 from U.k5 ; 5 / to U.g; /. Let us confirm that no newcomer appears at the passage from D3 .K3 =H / to D4 .K4 =H /. Assume that there exists A 2 U.k4 ; 4 / not belonging to U.k3 / C U.g/a . From assertion (iii) of Proposition 12.1.8, we may assume that A D X5 U C V with U; V 2 U.k3 /. Since k3 is commutative, ŒA; X4  D X3 U . By X3 62 U.g/a , it results that U 2 U.g/a and V 2 U.k3 ; 3 /. Thus, A 2 U.k3 ; 3 / C U.g/a and hence D3 .K3 =H / D D4 .K4 =H /. Now let us turn toward the question of the commutativity of D .G=H /. By a direct calculation, general G-orbits in  are of dimension 6 and general H -orbits are of dimension 1 by (12.3.12). Hence  D indG H f has infinite multiplicities. Therefore, the algebra D .G=H / should be non-commutative. Indeed, the groups Kj D exp.kj / are commutative for 1  j  3. Hence, the representation j D K indHj fj has finite multiplicities and the algebra Dj .Kj =H / is commutative. Let j D 5 and consider a general element ` 2  . The orbit K5 `5 is of dimension 5 4 and the orbit H  `5 is of dimension 1 by (12.3.12). Therefore, 5 D indK H f5 has infinite multiplicities. At the same time, X2 and X6 X3 C X2 X5 are two elements of U.k5 ; 5 / and, since ŒX2 ; X6 X3 C X2 X5  D X1 X3 ; X1 X3 62 U.g/a ; ŒX2 ; X6 X3 C X2 X5  6 0 mod U.g/a : In this way, we confirm that D5 .K5 =H / is non-commutative.

Chapter 13

Commutativity Conjecture: Restriction Case

13.1 Kernel of Representation As Frobenius reciprocity suggests, there is a kind of duality between the induction and the restriction of representations. Under this guiding principle, let us formulate and prove for the restriction of representations the counterpart of the commutativity conjecture. These results were obtained by [6]. As usual, let G D exp g be a connected and simply connected nilpotent Lie group with Lie algebra g and K D O ˝./ D ˝G ./ exp k an analytic subgroup of G with Lie algebra k. For  2 G, O denotes the corresponding coadjoint orbit of G. When  2 G is given, we examine the restriction jK of  to K. First of all, its canonical irreducible decomposition is described by Theorem 8.2.4 in Chap. 8. We denote by ker D fX 2 U.g/I .X / D 0g the primitive ideal of U.g/ associated with . Besides, for two subspaces M and N of U.g/, we denote the centralizer of M modulo N by c.M; N / D fA 2 U.g/I ŒA; M  N g: Let us set U .g/k D c.k; ker/. Clearly, ker is a two-sided ideal of U .g/k . When we define the algebra D .G/K as the image of U .g/k by the homomorphism , D .G/K Š U .g/k =ker D .U.g/=ker/K ; where the last algebra is the algebra composed of all K-invariant elements of U.g/=ker. The algebra D .G/K keeps H1 stable and is regarded as the algebra composed of all differential operators on it which are commutative with the action of K. We would like to understand the structure of the algebra D .G/K in connection with the irreducible decomposition of jK . To say more, when the © Springer Japan 2015 H. Fujiwara, J. Ludwig, Harmonic Analysis on Exponential Solvable Lie Groups, Springer Monographs in Mathematics, DOI 10.1007/978-4-431-55288-8__13

431

432

13 Commutativity Conjecture: Restriction Case

representation jK is irreducible, D .G/K is the trivial algebra composed of scalar operators. Exactly as in the case of monomial representations, there occurs the following alternative: in Theorem 8.2.4 of Chap. 8, the multiplicities n ./ are either uniformly bounded almost everywhere with respect to or n ./ D 1 almost everywhere with respect to . According to these two eventualities, we say that jK has finite or infinite multiplicities. Let us take up once again the composition series of ideals (9:1:1): S W f0g D g0  g1      gn1  gn D g; dim.gk / D k .0  k  n/; (9:1:1) and recall the notations introduced at the beginning of Chap. 9 :Xj 2 gj ngj 1 ; pj W g ! gj ; E; e 2 E; Ue ; S.e/; T .e/ etc. If we set gS D

X

RXj and gT D

j 2S.e/

X

RXj ;

j 2T .e/

˚ then g D gS ˚ gT . In E there exists a total ordering E D e .1/ >    > e .k/ such that Ue.k/ and [j i Ue.j / for any i become Zariski open sets of g . Hence, each layer Ue is an algebraic set, namely a difference of two Zariski open sets of g . We mention here some results of Pedersen [58]. Let Ue be an arbitrary layer and S.e/ D fj1 <    < jm g, where m is the dimension of G-orbits in Ue . Applying at the same time Theorem 6.1.15 of Chap. 6, there exist functions Rje W Ue  Rm ! R; 1  j  n: • when we fix ` 2 Ue , x D .x1 ; : : : ; xm / 7! Rje .`; x/ is a polynomial map of x 2 Rm and their coefficients are G-invariant functions on Ue ; • Rje .`; x/ D xk for j D jk 2 S.e/; ` 2 Ue ; • when jk  j  jkC1 , Rje .`; x/ depends only on x1 ; : : : ; xk ; • the coadjoint orbit G  ` is given by G`D

8 n <X :

Rje .`; x/Xj I x 2 Rm

j D1

9 = ;

for any ` 2 Ue . We replace the variables xk by iXjk in Rje .`; x/, form Rje .`; iXj1 ; : : : ; iXjm / in the symmetric algebra S.g/ and write its image in U.g/ by the symmetrization map as rje .`/. In particular, rjek .`/ D iXjk . Let Ve be the subspace of S.g/ spanned by the elements of the form Xjk11   Xjkmm ; k1 ; : : : ; km 2 N;

13.1 Kernel of Representation

433

Fe the image of Ve in U.g/ by the symmetrization map and Ee the subspace of U.g/ spanned by the elements of the form Xjk11   Xjkmm ; k1 ; : : : ; km 2 N: When e D ;, put Ve D Fe D Ee D C1. Pedersen [58] showed that the primitive ideal ker` , ` 2 Ue , is generated by uej .`/ D Xj  i rje .`/; j 2 T .e/ and that U.g/ D ker` ˚ Ee D ker` ˚ Fe : Here,  is faithful on Ee and on Fe . In this way, Ee and Fe are identified with U.g/=ker` . Abusing a little the notations, D` .G/K Š EeK Š FeK Š CŒXj1 ; : : : ; Xjm K : Of course, these isomorphisms are simply as vector spaces. Removing the dependence on ` from these results of Pedersen on ker` , Corwin and Greenleaf [17] constructed their e-central elements. These e-central elements are useful to construct elements of U .g/k and play a principal role to prove the commutativity. O We further We fix the coadjoint orbit ˝ D ˝./  g corresponding to  2 G. fix an element ` 2 ˝ and introduce the coordinates into ˝ by means of a Malcev basis fXQ k I 1  k  qg, q D dim.g=g.`// D dim ˝ relative to g.`/. That is to say, g.`/ C

r X

RXQ k ; 1  r  q;

kD1

are Lie subalgebras of g. The mapping Rq 3 T D .t1 ; : : : ; tq / 7! ˚.T / D exp.tq XQq /    exp.t1 XQ 1 /  ` 2 ˝ is a diffeomorphism and through this mapping the measure Q  turns out to be equivalent to the Lebesgue measure on Rq . Let f0g D k0  k1  k2      kd 1  kd D k; dim.ks / D s; 0  s  d; be a Jordan–Hölder sequence of k and fY1 ; : : : ; Yd g a basis of k adapted to this sequence. We define a matrix Mr ; 1  r  d; of type .r; d / by    Mr D ˚.T / ŒYi ; Yj  1i r; 1j d

434

13 Commutativity Conjecture: Restriction Case

and indicate the rank of Mr by er .˚.T /jk /. The set e .˚.T /jk / D .e1 .˚.T /jk / ; : : : ; ed .˚.T /jk // is nothing but the set of dimension indices of ˚.T /jk 2 k given from the above 0 0 Jordan–Hölder  sequence. For 1  i qd , put ei D maxT02Rq ei .˚.T /jk / and e D  0 0 e1 ; : : : ; ed . Evidently, D D fT 2 R I e .˚.T /jk / D e g is a Zariski open set of Rq . Hence, if we introduce a layer Uk ./ D Ue0 D f 2 k I e./ D e 0 g in k , it results that   KO n OK .Uk .// D 0: For A 2 S.k/, we define the principal closed set F .A/ of k by F .A/ D f 2 k I A./ D 0g. Then, FA D fT 2 Rq I ˚.T /jk 2 F .A/g becomes a Zariski closed set of Rq . Owing to the construction of e-central elements (cf. [17]), there is a Zariski open set Z of k such that Z \Ue0 is a non-empty K-invariant set where this construction is practised. Let pk be the restriction map g ! k . If Z \ pk .˝/ D ;, the construction is practised replacing Ue0 by the sub-layer Ue0 n .Z \ Ue0 /, and it continues likewise. In this fashion this process supplies Corwin–Greenleaf e 0 O central elements effectively -almost everywhere in K. Let 

I D fi1 < i2 <    < id g be the set of indices 1  i  n such that gi \ k ¤ gi 1 \ k in the composition series of ideals (9:1:1) and set J D fj1 < j2 <    < jp g D f1; 2; : : : ; ng n I .p D dim.g=k//: Setting l0 D k and lr D k C gjr for 1  r  p, we obtain the sequence k D l0  l1      lp1  lp D g; dim.lr =lr1 / D 1; of Lie subalgebras of g. On the other hand, considering ks D k \ gis .1  s  d /, we get a composition series f0g D k0  k1      kd 1  kd D k; dim.ks / D s; 0  s  d; of ideals in k. Extracting vectors Ys 2 ks n ks1 for 1  s  d we make a Jordan– Hölder basis fY1 ; : : : ; Yd g of k. Likewise, extracting vectors Xr 2 lr n lr1 for 1  r  p, we make a Malcev basis fX1 ; : : : ; Xp g of g relative to k. Let us begin with extending the result of Pedersen mentioned above.

13.1 Kernel of Representation

435

Theorem 13.1.1. We suppose that g.`/ \ kk ¤ g.`/ \ kk1 at almost all ` 2 ˝ relative to Q  . Then, using Pj 2 U.kk1 / .0  j  m/ with .Pm / ¤ 0, there P j exists an element W 2 U.kk / \ ker  of the form W D m j D0 Pj Yk . Proof. We put dj D max dim.Kj  `/ `2˝

with Kj D exp.kj /; 1  j  d; and O D f` 2 ˝I dim.Kk `/ D dk ; dim.Kk1 `/ D dk1 g: Clearly, O is a non-empty Zariski open set of ˝, while, as is easily seen, g.`/\kk ¤ g.`/ \ kk1 when and only when dim.Kk `/ D dim.Kk1 `/. Now, if we put Z D f` 2 ˝I g.`/ \ kk ¤ g.`/ \ kk1 g; Z contains a non-empty Zariski open set of ˝ provided Z \ O ¤ ;, otherwise 0 D exp .kk1 C RX1 / and considering the Q  .Z/ D 0. Similarly, letting Kk1 0 maximal values of dim.Kk1 `/ and dim.Kk1 `/ when ` moves in ˝, we see that the set V D f` 2 ˝I X1 2 kk1 C g.`/g contains a non-empty Zariski open set of ˝, unless Q  .V/ D 0. Let us proceed by induction on dim g C dim.g=k/. When dim.g=k/ D 0, the assertion is nothing but the well-known result of N. Pedersen. As above, we take a Jordan–Hölder sequence of g passing the centre z and put l D l1 ; L D exp l. Since dim.g=l/ D dim.g=k/1, our assertion is established for l. Besides, we may assume that k contains z and further, if a D z \ ker ` ¤ f0g at ` 2 ˝, we may descend to the quotient Lie algebra gQ D g=a and apply the induction hypothesis. Thus, as usual we are brought to the case where dim z D 1; `jz ¤ 0. We assume g.`/\kk ¤ g.`/\kk1 at almost all ` 2 ˝ relative to the measure Q  . From the induction hypothesis applied to l, there exists W 0 2 U.l \ gik / \ ker  which is written as W0 D

m X

aj0 Yk ; j

(13.1.1)

j D0 0 where aj0 2 U.l \ kik 1 / .0  j  m/ with .am / ¤ 0. Hence, if ik < j1 , the result follows immediately. Assume j1 < ik from now on. By means of aj 2 U.kk / .0  j  m0 / we rewrite (13.1.1) as 0

0

W D

m X j D0

j

aj X1

(13.1.2)

436

13 Commutativity Conjecture: Restriction Case

and choose this element W 0 in such a manner that m0 might be minimal. When j1 is a non-jump index in the dimension indices e.`/ D .e1 .`/; : : : ; en .`// of ` 2 ˝ relative to the sequence of ideals S, there exists WO D X1 C QO 2 ker  due to Pedersen [58]. Here, QO 2 U.gj1 1 /. Substituting this expression of X1 into formula (13.1.2) of W 0 and eliminating X1 , we obtain the desired result. In what follows, we assume that j1 is a jump-index in e.`/; ` 2 ˝. Let pl W g ! l be the canonical projection and OL W l ! LO the Kirillov map of L. From the irreducible decomposition of the restriction jK , the property .W 0 / D 0 means .W 0 / D 0 for almost all  2 LO with respect to the measure L D .OL ıpl / . Q  /. Let us examine separately two cases mentioned above. First consider the case where V contains a non-empty Zariski open set of ˝. Since X1 2 kk1 C g.`/ at a general ` 2 ˝, we take in ker  another element 00

00

W D

n X

bj0 Ysj ;

(13.1.3)

j D0

where bj0 2 U.l \ gis 1 / .0  j  n00 /; .bn0 00 / ¤ 0. Besides, j1 < is  ik1 and X1 2 ks C g.`/ almost everywhere relative to Q  . We rewrite the expression (13.1.3) as 0

W 00 D

n X

j

X 1 bj ;

(13.1.4)

j D0

where bj 2 U.ks / .0  j  n0 /. Let us remark that X1 appears effectively in this expression (13.1.4), in other words, .bn0 / ¤ 0. In fact, if all the bj0 2 U.ks / .0  j  n00 / belong to U.ks1 /, then .jK /.W 00 / D .jKs / .W 00 / D 0, while,  for almost all irreducible representations  of Ks relative to K , their coadjoint s ? orbits are saturated in the direction of .ks1 / , or there appear in the irreducible decomposition of jKs a one-parameter family of irreducible representations which are mutually inequivalent but whose restrictions to Ks1 are all to jKs1 .   equivalent This means as in the second case mentioned below that  bn0 00 D 0, but this is absurd. Here again we choose W 00 given by formula (13.1.3) so that n0 in expression (13.1.4) will be minimal. In order to get the desired result, let us eliminate X1 from formulas (13.1.2) and (13.1.4). Combining (13.1.2) with (13.1.4), we can write 00

am0 W D W

0

0 0 X1n m bn0

C

0 1 nX

j D0

j X1 cj0

DW

0

0 0 bn0 X1n m

C

0 1 nX

j D0

j

X1 cj

13.1 Kernel of Representation

437

when m0  n0 . Here cj0 ; cj 2 U.kk1 / .0  j  n0  1/. Next, putting W 000 D Pn0 1 j j D0 X1 cj and repeating this manipulation, 0

0

2 00 am D am0 W 0 bn0 X1n m C am0 W 000 0W

D am0 W

0

0 0 bn0 X1n m

CW

0

0 0 cn0 1 X1n m 1

C

0 2 nX

j

X 1 dj ;

j D0

where dj 2 U.kk1 / .0  j  n0  2/. If we continue likewise, we get 0

0

0

0

am0 W 00 D am0 W 0 bn0 X1n m C am0 W 0 cn0 1 X1n m 1 C    C WQ p

p1

p2

(13.1.5)

P j for some p 2 N. Here, WQ D rj D0 X1 bQj 2 ker  and r < m0 ; bQj 2 U.kk / .0  j  r/. If WQ contains effectively Yk , this contradicts the minimality of m0 . Hence WQ does not contain Yk effectively. Moreover, it does not happen that all the bQj .0  j  r/ are contained in ker . Otherwise, if we write 0

00

W D

n X

bj00 X1

j

  bj00 2 U.ks /; 0  j  n0 ;

j D0

we see that the constant term am0 b000 of Eq. (13.1.5) belongs to ker . Thus, b000 2 ker  and W 00  b000 becomes divisible by X1 , what contradicts the choice of W 00 . From these observations, we may assume m0 > n0 in the following arguments. Assume m0 > 0. From the expressions (13.1.2) and (13.1.4), we obtain p

0

0

W 0 bn0 D am0 X1m n W 00 C

0 1 m X

aj00 X1 ; j

j D0

where aj00 2 U.kk / .0  j  m0  1/. In the expression of W 0 bn0 , there is a power of Yk with degree larger than or equal to 1 and whose coefficient does not belong to P 0 1 00 j aj X1 contains effectively Yk ker . Since W 00 2 ker , it follows that WQ D jmD0 but this contradicts the choice of W 0 . Next, let us examine the case where Q  .V/ D 0. Let us assume m0 > 0 and show .am0 / D 0. Indeed, either the corresponding coadjoint orbits ! D .OL /1 ./ of L are saturated in the direction of k? for almost all  2 LO relative to L D .OL ıpl / . Q  / or there appear in the irreducible decomposition of jL a oneparameter family of irreducible representations which are mutually inequivalent but whose restrictions to Kk are all equivalent to jKk . When !  l is saturated in the direction of k? ,  is realized by means of a polarization contained in k and .X1 / is nothing but the partial differential operator with respect to the first coordinate introduced along X1 . From this, we know that .am0 / D 0 in this situation.

438

13 Commutativity Conjecture: Restriction Case

Henceforth, we suppose that !  l is non-saturated in the direction of k? . Then, decomposition of jL , the contribution of the subset  in the irreducible  OL ! C .kk /?  LO is not negligible. We find therein a one-parameter family of irreducible representations ` constructed at ` 2 ! C .kk /? . They admit the same e-central element D PN X1 C QN 2 U.l/ as those having the dimension indices e of L-orbits relative to the sequence of ideals f0g D k0  k1      kk  kk C RX1  kkC1 C RX1      l of l, and whose sub-layer ˝e encounters effectively the support of L . In more detail, PN ; QN 2 U.kk / and ` . / is a scalar operator whose value is given by p.` N 0 /`.X1 / C q.` N 0 /. Further, with `i D `.Yi / .1  i  k/, p; N qN are rational functions of `0 D .`1 ; : : : ; `k / and PN is also an e-central element such that ` .PN / D p.` N 0 /Id ¤ 0. Bringing these facts into expression (13.1.2) of W 0 2 ker  and dividing W 0 by

as above, we get 0

N m0

P

0

N m0

W DP

m X

m0

j X1 aO j

D aO m0 C

j D0

0 1 m X

j

X 1 bj ;

j D0

0 where aO j ; bj 2 U.kk / and aO m0 D am0 . Multiplying this equation by PN m 1 and repeating the process, we obtain

0 PN

2m0 1

W 0 D PN

m0 1

m0

@ am0 C

0 1 m X

1 X 1 bj A j

j D0

0 D PN

m0 1

m0

am0 C PN

m0 1

@

0 1 m X

1 X 1 bj A j

j D0 0

0

0

D PN m 1 m am0 C m 1 bm0 1 C

0 2 m X

j

X1 cj ;

j D0

where cj 2 U.kk / .0  j  m0  2/. Continuing this manipulation, there exists a certain r 2 N so that 0

Nr

0

P W D

m X

PN rj j hj ;

j D0

where hj 2 U.k/ .0  j  m0 /; hm0 D am0 and 0 D r0 < r1 <    < rm0 is a increasing sequence of integers. Taking .W 0 / D 0 into account, we procure

13.2 Commutativity of the Algebra D .G/K 0

m X

439

 j pN rj .`0 / p.` N 0 /`.X1 / C q.` N 0 / ` .hj / D 0:

j D0

Since this last expression is a polynomial of the variable `.X1 /, the coefficient of its maximal power is inevitably 0 and hence ` .hm0 / D ` .am0 / D 0 at almost all ` 2 l relative to L . In this way, .am0 / D 0 is deduced and we meet a contradiction. In consequence, we have shown that m0 D 0. 

13.2 Commutativity of the Algebra D .G /K Let us prove the first result concerning the commutativity of the algebra D .G/K . Theorem 13.2.1. Let G D exp g be a connected and simply connected nilpotent Lie group, K D exp k an analytic subgroup of G and  an irreducible unitary representation of G. If jK has finite multiplicities, the algebra D .G/K is commutative. Proof. Let us employ the induction on dim g. As is immediately seen, we may assume that k contains the centre z of g, and if a D z \ ker `; ` 2 ˝, is not trivial, we may descend to the quotient Lie algebra gQ D g=a and apply the induction hypothesis. Thus as usual we are brought to the case where dim z D 1 and the elements of ˝ do not vanish on z. Let fX; Y; Zg be a Heisenberg triplet such that z D RZ, `.Z/ D 1 for ` 2 ˝, ŒX; Y  D Z and g D RX C g0 with the centralizer g0 of Y in g. We put G 0 D exp.g0 / and consider the following two possible cases. The first case occurs when k  g0 . From the description of the irreducible decomposition of jK , if jK has finite multiplicities, almost all K-orbits in ˝ are saturated in the direction of .g0 /? . Hence, U .g/k  U.g0 / C ker . In fact, this is clear if Y 2 k. If Y 62 k, put l D k ˚ RY . From the saturation of orbits, l \ g.`/ ¤ k \ g.`/ at almost all ` 2 ˝, and by Theorem 13.1.1, there exists 0

W D

m X j D0

aj Ysj D

m X

Wr Y r 2 U.l/ \ ker 

rD0

for a certain s .1  s  d /. Here aj 2 U.l \ gis 1 / .0  j  m0 /, .am0 / ¤ 0 and Wr 2 U.ks / .0  r  m/. The Ks -coadjoint orbits in .ks / belong, almost everywhere relative to .pks / . Q  /, to a one-parameter family non-saturated in the direction of .ks1 /? and the corresponding irreducible unitary representations appear in the irreducible decomposition of jKs continuously in this parameter. From this, exactly as in the proof of Theorem 13.1.1, we see that W contains effectively Y . Namely, m > 0 and Wm 62 ker . Let us assume for a while that U .g/k 6 U.g0 / C ker . Then, there exists V D Xa C b 2 U .g/k with a; b 2 U.g0 / and a 62 ker . From this,

440

13 Commutativity Conjecture: Restriction Case

.adV /m .W / D mŠam Wm Z m 2 ker ; which contradicts the fact that  .U.g// has no non-trivial zero divisor. This last fact comes from Theorem 6.2.13 of Chap. 6 which says that the action of .A/ D d .A/; A 2 U.g/; on H1 , realized as S.Rk / for some k, is a linear differential operator with polynomial coefficients. 0 Let us realize as  D indG B ` using a polarization b  g at ` 2 ˝. Here B D exp b and ` is as usually the unitary character of B defined by ` .exp X / D e i `.X / ; X 2 b. We write the restriction of  to K as Z jK D .jG 0 / jK '

˚ R

t jK dt;

where t denotes the irreducible unitary representation of G 0 corresponding to .exp.tX /`/ jg0 : Since jK has finite multiplicities, the representation t jK too has finite multiplicities at almost all t 2 R, and hence Dt .G 0 /K is commutative by the induction hypothesis. On the other hand, it is easily seen that U .g/k D

\

Ut .g0 /k ;

t 2R

and D .G/K is commutative. The second case occurs when k 6 g0 . In this case, we choose X in k. If almost all K-orbits in ˝ are saturated in the direction of .g0 /? . The commutativity in question is shown exactly as in the first case. Therefore, it is enough to examine the case where almost all K-orbits in ˝ are non-saturated in the direction of .g0 /? . In this case, jK having finite multiplicities, almost all K-orbits in k relative to the measure .pk / . Q  / are non-saturated in the direction of .k \ g0 /? . From the preceding study on the Corwin–Greenleaf elements, there exists D aX C b 2 U.k/; a; b 2 U.k \ g0 /; such that . / is a scalar operator for almost all  2 KO relative to .OK ıpk / . Q  /, and a 62 ker  too has the same property. Hence, a; 2 U .g/k . Let A; B 2 U .g/k . When A; B belong to U.g0 /, considering k0 D .k\g0 /˚RY instead of k and reasoning exactly as in the first case, we see Œ.A/; .B/ D 0. We Pm Pm0 j j 0 write A D j D0 X Bj with Aj ; Bj 2 U.g /. Then, j D0 X Aj and B D Am ; Bm0 2 U .g/k . In order to show the commutativity in question, let us assume for instance m > 0 and employ a new induction on m C m0 . The degree of the element AQ D am A  m Am 2 U .g/k

13.2 Commutativity of the Algebra D .G/K

441

with respect to X is smaller than m. Besides, Œ ; B and ŒAm ; B belong to ker . So, 

 Q B am ŒA; B 2 ker : A;

Thus, ŒA; B 2 ker  and the proof of the theorem is complete.





Definition 13.2.2. Let ` 2 g and B` the bilinear form defined as before. We say that k is co-isotropic at ` if the subspace k` D fX 2 gI B` .X; k/ D f0gg is isotropic regarding B` . By means of this concept of being co-isotropic, we are able to characterize the case where jK has finite multiplicities. Proposition 13.2.3. When and only when k is co-isotropic almost everywhere on ˝, jK has finite multiplicities. Proof. We proceed by induction on the dimension of G. Since we may assume g ¤ k, we take an ideal g0 of codimension 1 in g containing k and put G0 D exp.g0 /. When ˝ is non-saturated with respect to g0 , g.`/ at ` 2 ˝ is not contained in g0 and the restriction jG0 is irreducible. Let ˝Q  g0 be the coadjoint orbit of G0 c0 . corresponding to jG0 2 G Since jK D .jG0 / jK , the induction hypothesis assures that jK has finite Q while, multiplicities when and only when k is co-isotropic almost everywhere on ˝, if we set `0 D `jg0 for ` 2 ˝, there exists a certain element X.`/ 2 g.`/ not belonging to g0 so that k` D k`0 ˚ X.`/. Hence, k is co-isotropic at ` if and only if it is co-isotropic at `0 . Supposing that ˝ is saturated relative to g0 , let us examine two possibilities. First, general K-orbits in ˝ are supposed saturated relative to g0 . The restriction jG0 is decomposed into a one-parameter family, Z jG0 '

˚ R

t dt:

Moreover, with g D RX ˚ g0 , t D exp.tX /0 and  ' indG G0 t for any t 2 R. Since general K-orbits in ˝ are saturated in the direction of .g0 /?  g , the set ft 2 RI .exp.tX /`0 / jk D .k`0 /jk ; 9k 2 Kg is a finite set for a general ` 2 ˝. Hence, jK has finite multiplicities if and only if .t /jK has finite multiplicities at almost all t 2 R. We denote by ˝t  g0 the c0 .t 2 R/. From what we have just coadjoint orbit of G0 corresponding to t 2 G seen and the induction hypothesis, jK has finite multiplicities if and only if k  g0 is co-isotropic almost everywhere on ˝t for almost all t 2 R. But the latter condition is equivalent to that k is co-isotropic almost everywhere on ˝ since k`  g0 at a general ` 2 ˝.

442

13 Commutativity Conjecture: Restriction Case

Lastly, suppose that almost all K-orbits in ˝ are non-saturated relative to g0 . Take such a general element ` 2 ˝. Then, there exist X.`/ 2 k` not belonging to g0 and Y .`/ 2 g0 .`0 / not belonging to g.`/. Because Y .`/ belongs to k` , k is not co-isotropic. Here, if we look at the K-orbit ! D Kpk .`/  pk .˝/  k ; it is clear that pk1 .!/ \ ˝ contains infinite K-orbits, and jK has infinite multiplicities.  Remark 13.2.4. As in the case of induced representations, Proposition 13.2.3 no longer holds for exponential solvable Lie groups. For example, let g D hA; X; Y iR W ŒA; X  D X Y; ŒA; Y  D X CY; k D RX ,  an infinite-dimensional irreducible unitary representation of G and ˝ the corresponding coadjoint orbit of G. Then, k` D RX ˚ RY at generic ` 2 ˝ and k is co-isotropic at `. However, it is immediately seen that jK has infinite multiplicities. Let us give another interpretation of this fact by a general principle due to Michel Duflo. Let ` 2 pk .˝/  k , ! the coadjoint orbit of K passing ` and  the irreducible unitary representation of K corresponding to !. Then, Proposition 13.2.3 is interpreted as follows: Proposition 13.2.5. When and only when K.`/-orbits in ˝ \ pk1 .`/ are open sets almost everywhere, the representation jK has finite multiplicities. In this situation, the multiplicity n ./ in Theorem 8.2.4 is equal to the number of K.`/-orbits contained in ˝ \ pk1 .`/. Writing kd Cr instead of lr .1  r  p/, we get a sequence of Lie subalgebras f0g D k0  k1      kd 1  k D kd  kd C1      kn1  kn D g of g. Here dim.kr / D r; 1  r  n and kr is an ideal of k for 1  r  d . Let us extract a vector Xs 2 ks n ks1 for 1  s  n. In reality, the dual basis fX1 ; : : : ; Xn g is a Jordan–Hölder basis of g relative to the unipotent coadjoint action Ad .K/. The projections pj W g ! kj .1  j  n/ commute with the action of K and exactly as before, the linear form ` 2 g gives an n-tuple of nonnegative integers e.`/ D .e1 .`/; : : : ; en .`// defined by ek .`/ D dim.Kpk .`// .1  k  n/. Conversely, each n-tuple e D .e1 ; : : : ; en / of non-negative integers gives a layer Ue of K-orbits in g defined by Ue D f` 2 g I ek .`/ D ek ; 1  k  ng: Then, there exists a unique layer Ue such that Ue \ ˝ becomes a non-empty Zariski open set of ˝. We consider the set S.e/ of the jump indices:

13.2 Commutativity of the Algebra D .G/K

443

S.e/ D f1  k  nI ek D ek1 C 1g and the set T .e/ of the non-jump indices: T .e/ D f1  k  nI ek D ek1 g; where we agree that e0 D 0. Put Kj D exp.kj /; 1  j  n  1. In particular, Kd D K. Theorem 13.2.6. When we pass from kj 1 to kj , the algebra U .g/k enlarges if j 2 T .e/, and does not enlarge if j 2 S.e/. More precisely, if we set U .kj /k D U .g/k \ U.kj / for 1  j  n:   (1) If j 2 T .e/, U .kj /k ¤ U .kj 1 /k C U.kj / U.kj 1 / \ ker  and there exists W 2 U .kj /k of the form W D aXj C b .a; b 2 U.kj 1 //, .a/  ¤ 0. (2) If j 2 S.e/, U .kj /k D U .kj 1 /k C U.kj / U.kj 1 / \ ker  . Proof. First we treat the case where j 2 T .e/. The claim for the first index 1 2 T .e/ is automatically obtained. Assume that the statements are true for the indices less than or equal to j  1. When 1  j  d , with e 0 D .e1 ; : : : ; ed /, the Corwin– Greenleaf e 0 -central element satisfies the condition of the theorem. Let d C 1  j . When k D f0g, the claim is evident. So, let us employ induction on dim k. From the induction hypothesis for kd 1 , there exists W D aXj C b .a; b 2 U.kj 1 //;

(13.2.1)

which satisfies the conditions for kd 1 . Namely, .ŒY; W / D 0 for arbitrary Y 2 kd 1 and .a/ ¤ 0. Let us first remark that we may assume Œkd 1 ; a  ker . In fact, if there exists Y 2 kd 1 such that ŒY; a 62 ker , the element ŒY; W  D ŒY; aXj C b 0 ; b 0 2 U.kj 1 /; of ker  is already our desired element. We operate ad.Yd / on W several times so that .ad.Yd //m .W / belongs for the first time to ker . Then, as is immediately seen, Wr D .ad.Yd //r1 .W /; r  1 too belongs to U .g/kd 1 and has the same form as in expression (13.2.1). Besides, using the maximal power m0 such that 0

.ad.Yd //m .W / 62 U.kj 1 / 0

modulo ker , we replace W by .ad.Yd //m .W /. Then, we may suppose that .ad.Yd // .W / 2 U.kj 1 / modulo ker .

444

13 Commutativity Conjecture: Restriction Case

If m D 1, then .Œk; W / D f0g and there is nothing left to do. Hence, we assume m  2. We first treat the case where m D 2q C 1  3. In this case, if we set WQ D .W1 Wm C Wm W1 /  .W2 Wm1 C Wm1 W2 / C    C .1/q2 .Wq1 WqC3 C WqC3 Wq1 / 2 ; C .1/q1 .Wq WqC2 C WqC2 Wq / C .1/q WqC1

ŒYd ; WQ  2 ker  and Œk; WQ   ker . Let us notice that we can write WQ D .aWm C Wm a/Xj C b 0 with a certain b 0 2 U.kj 1 /. As we have used this fact several times until now, when we realize  as a monomial representation in L2 .Rk / starting from a polarization, the space HC1 of the C 1 -vectors coincides with the Schwartz space S.Rk /, where .a/; .Wm / act as differential operators with polynomial coefficients. Hence, it results that .aWm C Wm a/ ¤ 0 and WQ possesses the required properties. Secondly, suppose that m is an even number. If we replace W by W 0 D W .Wm1 C cWm / .0 ¤ c 2 C/ and set Wr0 D .ad.Yd //r1 .W 0 /; 1  r  m C 1, then W20 W2 .Wm1 C cWm / C W1 Wm ;    ; 0 mWm2 Wm0 Wm .Wm1 C cWm / C .m  1/Wm1 Wm ; WmC1

modulo ker . Therefore, our new WQ is obtained by

˚ WQ D m W1 .Wm1 C cWm /Wm2 C Wm2 W1 .Wm1 C cWm /  W1 Wm fWm .Wm1 C cWm / C .m  1/Wm1 Wm g  fWm .Wm1 C cWm / C .m  1/Wm1 Wm g W1 Wm C VQ ; where VQ is a certain element of U.kj 1 /. We have WQ m.W1 Wm1 Wm2 C Wm2 W1 Wm1 /  .m  1/.W1 Wm Wm1 C Wm1 Wm W1 /Wm  .W1 Wm2 Wm1 C Wm Wm1 W1 Wm / C c.m  1/.W1 Wm2 C Wm2 W1 /Wm ;

13.2 Commutativity of the Algebra D .G/K

445

  modulo U.kj 1 /. Besides, since  .W1 Wm2 C Wm2 W1 /Wm ¤ 0 as we saw above, it is clear that we can choose 0 ¤ c 2 C so that WQ 62 U.kj 1 / C ker . This procedure does not work well when m D 2. Recall the definition i o n h U .kj 1 /kd 1 D WO 2 U.kj 1 /I  WO ; kd 1 D f0g :   Provided Yd ; U .kj 1 /kd 1 6 ker , we can increase m by multiplying W by WO 2 U .kj 1 /kd 1 h such that 

Yd ; WO

i

¤ 0. Indeed, take such a WO satisfying

i h i h WO 1 D Yd ; WO 62 ker ; WO 2 D Yd ; WO 1 2 ker : Then, by the change from W D W1 to WQ 1 D W1 WO , we get     WQ 2 D Yd ; WQ 1 D W2 WO C W1 WO 1 ; WQ 3 D Yd ; WQ 2 2W2 WO 1 .mod ker /: Here, if we produce WQ D WQ 1 WQ 3 C WQ 3 WQ 1  WQ 22 , regarding Xj the first two terms are of degree 1 but the last term is of degree 2. In fact, we can write  2 WQ D  aWO 1 Xj2 C a1 Xj C a2   with some a1 ; a2 2 U.kj 1 /. Let us remark that  aWO 1 ¤ 0. Next, in order to decrease the degree let  us make use of the element Xd C1 2 kd C1 belonging to the normalizer of k. Then, Xd C1 ; WQ also satisfies the relation 

   k; Xd C1 ; WQ D f0g:

Choosing Xd C1 freely, we may suppose ŒXd C1 ; Xj  62 ker . Otherwise, it results that Œk; Xj   ker  and Xj fulfils the request. Hence by continuity, letting  be a sufficiently small positive real number, " Xd C1 C

d X

# ˛i Xi ; Xj

62 ker 

i D1

i h for ˛i 2 C such that j˛i j <  .1  i  d /. If Xd C1 ; aWO 1 62 ker , by replacing WQ by .ad.Xd C1 //q .WQ / with an appropriate q 2 N, we may assume WQ D a0 Xj2 C a00 Xj C a000 ;

446

13 Commutativity Conjecture: Restriction Case

where a0 ; a00 ; a000 2 U.kj 1 /; .a0 / ¤ 0 and ŒXd C1 C k; a0   ker . Now, if there exist complex numbers ˛O i .1  i  d / with j˛O i j <  satisfying " Xd C1 C

d X

# 0

˛O i Xi ; 2a Xj C a

00

62 ker ;

i D1

h i P then, W  D Xd C1 C diD1 ˛O i Xi ; WQ fulfils our request. If there are no such ˛O i .1  i  d /, it turns out that Œk; 2a0 Xj C a00   ker  and 2a0 Xj C a00 is our desired element.   Provided Yd ; U .kj 1 /kd 1  ker , from the induction hypothesis for the index j  1,     k \ kj 1 .`jkj 1 / ¤ kd 1 \ kj 1 .`jkj 1 / at a general ` 2 ˝. From this, taking j 2 T .e/ into account,     k \ kj .`jkj / ¤ kd 1 \ kj .`jkj / : Using the multiplicity m./ and the measure j explained in Theorem 8.2.4 of Chap. 8, we write the canonical irreducible decomposition formula as Z jKj '

˚

bj K

m./d j ./:

cj relative to j , there exists an element By Theorem 13.1.1, for almost all  2 K V 2 U.k/ \ ker  of the form V D

r X

j

Pj Yd ;

j D0

where Pj 2 U.kd 1 / .0  j  r/ and .Pr / ¤ 0. We choose V in such a manner that r  1 might be as small as possible. Then, if .ad.Yd //2 .W / belongs to ker , 0 ŒW; V  @

r X

1 j 1 jPj Yd A ŒW; Yd 

mod ker :

j D1

P j 1 Since rj D1 jPj Yd does not belong to ker  and the actions in HC1 have no non  trivial zero divisor, we get  .ŒW; Yd / D 0 from  ŒW; V  D 0. Consequently,  .ŒYd ; W / D 0 and W 2 U .g/k . Next, let us examine the case where j 2 S.e/. We employ a new induction on dim g. First, let us remark that the study is reduced to the case where j D n.

13.2 Commutativity of the Algebra D .G/K

447

Indeed, if j  n  1, let us consider  0 D jKn1 . When  0 is irreducible, we apply the induction hypothesis. If not,  0 is decomposed into a one-parameter family t0 .t 2 R/ of irreducible unitary representations of Kn1 . Now, if there exists W 2 U.kj / n U.kj 1 / such that .Œk; W / D f0g, then t0 .Œk; W / D f0g. Hence W 2 U.kj / \ ker t0 by the induction hypothesis. In consequence, W 2 U.kj / \ ker . Henceforth, we suppose j D n and that the Kd 1 -orbit passing a general ` 2 ˝ is non-saturated relative to kn1 . We may assume without loss of generality that k contains the centre z of g. When dim z  2, we may pass to the quotient Lie algebra of g by z \ ker  and apply the induction hypothesis. In this way, we are as usual brought to the following case: dim z D 1; jz ¤ 0, namely z D g1 D RZ; .Z/ ¤ 0. Let g2 D RY ˚ z. We denote by gQ the centralizer of Y in g. Finally, taking X 2 g n gQ such that ŒX; Y  D Z we write g D RX C gQ . First, assume gQ D kn1 . Provided Y 2 k, the condition .ŒY; W / D 0 for an element W of U.g/ leads to W 2 U.Qg/ C ker . In fact, taking a polarization b at f 2 ˝ of g contained in gQ , we realize   G GQ  ' indG B f ' indGQ indB f : Here, of course, B D exp b; GQ D exp gQ . Now, we write W D

m X

a X ; a 2 U.Qg/; 0   m;

(13.2.2)

D0

with .am / ¤ 0. If m  1, we have .ŒY; W / D

m X

.a /.Z/.X / 1 D 0:

D1

However, since .X / is nothing but the partial differential operator relative to the first coordinate introduced along X , this is a contradiction. If Y 62 k, setting k0 D k ˚ RY D k C g2 , k0 \ g.`/ ¤ .kd 1 C g2 / \ g.`/ at almost P all ` 2j ˝. Therefore, taking Theorem 13.1.1 into account, there exists V D rj D0 Pj Yd 2 ker , where Pj 2 U.kd 1 Cg2 / .0  j  r/ with .Pr / ¤ 0. If we rewrite V as V

q X

bk Y k ; bk 2 U.k/ .0  k  q/

(13.2.3)

kD0

modulo ker  with .bq / ¤ 0, then q  1. In fact, in the irreducible decomposition

448

13 Commutativity Conjecture: Restriction Case

Z jK '

˚ KO

m./d ./;

 is written in the form t ˝ Q ; t 2 R, where t is a unitary character of K trivial on Kd 1 and where Q is an irreducible unitary representation of K with irreducible restriction to Kd 1 . Besides, there exists a measure 0 on Kd 1 so that the measure d ./Pis equivalent to the product measure dt  d 0 .Q /. From this, if q D 0 or j V D rj D0 Pj Yd 2 ker ; Pj 2 U.kd 1 / .0  j  r/, it follows that .Pr / D 0. This is contradictory. We choose V in such a manner that q  1 will be minimal in expression (13.2.3). Lastly, if there exists W 2 U.g/ of the form in expression (13.2.2) with m  1 such that .Œk; W / D f0g, we compute 1 0 q q m X X X     0 D .ŒW; V / D .bk / W; Y k D .bk / @ .a / X ; Y k A

1

kD0

D m.Z/

X

D0

kD0

!

q

!

k.bk /.Y /k1 .X /m1 C

m2 X

.cj /.X /j

j D0

kD1

Pq with cj 2 U.Qg/ .0  j  m  2/. We derive from this that V 0 D kD1 kbk Y k1 2 ker . But this contradicts the choice of V . Secondly, let us examine the case where gQ ¤ kn1 . Suppose that W 2 U.g/ Q satisfies .Œk; W / D f0g. Since Y 2 kn1 , the .K \ G/-orbits are already saturated with respect to kn1 almost everywhere in ˝. When k 6 gQ , W belongs to U.kn1 / C ker  by the induction hypothesis. Next assume k  gQ . If general K-orbits in ˝ are saturated relative to gQ , it follows from what we have seen until now that W 2 U.Qg/ C ker , and then from the induction hypothesis we conclude that W 2 U.Qg \ kn1 / C ker . As the last possibility, let us study the case where general K-orbits in ˝ are non-saturated with respect to gQ . This means that, at a general ` 2 ˝, the orbit K .`jkn1 /  kn1 is non-saturated with respect to gQ \ kn1 . Therefore, there exists V D aX C b 2 U.kn1 / such that .Œk; V / D f0g. Here, a; b 2 U.Qg \ kn1 / and .a/ ¤ 0. Now, P wekchoose in kn1 the element X 2 g n gQ introduced above. We write W D m g/ .0  k  m/ and assume m  1. Otherwise, it kD0 X ck ; ck 2 U.Q follows from the induction hypothesis applied to gQ that W 2 U.Qg \ kn1 / C ker . Because W 0 D ŒY; W  D

m X

kX k1 Zck

kD1

also satisfies the condition .Œk; W 0 / D f0g, the induction hypothesis on the degree of X in W implies ck 2 U.kn1 / .1  k  m/ modulo ker . Finally, we write V

m

Da X C m

m

m1 X j D0

X j bj

13.2 Commutativity of the Algebra D .G/K

449

with bj 2 U.Qg \ kn1 / .0  j  m  1/. Taking .Œk; a/ D f0g into consideration, W 00 D am W  V m cm D

m1 X j D0



X  m1 am X j cj  X j bj cm D X j cQj j D0

also satisfies .Œk; W 00 / D f0g. Here, cQj .0  j  m  1/ are all elements of U.Qg/. Since the degree of X in W 00 is less than or equal to m  1, the induction hypothesis asserts that cQj all belong to U.Qg \ kn1 / modulo ker . Because cQ0 D am c0 C c 0 with a certain c 0 2 U.Qg \ kn1 / C ker  and .a/ ¤ 0, we conclude that c0 2 U.kn1 / C ker . Finally, W 2 U.kn1 / C ker  as expected.  We are now ready to prove the main theorem of this chapter, the commutativity theorem for restrictions of representations. Theorem 13.2.7. Let G D exp g be a nilpotent Lie group with Lie algebra g,  2 GO and K D exp k an analytic subgroup of G. The algebra D .G/K is commutative if and only if the restriction jK has finite multiplicities. Proof. We already showed in Theorem 13.2.1 that D .G/K is commutative if jK has finite multiplicities. Here, we suppose that jK has infinite multiplicities and show that D .G/K is non-commutative. Owing to Proposition 13.2.3, there exist two non-jump indices j1 ; j2 2 T .e/ .j1 < j2 / which are mutually dual for the alternating bilinear form B` at a general ` 2 ˝. In order to show the noncommutativity in question, letting d C 1  j , we may replace K by Kj if jKj already has infinite multiplicities. So, the study is reduced to the situation where jK has infinite multiplicities but jKd C1 has finite multiplicities. From now on we assume this situation. Then, it follows that there exists only one pair of the non-jump indices j1 < j2 mentioned above and j1 D d C1. In particular, k contains the centre z of g. By Theorem 13.2.6, there exist two elements Wk D ak Xjk C bk .k D 1; 2/ so that .Œk; Wk / D f0g, where ak ; bk 2 U.kjk 1 / and .ak / ¤ 0. Especially a1 ; b1 2 U.k/. What we should show is .ŒW1 ; W2 / ¤ 0. By means of the Vergne polarization b of g constructed at f 2 ˝ from the Jordan–Hölder sequence given at the beginning, we realize  ' indG B f in L2 .Rq / .q D dim.g=b// as a monomial representation induced from the corresponding analytic subgroup B D exp b. The position of the index j1 is found in B and that of the index j2 in Rq . Hence, .Œk; a2 / D f0g. At a general ` 2 ˝, we denote by `Q the restriction of ` to kj1 and by `0 that to kj1 1 . Then Q D dim.G  `0 / C 1. Besides, since the orbit Kj  `Q is non-saturated with dim.G  `/ 1 Q j / independently respect to kj1 1 , we see .Œk; a1 / D f0g by changing the value `.X 1 of `0 . Put as usual a D z\ker f . If a ¤ f0g, we may descend to the quotient Lie algebra g=a and apply there the induction hypothesis. Assume a D f0g. Then, dim z D 1, namely z D g1 D k1 and f jz ¤ 0. Setting g2 D RY C z and gQ as the centralizer of g2 in g, it follows that gQ is an ideal of codimension 1 in g containing the polarization b. Let g D RX ˚ gQ .

450

13 Commutativity Conjecture: Restriction Case

Let us separate two cases. First, the case where g2  k. By Theorem 13.2.6, W1 ; W2 are included in U.Qg/ modulo ker , while the restriction of  to GQ D exp.Qg/ is decomposed into a one-parameter family of irreducible representations ft I t 2 Rg as Z jGQ '

˚

R

t dt:

Q

Indeed, t D exp.tX /0 with 0 D indG B f . This means that, if t1 ¤ t2 , t1 jK and t2 jK are disjointly decomposed. Thus, t jK has infinite multiplicities for almost all t 2 R. So, the desired result is derived from the induction hypothesis applied to these t . Secondly, the case where g2 6 k. In this case, kj1 D kd C1 D k C g2 and W1 D a1 Y C b10 with a new b10 2 U.k/. Moreover, kj2 1  gQ and Xj2 62 gQ . Taking these into account, since 0 ¤ ŒY; Xj2  2 z and jz ¤ 0, we have    a1 Y C b10 ; a2 Xj2 C b2 D  a1 Y; a2 Xj2 C b2      D  a1 Y; a2 Xj2 C b2 D .a1 /.a2 / Y:Xj2 ¤ 0:

 .ŒW1 ; W2 / D 





13.3 Commutativity of the Centralizer of Lie Subalgebras Keeping the notations, let G D exp g be a connected and simply connected nilpotent Lie group, K D exp k a connected closed subgroup of G and p D pk W g ! k the canonical projection. Let us study the commutativity of the centralizer c.k/ of k in U.g/. To begin with, we denote by O the set of the linear forms on g whose coadjoint G-orbits have maximal dimension. O is a non-empty Zariski open set of g . For  2 O, we set dK ./ D max dim.K`/; dK0 ./ D max dim .Kpk .`// : `2G

`2G

By definition, dK ; dK0 are G-invariant functions on O taking non-negative integers as their values. Now, we put dK D max dK ./ dK0 D max dK0 ./ 2O

2O

and ˚

Z D  2 OI dK ./ D dK ; dK0 ./ D dK0 :

13.3 Commutativity of the Centralizer of Lie Subalgebras

451

Z is G-invariant and contains a non-empty Zariski open set ZQ with following properties. The dimensions of the G-orbits and the K-orbits of the elements of ZQ are maximal and the dimensions of K-coadjoint orbits of their projections onto k are also maximal. We say that the irreducible unitary representations of G associated with the elements of ZQ are K-generic. Let  2 GO and ` 2 p .˝.//  k . We denote by ! the coadjoint orbit of K passing `, by  the irreducible unitary representation of K corresponding to ! and by K.`/ the stabilizer of ` in K. By Proposition 13.2.5, when and only when the K.`/-orbits in ˝./ \ p 1 .`/ are generally open sets, the representation jK has finite multiplicities. This is also equivalent to that the K-orbits in ˝./ \ p 1 .!/ are generally open sets. In other words, when and only when dim.˝.// D 2dim.K  /  dim.Kp.//;  2 ˝./ holds generally on ˝./, the representation jK has finite multiplicities. When the representation  is K-generic, jK has finite multiplicities if and only if dim.˝.// D 2dK  dK0 : Remarking that dim.˝.//  2dK C dK0 is a non-negative invariant quantity on a Q we admit the following alternative: ` jK has finite multiplicities Zariski open set Z, Q for all ` 2 Z or ` jK has generally infinite multiplicities on Z. Concerning the restriction to K of the coadjoint action of G, the maximal layer and Z intersect necessarily in a non-empty Zariski open set. Let S.e/; T .e/ be the sets of corresponding jump indices and non-jump indices. For a finite-dimensional vector space V , CŒV   denotes the algebra of all polynomial functions on the dual vector space V  . If we represent by the superscript K the totality of K-invariant elements, it is well known that the symmetrization map gives an isomorphism between CŒg K and U.g/K as vector spaces. Let us make use of this fact taking some kj ; j  d C 1 as g. When j 2 T .e/, the general K-orbits in kj  are non-saturated with respect to kj 1 and there exists a K-invariant polynomial function on kj  , having the form P D P1 Xj C P2 , where P1 ; P2 are elements of CŒkj 1   and P1 is not identically 0. Thus, transferring by the symmetrization map, we could find an element of c.k; f0g/ D c.k/ having the form W D aXj C b with a ¤ 0; b in U.kj 1 /. Next, let  be the Plancherel measure for the regular representation of G and assume that jK has finite multiplicities for almost all  2 GO relative to . Then, c.k/ D

\ 2GO

is commutative.

U .g/k

452

13 Commutativity Conjecture: Restriction Case

Conversely, let A be the set of all  2 GO such that jK has infinite multiplicities. If .A/ ¤ 0, A \ OG .Z/ ¤ ; and ` jK has infinite multiplicities for almost all ` 2 Z. When we consider a sequence of Lie subalgebras k D kd  kd C1      kn1  kn D g; there exists some index j0  d such that, for almost all  2 GO relative to , jKj0 has infinite multiplicities but jKj0 C1 has finite multiplicities. Then, it suffices to show the non-commutativity of c.kj0 /. That is to say, we may replace k by kj0 . This means that, using the reasoning of Theorem 13.2.7 on the intersection of Z and another non-empty Zariski open set if necessary, there exists only one pair of nonjump indices .j1 ; j2 / which are mutually dual for the alternating bilinear forms at almost all ` 2 Z. Then, there exist elements Wk D ak Xjk C bk .k D 1; 2/ of U.kjk /K such that ak ; bk are elements of U.kjk 1 / satisfying .ak / ¤ 0 for at least O Further, Wk is an element of c.k/ and also of U .kk /k . some K-generic  2 G. Fixing such a K-generic irreducible representation  and following the reasoning in the proof of Theorem 13.2.7, we verify .ŒW1 ; W2 / ¤ 0. Hence c.k/ is not commutative. In this way, we obtain the following result. Theorem 13.3.1. Let G be a connected and simply connected nilpotent Lie group and K a connected closed subgroup of G. Then, the following assertions are equivalent to each other: O (1) jK has finite multiplicities for almost all  2 G. (2) There exists a K-generic representation  2 GO such that jK has finite multiplicities. (3) The centralizer of k in U.g/ is commutative. Remark 13.3.2. The polynomial conjecture is also interpreted for restrictions of representations. However, the obtained conjecture is proved only in particular cases, for instance when K is a normal subgroup (cf. [7]). Example 13.3.3 (Non-nilpotent Case). Let g˛ D hA; X; Y iR W ŒA; X  D X  ˛Y; ŒA; Y  D ˛X C Y .0 ¤ ˛ 2 R/. Then, G˛ D exp.g˛ / is an exponential solvable but not completely solvable Lie group. Let fA ; X  ; Y  g be the dual basis of fA; X; Y g in g˛ . The coadjoint orbits of G˛ , which do not degenerate to one point, are those ˝ passing f D .cos /X  C .sin /Y  .0 <   2/: We denote by  the irreducible unitary representation of G˛ associated with f . Then, ˚

˝ D sA C e t cos.  ˛t/X  C e t sin.  ˛t/Y  I s; t 2 R :

13.3 Commutativity of the Centralizer of Lie Subalgebras

453

The polarization b D RX ˚RY at f satisfies the Pukanszky condition. Through this realization, the representation  acts on L2 .R/ and we get a self-adjoint operator i .A/ D i

d ; i .X / D e t cos.  ˛t/; i .Y / D e t sin.  ˛t/: dt

(1) Take k D RX ˚ RY . Then, Z  jK '

Z

˚

R

exp.tA/f dt '

˚ R

et cos. ˛t /X  Cet sin. ˛t /Y  dt;

while U .g/k is nothing but the centralizer of k in U.g/ and D .G/K Š U.g/K is commutative. In fact, if W D

m X

Aj Cj ; Cj 2 U.k/

j D0

is an element of U.g/K , 0 D ŒW; X C iY  D m.1 C i ˛/Am1 Cm C P; where the degree with respect to A of the element P of U.g/ is less than or equal to m  2. Hence it follows that m D 0 and W 2 U.k/. In consequence,   D .G/K Š CŒX; Y  Š C e t cos.  ˛t/; e t sin.  ˛t/ : (2) Next take k D RX . In this case,  jK has infinite multiplicities but D .G/K Š CŒX; Y  is commutative. As this example shows, Theorems 13.2.7, 13.3.1 no longer hold for exponential solvable Lie groups. Remark 13.3.4. As we easily become aware, at present many of the detailed analyses are possible only for nilpotent Lie groups, Even if we limit our study to exponential solvable Lie groups, there remains much to be clarified. In order to open a view on prospective developments, it may be instructive to deal with completely solvable Lie groups whose Lie algebras are normal j -algebras in the Pjatetskii– Shapiro sense, when we think about their rich algebraic structures.

References

1. Arnal, D., Fujiwara, H., Ludwig, J.: Opérateurs d’entrelacement pour les groupes de Lie exponentiels. Am. J. Math. 118, 839–878 (1996) 2. Arsac, G.: Opérateurs compacts dans l’espace d’une représentation. C. R. Acad. Sci. Paris 286, 189–192 (1982) 3. Auslander, L., Kostant, B.: Polarization and unitary representations of solvable Lie groups. Invent. Math. 14, 255–354 (1971) 4. Auslander, L., Moore, C.C.: Unitary Representations of Solvable Lie Groups. Memoirs of the American Mathematical Society, Number 62, 199 pp. (1966) 5. Baklouti, A., Fujiwara, H.: Opérateurs différentiels associés à certaines représentations unitaires d’un groupe de Lie résoluble exponentiel. Compos. Math. 139, 29–65 (2003) 6. Baklouti, A., Fujiwara, H.: Commutativité des opérateurs différentiels sur l’espace des représentations restreintes d’un groupe de Lie nilpotent. J. Math. Pures Appl. 83, 137–161 (2004) 7. Baklouti, A., Fujiwara, H., Ludwig, J.: Analysis of restrictions of unitary representations of a nilpotent Lie group. Bull. Sci. Math. 129, 187–209 (2005) 8. Benoist, Y.: Espaces symétriques exponentiels. Thèse de 3e cycle, Univ. Paris VII (1983) 9. Benoist, Y.: Multiplicité un pour les espaces symétriques exponentiels. Mém. Soc. Math. France 15, 1–37 (1984) 10. Bernat, P., et al.: Représentations des groupes de Lie résolubles. Dunod, Paris (1972) 11. Blattner, R.J.: On induced representations I. Am. J. Math. 83, 79–98 (1961); Blattner, R.J.: On induced representations II. Am. J. Math. 83, 499–512 (1961) 12. Bonnet, P.: Transformation de Fourier des distributions de type posotif sur un groupe de Lie unimodulaire. J. Funct. Anal. 55, 220–246 (1984) 13. Bourbaki, N.: Intégration. Hermann, Paris (1967) 14. Corwin, L., Greenleaf, F.P., Grélaud, G.: Direct integral decompositions and multiplicities for induced representations of nilpotent Lie groups. Trans. Am. Math. Soc. 304, 549–583 (1987) 15. Corwin, L., Greenleaf, F.P.: Spectrum and multiplicities for restrictions of unitary representations in nilpotent Lie groups. Pac. J. Math. 135, 233–267 (1988) 16. Corwin, L., Greenleaf, F.P.: Representations of Nilpotent Lie Groups and Their Applications, Part I: Basic Theory and Examples. Cambridge University Press, Cambridge (1990) 17. Corwin, L., Greenleaf, F.P.: Commutativity of invariant differential operators on nilpotent homogeneous spaces with finite multiplicity. Commun. Pure Appl. Math. 45, 681–748 (1992) 18. Dixmier, J.: L’application exponentielle dans les groupes de Lie résolubles. Bull. Soc. Math. France 85, 113–121 (1957)

© Springer Japan 2015 H. Fujiwara, J. Ludwig, Harmonic Analysis on Exponential Solvable Lie Groups, Springer Monographs in Mathematics, DOI 10.1007/978-4-431-55288-8

455

456

References

19. Dixmier, J.: Représentations irréductibles des algèbres de Lie nilpotentes. Anals Acad. Brasil. Ci. 35, 491–519 (1963) 20. Dixmier, J.: Les C*-algèbres et leurs représentations. Gauthier-Villars, Paris (1964) 21. Dixmier, J.: Enveloping Algebras. Graduate Studies in Mathematics, vol. 11. American Mathematical Society, Providence (1996) 22. Duflo, M.: Open problems in representation theory of Lie groups. In: Oshima, T. (ed.) Conference on Analysis on Homogrneous Spaces, pp 1–5. Katata in Japan (1986) 23. Effros, E.G.: Transformation groups and C*-algebras. Ann. Math. 81, 38–55 (1965) 24. Fell, J.M.G.: The dual space of C*-algebras. Trans. Am. Math. Soc. 94, 365–403 (1960) 25. Fell, J.M.G.: A new proof that nilpotent groups are CCR. Proc. Am. Math. Soc. 107, 93–99 (1962) 26. Fell, J.M.G.: Weak containment and induced representations of groups I. Can. J. Math. 14, 237–268 (1962); Fell, J.M.G.: Weak containment and induced representations of groups II. Trans. Am. Math. Soc. 110, 424–447 (1964) 27. Fujiwara, H.: On unitary representations of exponential groups. J. Fac. Sci. Univ. Tokyo 21, 465–471 (1974) 28. Fujiwara, H.: On holomorphically induced representations of exponential groups. Jpn. J. Math. 4, 109–170 (1978) 29. Fujiwara, H.: Polarisations réelles et représentations associées d’un groupe de Lie résoluble. J. Funct. Anal. 60, 102–125 (1985) 30. Fujiwara, H.: Représentations monomiales des groupes de Lie nilpotents. Pac. J. Math. 127, 329–351 (1987) 31. Fujiwara, H., Yamagami, S.: Certaines représentations monomiales d’un groupe de Lie résoluble exponentiel. Adv. St. Pure Math. 14, 153–190 (1988) 32. Fujiwara, H.: Représentations monomiales des groupes de Lie résolubles exponentiels. In: Duflo, M., Pedersen, N.V., Vergne, M. (eds.) The Orbit Method in Representation Theory. Proceedings of a Conference in Copenhagen, pp. 61–84. Birkhäser, Boston (1990) 33. Fujiwara, H.: La formule de Plancherel pour les représentations monomiales des groupes de Lie nilpotents. In: Kawazoe, T., Oshima, T., Sano, S. (eds.) Representation Theory of Lie Groups and Lie Algebras. The Proceedings of Fuji-Kawaguchiko Conference, pp. 140–150. World Scientific (1992) 34. Fujiwara, H.: Sur les restrictions des représentations unitaires des groupes de Lie résolubles exponentiels. Invent. Math. 104, 647–654 (1991) 35. Fujiwara, H., Lion, G., Mehdi, S.: On the commutativity of the algebra of invariant differential operators on certain nilpotent homogeneous spaces. Trans. Am. Math. Soc. 353, 4203–4217 (2001) 36. Fujiwara, H., Lion, G., Magneron, B.: Algèbres de fonctions associées aux représentations monomiales des groupes de Lie nilpotents. Prépub. Math. Univ. Paris 13, 2002–2 (2002) 37. Fujiwara, H., Lion, G., Magneron, B., Mehdi, S.: Commutativity criterion for certain algebras of invariant differential operators on nilpotent homogeneous spaces. Math. Ann. 327, 513–544 (2003) 38. Fujiwara, H.: Une réciprocité de Frobenius. In: Heyer, H., et al. (eds.) Infinite Dimensional Harmonic Analysis, pp. 17–35. World Scientific (2005) 39. Fujiwara, H.: Unitary representations of exponential solvable Lie groups–Orbit method (Japanese). Sugakushobo Co. (2010) 40. Gindikin, G., Pjatetskii-Shapiro, I.I., Vinberg, E.E.: Geometry of homogeneous bounded domains. C. I. M. E. 3, 3–87 (1967) 41. Godement, R.: Les fonctions de type positif et la théorie des groupes. Trans. Am. Math. Soc. 63, 1–84 (1948) 42. Grélaud, G.: Désintégration des représentations induites des groupes de Lie résolubles exponentiels. Thèse de 3e cycle, Univ. de Poitiers (1973) 43. Grélaud, G.: La formule de Plancherel pour les espaces homogènes des groupes de Heisenberg. J. Reine Angew. Math. 398, 92–100 (1989)

References

457

44. Halmos, P.R.: Measure Theory. D. Van Nostrand Company, Inc., New York (1950) 45. Hochschild, G.: The Structure of Lie Groups. Holden-Day, San Francisco (1965) 46. Howe, R.: On a connection between nilpotent groups and oscillatory integrals associated to singularities. Pac. J. Math. 73, 329–364 (1977) 47. Humphreys, J.: Linear algebraic groups. In: Graduate Texts in Mathematics, vol. 21. Springer, New York (1975) 48. Kirillov, A.A.: Unitary representations of nilpotent Lie groups. Russ. Math. Surv. 17, 57–110 (1962) 49. Kirillov, A.A.: Elements of the Theory of Representations. Springer, Berlin (1976) 50. Kirillov, A.A.: Lectures on the Orbit Method. Graduate Studies in Mathematics, vol. 64. American Mathematical Society, Providence (2004) 51. Leptin, H., Ludwig, J.: Unitary Representation Theory of Exponential Lie Groups. Walter de Gruyter, Berlin (1994) 52. Lion, G.: Indice de Maslov et représentation de Weil. Pub. Math. Univ. Paris VII 2, 45–79 (1978) 53. Lion, G., Vergne, M.: The Weil Representation, Maslov Index and Theta Series. Birkhäuser, Boston (1980) 54. Lipsman, R.: Attributes and applications of the Corwin–Greenleaf multiplicity function. Contemp. Math. 177, 27–46 (1994) 55. Mackey, G.W.: Induced representations of locally compact groups I. Ann. Math. 55, 101–139 (1952); Mackey, G.W.: Induced representations of locally compact groups II. Ann. Math. 58, 193–221 (1953) 56. Mackey, G.W.: The theory of unitary group representations. Chicago Lectures in Math. (1976) 57. Moore, C.C.: Representations of solvable and nilpotent groups and harmonic analysis on nil and solvmanifolds. Proc. Symp. Pure Math. 26, 3–44 (1973) 58. Pedersen, N.: On the infinitesimal kernel of irreducible representations of nilpotent Lie groups. Bull. Soc. Math. France 112, 423–467 (1984) 59. Penney, R.: Abstract Plancherel theorem and a Frobenius reciprocity theorem. J. Funct. Anal. 18, 177–190 (1975) 60. Pjatetskii-Shapiro, I.I.: Geometry of Classical Domains and Theory of Automorphic Functions. Gordon and Breach, New York (1969) 61. Pjatetskii-Shapiro, I.I.: On bounded homogeneous domains in an n-dimensional complex space. Izv. Acad. Nauk 26, 107–124 (1962) 62. Pontrjagin, L.: Topological Groups. Gordon & Breach Science Publishers (1966) 63. Poulsen, N.S.: On C 1 -vectors and intertwining bilinear forms for representations of Lie groups. J. Funct. Anal. 9, 87–120 (1972) 64. Pukanszky, L.: Leçon sur les représentations des groupes. Dunod, Paris (1967) 65. Pukanszky, L.: Representations of solvable Lie groups. Ann. Sci. École Norm. Sup. 4, 464–608 (1971) 66. Raïs, M.: Représentations des groupes de Lie nilpotents et méthode des orbites, Chap 5. In: d’Analyse Harmonique. Cours du CIMPA (1983) 67. Rossi, H.: Lectures on representations of groups of holomorphic transformations of Siegel domains. Lecture Notes, Brandeis University (1972) 68. Rossi, H., Vergne, M.: Representations of certain solvable Lie groups on Hilbert spaces of holomorphic functions and the application to the holomorphic discrete series of a semisimple Lie group. J. Funct. Anal. 13, 324–389 (1973) 69. Saito, M.: Sur certains groupes de Lie résolubles, I. Sci. Papers College Gen. Ed. Univ. Tokyo 7, 1–11 (1957); Saito, M.: Sur certains groupes de Lie résolubles, II. Sci. Papers College Gen. Ed. Univ. Tokyo 7, 157–168 (1957) 70. Satake, I.: Talks on Lie Groups (Japanese). Nihonhyôronsha (1993) 71. Satake, I.: Talks on Lie algebras (Japanese). Nihonhyôronsha (2002) 72. Spanier, H.: Algebraic Topology. McGraw-Hill, New York (1966)

458

References

73. Sternberg, S.: Lectures on Differential Geometry. Prentice-Hall, Englewood Cliffs (1964)/McGraw-Hill, New York (1966) 74. Sugiura, M.: Theory of Lie groups (Japanese). Kyôritu Pub. Co. (2001) 75. Takenouchi, O.: Sur la facteur-représentation des groupes de Lie de type (E). Math. J. Okayama Univ. 7, 151–161 (1957) 76. Treves, F.: Topological Vector Spaces, Distributions and Kernels. Academic, New York (1967) 77. Vergne, M.: Étude de certaines représentations induites d’un groupe de Lie résoluble exponentiel. Ann. Sci. Éc. Norm. Sup. 3, 353–384 (1970)

List of Notations

Chapter 1. Section 1.1. G; H (Lie group), g; h (Lie algebra), ŒX; Y  (Lie bracket), exp (exponential map), Ad (adjoint representation of Lie group), ad (adjoint representation of Lie algebra), D k g (derived sequence), C k g (descending central series), gC (complexification), T ./, T .ad/, CB.X; Y / Section 1.2. U.g/ (universal enveloping algebra), Uq .g/, ˙ (principal anti-isomorphism), S.g/ (symmetric algebra) Section 1.3. kAkHS (Hilbert–Schmidt norm), P Tr.A/ (trace), H , O (unitary dual),  '  (equivalence), G i i (direct sum), R  d.x/ (direct integral) X x Chapter 2. Section 2.1. Cc .G/, kf k1 (uniform norm), V, supp.f / (support), .s/ (left translation), f g (convolution), G .g/ (modular function) Section 2.2. Lp .G/, .s/ (right translation) Section 2.3. B.X /, .f /, BLG .H; H0 /, C  .G/ (C  -algebra), k  kop (operator norm) Chapter 3.

H Section 3.1. G;H .h/, E.G=H /, RG=H , QG=H , G=H k.x/dG;H .x/ Section 3.2. E.G=H; /, L2 .G=H; /, ;H D indG H  (induced representation), CB.G=H /, M ./ Section 3.3.  a Section 3.4. jC (restriction)

Chapter 4. Section 4.1. Rn (abelian Lie group), 'O (Fourier transform), Sn Section 4.2. Hn (Heisenberg group) © Springer Japan 2015 H. Fujiwara, J. Ludwig, Harmonic Analysis on Exponential Solvable Lie Groups, Springer Monographs in Mathematics, DOI 10.1007/978-4-431-55288-8

459

460

List of Notations

Section 4.3. ˙ Section 4.4. G (Grélaud group), g Chapter 5. 

Section 5.1. Ad (coadjoint representation), G.f / (stabilizer), g.f /, Bf , a?;g , a? (annihilator), af , S.f; g/, M.f; g/, P .f; G/, d, e, J (complex structure), Sf , P C .f; G/, f (unitary character), .f; f ; p; G/ (holomorphically induced representation), H.f; f ; p; G/ Section 5.2. .vj /j (co-exponential sequence), Ev , EB .w/, I g=h , E.2/, X g Y O Section 5.3. pker./, .f; O h; G/, H.f; h; G/, z (centre), a (minimal non-central ideal), I.f; g/, m.f; g/, F (Fourier transformation), O D OG (Kirillov–Bernat mapping), .g/ (pedestal) Section 5.4. Th2 h1 (formal intertwining operator), ij k (Maslov index) Chapter 6. Section 6.1. Z (Jordan–Hölder basis), I ` (index set), dO (invariant measure), EQ X , EY , EY;H , EG=H , `;h , `;h Section 6.2. PD.V /, X ˛ , D ˛ , S.V /, w.x/, !.x/, H1 , S.G=H; /, SK.G=P; ` / Section 6.4. d O (invariant measure) Section 6.5. QI , Imin , gmin , vgen , j .j 62 Imin / Chapter 7. Section 7.2. .g; j; ˇ/ (exponential Kähler algebra), 1 1 1 .g; j; !/ (exponential j -algebra),  2 .˛m ˛k / ,  2 ˛m ,  2 .˛m C˛k / Section 7.3. .abc/x Section 7.5. D.˝; Q/ (Siegel domain), H.D; ; ˚/ Section 7.7. U.f; h/, c.; f; h/, V .f; h/ Chapter 8. Section 8.1.  (monomial representation),  (affine space), m./; n ./ (multiplicity) Section 8.2. jK (restriction), p.g; k/ (projection), 1 ˝ 2 (tensor product) Chapter 9. Section 9.1. ej .`/, e.`/ (dimension index), E, Ue (layer), S.e/ (jump index set), T .e/ (non-jump index set), D .G=H / (algebra of invariant differential operators), a (kernel), U.g; /, CŒ H Section 9.2. M.gC ; b; /, .; b; gC /, Gr.gC ; r/ (Grassmannian manifold), Gr.gC /, Prim.U.g//, I./, ker , CŒX1 ; : : : ; Xp  (polynomial ring), T .eH /, U.e/, Z.g;  /

List of Notations

461

Chapter 10. K;  Section 10.1. H1 , H1 Section 10.2. ak Chapter 11. f

Section 11.1. ı (Dirac distribution), H Section 11.2. U.f; h/, c.˝; f; h; g/ Section 11.4. .; U /, ˝ Chapter 12. Section 12.1. fA; Bg, Tr .A/ Section 12.2. 'A .`/, CD .G=H /, UC .g; /, O.P; V; K; L/, S.e.P; V; K; L//, T .e.P; V; K; L// Chapter 13. Section 13.1. c.M; N /, U .g/k , D .G/K Section 13.3. dK ./; dK0 ./, dK ; dK0 , Z, c.k/

Index

Symbols C  -algebra, 47 C 1 -vector, 182 EY , 169 Imin , 201 ˝-hermitian form, 246 P .g/, 179 PU .g/, 179 n-admissible, 124 e-central element, 317 v Z , 201 v gen , 201 (finite-dimensional) representation, 4 (left) Haar measure, 31 (real) Lie algebra, 1 (real) Lie group, 1 ax + b group, 95

A abelian, 12 adjoint representation of g, 10 adjoint representation of G, 5 analytic subgroup, 12 analytic submanifold, 12 automorphism group, 4 automorphism group of g, 4 automorphism of G, 4

B Banach algebra, 38 Bonnet’s Plancherel formula, 372 Borel measure, 29

bounded approximate identity, 40 bracket product, 3

C Campbell–Hausdorff product, 15 canonical coordinate, 7 canonical irreducible decomposition, 28 CCR group, 158 CCR representation, 158 centralizer, 151 centre, 10 co-exponential sequence, 125 co-isotropic, 441 coadjoint orbit, 153 coadjoint representation, 117 coexponential basis, 135 commutative, 12 commutativity conjecture, 319 complete orthonormal system, 25 completely solvable, 130 complex Lie algebra, 1 composition formula, 166 composition series, 14 Corwin–Greenleaf function, 393 cyclic, 44

D direct integral, 28 direct sum, 27 dual cone, 246 dual space, 45

© Springer Japan 2015 H. Fujiwara, J. Ludwig, Harmonic Analysis on Exponential Solvable Lie Groups, Springer Monographs in Mathematics, DOI 10.1007/978-4-431-55288-8

463

464 E Engel’s theorem, 13 enveloping algebra, 21 equivalent, 27 exponential j -algebra, 214 exponential algebra, 130 exponential group, 130 exponential Kähler algebra, 214 exponential map, 4

F Fell topology, 158 Frobenius reciprocity, 315

G general position, 201 generalized vector, 333 Grélaud’s group, 104 group algebra, 37

H Heisenberg group, 85 Heisenberg triplet, 336 Hilbert–Schmidt class, 26 Hilbert–Schmidt norm, 26 holomorphically induced, 123 homomorphism of Lie algebra, 4 homomorphism of Lie groups, 4

I ideal, 12 induced representation, 61 Induction in Stages, 66 inner automorphism, 5 integral, 122 integral curve, 3 intertwining operator, 26 invariant subspace, 26 irreducible, 26 isomorphic as Lie algebra, 4 isomorphic as Lie group, 4 isomorphism, 4 isotropic, 117

J Jacobi identity, 1 Jordan decomposition, 18 Jordan–Hölder sequence, 14

Index Jordan–Hölder basis, 168 jump index, 317

K Kirillov’s character formula, 196 Kirillov–Bernat mapping, 158

L layer, 317 left translation, 2 left-invariant, 2 Lie subalgebra, 4 Lie subgroup, 12 Lie’s theorem, 16 local one-parameter subgroup, 3

M Mackey’s imprimitivity relation, 62 Mackey’s imprimitivity theorem, 73 Malcev sequence, 14 Maslov index, 163 minimal non-central ideal, 139 modular function, 33 monomial, 61 monomial representation, 61 multiplicity function, 28

N nilpotent Lie algebra, 13 nilpotent Lie group, 13 non-degenerate, 41 non-jump index, 317 non-saturated, 290 normalizer, 13

O one-parameter subgroup, 3

P passes through, 168 pedestal, 150 Penney’s Plancherel formula, 343 Plancherel formula, 208 Poincaré–Birkhoff–Witt theorem, 23 polarization, 118 polynomial conjecture, 319 positive, 119

Index prime ideal, 321 principal anti-isomorphism, 24 projective kernel, 136 proper, 246 Pukanszky condition, 120, 159

Q quotient Lie algebra, 12

R rational generators, 393 real, 119 representation space, 26 right translation, 2 right-invariant, 3 root, 17

S saturated, 290 Schur’s lemma, 44 Schwartz kernel, 188 Schwartz space, 179 Siegel domain of type II, 247 solvable, 12 solvable Lie algebra, 12 solvable Lie group, 13 stabilizer, 70 strong Malcev sequence, 14 strongly n-admissible, 124 support, 29

465 support S of , 50 symmetrization map, 24 symplectic, 214

T tangent space, 1 tangent vector, 1 tangent vector of curves, 3 tensor product, 315 totally complex, 125 trace, 26 trace class, 26 transcendental basis, 393 transitivity property, 58

U unipotent, 167 unitary character, 61 unitary dual, 27 unitary representation, 26

V vector field, 2 vector group, 35 Vergne polarization, 162

W weight, 17