Remarks concerning hyperbolic sets

Journal of Mathematical Sciences, Vol. 78, No. 5, 1996

REMARKS CONCERNING

H Y P E R B O L I C SETS

D. V. A n o s o v

UDC 517.987.5, 517.938, 515.168.3

1. The terminology and notations used in this article, as well as the background material, correspond to that in [1]. Naturally, all this can be found in many other papers and articles on the theory of smooth dynamical systems dealing with hyperbolic sets. Only for one detail can the reader encounter in the literature two variants that are in equal circulation. For flows, our stable and neutrally stable manifolds WS(x), W'~(x) are sometimes said to be strongly stable and stable and are denoted by W~(x) and W'(z) respectively. The same refers to unstable manifolds. We shall speak of a hyperbolic set A of the smooth dynamical system {g~}, a flow or a cascade, defined on some smooth manifold M. The questions of interest to us are semilocal and refer, in the final analysis, only to A itself and to what goes on in its neighborhood. Since we are not interested in what goes on far from A, M may be not compact and gtx may be defined, for certain x, not for all t. In the case of a cascade {g'~}, the mapping g may be defined only on some open s e t / / C M. In the case of a flow, the phase velocity vector field v in meant to be definite (and smooth) throughout M, but if M is not compact, then it is quite possible that for certain Xo E M the maximal interval of existence of tile solution x(t) = gtxo of the system = v(x) with the initial value x(0) = x0 may differ from R. The pairs (z, t) for which gtx has sense form an open subset :D C M x • that contains M x 0 but may differ from M x R for a noncompact M. However, by definition, gtx must have sense for all t E Z or t E R (and, of course, must belong to A) for all points x of the invariant, to be more precise, bilaterally invariant, set A (no matter whether it is hyperbolic or not). In the case of a cascade, it is sufficient to require that A C Lt n gu, gA C A, g-lA C A. In the case of a flow, what we have said means that A • R C :D. Under the additional condition of compactness of A, it is sufficient to require that for every x E A there should be r > 0 such that gtx be defined for all Itl < r and that gtx E A for these t. In the case of a hyperbolic A in which we are interested, it also follows from x E A that gty is defined for y E W~(x) for all t _> 0 and, when y e W~(z), for all t < 0. When, in this article, we speak of a stable or unstable manifold W~(x) or W"(x), we always have in mind that z e A. As is known, the simplest example of a hyperbolic set is a set consisting of a finite number of hyperbolic periodic trajectories. But it stands to reason that this concept was introduced not for the sake of this trivial example, but for the sake of objects like Smale's horseshoe in which the number of trajectories is infinite and even has the cardinality of continuum. It is well known that in these "real" hyperbolic sets the dynamics is much more complicated. This article adds a certain contrast of the properties of hyperbolic sets consisting of an infinite number of trajectories to the properties of the trivial example we have cited. In Sec. 3 we briefly discuss the "intermediate" case of a hyperbolic set A consisting of an infinite but countable number of trajectories. If such A is locally maximal, then its structure is very simple and in this respect the present case is certainly "not true." Now if A is not locally maximal, then its structure may be more complicated. I do not deal here with a detailed analysis of these A (although, in principle, such an investigation could have been carried out; perhaps it could have even yielded some results of a classificational nature). But it seems to me that the remarks given in this article present a sufficient answer to the natural question as to the role played by the "intermediate" case, showing that this case is not "true" either as concerns its complexity. 2. We say that the stable manifold Ws(z) is periodic with period r if g~W~(x) C Ws(Z). If, for any y E W~(x), gty are defined for all t, then the periodicity of W'(x) is equivalent to the fact that g~W'(x) = W'(z). Here we must check that if W'(z) is periodic and y e Ws(z), then g-*y e W'(z). It is clear from Translated from Itogi Nauki i Tekhniki, Seriya Sovremennaya Matematika i Ee Prilozheniya. Tematicheskie Obzory. Vol. 13, Dinamicheskie Sistemy-1, 1994. 1072-3374/96/7805-0497515.00 9

Plenum Publishing Corporation

497

the definition of periodicity that g~'x E W'(x), and therefore the distance

p(gty,gtg'~x) --* 0 as t --~ or This means that

p(gtg-~'y,

g'x) =

p(g'-*y, gt-*g'~x) --* 0

as t ~

oo,

so that, indeed, g-~y E W'(x). The periodicity of the unstable manifold W"(z) is defined as follrws: g-*W"(x) C W"(x). If, for all y e W~(x), g*y are defined for all t, then the periodicity of W"(x) is equivalent to the fact that g*W"(x) = W~(x). (We first make sure that g-~W~(x) = W~(x). This is similar to the statement we have made above and can even be reduced to it by means of time reversal.) If x is a periodic point, then the manifold WS(x) is periodic (with the same period). The converse is certainly not true, but the following theorem is valid.

Suppose that A is a hyperbolic set, x E A and W~(x) (or W"(x)) is periodic with period r. Then WS(x) (W~(x) respectively) passes through a periodic point y e A which has the same period v (of course, in this case, W'(x) = WS(y) or W"(x) = W"(y)). Theorem

1 . ( O n a p e r i o d i c s t a b l e or u n s t a b l e m a n i f o l d )

In the trivial cases of a hyperbolicset A consisting of a finite number of hyperbolic periodic trajectories,

W'(x) are different for different x E A, and the same is true of W ". Indeed, two distinct periodic points x and y cannot lie on the same stable (unstable) manifold: if we have, s~y, y E W'(x), then it would follow that limt_.~ p(g*x,gty) = 0, so that the point gty would not be able to return periodically to the initial position. The situation proves to be different in all other cases. T h e o r e m 2. If the hyperbolic set A consists of an infinite number of trajectories or if it consists of a finite number of trajectories but they are not all periodic, then there are trajectories L~, L2 C A such that (W'(LI) \ L1) M A ~ g, (W"(L2) \ L2) M A ~ ~. Remark.

It is obvious that if we take the point

y e (W'(L,) \ Lx)N A, in the same notations, we shall find that it lies on some W'(x), x E L1. Thus, the stable manifolds of two different points x, y E A coincide. Conversely, suppose it is known that there are two points z ~ y in A such that W'(x) = WS(y). Can we infer that the stable smooth manifold W'(L~) of a certain trajectory from A also contains another trajectory L ~ C A? Clearly, it is obviously so if one of the points x, y (say, x) is periodic, since then the other point, y, cannot be periodic and its trajectory L' is just the trajectory from A lying in W'(LI), where L~ = {gtx}, with L' ~ L1. Let us consider the case when neither x nor y is periodic. We denote by L1 and L r the trajectories of these points. It is clear that L ~ C W'(L1) as before, but this time it is not inconceivable that L ~ = L1. However, if L ~ = L1, then y = g*x or'z = g*y with a certain r > 0, and then the stable manifold W'(x) = W'(y) is periodic. According to Theorem 1 it passes through some periodic point z E A. Consequently, L~ = L' C W'({gtz}). In this case, the trajectory L~ = L' is not periodic and {gtz} is a periodic trajectory and so these trajectories are different. Thus the inference remains valid. What we have said remains true for unstable manifolds as well. 3. If the hyperbolic set A consists of an infinite number of trajectories, then it contains a nonperiodic trajectory. Theorem

This theorem follows from Theorem 2 since, as was mentioned, if L is a periodic trajectory, then neither W'(L) nor W~(L) can contain any other periodic trajectories except for L. This means that if A consists of periodic trajectories, then

(W'(L1) \ L1) N A = z, 498

(W~'(L2) \ L2) N A = o

for any of its trajectories L1, L2. However, we shall prove Theorem 3 before we complete the proof of Theorem 2 and, conversely, use Theorem 3 when completing the proof of Theorem 2. Theorems 1, 2, 3 will be proved in See. 8. Somewhat earlier,, in Sees. 4-7, we give some auxiliary "technical" material. A considerable part of this material is more or less well known, but I think it expedient to present the material successively, proceeding from universally known facts. Not mentioning the completeness of the exposition, I believe it to be more convenient for the reader than a number of references to papers in which, explicitly or implicitly, some more or less equivalent considerations appear in passing. Incidentally, what is said in Sees. 4, 6, 7 may be not without some new elements as concerns the methods of exposition. The treatment of the behavior of trajectories in the neighborhood of a fixed'trajectory {gtz} from the hyperbolic set A given there (with due account of the "uniformity" with respect to all {gtx} C A) differs from the treatment I gave earlier [2] and seems to be more convenient, at least for the purposes of the present article. (Maybe not only for them. However, in this article the exposition is constructed so as to achieve these purposes only.) It also seems to me that the construction of the Lyapunov metric (See. 5) for A y~ M, in the case of a flow, is formally new. 3. The set of trajectories lying in the hyperbolic set A is either finite or countably infinite, or has the cardinality of continuum. (And this is certainly also true in a considerably more general situation.) As to periodic trajectories, it contains a countable number of them (it follows from the property of separation of trajectories and compactness that the number of periodic trajectories with period _< T is finite for any T). This means that if the set of trajectories lying in A has a cardinality of continuum, then the "majority" of trajectories are nonperiodic. We do not need Theorem 3 in this case. And what happens when A consists of an infinite countable number of trajectories? According to Theorem 3, in that case there is at least one nonperiodic trajectory. The following example shows that we cannot state anything more. The example is based on the use of symbolic dynamics. It speaks directly of a closed invariant subset a of the Bernoulli topological cascade {~ri} acting on the-space f/2 of two-sided infinite sequences {a(i)} of symbois 0, 1. As is known, the Bernoulli topological cascade is realized as a hyperbolic set of a certain smooth cascade (this is done, for instance, with the aid of Smale's horseshoe). "Thus, closed invariant subsets of the Bernoulli topological cascade also give some examples of hyperbolic sets. In the example, A consists of the trajectories of the points a l , . . . , a,,,.., and b:. al = {al(i)}, where all at(i) = 0; a,, = {a,~(i)}, where a,(i) = 1 if i is divisible by n and a,(i) = 0 otherwise, b = {b(i)}, Where b(0) = 1, and all other b(i) = O. A contains an infinite number of periodic trajectories (each of the points a,~ is periodic with period n). In addition, A includes only one nonperiodic trajectory, the trajectory of the point b, which tends to al in both directions with respect to time. It should also be pointed out that periodic trajectories are dense everywhere in A so that the center (in the Birkhoff sense) of the cascade {ailA) (i.e., the closure of the set formed by the Poisson stable trajectories) coincides with the whole A. This example also shows that in the conclusion of Theorem 2 the trajectories Lx and L2 may coincide. In this example, the hyperbolic set A is not locally maximal. If A is a locally maximal hyperbolic set consisting of a countable number of trajectories, then it contains only a finite number of periodic trajectories. Indeed, if there were an infinite number of such trajectories, then, for any e > 0, there would be two different periodic trajectories L1 and L2 the minimum distance between whose points would be smaller than e. Then we can form an e-trajectory l = {x(t)) which coincides with L~ for t < 0, then "passes" on to L2 and revolves ones along it, and then again "passes" on to L1 and coincides with La further on its way. According to the theorem on the family of e-trajectories (actually, in this case the family reduces to only one e-trajectory), there is a "true" trajectory L = {g~x} close to this e-trajectory. (The proximity in the case of a cascade means the smallness of the distance p(gtx, x(t)) for all t and in the case of a flow the smallness of the Fr~chet distance between l and L as parametrized curves.) Since A is locally maximal, L C A. A certain negative 499

and a certain positive semitrajectory of the trajectory L remain close to L1 all the time. As is known, it follows that L C W"(L1) N W'(L~). (Incidentally, this also follows from what will be said in Sec. 7 of this article.) In this case, L ~ L1 since the point gtx, moving along L, recedes for some time from L1 moving close to L2. (Here again we speak not of the Hausdorff distance between L and L1 as sets of points - - it may be small (say, this is obviously so when L1 and L2 are "sufficiently dense" in A) - - but of p(gtxl,gtx) with some Xl E L1 in the case of a cascade and of the corresponding Fr~chet distance in the case of a flow.) Thus A contains a (transversal) homoclinic trajectory L. (The transversality of the intersection of W'(L1) and W"(LI) along L follows from the fact that W"(L~) = W"(L) and W~(L1) = W'(L) and so we speak of the transversality of the intersection of W"(L) and W'(L) along the trajectory L C A.) As is known, in that case, an arbitrarily small neighborhood of the closure L contains a hyperbolic set B which has a well-known symbolic description; from the latter we can see, in particular, that B contains a continuum of trajectories. But B C A since A is locally maximal. It follows furthermore that for a locally maximal hyperbolic set A consisting of a countable number of trajectories all nonwandering trajectories of the dynamical system {gtlA } are periodic. Indeed, we know that the periodic trajectories are dense everywhere in the set of nonwandering points of this system, and there is a finite number of trajectories of this kind. Finally, the theorem on the spectral decomposition "degenerates" into a statement that every trajectory L C A tends to a certain periodic trajectory Li C A as t --* - o o (i.e., L C W"(L~)) and to a certain periodic trajectory nj C A as t ---* oo (i.e., L C W'(Lj)), and the binary relation "Li > Lj, provided that there exists a trajectory L C A n W"(Li) fl W'(Li) '' is a strict-order relation on the set of the periodic trajectories from A. We can see that the structure of the locally maximal hyperbolic sets A consisting of a countable set of trajectories is very simple, essentially, as simple as in the case of a finite number of trajectories. Therefore, it is expedient to give an example showing that the number of trajectories in such an A can be infinite. We again give an example in terms of symbolic dynamics. In this example, the closed invariant set A C f~2 consists of the trajectories of the following points a, b, c, p, q, r, 8 1 , . . . , s,,,...: a = {a(i)), where all a(i) = O, b = {b(i)}, where b(i) = 1 for even i and b(i) = 0 for odd i, c= {c(i)), where all c(i) = 1, p = {p(i)}, where p(i) = 1 for even i > 0 and all other p(i) = O, q = {q(i)}, where q(i) = 0 for even i _< 0 and all other q(i) = 1, r = {r(i)}, where r(i) = 0 for i < 0 and r(i) = 1 for i _> 0, 8,, = {s,,(i)}, where s,(i) = 0 for i < 0 and for odd i = 1,... ,2n - 1 and all other a,(i) = 1. In- a hyperbolic set A which consists of a countable, even finite, number of trajectories but is not locally maximal, the limiting behavior of trajectories may be more complicated, namely, they (not necessarily) all tend to periodic trajectories. Here are some examples, "symbolic" again. Let us consider in f/2 points a0 = {a0(i)},..., a, = {an(i)} defined as follows. All ao(i) = 0; al(0) = 1 and all other ax(i) = 0. Furthermore, let k = 2 , . . . , n . If i < 0, then all ak(i) = 0. The symbols ak(i) with i > 0 written out in succession are divided into blocks, ak(0)ak(1).., ak(i)... = Bk(1)Bk(2)... Bk(j)..., which are defined as follows. We denote the block consisting of k symbols c by c k. Then B2(j) = 0~10j

(j = 1,2,...),

and if the blocks Bk-l(j) have already been defined, then Bk(1) = Bk-l(1), 500

Bk(2) = B k - l ( 1 ) B k - l ( 2 ) , . . . ,

Bk(j) ----Bk-l(1)Bk-l(2)... Bk-l(j), . . . . We denote by Ak (k = 0 , . . . , n) a set formed by the trajectories of the points n o , . . . , a,, under the action of the Bernoulli topological cascade {crl}. The point ao is fixed, and therefore the a- and w-limit sets of fts trajectory coincide with {a0} = Ao. Now if k > 0, then, as we can easily verify, the a-limit set of the trajectory {ainu} also reduces to Ao, whereas its w-limit set coincides with Ak-1. Moreover, denoting by NW({gt}) the set of nonwandering points of the dynamical system {gt }, we have

N W ( { f l A o } ) = Ao,

N W ( { f l A k } ) = Ak-1

(k = 1,...,n).

Recall the definition of the (Birkhof]) center C({gt}) of the dynamical system {gt}. We set

Ox = N W ( { g t } ) ,

~2 = N W ( { f ] ~ x } ) , . . . , =

N

~,, = N W ( { g ' l f l , - a } ) , . . . ,

=

k<w

(transfinite induction). If A is a metric compact set, then there is a least transfinite number a (not more than countable) for which fl= = ~2~+x . . . . . This set is precisely C({gt}) and a is the depth of the center. (In old papers a less expressive name "the ordinal number of central trajectories" was used. The center admits of other, equivalent, definitions [3].) In our example, gtj = A,_ i for j < n, C({ailA~}) = A0, and the depth of the center is n. In a hyperbolic set consisting of a countable number of trajectories the depth of the center may also be infinite. Here is an example of the corresponding subset in f/2- Let us consider the following points hi, h2, hi, b2, e l , C 2 , . . . , C n , - . . : al = {al(i)}, where all al(i) = 0; a2 = {a2(i)}, where all a2(i) = 1; b~ = {bl(i)}, where bl(i) = 0 for i < 0 and b~(i) = 1 for i > 1; b2 = {b2(i)}, where b2(i) = 1 for i < 0 and b~(i) = 0 for i > 1. Finally, for i < 0 all ck(i) = 0 and the symbols ck(i) with i _> 0 written out in succession are divided into blocks,

c,(i) . . . .

cdl)cd2).., c d j ) . . . ,

which are constructed as follows: Ok(l) = 1k Ck(2) = 0Ck+,(1),

Ck(j + 1)

=

(k = 1,2,...);

Ck(3) = 02Ck+l(l)Ck+l(2),...,

OJCk+l(1)Ck+l (2)... Ck+~(j), ....

Let A be the set {al,a2}, B be obtained by attaching the trajectories of the points bl, b2 to A, and Ok, k = 1, 2,..., be obtained by attaching the trajectories of the points cj, Mth j > k, to B. Then NW({a~[Ck}) = Ck+1 so that for the cascade {cri[Ca}

~'~1 = C 2 , . . . ,

~~j = C j + I , . . . ,

~~w = B,

~"~a+l =

A = C({ai[C~}).

The depth of the center is w + 1. These examples have something in common with Maier's examples of flows with an infinite depth of the center in R s. (See [4, 5]; a part of Maier's constructions is reproduced in [6]. In fact, Maler himself realized some of his examples not in R3 but in the space of unit tangent vectors of a closed surface of negative curvature9 It is indicated in [7] that they can also be realized in R 3 (and with smoothness of class C ~ at that)). To be more precise, they have something in common with the "symbolic part" of Maier's constructions 501

(we have naturally nothing to do with a witty technique which he invented to pass to a flow in ~3). The "symbolic" examples given above correspond only to the simplest Maier's examples. I do not doubt that the "symbolic part" of his more complicated examples (in which the depth of the center exceeds the specified countable transfinite number or which refer to other (nonequivalent) variants of the depth of the center) can also be presented as that referring to closed invariant subsets of the Bernoulli topological cascade consisting of a countable number of trajectories. The first example given in this section also testifies to some complication of the dynamics that may occur when we abandon the local maximality of A (and retain the condition that this hyperbolic set should consist of a countable number of trajectories), namely, there is a nonperiodic trajectory in the Birkhoff center of the system {gt[A}. But in another (and, I think, more important) respect, the dynamics in this case remains if not very simple, then not very complicated: only periodic trajectories can be Poisson stable (one-sided and two-sided) in A. This is not connected with hyperbolicity. Suppose that {gt} is a topological flow or cascade in a complete metric space and L = {9tx} is a Poisson stable (one-sided and two-sided) trajectory. If its closure L consists of a countable number of trajectories, then L is a periodic trajectory. The rephrasing of the well-known arguments going back to Birkhoff (who presented them in connection with some other case) may serve as a proof. To make the exposition more complete and convenient for the reader, I shall give the proof (especially since I do not know whether there are similar arguments in the literature given in the form we now need). From the fact that L is nonperiodic but Poisson stable, it follows that any finite arc of any trajectory in L is nowhere dense. Indeed, let us suppose that the arc I = {9'Y; ~ e [a, b]} is dense somewhere in L, i.e., that there is a ball v =

e L; p(z, u) <

e L,

in L in which the set U ~ I is dense. Then the closure ] contains the whole of this ball (and even its closure). But the set I is compact and therefore ] = I. Thus U C I. There is a sequence t,, ~ co or tn ~ - c o such that 9t"x ~ u. For sufficiently large n, say, n > N, all gt"x E U C I so that gt"x = g'"y for some s,, E [a, b]. Let n, m > N and It,,, - t.I > b - a. Then it follows from

gt,, x that

gt'x

=

gs=~,

gt,~x

__

gS,~t

= g ~ ' u = g ~ ' - ~ " g ~ " u = g~"-~"gt'=x = g t " + ~ ' - ~ " x ,

gtm-t"+sn-smX

=

X.

But and we find that {gtz} is a periodic trajectory. If L now consists of a countable number of trajectories, say, of the trajectories of the points z,~, then L coincides with the union of a countable number of finite arcs =

Itl _ m}.

But L is a complete metric space and, according to the Baire theorem, cannot be represented as the union of a countable number of subsets not dense anywhere. 4. This section and the next are not interconnected. They are both preparatory for Sec. 6 whose results are necessary for what will follow. In this section, the set A is assumed to be compact and invariant; its hyperbolicity is not necessary for the time being. (However, in the final analysis the results of this section are-used only for hyperbolic A.) 502

By analogy with ordinary functions, the map f : X ~ Y between metric spaces (with metric p) is known as a Lipschitzian map (satisfying the Lipschitz condition) if there is C such that p(fz, fy) < Cp(x, y) for all x, y E X. C is known as a Lipschitz constant for f. (A Lipschitz constant for f is not uniquely defined, i.e., if C is a Lipschitz constant for f, then any C' > C is also a Lipschitz constant.) A Lipschitz constant for several maps fi is a C which is a Lipschitz constant for every fi. f is said to be locally Lipschitzian if every point x has a neighborhood U= such that the restriction fJU= is Lipschitzian. If X is compact, then the locally Lipschitzian map f : X ~ Y is Lipschitzian. Indeed, then there is a finite open covering {Ui} of the space X such that all flUi are Lipschitzian with constants Ci, and this means that they have a common Lipschitz constant C = max Ci. According to the well-known Lebesgue lemma, there is r > 0 such that if p(x,y) < r, then there is a Ui containing x and y. Then p(fx, fy) < Cp(z,y). On the other hand, there is C' such that p(fx, fy) < C' for any x, y since the image f X is a compact and, hence, a bounded subset of Y. Now it is clear that

p(x, y) <_ m a x (C, C'/r)p(x, y) for any x, y E X. Suppose that f : N ~ M is a smooth map between Riemannian manifolds (for simplicity we consider the R.iemannian metric to be of class C~176 denote the norm of the vector by II " ]l and the distance by p) and K C N is a compact subset. Then fIK satisfies the Lipschitz condition. Indeed, it is sufficient to verify only that it is locally Lipschitzian. But for every point Xo E K there exists a chart (U, qa) (U is a coordinate neighborhood and qa: U ~ R", n = dimN, is coordinate map which associates to the point x E U its local coordinates) such that if x, y E U, then c,l x -

oyl < p(x, y) <

-

oyl for

y 9 u

(4.0

with certain Cl, c2 > 0. (Here J. l is a standard norm in. R".) We .can also suppose (diminishing U when necessary) that ~ U has an especially simple form, say, is a ball or a cube, and that fU entirely lies in some neighborhood V of the point fXo, where there are local coordinates for which the analog of (4.1) is satisfied. Therefore, in terms of local coordinates, the fact that f is locally Lipschitzian reduces to an elementary fact of analysis. Let A be a compact invariant set of a smooth cascade {g"} on M..There is rl > 0 such that the closure Urt(A) of the rt-neighborhood of A is compact (then it is so for any Ur(A) with r 9 [0, r~]). There is r2 > 0 such that ~-r2(A) C L/rh gH (U was introduced in Sec. 1). We can take it that r2 _< r~. Let C be a common Lipschitz constant for gJ0r2 CA) and g-~ IU,2(A). Slightly increasing C, we can assume that

p(g ,ay),

p(g-'x,g-'y)< cK ,y)

for x 9 Ur2(A) and x # y. Note that this C > l:since

y) =

< cp(ax, ay) < c p(x, y),

and that g(],(A) C ~rcr(A), g-'C)'~(A) C (.rc,(A) for r < r2/C. We introduce the notations

B,(z) = {v 9 T=M; Ilzll < r},.

B,(A) = {v 9 TMIA; I]vll < r}.

There is rz 9 (0, r2] such that for any x 9 A the exponential geodesic map exp IB,3(x ) is defined (throughout B,3(x)) and is a diffeomorphism B,3(z ) ~ U,3(z). We set ro = r3/C. Then, for every r 9 (0, to), the map F: &(A) ~ Bc~(A) that maps v 9 into F=(v) 9 Bc,(gx) is defined, where F=, = (exp I B c , ( g x ) ) - '

og o

(exp

(4.2)

The map F= is continuous and has a "fiberwise" Fr~chet derivative. (In accordance with the older terminology, the Fr~chet derivative in a finite-dimensional case is a full differential, but in this case it is not so "full" since 503

it is taken only o n / ) , ( x ) , and x does not vary.) This derivative (i.e., the de. =;-ative of the map v ~ F~(v) with a fixed x) is a linear map

D,F,(v) : T~M --* Tg~M. The latter continuously depends on (x, v). This statement must be explained if we do not want to use the universal incantation "in a definite sense." For any finite-dimensional vector spaces V, W the linear maps V --* W themselves form a finite-dimensional space L(V, W). If E ~ X, F ---* Y are vector bundles with finite-dimensional fibers E~, Fu, then a vector bundle L(E, F) ~ X x Y appears whose fiber over (x, y) is L(E~, Fu). (It is assumed to be known how we introduce the structure of a vector bundle in L(E, F).) In these terms (z, v) ~ D~F,(v) is the map

B,(A) ~ L ( T M I A , TMIA), and we speak of its continuity. Let Y0 = expx o Vo,

e

z0 = gxo,

Wo = gYo.

Let us take some C~176 charts (U1,T~) near Xo (this also defines the coordinates in the corresponding T,M), (U2, ~ ) near yo, (U3, T3) near Zo (in this way, we define the coordinates in the corresponding TzM), (U4, T4) near Wo. In these coordinates, the map F is locally represented as a composition of smooth maps describing (in the terms Of the coordinates being used) the maps (x, v) ~ exp~ v (for the moment we are considering them not only for x E A but also for all x 6 M close to x0),

x ~, z = gx,

y ~ w = gy,

(z, w) ~-~ exp~"1 w ( t o be more precise, (exp IBcr(z))-lw) (again for the moment we are considering all points z E M close to Zo). The continuity of D,,F,(v) with respect to (x, v) simply means the continuity of the corresponding partial derivatives. It is clear that F~(0~) = 0g~, where 0z is a zero of the space T , M , and D,,F,,(O~) = T~g (since To, exp is simply a standard identification of To.TM with T~M). We denote F~(v) - T~gv as f ( x , v) so that

S~(v) = T~gv + f ( x , v).

(4.3)

Clearly, f(x, v) has the same properties of continuity and smoothness as the properties of F~(v) indicated above. In addition, f(x,O~) = Og~ (4.4) and D,,f(x, v) = D,,F~(v) - D,,F(O,;) so that D,,f(x, 0~) = 0 (we imply the zero map T~M -+ Tg~M). Hence [[D,,f(x,v)[[ <_ wx(r), where w!(r ) is a continuous nondecreasing function of r >_ 0 equal to zero for r = 0. It follows that 9

v,) - f(--,

<

-

(4.5)

for all x E A, Vl, v2 E [~,(x). The map F~ describes the action of g near x. We can use the same construction to describe the action of g,1 near the points z E A. We may even go without changing the constants rl, r2, C, ra, to, since from the beginning we have taken them such that they suit g as well as g-1. The following maps arise: H=: /~,(x) ~ Bc,(x)

H= = (exp

(x 6 A, r 6 (0, to)),

I-~Cr(g--12~))-10 g-10 (exp I~r(Z)).

(4.6)

For them we have

(4.7) 504

h(x, 0~) = 0g-l~,

(4.8)

IIh(x, u1) - h(x, 732)[I __~ 2(r)11731 - 73~11, (4.9) where wz, analogously to wl, is a continuous nondecreasing function of r > 0 equal to zero for r = 0. Suppose now that A is a compact invariant set of the smooth floff {gt} on M. There are rl > 0 such that 0",, ( a ) i s compact and r2 E (0, rx] such that 0,,(A) x [-1,1] C 7) (we introduced 2:) in the second paragraph of Sec. 1; we supply 7:) and all M x R by a metric

p((X, 3), (y,•)) = (p2(~.,y) dl-(t -- 3)2)I/9, which, as we know, is Riemannian). Since the map

g: 7) ~ M

g(x,t) = gt(x)

is smooth, its restriction to 0~2 x [-1, 1] has a Lipschitz constant C. In particular,

p(gtx, gty) <_ Cp(x,y)

for

x,y E ~],2(A),

It[ < 1.

Slightly increasing C, we can take it that the inequality is strict for x ~ y. As before, C > 1. Finally, we introduce r3 and r0 by means of the same conditions as before. The following maps F~,t:/3~(x) ~ Bc,(gtx) are defined for every r E (0, r0) and any x E A, t E [-1, 1]:

F.,t(v) = (exp [Bc,(f x) ) -1 o g' o (exp I/~.(x)).

(4.10)

We denote F~::(v) - Tgtv as f(x, v, t) so that

F~:(v) = T=g% + f(z, 73,t).

(4.11)

We immediately note that

f(x,v,O) =F~,o(v)-T~g~

=v-v

= 0~.

(4.12)

It is clear that the map /3r(A) x [-1,1] --* Bo,(A)

(x,v,t) ~--~'F=,t(v)

is continuous and each F~,t has the same properties of continuity and smoothness as F~ mentioned above, but now D.F~:(v) is continuous with respect to (x, t, v). This is interpreted as the continuity of the corresponding map /3r(A) x [-1, 1] - . L((TMIA ) • [-1, 11, T M I A ) . (Here, in the right-hand side, [-1, 1] is regarded as a trivial bundle over itself, one point over a point of a base. Thus the fiber (TMIA) x [-1, 1] over (x, t)is T=M.) Therefore f(z, v, t) also has the requisite properties similar to those of f(x, v), but now the continuity of D J ( x , v, t) means the continuity with respect to (x, v, t). In particular, f(x,0=, t) = 0,,=. (4.13)

D,f(x,O,,t)=O

for all

zEA,

[t[
(4.14)

Moreover, we can state in a certain sense that there is a derivative D,D,f(x, v, t) continuous with respect to (z,v,t). We will not give a "global" interpretation of this statement (like that we gave above to D , F and D,f) but will only consider it to be a statement concerning a local property expressed by means of local coordinates. Let (x0, v0) E/3,(A),

to E [-1,1],

Y0 = exp,, Vo,

Zo g'xo,

Wo = g'Oyo.

Let us take smooth charts (U1, ~ol),..., (0"4, ~o4) like those we considered above. In these terms, F and T f can be locally represented as compositions of smooth mappings, the derivatives we need being continuous with respect to their arguments. Therefore, we can differentiate the right-hand side of the relation

ofiCz, v,t) Of~,t(v) OvJ OvJ

O(g'z) i OvJ 505

(in which the coordinates of the pair (x, v) in the chart (U1, qol) are again denoted by (x, v) = ( x l , . . . , v "~) and the coordinates of f , F, and gtx in the chart (U3, qoa) are denoted by F i, fi, (gtx)~) with respect to t (for (x, v, t) close to (z0, vo, to)) and the derivative is continuous with respect to (x, v, t). If we require that the smoothness of the phase velocity vector field v be of class C 2, then the mapping (x, t) ~ g~z will be ( l - s m o o t h and we can say the following. We extend somewhat the domain of definition of F and f so that it becomes a manifold, replace [~,(A), Bc,(A), [-1,1] by Br+~(B~(A)), Bc(r+~)(Bc~(A).), ( - 1 - e, 1 + e) with a very small e > 0. Then, as before, we can define F~,t as in (4.10), and then the resulting map Br+~(B~(A)) • ( - 1 - ~, 1 + ~) ~ BcC,+~)(Bc~(A)) (4.15) will be (2-smooth. True enough, the map (x, ,, ~) ~ Tg~(x)v will be only ( l - s m o o t h , but in this case the differentiation with respect to v does not decrease the smoothness, and so the map

(x, v, t) ~, n , f ( x , v, t) = n,E~,,(v) - Tg~x is a ( l - s m o o t h map

X --* L(TX, TY),

(4.16)

where X is the left-hand side and Y is the right-hand side of (4~15). One difference from the case v E C 1 is that in the last case we state not the smoothness of the mapping (4.16) (which, of course, can also be considered for v E C 1 since the extension of the domain of F and f indicated above does not require a high smoothness of v) but, so to speak, only the existence of a derivative with respect to one of the arguments (and its continuity). I would like to infer now that by virtue of (4.14), D~D,f(x, 0~, t) = 0 and therefore

IID,D.f(x,o=, t)ll<_ where w2 has the same properties as Wl above, and then, by virtue of (4.12),

II D,,f(x,v,t)II-< t,,2Cr).

C4.17)

However, these arguments do not have a definite sense, since we do not have a correctly defined geometrical quantity D~D,f(x, v, t). In this respect, the situation would not chafige if we required higher smoothness of the field-v. The most we can do is to interpret the pair

( D J ( x , v, t), D,D,f(x, v, t))

(4.18)

as a tangent vector to the right-hand side of (4.16) (incidentally, this is also possible for v E C1), but how can this substantiate the suggested "proof' of inequality (4.17)? Vector (4.18) is not small, its projection onto the base is (0, v(gtx)) E T:~M • Tg,~M. It seems that we can accurately reason with the aid of local coordinates, but this reasoning will be cumbersome (the more so that during the time from - 1 to 1 the point g~x can pass through several coordinate neighborhoods). Instead, we use a covariant derivative. Let x e A, v E B,(x), w e V~i, Ilwll = 1, t E [-1, 1]. (4.19) We introduce the designation w(t) = D J ( x , t, v)w. The covariant derivative Vtw(t) = v t ( n , F ( z , v , t ) w ) is a correctly defined geometrical object, namely, a vector from Tg,,M. It depends on x, v, w, t (satisfying (4.19)). We can see from the expression in terms of local coordinates

v,w'(t) = ='(t) + 506

d

t

k z) =

o,(..:(.,.,,).)' + v..j,(,'.)(..::.,,,.).)'o'(.'.) (l~jk axe Christoffel symbols and the field v(z) has components vk(z)) that it continuously depends on x, t, v, w satisfying (4.19). By virtue of (4.14), Vt(D,f(x, 0~, t)w) = 0 when x, t, w satisfy the conditions of (4.19) that refer to them. Therefore, there is a continuous nondecreasing function ws(r) of r > 0, which is equal to zero for r = 0, such that when x, t, v, w satisfy (4.19), then

IIV,(Dj'(~, v, t)w)ll ~ ,~3(r). As is known,

d l l •2,= (2(w(t), t ) lv,w(t)), l where (., .) is an inner product corresponding to the Pdemanniaa metric we use. Consequently, if w(t) ~ 0 everywhere on the interval w(t), then

]~tll~"(t)ll = I

i

d I[~.(t)l12 < IIV,~(t)ll < ~(.).

2llw(t)l I dt

-

-

On this interval

b,(t) - w(,)ll < ~,~(,-)lt- 4

(4.20)

By virtue of (4.12), D~f(x, v, O) = 0 so that w(0) = 0. This means that on the interval [0, t] there is at least one zero of t h e function t ~-* I[w(t)l[. Suppose that So is the largest of them. According to what we have proved, if so < t (i.e., w(t) 7t O) and s E (so, t), then (4.20) is satisfied. As s --* So, we get

b,(t)ll< ,,,~(,.)It-sol_< ,,,,(,-)t.

(4.21)

Now if w(t) = 0, then (4.21) also remains valid in a trivial way. Finally, in essence, (4.21) is precisely (4.17) since [[D,,f(x,v,t)l [ = m a x {IID,,.f(x,v,t)wll; w e T,:M, [[wl[= i}.

It follows from (4.1Z) that for 9 e A, v ~ ~,(x), ,,~~ ~,(:~), Itl_< I we have IIf(~:,vl,t) - f(~, v~,t)II-< t,,,3(,')II~,1 - "~ll-

(4.22)

Let us sum up what we have said. In order to describe the motion of the points f y which a r e close to the moving "reference point" gtx, we represent gty as expg,= v, where the variation of v with time occurs under the action of transformations (4.2), (4.6) in the case of a cascade and (4.10) in the case of a flow. These transformations are presented as (4.3), (4.7), or (4.11). On the right-hand side of the corresponding formula the first (linear) term corresponds to the tangent linear extension { T f } of our dynamical system (gt} and

relations (4.4), (4.S), (4.13) and estimates (4.5), (4.9), or (.4.22) hold for the second term. Using the classical language, we speak of the accuracy of linear approximation (i.e., variational equations) for the description of the behavior of the trajectories near gtx on a finite time interval. In fact, in the last phrase we have stated everything we need (and the rough answer we have given is well knov;'n), but we had to express the same things using another language, for all gtx, x 6 A, at once, and with due attention to the uniformity of the estimates we need with respect to the appropriate "input data" at that. 5. In Secs. 6-8 we shall use the Lyapunov metric I" I. Here are some of its properties that we shall need. There is a > 0 such that if x E A, ~ E E~, r/E E~, then

ITgt~l <_e-2"~'[~l,

ITg-t~l < e-2=tl~T[

(5.1)

for all t > 0. (I write here 2a and not a because, in what follows, I operate, for the most part, with other exponential estimates in which the exponent has to be somewhat diminished. A two-times diminishing is quite

507

sufficient, and, for this reason, I write 2~ instead of a from the very beginning m order to avoid writting a/2 later.) If {g~} is a flow with phase velocity vector field v(x), then T=M also includes a neutral subspace E~ generated by v(x). Without losing the high smoothness of the metric and without requiring a high smoothness of the flow, we cannot ensure ]v(z)[ = 1 for z E A but can only ensure that the inequalities 1 - e < Iv(•)l < 1 + e,

- ~ < d l T g ' v ( = ) l = dlvCg'=)l < e

(5.2)

be satisfied for x E A (and, hence,

lie in the indicated limits). Here e is "small." In particular, we require that e should satisfy the conditions

6 < 1/100,

~ < 5/2.

(5.3)

(An attentive verification of the arguments would probably show that e < 1/10 is suitable too, but I am not sure about 1/2 and 1/3. As concerns the smallness of e as compared to ~, in actual fact, for any preassigned > 0 there is a Lyapunov metric with e < 5a but E = 1/2 suits our purposes.) Note, in addition, that the construction of the Lyapunov metric given below also ensures an approximate orthogonality of the spaces E~ and E~ and (in the case of a flow) E=". Generally speaking, we could have attained a true orthogonality only for a continuous metric. But, without losing high smoothness, we can ensure the situation when the angles between the vectors ~ E E:, ~1 E E=", ~ E E~ are between, say, 89~ and 9I ~ We are not in a special need of this property of the metric (there are no direct references to it in Secs. 6-8), but since we get it without any additional efforts, it is natural to mention it.: In actual fact, we will have a chance to refer to the inequalities I

+ 1,712) for ~ e ~:,

8

,7 e E:,

l~ § ~ + r 2 > ~(I~I 2 + I~I2 + lr 2) for ~ e E:,

~ E EL

I,' +,712 > ~(1r

2

(5.4) r e Z~".

(5.5)

The construction of the Lyapunov metric as well as many other things are given in [2] as applied to the case A = M. Usually in a situation like this, I do not" think it is necessary to repeat myself if, as compared to [2], I only have to make small obvious changes. But, in this article, I decided to act differently. In the present section, I give the construction of the Lyapunov metric, dwelling on two circumstances. The first is that A ~t M of which I naturally could n o t speak in [2] because of the character of that book. The other circumstance was actually taken into account in [2] (it is not connected with whether A is coincident with M), but it deserves to be given special attention, w h i c h was not done in [2]. It will be indicated as the exposition goes on. Dwelling especially on these circumstances, I conformed to Littlewood's remark: usually the reader can easily fill in one gap in the proof, but "two missed trivialities taken together could form an insurmountable obstacle." We shall first prove that in a certain neighborhood N of the set A there is a continuous function ~ : TN --* Ir whose restriction on every T=N is a quadratic form (in such cases I shall speak of a continuous quadratic form on a vector bundle) possessing t h e following properties. It is positive definite (as a quadratic

T=N). In the case of a flow, for all w E TN, there is a derivative d t=o~(Tgtw) which is a continuous quadratic form on TN. (In tensor language, 9 is a certain twice covariant tensor and we speak of

form on every

its Lie derivative along the vector field of the phase velocity of the flow.) For all x E A, ~ E E:, ~/E E~ we have (5.5) r _< c~(~?)-i1~11~, r <_ ~(~) - I I , f f ' , in the case of a cascade and _a d~ ,=o~(Tz,~ ) < _11,,112 '

508

d ,=o~(Tg,r/) > ii~ll~ d~

(5.7)

in the case of a flow, and, in addition, ~(v(x)) = 1, where v is the phase velocity vector field of the flow. In (5.6) and (5.7) ]]. I] is the initial Riemannian metric on M with which we begin in order to arrive at the Lyapunov metric ] 9 [. We can somewhat simplify the reasoning if we assume that a certain neighborhood N1 D A is smoothly embedded into R '~ (for the neighborhood of the set A the embedding can be easily constructed by means of local coordinates [8, 9]; as is known, we can embed the whole M, but if it is not compact, it is much more difficult to do, whereas it is sufficient for us to embed only a certain neighborhood N1 D A) and if we also assume that ]]. ]] is a restriction on TN1 of a standard metric in R" which we also denote by [I" IILet z E A C R '~, X q R". Then we can represent X as X=~+r/+v

or

X=~+r/+~'+v,

where v.l.T=U, ~ 6 E=', r / e E~ and (in the case of a flow).~" e E•. We set UI(x,X) = H~ll2, v ~ ( z , x ) = Hr/]l2 and (in the case of a flow) W1 (x, X) = []r 2. These are quadratic forms on R '~ whose coefficients are continuous functions of z e A. These functions admit of a continuation up to the continuous functions R = --* R (a special case of the well-known theorem of Urysohn; this case was discovered by Brauer earlier and admits of a very simple proof). Continuing in this way the coefficients of the forms U1, V1, W1, we obtain some continuations U2, V~, W2 of these forms. We denote the restrictions of 0"2, V2, W2 on TN1 by U3, Va, W3. The latter are defined for the tangent vectors w e TN1, namely, u~(~) = v~(~, ~),

v~(w) = y ~ ( ~ , w ) ,

w~(w) = w 2 ( ~ , . , ) ,

where x is the point of N1 for which w e T=N1. These are continuous quadratic forms on TN1, and when

9 eA,

weT=g,,

~=~+r

or ~ = ~ + r 1 6 2

~eE;,

~eE;,

feE

2,

then

v~(w) = iir ~,

v~ = I[~ll ~,

w~ = [lr ~

We shall first consider the case of a cascade. We take an integer r > 1 (we shall later elucidate the choice of this parameter of the construction). We denote by N~ the neighborhood of the set A for which giN,. C N1 for all Iil -< T, and set r

,.-1

Oh(w) = 2 ~ U3(Tgiw) + 2 ~ Va(Tg-lw) i=0

i=0

for z e N,., w e T~N,.. Clearly, it is a continuous quadratic form on the vector bundle TN,.. At the points A it (i.e., its restriction on T=N,.) is positive definite, and therefore it is also positive definite when z lies in some neighborhood N; of the set A. If

zEA,

wET=M,

w =~+r/,

~EE~,

r / e E~,

then

~(w) = ~(~) + ~(~), ,-1 4,(~) = 2 ~ I[T9'@ 2,

~-I (~(,1) = 2 ~ [ITg-'~[] 2.

o

(5.8)

o

In particular, as can be seen from (5.8), E=~ and E~ are orthogonal in the Euclidean metric r Furthermore, it is easy to see that

r

- (I)(~) = 211Tg,.~H2 - 211~11=, (I)(Tg-ir/)- ~(rt) = 2llTg-,.r/I] 2 - 2l]r[[]2.

(5.9)

But the hyperbolicity of A means that estimates of the form

IlTgi~ll <_ "e-~'ll,~ll,

I[Tg%Tll - a~-'ql'Tl[, 509

where a, b > 0 are the same for all x, ~, r], are satisfied for i > 0. Hence, for a sufficiently large v, we have 2IITg~II 2 < ll~[15

2IITg-~II 2 _< I[~ll ~.

We shall take just such a ~" and N~' as N. Then (5.9) implies (5.6). Let us now consider the case of a flow. We take 7" > 1 (we shall elucidate its choice later on) and denote by N~ a neighborhood of the set A such that gtN~. C N1 for all Itl < r. We set

r

]

]

0

0

= 2 U3(Tg'w) ds + 2 V3(Tg-%) ds +

Wa(v(g'z)) ds 0

for z E N,, w E T,N,. (The last term is a new element as compared to the construction for the case A = M in [2]. In the terms we use now, we simply take l[r there,.where, as usual, ~ is the component w from E~. Formally, U1, V1 are not continued to Us, V3 there either, but this is naturally trivial.) Clearly, this is a continuous quadratic form on the vector bundle TN,. At the points of A it is positive definite (on the corresponding T~,NT), and, therefore, it is also positive definite when x lies in a certain open neighborhood Nr D A. Furthermore, if x E Nr then gtx E Nr as well for sufficiently small [tl (generally speaking, the degree of their smallness depends on x), and so it is reasonable to speak of

9 0

/ Wz(Tg t+'w)

0

2

ds + 2 "r

V3(Tg-'w) ds +

Wz(v(g'z)) d8

--t

for w E %N~. The integrand in these integrals is continuous with respect to s, and, therefore, there exists a derivative

d lo~(Tg'w) = 2U3(Tg ~'w) - 2U3(w)- 2V3(Tg -~w) + 2V3(w) +

Wa(Tg~w)

Wa(w)

W~(v(gl~))

w~(~(~))'

which is a continuous quadratic form of w. If

zEA,

wET~M, w=~+r/+~,

~eE~,

r/EEl,

~'EE~, r

zER,

then we have

r

= r

+ ~(~) + r162

(which, by the way, implies the orthogonality of E~, E~, E~' in the Euclidean metric ~I(TMIA)),

,~(r =/llTg'~ll 2 d,,

~(,7) =/llTg-',lll ~ as,

0

r

0

/IIT#%'II 2 z2 Itr 2 = 0j IIv(g'x)lP ds : = iiv(x)ll ~

(the latter is due to the fact that Tg'~ = zTg'v(x) = zv(g'x)),

510

211~tl2,

d o~(Tgt~ ) =

211Tg~ll = _

d o~(Tg_t~ ) =

211Tg_~II 2 _ 211~11=,

(5.10)

As before, for a sufficiently large r the subtrahend is twice as large as the minuend and we get (5.7). It is clear, finally, that ~(v(z)) = 1 for x E A. In order to pass from 9 to a smooth Lyapunov metric, we must approximate ~ by a smooth quadratic form k~ on the bundle T N . The meaning of the approximation is different in the case of a cascade and in the case of a flow. In the case of a cascade it is only required of the approximation that max {l~(w) - ~(w)l; w e T=M, x e A, Ilwll <__1} < 6.

(5.11)

We shall elucidate the choice of 6 somewhat later. For the time being, we shall make sure that for any 6 > 0 there exists a Coo-smooth (as a function of T M ~ g) quadratic form ~ satisfying (5.11). We fix the covering of the compact set A by a finite system of coordinate neighborhoods U I , . . . , U, with compact closures and suppose that (Ui, qai) are the corresponding charts. We also take an open set U,+x = M \ A and let {0~} be a Coo-smooth partition of unity subordinate to the covering {U1,..., Ur+l} of the manifold M. In actual fact, we shall only need 01,..., 0r. Note that their sum is equal to 1 in a certain neighborhood U of the set A and their supports are compact. We repiesented '~ITU in the form ~ ~i, where ~i(w) = #,(z)~(w) with x for which w E T~:U. Although ~ is defined on N (and ~ = E ~i is defined only on U, i.e., on TU), we can consider ~i to be defined throughout M. Outside of Ui and even outside of the support supp 01 C Ui the form ~i is identically zero (i.e., '~HTxM = 0 for x r supp/?i). In terms of the local coordinates corresponding to the chart (Ui, qvi), ~i can be regarded, as a quadratic form with coefficients depending on the coordinates ( y l , . . . , y'~) (rn = d i m M ) of the point y E U~ as parameters, i.e.,

~, = ~ ~,~(y~,..., y~')~Jw ~,

(5.12)

j,k

where the "vector part" of the coordinates T~,~iw = ( w X , . . . , w " )

w E T~M,

y E Ui,

r

= (y~,-..,y"),

(5.13) (we say the "vector part" of the coordinates since the total collection of coordinates of the vector T ~ i w also includes the coordinates (y~,..., y'~) of the point qa(y). Identifying %(~)R '~ with g " , we "forget the footpoint," i.e., "forget" at what point the vectors from T~,(y)R" were applied, and then only the "vector part" of the coordinates remains). Here the coefficients ~ k ( y ~ , . . . , y " ) are functions definite and continuous in qai(U~) with ~ k = 0 outside of supp (0~ o ~[~). Clearly, we can approximate qa~ by C ~ functions r

,y')

with supports supp Cjk c Ui (we can use the ordinary technique of approximation by means of a convolution with a suitable bell-shaped kernel). It is also clear that if all r = E~ ~i, where

9 ,(~) = ~ r ./k

are sufficiently close to ~ k , then the form

when (5.~3) is satisfied,

(5.~)

approximates the form ~. Since ~ is positive definite near A, the form 9 (for a sufficiently small 6) is also" positive definite in a certain (maybe smaller) neighborhood V D A. Finally, if we want a Lyapunov metric I" I to be a Pdemannian metric defined on the whole M, we can construct, near the boundary of V, a smooth "passage" from ~ to any other quadratic form C~-smooth and positive definite everywhere. But (5.1) refers only to the properties of a Lyapunov metric as a quadratic form on TM]A, i.e., the properties of ~. Since A is compact, there is c > 0 such that ~(w) < cll~ll ~ for all w ~ T~M, 9 ~ m. Then, by virtue of (5.11) and the positive definiteness of ~, 0 _ q'(w) _< (c + 6)11,oll~. If 6 > 0 is so small that 1 - 26 > 0 and 1-~s > ~ , then, in accordance with (5.6), qt(Tg~) < ~(Tg~)+~ll~ll ~ _< ~(~)-I1r

~+~11r ~ _< ~(~)+2gll~llZ-IIr ~- _< ~(~)

1 - 26

~ ~ ~(~) _< ( 1 - ~ ) ~ ( ~ ) 511

for ( E T=M, x E A. Similarly, ~(Tg-'7) <_(1 - 2-i/)~(7) for r/E TxM. Since ~he metric (i.e., on TM[A), it follows that

ITg l <

fi- 1/2c1 [,

I 15

=

on A

ITg-lnl <_ /1-1/2cl71.

Taking a such that e -5~ = V/i- - 1/2c, we get (5.1) for t = 1. Using again the resulting two inequalities and taking into account the invariance of E', E" with respect to Tg and Tg -1, we get (5.1) for integral t > 1. Let us consider an "approximate orthogonality" of the subspaces E~ and E~ in the metric I " I defined on TMIA by the form O. It follows from their orthogonality in the Euclidean metric r ) and the closeness of k~ to r In more detail we can show this as follows. Since ~ is positive definite on TM]A, there exists c~ > 0 such that Ilwll 5 __ c1r for all w e TMIA. Then, by virtue of (5.11), 1

_<

ll ll 5 _<

r

-< 1 -

c16 ( )

(we assume, of course, that c~6 < 1). Using (5.8), (5.9) and the last inequality, we find that for the vectors ( E E~, 7 6 E~ of unit length (in the metric l" I) the cosine of the angle ~o between t h e m (equal to their inner product) is estimated as I c~ ~l = I 1~+715- I@-1715 I -< ~ ( ~ + 7 ) - ~ ( ~ ) - ~(7)1 -< l e ( ~ + n ) - ~ ( ~ ) - ~ ( ~ ) l +6(11~+~115- II~ll5 -117115) <

2C1~

Cl~(lIl'(~ ' "1- 7) "l- 'CI~(~) + 1I~(7)) = 2C1~(1I)(~) -I" (I~('/'])) ~___1 ----'c~16(~('f) 4-

~(7))

-

2__C d (l[~2 4CI~ 1 -clN'" + 117115)-< 1 -- ~116"

When 6 is sufficiently small, the angle ~o is close to 90 ~ Now we can see that the smallness of 6 (even more "moderate") guarantees (5.4). In the case of a flow, the approximation must be understood in the sense that besides (5.11) one more condition,

max{

doO(Tg'w)-doO(Tg'w)];wETMIA, Ilwll < 1} < ~,

(,5.15)

is also satisfied. As before, we consider the quadratic forms r = Oi~, where {01,..., 0,} is a smooth partition Of unity in a certain neighborhood, of A and supp 0i ties in the coordinate neighborhood Ui With a compact closure. Since r differ from ~ only by smooth factors 0~, it follows from the existence of the derivative ~Ior which is a continuous quadratic form on TN, that there are also derivatives ~loi~i(Tg~w), i = 1,...,r, which are also continuous quadratic forms on TN. In order to construct ~, it is sufficient to approximate each gTi by a C~176 quadratic form ~i which is identically zero outside of a certain compact subset of the domain U~ (now we understand the approximation in the sense of conditions (5.11), (5.15)), and then q~ = ~ [ ~Ji will approximate r (maybe with r~ instead of ~, but this is inessential since ~ is arbitrary). Naturally, the approximation is again carried out in terms of local coordinates. It turns out that, as before, we can use the convolution of the coefficients of form (5.12) with a suitable "bell-shaped" kernel, but this time it is not so obvious. In the main it proved in [2]. To be more precise, in [2] I proved the corresponding statement concerning the approximation of functions (it is given below) and said that it should be applied to the coefficients of the quadratic forms (5.12). But these quadratic forms are some covariant tensors of valence 2. Upon a shift along the trajectories, the coefficients of these forms, i.e., the tensor components, are transformed according a law different from that used for functions. Therefore, the coefficients of the quadratic forms

w ~.-, dlo~i(Tg'w) do not simply reduce to the corresponding derivatives of the coefficients of r (Using the tensor language, we can say that the components of the Lie derivative of the tensor are not simply t h e derivatives of the 512

tensor components along the corresponding vector field.) However, we can easily verify that the factors or summands which are added for this reason (as compared with the case of functions) to the expression for the derivative being considered do not cause any difficulties, and so the part of the arguments of interest to us refers precisely to functions, and the rest is trivial. Still the latter fact deserves explanation, which is as follows. Now y will be a point from qoi(Ui). Suppose that

gi(t, y) = (g~ (t, y),..., g'['(t, y)) = ~ai(gt~.S'(u)), while gtqo~-X(y) remains within U~, and let the vector field

v/(~/)

=

(v:(y), . . . , v'['(y) ) = T~iv(~:, '(y) ).

It is clear that v~ in ~a~(U~) defines a "local flow" in which y passes into gi(t, y) during the time t, in other words,

Ogi(t, y) = vi(gi(t, y)). If (5.13) is satisfied, then ~i(Tg'w)

= E

vj' (t,y)d

',

(5.16)

jk

where

Og~(t'y) Og!(t'Y).cp i (g(t,y~,,.

(5.17)

In fact, it is a coordinate expression for the action of (Tgt). upon a twice covariant tensor ~i taken at the point gt~o:,l(y). We can naturally obtain the same thing in a direct way, by using the coordinate representation (5.12) for ~i and knowing that the "vector part" of the coordinates of T~oiTgtw is thepth

coordinate=y]Og~(t'Y)ur i. j

Oyi

Introducing the matrices

F(y)

=

F(t,Y) = (tPjk(t,Y)),

(~o~k(y)),

cOt, y) =,

cOror

)'

we can write (5.17) in the matrix form

F(t, y) = G(t, y)F(g(t, y))G'(t, y.) (the asterisk means transposition). Note that

G(t, y) is invertible so that

F(g(t, y)) = G-l(t, y)F(t, y)(G'(t, y))-~. Taking wj of the form 55 (a Kronecker delta), we find from (5.16). that

(5.18) ~oik(t,y ) are differentiable with

respect to t for t = 0 and ~10~.k(t , y) are continuous with respect to y. Since the field v, and in local coordinates the field vi, is smooth (at least of class C1), the partial derivatives ~ 01tJ smoothly depend on t and their t-derivatives are continuous with respect to (t, y). It should be pointed out for what follows that O~/Oy j satisfy the system of variational equations

o og;(t,y)

ova(z)

ogi (t,y)

(5.19) 513

and have initial values 09~(0,y) ~ ~ y j - = 6j.

(5.20)

Here we use the fact that by virtue of what was said, the matrix G-l(t,y) also smoothly depends on t and ~G-l(t,y) is also continuous with respect to (t,y). It follows from (5.18) and from the property of differentiability of qa~.k(t,y) pointed out above that qoik(g(t , y)) are differentiable with respect to t for t = 0 and o 0qaik(t,0) are continuous with respect to (t,y). The following statement is proved in [2] (the end of Sec. 6, p. 40). Let a smooth vector field f be defined in the domain U C R". We denote by h(t, y) a solution of the system ~ = f(y) with the initial value h(O, y) = y. Suppose that a continuous function qa with a compact support supp qa C U is defined in U (we mean that the closure of the set {y; qa(y) ~ 0} is taken in the whole R "~) which has a derivative at all points of U calculated in accordance with the system ~) = f(y), i.e., a derivative ~ o~(h(t , y)), and let the latter derivative be continuous in U. Then, for any 8 > 0, there exists a function r : U -+ g of class C ~ with a compact support, supp r C U6(supp T) (here U6(B) is a 6-neighborhood of the set B; we can assume that the closure of Us(supp qa) C Ui), for which

<,,

I

(5.21)

for all y. Applying this assertion to U = ~i(Ui), f = vi, and to the functions qa~k, we get some Let us define the form kgl which is identically zero outside of U; and has a local representation (5.14) in Ui. We must verify whether kg~ approximates r not only in the sense of (5:11) (which is obvious) but also in the sense of (5.15). Outside of Ui this is obvious since both forms are zero there. Thus everything reduces to the estimation

fjk.

of ~[o+i(Tgtw) - ~ o@,(Tgtw) for w + TU,, when +i(Tgtw) is expressed in terms of local coordinates in accordance with (5.16), (5.17), and the expression for k~i(Tgtw) is quite similar, with the only exception that ~bjk must be written instead of ~jki We differentiate (5.17) and a similar expression for r y) with respect to t for t = 0 taking into account that the values of cgg~/cgyJ and those of their derivatives with respect to t can be found from (5.19), (5.20). A simple computation yields

OqO~k(O'Y)ot "~"O~ik(g(O'Y))Of, -]"~ O'o~(y)oy j ~i1~k(y) "~"~q Ov~(y)oy k ~jq(y)i and a similar relation for 0r

y)/Ot. Since (5.21) is satisfied for ~ = ~ik and r = Cjk and Ov~/Oy j admits

of a uniform upper estimate (indeed, we have only to consider y E U~(supp (0i o ~ , a ) since outside of this set ~ k = Cjk = 0), we infer that there exists C such that

~162176 < c6 9for all y e ~i(Ui). It follows that if z e Ki, where Ki = ~i(U6(supp (Oi o ~,~))), then

for all w 6 T:N, and if z r Ki' then this difference is zero for w 6 T:N. (E Iw, I) -< C lwl when z g . and therefore

dtlo+,(Tg'w) 514

e

0 ,(Tg'w) _< O C lwl',

But there is C1 such that

and since 6 > 0 can be arbitrary, our approximation assertion is proved. As in the case of a cascade, we can change 9 outside of a certain neighborhood of A in order to get a true Pdemannian metric on M. But conditions (5.1), (5.2), (5.3) only refer to the properties of a Lyapunov metric as a quadratic form on TMIA , i.e., the properties of the unchanged ~. We again have c > 0 such that q~(w) < cl[w[[~ for all w E TMIA. Then 0 < ~(w) <__( c + 6)[Iwl[ ~, and, by virtue of (5.15) and (5.7),

~lolTg r

=

ot~(Tg'r

+ ~11r

o r162

<

--- --II~ll~ + 611~11~-<

-

c

~(r

--

~u

I~1~

for ~ E E ~. We set

1- ~ a = 4(c + 6)"

(~.22)

Then

d

t

l d

Tt

2_

Taking into account that TgtE" = E', we find that d t ~[olT9 ~1 = d [olZg'(Tg'~)l < -2alTgt~l for all t. When t > 0, we see that d In ITg'~I < -2~,

In ITg'~l - In I~1 -< - 2 a t .

This leads to the first inequality of (5.1). The second inequality can be obtained analogously. Let us consider (5.2), (5.3). ~(v) = 1 on A and, since A is compact, Ilvll on A does not exceed some C~. Using (5.11) and (5.15), we get

I l v ( ~ ) l - 1 t - I Iv(~)l~ - 11

I Iv(~)l + 11 <- I Iv(~)l~ - 11 = Ir

~tlolV(r

1

a

,

=

2(1- c~6

- 11 <_ 6tlv(~)ll ' ___ c~a,

o~(~(r

611v(~)ll ~ c~a G'26) < 2(1 C~6)"

< 2(1

-

-

This guarantees (5.2)with =

max

(G26,

C26 ) 2(1 - ~ 6 ) J

It is clear that for a sufficiently small 6 condition (5.3) is fulfilled. (By virtue of (5.22), a ~ ~ as 6 ~ 0, and therefore the "smallness of 6 as compared to a" is simply equivalent to the "smallness of e".) Finally, the reasoning concerning the "approximate orthogonality" of E~, E~, and E~ is the same as in the case of a cascade, since complications for flows arise because of.the differentiation along trajectories (because of the Lie derivative), and the angles between vectors from the indicated subspaces are irrelevant to it. The computations leading to (5.5) follow the sequence of computations leading to (5.4). 6. Beginning with this section, A is a hyperbolic set. To describe the motion of the points gty near gtz, z E A, we use the results of See. 4 implying now that the metric being used is the Lyapunov metric ] 9 I- In this case, it is expedient to use some other neighborhoods instead of Br(x). We denote by A~(z) C T~:M the domain {~ + ,7; ~ E E,', r/E E~, I~1 < r, I~l < r} in the case of a cascade and

515

in the case of a flow. (The definition only refers to x E A.) If E~, E~, and (1or a flow) E~ were orthogonal, then, for ~ E E~, 7 E E~, ff E E~, we would have max (1~1,171) <- I~ + 71 <- Vr~max Cl~l, I~1), max (1~t, 171, Ir

-< I~ + 7 + r -< V~max (1~1, 10t, Ir and therefore B,(z) C A~(x) C Bvs,(z) or B,(z) C A,(x) C Bv,~(x ). But since the orthogonality is approximate, we must somewhat extend the "boundaries enclosing A~(z)." It will suit us if B,/2(x) C ~ ( z ) C Bz~Cz). The right-hand inclusion simply follows from the fact that I~ + 7 + ~1 "-< I~1 + 171 + Ir it is not connected with the approximate orthogonality. The left-hand inclusion is connected with it; in actual fact, it follows from (5.4), (5.5). If 3r < v3, then exp diffeomorphically maps from A~(z) onto some neighborhood D,.(z) of the point x in M. In this way "coordinates" (~, 7) or (~, 7, ~) are introduced in D,(x). Omitting any reference to z and t, we have A--'~C Bar; if 3r < ro,-then FIB3' is defined and F(B-~z~) C B3c~ C AsCr C Blscr. The last ball lies in B~ (where exp has "good" properties) if 18Cr < r3 = Cro, r < r0/18. Thus: let us reduce r0 eighteen times and increase C six times; then, for r ~ (0, ro), the maps

F~: A~(~) --, ~Xc~(a~),

H~: ~(~) ~ zXc~(a-l~),

or Fz,t: At(x) --* Acr(gtx) are defined. (It is actually not very important that exp [Ac, is a diffeomorphic embedding of Ac~ in M, and therefore we could have made r0 only three times less. But, all the same, we will have to diminish r later on, and so it is inessential.) They are represented as (4.3), (4.7), or (4.11) with f, h = 0 for v = 0 (to be more precise, f = 0~ or 0~,~, h = 0~-~ for v = 0~) and f, h satisfy conditions (4.4), (4.5), (4.8), (4.9), (4.13), (4.22) with w~(3r) instead of wd(r). Let {g"} be a cascade. We write the maps F~ in terms of the "coordinates" (~, 7) in A,(z) and Ac~(gz) corresponding to the decompositions T~:M = E~ ~ E'~, T~M = E~ ~ E~. Since T~g maps from E~ into E$~ and from E~ into E ~ , we can represent F, as

~'

( A(~)~+ ~(~, ~,.) ) B(~)7 + r

(6.1)

'

and, by virtue of (5.1), (6.2) IA(z)~I < ,-2~1r IB-~(~)71 < e-2-~101. and ~b are defined for z E A, ~ E E~, 7 E E~, I~1,171 <-- r and take values in E ~ and E ~ ; they are continuous with respect to (~, 7, r and

~(~, 0, 0)

= 0,

r

0, 0) = 0,

[~(Z, ~1, ~1) -- ~(Z, ~2, 72)1 ~_~W(r)([~l -- ~2[-I-1771 -- r/2[), Ir

6 , 01) - r

(6.3)

6 , 72)1 _< ~(r)(16 - ~21 + I~1 - 721)

(I now take the liberty of writing 0 instead of 0z), where w(r) has the same properties as wd(r). Mutatis mutandis all that was said above also refers to H~. I shall not write out the corresponding analogs of (6.1), (6.2), and (6.3) but shall only recall that when passing from g to g -x, i.e., from F= to H~, the roles of the subspaces E~ and E~ are switched. Let us now consider the flow {at}. We write the maps F~,t in terms of coordinates (~, 7, ~') in T=M and Tg,~M corresponding to the decompositions of the respective T~M in" a direct sum of E~, E~, E~. Since T#t~ maps E~ into E~,~ and so on, we can represent F~,t as

F~,~:

516

7

~

B(~,t)7 + r 1 6 2 C(~, t)r + xCz, ~, ~, r

.

(6.4)

By virtue of (5.1), we have here

IA(x,t)h IB(z,-t)l,

IA-t(z,-t)l < e-~'; 9

IB-~(z,t)l < e -2~t

(z e A, 0 < t < 1).

(6.5)

As to c(z, t), it follows from T g t v = v o gt that c(z, t ) v ( z ) = v(gtz). If r e E~, then ( = z v ( z ) with a certain z q I~. Then, according to (5.2),

(1 -~)lzl < Izl Ivl = Ir d

d

]g/Ic(~,t)CI

= Izl ~ l v o g t [

< ~lzl <_ 1_-~1r t~

(6.6)

Ic(=,t)cI < (1 + ~--:S)IcI. Finally, for "nonlinear" terms % r

X, we have

~(~,o,o,o,t)

= o,

I~(~, 6 , ~1, C1, t) - ~(~, 6 , ~ , C2, t)l ___t-,(r)(16 - 61 + 17~ - 72l + 16 - 61) and similarly for r

(6.7)

X.

The properties of w are the same here as above, namely, it is a continuous nondecreasing function of r > 0 equal to zero for r = 0. L e m m a 1. For any k > 0 there ezists 6(k) > 0 such that the following is valid for any 6 E (O,6(k)). Suppose there are T > O, z E A and yt, y~ E Ds(z) such that

for all t E [0,TI,

gtyt,gty2 E Ds(gtx)

so that using the local coordinates (~, rl) in the case of a cascade and (~, q, () in the case of a flow, we can represent these points as gtYl = expa,=(~i(t) + r/i(t))

for all t e

[0, T].

gtyi = expgtr(~i(t ) %- rli(t ) %- Ci(t))

or

If 1~(0)-72(0)1

~k16(0)-6(o)I

(6.8)

in the case of a cascade or

171(0) -7=(0)1 _> k 1 6 ( 0 ) - 6 ( 0 ) h

kl6(0)-6(0)1

(6.9)

in the case of a flow, then

Inl(t) - ,2(t)1 _> kl6(t) - 6(t)1

(6.1o)

171(t) - ,7~(t)1 >__kl6(t) - 6(t)1, kl6(t) - 6(t)1,

(6.11)

and, respectively, .for all t E [0, T], and also

17~(t)

-

7~(t)1 > e~

- 72(0)1

(6.12)

in both cases (for a cascade and a flow). Now if I~,(T) -

5~(T)I >

(6.13)

kl,l,(T) - 72(T)1

in the case of a cascade or I~t(T) - ~2(T)I _> kln~(T)

-

72(T)1, klG(T)

-

G(T)I

(6.14) 517

in the case of a flow, then (6.15)

and, respectively, (6.16)

for all t ~ [0,T], and also

[~l(t)-~2(t)[ > e=(T-01~l(0-~2(t)l

(6.17)

in both cases. Proof. We begin with the case of a cascade. First we suppose that (6.8) holds. Then, using (6.2) and (6.3), we get - ~2(1)[ _< k]A(z)(~x(O) - ~2(0)1 + k[~(x, ~x(O), 771(0))

k[~r

e-2'~kl~x(O) - ~2(0)1 + r

-

-

~0(5~,~2(0), ~2(0))[

- ~2(0)1 + Ir/l(O) - 7/2(0)1) _< (e -2= + (k + 1)w(6))lrh(O) - r12(O)l.

Moreover, 1,71 = IB-XBuI <- e - 2 : l B ~ l ,

I,h(1) - rt2(1)l >_ IB(z)(,h(0) - ,72(0))1 - Ir

e2al~x(0 ) -

IBul _> e~:l,71,

~a(b)) - r

rt2(0))l >_

~ ( o ) l - wO)(l~l(o) - ~2(o)1 + 1~,(o) - ~ ( o ) l ) >_ (e 2~ - (1 + ~-)~0))1~1(o) - ~(O)l.

Let 6 > 0 be so small that e -2~ + (k + 1)w(6) < e -~, e 2~ - (1 + ~)w(6) > e% Then 17,(1) - ~20)1 > e ~ l ~ ( 0 )

- ~2(0)1,

kl~a(1) - ~,(1)1 < e-~

- ~2(0)1.

It follows that 1 ~ ( t ) - ~(1)1 > k t ~ ( 1 ) - ~(1)1,

i.e., the same inequality holds for the coordinates of the points 9'Yl, 9~y2 for i = 1 as for i = 0. Repeating these arguments, we infer that lug(i) - n2(i)l > kl~l(i) - ~2(i)1,

lux(i) - u2(i)l > e=ln,(i - 1) - ~2(i - 1)1

for all i = 1 , . . . ,T. The first of these inequalities is precisely (6.10) and the combination of the second one (with all these i) entails (6.12). We have thus proved that (6.8) implies (6.10) and (6.12). Finally, we can obtain the fact that (6.13) implies (6.15) and (6.17) as a formal consequence by shifting the zero of the time axis to the moment T and reversing the time. The arguments are similar for flows. It suffices to establish the fbllowing assertion: for sufficiently small from (6.9) it follows that (6.11) and (6.12) are satisfied for 0 < t < m i n ( 1 , T ) . Using (6.4)-(6.7), we find that k l ~ ( t ) - ~2(t)l

_<

klA(z,

t)(~(0) - ~(0))1 + kl~o(z, Ca(0), rtx(0), G(0), t) - ~oCz, ~(0), ~ ( 0 ) , ~2(0), t)] <

e - z " k l ~ ( 0 ) - ~ z ( 0 ) l +,~(6)k(l~1(0)-~=(0)l + I,h(0)-r~2(0)l + ICx(o)-Cu(O)l)" _< (e-~'+(2+k)tw(6))l~?x(O)-~?2(O)l,

Irh(t) - r/2(t)l _> [ B ( x , t ) ( o a ( O ) - r/~(0)l - Ithe difference of 0[ (e 2 ~ ' - (1 + k)tW(6))lrh(O)- 7/2(0)[ ,

kif,(t) - r

> k l c ( z , t ) ( G ( O ) - (2(0))1 + klthe difference of xl

-< (1 +

~

(6.18) te

+ (2 +

Expression (6.11) will be ensured if

e '~ - (1 + 518

t~(~) _> m a x ( e - ' ~

(2 + ~)t~(~), 1 + ~

+ (2 +

k)t,oCa)) I,~x(O)-,~(o)1.

for which purpose it is sufficient to ensure that

l+2at>max(l+(3+k+ 2a>max

3+k+

+ +

~(6),y=-~_ + 3 + k +

~(6) = 1 _ ~

and this is obviously so if (3 + k + ~)w(6) < a. (We must also take into account that, by virtue of (5.3),

< a/2 1 -- ~ -- 0, 99

~.)

We have also e n s u r e d t h e validity of (6.12). Indeed, for t > 0 we have e 2~t >_ e ~'

+ at

(for t = 0 we have an equality here and for the derivative of the right-hand side and the left-hand side it is obvious that 2he 2at _> ae ~t § a). This means that the right-hand side of (6.18) is not smaller than

For suj~ciently small 6 > O, the following is valid for the cascade {g~}. If x 9 A, y 9 De(x), gy • Ds(x), and y = exp~(~ q- y) in terms of the local coordinates (~, 7) ~n De(x), then L e m m a 2.

[7[ > 6~2max{IS(x)[; x 9 A}.

(6.19)

Proof. We consider 6 to be so small that Lemma 1 is valid for r = C6. First of all, from the fact that gy = expg~(~' q-7') and max([~'[, 17'[) > 6 it follows that 177'1> [~'[. Otherwise, I~'[ -> [7'[ and, according to Lemma 1 (with k -- 1), ]~] _> e~[~']. But since max (]~'], 177'1) > 6 and ]~'[ > [7'], it follows that [~'] > 6 so that [~[ > e~6 > 6 contrary to the fact that y 9 De(x). Thus, max(j7' [, [~'[) = 1'7'1, [7'[ > 6. But then, using (6.1) and (6.3), wefind that

max {IB(~)I; 9 9 A} > IB(~)TI > 17'- ~(~,~,7)1 > 17'1- 1~(~,~,7)1 > 6 -~(6)(1~1 + 171) > 6 - 26~(6). If 6 is so small that 2w(6) < 1/2, then (6.19) follows. How could the analog of Lemma 2 have looked for a flow? If the maps F~,t coincided with their linear approximations (i.e., with T~g ~) and the metric on TM]A was defined by the quadratic form 9 considered in Sec. 5 and still was sufficiently smooth to admit the use of exp~ then the trajectory of gty could leave De(g~x) with an increase (decrease) of t only as a result of a growth of the coordinate 7 (~ respectively). But, in actual fact, even [Tgt([ slightly varies with time and to the linear term c(x, t)( in (6.4) corresponding to

Tgt( a nonlinear one is added. Therefore, when ]~1 is close to 6, then g~y can leave De(gtx) because of a change in [([. We have to exclude the corresponding initial values from consideration. But if we speak of only a short time interval (say, not larger than of unit length) immediately before gty leaves De(gtx), then to assume that [([ is appreciably smaller than 6 at the beginning of this interval is to depreciate the problem to a considerable degree. It is a different matter if we speak of an arc of the trajectory Whose time length is not limited beforehand. Then the condition that, say, I(1 < 6/4 at the beginning of this time interval does not make the problem devoid of content. This is taken into account when formulating Lemma 3, which is an analog of Lemma 2 for flows. Note for the sake of comparison that we can also apply Lemma 2 to a long arc of the trajectory {gty, i = 0 , . . . , k -b 1} emanating from De(gtx) only at the last moment; then it asserts something about gky. But then the part of the trajectory preceding gky is of no importance, and its mention would only make the formulation cumbersome. This is not so in Lemma 3. Therefore, it seems to me that Lemma 3 is principally somewhat more complicated than Lemma 2. This is reflected in the proof of Lemma 3 - - we use in it locally stable and weakly stable manifolds. Therefore it is given in Sec. 7, where we speak of these manifolds. However, we can formulate Lemma 3 just now. 519

L e m m a 3. For sufficiently small 6 > 0 the following is valid for the flow {g'}. If z e A, gty 6 D6(x) for all t ~ [0, T] (where T > O) and

g'y = exp ,A(Ct) + 0(t) + r and Ir < 6/4 and iS arbitrarily close to T there are t such that gty q[ D,(g'z), then in D~(gtx) in terms of the local coordinates (~,~hr we have 1,7(T)I-- 6, Ir < 6, K(T)I < 6. 7. A piece of the manifold W'(z)near the point z, which is 6 in size, is called a local stable manifold of this point and is denoted by Wt(z ). We can use different means to make this more precise ([1, Sec. 1.3]). We shall consider W[(z) as a closed neighborhood of z on W'(z) (in the topology of an immersed manifold) lying in D,(z) and being, in terms of the "coordinates" (r r/) or (~, 77,(), the graph of some function • = r/~(r [el -< 6, in the case of a cascade or r / = r/~(~), ( = r I~1 -< 6, in the case of a flow. (In this variant WI(z ) is only defined for 6 < r, where r is a sufficiently small positive number. In particular, it must be so small that it would be reasonable to speak of the local coordinates (~, r/) or (~, r/, ~), i.e., that exp~ [A~(z) would be a diffeomorphism, but, of course, this does not exhaust the condition of smallness of r.) As is known, the construction of W'(z) begins just with the construction of W~(x), to be more precise, with the construction of r/~(~) or r/~(~), (~(~). These functions are continuous with respect to (x, ~) (z 6 A, ~ E E~, 1~1 _< 6). Moreover, they are differentiable (in the sense of Fr~chet) with respect to ~ and their derivatives Dr/~(~), D(~(~) continuously depend on (z, ~), with Dr/~(0) = 0, D(~:(0) = 0. It is clear that WZ(x ) is homeomorphic to a closed ball of the corresp9nding size. Mutatis mutandis all this also refers to the local unstable manifold W~(x) of the point z E A of size 6. The local manifolds Wt(x ) and W~'(x.) coincide with the linearly connected components of the intersections of W'(z), W~(x) with D6(z) containing x. W h e n dealing with a flow, we must also introduce a local weakly stable manifold W2'~(x) of the point z E A of size 6. This is a close neighborhood of the point x on W~'~(z) which is the graph of some function r / = r/~(~, ~), t~l, I(I < 6, and also a linearly connected component of the intersection of W"(z) with D6(z). The function r/~(~, ~) is continuous with respect to (z, ~; ~) and Fr~chet differentiable with respect to and (, the derivatives Der/~(~, (), Dcr/~(~, () being continuous with respect to (z, ~, ~) and Der/~(0, 0) = 0, Dr 0) = 0. If we could neglect the nonlinear terms in (6.4) and 9 would coincide with r (so that the third row in (6.4) would simply mean that [r does not change), then W~n(z) would be described by the relations 77= 0, ( = 0, 1~[ < 6, and W~'~(gtz) with It[_ 6 would be described by the relations r / = 0, ~ = t, It] < 6. In actual fact, this picture is distorted somehow. Here is what precisely will be used from this distorted picture. We would like to say that Wp(z) U{Wp(g'z); Itl <__6}, (7. 0 =

but this is, generally speaking, not true. (For instance, although W]"(gtz) is a linearly connected component of the intersection of W"(gtx) with D~(g'z), the linearly connected component of the intersection of W"(gtx) with D~(x) may be different. Even for very small t it may be slightly larger or slightly smaller, and, for t ~ [-6, 6] close to 6, a linearly connected component may be considerably smaller, as a m a t t e r of fact, for such t, g~x may not belong to Ds(z).) However, all this occurs close to the boundary FrDs(z) of the domain D~(z). Increasing slightly 6 on the right-hand side of (7.1), we shall cover the left-hand side. It is sufficient for us to have a rather rough fact

W~(z) = O,(z)fl (U{W~(gt(x); Itl ___2 6 } ) = O,(z)fl (U{g'W~n(x); Itl _< 26}),

(7.2)

and the fact (for sufficiently small 6) that if

z 6 A,

It[ < 26,

then

W~6(g'z) C D~6(z).

(7.3)

Finally, if y E W;6(gtz), 520

It] _< 26,

y = exp~(~ + ~/+ (),

then

[r - s] < 6/4

(7.4)

(here the formulation itself implies the use of the coordinates (~, r/, ~). in D3s(x), where, according to (7.3), y lies). Mutatis mutandis all that was said also refers to the local weakly unstable manifolds W~'"(z) of the points z E A of size 6. Note, furthermore, the following properties of the local manifolds Wt, W~': if 6 > 0 is sufficiently small, then s t e w ; ( ~ ) c w;-o,s(g ~) for t _> 0, I/. --I~ g-'wr c w:-o,6(g ~) for t _> o, W~(x) = {y; gty 6 Ds(gtz) for t > 0 and lim p(gtz, gty) = 0}

for all z E A (of course, notation of this kind automatically implies that

W~'(x) = {y; gty e Ds(gtx) for t < 0 and

lim

(7.5) (7.6) (7.7)

f y is defined for all t _> 0), p(gtx,gty) = 0}.

(7.8)

In fact, all this is established during the construction of W~, Ws~, but may not be especially emphasized. It stands to reason that I shall not repeat the construction (whose first "uniform with respect to x" variant also belongs to me [2]), but I deem it worthwhile to consider the connection of (7.5)-(7.8) with better known facts. We can reason as follows. Since Drb:(0 ) = 0, and, in the case of a flow, Dr = 0 as well, it follows that for sufficiently small 6 > 0 the local stable manifold W~(x) lies in the "cone" I~] -> It/[ or I~] -> It/I, Ir for all z 6 A. Since gtDs(x) C Dcs(gtx) for t 6 [0,1],

g'w;(~) c g'w'(~) c w'(g'~)

gtW~(z) lies entirely in a linearly connected component of the intersection WS(gtx) MDcs(gtx), which contains gtx, i.e., in W~s(fx ). This means that for all t E [0, 1] the point gt(x)W~(x) lies in the "cone" 1~1 _> [y[ or I~[ _> Iq[, 1r in Dcs(gtx). It follows that for any point y 6 W~(z) the coordinate ~(t) of its image gty decreases exponentially, (of course, the last fact is assumed to be known), and W~(z) is linearly connected, it follows that

i.e.,

Ir

_< :~'l~(0)l.

(We apply Lemma 1 with k = 1 to yl = y, y2 = z.) This means that

gtW;(z) C W':-o,s(g' x) for t E [0, 1].

(7.9)

Repeated applications of the result obtained (with a replacement of 6 by some e -'6, s > 0) prove (7.5). (Another variant: the reader may have dealt with the construction of Ws~ in which it was explicitly ensured that gtW](z) C Ds(gtx) (at least for t E [0, 1]) and that, moreover, gtW~(z) lies in the indicated cone. Then Lemma 1 immediately yields (7.9), whence follows (7.5).) Using the time reversal, we get (7.6). Relation (7.7) is obvious if'it is known that

w'(~) = {y; ~im p(g'~,g'~) = 0}. Often (7.10) is taken as the definition of the stable set with the immersed manifold

(7.10)

W'(z). But then we have to show that this set coincides

U{g-tW~(gtx); t > 0}, and this actually can be proved with the aid of (7.7). And so in every variant we have to prove (7.7) without using (7.10). It is clear from what we have just said about (7.5) that the left-hand side of (7.7) is contained in the right-hand side, and so we have to prove the inverse inclusion. Let us first consider a cascade. Suppose that gtyl G Ds(gtx) for all t > 0 and let

g'yl = exp:~(6(t) + ~(t)). 521

We want to prove that yl E W~(2~), i.e., that rh(0 ) = ?~x(~l(0)).

We assume tl:a', this is not so. We set

Y2 = exp=(~1(O)+ r/=(~(0))). Then y2 E W~(x), and therefore defined, namely,

g~y2 E D6(gtx) for all t > 0 so that the (~, rl)-coordinates of this point are g% = exp~,~(~(t) + ~(t)).

We have Irh(0)-r]~(0)l > [~1(0)-~2(0)1 = 0. According to Lemma 1 witla k = 1 (whose conditions are fulfilled for any T in this case), lot(t) - o2(t)l > e~'lr/l(0)

- rlz(0)l

for t > 0. But this contradicts the fact that gtyl, gty2 E D6(gtx) for all t > 0 since it follows from the last inclusion that 10~(t).-O~(t)l < 2~. In this reasoning we have not used the fact that limp(g'x,g'y~) = 0 and so in the case of a cascade we can omit mentioning the limit on the right-hand side of (7.7). Let us now consider a flow. We shall first prove that if

gtyx ~ D~(gtx) for all t > 0,

Using the (~, r], ~) coordinates in

y~ ~ W~"(z).

(7.11)

9s(g'z), we represent g'yl in the form gtyx = exp~,=(r

If

then

y~ r W;"(x), then rh(0 ) # r]=(r162 ~

+ rh(t ) -I- r

We set

= exp~(~,(0) + 0~(~,(0), C1(0)) + r

so that y~ e W~'~(x). We cannot guarantee that gty2 ~ D~(g'z) for all t-> 0, but at least y2 ~ W~s(g~'x) with some ~" e [-2~, 25] (see (7.2)), g~y~ e W~(g~'gtx), and therefore gty~ e D3~(g~x) (see (7.3)). We consider 8 to be so small that we can use (~, r/, () in Os~(gtz) as well and that Lemma 1 with k = 1 remains valid when 6 is replaced by 35. Hence gty2 can be represented as

g'Y2 =expg,~(~2(t)+y2(t)+~2(t)). In this case, we have It/l(0) -- r/z(0)t > [~x(0) -- ~2(0)1, 16(0) -- 6(0)1, since the last two differences are zero. Lemma 1 guarantees an exponential growth of I~l(t) - ~2(t)l, whereas rt2(t)l < 66. In order to prove (7.7) we have to show that if

gtyl, gty 2 remain in D35(gtz), and therefore I r h ( t ) -

g'yl e Ds(gtz)

for all

t > 0

lim(vtz,gtyl) = 0,

and

y~ E W~(x). We have shown that the first of these conditions ensures that Yx E W~n(x) so that !t~ E W~s(g~x) with some r E [-26, 25]. Then limt--.oo p(gt+*z, gtyl) 0. So it turns out that limt--.oo p ( g ' + ' z , g'z) =

then

=

0. This is possible for small r only when r = 0 (since there is no equilibrium point in A). Hence yx E W~6(x) whereas yx e Ds(x), and, consequently, yl e W~(z). Relations (7.6) and (7.8) result from (7.5) and (7.7) upon time reversal. P r o o f o f L e m m a 3. We make sure, first of all, that I~(T)} < 6. We apply L e m m a 1 with k = 1 to Yl = z, y2 = Y. In this case,

(~,(t),,llCt),r

= (0,0,0),

(~2(t),,72(t),r

= (r162

1~2(T) - ~a(T)I = I~(T)I = 6 > I~(T)I = 1,7~(T) - ~ , ( T ) I ,

522

I~2(T) - ~,(T)I >_ iG(T) - (x(T)l, and, consequently,

I~(0)1 = 16(0) - 6(0)1 > ~ T I 6 ( T ) - 6(T)I = ~"r6, contrary to the condition that y E Ds(x). It [s more difficult to show that I((T)I < 6 (the specificity of flows as compared to cascades is connected just with (). We assume that ]((T)I = 6. Let z = exp~,,:(~g,=(r/(T), ((T)), ~?(T), r so that z 6 W~'~(gTx). (It is not inconceivable yet that z = gTy.) Then, according to (7.2) (to be m o r e precise, according to the analog of (7.2) for W " , W"; in what follows I do not specify this, considering it to be obvious), z 6 W~5(g~gTx) with some r E [-26, 25], and, according to (7.4), Iv - 6{ < 5/4. This means that gt-W z 6 W~6(g~gtz) C Das(gtz) for t 6 [0, T] (see (7.5) and (7.3)) and we can represent gt-Tz as g ' - r z = exp~,A6Ct) + ~ ( t ) + 6 ( 0 )

for a small 6. We have {(~(0) - r I < 6/4 according to (7.4), and therefore Ir > 6/2. This already excludes the possibility admitted earlier that z = g~ since the ( = ('(0) coordinate for y is such that I([ < 5/4. Let us now apply Lemma 1 with k = 10 to yl = g-Wz, Y2 = Y. Since

1,7,(T) - ,7~(T)I = 0,

{G(T) - el(T)[ = 0,

gty # z,

it foLlows that ~2(T) # [~(T) and therefore }5,(T)

(,(T)I _> 101,72(T) -,7,(T)I,

IOIG(T)-

G(T)I.

Consequently, if 35 < 6(10), then 26___ 16(0)-6(0)l > I016(0)-6(0)l. But I(2(0){> 6/2, {(1(0){< 6/4, {(2(0)- (i(0){ > 5/4, and it turns out Chat 26 > 106/4, which is not true. L e m m a 4. Let a > O, b > O, a + b < 1. There exists 5 = 6(a, b) > 0 such that for 0 < 5 < 5(a, b) it follows from the fact that x 6 A and y 6 A N W~6(x ) that W:~(y) 6 W~(x). (Of course, 6(a, b) also depends on the dynamical system be!ng studied, on A, and on the metric used, but these objects are understood to be fixed, and therefore the dependence of 6 on these quantities is not explicitly indicated in the notations.) Proof. For any e > 0 there is 6(e) > 0 such that if 0 < 5 < 6(e) and z E A, ~ E E~, I~1 -< 6, then {'M~){ -< ~l~l or 1'/~(~)1 --< e{~{, IG(~){ -
l( + ~(()l < I~l + Ir

<_ (1 + ~)l~l,

l( - ~(()l >_ l~l - ln~(()l

_> (~ -

~)l~l,

and, in the case of a flow, we have

(i 2e)[(I -

_< 1~ + ,~=(()+ G((){ -< (I + 2e)l~l.

I f X ~ T~M and IXI < r, then p(x,exp=X) = IXl, Therefore, if~ 6 E~, I~1 _< 6, then

(1 -

2e)l({ < p(x, exp~(~ + ~/=(~))_< (I + 2~){~{ 523

in the case of a cascade (actually, we could have written 1 4- ~ in this case raok~r than 1 + 2e) and (1 - 2e)l~] _< p(x,exp~(~ + ~ ( ~ ) + (~(~))) _< (1 + 2e)l~[ in the case of a flow. Let us take e > 0 such that ~1-2~ ( a + b) = 1. We shall show that ~(a, b) = ~(e) possesses the required properties. In what follows, 0 < 6 < ~(a, b). In particular, then it follows from y E W~(x) that p(x, y) < (1 + 2r Let z E W~8(y) so that z = expy(~' + r/,(~'))

or

z = expy(~' + r/~(~') + ~,(~')),

where f' e E~, [f'] < a& We must prove that z E W~(x). Let us consider a curve [0,11 - .

where

w(O) = exp~(0~' + r/y(0~')) or w(O) = expy(0~' + r/y(0~') + r It starts at the point y E W~6(z) and connects it with z. Let us prove that it does not leave W~(x). We assume that this is not so. Then there is 01 E (0, 1) such that w([0,0a]) C W;(x),

w(O~)e OW;(x),

where OW~(x) is the boundary of W~(z) as a topological ball of the corresponding size or as a domain on W'(x) (in the topology of an immersed submanifold). As long as the point w(O) remains in W~(x), the local coordinates (~, r/) or (~, r/, r are defined for it, with 7/= r/~(~) or r/=.r/~(~), ~ = ~'~(~). Suppose that

w(O) = exp=({(O) + r/.({(e)))

or

w(O) = exp=(~(O) + r/=(~(O)) + ~'.(~(0)))

in these terms for all 0 E (0, 0~). Since w(O~) e OW~(x), it follows that ~(01) = & But

p(y,w(o))

= 1o ' +

or

1o '+

+

p(y, w(O)) <_(1 + 2e)01r < (x + 2e)Ob& We have used here the same arguments as at the beginning of the proof of the lemma, with the only difference that we replaced x E A by y E A. Hence pCz, w(e)) < p(x, y) + pCy, w(e)) <_ C1 + 2s)Ca + Ob)~, 1 +2e <_ 1 2t(a+Ob)a"

I (e)l < 1

But if 8~ < 1, then a + 8~b < a + b and L+-~la~-2=,+ 8~b) < 1. Therefore ~(0~) < 6, contrary to our assumption. 8. P r o o f o f T h e o r e m 1. It is sufficient to consider the case of a periodic W'(x) to which the case of a periodic W~(x) can be reduced by means of time reversal. Let r be a period of W~(x). We denote e - ~ by a and take some b E (0, 1 - a). In what follows, ~ is so small that c

for all z e A (see (7.5)) and that z E A, y E A n Wg~(z) entail W:s(y) C W~(z) (se e L e m m a 4). According to (7.10) and (7.7), for any y e W'(x) and ~ > 0 there exists T(y,e) such that gty e W~(g~x) for all t > T(y,8). We apply this to y = g*x E W'(x) and e = b& We take an integer'k for which kr > T(g'x, ba). Then g~y E W~(g~'z), i.e., g(}+')'a e W:s(g~rx). 524

(8.1)

At the same time c

But, as has been noted, (8.1) implies that

and it turns out that

g~Wl(g~ x) C W;(g~ x). Thus gr maps from the topological ball W~(gk~x) into itself. This means that there is a fixed point y there which is a periodic point of the dynamical system {gt} with period r. Since y E W~(gk'~x), it follows that

p(x, gs

= p(gj~+k,x, gi~+k~y) ~ 0 as j --* ~ ,

and since gJ~+~x E A and A is closed, it also follows that y E A. Finally, it is clear that y E W'(ga~x) C W'(x) (the last fact is true by virtue of the periodicity of W~(x)). We have proved Theorem 1. Next we denote by a(x) or a(L) the set of a-limit points and by w(x) or w(L) the set of w-limit points of the trajectory L = {gtx}. Lemma

5.

Suppose that the point x E A is a limit of the sequence of points x,~ E A and all x , r W;(x) U W~(x),

in the case of a cascade,

x,, q~ W~'~(x) U W~'~(x),

in the case of a flow,

(8.2) (8.3)

for a certain I~ > 0 (and then for any smaller 6 > 0 as well). Then there are v E a(x), w E w(x) such that (W'(v) \ {v}) f3 a r o, ( W " ( w ) \ {w}) r A r o. Proof. It is sufficient to prove only the existence of w. The existence of v, first, cart be proved by analogy and, second, can be obtained as a consequence of the existence of w by means of time reversal. Let us first consider the case of a cascade. We can assume that all z,, E Ds(x). By virtue of (7.7) and the remark we made when proving it stating that in the case of a cascade the condition concerning a limit is superflous, it follows from (8.2) that g~x,, leaves Ds(g~x) in due time. But since x,~ ~ x, the moment of the (first) exit from D~(g~x) increases indefinitely as n ---* oo. Thus, there exist k,~ > 0, k,~ --* cr such that

gix,, e D6(gix)

gk~+lx,~q~D6(gk~+xx).

for i e [0, k,,],

Suppose that

gixn=expgi~(~,~(i)(y)+~ln(i)(t))

for

i e [O, kn]

in terms of the (~, r/) coordinates in D6(g~x). According to Lemma 2,

(8.4)

I# (kn)l ___c6 with some c > 0 independent of z and x~. Let us prove that

-

<__2ae

(8.5)

When carrying out the proof, it is sufficient to consider only those n for which the left-hand side r 0. (In actual fact, on the assumptions of Lemma 3, this is so for all n since if it were not so for some n, then we would have

xn e wi'(g

nx),

e g-k wi,(g n ) c

(the latter by virtue of (7.6)) contrary to (8.2).) The point

525

and all its inverse images also belong to the corresponding W~(gJz). Let us apply Lemma 1 with k = 1 to Yl = z,~, Y2 = g-k"z. For them gizl, g~z2 E Ds(giz) for 0 < i < k, and for i = k~ 16(k.) - 6 ( k . ) l = l a c k . ) - ~gk.~C~.Ck.))l # 0, 1niCk=) - ~2(k=)l = I~=(k=) - #.(k.)l = 0.

Therefore 16(/)-~2(/)1 > 1 6 ( k = ) - ~ ( k ~ ) l e~r176 But since yl,y2 e Ds(z), it follows that 16(0)1, 1~2(0)1 < g(In actual fact they are considerably smaller, but this is irrelevant now.) Therefore, 16(0) - 6(0)1 _<_28,

16(k~) - 6 ( k , ) l _< 28' - ~ ,

I~n(kn) - ~at,,~x(on(kn))[ = ]~l(kn) - ~2(kn)l <__ 28e -~k*, and (8.5) is proved. We choose a subsequence lj = k,~j such that the sequences {glJz} C A, {~/,~j(/j)} C E ~, where E ~ is a bundle over A with fibres E~, converge. Let w E A and r/E E ~, respectively, be their limits. It is clear that ~ E~ and 171 > e8 (see (8.4)). We set w' = exp~(~w(r/) + r/). It is obvious that w' e W~'(w) and w' # w. Since lj ~ ~ (it has been pointed out that k= --* ~ ) , we have

I&,(tA - ~ , , . ( % (tA)l - . 0, by virtue of (8.5). Therefore g'Jz,,, ---, w' and so w' E A. This means that (W'~(w) \ {w}) t3 A # z indeed. Let us pass to a flow. We can assume that all z,, E Ds(z) and that they lie in the part of Ds(x) where, in terms of the corresponding "coordinates," Ir < 8/4. It follows from (8.4) and (7.11) that gtz,, leaves Ds(g'z) with the growth of t. But since z= ~ x, the moment of the (first) exit from D6(gtz) increases infinitely as n ~ cr Thus, there are t= > 0 such that t,, ~ cr gt(x,,) E close to t=, for which gtx q[ Ds(gtx). Suppose that

Ds(gtz) for t

g ' ~ . = exp~,=C&Ct) + ~,(t) + r

for

e [0, t~] and there are t, arbitrarily

t e [0,t~l

in terms of the "coordinates" (~, r/, r in Ds(gtz). In accordance with Lemma 3, I#=(t=)l = 8.

(8.6)

Let us prove that I~(t~)

-

~g,~(fh(t~), r

< 28e -~t~.

(8.7)

When proving this relation, it is sufficient to consider only those n for which the left-hand side # 0 (we again can show that this is so for all n, but it is of no importance now). The point = exp~,,~C~,.~C~Ct~), ~(t~)) + ~ ( t ~ ) + ~(t~)) e w ~ C g % ) , and, hence, z E W~s(g"gt"z) with a certain r E [-28,28] (see (7.2)). Therefore

for t < t,, (we use (7.6) and (7.3)). Let us apply Lemma 1 with k = 1 to yl = z,~, y2 = g-t"z (assuming that 38 < 8(1)). Since

16(t.) - ~2(t~)l > 0 = 10~(t~) - n~(t.)l,

16(t=) - 6(t=)l,

in our case in the notations of Lemma 1, it follows that 28 >__]~x(0) - ~2(0)] > e'~rl~l(t, ) - ~2(t,,)], 526

I~l(t,) - ~2(t,,)l <__28e -2=t',

and this is precisely (8.7). We choose a subsequence rj = t,j such that the sequences {g'~x} C A, {flnj(rj)} C E ~, {~,,~(rj)} C E" converge. Let w, q E E", ~ E E", respectively, be their limits. It is clear that w E A, 77 E E~,, ~ E E~, Ir/I = 6 (see (8.6)) and that --,

Since

-,

r

the left-hand side of (8.7) tends to zero and, therefore

--.

as well. We set

= exp~,(~,(r/, r + rt + r Then ~ = limg'Jz,,~ E a and ~ E W]'~(w) so that ~b fi W~s(g~w) with a certain r E [-26, 26]. For sufficiently small 6 the points of the arc {9%; I01 < 26},

(8.8)

that fall in D6(w) have a coordinate 77 there with [r/] < 6/2. (If one wishes, this formally follows from the fact that these points lie on a local weakly stable manifold W~"(w) at all Of whose points Ir/] < 6/2 because, for these points, r / = rho(~, ~'), and for the derivatives of rho, DU/~, = 0 and Dr = 0 for ~ = 0 and ~ = 0. In actual fact, most likely this property of the arc (8.8) is established at the very beginning of the construction of W~(w).) Therefore, ~ does not lie on the arc (8.8). Consequently, the point w' = 9-'tb is precisely the point A n W~(w) different from w. P r o o f of T h e o r e m 3. Let us assume that the hyperbolic ~et A consists of an infinite number of periodic trajectories. Let us take an infinite sequence of different periodic trajectories L,~, choose a point x, E L,~ for each L,,, and pass to a converging subsequence x,, h which we again denote by x,~. Then x,, ~ x, where the point x E A also lies on some periodic trajectory L according to our assumption. There are no other periodic points on W'(L) and W~(L) except for the points L and, since L,, are pairwise distinct, only one of them, at most, may coincide with L. Deleting, in the last case, the corresponding z,~, we find that conditions (8.2) or (8.3) are fulfilled. Then, according to our assumption, the points v and w yielded by Lemma 5 must also be periodic, but then there cannot be periodic points, points of the set A inclusive, on W~(v) \ {v} and

\ P r o o f of T h e o r e m 2. (a) Using Theorem 3 already proved, we "unify" the formulation of Theorem 2 as follows: if there is a nonperiodic trajectory L in the hyperbolic set A, then there exist trajectories L1 and

L2 in it such that (W'(Lx) \ L1) N A ~ o,

(W~(L2) \ L2) n A r o.

(b) Since the trajectory n is nonperiodic, it follows that w(L) ~ L. Indeed, if w(L) = L, then L is Poisson stable in a positive direction (with respect to time). But we have seen in Sec. 3 that the closure of a Poisson stable nonperiodic trajectory consists of an uncountable set of trajectories so that w(L) ~ L. We take a point y E w(L) \ L. There is a sequence of points x~ = gt"x converging to y, t,, ~ oo. If conditions (8.2) or (8.3) are satisfied, with x replaced by y (we call i~ case 1), then, with due account of the remark made after the formulation of Theorem 2, the conclusion of Lemma 5 proves this theorem. It remains to consider the cases (we shall have two of them) where these conditions are not satisfied. (c) First suppose that among the points x,~ there is an infinite subsequence x~k E W{(y), if we speak of a cascade, or Z~h E W~"(y), if we speak of a flow. (We call it case 2.) W.e again denote the points x,,~ by x,,. For a flow, it follows from x,, E W~'~(y) and x,~ ~ y that x,~ fi W~'(y) with certain r,~ ~ 0. We change the notations once again, denoting gt"-~"x,, by x,~. Now for a flow, just as for a cascade, x,~ ~ x and x, E W{(y). When t,,, > t,~, for a cascade and for a flow we have

V"-t,W~(y) = V - - t - W ~ ( x , ) = W ~ ( x , , ) = W~(y), i.e., the manifold W"(y) is periodic. According to Theorem 1, it passes through some periodic point z E A. Let Lx be the trajectory of the latter. Then L C W~(LI). The trajectory L' of the point y also ties in W~(L). Since L' C w(L), the closure L' C w(L) as well, and since L' C W~(L~) and L~ is periodic it follows that 527

LI = ct(L') C -'7 L C w(L). Hence there exists a sequence r, --* or such that y, = g " z --* z. It is clear that the points of the trajectory L = {gtz} for negative t with sufficiently large absolute values lie in W~(L) = U{W~'(x); x e L}, and when t increases, they leave it and never return to W~'(L) again. (Formally, if t > 0, then g-tWO(L) C W['(L), and so the negative semi-trajectory of any point from W~'(L) lies entirely in W~'(L), and therefore it is impossible to leave W~(L) and then return there.) This means that'z,, r W~'(z) in the case of a cascade and z,, r W~"(z) in the case of a flow (for sufficiently large n). If there are z,, in this case, with arbitrarily large n, for which condition (8.2) or (8.3) is fulfilled (with z and z replaced by x,, and z,,), then we have case 1 again. Now if all z,, with sufficiently large n lie on W~(z) or W~'~(z) (we have seen that they cannot lie on W~'(z) or W~"(z)), then it foUows that L C W'(L1) n W"(LI), with L # L~, since the trajectory Lx is periodic and L is nonperiodic. (d) It remains to consider only the case (case 3) where, for sufficiently large n, all x,~ E W~(y), if we speak of a cascade, and z~ E W~'~(y), if we speak of a flow. In the case of a flow, we can slightly "shift" z~ along the trajectory L to make z,, also belong to W~(y). As at the beginning of (c), here the manifold W'(y) is periodic, contains a periodic point z E A, and, if L1 is the trajectory of the latter, then L C Ws(L1). In this case, L # L1 since the trajectory L1 is periodic and L is not. This proves one of the two inferences of Theorem 2, namely, that in which we speak of W'. (e) In order to prove the second inference of the theorem in case 3, we consider a(L). As it was for w, a(L) # L. We take yx E a(L) \ L. There is a sequence t,, --* - c r such that the points z,, = gt"z tend to yx(We no longer need the t,, and z,, used before, and so I use the same letters here.) Now case 3 is divided into three subcases, for two of which we do not need the result given in the preceding paragraph, but we use it in the third subcase. If (8.2) or (8.3) is satisfied with yl substituted for x, then everything is proved, since the time reversal leads to case 1 with yl instead of y. If the infinite number x,, E W~(yl) (when we speak of a cascade) or zn 6 W;r*(yl) (when we speak of a flow), then, by means of time reversal, we reduce the situation to case 2, i.e., the a-limit set becomes the w-limit, the roles of W ~ and W s are switched. Now the statement proved in (c) means that either everything reduces to Case 1 or there is a periodic trajectory L2 C A for which L2 # L and L C W'(L2) f3 W"(L2). Finally, if all z,, E W~'(yx) in the case of a flow or all z,, E W~"(y~) in the case of a cascade, then we carry out time reversal and refer the reader to (d). The statement proved there means now that L lies on a nonstable manifold W~'(L~) of some periodic trajectory L2 C A. This proves the second inference of Theorem 2.

LITERATURE

CITED

. D. V. Anosov and V. V. Solodov, "Hyperbolic sets," In: Sovremennye Problemy Matematiki. Fundamental'aye Napravleniya, Vol. 66, Dynamicheskie Systemy-9, Itogi Nauki i Tekhn., All-Union Institute for Scientific and Technical Information (VINITI), Akad. Nauk SSSR, Moscow (1991), pp. 12-99. . D. V. Anosov, "Geodesic flows on closed Riemannian manifolds of negative curvature," Tr. Mat. Inst. Akad. Nauk SSSR, 90 (1967). .

528

D. V. Anosov and I. U. Bronshtein, "Topological dynamics," In: Sovremenrtye Problemy Matematiki. Fundarnental'nye Napravleniya, VoI. 1, Dynamicheskie Systemy-1, Itogi Nauki i Tekhn., All-Union Institute for Scientific and Technical Information (VINITI), Akad. Nauk SSSR, Moscow (1985), pp. 204-229.

4. A. G. Maier, "On the ordinal number of central trajectories," Dokl. Akad. Nauk SSSR, 59, No. 8, 1393-1396 (1948). 5. A. G. Maier, "On central trajectories and Birkhoff's problem," Mat. Sb., 26, No. 2, 265-290 (1950). 6. V. V. Nemytskii and V. V. Stepanov, The Qualitative Theory"of DifferentialEquations [In Russian], Gostekhizdat, Moscow-Leningrad (1949). 7. L. P. Shilnikov, "Concerning Maier's papers on central motions," Mat. Zametki, 5, No. 3, 335-339 (1969). 8. L. S. Pontryagin, Smooth Manifolds and Their Applications in t[~e Theory of Homotopy [In Russian], Nauka, Moscow (1976). 9. G. de Rham, Varigtgs Diffgrentiables, Hermann, Paris (1955).

529