Multiscale optimization methods and applications

(49)

where, for consistency, both sides are expressed as column vectors. Let Aj denote an eigenvalue of the matrix A~^B, and let r, denote the corresponding eigenvector. By equation (46), this eigenvector satisfies Qrj = 0, therefore we have

284

Ken Chan V^U. =0

Let the point P(xi, to ri(xi, X2,..., x„,l). lies on the surface Q. of <25 and obtain from

.

(50)

xg,..., x„) in nonhomogeneous coordinates correspond Since P lies on the surfaces A and B, therefore it also At this point, we evaluate the n-dimensional gradient equations (48) and (50)

(51) r Br)|^^^^ = AiV(r^Ar)|^^^^ . That is, the two quadratic surfaces A and B are tangent at point P. This proves the theorem. • Remarks: Note that this theorem is very general. If we are dealing with two n-diraensional surfaces other than hyper ellipsoids, then the singular quadratic surface may not be a point (which is a degenerate hyper elhpsoid). In general, we have a singular quadratic surface which may be a line, a plane, or any other surface Q as long as | Q |= 0. What this theorem says is that at the common point P the two surfaces A and B are tangent. This theorem does not say that this tangent point is necessarily an isolated point; it may be a point on a locus at which A and B are tangent. One example of this latter case is a sphere A inside a prolate ellipsoid of revolution B so that the locus of tangent points is a circle which is also the intersection of the plane Q with A (or B). Another example is an elhpsoid A inside a hyperboloid of one sheet B (both being concentric and having the same principal axes) so that the locus of tangent points is an ellipse which is also the intersection of the plane Q with A (or B). Corollary: If two quadratic surfaces A and B intersect (tangency excluded), then no point in the set of points comprising the intersection can correspond to an eigenvector of A~^B. Theorem 4. Converse Tangency. If two quadratic surfaces A and B are tangent at point P, then that point yields the homogeneous coordinates of an eigenvector Vi of A^^B. Proof. Let A and B be tangent at a point P(xj, X2,..., x^) whose homogeneous coordinates are denoted by r^(xj, X2,..., x^,l). Associated with A and B are two corresponding quadratic polynomials r-'^Ar and r-^Br. It follows that their gradients are proportional at point P V(r^Br) 1^^^,^ = A^ V ( / ^ A r ) |^^^,^

(52)

where AJ is some number. Let another quadratic surface Q be defined by r^Qr = r^Br - A-r^Ar = 0

.

(53)

Since A and B are symmetric, therefore Q is also symmetric. Associated with Q is the quadratic polynomial

Determining Intersection of Quadratic Surfaces ^ = r'^Qr = r ^ B r - A^r^Ar

.

285 (54)

Prom equations (52) and (54), we obtain V*lr=r', = 0

(55)

where V ^ is an n-dimensional row vector. Equation (55) states that it has n zero components. Since Q is symmetric, it may be shown that V ^ = 2[I„xnOnxi]Qr

.

(56)

If we evaluate the n-dimensional gradient V<5 at r^, it foUows from equations (55) and (56) that the column vector Qr^ has zero for the first n components, i.e., Qr'i = (0,0,...,0,a)

.

(57)

Since P lies on the surfaces A and B, therefore it also lies on the surface Q. Moreover, this point has nonhomogeneous coordinates obtained from v'^. Hence, the following equation holds

r'fQr', = 0 .

(58)

This yields 0 + l a = 0 so that a = 0

.

(59)

Consequently, Qr'i-0

.

(60)

That is, v[ is an eigenvector of A~^B. This proves the converse theorem. D Remarks: Let two quadratic surfaces A and B have tangent points Pi and P2. Then, the two vectors OPi and OP2 can be colhnear or non-colhnear. The first case can be easily constructed by using the example of a circle concentric with an ellipse whose minor axis is equal to the radius of the circle. This second case can be seen from the Illustrative Example for the case of a
286

Ken Chan p' + Po

.

(61)

However, this is not a convenient form for use in the extended system. Rather, let r and r ' respectively denote the homogeneous coordinates of the same point P. Then, they are related by the following matrix multiplication equation r = Tr'

(62)

where the (n+1) x (n+1) translation matrix T is given by T = T{po)

^nxn Po Olxn 1

(63)

By expanding the determinant of this matrix using the last row of elements and their cofactors, we obtain for all translations \T{Po)\ = l

•

(64)

Prom equation (62), it follows that r' = T^i(p„)r

.

(65)

v' = T{-po)v

,

(66)

Since we also have

it follows directly (without having to perform any matrix inversion) that T-i(po)=T(-p„)

.

(67)

T-\p,)^T^{p,) .

(68)

Note that

That is, the inverse of T is not equal to its transpose. However, by obtaining the inverse of the transpose of T given by equation (63), we obtain

[T'^{po)Y'=^[T-\p,)Y^l

.

(69)

That is, these two operations commute. The above discussion leads us to state the following: Theorem 5. Translation. The (n+1) x (n+1) translation matrix T given by equation (63) has the properties given by equations (64), (67) and (69). Let there be a rotation of axes from (e'^, 63,. • •, e^) to (e", e2,. • •, e^) where e^ and e" respectively denote the unit vectors associated with the positive x[ and x" axes. Let •jij denote the direction cosines of the "angle" between e[ and e'j

Determining Intersection of Quadratic Surfaces

lij=<-e';

.

287

(70)

Let S denote the n x n rotation matrix comprising these direction cosines S = [7d

•

(71)

Let p' = (x[,X2, • • •,x'^) and p" = {x'l,x'2, • • •,x'„) respectively denote the position of a point P in the ' system and the " system, both with the origin at O'. Then, we obtain p' = Sp"

.

(72)

Let r' and r" respectively denote the homogeneous coordinates of the same point P. Then, they are related by the following equation r' = R r "

(73)

where the (n+1) x (n+1) rotation matrix R is given by R = R(S)

^nxn

"nxl

Olxn 1

(74)

By expanding the determinant of this matrix using the last row or column of elements and noting that | S |= 1, we obtain for all rotations |R(S)|=:1

.

(75)

Since S is an orthogonal transformation (in the general sense), it follows from equation (68) that the extended rotation matrix R is also an orthogonal transformation. Consequently, we have R - i =: R^

.

(76)

That is, the inverse of the extended rotation matrix is precisely equal to the transpose. This is in direct contrast to the case of the extended translation matrix T. The above discussion leads us to state the following: Theorem 6. Rotation. The (n+1) x (n+1) rotation matrix R given by equation (74) has the properties given by equations (75) and (76). Remarks: For rotations, the usual and extended equations respectively given by (72) and (73) have the same form involving matrix multiplication. On the other hand, for translations, the usual and extended equations respectively given by (61) and (62) do not have the same form. One has vector sum while the other has matrix multiphcation. The older approaches in three dimensional analytic geometry (as in [Dre64]) have always used nonhomogeneous coordinates. As a result, they had to deal with translations and rotations in differing formulations. Thus, they encountered extensive algebraic

288

Ken Chan

manipulations which can be judiciously circumvented by using homogeneous coordinates. The advantages of this point are obvious in proving the following theorems on invariants. Theorem 7. Invariants. Let C denote the extended matrix whose elements are coefficients of a second degree polynomial with n variables. We shall associate these n variables with the coordinates in a rectangular system of the same dimension. We shall perform any sequence of translations and rotations taken in any order, even remembering that these operations do not commute. After such a sequence of transformations, we can replace the entire process by just a translation followed by a rotation or a rotation followed by a translation. However, in conformance with previously introduced notation in equations (62) and (73), we shall illustrate with a translation T and then a rotation R of axes taking the original system from the unprimed to the double primed variables. Then, the determinant of C is invariant under these transformations. Proof. Thus, we have r^Cr = (TRr")^C(TRr") = r"' R^T^CTRr" = r"' C"r"

(77)

so that the matrix C transforms according to C" = R'^T^CTR

.

(78)

If we take the determinant on both sides, it follows from Theorems 5 and 6 that |C"| = |C|

.

(79)

This proves the theorem. D Remarks: For the case of three dimensions. Theorem 7 is proved through much laborious pains by Dresden [Dre64] who used nonhomogeneous coordinates. If such an approach is taken for higher dimensions, it is conceivable that the methodology would soon break down. Suppose we wish to reduce C to its canonical form containing only diagonal second degree terms, only first degree terms that do not have a second degree term associated with them, and a constant term. This may be accomplished by first performing a rotation R* to eliminate the second degree off-diagonal terms, and then a translation T* to eliminate those first degree terms having a diagonal term associated with them. (Remember that the original coefficients of the linear terms have changed after performing the rotation R*.) Henceforth, we shall let C" denote the canonical form of C. This problem of reducing C to its canonical form is one of two very important transformations involving quadratic surfaces. The other is almost the inverse process: Given a canonical form C", transform it to another set of

Determining Intersection of Quadratic Surfaces

289

axes with orientation and displacement specified in the new coordinate system. A Httle consideration will reveal that, again, we first perform a rotation using the knowledge of direction cosines and then a translation to accomplish the required displacement in the new system. Interchanging these two steps will not yield the desired result. Corollary 1. It follows from the above theorem that (1) For the case of hyper ellipsoids with semi-axes ji, we have 1

ICI = |C"|

(80)

Til! (2) For the case of hyper hyperboloids, we have

ICI

1

|C"| = ±-

(81)

Uif Let us now return to the general case of two quadratic surfaces A and B defined in n-dimensional space, A being non-singular so that its inverse exists. Let D be defined as D =: A - ^ B

.

(82)

Even though A and B are symmetric (so that A~^ is also symmetric), it does not necessarily follow that A^^B is also symmetric. As an example, see the matrix in equation (18). From equation (82), we obtain |D|

(83)

IBI

Let (A ^)" and B " respectively denote the canonical forms of A ^ and B. This leads us to Theorem 8. Determinant. The determinant of the canonical form of the inverse of matrix A is equal to the determinant of the inverse of its canonical form, i.e., (84)

( A - i ) " = (A")-^ Proof. By Theorem 7, equation (79), we have |A"| = lAI (A-) Since | A

-1

=|(A-)|

(85) .

1, it follows from equations (85) and (86) that

(86)

290

Ken Chan IA"|

(87)

(A-^)

However, we also have = 1

(A")"

Consequently, we obtain equation (84). This proves the theorem. D

Remarks: Note that we proved the relation (84) relating determinants of (A~^) and (A )"•' even though it is not true in general that (A"^) = (A )"^. This can be seen from a simple example hke the matrix B given by equation (15) for which we observe that (B~^) ^ (B ) " ^ It is easy to prove that (B-^)" = ( B " ) - I if and only if T = I (i.e., Po = 0). Theorem 9. Combined Invariants. Let A and B be two hyper ellipsoids in n-dimensions. In this case, the inverse exists for both of them. Let D be defined by equation (82). Let ai and Pi respectively denote the semi-axes associated with A and B. Then, we have "

IDI

n\^f

2

(89)

Proof By Theorem 7, equation (79), and equation (83), we obtain (A-)

IDI

(90)

•IB''

Note that in deriving this equation, we do not assume that the transformations which take A to A" and B to B " are necessarily the same. Prom equation (80), we have |B"|

1

(91)

From equations (80) and (84), it follows that

(A-i)" = (A")-^ = - 1 1 " ?

(92)

i=l

By substituting equations (91) and (92) into (90), we obtain equation (89). This proves the theorem. D We can now rewrite equation (43) as |D - All - (-1)"+! [A"+i + d„A" + ... + diA + do]=0

(93)

where, in terms of the determinant | D |, we have according to equation (89)

Determining Intersection of Quadratic Surfaces

291

2

n

do-(-ir+MD| = (-l)"+^n?i ..:/?^

•

(94)

However, we can also write do in terms of the eigenvalues Aj as follows n+l

do = {-ir^'l[Xi

•

(95)

i=i

Prom these two equations, we obtain a relation between the product of the eigenvalues of A~^B and the product of semi-axes of the hyper ellipsoids A and B . This leads us to the following: Theorem 10. Eigenvalues Product. The (n+l) eigenvalues Aj of A~^B and the n semi-axes a, and /3j respectively of the hyper ellipsoids A and B are related by

n^'-nsA •

(96)

Remarks: If the two elhpsoids are translated or rotated, the matrix elements of A and B wiU change. Thus, the eigenvalues of A~^B wiU change, i.e., they are not invariant under rotation and translation. However, the semi-axes will not change because they are the diagonal elements of the canonical forms. This theorem states that do which is the product of the (n-)-l) eigenvalues is invariant under any combination of rotation and translation of A and B independently. This is not the case with the other coefficients dj in equation (93) which depend on the relative position and orientation of the two ellipsoids. This is evident even in the simple case studied in the Illustrative Example, as seen in equation (21). In these statements, it should be borne in mind that we are dealing with the eigenvalues {Aj ; i = 1,2,... ,(n-|-l)} of the extended matrix A~-^B in contrast to the eigenvalues {^,i ; i = 1,2,... ,n} associated with a quadratic form which are invariant under rotation. Needless to say, {ii, }
4 Application to ellipsoids The general theorems previously obtained for n-dimensional quadratic surfaces will now be apphed specifically to the three dimensional physical space so familiar to us. Moreover, we shall first restrict discussion to the case of two ellipsoids. (It helps to refer to Figure 3 which is really for elhpses in the two dimensional case.) Since n = 3, we have four eigenvalues of the matrix A^^B. Let us first consider a tangent point P when the ellipsoids A and B are exterior to each other (referred to as exterior tangency). We note that Vr-^Br is of opposite sign to Vr'^Ar. Hence, one eigenvalue (say Ai) is negative. Moreover, by Theorem 10, equation (96), the product of the four eigenvalues is positive.

292

Ken Chan

Consequently, there must be a second eigenvalue (say A2) which must also be negative. (The other two eigenvalues A3 and A4 must have a product which is positive. It does not matter presently what they are.) We note that when any negative A (not necessarily an eigenvalue) is substituted into equation (41), we obtain a quadratic surface Q which is an ellipsoid. However, when the eigenvalue Ai is substituted into this equation, Q becomes a singular ellipsoid which must therefore be a point. Let the eigenvalue Ai correspond to the eigenvector r i . Then, by Theorem 4, this eigenvector yields the nonhomogeneous coordinates of the tangent point P. Similarly, the eigenvalue A2 corresponds to the eigenvector r2 which yields the same tangent point P since there can be only one exterior tangent point between two ellipsoids. It follows that Ai = A2. Hence, the characteristic polynomial equation (43) has a repeated negative root when the surfaces A and B are exteriorly tangent. We note that the coefficients of this characteristic polynomial are functions of the elements of the matrices A and B. These are alternative ways of expressing the size, shape, orientation and relative position of the ellipsoids A and B. Hence, by subjecting these eUipsoids to undergo continuous variations in any fashion, the eigenvalues of equation (43) will also undergo continuous variations. This forms the basis of our approach in proving the ensuing general results without having to deal with the details of extremely complicated equations. If we move the two elhpsoids closer together or push them apart, then one of these configurations must correspond to the case in which the characteristic polynomial equation has two complex conjugate roots while the other configuration must correspond to the case in which it has two distinct negative roots. Let us first consider the case of two intersecting ellipsoids. For this, we have a locus of points in common between the ellipsoids A and B. Suppose this case corresponds to two negative eigenvalues Ai and A2. The eigenvalue Ai corresponds to an eigenvector ri which yields the nonhomogeneous coordinates of a point Pi on the singular surface Q which is a point. However, by Theorem 1, the set of points comprising the intersection between A and B must also lie on Q. This is impossible. Hence, the case of two intersecting ellipsoids must correspond to the characteristic polynomial equation having two complex conjugate roots. Consequently, the case of two ellipsoids exterior to each other must correspond to the characteristic polynomial equation having two distinct negative roots. Let us push the eUipsoids until they become tangent once again (referred to as interior tangency). At this tangent point, we note that Vr-^Br is of the same sign as Vr'^Ar. Hence, one eigenvalue (say Ai) is positive. Consequently, there must be a second eigenvalue (say A2) which must also be positive. In the configuration with interior tangency, there will be common volume between A and B. Depending on their size, shape, orientation and relative position, there are three possibilities: (i) They intersect again somewhere else; (ii) They become tangent again at another point; and (iii) They do not intersect again (one is contained entirely within the other). If we proceed as for exterior

Determining Intersection of Quadratic Surfaces

293

tangency, we can easily prove that for Cases (i) and (iii) we must have Ai = A2. To prove that Case (ii) also has two equal positive eigenvalues, let us consider Case (i) continuously deforming into Case (iii). We know that the eigenvalues also undergo continuous variations. It follows that at the transition (for which we have two interior tangent points), we stiU have Ai = A2. Note that this result is true for any two interior points in general. (In this process just discussed, the singular quadratic surface Q degenerates from a cone in Case (i) to a straight line or a plane in Case (ii) and then to a point in Case (iii).) We are able to prove this equahty of eigenvalues without having to deal with the details of the ehipsoidal surfaces. All the preceding theorems and discussion lead us to the following extremely general theorem on relative configuration based solely on eigenvalues. Theorem 11. Characteristic Equation. Let A and B denote two ellipsoids. If the extended 4^4 inatrix A~^B (or B~^A) has • • • • •

two distinct negative eigenvalues, then A and B are exterior to each other; two equal negative eigenvalues, then A and B are exteriorly tangent; two complex conjugate eigenvalues, then A and B intersect; two equal positive eigenvalues, then A and B are interiorly tangent at one or two points; two distinct positive eigenvalues, then A and B intersect or one is entirely within the other with no points of tangency.

Corollary 2. In view of Cases (2) and (4), it is also impossible for the matrix A~^B to have two pairs of equal eigenvalues of opposite signs (say Ai == A2 > 0 and A3 = A4 < 0). We will now apply this theorem to study the three cases above. For simplicity, we shall consider the two dimensional case as already exemplified by the Illustrative Example. We wish to observe how the eigenvalues change as the elUpse B (with semi-axes b and c) passes through the circle A (with radius a). Case (i) is illustrated by Stage 4 in Figure 4 for which c< a
For Case (ii), we consider a
294

Ken Chan

^r^f^

<==

(J) Two complex corrugate roots

(2) Two equal negative roots

-co

CD (4) Two equal positive roots

(1) Two distinct negative roots

(5) Two tlistinctpositivB roots

<=>

(9) Two distinct negative roots

c=0 (8) Two eqxjal negative roots

(e^ Two equal poatiye rooU

<:="

(7) Two complex conjugate roots

Fig. 4. Transition of Eigenvalues for the Case of c < a < b

(3) Two complex conjugate roots

(2) Two equal negative roots

(1) Two distinct negative roots

^

(4) Two equal positive roots

(9) Two distinctne^tive roots

(5) Two distinct positive roots

(8) Two equal ne^tive roots

(6) Two equal positive roots

(7) Two complex conjugate roots

Fig. 5. Transition of Eigenvalues for the Case of c < 6 < a

Determining Intersection of Quadratic Surfaces

295

stages are the same as before. If c is much larger than a, then Stages 4 through 6 (as well as Stages 8 through 10) collapse into one stage with only one tangent point. This critical condition depends on the curvature of the two conies. For example, see the Illustrative Example, equation (30). All these illustrations are included to clarify what could be an extremely difficult situation to describe analytically. They do not constitute a proof of the above theorem whose proof has been given already.

(J) Two complex corpugite roots

(2) Two equal regative roob

(1) Two distinct negative roob

(h ' CD ' (4) Two equal positive loots

(5) Two distinct positive mots


(9) Two distind positive roob

(6) Two eqoal positive loots

<^

(8) Two equal positive loots

(7) Two distinct pos itive roots

4

C10) Two eqaal positive roots

(ll)'Iwocon^lexcor5ugate loob

(12) Two equal negative roob

4

(i;^ Two distinct negative roob

Fig. 6. Transition of Eigenvalues for the Case of a < c < b

Ken Chan

296

Finally, consider Case (4) of Theorem 4 for which we have two equal positive eigenvalues such that A and B are interiorly tangent at two points Pi and P2. This is illustrated in Figure 7 for two ellipses. Note that Figure 7a depicts the case of two colhnear vectors OPi and OP2, each of which corresponds to two equal positive eigenvalues Ai = A2. Also note that these two colhnear vectors are not coUinear in the extended form since they both have 1 in the last component. What is not so obvious is that in Figures 7b through ??d it is also true that each of the two vectors OPi and OP2 has the same two equal positive eigenvalues Ai = A2 even though OPi and OP2 are not cohinear. This property follows from the fact that because Pi and P2 are common to A and B, they also lie on the singular quadratic surface Q defined by Q = B - AiA.

(a) - Collinear Vectors Figure

(b) - Non-collinear Vectors

0'

2 (c) - Non-coIIinear Vectors Figure

^2 (d) - Non-collinear Vectors

Fig. 7. Illustrative Cases of Two Vectors with Equal Eigenvalues

Theorem 12. Multiple Interior Tangent Points. Let A and B denote two ellipsoids. Suppose the extended 4 '>^ 4 matrix A~^B (or B~^A) has two equal positive eigenvalues (say Ai = A2) such that A and B are interiorly tangent at two points Pi and P2. Then, the two vectors OPi and OP2 are each associated with the same two equal positive eigenvalues Ai and A2. These

Determining Intersection of Quadratic Surfaces

297

tangent points may not be isolated points; they may be on a locus of points at which A and B are tangent. Corollary 3. / / the singular quadratic surface Q defined by Q = B - XiA is a straight line, then there can be at most only two interior tangent points common to A and B. Corollary 4. / / the singular quadratic surface Q defined by Q = B - XiA is a plane intersecting A (or B), then there are infinitely many interior tangent points common to A and B. These tangent points form a locus which is given by the intersection of Q and A (or B). Remarks: A singular conic can only be a point, a pair of intersecting lines, a pair of parallel lines or a coincident line. Therefore, there can be only one or two tangent points between two conies in general. In contrast, a singular degenerate quadric (which is of rank less than 3) can only be a straight line, a pair of intersecting planes, a pair of parallel planes or a coincident plane. If we also include a point (which is actually a singular non-degenerate quadric of rank 3), then for these singular quadrics Q there can be only one, two or infinitely many tangent points between two quadrics A and B in general. However, the author has not analyzed the case of other singular non-degenerate quadrics (a double elliptic cone and a cyhnder of the elliptic, hyperbolic or parabolic type) to arrive at any conclusion. It is possible that the above statement is still valid.

5 Discussion It is obvious that the foregoing theorems are applicable to the case of n dimensions. All the results we obtained in the simple Illustrative Example on conies become so obvious now. If we had attempted a more difficult two dimensional or a three dimensional problem, we would have been mired in excessive comphcated mathematical details without being able to obtain the answer. We achieved our objective by using abstract symbolism and invariant properties of the extended A~^B matrix which circumvented substantial unnecessary algebraic manipulations. Moreover, we may apply the above process to analyze other configurations such as an ellipsoid and a hyperboloid of one or two sheets, an ellipsoid and an elliptic paraboloid, two hyperboloids of two sheets, etc. At a tangent point for each of these cases, we have to cautiously determine whether Vr-^Br is of the same sign as Vr'^Ar or otherwise. This is especially true when we deal with open surfaces such as hyperboloids of one or two sheets, elliptic paraboloids and hyperbolic paraboloids. The convention for this sign determination is intrinsically given by the polynomials associated with the quadratic surfaces. Rather than delve into these more difficult topics, it is instructive to note that all the classical analyses of lines and planes intersecting or tangential

298

Ken Chan

to conies and quadrics can now be easily and compactly treated in a unified way. That is, all the equations previously derived for secants, tangents, normals and contact points can now be obtained and studied more qualitatively. Moreover, we can also extend this approach to study the intersection of other improper quadrics such as cyhnders of the elliptic, hyperbolic or parabolic types with any of the five proper quadrics. Even these simpler topics will provide substantial material for further investigations.

6 Conclusion The foregoing analysis presents a bridge between two seemingly disparate topics in mathematics. One deals with the intersection or tangency of surfaces in solid analytical geometry while the other reveals the relationship of the eigenvalues of an associated problem in linear algebra.

References [AGOl] S. Alfano and M. L. Greer, "Determining If Two Ellipsoids Share the Same Volume," AAS/AIAA Astrodynamics Specialists Conference, Quebec, Canada, July 2001. [ChaOl] K. Chan, "A Simple Mathematical Approach for Determining Intersection of Quadratic Surfaces," AAS/AIAA Astrodynamics Specialists Conference, Quebec, Canada, July 2001. [Pet66] A. J. Pettofrezzo, Matrices and Transformations, Dover Publications, New York, pp 103-110 (1966). [Dre64] A. Dresden, Solid Analytical Geometry and Determinants, Dover PubUcations. New York, pp 197-205, 230 (1964). [GKT68] Granino A. Korn and T. M. Korn, Mathematical Handbook for Scientists and Engineers, McGraw-Hill, New York, p 75 (1968).

Applications of Shape-Distance Metric to Clustering Shape-Databases Shantanu H. Joshi^ and Anuj Srivastava^ ' Department of Electrical Engineering, Florida State University, Tallahassee, PL 32310 USA. [email protected] ^ Department of Statistics, Florida State University, Tallahassee, FL 32306 USA. anujOstat.fsu.edu

Summary. Based upon the geometric approach to clustering outlined in [KSMJ04], this paper presents an application to hierarchical clustering of imaged objects according to the shapes of their boundaries. The shape-distance used in clustering is an intrinsic metric on a nonlinear, infinite-dimensional shape space, obtained using geodesic lengths defined on the manifold. This analysis is landmark free, does not require embedding shapes in R^, and uses ODEs for flows (as opposed to PDEs). Clustering is performed in a hierarchical fashion. At any level of hierarchy clusters are generated using a minimum dispersion criterion and an MCMC-type search algorithm is employed to ensure near-optimal configurations. The Hierarchical clustering potentially forms an efficient (O(logn) searches) tool for retrieval from shape databases.

1 Introduction Numerous problems in pattern recognition deal with classifying and labeling objects of interest present in the observed images. Discriminating features of the objects can be characterized according to their location, pose, textures, colors and shapes. Shape is a feature that is receiving widespread attention in analysis of images or improving understanding of objects that are changing in a scene. Analysis of shapes, especially those of complex objects, is a challenging task and requires sophisticated mathematical tools. Various applications of shape analysis include biomedical image analysis, automatic surveillance, biometrics, military target recognition and general computer vision. There have been various breakthroughs in understanding of shape. A majority of this research has been restricted to landmark-based analysis [DM98], where shapes are represented by a course, discrete sampling of the object contours. Shape clustering and learning using the resulting Procrustean analysis are discussed in [DJDOl, DSJ99]. Klassen et. al. [KSMJ04, SMKJ03], propose a new metric on the space of continuous, closed curves in IR'^, without a need for defining landmarks. The computation of this metric obviates the use of

300

Shantanu H. Joshi and Anuj Srivastava

computationally expensive tools such as Euclidean embeddings and nonlinear PDEs and uses ODEs instead. The idea involves identifying a space S of allowable shapes and imposing a Riemannian structure and further utilizing its geometry to solve optimization and inference problems. For planar curves in M?, of length 2TT and parameterized by the arc length, the coordinate function a{s) relates to the direction function 6{s) according to a{s) = e*^^*\ i = A/—1. The planar closed curves are represented by their angle functions. Considering closed curves, and making them invariant under rigid motions (rotations, translations), and uniform scaling, one obtains: C=ie£

L ^ l ^ f ^ e{s)ds = TT, f ^ e^^'Wds = o | .

(1)

The paper [KSMJ04] describes algorithms for computing geodesic paths between any given shapes. It also defines an intrinsic mean on the shape space and describes an algorithm of computing the mean for a set of shapes. The length of the geodesic d{9i,9j) for any 6i,9j S S can be used to quantify shape differences. This metric once computed, can be used to solve a host of challenging problems in image analysis, segmentation and feature extraction. In this paper in particular we focus on the clustering algorithm used to partition a group of shapes. Clustering can be further extended in a hierarchical fashion leading to the organization of a large database of objects according their shapes in a manner that allows for efficient searches and retrievals.

2 Shape Clustering An important need in shape studies is to classify and cluster previously observed shapes. In this section, we develop an algorithm for clustering of objects according to shapes of their boundaries. It is assumed that the task of boundary extraction or segmentation of the shape from the image is already completed. Thus we are presented with a set of 2D planar closed contours in the form of their co-ordinate functions. The shape-distance metric is computed pair-wise between all the set of shapes. In Euclidean spaces with the usual metric, there are several algorithms for clustering that can be adapted to the shape space S using its Riemannian structure. However, to keep the algorithm simple and efficient, we restrict to a simple modification of the kmean clustering idea. This modification is applied since the computation of means in S is an iterative and a computationally costly procedure, and we try to minimize that calculation. 2.1 Minimum Pair-wise dispersion Clustering We view clustering as a problem of partitioning n shapes (in S) into k clusters. To motivate our algorithm, we begin with a discussion of a classical clustering

Applications of Shape-Distance Metric to Clustering Shape-Databases

301

procedure for points in Euclidean spaces, which uses the minimization of the total variance of clusters as a clustering criterion. More precisely, consider a data set with n points {yi,y2, • • • ,yn} with each y, G R'^. If a collection C = {Ci, 1 < i < fc} of subsets of W^ partitions the data into k clusters, the total variance of C is defined by Q{C) = S j = i Ylyed 11^ ~ MilPi where n IS the mean of data points in d. The term Y^^Q. \\y — MilP can be interpreted as the total variance of the cluster Q . The total variance is used instead of the (average) variance to avoid placing a bias on large clusters, but when the data is fairly uniformly scattered, the difference is not significant and either term can be used. The widely used k-Means Clustering Algorithm is based on a similar clustering criterion (see e.g. [JD98]). The soft k-Means Algorithm is a variant that uses ideas of simulated annealing to improve convergence [BH91, R98]. These ideas can be extended to shape clustering using d{9, iXiY instead of \\y — HiW^, where d{-, •) is the geodesic length and /Xj is the Karcher mean of a cluster Ci on the shape space. The Karcher mean is an intrinsic mean on the shape space that is obtained after an iterative procedure that minimizes the average variance for all the set of shapes. However, the computation of Karcher means of large shape clusters is a computationally expensive operation. Thus, we propose a variation that replaces d{9,jii)'^ with the average distance-square Vi{9) from 9 to elements of Cj. If Ui is the size of Ci, then Vi(0) = :^ X^g'gc, d{9,9')'^. The cost Q associated with a partition C can be expressed as

Q(^) = E ^ ( E i=\

E

* \BaeCib
rf(^-^6)M.

(2)

J

The scale factor in the above equation of Q{C) ensures there is no additional preference to those clusters having a large number of shapes, while moving shapes from one cluster to other. The algorithm is outUned next. Algorithm 2.1 For n shapes and k clusters initialize by randomly distributing n shapes among k clusters. Set a high initial temperature T. 1. Compute pairwise geodesic distances between all n shapes. This requires n{n — l ) / 2 geodesic computations. 2. Pick a shape 9j randomly. If it is not a singleton in its cluster then compute Qf for alii = 1,2,...,k. 3. Compute the probability PT{J, i) for all i = 1,... ,k and re-assign 9j to a cluster chosen according to the probability PT{j,i). 4- Update temperature using T = T/0 and return to Step 2. We have used (3 = 1.0001 in our experiments. The set of all configurations of n shapes into k clusters is finite, and the stochastic nature of Algorithm 2.1 guarantees the convergence to the optimal

302

Shantanu H. Joshi and Anuj Srivastava

configuration as long as T is reduced slowly. Instead of analyzing convergence theoretically, we present some experimental results. The Algorithm 2.1 is apphed to a collection of 25 shapes from the Kimia database [SKK03] shown in Figure 1. Left panel of Figure 1 shows the 25 shapes. The Right panel of Figure 1 shows the cluster configuration for fc = 6 clusters. The success of Algorithm 2.1 in clustering these diverse shapes is visible in these results; similar shapes have been clustered together.

3J532:s-=21 iL^J^^JI^/.)} !>== i i = i > ^ 1== ZZSO-Y 3EaH E E H B E YYYY X fco-[>E

zzzz

Fig. 1. Left panel shows 25 objects whose shapes are analyzed here. They are numbered from top left to bottom right in increasing order. Right panel shows a clustering arrangement (row-wise) for the 25 objects into 6 clusters.

3 Hierarchical Organization An important goal of this paper is to organize large databases of shapes in a fashion that allows for efficient searches. One way of accomplishing this is to organize shapes in a tree structure, such that shapes are refined regularly as we move down the tree. In other words, objects are organized (clustered) according to coarser differences (in their shapes) at top levels and finer differences at lower levels. This is accomplished in a bottom to top construction as follows: start with all the shapes at the bottom level and cluster them according to Algorithm 2.1 for a pre-determined k. Then, compute mean shape for each cluster and at the second level cluster these means according to Algorithm 2.1. Applying this idea repeatedly, one obtains a tree organization of shapes in which shapes change from coarse to fine as we move down the tree. Critical to this organization is the notion of the intrinsic mean of shapes as discussed earlier. Shown in Figure 2 is an example of a tree structure obtained for clustering result for 25 shapes shown in Figure 1. At the bottom level, these 25 shapes are clustered in fc = 6 clusters, with the clusters denoted by the indices of their element shapes. Computing the means of each these six clusters, we obtain the

Applications of Shape-Distance Metric to Clustering Shape-Databases

303

last to bottom row of shapes. Repeating the clustering for fc = 4 clusters we obtain the next level and subsequently their means. In this example, we have chosen to organize shapes in four levels with a single shape at the top level. The choice of parameters such as the number of levels, and the number of clusters at each level, depends on the required search speed and performance.

19 22 10 13 20 21

3 5 6 7 24

9 15 16 17

4

11 12 18 23

2 14 25

Fig. 2. Hierarchical Organization of 25 shapes

4 Conclusion We have presented a hierarchical organization of shapes based upon the shapedistance metric which utilizes the Riemannian structure of the shape space. Clustering is performed efficiently by minimizing the pair-wise average variance within the clusters and can be used in clustering of shape databases of objects. Hierarchical clustering reduces the search and test times for shape

304

Shantanu H. Joshi and Anuj Srivastava

queries against large databases. This has enormous potential for systems which use shape based object retrieval.

References [BH91] Brown, D.E. and Huntley, C.L. (1991), "A practical application of simulated annealing to clustering," Technical Report IPC-TR-91-003, Institute for Parallel Computing, University of Virginia, Charlotesville, VA. [DM98] Dryden, I.L. and Mardia, K.V. (1998), Statistical Shape Analysis, John Wiley & Son. [DJDOl] Duta, N., Jain, A.K. and Dubuisson-Jolly, M.-P. (2001), "Automatic construction of 2D shape models," IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 23, No. 5. [DSJ99] Duta, N., Sonka, M., and Jain, A.K. (1999), "Learning shape models from examples using automatic shape clustering and Procrustes analysis," Proceedings of Information in Medical Image Processing. Vol. 1613 of Springer Lecture Notes in Computer Science, pp. 370-375. [JD98] Jain, A.K. and Dubes, R.C. (1998), Algorithms for Clustering Data, Prentice-Hall. [KSMJ04] Klassen, E., Srivastava, A., Mio, W., and Joshi, S.H. (2004), "Analysis of planar shapes using geodesic paths on shape spaces," IEEE Pattern Analysis and Machine Intelligence,Yol. 26 No. 3, pp. 372-383. [R98] Rose, K. (1998), "Deterministic annealing for clustering, compression, classification, regression, and related optimization problems," Proceedings of IEEE, 86(ll):2210-2239. [SKK03] Sebastian, T.B., Klein, P.N. and Kimia, B.B. (2003), "On aligning curves," IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 25, No.l. [SMKJ03] Srivastava, A., Mio, W., Klassen, E. and Joshi, S.H. (2003), "Geometric Analysis of Continuous, Planar Shapes," Proceedings of Energy Minimization Methods in Computer Vision and Pattern Recognition., Vol 2683 of Springer Lecture Notes in Computer Science, pp. 341-356.

Accurately Computing the Shape of Sandpiles"* Christopher M. Kuster and Pierre A. Gremaud Department of Mathematics and Center for Research in Scientific Computation, Raleigh, NC 27695, USA. {cmkuster,gremaud}9math.ncsu.edu

Summary. Using an Eikonal formulation, we model the surface of sandpiles formed in regions containing obstacles. The fast marching method is adapted to have the optimal rate of convergence. We also apply the fast marching method to an industrial problem. K e y w o r d s : Fast Marching, upwind methods, Eikonal equation.

1 Introduction There are many examples of granular materials in industrial applications. These range from ore in mining operations to fine powders in the pharmaceutical industry. It is common to store these materials in piles, either in bins or in the open. Often, the area in which these piles are formed contains soHd obstacles which the grains must flow around in the forming of the pile. Our goal is to accurately measure the volume of such piles using only observations of the surface. In [ABG03], we created a mathematical model of the system, discretized the corresponding equations, and used the Fast Marching method to solve for the surface profile. This gave a method t h a t converged to the proper solution upon refinement, b u t it did so more slowly t h a n expected. Here, we modify the procedure to improve accuracy. T h e only physical parameter t h a t we consider for our model is the angle of repose. This is the maximum angle of inclination the surface of the grain can have and still be at rest. It roughly corresponds to the angle of internal friction of the material. For the purposes of our model, we assume t h a t the angle of inclination of the surface from vertical is equal to the angle of repose at every point. In one dimension, our assumption leads to the equation This research was supported by the National Science Foundation (NSF) through grant DMS-0244488.

306

Christopher M. Kuster and Pierre A. Gremaud |/'(x)|=tan5

where f{x) defines the height of the material at a point x, and S is the angle of repose. Requiring this equation to be satisfied nearly everywhere leads to an infinite number of solutions (See figure 1 (left)). In order to obtain a unique solution, we add the condition that the surface decrease with distance from the source point (See figure 1 (right)). This is commonly referred to as the viscosity solution. Generalizing this to multiple dimensions results in the Eikonal equation |V/(a:)| = t a n ^

Fig. 1. Possible solutions to the 1-D equation Obstacles can be modeled by modifying the right hand side in an appropriate way, namely

\yf{x,y)\

=

S{x,yJ{x,y))

where S, the slowness function, is defined by f tan ^ if {x, y, f{x, y)) ^ Obstacle, loo if (a;,y,/(x,y)) S Obstacle Travel time problems are closely related to the above formulation. In this paper, we look at prismatic obstacles. That is, obstacles with a cross sectional area that does not change with height. In this case, the slowness function is only a function of x and y. It is possible that our methods can be adapted to non-prismatic obstacles.

2 Numerical M e t h o d s One standard way of solving Eikonal equations is the Fast Marching Method (FMM) ([Tsi95, Set99]). The FMM is a wave propagation method based on upwind formulations. It starts with known values on a region and propagates the solution outward from there. The FMM is described by the following algorithm

Accurately Computing the Shape of Sandpiles f = ( o o , . . . ,00) "front" = "source" while "front" ^ 0 fmin = m i n f G " f r o n t " remove tmm from " f r o n t " add n e i g h b o r s of fmin t o " f r o n t " f o r ie G " f r o n t " solve F(fi,,..,f^_i, ff,f^+i,...,fN) = 0 end end

for

307

f^

In order to find fmin, Fast Marching uses a heap sort [Wil64]. This is a d a t a structure t h a t requires O ( l o g n ) operations in order to add, remove, or u p d a t e a node, where n in the number of nodes in the d a t a structure. T h e Fast Marching Method malces the assumption t h a t the slowness function is continuous. This is not the case in our problem, and in fact, our slowness function has a singularity inside every obstacle. When applying the F M M with a uniform mesh (see figure 2 (right))to a region with a cyhndrical obstacle, a loss of accuracy is observed. By adding a small number of extra mesh points around the b o u n d a r y of the obstacle, the F M M can be made to work weU in a region with singularities. In particular, new mesh points are added at the intersection of the hnes in the uniform mesh with the obstacle boundary (see figure 2 (left)).

41

()

4

•

't

•

41

(I

(

' t ^

II ^1

•

m

i

f

p

• >—

•

• •

0

•


s

;,

• >—

A

^

——^

i

• •

h-*—s

«

Fig. 2. Uniform Grid (left), Modified Grid (right)

This addition of nodes creates a non-Cartesian mesh near the obstacle. We use the process described in [SVOO, SV03], and consider a node XQ at which the solution / is t o be computed and two neighboring nodes xi and X2 at which the values of / {fe, i = 1,2) and its derivatives {dxfe,dyfe, i = 1,2) are known or have already been computed (See figure 3). We define

Ne

XQ

— X(

\Xo — X(

1,2

and

Af •

No

308

Christopher M. Kuster and Pierre A. Gremaud

where A/" is a 2x2 nonsingular matrix, assuming XQ, xi and 0:2 are not co-linear. The directional derivatives in the directions A''i and A^2 are approximated by Def = ,

\

\xo

£=1,2

(to 1st order),

Def = 2/° 2^!\ -Nf\dJt,dyfe\ £=1,2 (to 2nd order). I Xo Xe\ Those approximate directional derivatives are linked to the gradient by D,J = Ne-Vf

(1)

-Xi\

+ 0{\xo-Xin

or

1 5 / = A ^ V / + 0(/i«),

(2)

(3)

where Df = [Di/, D2/], h = max{|xo — xi|,|a;o — X2I}, and a = 1 or 2 depending on whether (1) or (2) is respectively used. Solving for V / and plugging into (1) results in the quadratic equation defining the unknown /o Df\UM'r'Df

= {S{x^))\

(4)

Fig. 3. Direction vectors for non-Cartesian grid.

3 Results 3.1 Without Obstacle With no obstacle, the solution is a cone, see figure 4 (left). Since there is no obstacle, the slowness function is continuous and the results are identical to those obtained by the Fast Marching Method on a uniform grid. The order of convergence is first order or second order depending on the approximation used for the directional derivatives (1) or (2).

Accurately Computing the Shape of Sandpiles

309

Fig. 4. Contours of surface ( f ) with and without obstacles

3.2 With Cylindrical Obstacle The exact solution is the generalized distance function from the source. This is consistent with observation (see figure 4 (right)). Table 1 is a grid refinement study looking a t the convergence rate of the L~ and L" errors. With a uniform grid, accuracy is lost due t o the singularity at the obstacle. By adding in nodes around the obstacle boundary, the second order accuracy expected from our second order discretization is preserved in the L1 norm. The first order results obtained for the Lm norm are expected due t o the "shock" behind the obstacle. Figure 5 shows the absolute error in the solutions produced using the uniform grid (Left), and the modified grid (Right) with M = 1600.

Fig. 5. Error plots for: Uniform Grid (left), Modified Grid (right)

3.3 Application

An application of this process is the determination of the volume of material in a bin with inserts, see figure 6. From this picture, it can be seen that the insert is not a prismatic obstacle. Preserving optimal accuracy is much more challenging here as the (x, y)-projection of the line of contact between

310

Christopher M. Kuster and Pierre A. Gremaud

M 25 50 100 200 400 800 1600 3200

L' 2.91(0) 1.71(0) 1.07(0) 6.42(-l) 3.76(-l) 2.21(-1) 1.31(-1) 7.80(-2)

M 25 50 100 200 400 800 1600 3200

L' 2.52(0) 1.45(0) 6.36(-l) 3.02(-l) 1.52(-1) 6.91(-2) 3.42(-2) 1.48(-2)

first order methods standard method rate L' rate L°° rate L' 9.00(-l) 6.11(-1) 1.13(0) .77 5.24(-l) .78 3.52(-l) .79 7.58(-l) .68 3.27(-l) .68 2.23(-l) .66 4.41(-1) .74 2.00(-l) .71 1.46(-1) .61 2.39(-l) .77 1.20(-1) .74 9.36(-2) .64 1.27(-1) .77 7.24(-2) .73 6.07(-2) .62 6.38(-2) .76 4.40(-2) .72 3.82(-2) .67 3.07(-2) .75 2.68(-2) .72 2.44(-2) .65 1.49(-2) second order methods standard method rate L ^ rate L°° rate L' 1.07(0) 8.99(-l) 1.17(0) .80 5.94(-l) .85 7.39(-l) .28 2.83(-l) 1.2 2.45(-l) 1.3 3.10(-1) 1.3 1.23(-1) 1.1 1.20(-1) 1.0 1.73(-1) .84 3.29(-2) .99 6.16(-2) .97 8.49(-2) 1.0 1.52(-2) 1.1 2.91(-2) 1.1 4.41(-2) .95 2.06(-3) 1.0 1.45(-2) 1.0 2.31(-2) .93 4.82(-4) 1.2 6.36(-3) 1.2 1.12(-2) 1.0 1.15(-4)

modified method rate L^ rate L°° 3.04(-l) 1.44(-1) .58 2.00(-l) .60 9.18(-2) .78 1.13(-1) .82 4.92(-2) .88 6.10(-2) .89 2.65(-2) .92 3.24(-2) .91 2.02(-2) .99 1.64(-2) .99 1.54(-2) 1.1 7.91(-3) 1.0 1.14(-2) 1.0 3.88(-3) 1.0 6.77(-3) modified method rate L°° rate 1/ 2.84(-l) 1.16(-1) 2.0 9.59(-2) 1.6 4.54(-2) 1.2 4.26(-2) 1.2 1.97(-2) 1.9 1.14(-2) 1.9 1.22(-2) 1.1 5.32(-3) 1.1 5.93(-3) 2.9 8.22(-4) 2.7 3.11(-3) 2.1 2.47(-4) 1.7 1.68(-3) 2.1 7.64(-5) 1.7 8.83(-4)

rate .65 .88 .91 .39 .39 .43 .75

rate 1.4 1.2 .70 1.0 .94 .89 .93

Table 1. Convergence study for formally first and second order methods in the presence of a circular obstacle (Example 2); M measures the number of nodes on one edge of the domain, i.e., total number of nodes N = 0{M'^).

material and obstacle is not known in advance. The results shown below were computed on a uniform grid and are thus not optimally accurate. By applying the fast marching process to several different source heights, a chart can be made of the volume of material in the bin as a function of the height of the surface at the source. Figure 7 shows such a chart for materials with several different angles of repose.

4 Conclusions An Eikonal formulation of general sandpile formation is proposed. The problem is discretized and solved. It is shown how to maintain optimal accuracy with fast marching when obstacles are present. The feasibility of the approach is illustrated by the solution of an industrial problem. Future work will address how to maintain optimal accuracy for non-prismatic obstacles.

Accurately Computing the Shape of Sandpiles

311

Fig. 6. Example of a bin with an insert

Hopper wIBINSERT, Materials wldiffering angle of repose

0

I

I

I

I

2

4

6 Height (feet)

8

1

30 degrees 40 degrees ------50 degrees --.-----

10

Fig. 7. Volume as a function of height for various materials

12

312

Christopher M. Kuster and Pierre A. Gremaud

References [Abg96] R. ABGRALL, Numerical discretization of the first order Hamilton-Jacobi equations on triangular meshes, Comm. Pure Appl. Math., 49 (1996), pp. 1339-1377. [ABG03] S.A. AHMED, R . BUCKINGHAM, P.A. GREMAUD, C . D . HAUCK, C.M. KUSTER, M . PRODANOVIC, T . A . ROYAL, V. SILANTYEV, Volume

[HS99] [0S91] [QS02] [Set99] [SVOO]

[SV03]

[Tsi95] [Wil64]

determination for bulk materials in bunkers, submitted to Int. J. for Num. Math, in Eng., Center for Research in Scientific Computation, NCSU, Technical Report CRSC-TR03-24. C. Hu, AND C.-W. SHU, A discontinuous Galerkin method for HamiltonJacobi equations, SIAM J. Sci. Comput., 21 (1999), pp. 666-690. S. OSHER, AND C.-W. SHU, High-order essentially nonoscillatory schemes for Hamilton-Jacobi, SIAM J. Numer. Anal., 28 (1991), pp. 907-922. J. QiAN, AND W.W SYMES, An adaptive finite difference method for traveltimes and amplitudes. Geophysics, 67 (2002), pp. 167-176. J.A SETHIAN, Fast marching methods, SIAM Review, 41 (1999), pp. 199235. J. A. SETHIAN, AND A. VLADIMIRSKY, Fast methods for the Eikonal and related Hamilton-Jacobi equations on unstructured meshes, Proc. Natl. Acad. Sci. USA, 97 (2000), pp. 5699-5703. J. A. SETHIAN, AND A. VLADIMIRSKY, Ordered upwind methods for static Hamilton-Jacobi equations: theory and algorithms, SIAM J. Numer. Anal., 41 (2003), pp. 325-363. J.N. TsiTSiKLiS, Efficient algorithms for globally optimal trajectories, IEEE Trans. Automat. Control, 40 (1995), pp. 1528-1538. J.W.J WILLIAMS, Heapsort, Com. ACM, 7 (1964), pp. 347-348.

Shape Optimization of Transfer Functions^ Jiawang Nie and James W. Demmel Department of Mathematics, University of California, Berkeley, CA 94710, USA. {nj w,demmel}9math.berkeley.edu

Summary. We show how to optimize the shape of the transfer function of a linear time invariant (LTI) single-input-single-output (SISO) system. Since any transfer function is rational, this can be formulated as an optimization problem for the coefficients of polynomials. After characterizing the cone of polynomials which are nonnegative on intervals, we formulate this problem using semidefinite programming (SDP), which can be solved efficiently. This work extends prior results for discrete LTI SISO systems to continuous LTI SISO systems. K e y w o r d s : Linear system, transfer function, shape optimization, nonnegative polynomials, convex cone, semidefinite programming.

1 Introduction Consider the following linear time invariant (LTI) single-input-single-output (SISO) system i = Ax + bu

(1)

y = c^x + du

(2)

where yl € K " ^ " , 6, c G R " , d £ R. T h e transfer function is H{s) = d + A)~'^b, which can also be written as the rational function

c^{sl-

ELpQ'fc's'' = gi(g) Note t h a t deg[qi) < deg(g2) < n. Conversely any function H{s) of this kind is the transfer function of some LTI system. Any such a LTI system is called a realization of H{s). There are many such (algebraically equivalent) LTI systems [CD91, chap. 9]. * This research was supported by the National Science Foundation Grant No. EIA0122599.

314

Jiawang Nie and James W. Demmel

In many engineering applications, we want the transfer function to have certain attractive properties. For example, we may want the Bode plot (the graph of |i?(s)| along the pure imaginary axis s = j • UJ) to have a certain shape corresponding to some kind of filtering. In this paper we study the shape optimization problem of choosing the coefficients of the rational function H{s) so its Bode plot has some desired shape. Now consider a discrete LTI system, i.e. the governing differential equation (l)-(2) is replaced by the difference equation 52fc=i C(kz{n—k) = X]^=i Pe.v[n— t) where {u(fc)}^j is a sequence of discrete inputs and {z{k)}^^^ is the sequence of state variables. In this case there are several nice papers [AV02, GHNOO, WBV97] that show how to formulate the filter design problem as the solution of the feasibility problem for certain convex sets. The main idea is to apply the spectral factorization of trigonometric polynomials, a characterization of nonnegative univariate polynomials, and semi-infinite programming. This approach can be used to design the transfer function to be a bandpass filter, piecewise constant or polynomial, or even have an arbitrary shape. Our contribution is to extend these results to continuous time LTI SISO systems (l)-(2). In this case the transfer function is not a trigonometric polynomial and hence we cannot directly apply spectral factorization. Fortunately our transfer function is a univariate rational function, which lets us apply certain characterizations of nonnegative univariate polynomials over the whole axis (—00,00), semi-axis (0,oo), or some finite interval [a,5]; see section 2. Using these characterizations, we show how to solve the shape optimization problem for the following shapes: 1. 2. 3. 4.

standard bandpass filter design; arbitrary piecewise constant shape; arbitrary piecewise polynomial shape; general nonnegative function.

We will show that the first three shape optimization problems can be solved by testing the feasibility of certain convex sets, which are the intersections of certain hyperplanes and the cone of semidefinite matrices. This feasibility testing can be done efficiently using semidefinite programming (SDP) [VB96]. The fourth shape optimization problem can be solved by semi-infinite programming (SIP) [P0I97, WBV97]. We introduce some notation. For any m G N, denote by 5"* the vector space of m — by — m symmetric matrices, and let S^ be the intersection of S^ and the positive semidefinite matrices. A y B{A y B resp.) means that A — B is positive definite (semidefinite resp.). [r\ denotes the largest integer no greater than r. deg{p) is the degree of the polynomial p(-). Given a cone K C K^, y ^K 0 means that y G mtK, the interior of K. K* denotes the dual cone of K, i.e., /f* = {u e M^ : u^y > 0, Vy G K). The rest of this paper is organized as follows. In Section 2 we give a characterization of the cone of polynomials which are nonnegative on certain intervals. In Section 3, we reformulate the shape optimization problem for

Shape Optimization of Transfer Functions

315

transfer functions to be convex optimization, and also discuss related work. In Section 4 we show how to recover the transfer function from its absolute value. Section 5 draws conclusions.

2 Cone of nonnegative polynomials on intervals We characterize univariate polynomials which are nonnegative on certain intervals. For a survey paper see [PROO]. First, we characterize the nonnegative polynomials on the positive semiaxis [0, oo). The following result is due to Markov and Lukacs about one century ago. Theorem 1 (Markov, Lukacs [Lukl8, Mar48, PS76]). Let q{t) G be a real polynomial of degree n. Let rii = [^J and n2 = L^^J • //'?(*) ^ 0 for all f > 0, then q{t) = qiit^ + ^92(0^ where deg{qi) < n\ and deg{q2) < n2Now we apply this theorem to characterize the transfer function, which is similar to the spectral factorization for trigonometric polynomials. Observe that

l92(jt^)P

\q2,even{j(^) + q2,odd{joj)\'^

921(^2)2-I-w2g22(w2)2

___ Pljw) 2 = —-,—r where w = u P2[w) Here qi^even and qi^odd denotes the even and odd parts of the polynomial gj, and qij,i,j = 1,2 are defined accordingly. Note that pi{w) and P2{w) are nonnegative polynomials on w £ [0, oo). Conversely, by Theorem 1, given any such nonnegative pi{w) and P2{w), it is possible to reconstruct the qij{w), and so qi{ju)) and H{jio). In other words, p\{w) and P2{w) with deg{pi) < deg{p2) satisfy \H{ju)\'^ = Pi{w)/p2{w) where w = u'^ for some transfer function H{juj) if and only if they are nonnegative on [0, oo). The characterization of polynomials nonnegative on some finite interval [a, h] is analogous: Theorem 2 (Markov, Lukacs [Lukl8, Mar48, PS76]). Letq{t) € R[t] be a real polynomial. Suppose q{t) > 0 for all t G [o, b], then one of the following holds. 1. If deg[q) = n = 2m is even, then q{t) = gi(t)^ + (* •" o-){b — 092(0^ where deg{qi) < m and deg{q2) < m — 1. 2. If deg{q) = n = 2TO + 1 is odd, then q{t) = {t - a)qi{t)'^ + (6 - t)q2{tf where deg{q{) < m and deg{q2) < m.

316

Jiawang Nie and James W. Demmel

In our algorithms we will need to compute the polynomials qi{jio) from Pi{w), i.e. we need computationally effective versions of Theorems 1 and 2. These are given in section 4. To make the connection to semidefinite programming, we next characterize the polynomials nonnegative on an interval (either [0, oo) or [a,b]) by using certain convex cones. As introduced in [NesOO], define the vector of monomials v{t) = [1 t i^ • • • t""]"^ and the two convex cones of polynomials Ko,co = {P G K"+^ : P^v{t) > 0 V i > 0} Ka,b = {pe ]R"+i : p'^v{t) > 0 V ^ e [a, b]}. Let Hn,i G 5"+i be the i-th Hankel matrix, i.e., H^\=\^' ' lO,

if /e + / = i + 1, otherwise.

As introduced in [NesOO], define linear operators

by the following 2rn + l

^l{v) = Yl

2n2 + l

'^i^rii,i:

^2(-y) = Yl

i=l

^i+l^n2,i-

i=l

Another two operators A^, and A/^ are defined according to whether n is even or odd. When n = 2m, yl3 : ]R"+i-> S'™+\ Ai-.W+^^S"^ are defined as 2m+l M{v)

=

2m-1

Yl '"i^rn.i, i=l

Ai{v)

=

^ [(a + b)ViJ^i - Vi+2 " i=l

aK]^m-l,i-

When n = 2m + 1, yl3 :M»+i ^5-"^+!,

yl4 : ]R"+i-^ 5""+i

are defined as 2m+l

^3(w) = YZ [^»+l ~ "^il-^n^.i' i=l

2m + l

^4(^) = Yl, [^^» ~ •'^i+ll'f^m.ii=l

Let yli,yl2,yl3,^4 be their adjoint operators respectively, with respect to the inner product < A,B >= trace{A^B) for symmetric matrices of the same size. The following theorem is a compact characterization of cones ii'o.ooi -^o,b and their dual cones.

Shape Optimization of Transfer Functions

317

Theorem 3 (Nesterov [NesOO]). The cones Ko^(X), Ka^b can be chcLrcictevized as follows 1- -f^o.oo f^fid its dual KQ ^ are characterized as follows: i^o.oo = {P G R"+i : p = AUYi) + A*^iY2), Yi e 5!f>+\F2 G

Sl'^'},

^o.oo = {c G lR"+i : yli(c) h 0, A2{c) h 0}; 2. when n = 2m is even, Ka,b = {P G R"+i : p = AliYs) + yi:(F4),1^3 G 5!f'+\y4 G S!^}, Kl,

= {c G ]R"+i : yl3(c) ^ 0,yl4(c) >r 0};

when n — 2m + 1 is odd, K,,b = {P G R"+i : p = AUYs) + A^Y^), Y3 G 5!^+\y4 G 5!^+i}, i r ; ^ = {c G R"+i : .13(c) h 0 , ^ ( c ) ^ 0}; 3. Both Ko^aoiKa,b) and KQ^{K*I^) with non-empty interiors.

are convex, closed, and pointed cones

Now suppose we have L subintervals of [0,00): {[ai,bi]}i=,i- Let K = i^o.oo X Ka,,bi X • ••Ka^,bj^. Then its dual K* = K^^ x K*^f,^ x • • • K*^,,^. Given a matrix A of (L + l)(n + 1) rows and 2(n + 1) columns, consider the following problem: find a vector ( if it exists ) p e K^Cn+i) s.t. Ap G K. This can be done by solving a SDP feasibility problem by Theorem 3, say, using the SDP solver in [Stu99]. However it will introduce 2{L+ 1) symmetric matrices of size [n/2] or [n/2] + 1. In order to use interior-point methods to solve it, the complexity of one iteration will be at least 0(2(L + ^)n^) arithmetic operations. Fortunately, the dual cone K* (^ does not involve two symmetric matrices. A natural barrier function [NN94] for K* is given by L

F{c) = -lndetyli(co) -lndetyl2(co) - ^ ( l n d e t 4 ' ^ ( c i ) + Indetyl^'^(ci)), where 713(714) is the operator A3{A4) corresponding to Kai,bi in Theorem 3. Here the vector c = (CQ, • • • , cz,) G IR""*"-^ x • • • R""'"^ Now solve the following L+i times analytic center problem: min F{c) s.t. A'^c = 0,ceintK*.

(3) (4)

318

Jiawang Nie and James W. Demmel

The barrier function F{c) will tend to infinity as c approaches dK*. Hence the minimum will be attained in the interior of K*, which is not empty as guaranteed by Theorem 3. The optimality condition is that

VF{c)=AX,

ceintK*;

A'^c = 0. The optimal solution c* and its Lagrange multiplier A* can be found very efficiently using Newton's method. For any c G intK*, it can be shown [NN94] that VF(c) -
3 Shape optimization In this section, we will show how to design the transfer function of a LTI SISO system so that it has a desired Bode plot. Four kinds of shapes will be discussed: standard bandpass filter, piecewise constant, piecewise polynomial, and general shapes. 3.1 B a n d p a s s filter design The goal is to design a transfer function \H{ju)\'^ = ^ 4 ^ which is close to one on some squared frequency {w = w^) interval [u)^,u;''] and tiny in a neighborhood just outside this interval. The design rules can be formulated as Pi{w),P2{w) > 0, y w >0 P2{w) Pijw) <S, Piiw)

VwG[wlw'2]U[w{,wl2]

where the interval [wfjtOg] is to the left of \w^,w^], and [101,102] ^^ t° ^^'^ right. Here a and f3 are tiny tolerance parameters (say around .05). Let pi and p2 be the vectors of coefficients of pi{w) and P2{w) respectively. Then the constraints above can we restated as

Shape Optimization ofTransfer Functions

319

Pi,P2 e Xo.oo pi - (1 - a)p2 e K^e^^r

Using Theorem 3, we see that the above cone constraints can be expressed as Ap £ K where • In+i

0

0 In+l In+1 [a - l ) / „ + l A = -In+l (l+/3)/„+l —In+1 SIn+1 _—In+l ^In+1

P

and K ^ KQ^OO X ^O,OO X -f'^^^iu'- x -^-u^'.-iu x-f^w«,^ xii'^.^^r. Given (Q;,/3, 5), solve the analytic center problem (3)-(4) and then recover the coefficients p. As introduced in [GHNOO] for the discrete case, we can also consider the following objectives: • • •

minimize a + /9 for fixed S and n minimize S for fixed a, /3, and n minimize the degree n of pi and p2 for fixed a, /?, and 6.

These optimization problems with objectives are no longer convex, but quasiconvex. This means that we can use bisection to find the solution by solving a sequence of analytic center problems. A design example is given in figure 1. For the simplicity of programming, we used SeDuMi[Stu99] to solve the primal feasibility problem.

Fig. 1. The design filter shape for [w' w''] = [2 3], [w' w'2] = [0 1.8], [wl w^] = [3.2 5], a = /3 = 0.05, 5 = 0.05, n = 10.

320

Jiawang Nie and James W. Demmel

3.2 Piecewise constant shape design Here we extend the shape design technique from the last section to piecewise constant shapes. In other words we want the transfer function to be close to given constant values ci, ...,c„ in a set of m disjoint intervals ui"^ = w G \ak,bk], where ai < foi < a2 < 62 < • • • < Om < ^m- More precisely we want the transfer function to lie in the interval [{l—a)ck, (1 + /3)cfc] for w G [0^,6^]. By picking enough intervals (picking m large enough) we can approximate any continuous function as closely as we like. These constraints may be written Pii'w),P2{w) > 0, y w >0 pi{w) (1 - a)ck < < (1 + I3)ck, y w e [ak,bk], k = !,••• P2{w)

,m.

Using Theorem 3 as before, these constraints can be rewritten as the cone constraints Pliw),P2{w)

pi - (1 -a)ckP2,

e /tTo.oo

(l+/3)cfcP2 - P i e Ka^,bk,k = !,•

, m.

As before, find vector p such that Ap G K where 0 0 'n+1 /n+1 (a - l)ci/„+i (l+/?)ci/„+i -/„+i l)Cm-^n+l

(1 + f3)CmIn+l •X K? {, . Solve the analytic center problem and K ^ 0 , 0 0 ^ -^ai.bi ^ (3)-(4) again and recover the vector p. As in the preceding subsection, various design objectives can be considered by applying bisection. A design example for a step function with 3 steps is given in figure 2. 3.3 Piecewise polynomial shape design Here we extend the techniques of the last section to piecewise polynomials. Thus, on each interval [afc,&fc] we ask that Pi{w)/p2{w) be close to a given polynomial (j>k{w)-, in particular that it lie in an interval [(1 — a)<j)k{w)^ (1 + (5)(j)k{w)]. This leads to the constraints Pi{w),P2{w) > 0, V 10 > 0 {l-a)<j>k{w)k{w), P2\W)

Vu;G[afc,5fc],fc = l,--. ,m.

Shape Optimization of Transfer Functions

321

Fig. 2. The step function shape design for [ai bi] = [0 1.8], [02 62] = [2 3], [as 63] [3.2 5], ci = 1,C2 = 3,C3 = 2, a = /3 = 0.Q5, n = 10.

Using Theorem 3 once again, we transform these to the following cone constraints

Pi - (1 - a)4>k{w)p2{'w) e Kaf,,bk,k = !,••• ,m; (1 + (3)p2{w)(f)k{w) - p i G Ka^,bt,k = !,•••

,m.

which are again a set of linear equations and LMI's. As before, pi and P2 can be obtained by solving some appropriate analytic center problem (3)-(4), and bisection can be used to achieve certain design goals. 3.4 General shape design So far we have considered bandpass filter design, piecewise constant shape design, and piecewise polynomial shape design. Here we discuss general shape design. The goal is to design a transfer function \H{ju>)\'^ = ^^\^l so that it behaves like some general nonnegative function f{w) for w G [a, b] where 0 < a < 6. In other words we want: Piiw),p2{w) >0,yw& [0,00) Pijw) (l-a)/H< <{l+p)f{w), \/w€[a,b]. P2{w)

(5) (6)

Now we can not apply Theorem 3 directly, and must instead apply approximation methods. One obvious approach is to partition [a,b] into subintervals {[afc,&fc]}fcLi and approximate f{w) by a constant or more general polynomial in each subinterval. Then we can apply the method from the preceding sections. Another approach is to apply semi-infinite programming(SIP), as described in [WBV97]. The idea is to choose N sample points

322

Jiawang Nie and James W. Demmel a < Wi < W2 < • • • < wj^ < b

and replace the semi-infinite inequality constraints (5)-(6) by N simple inequality constraints. A standard rule of thumb is to choose A'' = 15n in practice [WBV97]. Then the approximate optimization problem is solved iteratively [Pol97]. 3.5 Related work There are several related papers [AV02, GHNOO, WBV97] on various filter design techniques. Most of them are for discrete systems, and some techniques can be applied to continuous systems with some modifications. Note that the mapping s = j ^ maps the pure imaginary axis onto the unit circle except (—1,0). Then our transfer function H{s) becomes a rational trigonometric function R{z). Each interval j[ai,bi] is mapped onto some arc {e^^ : u G [w/, wf ]} on the unit circle. The methods described in [AV02, GHNOO, WBV97] all can be applied. However, all of them will involve the constraints of the form that some trigonometric polynomial is nonnegative on some interval. The characterization of (trigonometric) polynomials nonnegative on some intervals will eventually need Theorem 3 or its equivalent form to transform to a LMI. In this paper, we transform our design problems using constraints of real polynomial nonnegativity on some intervals in the positive semi-axis. Then we may apply Theorem 3 directly to characterize these constraints using LMIs. As described at the end of Section 2, we solve an appropriate analytic center problem, instead of solving these LMIs directly. The structure of this problem can be exploited to use Newton's method to efficiently find the analytic center. There are also several good papers [Fab02, NesOO, GHY03] on polynomials on the real axis, unit circle, pure imaginary axis, and other curves. [NesOO] is the classical paper that characterizes polynomial nonnegativity constraints by LMIs; our paper is based mostly on it. In [GHY03] the authors characterized the cone of positive pseudopolynomial matrices and discussed optimization over this cone. The authors also discussed the conditioning of such optimizations, and proposed using the basis of Chebyshev polynomials to improve conditioning. [Fab02] gives an abstract version of [NesOO, GHY03], characterizing polynomials which are nonnegative on the disjoint union of several intervals.

4 Recovery of t h e transfer function In this section, we show how to use Theorems 1 and 2 effectively. First, given polynomials pi{w) and P2{w) {w = u"^) such that ^ desired shape, we need to find real polynomials qi and q2 so that Pijw) P2{w)

2

92 (jw)

Shape Optimization of Transfer Functions

323

To this end, given a polynomial p{w) that is nonnegative on [0, oo), we will provide an algorithm to find two polynomials qe{w) and qo{w) such that p{w) = q1{w) + w • Qoiw). Then qe contains the even coefficients and qo the odd coefficients (modulo signs) of the desired polynomial q as described in Section 2. Lemma 1. The following polynomial identities hold: 1. iff + wgJXfl + wgl) = {hh + wgxg2? +w{hg2 - /25i)^' 2. {w - r)2 + b^^{wv 9 2 T F ) 2 + w • 2{Vr^+lP - r). Proof. Verify directly. D Lemma 2. If a polynomial p{'w) is nonnegative on [0, oo), then its factorization must have the form

(

111

\

n2

J

i=l

nC-^ + Ci) i=l

Y[{.{w-rif

"3

+ b^)l[{w

+ ai)

i=l

where a >0, bi > 0, n\ + 2n2 + na = n, 0 < ai < • • • < a„3, Ci < 0. Proof. Write the factorization p(tf) = a YYk=i{'^~'Pk)• First consider the constant term a; it must clearly satisfy a > 0 for piw) to be nonnegative over [0, oo). Next consider the three classes of roots pk- real positive, complex, and real nonpositive. The positive roots must all have even multiplicity for p{w) to be nonnegative, so we can write the product of all their factors w—pk as 0^=1 (^"1" Cj)^ where Cj < 0. Next, the complex roots come in complex conjugate pairs Pk = Tk + j • bk and p^ = rk - j • bk, so we can write {w - Pk)iw - pk) = {w — rkY + b\ for all n2 complex conjugate pairs. Finally consider the na nonpositive real roots 0 > —ai > • • • > ~a„3 of p{w). Their corresponding factors u; — (—Oj) = u; + ttj are all nonnegative on [0, oo). D

Using the above two lemmas, we get the following algorithm. Algorithm 4.1 This algorithm will find qe{w) and qo{w) such that p{w) = q1{w) + « ; • q1{w) if p{w) is nonnegative on [0, oo). Let ge = l>9o = 0. Step 1 Find the factorization of where hi > 0, ni + 2n2 + ns = n, 0 < ai < • • • < a„3,0 > Cj £ K. Step 2 Find the {-f + w{-f form ofXXZ^iw + a^) for fc = 1 : n3 qe = y/alqe

+ W • qo

qo = \fa'iqo - qe qe •=qe,qo -^^qoend

324

Jiawang Nie and James W. Demmel

Step 3 Find the {-f + w{-f form ofY^^l^{w'^ + 6f) for k = 1 : n2 qe = qejw - ^rf + b'i) + w • g „ ^ 2 ( ^ K ' + 6f - r^) 9o = qeyj2{^rt^+yf^)qo{w - ^rf + b'j) Qe ••=qe,qo ••^'qoend Step 4 qe := aqe ]Xi=ii'^ + ^i), 9o := aQo n " = i ( ^ + ^i). Now apply algorithm 4.1 topi{w){i = 1,2) respectively, i.e., Und qij{w){i,j = 1, 2) such that Pi{w) = qfiiw) + w • ql2{'^) fo^ i = 1, 2. Then we obtain the desired transfer function H{s) 92,l(-'S^) + S 9 2 , 2 ( - S ^ ) '

Remark: If a polynomial p{w) is nonnegative on a finite interval [—1,1] (another finite interval can be changed to this one by a linear transformation), then we can also apply the above algorithm to find two polynomials pi and P2 such that p{w) = Pi{w) + (1 — w){w + l)p'2{w). ActuaUy, we only need to do the Goursat transform (see [PROO]) for piw), i.e.,

p(«;) = (^ + l ) " p ( i ^ ) , and then apply the above algorithm to find qe,qo such that

p{w)

=ql{w)+wqliw),

and then apply the inverse Goursat transform to get back p{w): p{w) = 2-"(w + l)'*''ff(P)p(i-^^).

5 Conclusions and discussion This paper discusses shape optimization for a transfer function for a LTI SISO system by formulating it via semidefinite programming. Given the shape (absolute value) of the transfer function, we show how to extract the transfer function itself. Since the optimization process uses semidefinite programming, it may be done efficiently. We do not consider any constraints on the components A, b, c, d of the LTI system. However in practice, these components may not be arbitrary, but instead have special structure and depend on certain design parameters. Thus an interesting question is finding those parameters to optimize the shape of the transfer function as we did in Section 3. This is in general not a convex

Shape Optimization of Transfer Functions

325

problem, and can be very hard to solve. But it is still a feasibility/optimization problem about polynomials, if {A, b, c, d) are polynomials in those parameters. Therefore, we may formulate t h e m using polynomial optimization, and then solve t h e m by techniques such as the sum of squares and positivstellensatz (see [ParOl]). B u t this is more difficult, and future work. A c k n o w l e d g e m e n t s : The authors would like to t h a n k Prof. El Ghaoui and the Referee for many useful comments and suggestions.

References [AV02] B. Alkire and L. Vandenberghe, Convex optimization problems involving finite autocorrelation sequences. Mathematical Programming Series A 93 (2002), 331-359. [CD91] Prank M. Callier, Charles A. Desoer, Linear System Theory, SpringerVerlag, New York, 1991. [Fab02] L. Faybusovich, On Nesterov's approach to semi-infinite programming. Acta Applicandae Mathematicae 74 (2002), 195-215. [GHNOO] Y. Genin, Y. Hachez, Yu. Nesterov, P. Van Dooren, " Convex Optimization over Positive Polynomials and filter design", Proceedings UKA CC Int. Conf. Control 2000, page SS41, 2000. [GHY03] Y. Genin, Y. Hachez, Yu. Nesterov, P. Van Dooren, Optimization problems over positive pseudopolynomial matrices, SIAM Journal on Matrix Analysis and Applications 25 (2003), 57-79. [KS95] T. Kailath and A.H. Sayed, "Displacement Structure: theory and applications", SIAM Rev. 37(1995), 297-386. [LuklS] Lukacs, "Verscharfung der ersten Mittelwersatzes der Integralrechnung fur rationale Polynome", Math. Zeitschrift, 2, 229-305, 1918. [Mar48] A.A. Markov, "Lecture notes on functions with the least deviation from zero", 1906. Reprinted in Markov A.A. Selected Papers (ed. N. Achiezer), GosTechlzdat, 244-291, 1948, Moscow(in Russian). [NN94] Yu. Nesterov and A. Nemirovsky, "interior-point polynomial algorithms in convex programming", SIAM Studies in Applied Mathematics, vol. 13, Society of Industrial and Applied Mathematics(SIAM), Philadelphia, PA, 1994. [NesOO] Yu. Nesterov, "Squared functional systems and optimization problems". High Performance Optimization(H.Frenk et al., eds), Kluwer Academic Publishers, 2000, pp.405-440. [Pol97] E. Polak, "Optimization: Algorithms and Consistent Approximations". Applied Mathematical Sciences, Vol. 124, Springer, New York, 1997. [PS76] G. Polya and G. Szego, Problems and Theorems in Analysis II, SpringerVerlag, New York, 1976 [PROO] V. Powers and B. Reznick, "Polynomials That are Positive on an Interval", Transactions of the American Mathematical Society, vol. 352, No. 10, pp. 4677-4692, 2000. [WBV97] S.-P. Wu, S. Boyd, and L. Vandenberghe, "FIR filter design via spectral factorization and convex optimization". Applied and Computational Control, Signals and Circuits, B. Datta, ed., Birkhauser, 1997, ch.2, pp.51-81.

326

Jiawang Nie and James W. Demmel

[ParOl] P.A. Parrilo. Semidefinite Programming relaxations for semialgebraic problems. Math. Prog., No. 2, Ser. B, 293-320, 96 (2003). [Stu99] J.F. Sturm, "SeDuMi 1.02, a MATLAB toolbox for optimization over symmetric cones". Optimization Methods and Software, ll&£:12(1999)625-653. [VB96] L. Vandenberghe and S. Boyd, "Semidefinite Programming", SIAM Review, 38(l):49-95, 1996.

Achieving Wide Field of View Using Double-Mirror Catadioptric Sensors Ronald Perline^ and Emek Kose^ ^ Department of Mathematics, Drexel University, Philadelphia, PA 19103, USA. rperlineOmcs.drexel.edu ^ Department of Mathematics, Drexel University, Philadelphia, PA 19103, USA. ek58®drexel.edu

Summary. For many applications such as surveillance, medical imaging, photography and robot navigation, it is required that the camera have a wide field of view. Traditional approaches to solve this problem include using a rotating camera, stitching images, complex lenses or multiple cameras. We are proposing a catadioptric sensor with a camera-mirror pair for enhancing the field of view. Devices consisting of a reflective surface (catoptrics) and a camera (dioptrics) are called catadioptric sensors. Our single viewpoint double-mirror system formed of a conical mirror coupled with a proper secondary mirror arises as a solution to a nonlinear first order ordinary differential equation. The ordinary diS'erential equations are obtained as geometric solutions to the problem. Our system is designed to image a plane at infinity, without distortion, requiring no digital unwarping. In this work, we are analyzing the family of cones and corresponding secondary mirrors to obtain a correct image and a large field of view.

1 Introduction Using mirrors combined with lenses is an effective solution to the general problem of enhancing the field of view, which is necessary for various apphcations, such as surveillance, medical imaging, photography and robot navigation. In robotics, especially in autonomous mobile robots, omnidirectional sensors are the best for navigation, and in dynamic environments, catadioptric sensors work the best. T h e images produced by the sensor can easily be transformed by various software. Presenting such advantages, interest in catadioptric sensors has increased recently. Catadioptric sensors are oriented such t h a t the camera is fixed, with the rotationally symmetric mirror suspended above it. A camera, the dioptric component, realizes a mapping between the 3D world and 2D image plane. We employed perspective and orthographic camera models in our designs. In perspective projection, the world points are projected onto the image plane with perspective rays originating at the center of

328

Ronald Perline and Emek Kose

projection (COP), which would lie in physical camera. The camera reahzing perspective projection is often referred to as pinhole camera. Orthographic projection can be viewed as a special case of perspective projection, it has a large focal distance, in other words, the center of projection is at infinity. In cameras realizing orthographic projection, objects close to the optical axis are imaged, and parallel hnes are mapped to parallel lines. So the difference of orthographic projection from perspective projection is that light rays travel parallel to optical axis, as opposed to passing through the center of projection. Traditionally, imaging systems have been designed to maintain a single effective viewpoint. In other words, all the rays of light entering the camera intersect at a single point, called the effective pinhole, so single effective viewpoint acts as a virtual center of projection. By placing the center of perspective camera lens at the outer focus of the hyperbohc mirror, the inner focus then becomes the single effective viewpoint. Single viewpoint allows the unwarping of accurate perspective images. The single viewpoint has been studied and recommended by Baker and Nayar [BN98]. Our system is designed to image a plane at infinity, compared to the diameter of the system, without distortion, requiring no digital unwarping. The catoptric component of the system is a double-mirror consisting of a primary mirror and its dual, as we call them, and both mirrors employed are rotationally symmetric. This allows us to picture the problem in 2D. The primary mirror has been chosen a concave up cone of different slopes under perspective and orthographic projections. The corresponding secondary mirror, which is also called its "dual" was designed according to the transformation reahzed by the primary mirror. For the special cone whose slope is 1, we can easily obtain the analytical solution to the ordinary differential equation for its dual whereas for cones with slopes different than 1, we have to solve the resulting ordinary differential equation by numerical methods. The scheme we used in designing our catadioptric sensors, as suggested by Hicks and Perhne [HPOl] consists of: 1) Differential formulation: Primary mirror is fixed, the desired design characteristics of the dual mirror are restated in terms of a set of ordinary differential equations. 2) RealizabiUty: A systematic check is performed to determine whether it is theoretically possible to construct a catoptric which exactly realizes the specified design characteristics. 3) Optimization: If the mirrors are not realizable, then we perform optimization methods, to find the 'best' possible catoptric component.

2 Related Work Here, we review past work on catadioptric sensors, with focus on design rather than applications. There has been an increasing interest in investigating the design and appfications of catadioptric sensors. Much of this work focuses on

Achieving Wide Field of View

329

designing panoramic or wide field of view sensors (see [BN98, Hic99, Nay97, Ree70, YK90, CM99]). An early use of employing mirrors for panoramic imaging is a patent by Rees [Ree70], who proposes the use of a hyperbolic mirror for television viewing. In [HPOl], Hicks and Perline describe a general method for catadioptric sensor design for a prescribed projections, using geometric distributions in 3-dimensional space, generalizing vector fields. Nayar presents a new single viewpoint, double-mirror, double-camera catadioptric design which is truly an omnidirectional sensor. His design consists of two parabolic mirrors, paired with two orthographic camera [Nay97]. In 1996, Nalwa [Nal96] proposed a panoramic sensor that consists of 4 planar mirrors coupled with 4 imaging systems. The system possesses single viewpoint and it has a large field of view, approximately 360° x 50°. In [HBOO], Hicks and Bajcsy obtain a mirror shape that is wide-angle but approximates perspective views. This design, which is not single effective viewpoint, distorts the view plane minimally. The polar sensor derived by Hicks, Perline and CoUetta [HPCOl] is very close to our Right Angle Mirror design, as we call it. They suggest a sensor design under cylindrical projection, such that the sensor images the world linearly.

3 Conical Mirrors 3.1 Right Angle Mirror In any of our designs, we start with a choice of primary mirror and the projection. Depicting the problem for the chosen parameters, we derive ordinary differential equations that describe the secondary mirror. Solving the ordinary differential equations obtained, either analytically, which is only valid for Right Angle Mirror under Orthographic Projection, or numerically, gives us the graph of the cross section of the secondary mirror. 3.1.1 Right Angle Mirror U n d e r O r t h o g r a p h i c P r o j e c t i o n This is the most basic design in the family of cones. The conical primary mirror we chose has slope equal to 1 and our camera realizes orthograpliic projection. The profile curve of the system can be seen in Figure 1. Assumptions made are: the view plane is infinitely far from the camera, compared to the diameter of the system and is in +y direction. Our camera is located on —y axis, which at the same time is the optical axis of the entire system. So, to set the problem from an inverse direction, all light rays emanating from the image plane travel parallel to each other to shoot the primary mirror,reflecting off parallel to a-axis, they intersect the dual mirror and they reflect off to shoot a point in the view plane, which is y = K. a is a. scaling

330

Ronald Perline and Emek Kose

-0.

Incoming L i g h t Ray

Fig. 1. Cross-section of the Right Angle Mirror under Orthographic Projection. constant that lets us adjust the amount of scaling of the view plane onto the image plane. The primary mirror, which is a cone has equation y=x. Let us define the vectors u and v; u = < - l , a ; i > ; v ==< Kaxi,K

> - < x',f{x')

>

(1)

Normalizing u and v, we get: u=<

-1 l+xi'l

Xi

X

,

Xi

+ xj

Since the view plane y = K is infinitely far, the limit X

lim

O/i

< axi - -r;,l —77 > = < axi, 1 >

K—>oo

K

(2)

K

The vector sum nd = u + v is parallel to the normal vector of the secondary mirror. By making use of the relation between the normal to the surface and the tangent to the surface, we can say that slope of the tangent line to the surface at the point x = x' equals the negative reciprocal of the slope of the normal to the surface at the same point.

That is, - S i = fix') We then have: f'{x)

= 2x+\/4x2 + l

(3)

Integrating both sides, we obtain the analytical solution to our problem:

Achieving Wide Field of View 1

331

1

fix) = x^ + -xV4x2 + 1 + - sinh-^(2x)

(4)

The graph of /(x) is the cross section for our secondary mirror. In Figure 1 you can see how this system images the checker board, far above. We use checkerboards for our simulations, because we are interested in how the system distorts lines. 3.1.2 Right Angle Mirror Under Perspective Projection Our primary mirror is the same cone with slope equal to 1 and we are considering a pinhole camera. The assumptions made are; view plane is at an infinite distance, the optical axis is the y-axis and the center of projection of the camera is at the point ( 0 , - 1 ) . All hght rays that pass through the pinhole are going to reflect off the conic mirror and then hit the secondary mirror, yet unknown. Reflecting off the secondary mirror, they shoot the plane y = Kxi is the intersection point of the incoming ray with the x-axis. We approach this problem with the vector-method discussed above. Let us define the vectors, u , v and w; u = < xi, 1 >, V = < —1, —xi >, w = < axi, 1 > . Let us also define m^ = slope of the incoming light ray, mr= slope of the reflected light ray, di= the angle that incoming light ray makes with positive X-axis, 9r=the angle that reflected light ray makes with positive x-axis and 9m= angle between positive x-axis and the mirror. It is obvious from the Figure 2 that rrii = ^ . Prom optics, we know that 0j. + Oi = 26^ , since we have 0m = f i this imphes; 61^ = I - 6li =» tan(6l^) = tan(f - Oi) =^ rtir = tan(f - di) = cot(6ii). Hence, this gives us ; m^ = xi and m, = —. The point of intersection of the conic mirror with the incoming hght ray can be found simply by simultaneously solving both equations. This point of intersection is: ( j ^ ^ , i r ^ ) Any point (x2,y2) on the secondary mirror can be written as: {X2,y2) =

{--,——,z——)+tv. 1 - xi 1 - xi This implies xi = —^TT . Also we know that irTr + jrhi is in the normal ^

^

X2 + ^

||V||

||W||

direction of the secondary mirror. After necessary substitutions and simplifications we obtain a nonlinear implicit ordinary differential equation which is not possible to solve analytically. However, numerical solution gives us the cross section of the dual mirror, seen in the Figure 2. 3.2 30-Degree Conical Primary Mirror One of most efficient systems we obtained is the 30-degree conic primary mirror system. Efficiency, in the sense that, the system can view a large portion

332

Ronald Perline and Emek Kose

Fig. 2. Cross-section of Riglit Angle Mirror under Perspective Projection. of the scene without having a large diameter. Keeping the diameter of the entire system small is important. We could sacrifice having a longer system for a narrow one. The reason for this is the future applications may include medical imaging. The primary mirror is a cone with slope equal to t a n ( | ) . The camera is orthographic.

0.5

Fig. 3. Cross-section of a 30-degree Primary Mirror under Orthographic Projection.

Achieving Wide Field of View

333

Fig. 4. POV-Ray simulation of the view from 30-degree system in the test room.

We are going t o employ the vector method t o derive the secondary mirror. The primary mirror has equation: y = tan(;)x. The vectors v and w after normalization are then;

So the vector addition of v and w is parallel to normal vector of the dual mirror at (x, f (x)). If n d = v w then the slope of n d equals the negative reciprocal of the slope of the tangent line of the secondary mirror at the same point. That is:

+

ndl nd2

--

=

f I(.)

Expressing tan(!) geometrically, using the figure helps us engage f (x) in the slope equation. That is

Solution of equation (8) for

gives us this relation;

334

Ronald Perline and Emek Kose

By equations (7) and (9) we derive the nonlinear, implicit ordinary differential equation for secondary mirror corresponding to 30-degree conic primary mirror. The cross section of this mirror can be seen in Figure 3. The simulation of this system after it has been revolved about the y-axis is as in Figure 5. The 30-degree conical mirror system inside the test room with checkerboard walls views close t o 180°, Figure 4.

Fig. 5. Sideview of a 30-degree Primary Mirror under Orthographic Projection.

4 Conclusion In this paper, we have exhibited a catadioptric sensor design which enables a normal camera an ultra wide field of view with minimal distortion. These sensors are based on a family of double-mirrors with conical primary mirrors, derived as numerical or analytical solutions of non-linear ordinary differential equations which describe how a plane perpendicular to the optical axis of the system is imaged on the film. The images obtained require no further processing. In the future work, we would like t o investigate the general conics as primary mirrors, such as parabola, ellipse and hyperbola. We also want to implement an actual system and experiment on it, perform full error analysis.

References [BN98] Baker, S, and Nayar, S.K. (1998), "A Theory of Catadioptric Image Formation," Proc. International Conference on Computer Vzsion, pp. 35-42.

Achieving Wide Field of View

335

[CM99J Conroy, T. and Moore, J. (1999), "Resolution Invariant Surfaces for Panoramic Vision Systems," Proc. International Conference on Computer Vision, 392-397. [Hic99] Hicks, R.A. (1999), "Reflective Surfaces as Catadioptric sensors ," Proceedings of the 2nd Workshop on Perception ofr Mobile Agents, CVPR 99 , 82-86. [HBOO] Hicks, R.A. and Bajcsy, R. (2000), "Catadioptric Sensors that Approximate Wide-Angle Perspective Projections," Proc. Computer Vision and Pattern Recognition, 545-551. [HPOl] Hicks, R.A. and Perline, R. (2001), "Geometric Distributions for Catadioptric Sensor Design," Proc. Computer Vision and Pattern Recognition 2001. [HPCOl] Hicks, R.A., Perline, R. and Coletta, M.L. (2001), Catadioptric Sensors For Panoramic Viewing, World Scientific. [Nal96] Nalwa, V. (1996), "A True Omnidirectional Viewer," Technical Report, Bell Laboratories, Homdel, NJ 07733, USA. [Nay97] Nayar, S.K. (1997), "Catadioptric Omnidirectional Camera," Proc. Computer Vision and Pattern Recognition, 482-488. [Ree70] Rees, D. (1970), "Panoramic Television Viewing System," United States Patent, (3,505,465) [YK90] Yagi, Y. and Kawato, S. (1990), "Panoramic Scene Analysis with Conic Projection ," Proceedings of International Conference on Robots and Systems , 82-86.

Darcy Flow, Multigrid, and Upscaling James M. R a t h Institute for Computational Engineering and Sciences, University of Texas at Austin, TX 78712, USA. organismQices.utexas.edu

Summary. Simulation of flow through a heterogeneous porous medium with finescale features can be computationally expensive if the flow is fully resolved. Multigrid algorithms are often used to compute approximations to such problems because of storage considerations, the high condition number of the problem, and multigrid's resolution independent convergence rate. The proposed algorithm takes an ordinary multigrid method and replaces the coarsening procedure with an upscaling one. Upscaling aims for performing computations on a coarser scale but still retaining information about the fine-scale flow and problem data. That is, an upscaled computation is just as cheap as a coarse one, but much more accurate. By replacing coarsening with upscaling in multigrid, faster convergence ensues. Numerous upscaling techniques have been proposed; variational multiscale subgrid upscaling is used here. By keeping within the usual finite element variational framework, known multigrid analysis techniques can be applied. Like multigrid, the proposed scheme has a convergence factor dependent on both the permeability field (the ratio of maximum to minimum values) and the relative resolution of the fine and coarse grids.

1 Introduction Darcy's law describes fluid flow through a porous medium. It is an empirical law t h a t asserts bulk flow of a fluid through the medium is proportional to the gradient of the pressure across the medium (accounting for hydrostatic differences from gravity) [Dar56, Bea72, Sch74, FC79]: u = — (Vp-pg) M Darcy's law has found wide applicability in modeling subsurface flows, and has been generalized to model multicomponent and multiphase flows. (The above differential form is itself a generalization of the relation Darcy formulated.) Our primary interest is in using Darcy's law to model oil reservoir and groundwater contaminant flow.

338

James M. Rath

Darcy's law alone is insufficient to describe the physics: conservation of mass (the continuity equation) and equations of state (relating density, viscosity, and permeability to phase fraction and temperature) are necessary. In the appUcations being considered, there is often a need for the velocity to be very accurate and to strictly (locally) observe mass conservation; hence our focus on mixed methods [RW83, DEW83, DEW84, ERW84]. Although our presentation ignores aspects of multiphase flow, the proposed ideas and software can readily be adapted to model such flows. 1.1 Heterogeneity and why it is a problem Geostatistical modeling is used to generate the necessary data (porosity and permeability) to specify the problem to be approximated [DJ97]. This data is typically given at a very fine resolution [Dur02], but the goal is to predict long-range flow behavior; it is tempting, then, to approximate the problem at a very coarse scale. However, fine-scale features of the problem data can have very large effects on the coarse-scale flow behavior [Dur02]. Therein lies one big difficulty: it is necessary to resolve the flow at very fine scales resulting in computationally poor conditioned problems to solve [G+85]. Moreover, the resolution cannot be reduced to shrink the size of the system: (1) heterogeneity in the permeabihty (irregular, short spatial-scale jumps) means p-refinements (high-order approximations) will not help, and (2) spatially-limited resolution and spatially-uniform heterogeneity means /i-refinements (coarse scaling) will not help either. The fine-scale resolution necessary in simulations makes for poor conditioning, yet there is still another difficulty: the jumps in the permeability can sometimes be quite severe (several orders of magnitude changes) between nearby locations (see for example Figure 4). This makes our computation of an approximation even more poorly conditioned. The more heterogeneous and fine-scale the problem, the more computationally expensive it becomes as all known direct/iterative linear solvers have behavior which worsens with greater condition number [GV96]. We seek to broaden the range of problems that are computationally feasible. 1.2 Proposed solution technique One approach used to overcome the poor-conditioning in modeling multiscale phenomena is upscaling [Dur02]. In upscaling techniques, an averaging process is used to determine the influence of fine-scales on the simulation and to adjust the coarse-scale computations accordingly. Typically an upscaled model will more accurately predict the coarse flow, and will also correctly display flne-scale features of the flow. With mixed variational multiscale subgrid upscahng [Arb, HFMQ98], the scales are spht into subgrid and coarse parts before approximating, thereby keeping all the fine-scale information in the model. The mixed framework keeps strict conservation of mass.

Darcy Flow, Multigrid, and Upscaling

339

Any upscaling technique must ignore some fine-scale information, though. As noted above, fine-scale features of the problem data can have very large effects on the coarse-scale flow behavior — sometimes a full resolution of the problem is inescapable to correctly predict flow features like production rates or breakthrough times. Not all the benefit of upscaling need be lost: we can use upscaling to aid the computation of the fine-scale flow. Since one goal of upscahng is to obtain a better coarse-scale approximation of the flow, it is a natural idea to try substituting the upscaled approximation for an ordinary coarse-scale approximation in a multigrid scheme. The upscaHng substitution is also natural in that it is about as computationally expensive as the ordinary coarsening. This substitution forms the basis for the proposed algorithm. Although we cannot hope to reduce the computational complexity of multigrid, we can dramatically reduce the time necessary to accurately model a fine-scale flow. As repeated single-phase flow simulations are at the heart of multi-phase simulations, geostatistical sampling, and flow optimization studies, this is a significant achievement.

2 Mixed Variational Multiscale Subgrid Upscaling Because of the complexity of the proposed algorithm, it is useful to first describe the underlying upscaling method and introduce some terminology and notation. Further details of this technique can be found in [Arb, Arb02, AB02]. Use /? to denote the spatial extent of the porous medium and assume that it is a connected, convex, polygonal domain in E" (where n is 1, 2, or 3). Let p and u denote the unknown pressure and macroscopic velocity. Our model problem in the interior of the domain consists of Darcy's Law u = --(Vp-pg), M

(1)

V • u = /,

(2)

and the continuity equation where K, /i, p, and g are the (spatially-varying) permeability, viscosity, density, and gravity, and / is a source term. On one piece of the boundary rV, specify the flow u-v = gN, (3) and on another, r b , specify the pressure P = 9D-

(4)

The pieces of the boundary add up to the whole dQ = PIM U PD , the pieces are disjoint F^ C] To = 0, and v is the unit outer normal vector to dSl. A cell-centered finite difference method (CCFD) is used to approximate fiow equations for the pressure.

340

James M. Rath

The proposed methods, however, apply to the problem (l)-(4) in mixed variational form. Let V = {v G iJ(div, /?) I V • 1/ = 0 on FN} , W = L'^{Q), and Vg„ be an element of if (div, Q) such that Vg^ • v = g^ on Fff and V(,„ • v = 0 on FD. (An important special case occurs if F^ = dQ. In that case, use W = L^/K instead; a compatibility condition on / and g^ is also required.) The problem is then to find u G V and p &W such that (V • n,w) = {b,w) (ku, v) - (p, V • v) = (c, v) -(£(£,,. VV' i •^ )iy)ro rD

Ww G W, Vv G V.

(5) (6)

The substitutions k = f^K.~^, b = f — V rg^, and c == pg — kvg„ have been made to simphfy the notation. Note that the velocity u = u + Vg„ is the solution to the original problem (l)-(4). In order for either formulation of the problem to have a solution, the data needs to meet some regularity constraints. In the variational formulation, one of interest in this proposal is that k must be a symmetric rank-two tensor in (L°°(i7))"^" (denote its essential supremum by k*), and must be uniformly elliptic (denote its essential infimum by fc*). The ratio of these two values fc*/fc* will have an impact on the performance of the proposed preconditioners. 2.1 A mixed finite element approximation For a fine-scale problem, assume the permeability data k is given as piecewise constant on a grid with spacing h, and that measurement uncertainties prevent specifying it at any finer resolution. It then makes sense to find solutions at the same resolution, and no finer. That is, find u/j G Vh C V and ph G Wh C W such that (V • Uh, Wh) = {b, Wh) (ku,i,v^)-(p/j,V-v/i) = (c,v/i)-(to,v/,-J/)rD

Wwh G Wh, Vv/j G V^,

(7) (8)

where Wh^Vh are Raviart-Thomas-Nedelec elements of order zero (RTO) [RT77, BF91, RT91] on the given grid. See Figure 1 for an illustration of the degrees of freedom of these elements. Using quadrature — the trapezoidal rule — to compute the term (ku/j, Yh) is equivalent to solving the original problem (5)-(6) using cell-centered finite differences [RW83, AWY97]. If a coarse-scale approximation is desired instead, then find u/f G VH C V and PH G WH C W such that {S7 •UH,WH)

= {b,WH)

(ku/f,v/f) - (p//,V-Vi^) = (c,Vfl') - (firo,v/f • j^jro

WwHeWn, Vv//G Vf/.

(9) (10)

Darcy Flow, Multigrid, and Upscaling —**—

^ »

: •

,

'

.i

.

•

:

•»

:

: * '

•

;

#:

•

•

•ii

•:

'

*

ill

;;

'!!!(•

,

r

<$ :

•

a- , ;

* ,

;

•

9

:

• ::

:•

:; »

:^

• ,:

: «:

: •"

: » .

:'i

•

• ;:

• : • ; : « ^; . :

•It

:

1!

.

:

.ill

:

:

•

II

;

;

# ::

• )

:

» : : • ;:

•

<•

::

f

;:

—fi—

III

:

,

.; • : :

s -

»

;

5

:

:

•

;

.

* ;

:

« ,

; . :

^

*

• :: .:

• :

•

. :
• : • ;

'::

• :: • ; : • ::

• : • :.

: • : : • ;: • ;: • :

m :: :

:

:

.

•

:;
.•

: •^ -

)

;

:,

« :: • ^ ,

m ^ ::

: • :

'

••

:

'-

^ . : ffl

;

$i

;

; « :

:

"

:

:

: ^

341

„: —

« :; »

•

• ,'

e .:

• ;:

« :• ,. ?

«

ti'

:i

®

:

11

;

:'. @ ;

• -

—

I

—

•

Fig. 1. Degrees of freedom for 2-D RTO elements on a fine rectangular grid: normal velocity degrees of freedom, and • pressure degrees of freedom.

•

•

*

'!•

•

a

•

•

^1^

Fig. 2. Degrees of freedom for 2-D BDMl [BDM85] elements on a coarse rectangular grid: :'. (linear) normal velocity degrees of freedom, and « pressure degrees of freedom.

342

James M. Rath

where WH X V^ are RTO elements or, perhaps, Brezzi-Douglas-Duran-Fortin order one (BDDFl) [BDDF87] elements) on a grid with spacing H, an integer multiple of h. See Figure 2 for an illustration. For the RTO elements, some error estimates [BF91, DR85, RT91] for a grid size h are ||u-u„||o
^0{h),

(11)

\\P - Phh < C\\p\\2h

=0{h).

(12)

For BDMl/BDDFl elements the velocity is an order of h more accurate ||u-u,,||o
=0{h').

(13)

2.2 The upscaling technique The subgrid upscaling technique uses the mixed finite element method (MFEM) with a variational multiscale [HFMQ98] technique. Before any approximation, decompose V and W (using a chosen coarse grid) so that (i) no information is lost, V = Vc ® SV, W = Wc® 5W; (ii) mass is conserved, V • Vg = Wc, V • 5Y = SW; and (iii) the fine-scale velocities are locally supported over the coarse grid, 6Y • i^ = 0 on the boundary of each coarse cell. (Section 3.1 — in particular Theorem 3.3 — of [Arb] show that this decomposition is always possible and satisfies some additional necessary properties. There is also more than one way to choose the decomposition.) Note that Condition (iii) above allows us to disconnect the subgrid problems from one another, but not from the coarse grid problem. A Green's or influence function approach is employed to effect a forward elimination of the subgrid problems into the coarse problem, and — once the coarse problem has been solved — a back substitution to recover the subgrid solutions. The two can then be recombined to obtain a fine scale solution. Since V and W have been decomposed into direct sums, the problem (5)~ (6) can be described as follows. Find Ug e Vc and pc G Wc, and Su G SV and Sp € 6W such that on the coarse scale {V •{uc + 5u),Wc)^{b,Wc)

Vu;c G Wc, (14)

(k(uc4-(5u),Vc) - (Pc + (5p, V • Vc) = (c,Vc) - igD,yc-'^)rD

Vvc e Vc, (15)

and on the subgrid scale {V-iuc + 5u),Sw) = {b,Sw) (k(uc-F(5u),(5v)-(pc + <5p,V-5v) = (c,(5v)

ySw&SW, VSveSY.

(16) (17)

Darcy Flow, Multigrid, and Upscaling

343

Then u = Uc + (5u and p = Pc + Sp solve the original variational problem (5)(6). To define the (5-subgrid operator and perform the forward elimination of the coarse components, rewrite the subgrid scale equations with coarse components on the right-hand-side. That is, consider the coarse components as sums of basis elements Pc

yjajW*

and

u^ = 2_.Pj'^i-

(18)

The parts of the 5-operator are obtained by substituting these expressions in (16) and (17) and solving for the a^- and /3j-influence function coefficients. The constant part of the 5 operator is given by solving (V • 5u, 5w) = {b, 5w)

\/Sw € 5W,

(19)

(k^u, Sv) - {5p, V • (5v) = (c, 5v)

V(5v e 5Y.

(20)

\/6w€ 5W, VJv G 5\.

(21) (22)

The V1/c-linear part of the S operator is given by solving (V • (5ui, Sw) = 0 {kSui, 6v) - {Spi, V • (5v) = {wi, V • (5v) The Vc-linear part of the 6 operator is given by solving (V • 5uj,5w) = -{V-vlSw) (k(5u^-,<5v)-(<5pj,V-(5v) = - ( k v ^ c H

\/6w e 5W,

(23)

ydv G 5Y.

(24)

Then 5p = Y^ UiSpi + ^ = 5p{p,)

pjSpj + Sp

+5p{n,)

+5p

(25)

and (5u = V ] ctiSui + 2_] P-jSvLj + 5vL = 5VL{PC)

+(5u(uc)

+5u

(26)

are implicit expressions for 5p and 6u in terms of Pc and Uc. Note that 5p{w].) = 5pi, 5p{vl) = Spj, 6u{wl) = Sui, and 5u(v^) = 6uj. Return to the coarse scale equations (14)-(15). Rewrite them substituting in the influence-function expressions for 5p and Su, and gathering the coarse coefficients ccj and /3j out front. Equations for the coarse components' coefficients result

344

James M. Rath ^ a i ( V • 5ui,wc) + X!'^i(V • (vi + Suj),We) = (6 - V • Su,Wc) i

i

Vu;eGVFc, (27) ^ a i ( k < 5 a i , V,) + ^ / 3 , ( k ( v ^ , + 5u,),Ve) i

3

- Y^ ai{wi + 5pi, V • Ve) - ^ i

pj{5pj, V • Ve)

J

= (c-k<5u,v,) + (<5p,V-Ve)

VVCGVC,

(28)

or using the operator notation (V • (ue + 5u{pc) + <5u(ue)), We) = {b-V( k ( U c + Su{Pc)

5u, Wc)

'iw, G Wc,

(29)

+ (5u(Uc)),Vc)

-(Pc + Sp{pc) + Sp{uc), V • Vc) = (c - k(5u, Vc) + {5p, V • v^) -igD,^c-'^)ro

VVeGVe,

(30)

Solve these equations for aj and /3j (that is, Uc and Pc), and write the solution as p = Pc

+Sp

= ^ atwl + Y2 (^i^Pi + Yl ^^^Pi + ^^ i

i

("^^)

j

and u = Uc

+ (5u

= E ^i^c + E "'"^^^ + E ^i'^"i + "^^^^ j

»

(32)

i

Lastly, (29)-(30) can be written in symmetric form by rearranging terms and using some identities not identified here. See [Arb] for details, as well as proofs that each of the above problems are well-posed. Also note that the i5-subgrid operator has a natural definition. No information has been lost in the reformulation of the problem, and no ad-hoc assumptions have been reintroduced to simplify the action of the 5 operator. 2.3 Approximating the upscaled problem Only after deciding on the decomposition is the approximation then made. We are free to pick discretizations for each subgrid independent of each of the others. The discretization of the coarse spaces is only constrained by the already chosen coarse grid. Standard mixed finite element spaces such as Raviart-Thomas or BDDF elements will do.

Darcy Flow, Multigrid, and Upscaling

345

The subgrid upscaling technique has a number of advantages. Each subgrid problem is independent from the others because of the Neumann boundary conditions along coarse cell edges; the subgrid problems can be solved in paraUel with no communication of boundary data needed across coarse cell edges. Each subgrid problem also has many fewer degrees of freedom than the whole problem (by about a factor of the number of coarse grid cells); these problems are far better conditioned, and may be amenable to a direct solver. Each subgrid problem has the same left-hand side and many different righthand sides creating an opportunity for computational efficiency. The coarse problem has many fewer degrees of freedom than the whole problem (by about a factor of the number of fine cells chosen per coarse ceU) and so is also much better conditioned. If the coarse grid spacing is a small multiple of the fine grid spacing, the work in solving the upscaled problem is nearly all done in solving the coarse problem. Lastly, there is only one coarse grid (not many as in multigrid). Further, the number of degrees of freedom in the upscaling space is nearly as many as that in the fine space (compare Figure 1 with Figure 3 below). However, for small upscaling factors the upscaled solution is only about as computationally expensive as a coarse solution. This would be only mildly interesting if the upscaling solution was only as accurate as a coarse solution, but in fact it captures much more (see example below in Subsection 2.5; also see Figures 10-17 in Section 3 on the multigrid-hke preconditioner). There are some algorithm implementation issues to be considered. First, the elements of SWk may not be easy to compute with. For instance, suppose RTO elements are used for Wh and WH, and SWh = {WH) with the complement taken in M^/,. Then elements of SWh do not have local support on fine cells — they must have zero average on coarse cells. A computational trick explained in [Arb02] and [AB02] avoids this comphcation. Second, there is obvious parallelism in the subgrid problems since each one is independent of all the others, and the coarse-scale influence functions allow disconnecting the subgrid from the coarse scale. However, for coarsescale problem there is not any obvious parallelism. To overcome this lack of paraUelism, one could use a domain decomposition method [GW88] or another technique. Lastly, all of the problems being considered are saddle point problems because of the mixed formulation. For small upscaling factors, the subgrid problems may be solved directly. In other cases and for the coarse problem, iterative solvers may be necessary. The Uzawa algorithm [BF91, BraOl] may be used to solve the indefinite system. Also, the hybrid mixed method [AB85, BF91] or a Schur complement could be used to transform the indefinite problem into a positive definite one.

James M. Rath

346

2.4 S a m p l e a p p r o x i m a t i o n of t h e u p s c a l e d p r o b l e m As an example, use RTO elements to approximate 5W and (5V, denoted SWh and 5Yh, and B D D F l elements for Wc and Vc, denoted WH and V ^ . Let WH,h = WH (B SWh and VHJI = ^H © SVh- Elements of WH are functions which are piecewise constant on coarse cells. Elements of SWh are functions which are piecewise constant on fine cells, but must have a zero average over each coarse cell. Elements of V/f are vector-valued functions where the components normal to coarse cell faces are continuous across those faces, and vary linearly along those faces. Lastly, elements of SVh, are vector-valued functions where the components normal to fine cell faces are continuous across those faces, and are constant along those faces; the normal components across coarse cell faces are zero. See Figure 3 for an illustration.

-»•

-«-

-»

T8^

S)

(5

U

r

J

(

a

ii

r

() if::

;: • (J

•

-ii-

(

o

(

o

)

»

o d)

z •

X

•

i'

()
Fig. 3. Degrees of freedom for 2-D RTO/BDDFl elements on an "upscaled" rectangular grid: ® coarse velocity (linear) degrees of freedom, x subgrid velocity degrees of freedom, and » pressure degrees of freedom. [Arb02]

No closure assumption is made regarding the permeability. Accuracy is simply a question of approximation theory — how well does VHJI approxim a t e V ? — and not a question of how much the physics is changed. Defining effective coarse permeabilities is not necessary. T h e closure assumption can be said to be t h a t ah flux across a coarse element face is due to the coarse-scale functions. (The 5-subgrid operator is approximated using well-worn techniques; it is not altered in an ad-hoc fashion to facilitate computation.) In this example, to compensate for the coarseness

Darcy Flow, Multigrid, and Upscaling

347

of the restriction of velocities along coarse edges, higher order accurate basis functions have been employed. For the BDDFl/RTO elements, some error estimates are [Arb] ||u - UHAO

< C{\\phh^

+ (||u||2 + ||u • 42,ro)H^}

= 0[H^),

(33)

lb - PH,hh < C{\\p\\ih + 1|V • u||i/i2 + (||u||2 + ||u • ^ i | 2 , r j ^ ' } = 0{h + H') (34) However, the simplicity of the above expressions belies some error analysis complications; for instance, the 'irH,h operator for the upscaled elements is obtained not just from gluing together the operators from the coarse elements TTH and each of the subgrid elements STTh,,Ec: but also solving an auxiliary problem (except under additional assumptions). The error estimates above also do not capture all the subtleties of the subgrid upscaled solution (see, for instance, the graphs of the eigenstructure of the subgrid upscahng preconditioner in Section 3). One thing to note, though, is that if H/h is small, then 0{h + H^) = 0{h) for the pressure and 0{H'^) = OHH/hfK^) for the velocity. A comparison between the upscaling error estimates with those for BDMl/BDDFl elements on fine grid, (12) and (13), shows that they are the

2.5 Some practical results To show the ability of the subgrid upscaling to approximate problems almost as well as a full fine-scale solution (and far better than a coarse-scale one), some results from a simulated quarter five-spot oil reservoir water flood are presented. The above ideas were adapted to two-phase immiscible displacements, and run on the following example [ArbOO]. The domain is a 40 m by 40 m square with a uniform, rectangular, 40 x 40 grid. The base-10 logarithm of the permeability field is shown in Figure 4; the porosity is 25%. The field initially has a 20% saturation of water. There is an injection well in the lower-left corner, and a production well in the top-right corner. Each has a rate of 0.2m'^/day, and water is being injected. Figures 5, 6, and 7 speak for themselves. The upscaled and fine-scale solutions are certainly close in the eyeball norm; the coarsened solution is not close to either and fails qualitatively to capture some behavior. The upscaled solution requires only about as much time to compute as the coarse solution, but has nearly as many degrees of freedom as the fine solution; hence the "success." Throughout most of the rest of the paper, only RTO/RTO elements will be used. Although the ideas are applicable to others, the description, analysis, and implementation of higher order combinations gets much more complicated.

348

James M. Rath

Fig. 4. Logarithm of the permeability field (geostatistically generated).

Fig. 5. Water saturation contours at 100 and 500 days for the fine 40 x 40 solution.

2.6 Other upscaling techniques

Upscaling techniques are numerous; surveys of various ones currently under study can be found in [Dur02, Far02, RdM97, WGH961. A few descriptions follow. Chen et al. [CDGWOS] describe their technique as a local/global one: use a global coarse simulation t o determine appropriate boundary conditions for local simulations that then determine an effective coarse permeability. Other techniques are local; for instance, periodic boundary conditions can be imposed on the subgrids when computing effective permeabilities as in [BLP78, SP801. Oversampling is another possibility where a local prob-

Darcy Flow, Multigrid, and Upscaling

349

Fig. 6. Water saturation contours at 100 and 500 days for the upscaled 5 x 5 solution.

Fig. 7. Water saturation contours at 100 and 500 days for the coarse 5 x 5 solution.

lem (with various boundary conditions) that is slightly larger than a subgrid is used to determine the effective permeability for that subgrid; Chen and Hou [CH02] and Hou and Wu [HW97] use a multiscale finite element technique to implement this idea. Wu, Efendiev, and Hou also use this idea [EHWOO], and in [WEH02] apply a flow-based gridding dynamic approach to limit the necessary upscaling. Holden and Nielsen [HNOO] have developed a global upscaling method that uses a least-squares problem to determine effective coarse permeabilities. Some distinguishing features of the upscaling technique used here are the use of mixed methods (and the variational setting), the lack of ad-hoc assumptions about the 6-subgrid operator, and the lack of computed effective coarse permeabilities.

350

James M. Rath

3 A Two-level Multigrid-like Scheme In traditional multigrid, one pairs smoothings on fine and coarse grids together with the intent of each smoother reducing the error in different parts of the spectrum. The fine-grid smoother reduces the error on short scales, and the coarse-grid smoother reduces it on larger scales [BHMOO]. The action of each smoother can be computed using a number of operations and storage units proportional to the number of unknowns. The tolerance for the stopping criterion (accuracy) can be chosen independently of the number of unknowns (but not the ratio of largest to smallest permeability). For problems with a great number of unknowns, multigrid is therefore preferable to direct methods. In our modified multigrid, upscaling is substituted for coarsening. Upscaling produces a better approximation than coarsening (at both fine and coarse scales), and is just as cheap. By substituting upscaling for coarsening we keep all of multigrid's benefits, and an even faster scheme results. To diagram the multigrid algorithm and our variant, let us introduce some notation for the matrices involved. If we cast equations (5)-(6) in operator form, we get " 0 B'^' P 'b (35) -B D _ u = c Forming the Schur complement of the velocities gives Ap={B'^D-^B)p = f = b-B'^D--^c.

(36)

For the discrete versions of these equations, add a subscript h for the fine scale, H for the coarse scale, and H, h for the upscaled. A schematic of a twolevel V-cycle multigrid is shown in Figure 8. Our modified version is shown in Figure 9 — the only change is the substitution of upscaled for coarse scale. Multigrid works so well partly because smoothing the solution iterates on different levels reduces the error in different parts of the spectrum. Towards that end, we have developed code to compute and plot the eigenvectors (and corresponding eigenvalues) for the action of each smoother on the error. The matrices analyzed have the form I — M A from the error update equation Gj+i = (/ — M A)ei that describes the action of the preconditioner M on the error e. The plots are of eigenvalue/eigenvector pairs. The horizontal axis shows the norm of the gradient of the eigenvectors (considered as functions) — a proxy for "eigenfrequency." The absolute value of the corresponding eigenvalue is used as the vertical axis. The results of applying the eigenanalysis to several two dimensional examples can be seen in Figures 10-17; some convergence histories are shown in Figures 18-19. Each of the examples has the unit square as the domain with uniform porosity and Dirichlet boundary conditions. The first example has a very simple permeability: it takes one value on the left half of the domain, and another value on the right half (shown in Figure 10). In Figure 1 1 a "typical" result for the eigenstructure analysis as applied to the simple two-level permeability field is shown. On the horizontal axis

Darcy Flow, Wlultigrid, and Upscaling

351

S k100'1'FI c, =

$ (cliagAhf- 1 r,

P I + I= p, -+ CI ~ + i + l

rs = fh START

- Ahpi A T>OSE'! Is llrt11 siritxll?

i=O

po = 0 ro = ftL

- ALP{) COAItSEY 13, = A;' Rr, P ~ + I= P I

+ PEI

i+-i+l

FISISII Ph

Pa

rl = f i t - Ahpv

Fig. 8. Schematic of two-level V-cycle multigrid as used to solve the linear system

Ahph = fh. The sequence of iterates for the pressure pi are computed using the ith-stage residual r , = f h - Ahp,, the diagonal elements of Ah (diagAh used in the Jacobi smoothing step), arid the coarse corrector from solving a reduced problem AH. The restriction operator R : Wh --' W I ~x 0, and the prolongation operator P : W H x V H + Wh are the natural projection and inclusion operators. Also note that there may be many ( u ) applications of the smoother between upscaling steps. is measured the norm of the gradient of the eigenvector when interpreted as a 2-D pressure field (this measure is a stand-in for frequency). The vertical axis shows on a logarithmic scale the absolute value of the corresponding eigenvalue. The blue dots show the check-mark-like structure for a weighted Jacobi smoother. This picture is similar to ones usually used in multigrid analysis [BHMOO]. The cyan dots show the structure one obtains for the corresponding weighted Jacobi smoother on the coarse level. I t has the same shape, but the small-eigenvalue dip comes at lower frequencies (coarser modes) as expected. The magenta dots show the structure for the preconditioner that exactly solves the coarse problem. It has eigenvalues greater than one in the coarser modes because the upscaling factor H l h is equal t o three, and not two (the more usual result with eigenvalues bounded by one can be seen in the bottom right of Figure 13). Lastly, the red dots show the structure for the upscaling preconditioner. Some notable features are its complicated organization, that coarse modes (over a large range) have quite small eigenvalues, that there is a steady rise in eigenvalues versus frequency, and that the eigenvalues top out at greater than one. However, this maximum occurs near where the weighted Jacobi smoother is performing at its best.

352

James M. Rath

i t i + l rs = f,,

-

.At,p,

A

1.- l'SC1,2 121;

E; = : 4 ~ !Rr+ ,~ pi+l = PI $* PEi

FTSISlI

i-i+l

PI,

" PI

r, = f f , - A f , p (

Fig. 9. Schematic of the modified two-level V-cycle multigrid as used to solve the

linear system Ahph = fh. The sequence of iterates for the pressure pi are computed using the ith-stage residual ri = fh - Ahpi, the diagonal elements of Ah (diagAh used in the Jacobi smoothing step), and the upscale corrector from solving a reduced problem AH,^. The restriction operator R : Wh --t WH,hx 0, and the prolongation ~ Wh are the natural projection and inclusion operaoperator P : W H , x~ V H , --t tors. Also note that there may be many (v) applications of the smoother between upscaling steps. Figure 12 shows the effect of varying h, the grid size, while leaving the upscaling factor H l h and the permeability field alone. As can be seen, the overall structure remains the same independent of h, but the level of detail one sees in the diagrams increases, as does the scale of the frequencies plotted. Figure 13 shows the effect of varying H l h , the upscaling factor, while leaving the fine grid size h and the permeability field alone. As H l h increases, there is a shift t o lower frequencies for the coarse smoother and the upscaling preconditioner. (As noted above, the eigenvalues of coarse modes for the exact coarse preconditioner get larger with increasing H l h . ) There is a general worsening of performance with increasing H l h as would be expected. Figure 14 shows the Arcoperm permeability field [Arc97], and Figure 16 shows the Brent permeability field [Lin92]. Both show considerably more variation than the simple two-level permability field. They both also have a greater ratio between the greatest and least values of the permeability: the Arcoperm field about a half an order of magnitude, and the Brent field more than four orders of magnitude. For the Arcoperm field, Figure 15 shows the same rich structure as seen for the two-level permeability field, but surprisingly the up-

Darcy Flow, Multigrid, and Upscaling

353

Fig. 10. A simple permeability field.

scaling preconditioner has a similar performance: small eigenvalues for coarse modes with a general trend towards larger (but not too large) eigenvalues for fine modes. At the greatest upscaling factor Hlh of six, the middle modes begin to suffer, too. The results in Figure 17 show the same general trends. However, the middle modes become much more of a problem (note the vertical scale has changed from a high of 10' before to 10' now). One obvious conclusion to draw from the eigenstructure diagrams is that the performance of the upscaling level suffers in the mid-frequency eigenvectors (see also the convergence histories below). Perhaps Krylov space methods (a CG-like iteration) [Bru95, GV96, Dem971, a third level with an upscaler based on a grid staggered relative to the first upscaler, semi-coarsening in different levels in different directions [CWW92], or perhaps a traditional multigrid coarse smoother might be used to improve the method. The diagrams also lead to the reasonable inference that the behavior of the proposed two-level scheme depends on Hlh and k*/k,, but not the absolute level of h. This mirrors the behavior of multigrid, but we have not proven this yet for this scheme. It also seems a reasonable inference that upscaling modified multigrid will always perform better than traditional multigrid. In this proposal, we have been assuming that a single coarsening of the grid is enough to result in a problem that is computationally inexpensive to use. Sometimes this "coarse" grid just is not coarse enough, and the associated problem is still too computationally expensive to work with. In this case, a

James M. Rath

354

amplify attenuate

8"

0

0.5

1

1.5

2

2.5

3

3.5

4

lo4 IUTX!

Eigen-Frequency

h igl I

Fig. 11. A "typical" eigenstructure result for the smoothers and upscaler (RTO elements used at all levels). The fine grid used was 24 x 24, the coarse grid 6 x 6 (with H / h = 4), and Icz = 0.1. The colored dots are eigenpairs for the * coarse

smoothing preconditioner, fine smoothing preconditioner, and preconditioner.

subgrid upscaling

true variational multi-scale (and not just two-scale) method for the upscaling level would be appropriate. This might be accomplished say by decomposing V = V , $ S V = V , , @ A V @ S V and W = W , @ S W = W , , $ A W @ S W into three levels (or more) with a correspoding approximation with multiple levels. The details of this possibility have yet to be worked out. It might also be possible to use traditional algebraic multigrid on the coarsened system that comes from the upscaled problem (29)-(30). The eigenstructure diagrams also give us a way to estimate how often the Jacobi smoother is applied relative t o the number of times the upscaling preconditioner is applied. It is also possible to then estimate the overall error reduction factor for an iteration of our two-level method. This is accomplished by examining frequency by frequency the product of the error reduction factors (eigenvalues) of the smoother and upscaling preconditioner, and by computing the maximum such product. This allows us also to estimate the relative performance of multigrid with the two-level upscaling/smoother method for a given problem. Of course, this is assuming that eigenvectors with

Darcy Flow, Multigrid, and Upscaling

Vary h ; F i x H/h= 3 ; Fix ka = 0.1 . . . .

to-

.""...t..:-...'..".

1'

,,

.

.

.... .

,:a'

A

. ..

..

, I

1

.

.

I-.---.

A

1

''0

. .

10'

355

d.

&

- 9 n r E g W d b V 1

+

*

I

.

,A

m

,

. .. . '

.I

I

..

E ,L & &

K& wMqlPa,.4-#IY1

An A Lm

mm

12 x 12

6x6 i

ID'

...

..

-. .

.. **

-

L

--ma-(*.-*-

24 x 24

_ .

& I

em.

;

.

'

ID.Y--Y-*-

A

-

.

.I$

48 x 48

Fig. 12. Eigenstructure results for the preconditioners for problems with various fine grid sizes h, but fixed upscaling ratio H l h and fixed permeability. The colored dots are eigenpairs for the * coarse smoothing preconditioner, e coarse exact preconditioner, fine smoothing preconditioner, and subgrid upscaling preconditioner.

like "frequency" are alike; in reality, the eigenbases differ and the comparison for a given frequency is not apples-to-apples. Some examples of convergence histories using this analysis are shown in Figure 18. The logarithm of the norm of the residuals/errors is shown on the vertical axis; the number of V-cycle iterations is shown on the horizontal axis. The red and blue pairs show the history for the upscaling modified multigrid; the pink and cyan pairs for an ordinary two-level multigrid. Significantly faster convergence is obtained with the upscaling than with multigrid (although both display linear convergence and dependence on k*/k,). A zero initial guess for the solution is used along with a random source term. T h e diagrams in the top row show the eigenstructure diagram and convergence history for the simple permeability field (shown in Figure 10) on a 24x 24 grid with H l h = 4 and k2 = 0.1. An estimation from the eigenstructure diagram indicates that a single Jacobi smoothing step pre- and post-upscaling in a V-cycle will ensure convergence. Indeed this is the case: the upscaling scheme has a convergence factor of about 0.6 per V-cycle step. On the other hand,

James M . Rath

356

Fix h = 1/24 ; Vary H / h ; Fix k2 = 0.1 w/RTO .... . .i>.. .,. .

.

,.

.

,d

."'

.

.

.

.

.

.

*..4.. 5 _.-.-.

..

r.

.......

.. ...

Lo-.

' ?.

.. .. .

.

7m-.

.

I

d

*- U . . - - y . p n C -

Ph

;'u

a

n

.

.

c

.

IS'

H/h =6 Id

d

.* R . * . r '. m C . W n r * . n , ~k.W

.

.

H/h =4 .

.

;'u

.

.

4

.I#

.

. . . .. +

d

*-.

... ..-. 5 :

... ?

,ma.

-

...

-

I

8

I

'

d"

;

*;

;

-4.mme*.e*..n*-

H/h = 3

0;

; '&

4

td

0

0;

;

'

.~l)*mnr~.w-~-

;r

i

*L

4

I,.'

H/h=2

Fig. 13. Eigenstructure results for the preconditioners for problems with various upscaling ratios H l h , but fixed fine grid size h and fixed permeability. The colored

dots are eigenpairs for the .r coarse smoothing preconditioner, o coarse exact preconditioner, fine smoothing preconditioner, and * subgrid upscaling preconditioner. multigrid has a factor of about 0.94 for the same V-cycle with a smoother on the coarse level. The diagrams in the middle row show the the eigenstructure diagram and convergence history for the Arcoperm permeability field shown in Figure 14 for H l h = 5. An estimation from the eigenstructure diagram indicates that two Jacobi smoothing steps pre- and post-upscaling in a V-cycle again will ensure convergence. Again, the upscaling scheme clearly outperforms ordinary multigrid. The diagrams in the bottom row show the the eigenstructure diagram and convergence history for the Brent permeability field shown in Figure 16 also for H l h = 5. This time estimating the number of necessary smoothings per iteration is more difficult. Forty smoothings were used; note the most smoothings comes from the coarsest eigenmode - the red dot furthest to the left. This is most likely an overestimate of the number of smoothings necessary

Darcy Flow, Multigrid, and Upscaling

357

Fig. 14. The Arcoperm permeability field. The minimum base-10 logarithm of the

permeability is -0.2916 and the maximum 0.1477. for a convergent scheme, but smoothings are inexpensive relative t o upscalings (and probably have a beneficial effect - see below). Figure 19 shows some effects of underestimating the number of necessary smoothings. In the case of the Arcoperm permeability field, changing the number of smoothings from two t o one only slows convergence (and both ordinary and modified multigrid perform comparably). In the case of the Brent permeability field, the iterations diverge. One feature t o note in the top diagram for the Arcoperm permeability field is that there is a steep initial decline in the size of the error. This is probably attributable t o the initial guess for the solution (all zeros) having an error that has large components in the direction of the coarse eigenmodes of the upscaling preconditioner (this probably also explains the large jump in the first or first few steps in each of the other convergence histories - note that the blue/cyan and red/pink dots coincide for step "zero"). The "stalling" of the convergence is probably attributable t o the poor performance of the preconditioner on middle eigenmodes. Another feature t o note is that the doubling of smoothings roughly doubles the rate of convergence of ordinary multigrid, but the modified multigrid experiences a dramatic improvement in performance. It would seem that overestimating the number of necessary fine smoothings has a synergistic effect (and is not very computationally expensive).

James M. Rath

358

Fix h = 1/40 ; Vary H / h ; Fix k w/RTO ,d -

d -

-

to4*

L

A

;

i*

i s .a' 7:. rn!m*rnC.C.ldlcN

A ;

A

x

'd

0

A

b

&

H/h =2 ,d 1

. ... . .-.

..,I

.

; *;

; ,

rnW.LM'IIY-I*

a. -

A

:.

*

r ID'

H/h=5

,.--.

.rr.. I.

..... 1 ! & A d a ; a ;

~ L - C - ~ Y P A

I;;;;I:I n w b L L b . L ;

H/h= 8

r d

; z ~ c I z. I ;

ryr-yawru-asw

H

d

H / h = 20

Fig. 15. Eigenstructure results for the preconditioners for problems with various upscaling ratios H l h with the Arcoperm permeability field in Figure 14. The colored dots are eigenpairs for the .- coarse smoothing preconditioner, * coarse exact preconditioner, fine smoothing preconditioner, and subgrid upscaling preconditioner.

One last feature t o note is t h a t for all the convergent schemes, the error is monotonically decreasing (as is expected for a multigrid-like scheme). Contrast this with (preconditioned) conjugate gradients where the convergence is typically non-monotone (although theoretical bounds are monotone). Conjugate gradients is also more costly asymptotically. In the future we would like t o produce plots of the size of spectrum of the error versus iteration number. A break-down of inter-step frequency reduction (the spectrum before and after smoothing, and before and after upscaling) would also be useful. The ability t o visualize a given eigenmode (that is, make an n-dimensional plot of the pressure that corresponds t o an eigenvector) by clicking on the eigenstructure plot would be a handy tool. In future research, we hope t o be able t o prove t h a t our two-level scheme converges linearly (independent of h, but dependent on H l h and k*/k,), and does so faster than ordinary multigrid. Toward that end, the methods used in standard multigrid will certainly be useful as the upscaling scheme is

Darcy Flow, Multigrid, and Upscaling

359

Fig. 16. The Brent permeability field. The minimum base-10 logarithm of the

permeability is -0.6909 and the maximum 3.5230. posed in a variational setting. There are several monographs [Hac85, McC87, Wes92, Bra93, Hac93, Bru95, Saa95, Sha95, BHMOO, Bra011 and review articles [Xu92, Yse93, Xu97a, Xu97bj that provide useful information (several of the monographs on iterative methods generally). As well, Bramble, Pasciak, and Xu have developed a general framework for subspace correction methods [BPX91, BPWXgla, BPWXglb, Xu92, Bra93, Xu97a, Xu97bl. Their work will be useful in having developed a unified approach to multigrid-like methods, and also specifically because they treat the possibility of non-nested spaces for corrections ( V H , is~ not necessarily a subset of V h ) , and perturbed linear forms (e.g., using quadrature on the term (kuh,v h ) ) . Lastly, we might try to modify the upscaling preconditioner so that it itself is a smoother. Also, the upscaling preconditioner might be modified for use in discontinous Galerkin (DG) methods, or the expanded mixed method [AWY97].

360

M,Rath

James

. .

d

Fix h = 1/40 ; Vary H / h ; Fix k w/RTO .

,

,

.

,

,

,

.

lf

.

.

.

.

.

.

.

\.

.

d.

.

........

.

.

- ..

a .

"

I@',

A

.

.

.

.

.

.

..>

'

"

U

"

U

.

.

.

*;

.

;A

.

t

mid

'4 &

A

A

; *;

A

h

;A

.IIIC-.*cIp.dCI*

2

I lb

H/h =5

.

'*..,, ..-,

.

-

.

.

.

.

.

.

ih.

..

'

.

U

,:,

&

Hjh = 2

.. ..... 3:: . :. O

;

A &

mulbl-.Ips-dLW*

.

..

.Z

. U

'

!

%

1

.

< 1

II*(.IL-C~eLLUI

H/h=8

I

4

l

d

I

r d

A

I

~

U

D

I

U

I

I

B

rnlllnn..,.r*l.l...-N

U

t

A

. U

1

1

I

r d

H / h = 20

Fig. 17. Eigenstructure results for the preconditioners for problems with various upscaling ratios H l h with the Brent permeability field in Figure 16. The colored dots are eigenpairs for the coarse smoothing preconditioner, e coarse exact preconditioner, fine smoothing preconditioner, and 0 subgrid upscaling preconditioner.

Darcy Flow, Multigrid, and Upscaling

361

Fig. 18. Convergence histories (right column) of the two-level upscaling and multigrid schemes for the three sample permeability fields in Figures 10, 14, and 16. The vertical axis shows the logarithm of the residuals/errors; the horizontal axis the number of V-cycle iterations. The light and dark blue colored dots mark the sizes of the residual for the multigrid and upscaling schemes, respectively. The light and dark red dots mark the sizes of the error. For reference, the left column shows the eigenstructure diagrams for the corresponding permeability fields.

362

James M. Rath

0

10

20

90

40

-1

50

Brent (v = 5 )

I

80

70

..

10''

lo'*

BO

numbex

.

-

Resklual(UP) Ermr (UP) Residual (MQ) Ermr(MG)

.....

10'~-

......... ............ .-aL

-

...

........ . .....*..........-.

Fig. 19. More convergence histories of the two-level upscaling and multigrid schemes that illustrate the effect of and underestimating the number of necessary smoothings. The coloring scheme is the same as used in Figure 18. The diagram on the top is comparable to the one in the center-right of Figure 18; one smoothing per step are used instead of two. The diagram on the bottom is comparable to the one in the bottom-right of Figure 18; five smoothings per step are used instead of forty.

Darcy Flow, Multigrid, and Upscaling

363

References [AB85]

D. N. Arnold and F. Brezzi. Mixed and nonconforming finite element methods: implementation, postprocessing and error estimates. RAIRO Model. Math. Anal. Numer., 19:7-32, 1985. [AB02] T. Arbogast and S. L. Bryant. A two-scale numerical subgrid technique for waterflood simulations. SPE J., pages 446-457, Dec. 2002. [Arb] T. Arbogast. Analysis of a two-scale, locally conservative subgrid upscaling for elliptic problems. [ArbOO] T. Arbogast. Numerical subgrid upscaling of two-phase flow in porous media. In Z. Chen, R. E. Ewing, and Z.-C. Shi, editors. Numerical treatment of multiphase flows in porous media, volume 552 of Lecture Notes in Physics, pages 35-49. Springer, Berlin, 2000. [Arb02] T. Arbogast. Implementation of a locally conservative numerical subgrid upscaling scheme for two-phase Darcy flow. Computational Geosciences, 6:453-481, 2002. [Arc97] Arcoperm permeability field data. Personal correspondence, 1997. [AWY97] T. Arbogast, M. F. Wheeler, and I. Yotov. Mixed finite elements for elliptic problems with tensor coefficients as cell-centered finite differences. SIAM J. Numer. Anal., 34:828-852, 1997. [BDDF87] F. Brezzi, J. Douglas, Jr., R. Duran, and M. Fortin. Mixed finite elements for second order elliptic problems in three variables. Numer. Math., 51:237-250, 1987. [BDM85] F. Brezzi, J. Douglas, Jr., and L. D. Marini. Two families of mixed elements for second order elliptic problems. Numer. Math., 47:217-235, 1985. [Bea72] J. Bear. Dynamics of Fluids in Porous Media. Dover, New York, 1972. [BF91] F. Brezzi and M. Fortin. Mixed and hybrid finite element methods. Springer-Verlag, New York, 1991. [BHMOO] William L. Briggs, Van Emden Henson, and Stephen Fahrney McCormick. A Multigrid Tutorial. SIAM, 2000. [BLP78] A. Bensoussan, J. L. Lions, and G. Papanicolaou. Asymptotic Analysis for Periodic Structure. North Holland, Amsterdam, 1978. [BPWX91a] James H. Bramble, Joseph E. Pasciak, J. Wang, and Jinchao Xu. Convergence estimates for multigrid algorithms without regularity assumptions. Mathematics of Computation, 1991. [BPWXQlb] James H. Bramble, Joseph E. Pasciak, J. Wang, and Jinchao Xu. Convergence estimates for product iterative methods with applications to domain decomposition. Mathematics of Computation, 57:1-21, 1991. [BPX91] James H. Bramble, Joseph E. Pasciak, and Jinchao Xu. The analysis of multigrid algorithms with nonnested spaces or noninherited quadratic forms. Mathematics of Computation, 56:1-34, 1991. [Bra93] James H. Bramble. Multigrid Methods. Number 294 in Pitman Research Notes in Mathematics. Longman Scientific and Technical, New York, 1993. [BraOl] Dietrich Braess. Finite elements : theory, fast solvers, and applications in solid mechanics. Cambridge University Press, New York, second edition, 2001.

364 [Bru95]

James M. Rath

Are Magnus Bruaset. A Survey of Preconditioned Iterative Methods. Number 328 in Pitman Research Notes in Mathematics. Longman Scientific and Technical (John Wiley), New York, 1995. [CDGW03] Y. Chen, Louis J. Durlofsky, M. Gerritsen, and Xian-Huan Wen. A coupled local-global upscaling approach for simulating flow in highly heterogeneous formations. Advances in Water Resources, 2003. Submitted. [CH02] Z. Chen and T. Y. Hou. A mixed multiscale finite element method for elliptic problems with oscillating coefficients. Math. Camp., 2002. [CWW92] L. C. Cowsar, A. Weiser, and M. F. Wheeler. Parallel multigrid and domain decomposition algorithms for elliptic equations. In D. Keyes et al., editors. Fifth International Symposium on Domain Decomposition Methods for Partial Differential Equations, pages 376-385. SIAM, Philadelphia, 1992. [Dar56] Henry Darcy. The Public Fountains of the City of Dijon, chapter Appendix D. Victor Dalmont, Paris, 1856. [Dem97] James W. Demmel. Applied numerical linear algebra. SIAM, Philadelphia, 1997. [DEW83] J. Douglas, Jr., R. E. Ewing, and M. F. Wheeler. Approximation of the pressure by a mixed method in the simulation of miscible displacement. R.A.I.R.O. Model. Math. Anal Numer., 17:17-33, 1983. [DEW84] B. L. Darlow, R. E. Ewing, and M. F. Wheeler. Mixed finite element methods for miscible displacement problems in porous media, SPE 10501. Soc. Petrol Eng. J., 24:391-398, 1984. [DJ97] Clayton V. Deutsch and Andre G. Journel. GSLIB geostatistical software library and user's guide. Oxford University Press, New York, second edition, 1997. [DR85] J. Douglas, Jr. and J. E. Roberts. Global estimates for mixed methods for second order elliptic equations. Math. Comp., 44:39-52, 1985. [Dur02] Louis J. Durlofsky. Upscaling of geological models for reservoir simulation: Issues and approaches. Computational Geosciences, 6:1-4, 2002. [EHWOO] Y. R. Efendiev, T. Y. Hou, and X.-H. Wu. Convergence of a nonconforming multiscale finite element method. SIAM J. Numer. Anal, 37:888-910, 2000. [ERW84] R. E. Ewing, T. F. Russell, and M. F. Wheeler. Convergence analysis of an approximation of miscible displacement in porous media by mixed finite elements and a modified method of characteristics. Comp. Meth. in Appl. Mech. and Engng., 47:73-92, 1984. [Far02] C. L. Farmer. Upscaling: a review. International Journal for Numerical Methods in Fluids, 40(l-2):63-78, 2002. [FC79] R. A. Freeze and J. A. Cherry. Groundwater. Prentice-Hall, Englewood Cliff's, New Jersey, 1979. [G'*'85] James Glimm et al. Sharp and diffuse fronts in oil reservoirs: Front tracking and capillarity. In William E. Fitzgibbon, editor, Mathematical and Computational Methods in Seismic Exploration and Reservoir Modeling, pages 54-76, Philadelphia, 1985. SIAM. [GV96] Gene H. Golub and Charles F. Van Loan. Matrix Computations. The Johns Hopkins University Press, Baltimore, third edition, 1996.

Darcy Flow, Multigrid, and Upscaling [GW88]

365

R, Glowinski and M. F. Wheeler. Domain decomposition and mixed finite element methods for elliptic problems. In R. Glowinski et al., editors, First International Symposium on Domain Decomposition Methods for Partial Differenti