Introduction to Linear Algebra with Applications

DeFranza

Linear Algebra Introduction to Linear Algebra with Applications by Jim DeFranza and Daniel Gagliardi provides the proper balance between computation, problem solving, and abstraction that will equip students with the necessary skills and problem solving strategies to allow for a greater understanding and appreciation of linear algebra and its numerous applications.

Abstract theory is essential to understanding how linear algebra is applied. Each concept is fully developed presenting natural connections between topics giving students a working knowledge of the theory and techniques for each module covered. Applications have been carefully chosen to highlight the utility of linear algebra in order to see the relevancy of the subject matter in other areas of science as well as in mathematics.

MD DALIM 976667 7/29/08 CYAN MAG YELO BLACK

Ranging from routine to more challenging, each exercise set extends the concepts or techniques by asking the student to construct complete arguments. End of chapter True/False questions help students connect concepts and facts presented in the chapter. Examples are designed to develop intuition and prepare students to think more conceptually about new topics as they are introduced. Students are introduced to the study of linear algebra in a sequential and thorough manner through an engaging writing style gaining a clear understanding of the theory essential for applying linear algebra to mathematics or other fields of science. Summaries conclude each section with important facts and techniques providing students with easy access to the material needed to master the exercise sets.

www.mhhe.com/defranza

ISBN 978-0-07-353235-6 MHID 0-07-353235-5

www.mhhe.com

Linear Algebra

Introduction to Linear Algebra with Applications provides students with the necessary tools for success:

Linear Algebra

Revised Pages

INTRODUCTION TO LINEAR ALGEBRA WITH APPLICATIONS

Jim DeFranza St. Lawrence University

Dan Gagliardi SUNY Canton

First Pages

INTRODUCTION TO LINEAR ALGEBRA WITH APPLICATIONS Published by McGraw-Hill, a business unit of The McGraw-Hill Companies, Inc., 1221 Avenue of the Americas, New York, NY 10020. Copyright  2009 by The McGraw-Hill Companies, Inc. All rights reserved. No part of this publication may be reproduced or distributed in any form or by any means, or stored in a database or retrieval system, without the prior written consent of The McGraw-Hill Companies, Inc., including, but not limited to, in any network or other electronic storage or transmission, or broadcast for distance learning. Some ancillaries, including electronic and print components, may not be available to customers outside the United States. This book is printed on acid-free paper. 1 2 3 4 5 6 7 8 9 0 DOC/DOC 0 9 8 ISBN 978–0–07–353235–6 MHID 0–07–353235–5 Editorial Director: Stewart K. Mattson Senior Sponsoring Editor: Elizabeth Covello Director of Development: Kristine Tibbetts Developmental Editor: Michelle Driscoll Director of Marketing: Ryan Blankenship Marketing Coordinator: Sabina Navsariwala Project Manager: Joyce Watters Senior Production Supervisor: Sherry L. Kane Senior Media Project Manager: Tammy Juran Designer: Laurie B. Janssen Cover Designer: Ron Bissell (USE) Cover Image: Royalty-Free/CORBIS Senior Photo Research Coordinator: John C. Leland Supplement Producer: Melissa M. Leick Compositor: Laserwords Private Limited Typeface: 10.25/12 Times Printer: R. R. Donnelly Crawfordsville, IN Library of Congress Cataloging-in-Publication Data DeFranza, James, 1950– Introduction to linear algebra / James DeFranza, Daniel Gagliardi. —1st ed. p. cm. Includes index. ISBN 978–0–07–353235–6—ISBN 0–07–353235–5 (hard copy : alk. paper) 1. Algebras, Linear—Textbooks. 2. Algebras, Linear—Problems, exercises, etc. I. Gagliardi, Daniel. II. Title. QA184.2.D44 2009 515
.5—dc22 2008026020

www.mhhe.com

Revised Pages

To Regan, Sara, and David —JD To Robin, Zachary, Michael, and Eric —DG

Revised Pages

Revised Confirming Pages

About the Authors

Jim DeFranza was born in 1950 in Yonkers New York and grew up in Dobbs Ferry New York on the Hudson River. Jim DeFranza is Professor of Mathematics at St. Lawrence University in Canton New York where he has taught undergraduate mathematics for 25 years. St. Lawrence University is a small Liberal Arts College in upstate New York that prides itself in the close interaction that exists between students and faculty. It is this many years of working closely with students that has shaped this text in Linear Algebra and the other texts he has written. He received his Ph.D. in Pure Mathematics from Kent State University in 1979. Dr. DeFranza has coauthored PRECALCULUS, Fourth Edition and two other texts in single variable and multivariable calculus. Dr. DeFranza has also published a dozen research articles in the areas of Sequence Spaces and Classical Summability Theory. Jim is married and has two children David and Sara. Jim and his wife Regan live outside of Canton New York in a 150 year old farm house. Daniel Gagliardi is an Assistant Professor of Mathematics at SUNY Canton, in Canton New York. Dr. Gagliardi began his career as a software engineer at IBM in East Fishkill New York writing programs to support semiconductor development and manufacturing. He received his Ph.D. in Pure Mathematics from North Carolina State University in 2003 under the supervision of Aloysius Helminck. Dr. Gagliardi’s principle area of research is in Symmetric Spaces. In particular, his current work is concerned with developing algorithmic formulations to describe the fine structure (characters and Weyl groups) of local symmetric spaces. Dr. Gagliardi also does research in Graph Theory. His focus there is on the graphical realization of certain types of sequences. In addition to his work as a mathematician, Dr. Gagliardi is an accomplished double bassist and has recently recorded a CD of jazz standards with Author/Pianist Bill Vitek. Dr. Gagliardi lives in northern New York in the picturesque Saint Lawrence River Valley with his wife Robin, and children Zachary, Michael, and Eric.

v

Revised Pages

Contents

Preface ix CHAPTER

1

Systems of Linear Equations and Matrices

1

1.1 Systems of Linear Equations 2 Exercise Set 1.1 12 1.2 Matrices and Elementary Row Operations 14 Exercise Set 1.2 23 1.3 Matrix Algebra 26 Exercise Set 1.3 37 1.4 The Inverse of a Square Matrix 39 Exercise Set 1.4 45 1.5 Matrix Equations 48 Exercise Set 1.5 51 1.6 Determinants 54 Exercise Set 1.6 65 1.7 Elementary Matrices and LU Factorization 68 Exercise Set 1.7 77 1.8 Applications of Systems of Linear Equations 79 Exercise Set 1.8 84 Review Exercises 89 Chapter Test 90 CHAPTER

2

Linear Combinations and Linear Independence 2.1 Vectors in ⺢n 94 Exercise Set 2.1 99 2.2 Linear Combinations 101 Exercise Set 2.2 108 2.3 Linear Independence 111 Exercise Set 2.3 120 Review Exercises 123 Chapter Test 125

vi

93

Revised Pages

Contents

CHAPTER

3

Vector Spaces

127

3.1 Definition of a Vector Space 129 Exercise Set 3.1 137 3.2 Subspaces 140 Exercise Set 3.2 154 3.3 Basis and Dimension 156 Exercise Set 3.3 171 3.4 Coordinates and Change of Basis 173 Exercise Set 3.4 182 3.5 Application: Differential Equations 185 Exercise Set 3.5 193 Review Exercises 194 Chapter Test 195 CHAPTER

4

Linear Transformations

199

4.1 Linear Transformations 200 Exercise Set 4.1 211 4.2 The Null Space and Range 214 Exercise Set 4.2 223 4.3 Isomorphisms 226 Exercise Set 4.3 233 4.4 Matrix Representation of a Linear Transformation Exercise Set 4.4 245 4.5 Similarity 249 Exercise Set 4.5 253 4.6 Application: Computer Graphics 255 Exercise Set 4.6 268 Review Exercises 270 Chapter Test 272 CHAPTER

5

235

Eigenvalues and Eigenvectors 275 5.1 Eigenvalues and Eigenvectors 276 Exercise Set 5.1 285 5.2 Diagonalization 287 Exercise Set 5.2 298 5.3 Application: Systems of Linear Differential Equations Exercise Set 5.3 309

300

vii

Revised Pages

viii

Contents

5.4 Application: Markov Chains Exercise Set 5.4 315 Review Exercises 316 Chapter Test 318 CHAPTER

6

Inner Product Spaces

310

321

6.1 The Dot Product on ⺢n 323 Exercise Set 6.1 331 6.2 Inner Product Spaces 333 Exercise Set 6.2 341 6.3 Orthonormal Bases 342 Exercise Set 6.3 352 6.4 Orthogonal Complements 355 Exercise Set 6.4 364 6.5 Application: Least Squares Approximation 366 Exercise Set 6.5 375 6.6 Diagonalization of Symmetric Matrices 377 Exercise Set 6.6 383 6.7 Application: Quadratic Forms 385 Exercise Set 6.7 392 6.8 Application: Singular Value Decomposition 392 Exercise Set 6.8 403 Review Exercises 404 Chapter Test 406

Appendix 409 Answers to Odd-Numbered Exercises 440 Index 479

Revised Pages

Preface

Introduction to Linear Algebra with Applications is an introductory text targeted to second-year or advanced first-year undergraduate students. The organization of this text is motivated by what our experience tells us are the essential concepts that students should master in a one-semester undergraduate linear algebra course. The centerpiece of our philosophy regarding the presentation of the material is that each topic should be fully developed before the reader moves onto the next. In addition, there should be a natural connection between topics. We take great care to meet both of these objectives. This allows us to stay on task so that each topic can be covered with the depth required before progression to the next logical one. As a result, the reader is prepared for each new unit, and there is no need to repeat a concept in a subsequent chapter when it is utilized. Linear algebra is taken early in an undergraduate curriculum and yet offers the opportunity to introduce the importance of abstraction, not only in mathematics, but in many other areas where linear algebra is used. Our approach is to take advantage of this opportunity by presenting abstract vector spaces as early as possible. Throughout the text, we are mindful of the difficulties that students at this level have with abstraction and introduce new concepts first through examples which gently illustrate the idea. To motivate the definition of an abstract vector space, and the subtle concept of linear independence, we use addition and scalar multiplication of vectors in Euclidean space. We have strived to create a balance among computation, problem solving, and abstraction. This approach equips students with the necessary skills and problemsolving strategies in an abstract setting that allows for a greater understanding and appreciation for the numerous applications of the subject.

Pedagogical Features 1. Linear systems, matrix algebra, and determinants: We have given a streamlined, but complete, discussion of solving linear systems, matrix algebra, determinants, and their connection in Chap. 1. Computational techniques are introduced, and a number of theorems are proved. In this way, students can hone their problem-solving skills while beginning to develop a conceptual sense of the fundamental ideas of linear algebra. Determinants are no longer central in linear algebra, and we believe that in a course at this level, only a few lectures should be devoted to the topic. For this reason we have presented all the essentials on determinants, including their connection to linear systems and matrix inverses, in Chap. 1. This choice also enables us to use determinants as a theoretical tool throughout the text whenever the need arises.

ix

Revised Pages

x

Preface

2. Vectors: Vectors are introduced in Chap. 1, providing students with a familiar structure to work with as they start to explore the properties which are used later to characterize abstract vector spaces. 3. Linear independence: We have found that many students have difficulties with linear combinations and the concept of linear independence. These ideas are fundamental to linear algebra and are essential to almost every topic after linear systems. When students fail to grasp them, the full benefits of the course cannot be realized. In Introduction to Linear Algebra with Applications we have devoted Chap. 2 to a careful exposition of linear combinations and linear independence in the context of Euclidean space. This serves several purposes. First, by placing these concepts in a separate chapter their importance in linear algebra is highlighted. Second, an instructor using the text can give exclusive focus to these ideas before applying them to other problems and situations. Third, many of the important ramifications of linear combinations and linear independence are considered in the familiar territory of Euclidean spaces. 4. Euclidean spaces ⺢n : The Euclidean spaces and their algebraic properties are introduced in Chap. 2 and are used as a model for the abstract vectors spaces of Chap. 3. We have found that this approach works well for students with limited exposure to abstraction at this level. 5. Geometric representations: Whenever possible, we include figures with geometric representations and interpretations to illuminate the ideas being presented. 6. New concepts: New concepts are almost always introduced first through concrete examples. Formal definitions and theorems are then given to describe the situation in general. Additional examples are also provided to further develop the new idea and to explore it in greater depth. 7. True/false chapter tests: Each chapter ends with a true/false Chapter Test with approximately 40 questions. These questions are designed to help the student connect concepts and better understand the facts presented in the chapter. 8. Rigor and intuition: The approach we have taken attempts to strike a balance between presenting a rigorous development of linear algebra and building intuition. For example, we have chosen to omit the proofs for theorems that are not especially enlightening or that contain excessive computations. When a proof is not present, we include a motivating discussion describing the importance and use of the result and, if possible, the idea behind a proof. 9. Abstract vector spaces: We have positioned abstract vector spaces as a central topic within Introduction to Linear Algebra with Applications by placing their introduction as early as possible in Chap. 3. We do this to ensure that abstract vector spaces receive the appropriate emphasis. In a typical undergraduate mathematics curriculum, a course on linear algebra is the first time that students are exposed to this level of abstraction. However, Euclidean spaces still play a central role in our approach because of their familiarity and since they are so widely used. At the end of this chapter, we include a section on differential equations which underscores the need for the abstract theory of vector spaces.

Revised Pages

Preface

xi

10. Section fact summaries: Each section ends with a summary of the important facts and techniques established in the section. They are written, whenever possible, using nontechnical language and mostly without notation. These summaries are not meant to give a recapitulation of the details and formulas of the section; rather they are designed to give an overview of the main ideas of the section. Our intention is to help students to make connections between the concepts of the section as they survey the topic from a greater vantage point.

Applications Over the last few decades the applications of linear algebra have mushroomed, increasing not only in their numbers, but also in the diversity of fields to which they apply. Much of this growth is fueled by the power of modern computers and the availability of computer algebra systems used to carry out computations for problems involving large matrices. This impressive power has made linear algebra more relevant than ever. Recently, a consortium of mathematics educators has placed its importance, relative to applications, second only to calculus. Increasingly, universities are offering courses in linear algebra that are specifically geared toward its applications. Whether the intended audience is engineering, economics, science, or mathematics students, the abstract theory is essential to understanding how linear algebra is applied. In this text our introduction to the applications of linear algebra begins in Sec. 1.8 where we show how linear systems can be used to solve problems related to chemistry, engineering, economics, nutrition, and urban planning. However, many types of applications involve the more sophisticated concepts we develop in the text. These applications require the theoretical notions beyond the basic ideas of Chap. 1, and are presented at the end of a chapter as soon as the required background material is completed. Naturally, we have had to limit the number of applications considered. It is our hope that the topics we have chosen will interest the reader and lead to further inquiry. Specifically, in Sec. 4.6, we discuss the role of linear algebra in computer graphics. An introduction to the connection between differential equations and linear algebra is given in Secs. 3.5 and 5.3. Markov chains and quadratic forms are examined in Secs. 5.4 and 6.7, respectively. Section 6.5 focuses on the problem of finding approximate solutions to inconsistent linear systems. One of the most familiar applications here is the problem of finding the equation of a line that best fits a set of data points. Finally, in Sec. 6.8 we consider the singular value decomposition of a matrix and its application to data compression.

Technology Computations are an integral part of any introductory course in mathematics and certainly in linear algebra. To gain mastery of the techniques, we encourage the student to solve as many problems as possible by hand. That said, we also encourage the student to make appropriate use of the available technologies designed to facilitate, or to completely carry out, some of the more tedious computations. For example, it is quite reasonable to use a computer algebra system, such as MAPLE or MATLAB,

Revised Pages

xii

Preface

to row-reduce a large matrix. Our approach in Introduction to Linear Algebra with Applications is to assume that some form of technology will be used, but leave the choice to the individual instructor and student. We do not think that it is necessary to include discussions or exercises that use particular software. Note that this text can be used with or without technology. The degree to which it is used is left to the discretion of the instructor. From our own experience, we have found that Scientific Notebook,TM which offers a front end for LATEX along with menu access to the computer algebra system MuPad, allows the student to gain experience using technology to carry out computations while learning to write clear mathematics. Another option is to use LATEX for writing mathematics and a computer algebra system to perform computations. Another aspect of technology in linear algebra has to do with the accuracy and efficiency of computations. Some applications, such as those related to Internet search engines, involve very large matrices which require extensive processing. Moreover, the accuracy of the results can be affected by computer roundoff error. For example, using the characteristic equation to find the eigenvalues of a large matrix is not feasible. Overcoming problems of this kind is extremely important. The field of study known as numerical linear algebra is an area of vibrant research for both software engineers and applied mathematicians who are concerned with developing practical solutions. In our text, the fundamental concepts of linear algebra are introduced using simple examples. However, students should be made aware of the computational difficulties that arise when extending these ideas beyond the small matrices used in the illustrations.

Other Features 1. Chapter openers: The opening remarks for each chapter describe an application that is directly related to the material in the chapter. These provide additional motivation and emphasize the relevance of the material that is about to be covered. 2. Writing style: The writing style is clear, engaging, and easy to follow. Important new concepts are first introduced with examples to help develop the reader’s intuition. We limit the use of jargon and provide explanations that are as readerfriendly as possible. Every explanation is crafted with the student in mind. Introduction to Linear Algebra with Applications is specifically designed to be a readable text from which a student can learn the fundamental concepts in linear algebra. 3. Exercise sets: Exercise sets are organized with routine exercises at the beginning and the more difficult problems toward the end. There is a mix of computational and theoretical exercises with some requiring proof. The early portion of each exercise set tests the student’s ability to apply the basic concepts. These exercises are primarily computational, and their solutions follow from the worked examples in the section. The latter portion of each exercise set extends the concepts and techniques by asking the student to construct complete arguments. 4. Review exercise sets: The review exercise sets are organized as sample exams with 10 exercises. These exercises tend to have multiple parts, which connect the various techniques and concepts presented in the text. At least one problem in each of these sets presents a new idea in the context of the material of the chapter.

Revised Pages

Preface

xiii

5. Length: The length of the text reflects the fact that it is specifically designed for a one-semester course in linear algebra at the undergraduate level. 6. Appendix: The appendix contains background material on the algebra of sets, functions, techniques of proof, and mathematical induction. With this feature, the instructor is able to cover, as needed, topics that are typically included in a Bridge Course to higher mathematics.

Course Outline The topics we have chosen for Introduction to Linear Algebra with Applications closely follow those commonly covered in a first introductory course. The order in which we present these topics reflects our approach and preferences for emphasis. Nevertheless, we have written the text to be flexible, allowing for some permutations of the order of topics without any loss of consistency. In Chap. 1 we present all the basic material on linear systems, matrix algebra, determinants, elementary matrices, and the LU decomposition. Chap. 2 is entirely devoted to a careful exposition of linear combinations and linear independence in ⺢n . We have found that many students have difficulty with these essential concepts. The addition of this chapter gives us the opportunity to develop all the important ideas in a familiar setting. As mentioned earlier, to emphasize the importance of abstract vector spaces, we have positioned their introduction as early as possible in Chap. 3. Also, in Chap. 3 is a discussion of subspaces, bases, and coordinates. Linear transformations between vector spaces are the subject of Chap. 4. We give descriptions of the null space and range of a linear transformation at the beginning of the chapter, and later we show that every finite dimensional vector space, of dimension n, is isomorphic to ⺢n . Also, in Chap. 4 we introduce the four fundamental subspaces of a matrix and discuss the action of an m × n matrix on a vector in ⺢n . Chap. 5 is concerned with eigenvalues and eigenvectors. An abundance of examples are given to illustrate the techniques of computing eigenvalues and finding the corresponding eigenvectors. We discuss the algebraic and geometric multiplicities of eigenvalues and give criteria for when a square matrix is diagonalizable. In Chap. 6, using ⺢n as a model, we show how a geometry can be defined on a vector space by means of an inner product. We also give a description of the Gram-Schmidt process used to find an orthonormal basis for an inner product space and present material on orthogonal complements. At the end of this chapter we discuss the singular value decomposition of an m × n matrix. The Appendix contains a brief summary of some topics found in a Bridge Course to higher mathematics. Here we include material on the algebra of sets, functions, techniques of proof, and mathematical induction. Application sections are placed at the end of chapters as soon as the requisite background material has been covered.

Supplements 1. Instructor solutions manual: This manual contains detailed solutions to all exercises. 2. Student solutions manual: This manual contains detailed solutions to oddnumbered exercises.

xiv

Preface

3. Text website www.mhhe.com/defranza: This website accompanies the text and is available for both students and their instructors. Students will be able to access self-assessment quizzes and extra examples for each section and end of chapter cumulative quizzes. In addition to these assets, instructors will be able to access additional quizzes, sample exams, the end of chapter true/false tests, and the Instructor’s Solutions Manual.

Acknowledgments We would like to give our heartfelt thanks to the many individuals who reviewed the manuscript at various stages of its development. Their thoughtful comments and excellent suggestions have helped us enormously with our efforts to realize our vision of a reader-friendly introductory text on linear algebra. We would also like to give special thanks to David Meel of Bowling Green State University, Bowling Green, Ohio, for his thorough review of the manuscript and insightful comments that have improved the exposition of the material in the text. We are also grateful to Ernie Stitzinger of North Carolina State University who had the tiring task of checking the complete manuscript for accuracy, including all the exercises. A very special thanks goes to our editors (and facilitators), Liz Covello (Sr. Sponsoring Editor), Michelle Driscoll (Developmental Editor), and Joyce Watters (Project Manager) who have helped us in more ways than we can name, from the inception of this project to its completion. On a personal level, we would like to thank our wives, Regan DeFranza and Robin Gagliardi, for their love and support; and our students at Saint Lawrence University and SUNY Canton who provided the motivation to write the text. Finally, we want to express our gratitude to the staff at McGraw-Hill Higher Education, Inc., for their work in taking our manuscript and producing the text.

Confirming Pages

Preface

.

List of Reviewers Marie Aratari, Oakland Community College Cik Azizah, Universiti Utara Malaysia (UUM) Przcmyslaw Bogacki, Old Dominion University Rita Chattopadhyay, Eastern Michigan University Eugene Don, Queens College Lou Giannini, Curtin University of Technology Gregory Gibson, North Carolina A&T University Mark Gockenback, Michigan Technological University Dr. Leong Wah June, Universiti Putra Malaysia Cerry Klein, University of Missouri–Columbia Kevin Knudson, Mississippi State University Hyungiun Ko, Yonsei University Jacob Kogan, University of Maryland–Baltimore County David Meel, Bowling Green State University Martin Nakashima, California State Poly University–Pomona Eugene Spiegel, University of Connecticut–Storrs Dr. Hajar Sulaiman, Universiti Sains Malaysia (USM) Gnana Bhaskar Tenali, Florida Institute of Technology–Melbourne Peter Wolfe, University of Maryland–College Park

xv

Revised Pages

To The Student

You are probably taking this course early in your undergraduate studies after two or three semesters of calculus, and most likely in your second year. Like calculus, linear algebra is a subject with elegant theory and many diverse applications. However, in this course you will be exposed to abstraction at a much higher level. To help with this transition, some colleges and universities offer a Bridge Course to Higher Mathematics. If you have not already taken such a course, this may likely be the first mathematics course where you will be expected to read and understand proofs of theorems, provide proofs of results as part of the exercise sets, and apply the concepts presented. All this is in the context of a specific body of knowledge. If you approach this task with an open mind and a willingness to read the text, some parts perhaps more than once, it will be an exciting and rewarding experience. Whether you are taking this course as part of a mathematics major or because linear algebra is applied in your specific area of study, a clear understanding of the theory is essential for applying the concepts of linear algebra to mathematics or other fields of science. The solved examples and exercises in the text are designed to prepare you for the types of problems you can expect to see in this course and other more advanced courses in mathematics. The organization of the material is based on our philosophy that each topic should be fully developed before readers move onto the next. The image of a tree on the front cover of the text is a metaphor for this learning strategy. It is particularly applicable to the study of mathematics. The trunk of the tree represents the material that forms the basis for everything that comes afterward. In our text, this material is contained in Chaps. 1 through 4. All other branches of the tree, representing more advanced topics and applications, extend from the foundational material of the trunk or from the ancillary material of the intervening branches. We have specifically designed our text so that you can read it and learn the concepts of linear algebra in a sequential and thorough manner. If you remain committed to learning this beautiful subject, the rewards will be significant in other courses you may take, and in your professional career. Good luck! Jim DeFranza [email protected] Dan Gagliardi [email protected]

xvi

Revised Pages

Applications Index

Aircraft Design, 199 Astronomy, 61 Average Global Temperatures, 371 Balancing Chemical Equations, 1, 79, 84 Carbon Emissions, 321 Cellular Phone Subscribers, 86 Computer Graphics, 199, 255 Cumulative HIV Infections, 376 Data Compression, 401 Differential Equations, 185 Digital Transmissions, 127, 402 Economics, 82, 86, 87 Electrical Networks, 88, 191 Engineering, 185, 191 Epidemics, 87, 315 Exponential Growth and Decay, 186 Fitting Data with a Curve, 11, 61, 366, 372, 376 Fourier Approximation, 373 Infant Mortality Rates, 376 Least Squares Approximation, 321, 366 Linear Regression, 371 Markov Chains, 275, 310 Mass Transit Systems, 315 Mixing, 307, 310 National Health Care, 86 Networks, 79, 85, 89 Nutrition, 81, 85 Photosynthesis, 1 Plant Breading, 315 Population Distribution, 87, 275, 315 Powers of a Matrix, 287 Predator–Prey Models, 300 Projections, 265, 343, 345, 361, 365 Quadratic Forms, 385 Rotation of Axes, 385 Search Engines, 276

xvii

Revised Pages

xviii

Applications Index

Signal Processing, 93 Singular Value Decomposition, 392 Systems of Differential Equations, 300 Thermal Equilibrium, 88, 310 Vibrating Spring, 192, 194 World Hydroelectricity Use, 376 World Population, 376

Confirming Pages

CHAPTER

Systems of Linear Equations and Matrices CHAPTER OUTLINE

1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8

Systems of Linear Equations 2 Matrices and Elementary Row Operations 14 Matrix Algebra 26 The Inverse of a Square Matrix 39 Matrix Equations 48 Determinants 54 Elementary Matrices and LU Factorization 68 Applications of Systems of Linear Equations 79

n the process of photosynthesis solar energy is converted into forms that are used by living organisms. The chemical reaction that occurs in the leaves of plants converts carbon dioxide and water to carbohydrates with the release of oxygen. The chemical equation of the reaction takes the form

I

aCO2 + bH2 O → cO2 + dC6 H12 O6 where a, b, c, and d are some positive whole numbers. The law of conservation of mass states that the total mass of all substances present before and after a chemical reaction remains the same. That is, atoms are neither created nor destroyed Photograph by Jan Smith/RF in a chemical reaction, so chemical equations must be balanced. To balance the photosynthesis reaction equation, the same number of carbon atoms must appear on both sides of the equation, so a = 6d The same number of oxygen atoms must appear on both sides, so 2a + b = 2c + 6d and the same number of hydrogen atoms must appear on both sides, so 2b = 12d 1

Confirming Pages

Chapter 1 Systems of Linear Equations and Matrices

This gives us the system of three linear equations in four variables   − 6d = 0  a 2a + b − 2c − 6d = 0   2b − 12d = 0 Any positive integers a, b, c, and d that satisfy all three equations are a solution to this system which balances the chemical equation. For example, a = 6, b = 6, c = 6, and d = 1 balances the equation.

Many diverse applications are modeled by systems of equations. Systems of equations are also important in mathematics and in particular in linear algebra. In this chapter we develop systematic methods for solving systems of linear equations.

1.1

ß

2

Systems of Linear Equations As the introductory example illustrates, many naturally occurring processes are modeled using more than one equation and can require many equations in many variables. For another example, models of the economy contain thousands of equations and thousands of variables. To develop this idea, consider the set of equations  2x − y = 2 x + 2y = 6 which is a system of two equations in the common variables x and y. A solution to this system consists of values for x and y that simultaneously satisfy each equation. In this example we proceed by solving the first equation for y, so that y = 2x − 2 To find the solution, substitute y = 2x − 2 into the second equation to obtain x + 2(2x − 2) = 6

and solving for x gives

x=2

Substituting x = 2 back into the first equation yields 2(2) − y = 2, so that y = 2. Therefore the unique solution to the system is x = 2, y = 2. Since both of these equations represent straight lines, a solution exists provided that the lines intersect. These lines intersect at the unique point (2, 2), as shown in Fig. 1(a). A system of equations is consistent if there is at least one solution to the system. If there are no solutions, the system is inconsistent. In the case of systems of two linear equations with two variables, there are three possibilities: 1. The two lines have different slopes and hence intersect at a unique point, as shown in Fig. 1(a). 2. The two lines are identical (one equation is a nonzero multiple of the other), so there are infinitely many solutions, as shown in Fig. 1(b).

Confirming Pages

3

1.1 Systems of Linear Equations x + 2y = 6

y 5

2x − y = 2

y

y

x+y =1 5

5

(2, 2) ⫺5

x−y =1 5

⫺5

(a)

−x + y = 1

x

⫺5

5

⫺5

x

2x + 2y = 2

(b)

⫺5

5

x

⫺5

(c)

Figure 1

3. The two lines are parallel (have the same slope) and do not intersect, so the system is inconsistent, as shown in Fig. 1(c). When we are dealing with many variables, the standard method of representing linear equations is to affix subscripts to coefficients and variables. A linear equation in the n variables x1 , x2 , . . . , xn is an equation of the form a1 x1 + a2 x2 + · · · + an xn = b To represent a system of m linear equations in n variables, two subscripts are used for each coefficient. The first subscript indicates the equation number while the second specifies the term of the equation.

DEFINITION 1

System of Linear Equations A system of m linear equations in n variables, or a linear system, is a collection of equations of the form  a11 x1 + a12 x2 + · · · + a1n xn = b1       a21 x1 + a22 x2 + · · · + a2n xn = b2 a31 x1 + a32 x2 + · · · + a3n xn = b3  .. .. .. .. ..   . . . . .    am1 x1 + am2 x2 + · · · + amn xn = bm This is also referred to as an m × n linear system.

For example, the collection of equations   −2x1 + 3x2 + x3 − x4 = −2 x1 + x3 − 4x4 = 1  3x1 − x2 − x4 = 3 is a linear system of three equations in four variables, or a 3 × 4 linear system. A solution to a linear system with n variables is an ordered sequence (s1 , s2 , . . . , sn ) such that each equation is satisfied for x1 = s1 , x2 = s2 , . . . , xn = sn . The general solution or solution set is the set of all possible solutions.

Confirming Pages

4

Chapter 1 Systems of Linear Equations and Matrices

The Elimination Method The elimination method, also called Gaussian elimination, is an algorithm used to solve linear systems. To describe this algorithm, we first introduce the triangular form of a linear system. An m × n linear system is in triangular form provided that the coefficients aij = 0 whenever i > j. In this case we refer to the linear system as a triangular system. Two examples of triangular systems are    x1 − 2x2 + x3 = −1  x1 + x2 − x3 − x4 = 2 x2 − 3x3 = 5 x2 − x3 − 2x4 = 1 and   x3 = 2 2x3 − x4 = 3 When a linear system is in triangular form, then the solution set can be obtained using a technique called back substitution. To illustrate this technique, consider the linear system given by   x1 − 2x2 + x3 = −1 x2 − 3x3 = 5  x3 = 2 From the last equation we see that x3 = 2. Substituting this into the second equation, we obtain x2 − 3(2) = 5, so x2 = 11. Finally, using these values in the first equation, we have x1 − 2(11) + 2 = −1, so x1 = 19. The solution is also written as (19, 11, 2).

DEFINITION 2

Equivalent Linear Systems Two linear systems are equivalent if they have the same solutions

For example, the system

  x1 − 2x2 + x3 = −1 2x1 − 3x2 − x3 = 3  x1 − 2x2 + 2x3 = 1

has the unique solution x1 = 19, x2 = 11, and x3 = 2, so the linear systems    x1 − 2x2 + x3 = −1  x1 − 2x2 + x3 = −1 x2 − 3x3 = 5 2x1 − 3x2 − x3 = 3 and   x3 = 2 x1 − 2x2 + 2x3 = 1 are equivalent. The next theorem gives three operations that transform a linear system into an equivalent system, and together they can be used to convert any linear system to an equivalent system in triangular form.

Confirming Pages

1.1 Systems of Linear Equations

THEOREM 1

 a11 x1 + a12 x2 + · · · + a1n xn = b1     a   21 x1 + a22 x2 + · · · + a2n xn = b2 a31 x1 + a32 x2 + · · · + a3n xn = b3  .. .. .. .. ..   . . . . .    am1 x1 + am2 x2 + · · · + amn xn = bm

Let

be a linear system. Performing any one of the following operations on the linear system produces an equivalent linear system. 1. Interchanging any two equations. 2. Multiplying any equation by a nonzero constant. 3. Adding a multiple of one equation to another. Proof Interchanging any two equations does not change the solution of the linear system and therefore yields an equivalent system. If equation i is multiplied by a constant c = 0, then equation i of the new system is cai1 x1 + cai2 x2 + · · · + cain xn = cbi Let (s1 , s2 , . . . , sn ) be a solution to the original system. Since ai1 s1 + ai2 s2 + · · · + ain sn = bi ,

then

cai1 s1 + cai2 s2 + · · · + cain sn = cbi

Hence (s1 , s2 , . . . , sn ) is a solution of the new linear system. Consequently, the systems are equivalent. For part (3) of the theorem, consider the new system obtained by adding c times equation i to equation j of the original system. Thus, equation j of the new system becomes (cai1 + aj 1 )x1 + (cai2 + aj 2 )x2 + · · · + (cain + aj n )xn = cbi + bj or equivalently, c(ai1 x1 + ai2 x2 + · · · + ain xn ) + (aj 1 x1 + aj 2 x2 + · · · + aj n xn ) = cbi + bj Now let (s1 , s2 , . . . , sn ) be a solution for the original system. Then ai1 s1 + ai2 s2 + · · · + ain sn = bi

and

aj 1 s1 + aj 2 s2 + · · · + aj n sn = bj

Therefore, c(ai1 s1 + ai2 s2 + · · · + ain sn ) + (aj 1 s1 + aj 2 s2 + · · · + aj n sn ) = cbi + bj so that (s1 , s2 , . . . , sn ) is a solution of the modified system and the systems are equivalent.

5

Confirming Pages

6

Chapter 1 Systems of Linear Equations and Matrices

EXAMPLE 1

Solution

Use the elimination method to solve the linear system.  x+y=1 −x + y = 1 Adding the first equation to the second gives the equivalent system  x + y =1 2y = 2 From the second equation, we have y = 1. Using back substitution gives x = 0. The graphs of both systems are shown in Fig. 2. Notice that the solution is the same in both, but that adding the first equation to the second rotates the line −x + y = 1 about the point of intersection.

y

x+y =1 5

y

−x + y = 1

x+y =1 5

y=1

(1, 0) ⫺5

−x + y = 1

5

x

⫺5

⫺5

5

x

⫺5

(a)

(b) Figure 2

Converting a linear system to triangular form often requires many steps. Moreover, the operations used to convert one linear system to another are not unique and may not be apparent on inspection. To articulate this process, the notation, for example, (−2) · E1 + E3 −→ E3 will mean add −2 times equation 1 to equation 3, and replace equation 3 with the result. The notation Ei ↔ Ej will be used to indicate that equation i and equation j are interchanged.

EXAMPLE 2

Solution

Solve the linear system.

  x+y+ z= 4 −x − y + z = −2  2x − y + 2z = 2

To convert the system to an equivalent triangular system, we first eliminate the variable x in the second and third equations to obtain

Confirming Pages

1.1 Systems of Linear Equations

  x+y+ z= 4 −x − y + z = −2  2x − y + 2z = 2

E1 + E2 → E2 − 2E1 + E3 → E3

7

 x + y + z = 4 2z = 2 −→  − 3y = −6

Interchanging the second and third equations gives the triangular linear system   x + y + z = 4 x + y + z = 4 2z = 2 E2 ↔ E3 − 3y = −6 −→   − 3y = −6 2z = 2 Using back substitution, we have z = 1, y = 2, and x = 4 − y − z = 1. Therefore, the system is consistent with the unique solution (1, 2, 1). Recall from solid geometry that the graph of an equation of the form ax + by + cz = d is a plane in three-dimensional space. Hence, the unique solution to the linear system of Example 2 is the point of intersection of three planes, as shown in Fig. 3(a). For another perspective on this, shown in Fig. 3(b) are the lines of the pairwise intersections of the three planes. These lines intersect at a point that is the solution to the 3 × 3 linear system.

(1, 2, 1)

(a)

(b)

Figure 3

Similar to the 2 × 2 case, the geometry of Euclidean space helps us better understand the possibilities for the general solution of a linear system of three equations in three variables. In particular, the linear system can have a unique solution if the three planes all intersect at a point, as illustrated by Example 2. Alternatively, a 3 × 3 system can have infinitely many solutions if 1. 2. 3. For

The three planes are all the same. The three planes intersect in a line (like the pages of a book). Two of the planes are the same with a third plane intersecting them in a line. example, the linear system given by  −y +z=0 y =0  z=0

Confirming Pages

8

Chapter 1 Systems of Linear Equations and Matrices

represents three planes whose intersection is the x axis. That is, z = 0 is the xy plane, y = 0 is the xz plane, and y = z is the plane that cuts through the x axis at a 45◦ angle. Finally, there are two cases in which a 3 × 3 linear system has no solutions. First, the linear system has no solutions if at least one of the planes is parallel to, but not the same as, the others. Certainly, when all three planes are parallel, the system has no solutions, as illustrated by the linear system    z = 0 z = 1   z = 2 Figure 4

EXAMPLE 3

Solution

Also, a 3 × 3 linear system has no solutions, if the lines of the pairwise intersections of the planes are parallel, but not the same, as shown in Fig. 4. From the previous discussion we see that a 3 × 3 linear system, like a 2 × 2 linear system, has no solutions, has a unique solution, or has infinitely many solutions. We will see in Sec. 1.4 that this is the case for linear systems of any size. In Example 3 we consider a linear system with four variables. Of course the geometric reasoning above cannot be applied to the new situation directly, but provides the motivation for understanding the many possibilities for the solutions to linear systems with several variables.

Solve the linear system.

  4x1 − 8x2 − 3x3 + 2x4 = 13 3x1 − 4x2 − x3 − 3x4 = 5  2x1 − 4x2 − 2x3 + 2x4 = 6

Since every term of the third equation can be divided evenly by 2, we multiply the third equation by 12 . After we do so, the coefficient of x1 is 1. We then interchange the first and third equations, obtaining    4x1 − 8x2 − 3x3 + 2x4 = 13  4x1 − 8x2 − 3x3 + 2x4 = 13 3x1 − 4x2 − x3 − 3x4 = 5 12 E3 → E3 −→ 3x1 − 4x2 − x3 − 3x4 = 5   2x1 − 4x2 − 2x3 + 2x4 = 6 x1 − 2x2 − x3 + x4 = 3   x1 − 2x2 − x3 + x4 = 3 E1 ↔ E3 −→ 3x1 − 4x2 − x3 − 3x4 = 5  4x1 − 8x2 − 3x3 + 2x4 = 13 Next using the operations −3E1 + E2 → E2 and −4E1 + E3 → E3 , we obtain the linear system   x1 − 2x2 − x3 + x4 = 3 2x2 + 2x3 − 6x4 = −4  x3 − 2x4 = 1

Confirming Pages

1.1 Systems of Linear Equations

9

which is an equivalent system in triangular form. Using back substitution, the general solution is x3 = 2x4 + 1 x2 = x4 − 3 x1 = 3x4 − 2 with x4 free to assume any real number. It is common in this case to replace x4 with the parameter t. The general solution can now be written as S = {(3t − 2, t − 3, 2t + 1, t) | t ∈ ⺢} and is called a one-parameter family of solutions. The reader can check that x1 = 3t − 2, x2 = t − 3, x3 = 2t + 1, and x4 = t is a solution for any t by substituting these values in the original equations. A particular solution can be obtained by letting t be a specific value. For example, if t = 0, then a particular solution is (−2, −3, 1, 0).

In Example 3, the variable x4 can assume any real number, giving infinitely many solutions for the linear system. In this case we call x4 a free variable. When a linear system has infinitely many solutions, there can be more than one free variable. In this case, the solution set is an r-parameter family of solutions where r is equal to the number of free variables.

EXAMPLE 4

Solve the linear system.  

x1 − x2 − 2x3 − 2x4 − 2x5 = 3 3x1 − 2x2 − 2x3 − 2x4 − 2x5 = −1  −3x1 + 2x2 + x3 + x4 − x5 = −1

Solution

After performing the operations E3 + E2 → E3 followed by E2 − 3E1 → E2 , we have the equivalent system  3  x1 − x2 − 2x3 − 2x4 − 2x5 = x2 + 4x3 + 4x4 + 4x5 = − 10  − x3 − x4 − 3x5 = − 2 The variables x4 and x5 are both free variables, so to write the solution, let x4 = s and x5 = t. From the third equation, we have x3 = 2 − x4 − 3x5 = 2 − s − 3t Substitution into the second equation gives x2 = −10 − 4x3 − 4x4 − 4x5 = −10 − 4(2 − s − 3t) − 4s − 4t = −18 + 8t

Confirming Pages

10

Chapter 1 Systems of Linear Equations and Matrices

Finally, substitution into the first equation gives x1 = 3 + x2 + 2x3 + 2x4 + 2x5 = 3 + (−18 + 8t) + 2(2 − s − 3t) + 2s + 2t = −11 + 4t The two-parameter solution set is therefore given by S = {(−11 + 4t, −18 + 8t, 2 − s − 3t, s, t) | s, t ∈ ⺢} Particular solutions for s = t = 0 and s = 0, t = 1 are (−11, −18, 2, 0, 0) and (−7, −10, −1, 0, 1), respectively.

EXAMPLE 5

Solution

Solve the linear system.

 x1 − x2 + 2x3 = 5    2x1 + x2 =2 x + 8x − x =3  1 2 3   −x1 − 5x2 − 12x3 = 4

To convert the linear system to an equivalent triangular system, we will eliminate the first terms in equations 2 through 4, and then the second terms in equations 3 and 4, and then finally the third term in the fourth equation. This is accomplished by using the following operations.   x1 − x2 + 2x3 = 5 x1 − x2 + 2x3 = 5     − 2E1 + E2 → E2   2x1 + x2 3x2 − 4x3 = − 8 =2 −E1 + E3 → E3 → x1 + 8x2 − x3 = 3 9x2 − 3x3 = − 2     E1 + E4 → E4   −x1 − 5x2 − 12x3 = 4 − 6x2 − 10x3 = 9  x1 − x2 + 2x3 = 5    3x2 − 4x3 = −8 −3E2 + E3 → E3 → 9x3 = 22 2E2 + E4 → E4    − 18x3 = −7  x1 − x2 + 2 x3 = 5    3x2 − 4 x3 = −8 2E3 + E4 → E4 → 9 x3 = 22    0 = −37 The last equation of the final system is an impossibility, so the original linear system is inconsistent and has no solution.

In the previous examples the algorithm for converting a linear system to triangular form is based on using a leading variable in an equation to eliminate the same variable in each equation below it. This process can always be used to convert any linear system to triangular form.

Confirming Pages

1.1 Systems of Linear Equations

EXAMPLE 6 Solution

11

Find the equation of the parabola that passes through the points (−1, 1), (2, −2), and (3, 1). Find the vertex of the parabola. The general form of a parabola is given by y = ax 2 + bx + c. Conditions on a, b, and c are imposed by substituting the given points into this equation. This gives 1 = a(−1)2 + b(−1) + c = a − b + c −2 = a(2)2 + b(2) + c = 4a + 2b + c 1 = a(3)2 + b(3) + c = 9a + 3b + c From these conditions we obtain the linear system   a− b+c= 1 4a + 2b + c = −2  9a + 3b + c = 1 First, with a as the leading variable, we use row operations to eliminate a from equations 2 and 3. In particular, we have    a− b+c= 1 a − b + c = 1 −4E1 + E2 → E2 4a + 2b + c = −2 6b − 3c = −6 → −9E1 + E3 → E3   9a + 3b + c = 1 12b − 8c = −8 Next, with b as the leading variable we, eliminate b from equation 3, so that   a − b + c = 1 a − b + c = 1 6b − 3c = −6 −2E2 + E3 → E3 6b − 3c = −6 →   12b − 8c = −8 − 2c = 4 Now, using back substitution on the last system gives c = −2, b = −2, and a = 1. Thus, the parabola we seek is

y

y = x 2 − 2x − 2

y = x 2 − 2x − 2

5

Completing the square gives the parabola in standard form (−1, 1)

(3, 1)

⫺5

5

(2, −2) (1, −3)

y = (x − 1)2 − 3

x

with vertex (1, −3), as shown in Fig. 5.

⫺5

Figure 5

Fact Summary 1. A m × n linear system has a unique solution, infinitely many solutions, or no solutions. 2. Interchanging any two equations in a linear system does not alter the set of solutions. 3. Multiplying any equation in a linear system by a nonzero constant does not alter the set of solutions.

Confirming Pages

12

Chapter 1 Systems of Linear Equations and Matrices

4. Replacing an equation in a linear system with the sum of the equation and a scalar multiple of another equation does not alter the set of solutions. 5. Every linear system can be reduced to an equivalent triangular linear system.

Exercise Set 1.1 Perform the operations −E1 + E2 → E2 and −2E1 + E3 → E3 , and write the new equivalent system. Solve the linear system. In Exercises 5–18, solve the linear system using the elimination method.

1. Consider the linear system   x1 − x2 − 2x3 = 3 −x1 + 2x2 + 3x3 = 1  2x1 − 2x2 − 2x3 = −2 Perform the operations E1 + E2 → E2 and −2E1 + E3 → E3 , and write the new equivalent system. Solve the linear system. 2. Consider the linear system   2x1 − 2x2 − x3 = −3 x1 − 3x2 + x3 = −2  x1 − 2x2 = 2 Perform the operations E1 ↔ E2 , −2E1 + E2 → E2 , −E1 + E3 → E3 , E2 ↔ E3 , and −4E2 + E3 → E3 , and write the new equivalent system. Solve the linear system. 3. Consider the linear system  x1 + 3x4 = 2    x1 + x2 + 4x4 = 3 2x + x  1 3 + 8x4 = 3   x1 + x2 + x3 + 6x4 = 2 Perform the operations −E1 + E2 → E2 , −2E1 + E3 → E3 , −E1 + E4 → E4 , −E2 + E4 → E4 , and −E3 + E4 → E4 , and write the new equivalent system. Solve the linear system. 4. Consider the linear system  + x3 = −2  x1 x1 + x2 + 4x3 = −1  2x1 + 2x3 + x4 = −1

 5.  6.  7.  8.  9.  10.

2x + 3y = −2 −2x = 0 x + 3y = −1 − y = −1 4x = 4 −3x + 2y = −3 2x + 3y = −1 x− y= 0 3x − 2y = 4 x − 23 y = 43 3x − 5y = 1 −x + 53 y = − 13

  − 3x − 2y + 2z = −2 − x − 3y + z = −3 11.  x − 2y + z = −2   12.

x + 3y + z = 2 − 2x + 2y − 4z = −1  − y + 3z = 1

Confirming Pages

1.1 Systems of Linear Equations

13.

14.

15. 16. 17. 18.

  −2x − 2y + 2z = 1 x + 5z = −1  3x + 2y + 3z = −2   −x + y + 4z = −1 3x − y + 2z = 2  2x − 2y − 8z = 2  3x1 + 4x2 + 3x3 = 0 3x1 − 4x2 + 3x3 = 4  −2x1 + x2 =2 3x1 − x2 + 2x3 = 1  x1 − 2x2 − 2x3 − x4 = −3 − 2x1 + x2 + x3 − 2x4 = −3  =1 2x1 + 2x2 − x3 − x2 + 3x4 = 2

In Exercises 19–22, solve for x, y, and z in terms of a, b, and c.  −2x + y = a 19. −3x + 2y = b  2x + 3y = a 20. x+ y=b   3x + y + 3z = a −x − z=b 21.  −x + 2y =c   −3x + 2y + z = a x− y− z=b 22.  x − y − 2z = c In Exercises 23–28, give restrictions on a, b, and c such that the linear system is consistent.  x − 2y = a 23. −2x + 4y = 2  −x + 3y = a 24. 2x − 6y = 3  x − 2y = a 25. −x + 2y = b  6x − 3y = a 26. −2x + y = b

13

  x − 2y + 4z = a 2x + y − z = b 27.  3x − y + 3z = c   x − y + 2z = a 2x + 4y − 3z = b 28.  4x + 2y + z = c In Exercises 29–32, determine the value of a that makes the system inconsistent.  x + y = −2 29. 2x + ay = 3  2x − y = 4 30. ax + 3y = 2  x− y=2 31. 3x − 3y = a  2x − y = a 32. 6x − 3y = a In Exercises 33–36, find an equation in the form y = ax 2 + bx + c for the parabola that passes through the three points. Find the vertex of the parabola. 33. (0, 0.25), (1, −1.75), (−1, 4.25) 34. (0, 2), (−3, −1), (0.5, 0.75) 35. (−0.5, −3.25), (1, 2), (2.3, 2.91) 36. (0, −2875), (1, −5675), (3, 5525) 37. Find the point where the three lines −x + y = 1, −6x + 5y = 3, and 12x + 5y = 39 intersect. Sketch the lines. 38. Find the point where the four lines 2x + y = 0, x + y = −1, 3x + y = 1, and 4x + y = 2 intersect. Sketch the lines. 39. Give an example of a 2 × 2 linear system that a. Has a unique solution b. Has infinitely many solutions c. Is inconsistent 40. Verify that if ad − bc = 0, then the system of equations  ax + by = x1 cx + dy = x2 has a unique solution.

Confirming Pages

14

Chapter 1 Systems of Linear Equations and Matrices

41. Consider the system  x1 − x2 + 3x3 − x4 = 1 x2 − x3 + 2x4 = 2 a. Describe the solution set where the variables x3 and x4 are free. b. Describe the solution set where the variables x2 and x4 are free. 42. Consider the system   x1 − x2 + x3 − x4 + x5 = 1 x2 − x4 − x5 = −1  x3 − 2x4 + 3x5 = 2 a. Describe the solution set where the variables x4 and x5 are free. b. Describe the solution set where the variables x3 and x5 are free. ß

1.2

43. Determine the values of k such that the linear system  9x + ky = 9 kx + y = −3 has a. No solutions b. Infinitely many solutions c. A unique solution 44. Determine the values of k such that the linear system   kx + y + z = 0 x + ky + z = 0  x + y + kz = 0 has a. A unique solution b. A one-parameter family of solutions c. A two-parameter family of solutions

Matrices and Elementary Row Operations In Sec. 1.1 we saw that converting a linear system to an equivalent triangular system provides an algorithm for solving the linear system. The algorithm can be streamlined by introducing matrices to represent linear systems.

DEFINITION 1

Matrix

An m × n matrix is an array of numbers with m rows and n columns.

For example, the array of numbers   2 3 −1 4  3 1 0 −2  −2 4 1 3 is a 3 × 4 matrix. When solving a linear system by the elimination method, only the coefficients of the variables and the constants on the right-hand side are needed to find the solution. The variables are placeholders. Utilizing the structure of a matrix, we can record the coefficients and the constants by using the columns as placeholders for the variables. For example, the coefficients and constants of the linear system  −4x1 + 2x2 − 3x4 = 11    2x1 − x2 − 4x3 + 2x4 = − 3  3x2 − x4 = 0   −2x1 + x4 = 4

Confirming Pages

1.2 Matrices and Elementary Row Operations

15

can be recorded in matrix form as 

−4  2   0 −2

2 0 −3 −1 −4 2 3 0 −1 0 0 1

 11 −3   0  4

This matrix is called the augmented matrix of the linear system. Notice that for an m × n linear system the augmented matrix is m × (n + 1). The augmented matrix with the last column deleted  −4 2 0 −3  2 −1 −4 2     0 3 0 −1  −2 0 0 1 

is called the coefficient matrix. Notice that we always use a 0 to record any missing terms. The method of elimination on a linear system is equivalent to performing similar operations on the rows of the corresponding augmented matrix. The relationship is illustrated below: Linear system   x+y− z= 1 2x − y + z = −1  −x − y + 3z = 2

Corresponding augmented matrix   1 1 −1 1  2 −1 1 −1  −1 −1 3 2

Using the operations −2E1 + E2 → E2 Using the operations −2R1 + R2 → R2 and E1 + E3 → E3 , we obtain the equiv- and R1 + R3 → R3 , we obtain the equivalent triangular system alent augmented matrix    1 1 −1 1 x + y − z= 1  0 −3 3 −3  − 3y + 3z = −3  0 0 2 3 2z = 3 The notation used to describe the operations on an augmented matrix is similar to the notation we introduced for equations. In the example above, −2R1 + R2 −→ R2 means replace row 2 with −2 times row 1 plus row 2. Analogous to the triangular form of a linear system, a matrix is in triangular form provided that the first nonzero entry for each row of the matrix is to the right of the first nonzero entry in the row above it. The next theorem is a restatement of Theorem 1 of Sec. 1.1, in terms of operations on the rows of an augmented matrix.

Confirming Pages

16

Chapter 1 Systems of Linear Equations and Matrices

THEOREM 2

Any one of the following operations performed on the augmented matrix, corresponding to a linear system, produces an augmented matrix corresponding to an equivalent linear system. 1. Interchanging any two rows. 2. Multiplying any row by a nonzero constant. 3. Adding a multiple of one row to another.

Solving Linear Systems with Augmented Matrices The operations in Theorem 2 are called row operations. An m × n matrix A is called row equivalent to an m × n matrix B if B can be obtained from A by a sequence of row operations. The following steps summarize a process for solving a linear system. 1. 2. 3. 4.

Write the augmented matrix of the linear system. Use row operations to reduce the augmented matrix to triangular form. Interpret the final matrix as a linear system (which is equivalent to the original). Use back substitution to write the solution. Example 1 illustrates how we can carry out steps 3 and 4.

EXAMPLE 1

Solution

Given the augmented matrix, find the solution of the corresponding linear system.       1 0 0 1 1 0 0 0 5 1 2 1 −1 1 0 1  a.  0 1 0 2  b.  0 1 −1 0 1  c.  0 3 −1 0 0 1 3 0 0 0 1 3 0 0 0 0 0 a. Reading directly from the augmented matrix, we have x3 = 3, x2 = 2, and x1 = 1. So the system is consistent and has a unique solution. b. In this case the solution to the linear system is x4 = 3, x2 = 1 + x3 , and x1 = 5. So the variable x3 is free, and the general solution is S = {(5, 1 + t, t, 3) | t ∈ ⺢}. c. The augmented matrix is equivalent to the linear system  x1 + 2x2 + x3 − x4 = 1 3x2 − x3 =1 Using back substitution, we have 1 x2 = (1 + x3 ) and 3

x1 = 1 − 2x2 − x3 + x4 =

1 5 − x3 + x4 3 3

Confirming Pages

1.2 Matrices and Elementary Row Operations

17

So the variables x3 and x4 are free, and the two-parameter solution set is given by

    1 s 1 5s − + t, + , s, t  s, t ∈ ⺢ S= 3 3 3 3

EXAMPLE 2

Solution

Write the augmented matrix and solve the linear system.  x − 6y − 4z = −5  2x − 10y − 9z = −4  − x + 6y + 5z = 3 To solve this system, we write the augmented matrix   1 −6 −4 −5  2 −10 −9 −4  −1 6 5 3 where we have shaded the entries to eliminate. Using the procedure described above, the augmented matrix is reduced to triangular form as follows:     1 −6 −4 −5 1 −6 −4 −5 −2R1 + R2 → R2  2 −10 −9 −4  2 −1 6  −→  0 R1 + R3 → R3 −1 6 5 0 0 1 −2 3 The equivalent triangular linear system is   x − 6y − 4z = −5 2y − z = 6  z = −2 which has the solution x = −1, y = 2, and z = −2.

Echelon Form of a Matrix In Example 2, the final augmented matrix  1 −6 −4  0 2 −1 0 0 1

 −5 6  −2

is in row echelon form. The general structure of a matrix in row echelon form is shown in Fig. 1. The height of each step is one row, and the first nonzero term in a row, denoted in Fig. 1 by *, is to the right of the first nonzero term in the previous row. All the terms below the stairs are 0.

Confirming Pages

18

Chapter 1 Systems of Linear Equations and Matrices

0 .... 0 * * * *

...

...

0 Figure 1

Although, the height of each step in Fig. 1 is one row, a step may extend over several columns. The leading nonzero term in each row is called a pivot element. The matrix is in reduced row echelon form if, in addition, each pivot is a 1 and all other entries in this column are 0. For example, the reduced row echelon form of the matrix   1 −6 −4 −5  0 2 −1 6  0 0 1 −2 is given by

 1 0 0 −1  0 1 0 2  0 0 1 −2 

Transforming a matrix in row echelon form to reduced row echelon form in effect incorporates back substitution as row operations on the matrix. If we read from the last matrix above, the solution to the corresponding linear system is, as before, x = −1, y = 2, and z = −2. Here are three additional matrices that are in reduced row echelon form       1 0 0 0 1 −2 0 1 −1 1 0 0 −1  0 1 0 0     0  0 1 0 0 1 −1 2  2   0 0 1 0  0 0 0 0 0 0 0 1 4 0 0 0 1 and two that are not in reduced row echelon form     1 −2 1 0 1 3 2 1  0  0 1 −5 6  0 2 1  0 0 0 3 0 1 4 1 In general, for any m × n matrix in reduced row echelon form, the pivot entries correspond to dependent variables, and the nonpivot entries correspond to independent or free variables. We summarize the previous discussion on row echelon form in the next definition.

Confirming Pages

1.2 Matrices and Elementary Row Operations

DEFINITION 2

19

Echelon Form An m × n matrix is in row echelon form if 1. Every row with all 0 entries is below every row with nonzero entries. 2. If rows 1, 2, . . . , k are the rows with nonzero entries and if the leading nonzero entry (pivot) in row i occurs in column ci , for 1, 2, . . . , k, then c1 < c 2 < · · · < c k . The matrix is in reduced row echelon form if, in addition, 3. The first nonzero entry of each row is a 1. 4. Each column that contains a pivot has all other entries 0. The process of transforming a matrix to reduced row echelon form is called Gauss-Jordan elimination.

EXAMPLE 3

Solution

Solve the linear system by transforming the augmented matrix to reduced row echelon form.  x1 − x2 − 2x3 + x4 = 0    2x1 − x2 − 3x3 + 2x4 = −6 −x1 + 2x2 + x3 + 3x4 = 2    x1 + x2 − x3 + 2x4 = 1 The augmented matrix of the linear system is   1 −1 −2 1 0  2 −1 −3 2 −6     −1 2 1 3 2  1 1 −1 2 1 To transform the matrix into reduced row echelon form, we first use the leading 1 in row 1 as a pivot to eliminate the terms in column 1 of rows 2, 3, and 4. To do this, we use the three row operations −2R1 + R2 → R2 R1 + R3 → R3 −R1 + R4 → R4 in succession, transforming the matrix   1 −1 −2 1 0  2 −1 −3 2 −6     −1 2 1 3 2  1 1 −1 2 1

 to

1 −1 −2 1  0 1 1 0   0 1 −1 4 0 2 1 1

 0 −6   2  1

For the second step we use the leftmost 1 in row 2 as the pivot and eliminate the term in column 2 above the pivot, and the two terms below the pivot. The required row operations are R2 + R1 → R1 −R2 + R3 → R3 −2R2 + R4 → R4

Confirming Pages

20

Chapter 1 Systems of Linear Equations and Matrices

reducing the matrix  1 −1 −2 1  0 1 1 0   0 1 −1 4 0 2 1 1

 0 −6   2  1

 to

0 −1 1 1 1 0 0 −2 4 0 −1 1

1  0   0 0

 −6 −6   8  13

Notice that each entry in row 3 is evenly divisible by 2. Therefore, a leading 1 in row 3 is obtained using the operation − 12 R3 → R3 , which results in the matrix   1 0 −1 1 −6  0 1 1 0 −6     0 0 1 −2 −4  0 0 −1 1 13 Now, by using the leading 1 in row 3 as a pivot, the operations R3 + R1 → R1 −R3 + R2 → R2 R3 + R4 → R4 row-reduce the  1  0   0 0

matrix 0 −1 1 1 1 0 0 1 −2 0 −1 1

 −6 −6   −4  13

Using the operation −R4 → R4 , obtain the matrix  1  0   0 0



1  0   0 0

to

0 1 0 0

0 −1 0 2 1 −2 0 −1

 −10 −2   −4  9

we change the signs of the entries in row 4 to  0 0 −1 −10 1 0 2 −2   0 1 −2 −4  0 0 1 −9

Finally, using the leading 1 in row 4 as the pivot, we eliminate the terms above it in column 4. Specifically, the operations R4 + R1 → R1 −2R4 + R2 → R2 2R4 + R3 → R3 applied to the last matrix give 

1  0   0 0

0 1 0 0

0 0 1 0

0 0 0 1

 −19 16   −22  −9

which is in reduced row echelon form. The solution can now be read directly from the reduced matrix, giving us x1 = −19

x2 = 16

x3 = −22

and

x4 = −9.

Confirming Pages

1.2 Matrices and Elementary Row Operations

EXAMPLE 4

Solve the linear system.

21

 

3x1 − x2 + x3 + 2x4 = −2 x1 + 2x2 − x3 + x4 = 1  − x1 − 3x2 + 2x3 − 4x4 = −6 Solution

The linear system in matrix form is  3 −1 1 2  1 2 −1 1 −1 −3 2 −4 which can be reduced to



1 2  0 1 0 0

−1 1 −1 3 1 − 20 3

 −2 1  −6  1 5  −10

Notice that the system has infinitely many solutions, since from the last row we see that the variable x4 is a free variable. We can reduce the matrix further, but the solution can easily be found from the echelon form by back substitution, giving us 20 x3 = −10 + x4 3

20 11 x2 = 5 + x3 − 3x4 = 5 + −10 + x4 − 3x4 = −5 + x4 3 3 5 x1 = 1 − 2x2 + x3 − x4 = 1 − x4 3 Letting x4 be the arbitrary parameter t, we see the general solution is

    11t 20t 5t  , −10 + ,t t ∈ ⺢ S= 1 − , −5 + 3 3 3 Example 5 gives an illustration of a reduced matrix for an inconsistent linear system.

EXAMPLE 5

Solution

Solve the linear system.

  x+ y+ z=4 3x − y − z = 2  x + 3y + 3z = 8

To solve this system, we reduce the augmented matrix to triangular form. The following steps describe the process.     1 1 1 4 1 1 1 4 −3R + R → R 1 2 2  3 −1 −1 2  −→  0 −4 −4 −10  −R1 + R3 → R3 1 3 3 8 0 2 2 4

Confirming Pages

22

Chapter 1 Systems of Linear Equations and Matrices



1 1 2 R2 ↔ R3 −→  0 0 −4  1 1 1  0 1 −→ R → R 2 2 2 0 −4  1 4R2 + R3 → R3 −→  0 0

 4 4  −10  1 4 1 2  −4 −10  1 1 4 1 1 2  0 0 −2

1 2 −4

The third row of the last matrix corresponds to the equation 0 = −2. As this system has no solution, the system is inconsistent. This can also be seen from the fact that the three planes do not have a common intersection, as shown in Fig.2.

Figure 2

EXAMPLE 6

Solution

In Example 5, each entry in the last row of the reduced coefficient matrix is 0, but the constant term is nonzero and the linear system is inconsistent. The reduced augmented matrix for a consistent linear system can have a row of zeros. However, in this case the term in the last column of this row must also be zero. Example 6 gives an illustration.

Determine when the augmented matrix represents a consistent linear system.   1 0 2 a  2 1 5 b  1 −1 1 c The operation −2R1 + R2 → R2 followed by −R1 + R3 → R3 and finally followed by R2 + R3 → R3 reduces the augmented matrix to   1 0 2 a  0 1 1 b − 2a  0 0 0 b + c − 3a Hence, the corresponding linear system is consistent provided that b + c − 3a = 0. That is, the system is consistent for all a, b, and c such that the point (a, b, c) lies on the plane b + c − 3a = 0. Notice also that when the system is consistent, the third row will contain all zeros and the variable x3 is a free variable. Shown in the following list is an outline that summarizes the process for transforming a matrix to its equivalent reduced row echelon form. 1. If necessary, interchange rows so that the leading nonzero entry of row 1 is the leftmost nonzero entry of the matrix. Then divide each entry of row 1 by the leading entry.

Confirming Pages

1.2 Matrices and Elementary Row Operations

23

2. Eliminate all other nonzero terms in this leading column. 3. Repeat the first two steps, starting with row 2. Note that the leading entry may not be in column 2. 4. Continue in this way, making sure that the leading entry of each row is a 1 with zeros elsewhere in that column. 5. The leading 1 in any row should be to the right of a leading 1 in the row above it. 6. All rows of zeros are placed at the bottom of the matrix. We have implicitly assumed in our discussion that every matrix is row equivalent to exactly one matrix in reduced row echelon form. It is an important fact that we will state here as a theorem without proof.

THEOREM 3

The reduced row echelon form of every matrix is unique.

Fact Summary 1. Altering an augmented matrix by interchanging two rows, or multiplying a row by a nonzero constant, or replacing a row with the sum of the same row and a scalar multiple of another row does not alter the set of solutions of the corresponding linear system. 2. If an augmented matrix is row-reduced to triangular form, the coefficient matrix has a row of zeros, and the corresponding augmented term is not zero, then the linear system has no solutions. 3. Every matrix has a unique reduced row echelon form. 4. If the augmented matrix of an n × n linear system is row-reduced to triangular form and the coefficient matrix has no rows of zeros, then the linear system has a unique solution. 5. If the augmented matrix of an n × n linear system is row-reduced to triangular form, the coefficient matrix has rows of zeros, and each corresponding augmented term is 0, then the linear system has infinitely many solutions.

Exercise Set 1.2 In Exercises 1–8, write the linear system as an augmented matrix. Do not solve the system.  2x − 3y = 5 1. −x + y = −3

 2.

2x − 2y = 1 3x =1

 −z=4  2x x + 4y + z = 2 3.  4x + y − z = 1

Confirming Pages

24

Chapter 1 Systems of Linear Equations and Matrices

  −3x + y + z = 2 − 4z = 0 4.  −4x + 2y − 3z = 1  5.  6.

15.

2x1 − x3 = 4 x1 + 4x2 + x3 = 2

16.

4x1 + x2 − 4x3 = 1 4x1 − 4x2 + 2x3 = −2

17.

  2x1 + 4x2 + 2x3 + 2x4 = −2 4x1 − 2x2 − 3x3 − 2x4 = 2 7.  x1 + 3x2 + 3x3 − 3x4 = −4  − 3x3 + 4x4 = −3  3x1 −4x1 + 2x2 − 2x3 − 4x4 = 4 8.  4x2 − 3x3 + 2x4 = −3 In Exercises 9–20, write the solution of the linear system corresponding to the reduced augmented matrix. 

1 0 0 9.  0 1 0 0 0 1 

1 0 0 10.  0 1 0 0 0 1 

1 0 2 11.  0 1 −1 0 0 0 

1 0 − 13 12.  0 1 3 0 0 0 

1 −2 0 0 1 13.  0 0 0 0 



1 5 5 14.  0 0 0 0 0 0

−1 1 2

 

0  2 0  − 23  −3 2  0 4 4 3

 

0  −3 2  0  −1 0  0

18.

19.

20.

1 0  0 1 0 0  1 0  0 0 0 0  1 0 0 1  1 3 0 0  1 0  0 1 0 0  1 0  0 1 0 0

 0 0  1  0 0  1

0 0 0 0 1 0

−2 5 −1 2

3 2

−3 0 0 1

1 4

0 −3 0 −1 1 2 2 5

0 −3 0 0 1

 

 1 7  −1  −1 1  4 5

In Exercises 21–28, determine whether the matrices are in  echelon form.  reduced row 1 0 2 21. 0 1 3   1 2 0 22. 0 0 1   1 2 3 23.  0 1 2  0 0 1   1 2 0 24.  0 0 2  0 0 0   1 2 0 −1 25.  0 0 1 −2  0 0 0 0   1 0 −3 4 1 5  26.  0 1 0 0 0 0   1 0 0 4 −1 2  27.  0 0 1 5 0 1 0 0 −1

Confirming Pages

1.2 Matrices and Elementary Row Operations



1 1 0 4 28.  0 1 1 5 0 0 0 1

2 3



6 

40.

1 3

In Exercises 29–36, find the reduced row echelon form of the matrix.   2 3 29. −2 1   −3 2 30. 3 3   3 3 1 31.  3 −1 0  −1 −1 2   0 2 1 32.  1 −3 −3  1 2 −3   −4 1 4 33. 3 4 −3   −4 −2 −1 34. −2 −3 0   −2 2 −1 2 3 3 −3  35.  0 1 −4 2 2   4 −3 −4 −2 2 1 −4  36.  −4 −1 −3 1 −4 In Exercises 37–48, write the linear system as an augmented matrix. Convert the augmented matrix to reduced row echelon form, and find the solution of the linear system.  x+ y=1 37. 4x + 3y = 2  −3x + y = 1 38. 4x + 2y = 0  = 3  3x − 3y 4x − y − 3z = 3 39.  −2x − 2y = −2

41.

42.

43.

44.

45.

46.

47.

48.

25

 − 4z = 1  2x 4x + 3y − 2z = 0  2x + 2z = 2   x + 2y + z = 1 2x + 3y + 2z = 0  x + y + z=2   3x − 2z = −3 −2x + z = −2  − z= 2  3x1 + 2x2 + 3x3 = −3 x1 + 2x2 − x3 = −2  − 3x2 − x3 = 2 x1 + x3 = −2  + 3x3 + x4 = 2  −x1 2x1 + 3x2 − 3x3 + x4 = 2  2x1 − 2x2 − 2x3 − x4 = −2   −3x1 − x2 + 3x3 + 3x4 = −3 x1 − x2 + x3 + x4 = 3  −3x1 + 3x2 − x3 + 2x4 = 1   3x1 − 3x2 + x3 + 3x4 = −3 x1 + x2 − x3 − 2x4 = 3  4x1 − 2x2 + x4 = 0   −3x1 + 2x2 − x3 − 2x4 = 2 x1 − x2 − 3x4 = 3  4x1 − 3x2 + x3 − x4 = 1

49. The augmented matrix of a linear system has the form   1 2 −1 a  2 3 −2 b  −1 −1 1 c a. Determine the values of a, b, and c for which the linear system is consistent. b. Determine the values of a, b, and c for which the linear system is inconsistent. c. When it is consistent, does the linear system have a unique solution or infinitely many solutions?

Confirming Pages

26

Chapter 1 Systems of Linear Equations and Matrices

d. Give a specific consistent linear system and find one particular solution. 50. The augmented matrix of a linear system has the form 

ax 2x

y (a − 1)y

1 1

a. Determine the values of a, b, and c for which the linear system is consistent. b. Determine the values of a, b, and c for which the linear system is inconsistent. c. When it is consistent, does the linear system have a unique solution or infinitely many solutions? d. Give a specific consistent linear system and find one particular solution.



a. Determine the values of a for which the linear system is consistent. b. When it is consistent, does the linear system have a unique solution or infinitely many solutions? c. Give a specific consistent linear system and find one particular solution.

ß

1.3

51. The augmented matrix of a linear system has the form   −2 3 1 a  1 1 −1 b  0 5 −1 c

52. Give examples to describe all 2 × 2 reduced row echelon matrices.

Matrix Algebra Mathematics deals with abstractions that are based on natural concepts in concrete settings. For example, we accept the use of numbers and all the algebraic properties that go with them. Numbers can be added and multiplied, and they have properties such as the distributive and associative properties. In some ways matrices can be treated as numbers. For example, we can define addition and multiplication so that algebra can be performed with matrices. This extends the application of matrices beyond just a means for representing a linear system. Let A be an m × n matrix. Then each entry of A can be uniquely specified by using the row and column indices of its location, as shown in Fig. 1. Column j ↓



···

a11  ..  .  Row i −→   ai1  .  .. am1 For example, if

···

a1j .. .

aij .. . · · · amj Figure 1

 a1n ..  .   · · · ain  =A ..  .  · · · amn ···

 −2 1 4 A =  5 7 11  2 3 22 

Confirming Pages

1.3 Matrix Algebra

27

then a11 = −2 a21 = 5 a31 = 2

a12 = 1 a22 = 7 a32 = 3

a13 = 4 a23 = 11 a33 = 22

A vector is an n × 1 matrix. The entries of a vector are called its components. For a given matrix A, it is convenient to refer to its row vectors and its column vectors. For example, let   1 2 −1 0 1  A= 3 4 −1 2 Then the column vectors of A are  1  3  4 

 2  0  −1

 −1  1  2 



and

while the row vectors of A, written vertically, are  1  2  −1 

 3  0  1

 4  −1  2 



and

Two m × n matrices A and B are equal if they have the same number of rows and columns and their corresponding entries are equal. Thus, A = B if and only if aij = bij , for 1 ≤ i ≤ m and 1 ≤ j ≤ n. Addition and scalar multiplication of matrices are also defined componentwise.

DEFINITION 1

EXAMPLE 1

Addition and Scalar Multiplication If A and B are two m × n matrices, then the sum of the matrices A + B is the m × n matrix with the ij term given by aij + bij . The scalar product of the matrix A with the real number c, denoted by cA, is the m × n matrix with the ij term given by caij .

Perform the operations on the matrices   2 0 1 and A =  4 3 −1  −3 6 5 a. A + B

b. 2A − 3B

 −2 3 −1 6  B= 3 5 4 2 1 

Confirming Pages

28

Chapter 1 Systems of Linear Equations and Matrices

Solution

a. We add the two matrices by adding their corresponding    −2 3 2 0 1 A + B =  4 3 −1  +  3 5 4 2 −3 6 5  2 + (−2) 0 + 3 1 + (−1) 4+3 3+5 −1 + 6 = −3 + 4 6 + 2 5+1   0 3 0 = 7 8 5  1 8 6

entries, so that  −1 6  1  

b. To evaluate this expression, we first multiply each entry of the matrix A by 2 and each entry of the matrix B by −3. Then we add the resulting matrices. This gives     −2 3 −1 2 0 1 6  2A + (−3B) = 2  4 3 −1  + (−3)  3 5 4 2 1 −3 6 5     6 −9 3 4 0 2 6 −2  +  −9 −15 −18  = 8 −12 −6 −3 −6 12 10   10 −9 5 =  −1 −9 −20  −18 6 7

In Example 1(a) reversing the order of the addition of the matrices gives the same result. That is, A + B = B + A. This is so because addition of real numbers is commutative. This result holds in general, giving us that matrix addition is also a commutative operation. Some other familiar properties that hold for real numbers also hold for matrices and scalars. These properties are given in Theorem 4. THEOREM 4

Properties of Matrix Addition and Scalar Multiplication Let A, B, and C be m × n matrices and c and d be real numbers. 1. A + B = B + A 2. A + (B + C) = (A + B) + C 3. c(A + B) = cA + cB 4. (c + d)A = cA + dA 5. c(dA) = (cd)A 6. The m × n matrix with all zero entries, denoted by 0, is such that A + 0 = 0 + A = A. 7. For any matrix A, the matrix −A, whose components are the negative of each component of A, is such that A + (−A) = (−A) + A = 0.

Confirming Pages

1.3 Matrix Algebra

29

Proof In each case it is sufficient to show that the column vectors of the two matrices agree. We will prove property 2 and leave the others as exercises. (2) Since the matrices A, B, and C have the same size, the sums (A + B) + C and A + (B + C) are defined and also have the same size. Let Ai , Bi , and Ci denote the ith column vector of A, B, and C, respectively. Then       a1i b1i c1i       (Ai + Bi ) + Ci =  ...  +  ...  +  ...    =

ami



bmi 

a1i + b1i c1i   .. .. + . . ami + bmi cmi

cmi   (a1i + b1i ) + c1i    .. =  . (ami + bmi ) + cmi 

Since the components are real numbers, where the associative property of addition holds, we have   (a1i + b1i ) + c1i   .. (Ai + Bi ) + Ci =   . 

(ami + bmi ) + cmi

 a1i + (b1i + c1i )   .. =  = Ai + (Bi + Ci ) . ami + (bmi + cmi ) As this holds for every column vector, the matrices (A + B) + C and A + (B + C) are equal, and we have (A + B) + C = A + (B + C).

Matrix Multiplication We have defined matrix addition and a scalar multiplication, and we observed that these operations satisfy many of the analogous properties for real numbers. We have not yet considered the product of two matrices. Matrix multiplication is more difficult to define and is developed from the dot product of two vectors.

DEFINITION 2

Dot Product of Vectors Given two vectors    u1   u2     and v= u= .    ..  un

v1 v2 .. .

    

vn

the dot product is defined by u · v = u 1 v1 + u2 v2 + · · · + un vn =

n  i=1

u i vi

Confirming Pages

30

Chapter 1 Systems of Linear Equations and Matrices

Observe that the dot product of two vectors is a scalar. For example,     −5 2  −3  ·  1  = (2)(−5) + (−3)(1) + (−1)(4) = −17 4 −1 Now to motivate the concept and need for matrix multiplication we first introduce the operation of multiplying a vector by a matrix. As an illustration let     1 1 −1 and v= B= 3 −2 1 The product of B and v, denoted by Bv, is a vector, in this case with two components. The first component of Bv is the dot product of the first row vector of B with v, while the second component is the dot product of the second row vector of B with v, so that        −2 (1)(1) + (−1)(3) 1 1 −1 = = Bv = 1 (−2)(1) + (1)(3) 3 −2 1   1 to the vector Using this operation, the matrix B transforms the vector v = 3     −1 2 −2 is another matrix, then the product of A and Bv . If A = Bv = 0 1 1 is given by      4 −2 −1 2 = A(Bv) = 1 1 0 1 The question then arises, is there a single matrix which can be used to transform     4 1 ? To answer this question, let to the original vector 1 3       a11 a12 x b11 b12 A= v= and B= y a21 a22 b21 b22 The product of B and v is  Bv =

b11 x + b12 y b21 x + b22 y



Now, the product of A and Bv is    a11 a12 b11 x + b12 y A(Bv) = b21 x + b22 y a21 a22   a11 (b11 x + b12 y) + a12 (b21 x + b22 y) = a21 (b11 x + b12 y) + a22 (b21 x + b22 y)   (a11 b11 + a12 b21 )x + (a11 b12 + a12 b22 )y = (a21 b11 + a22 b21 )x + (a21 b12 + a22 b22 )y    a11 b11 + a12 b21 a11 b12 + a12 b22 x = y a21 b11 + a22 b21 a21 b12 + a22 b22

Confirming Pages

31

1.3 Matrix Algebra

Thus, we see that A(Bv) is the product of the matrix   a11 b11 + a12 b21 a11 b12 + a12 b22 a21 b11 + a22 b21 a21 b12 + a22 b22   x . We refer to this matrix as the product of A and B, denoted by and the vector y AB, so that A(Bv) = (AB)v See Fig. 2.

AB y

y

y

B

5

A

5

5

v

ABv

Bv ⫺5

5

⫺5

x

⫺5

5

⫺5

x

⫺5

5

x

⫺5

Figure 2

Notice that the product matrix AB is obtained by computing the dot product of each row vector of the matrix A, on the left, with each column vector of the matrix B, on the right. Using the matrices A and B given above, we have    1 −1 −1 2 AB = −2 1 0 1     −5 3 (−1)(1) + (2)(−2) (−1)(−1) + (2)(1) = = −2 1 (0)(1) + (1)(−2) (0)(−1) + (1)(1)     4 1 in one step. That is, to This matrix transforms the vector 1 3      4 1 −5 3 = (AB)v = 1 3 −2 1 which was our original aim. The notion of matrices as transformations is taken up again in Chap. 4 where we consider more general transformations of vectors. For another illustration of the matrix product let     3 −2 5 1 3 0   A= and B =  −1 4 −2  2 1 −3  −4 6 2 1 0 3 The entries across the first row of the product matrix AB are obtained from the dot product of the first row vector of A with the first, second, and third column vectors of

Confirming Pages

32

Chapter 1 Systems of Linear Equations and Matrices

B, respectively. The terms in the second row of AB are the dot products of the second row vector of A with the first, second, and third column vectors of B, respectively. Finally, the terms in the third row of AB are the dot products of the third row vector of A again with the first, second, and third column vectors of B, respectively. Thus, the product matrix AB is 

(1)(3)+(3)(−1)+(0)(1) AB =  6−1−3 −12 − 6 + 2   0 10 −1 2 0 −1  = −16 32 −26

−2 + 12 + 0 −4 + 4 + 0 8 + 24 + 0

 5−6+0 10 − 2 − 9  −20 − 12 + 6

In the previous example, the product AB exists since the matrices A and B have the same number of rows and the same number of columns. This condition can be relaxed somewhat. In general, the product of two matrices A and B exists if the number of columns of A is equal to the number of rows of B.

DEFINITION 3

Matrix Multiplication Let A be an m × n matrix and B an n × p matrix; then the product AB is an m × p matrix. The ij term of AB is the dot product of the ith row vector of A with the j th column vector of B, so that (AB)ij = ai1 b1j + ai2 b2j + · · · + ain bnj =

n 

aik bkj

k=1

It is important to recognize that not all properties of real numbers carry over to properties of matrices. Because matrix multiplication is only defined when the number of columns of the matrix on the left equals the number of rows of the matrix on the right, it is possible for AB to exist with BA being undefined. For example,     3 −2 5 1 3 0  −1 4 −2  AB = 2 1 −3 1 0 3 is defined, but

  3 −2 5  1 3 0   4 −2 BA = −1 2 1 −3 1 0 3 is not. As a result, we cannot interchange the order when multiplying two matrices unless we know beforehand that the matrices commute. We say two matrices A and B commute when AB = BA. Example 2 illustrates that even when AB and BA are both defined, they might not be equal. 

Confirming Pages

1.3 Matrix Algebra

EXAMPLE 2

Verify that the matrices



A=

1 0 −1 2



 B=

and



0 1 1 1

do not satisfy the commutative property for multiplication. Solution

The products are



1 0 −1 2

AB = 

and

0 1 1 1

BA =





0 1 1 1

1 0 −1 2



 =



 =



0 1 2 1 −1 2 0 2



so that AB = BA.

In Example 3 we describe all matrices that commute with a particular matrix.

EXAMPLE 3

Solution

Find all 2 × 2 matrices that commute with the matrix   1 0 A= 1 1 We start by letting B denote an arbitrary 2 × 2 matrix   a b B= c d Then the product of matrix A on the left with matrix B on the right is given by      a b a b 1 0 = AB = a+c b+d c d 1 1 On the other hand,

 BA =

a c

b d



1 0 1 1



 =

a+b c+d

b d



Setting AB = BA, we obtain a =a+b

a+c =c+d

and

b+d =d

so that b = 0 and a = d. Let S be the set of all 2 × 2 matrices defined by    a 0  a, c ∈ ⺢ S= c a  Then each matrix in S commutes with the matrix A.

33

Confirming Pages

34

Chapter 1 Systems of Linear Equations and Matrices

EXAMPLE 4

Perform the  −3 A= 2 −1

operations on the matrices    1 −1 1 −1 3 2  B= 2 5 −3 1 5

a. A(B + C) Solution

 and

C=

3 2 −2 1 1 6 −2 4

b. AB + AC

We first notice that the matrix A is 3 × 2 and both B and C are 2 × 4, so AB and AC are defined. Also since the matrices B and C have the same number of rows and columns, the matrix B + C is defined, so the expressions in parts (a) and (b) are defined. a. We first add the matrices B and C inside the parentheses and then multiply on the left by the matrix A. This gives us A(B + C)     −3 1  3 2 −2 −1 1 −1 3   2 2 + = 1 6 −2 2 5 −3 1 −1 5    −3 1  2 3 −3 4 = 2 2  3 11 −5 5 −1 5  −3(2) + 1(3) −3(3) + 1(11) −3(−3) + 1(−5) 2(3) + 2(11) 2(−3) + 2(−5) =  2(2) + 2(3) −1(2) + 5(3) −1(3) + 5(11) −1(−3) + 5(−5)   −3 2 4 −7 =  10 28 −16 18  13 52 −22 21

1 4



 −3(4) + 1(5) 2(4) + 2(5)  −1(4) + 5(5)

b. In this case we compute AB and AC separately and then add the two resulting matrices. We have    −3 1  −1 1 −1 3 AB + AC =  2 2  2 5 −3 1 −1 5    −3 1  3 2 −2 1 + 2 2  1 6 −2 4 −1 5     −8 0 4 1 5 2 0 −8 8  +  8 16 −8 10  =  2 12 −8 2 28 −8 19 11 24 −14 2   −3 2 4 −7 =  10 28 −16 18  13 52 −22 21



Confirming Pages

1.3 Matrix Algebra

35

Notice that in Example 4 the matrix equation A(B + C) = AB + AC holds. This and other familiar properties involving multiplication and addition of real numbers hold for matrices. They are listed in Theorem 5.

THEOREM 5

Properties of Matrix Multiplication Let A, B, and C be matrices with sizes so that the given expressions are all defined, and let c be a real number. 1. 2. 3. 4.

A(BC) = (AB)C c(AB) = (cA)B = A(cB) A(B + C) = AB + AC (B + C)A = BA + CA

We have already seen that unlike with real numbers, matrix multiplication does not commute. There are other properties of the real numbers that do not hold for matrices. Recall that if x and y are real numbers such that xy = 0, then either x = 0 or y = 0. This property does not hold for matrices. For example, let     −1 −1 1 1 and B= A= 1 1 1 1 

Then AB =

1 1 1 1



−1 1

−1 1



 =

0 0 0 0



Transpose of a Matrix The transpose of a matrix is obtained by interchanging the rows and columns of a matrix.

DEFINITION 4

Transpose If A is an m × n matrix, the transpose of A, denoted by At , is the n × m matrix with ij term (At )ij = aj i where 1 ≤ i ≤ n and 1 ≤ j ≤ m. For example, the transpose of the matrix   1 2 −3 4  is A= 0 1 −1 2 1

 1 0 −1 2  At =  2 1 −3 4 1 

Notice that the row vectors of A become the column vectors of At . Theorem 6 gives some properties of the transpose.

Confirming Pages

36

Chapter 1 Systems of Linear Equations and Matrices

THEOREM 6

Suppose A and B are m × n matrices, C is an n × p matrix, and c is a scalar. 1. (A + B)t = At + B t 2. (AC)t = C t At 3. (At )t = A 4. (cA)t = cAt Proof (2) We start by showing that the products involved are all defined. Since AC is m × p, then (AC)t is p × m. As C t is p × n and At is n × m, then C t At is also p × m. So the sizes of the products agree. Now to show that the products are equal, we apply the definitions of multiplication and transpose to obtain (C t At )ij =

n 

(C t )ik (At )kj

k=1

=

n 

cki aj k =

k=1

n 

aj k cki = (AC)j i

k=1

= ((AC)t )ij The proofs of parts 1, 3, and 4 are left as exercises.

DEFINITION 5

EXAMPLE 5 Solution

Symmetric Matrix

An n × n matrix is symmetric provided that At = A.

Find all 2 × 2 matrices that are symmetric. 

Let A=

a c

b d

Then A is symmetric if and only if    a a b = A= b c d



c d

 = At

which holds if and only if b = c. So a 2 × 2 matrix is symmetric if and only if the matrix has the form   a b b d

Fact Summary Let A, B, and C be matrices. 1. The definitions of matrix addition and scalar multiplication satisfy many of the properties enjoyed by real numbers. This allows algebra to be carried out with matrices.

Confirming Pages

1.3 Matrix Algebra

2. When AB is defined, the ij entry of the product matrix is the dot product of the ith row vector of A with the j th column vector of B. 3. Matrix multiplication does not in general commute. Even when AB and BA are both defined, it is possible for AB
= BA. 4. The distributive properties hold. That is, A(B + C) = AB + AC and (B + C)A = BA + CA. t t t t t t t 5. (A + B)t = At +
B t , (AB)  = B A , (A ) = A, (cA) = cA a b is symmetric if and only if b = c. 6. The matrix A = c d

Exercise Set 1.3 In Exercises 1–4, use the matrices  

−1 3 2 −3 B= A= −2 5 4 1 
1 1 C= 5 −2 1. Find A + B and B + A. 2. Find 3A − 2B. 3. Find (A + B) + C and A + (B + C).

7. Find AB and BA. 8. Show that 3(AB) = A(3B). In Exercises 9 and 10, use the matrices 
2 −3 −3 A= −3 −2 0   3 −1 B =  2 −2  3 0 9. Find AB.

4. Find 3(A + B) − 5C.

10. Find BA.

In Exercises 5 and 6, use the matrices   −3 −3 3 0 2  A= 1 0 −2 3   −1 3 3 B =  −2 5 2  1 2 4   −5 3 9 C =  −3 10 6  2 2 11

11. Let

and

−1 1 A =  3 −3 −1 2  −2 3 B =  0 −1 3 −2

 1 3  1  −3 2  −1

Find AB. 12. Let

5. Find (A − B) + C and 2A + B. 6. Show that A + 2B − C = 0. In Exercises 7 and 8, use the matrices 

2 3 1 B= A= 1 −2 4



and 0 −2

 Find AB.

 −2 −2 −1 2 1  A =  −3 1 −1 −1   1 −1 −2 3  B =  −2 −2 −3 1 −3 

37

Confirming Pages

38

Chapter 1 Systems of Linear Equations and Matrices

Find a 2 × 2 matrix B that is not the zero matrix, such that AB is the zero matrix.

In Exercises 13–16, use the matrices     2 0 −2 −3 B= A= −2 0 3 0   2 0 C= −1 −1

27. Find all 2 × 2 matrices of the form   a b A= 0 c A2 = AA =

14. Find (A + B)C.

−

29. Find matrices A and B such that AB = 0 but BA = 0. 30. Show there are no 2 × 2 matrices A and B such that   1 0 AB − BA = 0 1 31. Determine all values of a and b such that      −5 6 3 b 1 2 = 12 16 −4 1 a 0

Bt

18. B t − 2A 19. AB t

32. If A and B are 2 × 2 matrices, show that the sum of the terms on the diagonal of AB − BA is 0.

20. BAt 21. (At + B t )C 22.



 2 1 . Find all matrices of the form 28. Let A = 1 1  a b such that AM = MA. M= c d

16. Find (A + 2B)(3C). In Exercises 17–24, use the matrices   2 0 −1 A= 1 0 −2   −3 1 1 B= −3 −3 −2   3 −1 C= −1 −3 Whenever possible, perform the operations. If a computation cannot be made, explain why. 17.

1 0 0 1



15. Find 2A(B − 3C).

2At



such that

13. Find A(B + C).

C(At

+

33. Let

23. (At C)B

 A=

 0 0 −1 0  0 1

Find the matrix A20 .

24. (At B t )C 25. Let



1 A= 0 0

Bt )

−1 −2 1 2  C=



 B= 

1 2

7 5 −1 −2 Show that AB = AC and yet B = C. 

26. Let A=

0 2 0 5



3 −1



34. If A and B are n × n matrices, when does (A + B)(A − B) = A2 − B 2 ? 35. If the matrices A and B commute, show that A2 B = BA2 . 36. Suppose A, B, and C are n × n matrices and B and C both commute with A. a. Show that BC and A commute. b. Give specific matrices to show that BC and CB do not have to be equal.

Confirming Pages

1.4 The Inverse of a Square Matrix

37. Suppose that A is an n × n matrix. Show that if for each vector x in ⺢n , Ax = 0, then A is the zero matrix. 38. For each positive integer n, let   1−n −n An = n 1+n Show that An Am = An+m . 39. Find all 2 × 2 matrices that satisfy AAt = 0.

39

42. An n × n matrix A is called idempotent provided that A2 = AA = A. Suppose that A and B are n × n idempotent matrices. Show that if AB = BA, then the matrix AB is idempotent. 43. An n × n matrix A is skew-symmetric provided At = −A. Show that if a matrix is skew-symmetric, then the diagonal entries are 0. 44. The trace of an n × n matrix A is the sum of the diagonal terms, denoted tr(A).

40. Suppose that A and B are symmetric matrices. Show that if AB = BA, then AB is symmetric.

a. If A and B are n × n matrices, show that tr(A + B) = tr(A) + tr(B).

41. If A is an m × n matrix, show that AAt and At A are both defined and are both symmetric.

b. If A is an n × n matrix and c is a scalar, show that tr(cA) = c tr(A).

ß

1.4

The Inverse of a Square Matrix In the real number system, the number 1 is the multiplicative identity. That is, for any real number a, a·1=1·a =a We also know that for every number x with x = 0, there exists the number x1 , also written x −1 , such that 1 x· =1 x We seek a similar relationship for square matrices. For an n × n matrix A, we can check that the n × n matrix   1 0 0 ··· 0  0 1 0 ··· 0      I =  0 0 1 ··· 0   .. .. .. . . ..   . . . . .  0 0 0 ··· 1 is the multiplicative identity. That is, if A is any n × n matrix, then AI = I A = A This special matrix is called the identity matrix. For 4 × 4 identity matrices are, respectively,     1 0 0 1 0  0 1 0  and 0 1 0 0 1

example, the 2 × 2, 3 × 3, and 

1  0   0 0

0 1 0 0

0 0 1 0

 0 0   0  1

Confirming Pages

40

Chapter 1 Systems of Linear Equations and Matrices

DEFINITION 1

Let A be an n × n matrix. If there exists an n × n

Inverse of a Square Matrix matrix B such that

AB = I = BA then the matrix B is a (multiplicative) inverse of the matrix A.

EXAMPLE 1



Find an inverse of the matrix A= 

Solution

In order for a 2 × 2 matrix B = satisfy



1 1 1 2



x1 x3

x2 x4

x1 x3



 =

x2 x4

1 1 1 2



 to be an inverse of A, matrix B must

x1 + x3 x1 + 2x3

x2 + x4 x2 + 2x4



 =

1 0 0 1



This matrix equation is equivalent to the linear system  x1 + x3 =1    x2 + x4 = 0 =0  x1 + 2x3   x2 + 2x4 = 1 The augmented matrix  1  0   1 0

and the reduced row echelon   0 1 0 1 1 0  0 1 1 0 1 0  →  0 0 0 2 0 0  1 0 2 1 0 0

form are given by  2 0 0 0 0 −1   1 0 −1  0 1 1

Thus, the solution is x1 = 2, x2 = −1, x3 = −1, x4 = 1, and an inverse matrix is   2 −1 B= −1 1 The reader should verify that AB = BA = I .

Theorem 7 establishes the uniqueness, when it exists, of the multiplicative inverse.

THEOREM 7

The inverse of a matrix, if it exists, is unique. Proof Assume that the square matrix A has an inverse and that B and C are both inverse matrices of A. That is, AB = BA = I and AC = CA = I. We show that B = C. Indeed, B = BI = B(AC) = (BA)C = (I )C = C

Confirming Pages

1.4 The Inverse of a Square Matrix

41

We refer to the unique inverse as the inverse of A and denote it by A−1 . When the inverse of a matrix A exists, we call A invertible. Otherwise, the matrix A is called noninvertible.  THEOREM 8

The inverse of the matrix A =

a c

b d

 exists if and only if ad − bc = 0. In this

case the inverse is the matrix −1

A

1 = ad − bc



d −c

−b a



Proof First, assume that ad − bc = 0, and let   x1 x2 B= x3 x4 We need to find x1 , x2 , x3 , and x4 such that   1 0 AB = 0 1 Taking the product of the two matrices yields     ax1 + bx3 ax2 + bx4 1 0 = 0 1 cx1 + dx3 cx2 + dx4 which results in the linear system  ax1   

+ bx3

=1 + bx4 = 0 cx1 + dx3 =0    cx2 + dx4 = 1 ax2

The augmented matrix of this linear  a  0   c 0

system is given by  0 b 0 1 a 0 b 0   0 d 0 0  c 0 d 1

 d 1 0 0 0 ad−bc  0 1 0 0 − b   ad−bc    c   0 0 1 0 − ad−bc a 0 0 0 1 ad−bc Since ad − bc = 0, the inverse of the matrix A is    d b 1 − ad−bc d −1 ad−bc = A = a c −c − ad−bc ad − bc ad−bc which reduces to



−b a



To prove the reverse claim, we use the contrapositive. That is, if ad − bc = 0, then the inverse does not exist. An outline of the proof is given in Exercise 41.

Confirming Pages

42

Chapter 1 Systems of Linear Equations and Matrices

To illustrate the use of the formula, let   2 −1 A= 1 3 then −1

A

1 = 6 − (−1)



3 1 −1 2



 =

1 7 2 7

3 7 − 17



For an example which underscores the necessity of the condition that ad − bc = 0, we consider the matrix   1 1 A= 1 1 Observe that in this case ad − bc = 1 − 1 = 0. Now, the matrix A is invertible if x1 x2 there is a B = such that x3 x4      x1 x2 1 1 1 0 = 1 1 0 1 x3 x4 This matrix equation yields the inconsistent system  x1 + x3 =1    x2 + x4 = 0 x1 + x3 =0    x2 + x4 = 1 Hence, A is not invertible. To find the inverse of larger square matrices, we extend the method of augmented matrices. Let A be an n × n matrix. Let B be another n × n matrix, and let B1 , B2 , . . . , Bn denote the n column vectors of B. Since AB1 , AB2 , . . . , ABn are the column vectors of AB, in order for B to be the inverse of A, we must have       1 0 0  0   1   0        AB2 =  .  ... ABn =  .  AB1 =  .   ..   ..   ..  0

0

That is, the matrix equations   1  0    Ax =  .   .. 

   Ax =  

0 1 .. .

1 

    

...

  Ax =  

    

1

0

0

0 0 .. .

must all have unique solutions. But all n linear systems can be solved simultaneously by row-reducing the n × 2n augmented matrix   a11 a12 . . . a1n 1 0 . . . 0  a21 a22 . . . a2n 0 1 . . . 0     .. .. .. . . ..  .. .. ..  . . . . . .  . . an1

an2

. . . ann

0 0 ... 1

Confirming Pages

1.4 The Inverse of a Square Matrix

43

On the left is the matrix A, and on the right is the matrix I. Then A will have an inverse if and only if it is row equivalent to the identity matrix. In this case, each of the linear systems can be solved. If the matrix A does not have an inverse, then the row-reduced matrix on the left will have a row of zeros, indicating at least one of the linear systems does not have a solution. Example 2 illustrates the procedure.

EXAMPLE 2

Find the inverse of the matrix



1 A =  −1 0 Solution

 1 −2 2 0  −1 1

To find the inverse of this matrix, place the identity on the right to form the 3 × 6 matrix   1 1 −2 1 0 0  −1 2 0 0 1 0  0 −1 1 0 0 1 Now use row operations to reduce the matrix on the left to the identity, while applying the same operations to the matrix on the right. The final result is   1 0 0 2 1 4  0 1 0 1 1 2  0 0 1 1 1 3 so the inverse matrix is   2 1 4 A−1 =  1 1 2  1 1 3 The reader should check that AA−1 = A−1 A = I.

EXAMPLE 3

Use the method of Example 2 to determine whether the matrix   1 −1 2 A =  3 −3 1  3 −3 1 is invertible.

Solution

Following the procedure described  1  3 3

above, we start with the matrix  −1 2 1 0 0 −3 1 0 1 0  −3 1 0 0 1

Confirming Pages

44

Chapter 1 Systems of Linear Equations and Matrices

After the two row operations −3R1 + R2 → R2 followed by −3R1 + R3 → R3 , this matrix is reduced to   1 −1 2 1 0 0  0 0 −5 −3 1 0  0 0 −5 −3 0 1 Next perform the row operation −R2 + R3 → R3 to obtain 

1 −1 2  0 0 −5 0 0 0

 1 0 0 −3 1 0  0 −1 1

The 3 × 3 matrix of coefficients on the left cannot be reduced to the identity matrix, and therefore, the original matrix does not have an inverse. Also notice that a solution does exist to   1 Ax =  0  0 while solutions to     0 0 and Ax =  0  Ax =  1  1 0 do not exist.

The matrix A of Example 3 has two equal rows and cannot be row-reduced to the identity matrix. This is true for any n × n matrix with two equal rows and provides an alternative method for concluding that such a matrix is not invertible. Theorem 9 gives a formula for the inverse of the product of invertible matrices.

THEOREM 9

Let A and B be n × n invertible matrices. Then AB is invertible and (AB)−1 = B −1 A−1 Proof Using the properties of matrix multiplication, we have (AB)(B −1 A−1 ) = A(BB −1 )A−1 = AI A−1 = AA−1 = I and

(B −1 A−1 )(AB) = B −1 (A−1 A)B = B −1 I B = BB −1 = I Since, when it exists, the inverse matrix is unique, we have shown that the inverse of AB is the matrix B −1 A−1 .

Confirming Pages

1.4 The Inverse of a Square Matrix

EXAMPLE 4 Solution

45

Suppose that B is an invertible matrix and A is any matrix with AB = BA. Show that A and B −1 commute. Since AB = BA, we can multiply both sides on the right by B −1 to obtain (AB)B −1 = (BA)B −1 By the associative property of matrix multiplication this last equation can be written as A(BB −1 ) = BAB −1 and since BB −1 = I, we have

A = BAB −1

Next we multiply on the left by B −1 to obtain B −1 A = B −1 BAB −1

B −1 A = AB −1

so

as required.

Fact Summary Let A and B denote matrices. 1. The inverse of amatrix, when it exists, is unique.    d −b a b 1 −1 . and ad − bc = 0, then A = ad−bc 2. If A = −c a c d 3. A matrix A is invertible if and only if it is row equivalent to the identity matrix. 4. If A and B are invertible n × n matrices, then AB is invertible and (AB)−1 = B −1 A−1 .

Exercise Set 1.4 In Exercises 1–16, a matrix A is given. Find A−1 or indicate that it does not exist. When A−1 exists, check your answer by showing that AA−1 = I .   1 −2 1. 3 −1   −3 1 2. 1 2

 3.  4.

−2 2 1 2

1 2

4 −4 



 0 1 −1 1  5.  3 1 1 2 −1 

Confirming Pages

46

Chapter 1 Systems of Linear Equations and Matrices

 −2 −3 3 0  2 0 −2 0   16.   2 0 −1 −1  −2 0 1 1 

 0 2 1 6.  −1 0 0  2 1 1   3 −3 1 0 1  7.  0 −2 2 −1   1 3 0 2 3  8.  1 0 −1 3 

17. Let

3 3 0 −3  0 1 2 0   9.   0 0 −1 −1  0 0 0 −2 

1  0 10.   0 0

3 1 0 0

 0 −3 2 −3   2 −2  0 2

 1 0 0 0  2 1 0 0   11.   −3 −2 −3 0  0 1 3 3 

 1 0 0 0  −2 1 0 0   12.   1 −1 −2 0  2 −2 0 2   2 −1 4 −5  0 −1 3 −1   13.   0 0 0 2  0 0 0 −1   3 0 0 0  −6 1 0 0   14.   2 −5 0 0  1 −3 4 2   −1 1 0 −1  −1 1 −1 0   15.   −1 0 0 0  −2 1 −1 1 



2 1 3 −4

 B=

1 2 −1 3



Verify that AB + A can be factored as A(B + I ) and AB + B can be factored as (A + I )B. 



 A=

18. If A is an n × n matrix, write A2 + 2A + I in factored form. 

19. Let A=

1 2 −2 1



a. Show that A2 − 2A + 5I = 0. b. Show that A−1 = 15 (2I − A). c. Show in general that for any square matrix A satisfying A2 − 2A + 5I = 0, the inverse is A−1 = 15 (2I − A). 20. Determine those values  1  3 1

of λ 2 2

λ for which the matrix  0 0  1

is not invertible. 21. Determine those values  1  1 2

of λ for which the matrix  λ 0 3 1  1 1

is not invertible. 22. Determine those values  2  3 1

of λ for which the matrix  λ 1 2 1  2 1

is not invertible. 23. Let



1 A= 1 0

 λ 0 1 1  0 1

Confirming Pages

1.4 The Inverse of a Square Matrix

a. Determine those values of λ for which A is invertible. b. For those values found in part (a) find the inverse of A. 24. Determine those values of λ for which the matrix   λ −1 0  −1 λ −1  0 −1 λ is invertible. 25. Find 2 × 2 matrices A and B that are not invertible but A + B is invertible. 26. Find 2 × 2 matrices A and B that are invertible but A + B is not invertible. 27. If A and B are n × n matrices and A is invertible, show that (A + B)A−1 (A − B) = (A − B)A−1 (A + B) 28. If B = P AP −1 , express B 2 , B 3 , . . . , B k , where k is any positive integer, in terms of A, P , and P −1 . 29. Let A and B be n × n matrices. a. Show that if A is invertible and AB = 0, then B = 0. b. If A is not invertible, show there is an n × n matrix B that is not the zero matrix and such that AB = 0. 30. Show that if A is symmetric and invertible, then A−1 is symmetric. In Exercises 31–34, the matrices A and B are invertible symmetric matrices and AB = BA. 31. Show that AB is symmetric. 32. Show that A−1 B is symmetric. 33. Show that AB −1 is symmetric. 34. Show that A−1 B −1 is symmetric. 35. A matrix A is orthogonal provided that At = A−1 . Show that the product of two orthogonal matrices is orthogonal.

36. Show the matrix A=



cos θ − sin θ sin θ cos θ

47



is orthogonal. (See Exercise 35.) 37. a. If A, B, and C are n × n invertible matrices, show that (ABC)−1 = C −1 B −1 A−1 b. Use mathematical induction to show that for all positive integers k, if A1 , A2 , . . . , Ak are n × n invertible matrices, then −1 −1 (A1 A2 · · · Ak )−1 = A−1 k Ak−1 · · · A1

38. An n × n matrix A is diagonal provided that aij = 0 whenever i = j. Show that if ann = 0 for all n, then A is invertible and the inverse is  1  0 0 ... 0 a11 1  0 0 ... 0    a22  . .. .. ..  ..  .  . . . .   .   1 0  0 . . . an−1,n−1  0 1 0 0 ... 0 ann 39. Let A be an n × n invertible matrix. Show that if A is in upper (lower) triangular form, then A−1 is also in upper (lower) triangular form. 40. Suppose B is row equivalent to the n × n invertible matrix A. Show that B is invertible.   a b is 41. Show that if ad − bc = 0, then A = c d not invertible. a. Expand the matrix equation      x1 x2 a b 1 0 = c d 0 1 x3 x4 b. Show the 2 × 2 linear system in the variables x1 and x3 that is generated in part (a) yields d = 0. Similarly, show the system in the variables x2 and x4 yields b = 0. c. Use the results of part (b) to conclude that ad − bc = 0.

Confirming Pages

Chapter 1 Systems of Linear Equations and Matrices

1.5

ß

48

Matrix Equations In this section we show how matrix multiplication can be used to write a linear system in terms of matrices and vectors. We can then write a linear system as a single equation, using a matrix and two vectors, which generalizes the linear equation ax = b for real numbers. As we will see, in some cases the linear system can then be solved using algebraic operations similar to the operations used to solve equations involving real numbers. To illustrate the process, consider the linear system   x − 6y − 4z = −5 2x − 10y − 9z = −4  −x + 6y + 5z = 3 The matrix of coefficients is given by 

 1 −6 −4 A =  2 −10 −9  −1 6 5

Now let x and b be the vectors   x x= y  z

 −5 b =  −4  3 

and

Then the original linear system can be rewritten as Ax = b We refer to this equation as the matrix form of the linear system and x as the vector form of the solution. In certain cases we can find the solution of a linear system in matrix form directly by matrix multiplication. In particular, if A is invertible, we can multiply both sides of the previous equation on the left by A−1 , so that A−1 (Ax) = A−1 b Since matrix multiplication is associative, we have  −1  A A x = A−1 b therefore, x = A−1 b For the example above, the inverse of the matrix    2 1 −6 −4 is A−1 =  − 12 A =  2 −10 −9  −1 6 5 1

3

7

1 2

1 2

0

1

 

Confirming Pages

1.5 Matrix Equations

49

Therefore, the solution to the linear system in vector form is given by      2 3 7 −1 −5 x = A−1 b =  − 12 21 21   −4  =  2  −2 3 1 0 1 That is, x = −1 y=2 and z = −2 We have just seen that if the matrix A has an inverse, then the equation Ax = b has a unique solution. This fact is recorded in Theorem 10.

THEOREM 10

If the n × n matrix A is invertible, then for every vector b, with n components, the linear system Ax = b has the unique solution x = A−1 b.

EXAMPLE 1

Write the linear system in matrix form and solve.  2x + y = 1 −4x + 3y = 2

Solution

The matrix form of the linear system is given by      1 x 2 1 = 2 y −4 3 Notice that since 2(3) − (1)(−4) = 10 = 0, the coefficient matrix is invertible. By Theorem 8, of Sec. 1.4, the inverse is   3 −1 1 10 4 2 Now, by Theorem 10, the solution to the linear system is  x=

1 10

3 4

−1 2



1 2



 =

1 10

1 8





=

1 10

 

8 10

so that x=

DEFINITION 1

1 10

and

y=

8 10

Homogeneous Linear System A homogeneous linear system is a system of the form Ax = 0. The vector x = 0 is always a solution to the homogeneous system Ax = 0, and is called the trivial solution.

Confirming Pages

50

Chapter 1 Systems of Linear Equations and Matrices

EXAMPLE 2

Solution

 1 2 1 A= 1 3 0  1 1 2 Find all vectors x such that Ax = 0.

Let





and

 x1 x =  x2  x3

First observe that x = 0 is one solution. To find the general solution, we row-reduce the augmented matrix     1 2 1 0 1 2 1 0  1 3 0 0   0 1 −1 0  to 1 1 2 0 0 0 0 0 From the reduced matrix we see that x3 is free with x2 = x3 , and x1 = −2x2 − x3 = −3x3 . The solution set in vector form is given by      −3t  t  t ∈ ⺢ . S=    t  Notice that the trivial solution is also included in S as a particular solution with t = 0.

Observe that in Example 2, the coefficient matrix is not row equivalent to I , and hence A is not invertible. If a homogeneous linear system Ax = 0 is such that A is invertible, then by Theorem 10, the only solution is x = 0. In Sec. 1.6 we will show that the converse is also true.

EXAMPLE 3 Solution

Show that if x and y are distinct solutions to the homogeneous system Ax = 0, then x + cy is a solution for every real number c. Using the algebraic properties of matrices, we have that A(x + cy) = A(x) + A(cy) = Ax + cAy = 0+c0 =0 Hence, x + cy is a solution to the homogeneous system.

The result of Example 3 shows that if the homogeneous equation Ax = 0 has two distinct solutions, then it has infinitely many solutions. That is, the homogeneous

Confirming Pages

1.5 Matrix Equations

51

equation Ax = 0 either has one solution (the trivial solution) or has infinitely many solutions. The same result holds for the nonhomogeneous equation Ax = b, with b = 0. To see this, let u and v be distinct solutions to Ax = b and c a real number. Then A(v + c(u − v)) = Av + A(c(u − v)) = Av + cAu − cAv = b + cb − cb = b These observations are summarized in Theorem 11.

THEOREM 11

If A is an m × n matrix, then the linear system Ax = b has no solutions, one solution, or infinitely many solutions.

Fact Summary Let A be an m × n matrix. 1. If A is invertible, then for every n × 1 vector b the matrix equation Ax = b has a unique solution given by x = A−1 b. 2. If A is invertible, then the only solution to the homogeneous equation Ax = 0 is the trivial solution x = 0. 3. If u and v are solutions to Ax = 0, then the vector u + cv is another solution for every scalar c. 4. The linear system Ax = b has a unique solution, infinitely many solutions, or no solution.

Exercise Set 1.5 In Exercises 1–6, find a matrix A and vectors x and b such that the linear system can be written as Ax = b.  2x + 3y = −1 1. −x + 2y = 4  2.

−4x − y = 3 −2x − 5y = 2

  2x − 3y + z = −1 −x − y + 2z = −1 3.  3x − 2y − 2z = 3

  4.

−x  −x

3y − 2z = 2 + 4z = − 3 − 3z = 4

  4x1 + 3x2 − 2x3 − 3x4 = −1 −3x1 − 3x2 + x3 = 4 5.  2x1 − 3x2 + 4x3 − 4x4 = 3   6.

3x2 + x3 − 2x4 = −4 4x2 − 2x3 − 4x4 = 0  x1 + 3x2 − 2x3 = 3

Confirming Pages

52

Chapter 1 Systems of Linear Equations and Matrices

In Exercises 7–12, given the matrix A and vectors x and b, write the equation Ax = b as a linear system.       3 x 2 −5 b= x= 7. A = 2 y 2 1       −1 x −2 4 b= x= 8. A = 1 y 0 3   0 −2 0 9. A =  2 −1 −1  3 −1 2     3 x x =  y b =  1  −1 z   −4 −5 5 10. A =  4 −1 1  −4 3 5     −3 x x =  y b =  2  1 z   2 5 −5 3 11. A = 3 1 −2 −4   x1    x2  2   x= b= x3  0 x4   0 −2 4 −2 0 1 1  12. A =  2 1 0 1 −2     x1 4  x2     x=  x3  b = −3 1 x4 In Exercises 13–16, use the information given to solve the linear system Ax = b.  2 0 −1 4  = 4 1 1 2 4   1 b =  −4  1 

13. A−1

 −4 3 −4 0  = 2 2 1 2 4   2 b= 2  −2 

14. A−1

 −3 −2 0 3  −1 2 −2 3   =  0 1 2 −3  −1 0 3 1   2  −3   b=  2  3 

15. A−1

 3 0 −2 −2  2 0 1 −1   =  −3 −1 −1 1  2 −1 −2 −3   1  −4   b=  1  1 

16. A−1

In Exercises 17–22, solve the linear system by finding the inverse of the coefficient matrix.  17.  18.

x + 4y = 2 3x + 2y = −3 2x − 4y = 4 −2x + 3y = 3

 − z=−1  −x −3x + y − 3z = 1 19.  x − 3y + 2z = 1   −2x − 2y − z = 0 −x − y =−1 20.  − y + 2z = 2

Confirming Pages

1.5 Matrix Equations

 − x1 − x2 − 2x3 + x4 = −1    2x1 + x2 + 2x3 − x4 = 1 21. − 2x1 − x2 − 2x3 − 2x4 = 0    − 2x1 − x2 − x3 − x4 = 0

27. Find a nonzero 3 × 3 matrix A such that the vector   1  −1  1

 + x4 = −3 −x1 − 2x2    −x1 + x2 − 2x3 + x4 = −2 22. −x1 + 2x2 − 2x3 + x4 = 3    − 2x2 + 2x3 − 2x4 = −1

is a solution to Ax = 0.

 1 −1 2 3 Use the inverse matrix to solve the linear system Ax = b for the given vector b.   2 a. b = 1   −3 b. b = 2 

23. Let

A=

24. Let

 −1 0 −1 1 −3  A =  −3 1 −3 2 

Use the inverse matrix to solve the linear system Ax = b for the given vector b.   −2 a. b =  1  1   1 b. b =  −1  0 25. Let

 A=

−1 3 2



−4 12  8

Find a nontrivial solution to Ax = 0.  1 −2 4 A =  2 −4 8  3 −6 12 Find a nontrivial solution to Ax = 0.

26. Let



53

28. Find a nonzero 3 × 3 matrix A such that the vector   −1  2  1 is a solution to Ax = 0. 29. Suppose that A is an n × n matrix and u and v are vectors in ⺢n . Show that if Au = Av and u = v, then A is not invertible. 30. Suppose that u is a solution to Ax = b and that v is a solution to Ax = 0. Show that u + v is a solution to Ax = b. 31. Consider the linear system   2x + y = 1 −x + y = −2  x + 2y = −1 a. Write the linear system in matrix form Ax = b and find the solution. b. Find a 2 × 3 matrix C such that CA = I. (The matrix C is called a left inverse.) c. Show that the solution to the linear system is given by x = Cb. 32. Consider the linear system   2x + y = 3 −x − y = −2  3x + 2y = 5 a. Write the linear system in matrix form Ax = b and find the solution. b. Find a 2 × 3 matrix C such that CA = I. c. Show that the solution to the linear system is given by x = Cb.

Confirming Pages

Chapter 1 Systems of Linear Equations and Matrices

1.6

ß

54

Determinants In Sec. 1.4 we saw that the number ad − bc, associated with the 2 × 2 matrix   a b A= c d has special significance. This number is called the determinant of A and provides useful information about the matrix. In particular, using this terminology, the matrix A is invertible if and only if the determinant is not equal to 0. In this section the definition of the determinant is extended to larger square matrices. The information provided by the determinant has theoretical value and is used in some applications. In practice, however, the computational difficulty in evaluating the determinant of a very large matrix is significant. For this reason the information desired is generally found by using other more efficient methods. 

DEFINITION 1

Determinant of a 2 × 2 Matrix

The determinant of the matrix A =

denoted by |A| or det(A), is given by

  a |A| = det(A) =  c

a c

 b , d

 b  = ad − bc d 

Using this terminology a 2 × 2 matrix is invertible if and only if its determinant is nonzero.

EXAMPLE 1

Solution

Find the determinant of the matrix.     3 1 3 5 a. A = b. A = −2 2 4 2    3 1   = (3)(2) − (1)(−2) = 8 a. |A| =  −2 2     3 5   = (3)(2) − (5)(4) = −14 b. |A| =  4 2     1 0   = (1)(0) − (0)(−3) = 0  c. |A| =  −3 0 

 c. A =

1 0 −3 0



Using the determinant of a 2 × 2 matrix, we now extend this definition to 3 × 3 matrices. DEFINITION 2

Determinant of a 3 × 3 Matrix

The determinant of the matrix  a11 a12 a13 A =  a21 a22 a23  a31 a32 a33 

Confirming Pages

55

1.6 Determinants

is

  a |A| = a11  22 a32

The computation   ∗  |A| = a11  ∗  ∗

   a a23  − a12  21 a33  a31

   a a23  + a13  21 a33  a31

of the 3 × 3 determinant takes the form      ∗  ∗ ∗ ∗  ∗ ∗      a22 a23  − a12  a21 ∗ a23  + a13  a21  a31  a31 ∗ a33  a32 a33  

 a22  a32 

∗ a22 a32

∗ ∗ ∗

      

where the first 2 × 2 determinant is obtained by deleting the first row and first column, the second by deleting the first row and second column, and the third by deleting the first row and third column.

EXAMPLE 2

Find the determinant of the matrix



2 A= 3 5 Solution



+  − +

− + −

 + −  +

Figure 1

 1 −1 1 4  −3 3

By Definition 2, the determinant is given by       3  3 4   1 4  1     + (−1)  − 1 det(A) = |A| = 2    5 −3 5 3 −3 3 = (2) [3 − (−12)] − (1)(9 − 20) + (−1)(−9 − 5) = 30 + 11 + 14 = 55

   

In Example 2, we found the determinant of a 3 × 3 matrix by using an expansion along the first row. With an adjustment of signs the determinant can be computed by using an expansion along any row. The pattern for the signs is shown in Fig. 1. The expansion along the second row is given by        2  2 −1   1 −1  1       − 4 + 1 det(A) = |A| = −3  5 −3  5 3  −3 3  = −3(3 − 3) + (6 + 5) − 4(−6 − 5) = 55 The 2 × 2 determinants in this last equation are found from the original matrix by deleting the second row and first column, the second row and second column, and the second row and the third column, respectively. Expansion along the third row is found in a similar way. In this case            1 −1   − (−3)  2 −1  + 3  2 1  det(A) = |A| = 5      3 1  3 4 1 4 = 5(4 + 1) + 3(8 + 3) + 3(2 − 3) = 55

Confirming Pages

56

Chapter 1 Systems of Linear Equations and Matrices

The determinant can also be computed using expansions along any column in a similar manner. The method used to compute the determinant of a 3 × 3 matrix can be extended to any square matrix. DEFINITION 3

Minors and Cofactors of a Matrix If A is a square matrix, then the minor Mij , associated with the entry aij , is the determinant of the (n − 1) × (n − 1) matrix obtained by deleting row i and column j from the matrix A. The cofactor of aij is Cij = (−1)i+j Mij . For the matrix of Example 2, several minors are      1 4   3 4      M12 =  and M11 =  −3 3  5 3 

M13

  3 =  5

 1  −3 

Using the notation of Definition 3, the determinant of A is given by the cofactor expansion det(A) = a11 C11 + a12 C12 + a13 C13 = 2(−1)2 (15) + 1(−1)3 (−11) − 1(−1)4 (−14) = 30 + 11 + 14 = 55 DEFINITION 4

Determinant of a Square Matrix

If A is an n × n matrix, then

det(A) = a11 C11 + a12 C12 + · · · + a1n C1n =

n 

a1k C1k

k=1

Similar to the situation for 3 × 3 matrices, the determinant of any square matrix can be found by expanding along any row or column. THEOREM 12

Let A be an n × n matrix. Then the determinant of A equals the cofactor expansion along any row or any column of the matrix. That is, for every i = 1, . . . , n and j = 1, . . . , n, det(A) = ai1 Ci1 + ai2 Ci2 + · · · + ain Cin =

n 

aik Cik

k=1

and det(A) = a1j C1j + a2j C2j + · · · + anj Cnj =

n 

akj Ckj

k=1

For certain square matrices the computation of the determinant is simplified. One such class of matrices is the square triangular matrices. DEFINITION 5

Triangular Matrices An m × n matrix is upper triangular if aij = 0, for all i > j , and is lower triangular if aij = 0, for all i < j. A square matrix is a diagonal matrix if aij = 0, for all i = j.

Revised Confirming Pages

1.6 Determinants

Some examples of upper triangular matrices are   
2 −1 0 1 1  0 0 3  and 0 2 0 0 2

 1 1 0 1  0 0 0 1  0 0 1 1 

and some examples of lower triangular matrices are


THEOREM 13

1 0 1 1



 2 0 0  0 1 0  1 0 2

57





1  0   1 0

and

0 0 3 1

0 0 1 2

 0 0   0  1

If A is an n × n triangular matrix, then the determinant of A is the product of the terms on the diagonal. That is, det(A) = a11 · a22 · · · ann Proof We present the proof for an upper triangular matrix. The proof for a lower triangular matrix is identical. The proof is by induction on n. If n = 2, then det(A) = a11 a22 − 0 and hence is the product of the diagonal terms. Assume that the result holds for an n × n triangular matrix. We need to show that the same is true for an (n + 1) × (n + 1) triangular matrix A. To this end let   a11 a12 a13 · · · a1n a1,n+1  0 a22 a23 · · · a2n a2,n+1     0 0 a33 · · · a3n a3,n+1    A= .  .. .. .. .. ..  ..  . . . . .    0 an,n+1  0 0 · · · ann 0 0 0 · · · 0 an+1,n+1 Using the cofactor expansion along row n + 1, we a11 0 (n+1)+(n+1) det(A) = (−1) an+1,n+1 0 .. . 0

have a12 a22 0 .. .

a13 a23 a33 .. .

0

0

··· ··· ··· .. .

a1n a2n a3n .. .

· · · ann



Since the determinant on the right is n × n and upper triangular, by the inductive hypothesis det(A) = (−1)2n+2 (an+1,n+1 )(a11 a22 · · · ann ) = a11 a22 · · · ann an+1,n+1

Properties of Determinants Determinants for large matrices can be time-consuming to compute, so any properties of determinants that reduce the number of computations are useful. Theorem 14 shows how row operations affect the determinant.

Confirming Pages

58

Chapter 1 Systems of Linear Equations and Matrices

THEOREM 14

Let A be a square matrix. 1. If two rows of A are interchanged to produce a matrix B, then det(B) = − det(A). 2. If a multiple of one row of A is added to another row to produce a matrix B, then det(B) = det(A). 3. If a row of A is multiplied by a real number α to produce a matrix B, then det(B) = αdet(A). Proof (1) The proof is by induction on n. For the case n = 2 let   a b A= c d Then det(A) = ad − bc. If the two rows of A are interchanged to give the matrix   c d B= a b then det(B) = bc − ad = − det(A). Assume that the result holds for n × n matrices and A is an (n + 1) × (n + 1) matrix. Let B be the matrix obtained by interchanging rows i and j of A. Expanding the determinant of A along row i and of B along row j, we have det(A) = ai1 Ci1 + ai2 Ci2 + · · · + ain Cin and det(B) = aj 1 Dj 1 + aj 2 Dj 2 + · · · + aj n Dj n = ai1 Dj 1 + ai2 Dj 2 + · · · + ain Dj n where Cij and Dij are the cofactors of A and B, respectively. To obtain the result there are two cases. If the signs of the cofactors Cij and Dij are the same, then they differ by one row interchanged. If the signs of the cofactors Cij and Dij are opposite, then they differ by two rows interchanged. In either case, by the inductive hypothesis, we have det(B) = − det(A) The proofs of parts 2 and 3 are left as exercises. We note that in Theorem 14 the same results hold for the similar column operations. To highlight the usefulness of this theorem, recall that by Theorem 13, the determinant of a triangular matrix is the product of the diagonal entries. So an alternative approach to finding the determinant of a matrix A is to row-reduce A to triangular form and apply Theorem 14 to record the effect on the determinant. This method is illustrated in Example 3.

EXAMPLE 3

Find the determinant of the matrix 

 0 1 3 −1  2 4 −6 1   A=  0 3 9 2  −2 −4 1 −3

Confirming Pages

1.6 Determinants

Solution

Since column 1 has two zeros, an expansion along this column will involve the fewest computations. Also by Theorem 14, if row 2 is added to row 4, then the determinant is unchanged and    0 1 3 −1    2 4 −6 1  det(A) =  9 2   0 3  0 0 −5 −2  Expansion along the first column gives

   1 3 −1   9 2  det(A) = −2  3  0 −5 −2 

We next perform the operation −3R1 + R2 −→ R2 , leaving the determinant again unchanged, so that    1 3 −1   0 5  det(A) = −2  0  0 −5 −2  Now, interchanging the second and third rows gives    1 3 −1   det(A) = (−2)(−1)  0 −5 −2   0 0 5  This last matrix is triangular, thus by Theorem 13, det(A) = (−2)(−1)[(1)(−5)(5)] = −50

Theorem 15 lists additional useful properties of the determinant.

THEOREM 15

Let A and B be n × n matrices and α a real number. 1. The determinant computation is multiplicative. That is, det(AB) = det(A) det(B) 2. 3. 4. 5. 6.

det(αA) = αn det(A) det(At ) = det(A) If A has a row (or column) of all zeros, then det(A) = 0. If A has two equal rows (or columns), then det(A) = 0. If A has a row (or column) that is a multiple of another row (or column), then det(A) = 0.

59

Confirming Pages

60

Chapter 1 Systems of Linear Equations and Matrices



EXAMPLE 4 Solution

Let A =

1 2 3 −2



 and B =

1 1

−1 4

In this case the product is

 . Verify Theorem 15, part 1.

 3 7 1 −11 so that det(AB) = −33 − 7 = −40. We also have det(A) det(B) = (−8)(5) = −40. 

AB =

Properties of the determinant given in Theorem 15 can be used to establish the connection between the determinant and the invertibility of a square matrix. THEOREM 16

A square matrix A is invertible if and only if det(A) = 0. Proof If the matrix A is invertible, then by Theorem 15, 1 = det(I ) = det(AA−1 ) = det(A) det(A−1 ) Since the product of two real numbers is zero if and only if at least one of them is zero, we have det(A) = 0 [also det(A−1 ) = 0]. To establish the converse, we will prove the contrapositive statement. Assume that A is not invertible. By the remarks at the end of Sec. 1.4, the matrix A is row equivalent to a matrix R with a row of zeros. Hence, by Theorem 14, there is some real number k = 0 such that det(A) = k det(R), and therefore by Theorem 15, part 4, det(A) = k det(R) = k(0) = 0

COROLLARY 1

Let A be an invertible matrix. Then det(A−1 ) =

1 det(A)

Proof If A is invertible, then as in the proof of Theorem 16, det(A) = 0, det(A−1 ) = 0, and det(A) det(A−1 ) = 1 Therefore, det(A−1 ) =

1 det(A)

The final theorem of this section summarizes the connections between inverses, determinants, and linear systems. THEOREM 17

Let A be a square matrix. Then the following statements are equivalent. 1. The matrix A is invertible. 2. The linear system Ax = b has a unique solution for every vector b.

Confirming Pages

1.6 Determinants

61

3. The homogeneous linear system Ax = 0 has only the trivial solution. 4. The matrix A is row equivalent to the identity matrix. 5. The determinant of the matrix A is nonzero.

The graph of the equation (x − h)2 (y − k)2 + =1 a2 b2 is an ellipse with center (h, k), horizontal axis of length 2a, and vertical axis of length 2b.

EXAMPLE 5

Solution

Determinants can be used to find the equation of a conic section passing through specified points. In the 17th century, Johannes Kepler’s observations of the orbits of planets about the sun led to the conjecture that these orbits are elliptical. It was Isaac Newton who, later in the same century, proved Kepler’s conjecture. The graph of an equation of the form Ax 2 + Bxy + Cy 2 + Dx + Ey + F = 0 is a conic section. Essentially, the graphs of conic sections are circles, ellipses, hyperbolas, or parabolas.

An astronomer who wants to determine the approximate orbit of an object traveling about the sun sets up a coordinate system in the plane of the orbit with the sun at the origin. Five observations of the location of the object are then made and are approximated to be (0, 0.31), (1, 1), (1.5, 1.21), (2, 1.31), and (2.5, 1). Use these measurements to find the equation of the ellipse that approximates the orbit. We need to find the equation of an ellipse in the form Ax 2 + Bxy + Cy 2 + Dx + Ey + F = 0 Each data point must satisfy this equation; for example, since the point (2, 1.31) is on the graph of the conic section, A(2)2 + B(2)(1.31) + C(1.31)2 + D(2) + E(1.31) + F = 0 so 4A + 2.62B + 1.7161C + 2D + 1.31E + F = 0 Substituting the five points in the general equation, we obtain the 5 × 6 linear system (with coefficients rounded to two decimal places)      

0.1C + 0.31E + F A+ B+ C+ D+ E+F 4A + 2.62B + 1.72C + 2D + 1.31E + F   2.25A + 1.82B + 1.46C + 1.5D + 1.21E + F    6.25A + 2.5B + C + 2.5D + E+F

=0 =0 =0 =0 =0

Since the equation Ax 2 + Bxy + Cy 2 + Dx + Ey + F = 0 describing the ellipse passing through the five given points has infinitely many solutions, by Theorem 17, we have

Revised Confirming Pages

62

Chapter 1 Systems of Linear Equations and Matrices




x2 xy y2 x y 1



0 0 0.1 0 0.31 1



1 1 1 1 1 1


=0
4 2.62 1.72 2 1.31 1



2.25 1.82 1.46 1.5 1.21 1



6.25 2.5 1 2.5 1 1
Expanding the determinant gives us the equation

y

x

−0.014868x 2 + 0.0348xy − 0.039y 2 + 0.017238x − 0.003y + 0.00483 = 0 The graph of the orbit is shown in Fig. 2.

Figure 2

Cramer’s Rule Determinants can also be used to solve linear systems. To illustrate the technique consider the 2 × 2 linear system  ax + by = u cx + dy = v with ad − bc
= 0. By Theorem 17, the linear system has a unique solution. To eliminate the variable y, we multiply the first equation by d and the second equation by b, and then we subtract the two equations. This gives adx + bdy − (bcx + bdy) = du − bv Simplifying, we have (ad − bc)x = du − bv

so that

x=

Using a similar procedure, we can solve for y. av − cu y= ad − bc Using determinants, we can write the solution as



a
u b




c
v d

and y =

x =



a b

a
c d

c

du − bv ad − bc

u v b d











Notice that the solutions for x and y are similar. The denominator for each is the determinant of the coefficient matrix. The determinant in the numerator for x is formed by replacing the first column of the coefficient matrix with the column of constants on the right-hand side of the linear system. The determinant in the numerator for y is formed by replacing the second column of the coefficient matrix with the column of constants. This method of solving a linear system is called Cramer’s rule.

Confirming Pages

1.6 Determinants

EXAMPLE 6

63

Use Cramer’s rule to solve the linear system. 2x + 3y = 2 −5x + 7y = 3

Solution

The determinant of the coefficient matrix is given by    2 3     −5 7  = 14 − (−15) = 29 and since the determinant is not zero, the system has a unique solution. The solution is given by      2 3   2 2       3 7   −5 3  14 − 9 5 6 − (−10) 16 x= = = and y= = = 29 29 29 29 29 29

THEOREM 18

Cramer’s Rule Let A be an n × n invertible matrix, and let b be a column vector with n components. Let Ai be the matrix obtained by replacing the ith column of A   x1  x2    with b. If x =  .  is the unique solution to the linear system Ax = b, then  ..  xn xi =

det(Ai ) det(A)

for

i = 1, 2, . . . , n

Proof Let Ii be the matrix obtained by replacing the ith column of the identity matrix with x. Then the linear system is equivalent to the matrix equation AIi = Ai

so

det(AIi ) = det(Ai )

By Theorem 15, part 1, we have det(A) det(Ii ) = det(AIi ) = det(Ai ) Since A is invertible, det(A) = 0 and hence det(Ai ) det(Ii ) = det(A) Expanding along the ith row to find the determinant of Ii gives det(Ii ) = xi det(I ) = xi where I is the (n − 1) × (n − 1) identity. Therefore, xi =

det(Ai ) det(A)

Confirming Pages

64

Chapter 1 Systems of Linear Equations and Matrices

If a unique solution exists, then Cramer’s rule can be used to solve larger square linear systems. Example 7 illustrates the technique for a 3 × 3 system.

EXAMPLE 7

Solve the linear system.

  

2x + 3y − z = 2 3x − 2y + z = −1   −5x − 4y + 2z = 3 Solution

The determinant of the coefficient matrix is given by    2 3 −1    3 −2 1  = −11   −5 −4 2  By Cramer’s rule the solution to the system is    2 3 −1  1  5 1  = − x = −  −1 −2 11  3 −4 11 2     2 2 −1  36 1  1  = y = −  3 −1 11  −5 3 2  11    2 3 2  76 1  z = −  3 −2 −1  = 11  −5 −4 3  11 The reader should verify this solution by substitution into the original system.

Fact Summary Let A and B be n × n matrices.   a b = ad − bc. 1. det c d 2. The determinant of A can be computed by expanding along any row or column provided that the signs are adjusted using the pattern   + − + − ···  − + − + ···     + − + − ···     − + − + ···    .. .. .. .. . . . . . . . 3. The matrix A is invertible if and only if det(A) = 0.

Confirming Pages

1.6 Determinants

65

4. If A is a triangular matrix, then the determinant of A is the product of the diagonal terms. 5. If two rows of A are interchanged, the determinant of the resulting matrix is the negative of the determinant of A. 6. If a multiple of one row of A is added to another row, the determinant is not altered. 7. If one row of A is multiplied by a scalar c, the determinant of the resulting matrix is c times the determinant of A. 8. det(AB) = det(A) det(B), det(cA) = cn det(A), det(At ) = det(A) 9. If A has a row or column of zeros, then det(A) = 0. 10. If one row or column of A is a multiple of another row or column, then det(A) = 0. 1 11. If A is invertible, then det(A−1 ) = det(A) .

Exercise Set 1.6 In Exercises 1–4, evaluate the determinant of the matrix by inspection.   2 −40 10 3 12  1.  0 0 0 4   1 2 3 2.  4 5 6  1 2 3   1 0 0 0  3 −1 0 0   3.   4 2 2 0  1 1 6 5   1 −1 2 4  4.  2 −2 1 2 −1 In Exercises 5–8, use determinants to decide if the matrix is invertible.   2 −1 5. −2 2   1 3 6. 5 −2

 1 0 0 0  7.  3 6 0 8 −1   7 2 1 8.  7 2 1  3 6 6 

9. Answer the questions using the matrix   2 0 1 4  A =  3 −1 −4 1 −2 a. Find the determinant of the matrix by using an expansion along row 1. b. Find the determinant of the matrix by using an expansion along row 2. c. Find the determinant of the matrix by using an expansion along column 2. d. Interchange rows 1 and 3 of the matrix, and find the determinant of the transformed matrix. e. Multiply row 1 of the matrix found in part (d) by −2, and find the determinant of the new matrix. Use the value to find the determinant of the original matrix.

Confirming Pages

66

Chapter 1 Systems of Linear Equations and Matrices

f. Replace row 3 of the matrix found in part (e) with the sum of row 3 and −2 times row 1 and find the determinant of the new matrix in two ways. First, use an expansion along row 3 of the new matrix. Second, use the value for the determinant of A that has already been calculated. g. Does the matrix A have an inverse? Do not try to compute the inverse. 10. Answer the questions using  −1 1  3 −2 A=  0 1 3 3

the matrix  1 2 0 −1   0 1  3 3

a. Find the determinant of the matrix by using an expansion along row 4. b. Find the determinant of the matrix by using an expansion along row 3. c. Find the determinant of the matrix by using an expansion along column 2. d. In (a), (b), and (c), which computation do you prefer, and why? e. Does the matrix A have an inverse? Do not try to compute the inverse. In Exercises 11–26, find the determinant of the matrix. Specify whether the matrix has an inverse without trying to compute the inverse.   5 6 11. −8 −7   4 3 12. 9 2   −1 −1 13. −11 5   1 2 14. 4 13   −1 4 15. 1 −4   1 1 16. 2 2

 5 −5 −4 5  17.  −1 −3 −3 1 3   3 −3 5 4 −3  18.  2 −3 −1 −5   −3 4 5 1 4  19.  1 −1 −3 4   −2 −2 −4 1 3  20.  1 −4 0 4   1 −4 1 21.  1 −2 4  0 2 3   1 2 4 22.  4 0 0  1 2 4  2 −2 −2 −2  −2 2 3 0 23.   −2 −2 2 0 1 −1 −3 −1  1 −1 0 0  −3 −3 −1 −1 24.   −1 −1 −3 2 −1 −2 2 1  −1 1 1 0  0 0 −1 0  0 0 1 −1 25.    0 1 1 0 1 −1 1 1  1 0 −1 0  −1 −1 0 0  1 0 0 0 26.    0 1 1 1 −1 1 1 −1 

In Exercises 27–30, let



       

a A= d g and assume det(A) = 10.

0 0 0 1 0

     

−1 −1 −1 0 0

     

 b c e f  h i

Confirming Pages

1.6 Determinants

35. Answer the questions about the linear system.   x − y − 2z = 3 −x + 2y + 3z = 1  2x − 2y − 2z = −2

27. Find det(3A). 28. Find det(2A−1 ). ! " 29. Find det (2A)−1 . 

30. Find

67

a det  b c 31. Find x, assuming 

x2 det  2 0

32. Find the determinant of  1 1  0 1   1 0   1 1 1 1

g h i

 d e  f

 x 2 1 1 =0 0 −5 the matrix  1 1 1 1 1 1   1 1 1   0 1 1  1 0 1

33. Suppose a1 = b1 . Describe the set of all points (x, y) that satisfy the equation   1 1 1 det  x a1 b1  = 0 y a2 b2 34. Use the three systems to answer the questions.   x+ y=3 x+ y=3 (1) (2) 2x + 2y = 1 2x + 2y = 6  x+ y=3 (3) 2x − 2y = 1 a. Form the coefficient matrices A, B, and C, respectively, for the three systems. b. Find det(A), det(B), and det(C). How are they related? c. Which of the coefficient matrices have inverses? d. Find all solutions to system (1). e. Find all solutions to system (2). f. Find all solutions to system (3).

a. Form the coefficient matrix A for the linear system. b. Find det(A). c. Does the system have a unique solution? Explain. d. Find all solutions to the system. 36. Answer the questions about the linear system.   x + 3y − 2z = −1 2x + 5y + z = 2  2x + 6y − 4z = −2 a. Form the coefficient matrix A for the linear system. b. Find det(A). c. Does the system have a unique solution? Explain. d. Find all solutions to the system. 37. Answer the questions about the linear system.  − z=−1  −x 2x + 2z = 1  x − 3y − 3z = 1 a. Form the coefficient matrix A for the linear system. b. Find det(A). c. Does the system have a unique solution? Explain. d. Find all solutions for the system. In Exercises 38–43, use the fact that the graph of the general equation Ax 2 + Bxy + Cy 2 + Dx + Ey + F = 0 is essentially a parabola, circle, ellipse, or hyperbola. 38. a. Find the equation of the parabola in the form Ax 2 + Dx + Ey + F = 0 that passes through the points (0, 3), (1, 1), and (4, −2). b. Sketch the graph of the parabola.

Confirming Pages

68

Chapter 1 Systems of Linear Equations and Matrices

39. a. Find the equation of the parabola in the form Cy + Dx + Ey + F = 0 2

that passes through the points (−2, −2), (3, 2), and (4, −3). b. Sketch the graph of the parabola. 40. a. Find the equation of the circle in the form A(x 2 + y 2 ) + Dx + Ey + F = 0 that passes through the points (−3, −3), (−1, 2), and (3, 0). b. Sketch the graph of the circle. 41. a. Find the equation of the hyperbola in the form Ax 2 + Cy 2 + Dx + Ey + F = 0 that passes through the points (0, −4), (0, 4), (1, −2), and (2, 3). b. Sketch the graph of the hyperbola. 42. a. Find the equation of the ellipse in the form Ax 2 + Cy 2 + Dx + Ey + F = 0 that passes through the points (−3, 2), (−1, 3), (1, −1), and (4, 2). b. Sketch the graph of the ellipse. 43. a. Find the equation of the ellipse in the form Ax 2 + Bxy + Cy 2 + Dx + Ey + F = 0 that passes through the points (−1, 0), (0, 1), (1, 0), (2, 2), and (3, 1). b. Sketch the graph of the ellipse.

ß

1.7

In Exercises 44–51, use Cramer’s rule to solve the linear system.  2x + 3y = 4 44. 2x + 2y = 4  5x − 5y = 7 45. 2x − 3y = 6  2x + 5y = 4 46. 4x + y = 3  −9x − 4y = 3 47. −7x + 5y = −10  −10x − 7y = −12 48. 12x + 11y = 5  −x − 3y = 4 49. −8x + 4y = 3   −2x + y − 4z = −8 − 4y + z = 3 50.  4x − z = −8   2x + 3y + 2z = −2 −x − 3y − 8z = −2 51.  −3x + 2y − 7z = 2 52. An n × n matrix is skew-symmetric provided At = −A. Show that if A is skew-symmetric and n is an odd positive integer, then A is not invertible. 53. If A is a 3 × 3 matrix, show that det(A) = det(At ). 54. If A is an n × n upper triangular matrix, show that det(A) = det(At ).

Elementary Matrices and LU Factorization In Sec. 1.2 we saw how the linear system Ax = b can be solved by using Gaussian elimination on the corresponding augmented matrix. Recall that the idea there was to use row operations to transform the coefficient matrix to row echelon form. The upper triangular form of the resulting matrix made it easy to find the solution by using back substitution. (See Example 1 of Sec. 1.2.) In a similar manner, if an augmented matrix is reduced to lower triangular form, then forward substitution can be used to find the solution of the corresponding linear system. For example, starting from the

Confirming Pages

1.7 Elementary Matrices and LU Factorization

69

first equation of the linear system  = 3  x1 −x1 + x2 = −1  2x1 − x2 + x3 = 5 we obtain the solution x1 = 3, x2 = 2, and x3 = 1. Thus, from a computational perspective, to find the solution of a linear system, it is desirable that the corresponding matrix be either upper or lower triangular. In this section we show how, in certain cases, an m × n matrix A can be written as A = LU, where L is a lower triangular matrix and U is an upper triangular matrix. We call this an LU factorization of A. For example, an LU factorization of the   −3 −2 is given by matrix 3 4      3 2 −1 0 −3 −2 = 0 1 1 2 3 4     3 2 −1 0 . We also show in this section that when such and U = with L = 0 1 1 2 a factorization of A exists, a process that involves both forward and back substitution can be used to find the solution to the linear system Ax = b.

Elementary Matrices As a first step we describe an alternative method for carrying out row operations using elementary matrices.

DEFINITION 1

Elementary Matrix An elementary matrix is any matrix that can be obtained from the identity matrix by performing a single elementary row operation.

As an illustration, the elementary matrix E1 is formed by interchanging the first and third rows of the 3 × 3 identity matrix I , that is,   0 0 1 E1 =  0 1 0  1 0 0 Corresponding to the three row operations given in Theorem 2 of Sec. 1.2, there are three types of elementary matrices. For example, as we have just seen, E1 is derived from I by means of the row operation R1 ↔ R3 which interchanges the first and third rows. Also, the row operation kR1 + R2 −→ R2 applied to I yields the elementary matrix   1 0 0 E2 =  k 1 0  0 0 1

Confirming Pages

70

Chapter 1 Systems of Linear Equations and Matrices

Next, if c = 0, the row operation cR2 −→ R2  1 0 E3 =  0 c 0 0

performed on I produces the matrix  0 0  1

Using any row operation, we can construct larger elementary matrices from larger identity matrices in a similar manner. We now show how elementary matrices can be used to perform row operations. To illustrate the process, let A be the 3 × 3 matrix given by   1 2 3 A= 4 5 6  7 8 9 Multiplying A by the matrix E1 , defined above,  7 8 E1 A =  4 5 1 2

we obtain  9 6  3

Observe that E1 A is the result of interchanging the first and third rows of A. Theorem 19 gives the situation in general.

THEOREM 19

Let A be an m × n matrix and E the elementary matrix obtained from the m × m identity matrix I by a single row operation R. Denote by R(A) the result of performing the row operation on A. Then R(A) = EA. By repeated application of Theorem 19, a sequence of row operations can be performed on a matrix A by successively multiplying A by the corresponding elementary matrices. Specifically, let Ei be the elementary matrix corresponding to the row operation Ri with 1 ≤ i ≤ k. Then Rk · · · R2 (R1 (A)) = Ek · · · E2 E1 A

EXAMPLE 1

Solution

 1 2 −1 0  A= 3 5 −1 1 1 Use elementary matrices to perform the row operations R1 : R2 − 3R1 −→ R2 , R2 : R3 + R1 −→ R3 , and R3 : R3 + 3R2 −→ R3 . Let A be the matrix



The elementary matrices corresponding to these    1 0 0 1 0 E2 =  0 1 E1 =  −3 1 0  0 0 1 1 0

row operations are given   0 1 0 0  E3 =  0 1 1 0 3

by

 0 0  1

Confirming Pages

1.7 Elementary Matrices and LU Factorization

respectively, so that

71

 1 2 −1 3  E3 E2 E1 A =  0 −1 0 0 9 The reader should check that the matrix on the right-hand side is equal to the result of performing the row operations above on A in the order given. 

The Inverse of an Elementary Matrix An important property of elementary matrices is that they are invertible.

THEOREM 20

Let E be an n × n elementary matrix. Then E is invertible. Moreover, its inverse is also an elementary matrix. Proof Let E be an elementary matrix. To show that E is invertible, we compute its determinant and apply Theorem 17 of Sec. 1.6. There are three cases depending on the form of E. First, if E is derived from I by an interchange of two rows, then det(E) = − det(I ) = −1. Second, if E is the result of multiplying one row of I by a nonzero scalar c, then det(E) = c det(I ) = c = 0. Third, if E is formed by adding a multiple of one row of I to another row, then det(E) = det(I ) = 1. In either case, det(E) = 0 and hence E is invertible. To show that E −1 is an elementary matrix, we use the algorithm of Sec. 1.4 to compute the inverse. In this case starting with the n × 2n augmented matrix [E | I ] we reduce the elementary matrix on the left (to I ) by applying the reverse operation used to form E, obtaining ! " I | E −1 That E −1 is also an elementary matrix follows from the fact that the reverse of each row operation is also a row operation.

As an illustration of the above theorem, let R be the row operation 2R2 + R1 −→ R1 , which says to add 2 times row 2 to row 1. The corresponding elementary matrix is given by   1 2 0 E= 0 1 0  0 0 1 Since det(E) = 1, then E is invertible with   1 −2 0 1 0  E −1 =  0 0 0 1

Confirming Pages

72

Chapter 1 Systems of Linear Equations and Matrices

Observe that E −1 corresponds to the row operation R2 : −2R2 + R1 −→ R1 which says to subtract 2 times row 2 from row 1, reversing the original row operation R. Recall from Sec. 1.2 that an m × n matrix A is row equivalent to an m × n matrix B if B can be obtained from A by a finite sequence of row operations. Theorem 21 gives a restatement of this fact in terms of elementary matrices. THEOREM 21

Let A and B be m × n matrices. The matrix A is row equivalent to B if and only if there are elementary matrices E1 , E2 , . . . , Ek such that B = Ek Ek−1 · · · E2 E1 A.

In light of Theorem 21, if A is row equivalent to B, then B is row equivalent to A. Indeed, if A is row equivalent to B, then B = Ek Ek−1 · · · E2 E1 A for some elementary matrices E1 , E2 , . . . , Ek . Successively multiplying both sides of −1 this equation by Ek−1 , Ek−1 , . . . , and E1−1 , we obtain −1 Ek−1 B A = E1−1 · · · Ek−1

Since each of the matrices E1−1 , E2−1 , . . . , Ek−1 is an elementary matrix, B is row equivalent to A. Theorem 22 uses elementary matrices to provide a characterization of invertible matrices. THEOREM 22

An n × n matrix A is invertible if and only if it can be written as the product of elementary matrices. Proof First assume that there are elementary matrices E1 , E2 , . . . , Ek such that A = E1 E2 · · · Ek−1 Ek We claim that the matrix B = Ek−1 · · · E2−1 E1−1 is the inverse of A. To show this, we multiply both sides of A = E1 E2 · · · Ek−1 Ek by B to obtain BA = (Ek−1 · · · E2−1 E1−1 )A = (Ek−1 · · · E2−1 E1−1 )(E1 E2 · · · Ek−1 Ek ) = I establishing the claim. On the other hand, suppose that A is invertible. In Sec. 1.4, we showed that A is row equivalent to the identity matrix. So by Theorem 21, there are elementary matrices E1 , E2 , . . . , Ek such that I = Ek Ek−1 · · · E2 E1 A. Consequently, A = E1−1 · · · Ek−1 I. Since E1−1 , . . . , Ek−1 and I are all elementary matrices, A is the product of elementary matrices as desired.

LU Factorization There are many reasons why it is desirable to obtain an LU factorization of a matrix. For example, suppose that A is an m × n matrix and bi , with 1 ≤ i ≤ k, is a collection of vectors in ⺢n , which represent outputs for the linear systems Ax = bi . Finding input vectors xi requires that we solve k linear systems. However, since the matrix A is the

Confirming Pages

1.7 Elementary Matrices and LU Factorization

A=

0 0 Figure 1

73

same for each linear system, the process is greatly simplified if A is replaced with its LU factorization. The details for solving a linear system using an LU factorization are presented later in this section. If A is an n × n matrix with an LU factorization given by A = LU, then L and U are also n × n. See Fig. 1. Then by Theorem 13 of Sec. 1.6, the determinant of A is given by det(A) = (
11 · · ·
nn )(u11 · · · unn ) where
ii and uii are the diagonal entries of L and U, respectively. If this determinant is not zero, then by Theorem 9 of Sec. 1.4 the inverse of the matrix A is given by A−1 = (LU )−1 = U −1 L−1 To describe the process of obtaining an LU factorization of an m × n matrix A, suppose that A can be reduced to an upper triangular matrix by a sequence of row operations which correspond to lower triangular elementary matrices. That is, there exist lower triangular elementary matrices L1 , L2 , . . . , Lk such that Lk Lk−1 · · · L1 A = U Since each of the matrices Li with 1 ≤ i ≤ k is invertible, we have −1 −1 A = L−1 1 L2 · · · Lk U −1 −1 By Theorem 20, L−1 1 , L2 , . . . , and Lk are elementary matrices. They are also lower −1 −1 triangular. Now let L = L1 L2 · · · L−1 k . Observe that L is lower triangular as it is the product of lower triangular matrices. The desired factorization is thus given by A = LU .

EXAMPLE 2

Find an LU factorization of the matrix 

3 A= 6 −1

Solution

 6 −3 15 −5  −2 6

Observe that A can be row-reduced to an upper triangular matrix by means of the row operations R1 : 13 R1 −→ R1 , R2 : −6R1 + R2 −→ R2 , and R3 : R1 + R3 −→ R3 . The corresponding elementary matrices are therefore given by     1   1 0 0 1 0 0 0 0 3 E3 =  0 1 0  E2 =  −6 1 0  E1 =  0 1 0  0 0 1 1 0 1 0 0 1 respectively, so that  1 E3 E2 E1 A =  0 1  1 = 0 0

 1  1 0 0 0 0 3 1 0   −6 1 0   0 0 0 1 0 1 0  2 −1 3 1 =U 0 5

 3 0 0 1 0  6 −1 0 1

 6 −3 15 −5  −2 6

Confirming Pages

74

Chapter 1 Systems of Linear Equations and Matrices

An LU factorization of A is then given by A =    3 0 3 6 −3  6 15 −5  =  6 1 −1 0 −1 −2 6

$ # E1−1 E2−1 E3−1 U, so that   1 2 −1 0 1  0  0 3 0 0 5 1

From the remarks preceding Example 2, we see that to use the above procedure with success, there are limitations on the matrix A. Specifically, A must be reducible to upper triangular form without any row interchanges. This will ensure that the elementary matrices used in the elimination process will all be lower triangular. Theorem 23 summarizes these results.

THEOREM 23

Let A be an m × n matrix that can be reduced to the upper triangular matrix U without row interchanges by means of the m × m lower triangular matrices L1 , −1 −1 L2 , . . . Lk . If L = L−1 1 L2 · · · Lk , then A has the LU factorization A = LU.

A simple example of a matrix that cannot be reduced to upper triangular form   0 1 . This matrix does not have an LU without interchanges is given by P = 1 0 factorization. (See Exercise 29.) As stated in Theorem 23, it is not necessary for A to be square, as shown in Example 3.

EXAMPLE 3

Solution

Find an LU factorization of the matrix   1 −3 −2 0 1 −1  A =  1 −2 2 −4 3 2

 −3 −2 0 1 3 −1  0 1 4



1 Observe that A can be reduced to the upper triangular matrix U = 0 0 by means of the elementary matrices      1 0 0 1 0 0 E1 =  −1 1 0  E2 =  0 1 0  E3 =  0 0 1 −2 0 1

1 0 1

 0 0 1 0  −2 1

Confirming Pages

1.7 Elementary Matrices and LU Factorization

75

Hence,

$ # A = E1−1 E2−1 E3−1 U   1 0 0 =  1 1 0   0 0 1   1 1 0 0 =  1 1 0  0 0 2 2 1

 1 1 0 0 0 1 0  0 0 2 0 1 −3 −2 0 1 3 −1 0 1 4

  1 −3 −2 0 0 1 3 1 0   0 0 0 1 2 1 

 0 −1  4

 = LU

Solving a Linear System Using LU Factorization We now turn our attention to the process of solving a linear system by using an LU factorization. To illustrate the procedure, consider the linear system Ax = b with   3 b =  11  and A the matrix of Example 2. By using the LU factorization of A 9 found in Example 2,      x1 3 6 −3 3 Ax =  6 15 −5   x2  =  11  −1 −2 6 9 x3 can be written equivalently  3 LU x =  6 −1

as

     x1 1 2 −1 0 0 3 1   x2  =  11  1 0  0 3 0 0 5 0 1 9 x3   y1 To solve this equation efficiently, we define the vector y =  y2  by the equation y3 U x = y, so that      x1 y1 1 2 −1  0 3 1   x2  =  y2  0 0 5 x3 y3

Making this substitution in the  3  6 −1

linear system L(U x) = b gives     y1 3 0 0 1 0   y2  =  11  9 0 1 y3

Confirming Pages

76

Chapter 1 Systems of Linear Equations and Matrices

Using forward substitution, we solve the system Ly = b for y, obtaining y1 = 1, y2 = 5, and y3 = 10. Next we solve the linear system U x = y. That is,      x1 1 1 2 −1  0 3 1   x2  =  5  10 0 0 5 x3 Using back substitution, we obtain x3 = 2, x2 = 1, and x1 = 1. The following steps summarize the procedure for solving the linear system Ax = b when A admits an LU factorization. 1. 2. 3. 4.

Use Theorem 23 to write the linear system Ax = b as L(U x) = b. Define the vector y by means of the equation U x = y. Use forward substitution to solve the system Ly = b for y. Use back substitution to solve the system U x = y for x. Note that x is the solution to the original linear system.

PLU Factorization We have seen that a matrix A has an LU factorization provided that it can be rowreduced without interchanging rows. We conclude this section by noting that when row interchanges are required to reduce A, a factorization is still possible. In this case the matrix A can be factored as A = P LU, where P is a permutation matrix, that is, a matrix that results from interchanging rows of the identity matrix. As an illustration, let   0 2 −2 3  A= 1 4 1 2 0 The matrix A can be reduced to



1 2 U = 0 2 0 0

 0 3  −5

by means of the row operations R1 : R1 ↔ R3 , R2 : −R1 + R2 −→ R2 , and R3 : −R2 + R3 −→ R3 . The corresponding elementary matrices are given by       0 0 1 1 0 0 1 0 0 1 0  E2 =  −1 1 0  and E3 =  0 E1 =  0 1 0  1 0 0 0 0 1 0 −1 1

Confirming Pages

1.7 Elementary Matrices and LU Factorization

Observe that the elementary matrix E1 is lower triangular. Hence, # $ A = E1−1 E2−1 E3−1 U   1 0 0 1 =  0 1 0  1 0 1 0 0 = P LU

77

a permutation matrix while E2 and E3 are

  1 2 0 0 0 3  1 0  0 2 0 0 −5 1 1

Fact Summary 1. A row operation on a matrix A can be performed by multiplying A by an elementary matrix. 2. An elementary matrix is invertible, and the inverse is an elementary matrix. 3. An n × n matrix A is invertible if and only if it is the product of elementary matrices. 4. An m × n matrix A has an LU factorization if it can be reduced to an upper triangular matrix with no row interchanges. 5. If A = LU, then L is invertible. 6. An LU factorization of A provides an efficient method for solving Ax = b.

Exercise Set 1.7 In Exercises 1–4: a. Find the 3 × 3 elementary matrix E that performs the row operation. b. Compute EA, where   1 2 1 2  A= 3 1 1 1 −4 1. 2R1 + R2 −→ R2

5. 6.

7.

2. R1 ↔ R2 3. −3R2 + R3 −→ R3

8.

4. −R1 + R3 −→ R3 In Exercises 5–10: a. Find the elementary matrices required to reduce A to the identity.

9.

b. Write A as the product of elementary matrices.   1 3 A= −2 4   −2 5 A= 2 5   1 2 −1 3  A= 2 5 1 2 0   −1 1 1 A= 3 1 0  −2 1 1   0 1 1 A= 1 2 3  0 1 0

Confirming Pages

78

Chapter 1 Systems of Linear Equations and Matrices



0  0 10. A =   0 1

0 0 1 0

0 1 0 0

 1 0   0  0

In Exercises 11–16, find the LU factorization of the matrix A.   1 −2 11. A = −3 7   3 9 12. A = 1 1 2   1 2 1 5 5  13. A =  2 −3 −6 −2   1 1 1 14. A =  −1 0 −4  2 2 3   1 1 −3 2   3 15. A =  1 1  2 −1 −1 4   1 −2 1 3  −2 5 −3 −7   16. A =   1 −2 2 8  3 −6 3 10 In Exercises 17–22, solve the linear system by using LU factorization.  −2x + y = −1 17. 4x − y = 5  3x − 2y = 2 18. −6x + 5y = − 72   x + 4y − 3z = 0 −x − 3y + 5z = −3 19.  2x + 8y − 5z = 1   x − 2y + z = −1 2x − 3y + 6z = 8 20.  −2x + 4y − z = 4  x − 2y + 3z + w = 5    x − y + 5z + 3w = 6 21. 2x − 4y + 7z + 3w = 14    −x + y − 5z − 2w = −8

 x + 2y + 2z − w = 5    y + z − w = −2 22. −x − 2y − z + 4w = 1    2x + 2y + 2z + 2w = 1 In Exercises 23 and 24, find the P LU factorization of the matrix A.   0 1 −1 0  23. A =  2 −1 1 −3 2   0 0 1 1  24. A =  2 1 1 0 −3 In Exercises 25–28, find the inverse of the matrix A by using an LU factorization.   1 4 25. A = −3 −11   1 7 26. A = 2 20   2 1 −1 27. A =  2 2 −2  2 2 1   −3 2 1 28. A =  3 −1 1  −3 1 0   0 1 does 29. Show directly that the matrix A = 1 0 not have an LU factorization. 30. Let A, B, and C be m × n matrices. Show that if A is row equivalent to B and B is row equivalent to C, then A is row equivalent to C. 31. Show that if A and B are n × n invertible matrices, then A and B are row equivalent. 32. Suppose that A is an n × n matrix with an LU factorization, A = LU. a. What can be said about the diagonal entries of L? b. Express det(A) in terms of the entries of L and U. c. Show that A can be row-reduced to U using only replacement operations.

Confirming Pages

1.8 Applications of Systems of Linear Equations

ß

1.8

79

Applications of Systems of Linear Equations In the opening to this chapter we introduced linear systems by describing their connection to the process of photosynthesis. In this section we enlarge the scope of the applications we consider and show how linear systems are used to model a wide variety of problems.

Balancing Chemical Equations Recall from the introduction to this chapter that a chemical equation is balanced if there are the same number of atoms, of each element, on both sides of the equation. Finding the number of molecules needed to balance a chemical equation involves solving a linear system.

EXAMPLE 1

Propane is a common gas used for cooking and home heating. Each molecule of propane is comprised of 3 atoms of carbon and 8 atoms of hydrogen, written as C3 H8 . When propane burns, it combines with oxygen gas, O2 , to form carbon dioxide, CO2 , and water, H2 O. Balance the chemical equation C3 H8 + O2 −→ CO2 + H2 O that describes this process.

Solution

We need to find whole numbers x1 , x2 , x3 , and x4 , so that the equation x1 C3 H8 + x2 O2 −→ x3 CO2 + x4 H2 O is balanced. Equating the number of carbon, hydrogen, and oxygen atoms on both sides of this equation yields the linear system  − x3 =0  3x1 8x1 − 2x4 = 0  2x2 − 2x3 − x4 = 0 Solving this system, we obtain the solution set

    1 5 3  t, t, t, t  t ∈ ⺢ S= 4 4 4 Since whole numbers are required to balance the chemical equation, particular solutions are obtained by letting t = 0, 4, 8, . . . . For example, if t = 8, then x1 = 2, x2 = 10, x3 = 6, and x4 = 8. The corresponding balanced equation is given by 2C3 H8 + 10O2 −→ 6CO2 + 8H2 O

Network Flow To study the flow of traffic through city streets, urban planners use mathematical models called directed graphs or digraphs. In these models, edges and points are used to represent streets and intersections, respectively. Arrows are used to indicate

Confirming Pages

80

Chapter 1 Systems of Linear Equations and Matrices

the direction of traffic. To balance a traffic network, we assume that the outflow of each intersection is equal to the inflow, and that the total flow into the network is equal to the total flow out.

EXAMPLE 2

Partial traffic flow information, given by average hourly volume, is known about a network of five streets, as shown in Fig. 1. Complete the flow pattern for the network. 100

300

500 300

200

400 400 600

500

Figure 1

Solution

To complete the traffic model, we need to find values for the eight unknown flows, as shown in Fig. 2. x1

100

x6

300

x2 200

x3 x8

x7 400

500 300 400

x4 600

x5 500

Figure 2

Our assumptions about the intersections give us the set of linear equations  x + x6 = 300 + x1    2   100 + 500 = x6 + 300    200 + x = x2 + x7 3 300 + x7 = 400 + x4      400 + 500 = x3 + x8    x4 + 600 = 400 + x5

Confirming Pages

1.8 Applications of Systems of Linear Equations

81

Moreover, balancing the total flow into the network with the total flow out gives us the additional equation 500 + 600 + 500 + 200 + 100 = 400 + x5 + x8 + 300 + x1 The final linear system is  −x1 + x2         x2 − x3          

x3 x1

+ x6 x6 − x4 − x4 + x5 + x5

= 300 = 300 + x7 = 200 + x7 = 100 + x8 = 900 = 200 + x8 = 1200

The solution is given by x2 = 1100 − s − t x3 = 900 − t x1 = 1100 − s − t x5 = 100 + s x6 = 300 x7 = s x8 = t

x4 = −100 + s

Notice that x7 and x8 are free variables. However, to obtain particular solutions, we must choose numbers for s and t that produce positive values for each xi in the system (otherwise we will have traffic going in the wrong direction!) For example, s = 400 and t = 300 give a viable solution.

Nutrition Designing a healthy diet involves selecting foods from different groups that, when combined in the proper amounts, satisfy certain nutritional requirements.

EXAMPLE 3

Table 1 gives the amount, in milligrams (mg), of vitamin A, vitamin C, and calcium contained in 1 gram (g) of four different foods. For example, food 1 has 10 mg of vitamin A, 50 mg of vitamin C, and 60 mg of calcium per gram of food. Suppose that a dietician wants to prepare a meal that provides 200 mg of vitamin A, 250 mg of vitamin C, and 300 mg of calcium. How much of each food should be used? Table 1 Food 1

Food 2

Food 3

Food 4

Vitamin A

10

30

20

10

Vitamin C

50

30

25

10

Calcium

60

20

40

25

Revised Confirming Pages

82

Chapter 1 Systems of Linear Equations and Matrices

Solution

Let x1 , x2 , x3 , and x4 denote the amounts of foods 1 through 4, respectively. The amounts for each of the foods needed to satisfy the dietician’s requirement can be found by solving the linear system    10x1 + 30x2 + 20x3 + 10x4 = 200 50x1 + 30x2 + 25x3 + 10x4 = 250   60x1 + 20x2 + 40x3 + 25x4 = 300 Rounded to two decimal places, the solution to the linear system is given by x1 = 0.63 + 0.11t x3 = 5 − 0.92t

x2 = 3.13 + 0.24t x4 = t

Observe that each of these values must be nonnegative. Hence, particular solutions can be found by choosing nonnegative values of t such that 0 ≤ 5 − 0.92t Isolating t gives t≤

5 ≈ 5.4 0.92

Economic Input-Output Models Constructing models of the economy is another application of linear systems. In a real economy there are tens of thousands of goods and services. By focusing on specific sectors of the economy the Leontief input-output model gives a method for describing a simplified, but useful model of a real economy. For example, consider an economy for which the outputs are services, raw materials, and manufactured goods. Table 2 provides the inputs needed per unit of output. Table 2 Services

Raw materials

Manufacturing

Services

0.04

0.05

0.02

Raw materials

0.03

0.04

0.04

Manufactured goods

0.02

0.3

0.2

Here to provide $1.00 worth of service, the service sector requires $0.04 worth of services, $0.05 worth of raw materials, and $0.02 worth of manufactured goods. The data in Table 2 are recorded in the matrix   0.04 0.05 0.02 A =  0.03 0.04 0.04  0.02 0.3 0.2

Confirming Pages

1.8 Applications of Systems of Linear Equations

83

This matrix is called the input-output matrix. The demand vector D gives the total demand on the three sectors, in billions of dollars, and the production vector x, also in billions of dollars, contains the production level information for each sector. Each component of Ax represents the level of production that is used by the corresponding sector and is called the internal demand. As an example, suppose that the production vector is   200 x =  100  150 Then the the internal demand is  0.04 Ax =  0.03 0.02

given by

    16 200 0.05 0.02 0.04 0.04   100  =  16  64 150 0.3 0.2

This result means that the service sector requires $16 billion of services, raw materials, and manufactured goods. It also means that the external demand cannot exceed $184 billion of services, $84 billion of raw materials, and $86 billion in manufactured goods. Alternatively, suppose that the external demand D is given. We wish to find a level of production for each sector such that the internal and external demands are met. Thus, to balance the economy, x must satisfy x − Ax = D that is, (I − A)x = D When I − A is invertible, then x = (I − A)−1 D

EXAMPLE 4

Suppose that the external demand for services, raw materials, and manufactured goods in the economy described in Table 2 is given by   300 D =  500  600 Find the levels of production that balance the economy.

Solution

From the discussion above we have that the production vector x must satisfy (I − A)x = D

Confirming Pages

84

Chapter 1 Systems of Linear Equations and Matrices

    x1 300 0.96 −0.05 −0.02  −0.03 0.96 −0.04   x2  =  500  600 −0.02 −0.3 0.8 x3 Since the matrix on the left is invertible, the production vector x can be found by multiplying both sides by the inverse. Thus,      x1 300 1.04 0.06 0.03  x2  =  0.03 1.06 0.05   500  600 0.04 0.4 1.27 x3   360 ≈  569  974 that is,



So the service sector must produce approximately $360 billion worth of services, the raw material sector must produce approximately $569 billion worth of raw materials, and the manufacturing sector must produce approximately $974 billion worth of manufactured goods.

Exercise Set 1.8 In Exercises 1–4, use the smallest possible positive integers to balance the chemical equation. 1. When subjected to heat, aluminium reacts with copper oxide to produce copper metal and aluminium oxide according to the equation Al3 + CuO −→ Al2 O3 + Cu Balance the chemical equation. 2. When sodium thiosulfate solution is mixed with brown iodine solution, the mixture becomes colorless as the iodine is converted to colorless sodium iodide according to the equation I2 + Na2 S2 O3 −→ NaI + Na2 S4 O6 Balance the chemical equation. 3. Cold remedies such as Alka-Seltzer use the reaction of sodium bicarbonate with citric acid in

solution to produce a fizz (carbon dioxide gas). The reaction produces sodium citrate, water, and carbon dioxide according to the equation NaHCO3 + C6 H8 O7 −→ Na3 C6 H5 O7 + H2 O + CO2 Balance the chemical equation. For every 100 mg of sodium bicarbonate, how much citric acid should be used? What mass of carbon dioxide will be produced? 4. Balance the chemical equation MnS + As2 Cr10 O35 + H2 SO4 −→ HMnO4 + AsH3 + CrS3 O12 + H2 O 5. Find the traffic flow pattern for the network in the figure. Flow rates are in cars per hour. Give one specific solution.

Confirming Pages

85

1.8 Applications of Systems of Linear Equations

8. Find the traffic flow pattern for the network in the figure. Flow rates are in cars per half-hour. What is the smallest possible value for x8 ?

300

800

500 150 100

x5 300

300

x6

250

x7

x3

x4

200

x2 x8

300

x1

700

6. Find the traffic flow pattern for the network in the figure. Flow rates are in cars per hour. Give one specific solution. 100

400

500 300

500

300 400 200

100 200

9. The table lists the number of milligrams of vitamin A, vitamin B, vitamin C, and niacin contained in 1 g of four different foods. A dietician wants to prepare a meal that provides 250 mg of vitamin A, 300 mg of vitamin B, 400 mg of vitamin C, and 70 mg of niacin. Determine how many grams of each food must be included, and describe any limitations on the quantities of each food that can be used.

500

7. Find the traffic flow pattern for the network in the figure. Flow rates are in cars per half-hour. What is the current status of the road labeled x5 ? 150

x1

Group 1

Group 2

Group 3

Group 4

Vitamin A

20

30

40

10

Vitamin B

40

20

35

20

Vitamin C

50

40

10

30

5

5

10

5

Niacin

x4 x5

100

50

x2

x3

100

10. The table lists the amounts of sodium, potassium, carbohydrates, and fiber in a single serving of three food groups. Also listed are the daily recommended amounts based on a 2000-calorie diet. Is it possible to prepare a diet using the three food groups alone that meets the recommended amounts?

Confirming Pages

86

Chapter 1 Systems of Linear Equations and Matrices

Group 1

Group 2

Group 3

Requirement

Sodium (mg)

200

400

300

2400

Potassium (mg)

300

500

400

3500

40

50

20

300

5

3

2

25

Carbohydrates (g) Fiber (g)

a. Write the input-output matrix A for the economy. b. If the levels of production, in billions, of the three sectors of the economy are 300, 150, and 200, respectively, find the internal demand vector for the economy. What is the total external demand that can be met by the three sectors?

11. An economy is divided into three sectors as described in the table. Each entry represents the number of units required by the sector to produce 1 unit of output. Services

Raw materials

Manufacturing

Services

0.02

0.04

0.05

Raw materials

0.03

0.02

0.04

Manufacturing

0.03

0.3

0.1

c. Find the inverse of the matrix I − A. d. If the external demands on the three sectors are 350, 400, and 600, respectively, determine the levels of production that balance the economy.

12. Economies are, in general, very complicated with many sectors. The input-output matrix A is based on grouping the different industries and services into 10 separate sectors. If the external demands to the sectors are given in the vector D, determine the levels of production that balance the economy.         A=       

0.041 0.023 0.018 .034 0.022 0.044 0.018 0.026 0.01 0.048

0.032 0.037 0.03 0.005 0.019 0.005 0.001 0.004 0.011 0.03

0.018 0.046 0.039 0.034 0.021 0.02 0.049 0.03 0.039 0.019

0.041 0.011 0.05 0.039 0.009 0.006 0.011 0.015 0.025 0.045

0.009 0.004 0.038 0.023 0.007 0.013 0.043 0.044 0.005 0.044

0.002 0.024 0.011 0.007 0.035 0.005 0.003 0.021 0.029 0.033

0.039 0.041 0.049 0.009 0.044 0.032 0.024 0.01 0.024 0.014

0.048 0.006 0.001 0.023 0.023 0.016 0.047 0.004 0.023 0.03

0.04 0.004 0.028 0.05 0.019 0.047 0.027 0.011 0.021 0.042

13. The table contains estimates for national health care in billions of dollars. Year

Dollars (billions)

1965

30

1970

80

1975

120

1980

250

1985

400

1990

690

0.021 0.007 0.047 0.006 0.019 0.02 0.042 0.044 0.042 0.05





              

       D=       

45 10 11 17 48 32 42 21 34 40

               

a. Make a scatter plot of the data. b. Use the 1970, 1980, and 1990 data to write a system of equations that can be used to find a parabola that approximates the data. c. Solve the system found in part (b). d. Plot the parabola along with the data points. e. Use the model found in part (c) to predict an estimate for national health care spending in 2010. 14. The number of cellular phone subscribers worldwide from 1985 to 2002 is given in the

Confirming Pages

1.8 Applications of Systems of Linear Equations

table. Use the data from 1985, 1990, and 2000 to fit a parabola to the data points. Use the quadratic function to predict the number of cellular phone subscribers expected in 2010. Year

Cellular Phone Subscribers(millions)

1985

1

1990

11

2000

741

2001

955

2002

1155

In Exercises 15–18, use the power of a matrix to solve the problems. That is, for a matrix A, the nth power is An = A % · A ·&'A · · · A( n times

15. Demographers are interested in the movement of populations or groups of populations from one region to another. Suppose each year it is estimated that 90 percent of the people of a city remain in the city, 10 percent move to the suburbs, 92 percent of the suburban population remain in the suburbs, and 8 percent move to the city. a. Write a 2 × 2 transition matrix that describes the percentage of the populations that move from city to city (remain in the city), city to suburbs, suburbs to suburbs (remain in the suburbs), and suburbs to city. b. If in the year 2002 the population of a city was 1,500,000 and of the suburbs was 600,000, write a matrix product that gives a 2 × 1 vector containing the populations in the city and in the suburbs in the year 2003. Multiply the matrices to find the populations. c. If in the year 2002 the population of a city was 1,500,000 and of the suburbs was 600,000, write a matrix product that gives a 2 × 1 vector containing the populations in the city and in

87

the suburbs in the year 2004. Multiply the matrices to find the populations. d. Give a matrix product in terms of powers of the matrix found in part (a) for the size of the city and suburban populations in any year after 2002. 16. To study the spread of a disease, a medical researcher infects 200 laboratory mice of a population of 1000. The researcher estimates that it is likely that 80 percent of the infected mice will recover in a week and 20 percent of healthy mice will contract the disease in the same week. a. Write a 2 × 2 matrix that describes the percentage of the population that transition from healthy to healthy, healthy to infected, infected to infected, and infected to healthy. b. Determine the number of healthy and infected mice after the first week. c. Determine the number of healthy and infected mice after the second week. d. Determine the number of healthy and infected mice after six weeks. 17. In a population of 50,000 there are 20,000 nonsmokers, 20,000 smokers of one pack or less a day, and 10,000 smokers of more than one pack a day. During any month it is likely that only 10 percent of the nonsmokers will become smokers of one pack or less a day and the rest will remain nonsmokers, 20 percent of the smokers of a pack or less will quit smoking, 30 percent will increase their smoking to more than one pack a day, 30 percent of the heavy smokers will remain smokers but decrease their smoking to one pack or less, and 10 percent will go cold turkey and quit. After one month what part of the population is in each category? After two months how many are in each category? After one year how many are in each category? 18. An entrepreneur has just formed a new company to compete with the established giant in the market. She hired an advertising firm to develop a

Confirming Pages

88

Chapter 1 Systems of Linear Equations and Matrices

campaign to introduce her product to the market. The advertising blitz seems to be working, and in any given month 2 percent of the consumers switch from the time-honored product to the new improved version, but at the same time 5 percent of those using the new product decide to switch back to the old established brand. How long will it take for the new company to acquire 20 percent of the consumers? In Exercises 19 and 20, the figure shows an electrical network. In an electrical network, current is measured in amperes, resistance in ohms, and the product of current and resistance in volts. Batteries are represented using two parallel line segments of unequal length, and it is understood the current flows out of the terminal denoted by the longer line segment. Resistance is denoted using a sawtooth. To analyze an electrical network requires Kirchhoff’s laws, which state all current flowing into a junction, denoted using a black dot, must flow out and the sum of the products of current I and resistance R around a closed path (a loop) is equal to the total voltage in the path. 19. a. Apply Kirchhoff’s first law to either junction to write an equation involving I1 , I2 , and I3 . b. Apply Kirchhoff’s second law to the two loops to write two linear equations. c. Solve the system of equations from parts (a) and (b) to find the currents I1 , I2 , and I3 . I1

20. a. Apply Kirchhoff’s first law to the four junctions to write four equations involving currents. b. Apply Kirchhoff’s second law to the three loops to write three linear equations. c. Solve the system of equations from parts (a) and (b) to find the currents I1 , I2 , I3 , I4 , I5 , and I6 . I1 R1 = 4 Loop

I2 R2 = 6 I5

R5 = 2

I4 R4 = 6

Loop

16 V

In Exercises 21 and 22, use the fact that if a plate has reached a thermal equilibrium, then the temperature at a grid point, not on the boundary of the plate, is the average of the temperatures of the four closest grid points. The temperatures are equal at each point on a boundary, as shown in the figure. Estimate the temperature at each interior grid point. 21.

30

20

Loop

25

20

22.

30

R2 = 3 Loop

R3 = 5

I6 R6 = 3

R3 = 4

R1 = 4

20

I3

18 V Loop

I3

8V

I2

14 V

25

10 V 20

Confirming Pages

1.8 Applications of Systems of Linear Equations

89

Review Exercises for Chapter 1 1. Consider the linear system  x + y + 2z + w = 3    −x + z + 2w = 1 2x + 2y + w=−2    x + y + 2z + 3w = 5 a. Define the coefficient matrix A for the linear system. b. Find det(A). c. Is the linear system consistent? Explain. d. Find all solutions to Ax = 0. e. Is the matrix A invertible? If yes, then find the inverse. f. Solve the linear system. 2. The augmented matrix of a linear system has the form   1 −1 2 1 a  −1 3 1 1 b     3 −5 5 1 c  2 −2 4 2 d a. Can you decide by inspection whether the determinant of the coefficient matrix is 0? Explain.

3. Find all idempotent matrices of the form   a b 0 c 4. Let S denote the set of all 2 × 2 matrices. Find all   a b that will commute with every matrices c d matrix in S. 5. Let A and B be 2 × 2 matrices. a. Show that the sum of the terms on the main diagonal of AB − BA is 0. b. If M is a 2 × 2 matrix and the sum of the main diagonal entries is 0, show there is a constant c such that M 2 = cI c. If A, B, and C are 2 × 2 matrices, then use parts (a) and (b) to show that (AB − BA)2 C = C(AB − BA)2 6. Find the traffic flow pattern for the network in the figure. Flow rates are in cars per hour. Give one specific solution.

b. Can you decide by inspection whether the linear system has a unique solution for every choice of a, b, c, and d? Explain.

100

c. Determine the values of a, b, c, and d for which the linear system is consistent.

300

d. Determine the values of a, b, c, and d for which the linear system is inconsistent.

200

500 300 400 400

e. Does the linear system have a unique solution or infinitely many solutions? f. If a = 2, b = 1, c = −1, and d = 4, describe the solution set for the linear system.

600

500

Confirming Pages

90

Chapter 1 Systems of Linear Equations and Matrices

7. a. Explain why the matrix  1 1  0 1  A=  0 0  0 0 0 0

1 1 1 0 0

1 1 1 1 0

1 1 1 1 1

     

is invertible. b. Determine the maximum number of 1’s that can be added to A such that the resulting matrix is invertible. 8. Show that if A is invertible, then At is invertible and (At )−1 = (A−1 )t .

9. A matrix A is skew-symmetric provided At = −A. a. Let A be an n × n matrix and define B = A + At

and

C = A − At

Show that B is symmetric and C is skew-symmetric. b. Show that every n × n matrix can be written as the sum of a symmetric and a skew-symmetric matrix. 10. Suppose u and v are solutions to the linear system Ax = b. Show that if scalars α and β satisfy α + β = 1, then αu + βv is also a solution to the linear system Ax = b.

Chapter 1: Chapter Test In Exercises 1–45, determine whether the statement is true or false. 1. A 2 × 2 linear system has one solution, no solutions, or infinitely many solutions. 2. A 3 × 3 linear system has no solutions, one solution, two solutions, three solutions, or infinitely many solutions.

11. The matrix      

1 0 0 0 0

−2 −1 0 0 0

3 4 3 0 0

 1 0 3 2   5 −2   0 4  0 6

does not have an inverse.

3. If A and B are n × n matrices with no zero entries, then AB = 0.

12. Interchanging two rows of a matrix changes the sign of its determinant.

4. Homogeneous linear systems always have at least one solution.

13. Multiplying a row of a matrix by a nonzero constant results in the determinant being multiplied by the same nonzero constant.

5. If A is an n × n matrix, then Ax = 0 has a nontrivial solution if and only if the matrix A has an inverse. 6. If A and B are n × n matrices and Ax = Bx for every n × 1 matrix x, then A = B. 7. If A, B, and C are invertible n × n matrices, then (ABC)−1 = A−1 B −1 C −1 . 8. If A is an invertible n × n matrix, then the linear system Ax = b has a unique solution. 9. If A and B are n × n invertible matrices and AB = BA, then A commutes with B −1 . 10. If A and B commute, then A2 B = BA2 .

14. If two rows of a matrix are equal, then the determinant of the matrix is 0. 15. Performing the operation aRi + Rj → Rj on a matrix multiplies the determinant by the constant a.   1 2 , then A2 − 7A = 2I. 16. If A = 4 6 17. If A and B are invertible matrices, then A + B is an invertible matrix. 18. If A and B are invertible matrices, then AB is an invertible matrix.

Confirming Pages

1.8 Applications of Systems of Linear Equations

19. If A is an n × n matrix and A does not have an inverse, then the linear system Ax = b is inconsistent. 20. The linear system      1 x 1 2 3  6 5 4  y  =  2  3 z 0 0 0



1 1 −3 2





In Exercises 26–32, use the linear system  2x + 2y = 3 x− y=1 26. The coefficient matrix is   2 2 A= 1 −1 27. The coefficient matrix A has determinant det(A) = 0 28. The linear system has a unique solution. 29. The only solution to the linear system is x = −7/4 and y = −5/4. 30. The inverse of the coefficient matrix A is   1 1 A−1 =

4 1 4

2

In Exercises 33–36, use the linear system   x1 + 2x2 − 3x3 = 1 2x1 + 5x2 − 8x3 = 4  −2x1 − 4x2 + 6x3 = −2

 2 −1 4 −2 does not have an inverse. 23. If the n × n matrix A is idempotent and invertible, then A = I. 24. If A and B commute, then At and B t commute. 25. If A is an n × n matrix and det(A) = 3, then det(At A) = 9. 22. The matrix

32. The solution to the system is given by the matrix equation     1  1 x 3 4 2 = 1 y 1 −1 4

is inconsistent. 21. The inverse of the matrix   2 −1 is 3 1

91

2

− 12

31. The linear system is equivalent to the matrix equation      3 x 2 2 = 1 y 1 −1

33. The determinant of the coefficient matrix is        5 −8   2 −8   2 5  + +   −4 6   −2 −4  6   −2 34. The determinant of the coefficient matrix is 0. 35. A solution to the linear system is x1 = −4, x2 = 0, and x3 = −1. 36. The linear system has infinitely many solutions, and the general solution is given by x3 is free, x2 = 2 + 2x3 , and x1 = −3 − x3 . In Exercises 37–41, use the matrix   −1 −2 1 3 0 1 −1  A= 1 2 1 2 1 37. After the operation R1 ←→ R2 is performed, the matrix becomes   1 0 1 −1  −1 −2 1 3  2 1 2 1 38. After the operation −2R1 + R3 −→ R3 is performed on the matrix found in Exercise 37, the matrix becomes   1 0 1 −1  −1 −2 1 3  0 −2 0 −3 39. The matrix A is row equivalent to   1 0 1 −1  0 −2 2 2  0 0 1 4

Confirming Pages

92

Chapter 1 Systems of Linear Equations and Matrices

40. The reduced row echelon form of A is   1 0 0 −5  0 1 0 3  0 0 1 4

42. The matrix products AB and BA are both defined.

41. If A is viewed as the augmented matrix of a linear system, then the solution to the linear system is x = −5, y = 3, and z = 4.

44. The matrix expression −2BA + 3B equals

43. The matrix expression −2BA + 3B simplifies to a 2 × 3 matrix. 

In Exercises 42–45, use the matrices  1 1 2 1  A =  −2 3 4 0 −3   1 2 1 B= −1 3 2 

45. The matrix A2 is 

−3 −5 3 −5 7 16



 7 4 −3  −4 7 −4  −8 4 17

Confirming Pages

CHAPTER

Linear Combinations and Linear Independence CHAPTER OUTLINE

2.1 2.2 2.3

Vectors in ⺢n 94 Linear Combinations 101 Linear Independence 111

I

n the broadest sense a signal is any timevarying quantity. The motion of a particle through space, for example, can be thought of as a signal. A seismic disturbance is detected as signals from within the earth. Sound caused by the vibration of a string is a signal, radio waves are signals, and a digital picture with colors represented numerically also can be considered a signal. A video signal is a sequence of images. Signals represented using real numbers are called continuous while others that use integers are called discrete. A compact disc contains discrete signals representing sound. Some signals are periodic; that is, the waveform or shape of the signal repeats at regular intervals. The period of a wave is the time it takes for one cycle of the wave, and the frequency is the number of cycles that occur per unit of time. If the period of a wave is 2T , then the frequency is F = 2T1 . Every periodic motion is the mixture of sine and cosine waves with frequencies proportional to a common frequency, called the fundamental frequency. A signal with period 2T is a mixture of the functions πx πx 2πx 2πx 3πx 3πx 1, cos , sin , cos , sin , cos , sin ,... T T T T T T and for any n, the signal can be approximated by the fundamental set πx πx 2πx 2πx nπx nπx 1, cos , sin , cos , sin , . . . , cos , sin T T T T T T

93

Confirming Pages

Chapter 2 Linear Combinations and Linear Independence

The approximation obtained from the sum of the elements of the fundamental set with appropriate coefficients, or weights, has the form πx πx 2πx a0 + a1 cos + b1 sin + a2 cos T T T 2πx nπx nπx + · · · + an cos + bn sin + b2 sin T T T This sum is called a linear combination of the elements of the fundamental set. A square wave on the interval [−π, π] along with the approximations 4 4 4 4 4 4 sin x, sin x + sin 3x, sin x + sin 3x + sin 5x π π 3π π 3π 5π and

4 4 4 4 sin x + sin 3x + sin 5x + sin 7x π 3π 5π 7π are shown in Fig. 1. As more terms are added, the approximations become better. y

y

1

1

Square wave x

x ⫺1

⫺1

Figure 1

In Chap. 1 we defined a vector, with n entries, as an n × 1 matrix. Vectors are used not only in mathematics, but in virtually every branch of science. In this chapter we study sets of vectors and analyze their additive properties. The concepts presented here, in the context of vectors, are fundamental to the study of linear algebra. In Chap. 3, we extend these concepts to abstract vector spaces, including spaces of functions as described in the opening example.

2.1

ß

94

Vectors in ⺢n Euclidean 2-space, denoted by ⺢2 ,
 x1 2 ⺢ = x2

is the set of all vectors with two entries, that is,     x1 , x2 are real numbers 

Confirming Pages

2.1 Vectors in ⺢n

95

Similarly Euclidean 3-space, denoted by ⺢3 , is the set of all vectors with three entries, that is,     x1   ⺢3 =  x2  x1 , x2 , x3 are real numbers   x3  In general, Euclidean n-space consists of vectors with n entries. Vectors in ⺢n

DEFINITION 1

Euclidean n-space, denoted by ⺢n , or simply n-space, is defined by    x1         x   2 n    ⺢ =  ..  xi ∈ ⺢, for i = 1, 2, . . . , n   .       x n

y

The entries of a vector are called the components of the vector.

(1, 2)

x

Figure 2

y

v v x

Figure 3

DEFINITION 2

Geometrically, in ⺢2 and ⺢3 a vector is a directed line segment from the origin to the point whose coordinates are equal to the components of the vector. For example, the vector in ⺢2 given by   1 v= 2 is the directed line segment from the origin (0, 0) to the point (1, 2), as shown in Fig. 2. The point (0, 0) is the initial point, and the point (1, 2) is the terminal point. The length of a vector is the length of the line segment  from the initial point  √ √ 1 is 12 + 22 = 5. A to the terminal point. For example, the length of v = 2 vector is unchanged if it is relocated elsewhere in the plane, provided that the length and direction remain unchanged. For example, the directed line segments between (0, 0) and (1,  2)and between (2, 2) and (3, 4) are both representations of the same 1 . See Fig. 3. When the initial point of a vector is the origin, we say vector v = 2 that the vector is in standard position. Since vectors are matrices, two vectors are equal provided that their corresponding components are equal. The operations of addition and scalar multiplication are defined componentwise as they are for matrices. Addition and Scalar Multiplication of Vectors ⺢n and c a scalar. 1. The sum of u and v is

  u1  u2     u+v=  ...  +  

un

Let u and v be vectors in

  v1 u 1 + v1 v 2   u 2 + v2 ..  .. = .   . vn un + vn

   

Confirming Pages

Chapter 2 Linear Combinations and Linear Independence

2. The scalar product of c and u is 

   u1 cu1  u2   cu2     cu = c   ...  =  ...  un cun

These algebraic definitions of vector addition and scalar multiplication agree with the standard geometric definitions. Two vectors u and v are added according to the parallelogram rule, as shown in Fig. 4(a). The vector cu is a scaling of the vector u. In Fig. 4(b) are examples of scaling a vector with 0 < c < 1 and c > 1. In addition, if c < 0, then the vector cu is reflected through the origin, as shown in Fig. 4(b). The difference u − v = u + (−v) is the vector shown in Fig. 4(c). As shown in Fig. 4(c), it is common to draw the difference vector u − v from the terminal point of v to the terminal point of u.

Solution

 1 u =  −2  3 Find (2u + v) − 3w. 

Let

 −1 v= 4  3

 4 w= 2  6 



and

Using the componentwise definitions of addition and scalar multiplication, we have        4 −1 1 (2u + v) − 3w = 2  −2  +  4  − 3  2  6 3 3       −12 −1 2 =  −4  +  4  +  −6  −18 3 6       −11 −12 1 =  0  +  −6  =  −6  −9 −18 9

y

y

y

2u

v

EXAMPLE 1

v

u

u+

96

1 2u

u x

(a)

v x

−2u

(b)

Figure 4

u−v u x

−v

(c)

Confirming Pages

2.1 Vectors in ⺢n

97

Vectors in ⺢n , being matrices with n rows and 1 column, enjoy all the algebraic properties of matrices that we saw in Chap. 1.

EXAMPLE 2 Solution

Show that vector addition is commutative. If u and v are vectors in ⺢n , then  u1  u2 u+v=  ...





  v1 u 1 + v1   v 2   u 2 + v2 + . = ..   ..   . un vn un + vn

Since addition for real numbers is commutative,    u 1 + v1 v 1 + u1  u 2 + v2   v 2 + u 2 = .. .. u+v=    . . un + vn

   

  =v+u 

vn + u n

The zero vector in ⺢n is the vector with  0  0 0=  ...

each component equal to 0, that is,    

0 Hence, for any vector v in ⺢n , we have v + 0 = v. Recall that for any real number a there is a unique number −a such that a + (−a) = 0. This enables us to define the additive inverse of any vector v as the vector   −v1  −v2   −v =   ...  −vn so that v + (−v) = 0. Theorem 1 summarizes the essential algebraic properties of vectors in ⺢n . These properties serve as a model for the structure we will require for the abstract vector spaces of Chap. 3. The first of these properties was proved in Example 2. The remaining justifications are left as exercises.

THEOREM 1

Let u, v, and w be vectors in ⺢n , and let c and d be scalars. The following algebraic properties hold. 1. Commutative property: 2. Associative property:

u+v=v+u (u + v) + w = u + (v + w)

Confirming Pages

98

Chapter 2 Linear Combinations and Linear Independence

3. Additive identity: The vector 0 satisfies 0 + u = u + 0 = u. 4. Additive inverse: For every vector u, the vector −u satisfies u + (−u) =−u + u = 0. 5. c(u + v) = cu + cv 6. (c + d)u = cu + du 7. c(du) = (cd)u 8. (1)u = u By the associative property, the vector sum u1 + u2 + · · · + un can be computed unambiguously, without the need for parentheses. This will be important in Sec. 2.2.

EXAMPLE 3

     4 2 1 and w= v= −3 3 −1 Verify that the associative property holds for these three vectors. Also verify that for any scalars c and d, c(du) = (cd)u. 

Let

u=

Solution

To verify the associative property, we have           4 1+2 4 2 1 + = + + (u + v) + w = −3 −1 + 3 −3 3 −1       7 4 3 = + = −1 −3 2 and

   4 2 + + u + (v + w) = −3 3       7 6 1 = + = −1 0 −1 

1 −1





Hence, (u + v) + w = u + (v + w). For the second verification, we have          1 cd d 1 = (cd)u = cd = =c c(du) = c d −1 −cd −d −1

The properties given in Theorem 1 can be used to establish other useful properties of vectors in ⺢n . For example, if u ∈ ⺢n and c is a scalar, then     0 u1  u2   0     and c0 = 0 0u = 0   ...  =  ...  = 0 un

0

Confirming Pages

2.1 Vectors in ⺢n

99

We also have the property that (−1)u = −u. That is, the scalar product of −1 with u is the additive inverse of u. In the case of real numbers, the statement xy = 0 is equivalent to x = 0 or y = 0. A similar property holds for scalar multiplication. That is, if cu = 0, then either c = 0 or u = 0. To see this, let     cu1 0  cu2   0   . = .   ..   ..  cun

0

so that cu1 = 0, cu2 = 0, . . . , cun = 0. If c = 0, then the conclusion holds. Otherwise, u1 = u2 = · · · = un = 0, that is, u = 0.

Fact Summary 1. The definitions of vector addition and scalar multiplication in ⺢n agree with the definitions for matrices in general and satisfy all the algebraic properties of matrices. 2. The zero vector, whose components are all 0, is the additive identity for vectors in ⺢n . The additive inverse of a vector v, denoted by −v, is obtained by negating each component of v. 3. For vectors in ⺢2 and ⺢3 vector addition agrees with the standard parallelogram law. Multiplying such a vector by a positive scalar changes the length of the vector but not the direction. If the scalar is negative, the vector is reflected through the origin.

Exercise Set 2.1 In Exercises 1–6, use the vectors     −2 1 v= 4  u =  −2  0 3   2 w= 1  −1 1. Find u + v and v + u. 2. Find (u + v) + w and u + (v + w). 3. Find u − 2v + 3w. 4. Find −u + 12 v − 2w.

5. Find −3(u + v) − w. 6. Find 2u − 3(v − 2w). In Exercises 7–10, use the vectors    3 1  2  −2   v= u=  −1  3  1 0 7. Find −2(u + 3v) + 3u. 8. Find 3u − 2v. 9. If x1 and x2 are real scalars, verify that (x1 + x2 )u = x1 u + x2 u.

   

Confirming Pages

100

Chapter 2 Linear Combinations and Linear Independence



10. If x1 is a real scalar, verify that x1 (u + v) = x1 u + x1 v. In Exercises 11–14, let   1 e1 =  0  0   0 e3 =  0  1



18. c1 

0 e2 =  1  0

Write the given vector in terms of the vectors e1 , e2 , and e3 .   2 11. v =  4  1   −1 12. v =  3  2   0 13. v =  3  −2   −1 14. v =  0  1 2

In Exercises 15 and 16, find w such that −u + 3v − 2w = 0.     −2 1 v= 2  15. u =  4  0 2     2 −2 v =  −3  16. u =  0  4 1 In Exercises 17–24, write the vector equation as an equivalent linear system and then solve the system. Explain what the solution to the linear system implies about the vector equation.       −2 1 3 17. c1 + c2 = −1 −2 −2

 19. c1  20. c1

2 5 1 2



 + c2



−1 3



−1 −2

+ c2

−1 −2



 + c2



 =



2 −6





0 5

=

3 1



 =



−1 1



       −3 −4 0 −5 21. c1  4  + c2  3  + c3  1  =  −3  4 3 −1 −5 

       −1 0 1 1 22. c1  −1  + c2  1  + c3  1  =  0  −1 1 0 −1 

       −1 −1 −1 1 23. c1  0  + c2  1  + c3  −1  =  0  2 1 1 −1 

       6 −1 0 2 24. c1  2  + c2  2  + c3  1  =  7  3 4 4 2   a , so that the In Exercises 25–28, find all vectors b vector equation can be solved.       a 1 2 25. c1 + c2 = b −1 1 

 26. c1  27. c1  28. c1

1 1



1 −1 3 1

 + c2

−1 1



 + c2



 + c2

6 2



 =

2 −2





 =

 =

a b



a b

a b





In Exercises 29–32, find all vectors   a v= b  c so that the vector equation c1 v1 + c2 v2 + c3 v3 = v can be solved.

Confirming Pages

2.2 Linear Combinations

29. v1

v3

30. v1

v3

31. v1

v3

 0 v2 =  1  1 

 0 v2 =  1  0 

 −1 32. v1 =  0  2   1 v3 =  −1  3 

 1 v2 =  −2  8 

In Exercises 33–39, verify the indicated vector property of Theorem 1 for vectors in ⺢n . 33. Property 2. 34. Property 3. 35. Property 4.

 2 v2 =  1  1 

36. Property 5. 37. Property 6. 38. Property 7. 39. Property 8. 40. Prove that the zero vector in ⺢n is unique.

2.2

ß

 1 = 0  1   2 = 1  0   1 = 1  1   1 = 1  0   1 = 1  −1   3 = 2  0 

101

Linear Combinations In three-dimensional Euclidean space ⺢3 the coordinate vectors that define the three axes are the vectors       1 0 0 e2 =  1  and e3 =  0  e1 =  0  0 0 1 Every vector in ⺢3 can then be obtained from example, the vector      1 2 v =  3  = 2 0 + 3 0 3

these three coordinate vectors, for    0 0 1  + 3 0  1 0

Geometrically, the vector v is obtained by adding scalar multiples of the coordinate vectors, as shown in Fig. 1. The vectors e1 , e2 , and e3 are not unique in this respect. For example, the vector v can also be written as a combination of the vectors       1 0 −1 v2 =  1  and v3 =  1  v1 =  1  1 1 1

Confirming Pages

102

Chapter 2 Linear Combinations and Linear Independence z

3e3 v 2e1 x

3e2

2e1 + 3e2

y

Figure 1

 2 3v1 − v2 + v3 =  3  3 A vector written as a combination of other vectors using addition and scalar multiplication is called a linear combination. Combining vectors in this manner plays a central role in describing Euclidean spaces and, as we will see in Chap. 3, in describing abstract vector spaces. 

that is,

DEFINITION 1

Linear Combination Let S = {v1 , v2 , . . . , vk } be a set of vectors in ⺢n , and let c1 , c2 , . . . , ck be scalars. An expression of the form c1 v1 + c2 v2 + · · · + ck vk =

k 

ci vi

i=1

is called a linear combination of the vectors of S. Any vector v that can be written in this form is also called a linear combination of the vectors of S.

In Example 1 we show how linear systems are used to decide if a vector is a linear combination of a set of vectors.

EXAMPLE 1

 −1 v= 1  10 is a linear combination of the vectors     1 −2 v1 =  0  v2 =  3  and 1 −2 Determine whether the vector



 −6 v3 =  7  5 

Confirming Pages

2.2 Linear Combinations

Solution

103

The vector v is a linear combination of the vectors v1 , v2 , and v3 if there are scalars c1 , c2 , and c3 , such that   −1 v =  1  = c1 v1 + c2 v2 + c3 v3 10       1 −2 −6 = c1  0  + c2  3  + c3  7  1 −2 5   c1 − 2c2 − 6c3 3c2 + 7c3  = c1 − 2c2 + 5c3 Equating components gives the linear system   c1 − 2c2 − 6c3 = −1 3c2 + 7c3 = 1  c1 − 2c2 + 5c3 = 10 To solve this linear system, we reduce the augmented matrix    1 −2 −6 −1 1 0 0  0  0 1 0 3 7 1  to 1 −2 5 10 0 0 1

 1 −2  1

From the last matrix, we see that the linear system is consistent with the unique solution c1 = 1 c2 = −2 and c3 = 1 Using these scalars, we can write v as the linear combination         −6 −2 1 −1 v =  1  = 1  0  + (−2)  3  + 1  7  5 −2 1 10

The case for which a vector is not a linear combination of a set of vectors is illustrated in Example 2.

EXAMPLE 2

 −5 v =  11  −7 is a linear combination of the vectors     1 0 v1 =  −2  v2 =  5  and 2 5 Determine whether the vector



 2 v3 =  0  8 

Confirming Pages

104

Chapter 2 Linear Combinations and Linear Independence

Solution

The vector v is a linear combination of the vectors v1 , v2 , and v3 if there are scalars c1 , c2 , and c3 , such that         1 0 2 −5  11  = c1  −2  + c2  5  + c3  0  2 5 8 −7 The augmented matrix corresponding to this equation is given by   1 0 2 −5  −2 5 0 11  2 5 8 −7 Reducing the augmented matrix   1 0 2 −5  −2 5 0 11  2 5 8 −7

 to

1 0 2  0 5 4 0 0 0

 −5 1  2

shows that the linear system is inconsistent. Therefore, the vector v cannot be written as a linear combination of the three vectors v1 , v2 , and v3 . To see this geometrically, first observe that the vector v3 is a linear combination of v1 and v2 . That is, 4 v3 = 2v1 + v2 5 Therefore, any linear combination of the three vectors v1 , v2 , and v3 , is just a linear combination of v1 and v2 . Specifically,   4 c1 v1 + c2 v2 + c3 v3 = c1 v1 + c2 v2 + c3 2v1 + v2 5   4 = (c1 + 2c3 )v1 + c2 + c3 v2 5 The set of all vectors that are linear combinations of v1 and v2 is a plane in ⺢3 , which does not contain the vector v, as shown in Fig. 2.

v Figure 2

Confirming Pages

2.2 Linear Combinations

In ⺢n the coordinate vectors are the    1  0    e2 =  e1 =   ...   0

n vectors given by   0 0  0 1  ..  ... en =   ... .  0 1

105

   

These vectors can also be defined by the equations
1 if i = k (ek )i = 0 if i = k where 1 ≤ k ≤ n. An important property of the coordinate vectors is that every vector in ⺢n can be written as a linear combination of the coordinate vectors. Indeed, for any vector v in ⺢n , let the scalars be the components of the vector, so that         v1 1 0 0  v2   0   1   0         v=  ...  = v1  ...  + v2  ...  + · · · + vn  ...  vn

0

0

1

= v 1 e 1 + v2 e 2 + · · · + vn e n Linear combinations of more abstract objects can also be formed, as illustrated in Example 3 using 2 × 2 matrices. This type of construction is used extensively in Chap. 3 when we consider abstract vector spaces.

EXAMPLE 3



Show that the matrix A=

1 1 1 0

is a linear combination of the matrices     1 0 0 1 M1 = M2 = 0 1 1 1 Solution



 and

M3 =

1 1 1 1



Similar to the situation with vectors, we must find scalars c1 , c2 , and c3 such that c1 M1 + c2 M2 + c3 M3 = A        1 1 1 0 0 1 1 1 + c2 + c3 = 1 0 0 1 1 1 1 1 After performing the scalar multiplication and addition, we obtain     c 1 + c3 c2 + c3 1 1 = 1 0 c2 + c3 c1 + c2 + c3 

that is,

c1

Confirming Pages

106

Chapter 2 Linear Combinations and Linear Independence

Equating corresponding entries gives the linear   c3 c1 + c2 + c3   c1 + c2 + c3

system =1 =1 =0

This system is consistent with solution c1 = −1, c2 = −1, and c3 = 2. Thus, the matrix A is a linear combination of the matrices M1 , M2 , and M3 .

EXAMPLE 4

Solution

Consider the homogeneous equation Ax = 0. Show that if x1 , x2 , . . . , xn are solutions of the equation, then every linear combination c1 x1 + c2 x2 + · · · + cn xn is also a solution of the equation. Since x1 , x2 , . . . , xn are solutions of the matrix equation, we have Ax1 = 0

Ax2 = 0

...

Axn = 0

Then using the algebraic properties of matrices, we have A(c1 x1 + c2 x2 + · · · + cn xn ) = A(c1 x1 ) + A(c2 x2 ) + · · · + A(cn xn ) = c1 (Ax1 ) + c2 (Ax2 ) + · · · + cn (Axn ) = c1 0 + c2 0 + · · · + cn 0 =0

The result of Example 4 is an extension of the one given in Example 3 of Sec. 1.5.

Vector Form of a Linear System We have already seen that a linear system with m equations and n variables  a11 x1 + a12 x2 + · · · + a1n xn = b1     a21 x1 + a22 x2 + · · · + a2n xn = b2 .. .. .. ..  . . . .    am1 x1 + am2 x2 + · · · + amn xn = bm can be written in matrix form as Ax = b, where A is the m × n coefficient matrix, x is the vector in ⺢n of variables, and b is the vector in ⺢m of constants. If we use the column vectors of the coefficient matrix A, then the matrix equation can be written

Confirming Pages

2.2 Linear Combinations

in the equivalent form  a11  a21 x1   ... am1





  a12 a1n   a22   a2n  + x2  .  + · · · + xn  .   ..   .. am2 amn



107



 b1   b2  = .    ..  bm

This last equation is called the vector form of a linear system. This equation can also be written as x1 A1 + x2 A2 + · · · + xn An = b where Ai denotes the ith column vector of the matrix A. Observe that this equation is consistent whenever the vector b can be written as a linear combination of the column vectors of A.

THEOREM 2

The linear system Ax = b is consistent if and only if the vector b can be expressed as a linear combination of the column vectors of A.

Matrix Multiplication Before concluding this section, we comment on how linear combinations can be used to describe the product of two matrices. Let A be an m × n matrix and B an n × p matrix. If Bi is the ith column vector of B, then the ith column vector of the product AB is given by   a11 a12 . . . a1n  b1i   a21 a22 . . . a2n   b  2i   ..  ABi =  . .. ..   .  . .  . .  . . .  bni am1 am2 . . . amn   a11 b1i + a12 b2i + · · · + a1n bni  a21 b1i + a22 b2i + · · · + a2n bni    =  .. .. ..   . . . am1 b1i + am2 b2i + · · · + amn bni 

  a11 b1i a12 b2i  a21 b1i   a22 b2i + .. .. =    . . am1 b1i am2 b2i





 a1n bni   a b   + · · · +  2n. ni  ..    amn bni

= b1i A1 + b2i A2 + · · · + bni An Since for i = 1, 2, . . . , p the product ABi is the ith column vector of AB, each column vector of AB is a linear combination of the column vectors of A.

Confirming Pages

108

Chapter 2 Linear Combinations and Linear Independence

Fact Summary 1. Every vector in ⺢n is a linear combination of the coordinate vectors e 1 , e2 , . . . , en . 2. If x1 , x2 , . . . , xk are all solutions to the homogeneous equation Ax = 0, then so is every linear combination of these vectors. 3. The linear system Ax = b can be written in the equivalent vector form as x1 A1 + x2 A2 + · · · + xn An = b. The left side is a linear combination of the column vectors of A. 4. The linear system Ax = b is consistent if and only if b is a linear combination of the column vectors of A.

Exercise Set 2.2 In Exercises 1–6, determine whether the vector v is a linear combination of the vectors v1 and v2 .     −4 1 v1 = 1. v = 11 1   −2 v2 = 3     13 −1 v1 = 2. v = −2 2   3 v2 = 0     1 −2 v1 = 3. v = 1 4   3 v2 = −6     3 1 v1 = 4. v = 2 −2  v2 =

1 2

−1



 −3 5. v =  10  10   1 v2 =  4  2   −2 6. v =  6  8 

 −2 v1 =  3  4 

 3 v1 =  4  −1 

 2 v2 =  7  3 

In Exercises 7–12, determine whether the vector v is a linear combination of the vectors v1 , v2 , and v3 .     2 2 v1 =  −2  7. v =  8  0 2     3 −2 v2 =  0  v3 =  0  −3 −1

Confirming Pages

2.2 Linear Combinations

 5 8. v =  −4  −7   −2 v2 =  −1  −1   −1 9. v =  1  5 

 −1 v2 =  −1  3 

 −3 10. v =  5  5 

 1 v2 =  4  1 

 3  −17   11. v =   17  7 

 1 v1 =  −1  0   3 v3 =  −1  −3   1 v1 =  2  −1   0 v3 =  1  2 

 −3 v1 =  2  1   −1 v3 =  10  3   2  −3   v1 =   4  1 

  1 −1  6   −1  v2 =  v3 =   −1   2 2 3     6 2  3   3    v1 =  12. v =   3   4  7 5 

 1  −1   v2 =   2  3 

   

 3  1   v3 =   −3  1 

109

In Exercises 13–16, find all the ways that v can be written as a linear combination of the given vectors.     3 3 v1 = 13. v = 0 1     0 −1 v2 = v3 = −1 2     −1 1 v1 = 14. v = −1 −1     −2 3 v3 = v2 = −1 0     0 0 v1 =  1  15. v =  −1  −3 1     −2 −2 v3 =  −3  v2 =  −1  2 −1   2 v4 =  −1  −2     −3 −1 v1 =  −1  16. v =  −3  1 2     0 0 v2 =  −1  v3 =  −1  −1 −2   −3 v4 =  −1  −2 In Exercises 17–20, determine if the matrix M is a linear combination of the matrices M1 , M2 , and M3 .   −2 4 17. M = 4 0     1 2 −2 3 M2 = M1 = 1 −1 1 4   −1 3 M3 = 2 1

Confirming Pages

110

Chapter 2 Linear Combinations and Linear Independence

 18. M =  M1 =  M3 =  19. M =  M1 =

2 3 1 2



2 2 1 1 1 2 3 1

In Exercises 25 and 26, write the polynomial p(x), if possible, as a linear combination of the polynomials



 M2 =



2 1 −1 2 2 2 −1 3

−1 1 2 1



1+x 26. p(x) = −x 2 + 3x + 3



In Exercises 27 and 28, write the polynomial p(x), if possible, as a linear combination of the polynomials



 M2 =

3 2

−1 −2



3 2

product Ax as a vectors of A.  1 22. Let A =  2 −3

x2

25. p(x) = 2x 2 − 3x − 1

 −1 M3 = 2   2 1 20. M = 3 4     1 0 0 1 M1 = M2 = 0 −1 0 0   0 0 M3 = 0 1     2 1 3 . Write the and x = 21. Let A = −1 −2 1 

and

linear combination of the column    −1 2 −1 3 4  and x =  −1 . 3 2 1

Write the product Ax as a linear combination of the column vectors of A.     3 2 −1 −2 . Write and B = 23. Let A = 2 5 3 4 each column vector of AB as a linear combination of the column vectors of A.   2 0 −1 4  and 24. Let A =  1 −1 −4 3 1   3 2 1 1 0 . Write each column B =  −2 2 −1 1 vector of AB as a linear combination of the column vectors of A.

1 + x, −x, x 2 + 1

and

2x 3 − x + 1

27. p(x) = x 3 − 2x + 1 28. p(x) = x 3 29. Describe all vectors in ⺢3 that can be written as a linear combination of the vectors       1 3 1  3   7   2  and 0 −2 −1 30. Describe all 2 × 2 matrices that can be written as a linear combination of the matrices       0 0 0 1 1 0 and 0 1 1 0 0 0 31. If v = v1 + v2 + v3 + v4 and v4 = v1 − 2v2 + 3v3 , write v as a linear combination of v1 , v2 , and v3 . 32. If v = v1 + v2 + v3 + v4 and v2 = 2v1 − 4v3 , write v as a linear combination of v1 , v3 , and v4 . 33. Suppose that the vector v is a linear combination of the vectors v1 , v2 , . . . , vn , and c1 v1 + c2 v2 + · · · + cn vn = 0, with c1 = 0. Show that v is a linear combination of v2 , . . . , vn . 34. Suppose that the vector v is a linear combination of the vectors v1 , v2 , . . . , vn , and w1 , w2 , . . . , wm , are another m vectors. Show that v is a linear combination of v1 , v2 , . . . , vn , w1 , w2 , . . . , wm . 35. Let S1 be the set of all linear combinations of the vectors v1 , v2 , . . . , vk in ⺢n , and S2 be the set of all linear combinations of the vectors v1 , v2 , . . . ,

Confirming Pages

2.3 Linear Independence

vk , cvk , where c is a nonzero scalar. Show that S1 = S2 . 36. Let S1 be the set of all linear combinations of the vectors v1 , v2 , . . . , vk in ⺢n , and S2 be the set of all linear combinations of the vectors v1 , v2 , . . . , vk , v1 + v2 . Show that S1 = S2 . 37. Suppose that Ax = b is a 3 × 3 linear system that is consistent. If there is a scalar c such that A3 = cA1 , then show that the linear system has infinitely many solutions. ß

2.3 y

v u = 12 v x

−v

111

38. Suppose that Ax = b is a 3 × 3 linear system that is consistent. If A3 = A1 + A2 , then show that the linear system has infinitely many solutions. 39. The equation 2y  − 3y  + y = 0 is an example of a differential equation. Show that 1 y = f (x) = ex and y = g(x) = e 2 x are solutions to the equation. Then show that any linear combination of f (x) and g(x) is another solution to the differential equation.

Linear Independence In Sec. 2.2 we saw that given a set S of vectors in ⺢n , it is not always possible to express every vector in ⺢n as a linear combination of vectors from S. At the other extreme, there are infinitely many different subsets S such that the collection of all linear combinations of vectors from S is ⺢n . For example, the collection of all linear combinations of the set of coordinate vectors S = {e1 , . . . , en } is ⺢n , but so is the collection of linear combinations of T = {e1 , . . . , en , e1 + e2 }. In this way S and T both generate ⺢n . To characterize those minimal sets S that generate ⺢n , we require the concept of linear independence. As motivation let two vectors u and v in ⺢2 lie on the same line, as shown in Fig. 1. Thus, there is a nonzero scalar c such that u = cv

Figure 1

This condition can also be written as

y

u − cv = 0

v u x

In this case we say that the vectors u and v are linearly dependent. Evidently we have that two vectors u and v are linearly dependent provided that the zero vector is a nontrivial (not both scalars 0) linear combination of the vectors. On the other hand, the vectors shown in Fig. 2 are not linearly dependent. This concept is generalized to sets of vectors in ⺢n .

Figure 2

DEFINITION 1

Linearly Indpendent and Linearly Dependent The set of vectors S = {v1 , v2 , . . . , vm } in ⺢n is linearly independent provided that the only solution to the equation c1 v1 + c2 v2 + · · · + cm vm = 0 is the trivial solution c1 = c2 = · · · = cm = 0. If the above linear combination has a nontrivial solution, then the set S is called linearly dependent.

Confirming Pages

112

Chapter 2 Linear Combinations and Linear Independence

For example, the set of coordinate vectors S = {e1 , . . . , en } in ⺢n is linearly independent.

EXAMPLE 1

Determine whether the vectors    1 0  0   1  v2 =  v1 =   1   1 2 2



 1  1   v3 =   1  3 

  

and

are linearly independent or linearly dependent. Solution

We seek solutions to the vector equation      1 0  0   1       c1   1  + c2  1  + c3  2 2

  0 1  1  = 0 1   0 0 3

   

From this we obtain the linear system  c1 +   

c3 = 0 c2 + c3 = 0 c 1 + c2 + c3 = 0    2c1 + 2c2 + 3c3 = 0

Subtracting the first equation from the third equation gives c2 = 0. Then, from equation 2, we have c3 = 0 and from equation 1 we have c1 = 0. Hence, the only solution to the linear system is the trivial solution c1 = c2 = c3 = 0. Therefore, the vectors are linearly independent.

EXAMPLE 2

Determine whether the vectors     1 −1 v2 =  1  v1 =  0  2 2

 −2 v3 =  3  1 

 2 v4 =  1  1 

are linearly independent. Solution

As in Example 1, we need to solve           0 1 −1 −2 2 c1  0  + c 2  1  + c 3  3  + c 4  1  =  0  0 2 2 1 1

Confirming Pages

2.3 Linear Independence

113

This leads to the homogeneous linear system   c1 − c2 − 2c3 + 2c4 = 0 c2 + 3c3 + c4 = 0  2c1 + 2c2 + c3 + c4 = 0 with solution set given by S = {(−2t, 2t, −t, t) | t ∈ ⺢} Since the linear system has infinitely many solutions, the set of vectors {v1 , v2 , v3 , v4 } is linearly dependent.

In Example 2, we verified that the set of vectors {v1 , v2 , v3 , v4 } is linearly dependent. Observe further that the vector v4 is a linear combination of v1 , v2 , and v3 , that is, v4 = 2v1 − 2v2 + v3 In Theorem 3 we establish that any finite collection of vectors in ⺢n , where the number of vectors exceeds n, is linearly dependent.

THEOREM 3

Let S = {v1 , v2 , . . . , vn } be a set of n nonzero vectors in ⺢m . If n > m, then the set S is linearly dependent. Proof Let A be the m × n matrix with column vectors the vectors of S so that Ai = vi

for

i = 1, 2, . . . , n

In this way we have c1 v1 + c2 v2 + · · · + cn vn = 0 in matrix form, is the homogeneous linear system  c1  c2 Ac = 0 where c=  ...

   

cn As A is not square with n > m, there is at least one free variable. Thus, the solution is not unique and S = {v1 , . . . , vn } is linearly dependent.

Notice that from Theorem 3, any set of three or more vectors in ⺢2 , four or more vectors in ⺢3 , five or more vectors in ⺢4 , and so on, is linearly dependent. This theorem does not address the case for which n ≤ m. In this case, a set of n vectors in ⺢m may be either linearly independent or linearly dependent. The notions of linear independence and dependence can be generalized to include other objects, as illustrated in Example 3.

Confirming Pages

114

Chapter 2 Linear Combinations and Linear Independence

EXAMPLE 3

Determine whether the matrices     1 0 −1 2 M2 = M1 = 3 2 3 2

 M3 =

and

5 −6 −3 −2



are linearly independent. Solution

Solving the equation      1 0 −1 2 5 + c2 + c3 c1 3 2 3 2 −3

−6 −2



 =

0 0 0 0



is equivalent to solving     c1 − c2 + 5c3 2c2 − 6c3 0 0 = 0 0 3c1 + 3c2 − 3c3 2c1 + 2c2 − 2c3 Equating corresponding entries gives the linear system  c1 − c2 + 5c3 = 0    2c2 − 6c3 = 0 3c + 3c2 − 3c3 = 0  1   2c1 + 2c2 − 2c3 = 0 The augmented matrix of the linear system is   1 −1 5 0  0 2 −6 0    which reduces to  3 3 −3 0  2 2 −2 0



1  0   0 0

0 1 0 0

2 −3 0 0

 0 0   0  0

Therefore, the solution set is S = {(−2t, 3t, t) | t ∈ ⺢} Since the original equation has infinitely many solutions, the matrices are linearly dependent.

Criteria to determine if a set of vectors is linearly independent or dependent are extremely useful. The next several theorems give situations where such a determination can be made. THEOREM 4

If a set of vectors S = {v1 , v2 , . . . , vn } contains the zero vector, then S is linearly dependent. Proof Suppose that the vector vk = 0, for some index k, with 1 ≤ k ≤ n. Setting c1 = c2 = · · · = ck−1 = 0, ck = 1, and ck+1 = ck+2 = · · · = cn = 0, we have 0v1 + · · · + 0vk−1 + 1vk + 0vk+1 + · · · + 0vn = 0 which shows that the set of vectors is linearly dependent.

Confirming Pages

2.3 Linear Independence

THEOREM 5

115

A set of nonzero vectors is linearly dependent if and only if at least one of the vectors is a linear combination of other vectors in the set. Proof Let S = {v1 , v2 , . . . , vn } be a set of nonzero vectors that is linearly dependent. Then there are scalars c1 , c2 , . . . , cn , not all 0, with c1 v1 + c2 v2 + · · · + cn vn = 0 Suppose that ck = 0 for some index k. Then solving the previous equation for the vector vk , we have c1 ck−1 ck+1 cn vk = − v1 − · · · − vk−1 − vk+1 − · · · − vn ck ck ck ck Conversely, let vk be such that vk = c1 v1 + c2 v2 + · · · + ck−1 vk−1 + ck+1 vk+1 + · · · + cn vn Then c1 v1 + c2 v2 + · · · + ck−1 vk−1 − vk + ck+1 vk+1 + · · · + cn vn = 0 Since the coefficient of vk is −1, the linear system has a nontrivial solution. Hence, the set S is linearly dependent. As an illustration, let S be the set of vectors      2  −1  1 S =  3 ,  2 ,  6    2 1 1 Notice that the third vector is twice the first vector, that is,     1 2  6  = 2 3  1 2 Thus, by Theorem 5, the set S is linearly dependent.

EXAMPLE 4

Verify that the vectors   −1 v1 = 0

 v2 =

0 1



 and

v3 =

1 0



are linearly dependent. Then show that not every vector can be written as a linear combination of the others. Solution

By Theorem 3, any three vectors in ⺢2 are linearly dependent. Now, observe that v1 and v3 are linear combinations of the other two vectors, that is, v1 = 0v2 − v3

and

v3 = 0v2 − v1

Confirming Pages

116

Chapter 2 Linear Combinations and Linear Independence

However, v2 cannot be written as a linear combination of v1 and v3 . To see this, notice that the equation av1 + bv3 = v2

y

v2 v1

v3 x

is equivalent to the inconsistent linear system  −a + b = 0 0 =1 As shown in Fig. 3, any linear combination of the vectors v1 and v3 is a vector that is along the x axis. Therefore, v2 is not a linear combination of v1 and v3 .

Figure 3

THEOREM 6

1. If a set of vectors S is linearly independent, then any subset of S is also a linearly independent set of vectors. 2. If a set of vectors T is linearly dependent and S is a set of vectors that contains T , then S is also a linearly dependent set of vectors. Proof (1) Let T be a subset of S. Reorder and relabel the vectors of S, if necessary, so that T = {v1 , . . . , vk } and S = {v1 , . . . , vk , vk+1 , . . . , vm }. Consider the equation c1 v1 + c2 v2 + · · · + ck vk = 0 Next let ck+1 = ck+2 = · · · = cm = 0, and consider the linear combination c1 v1 + c2 v2 + · · · + ck vk + 0vk+1 + · · · + 0vm = 0 Since S is linearly independent, c1 = c2 = · · · = ck = 0 and hence T is linearly independent. (2) Let T = {v1 , . . . , vk } and suppose that T ⊂ S. Label the vectors of S that are not in T as vk+1 , . . . , vm . Since T is linearly dependent, there are scalars c1 , . . . , ck , not all 0, such that c1 v1 + c2 v2 + · · · + ck vk = 0 Then c1 , c2 , . . . , ck , ck+1 = ck+2 = · · · = cm = 0 is a collection of m scalars, not all 0, with c1 v1 + c2 v2 + · · · + ck vk + 0vk+1 + · · · + 0vm = 0 Consequently, S is linearly dependent.

Given a set of vectors S = {v1 , . . . , vn } and an arbitrary vector not in S, we have seen that it may or may not be possible to write v as a linear combination of S. We have also seen that sometimes v can be written as a linear combination of the vectors of S in infinitely many ways. That this cannot happen for a linearly independent set is the content of Theorem 7.

Confirming Pages

2.3 Linear Independence

THEOREM 7

117

Let S = {v1 , v2 , . . . , vn } be a linearly independent set. Suppose that there are scalars c1 , c2 , . . . , cn such that n  ck vk v= k=1

Then the scalars are unique. Proof To prove the result, let v be written as v=

n 

v=

and as

ck vk

k=1

n 

dk vk

k=1

Then 0=v−v=

n 

ck vk −

k=1

=

n 

dk vk

k=1

n  (ck − dk )vk k=1

Since the set of vectors S is linearly independent, the only solution to this last equation is the trivial one. That is, c1 − d1 = 0, c2 − d2 = 0, . . . , cn − dn = 0,

or c1 = d1 , c2 = d2 , . . . , cn = dn

Linear Systems At the end of Sec. 2.2, in Theorem 2, we made the observation that a linear system Ax = b is consistent if and only if the vector b is a linear combination of the column vectors of the matrix A. Theorem 8 gives criteria for when the solution is unique.

THEOREM 8

Let Ax = b be a consistent m × n linear system. The solution is unique if and only if the column vectors of A are linearly independent. Proof First we prove that the condition is necessary. Suppose that the column vectors A1 , A2 , . . . , An are linearly independent, and let     c1 d1  c2   d2    c= and d=  ...   ...  cn

dn

Confirming Pages

118

Chapter 2 Linear Combinations and Linear Independence

be solutions to the linear system. In vector form, we have c 1 A1 + c 2 A2 + · · · + c n An = b

and

d1 A1 + d2 A2 + · · · + dn An = b

By Theorem 7, c1 = d1 , c2 = d2 , . . . , cn = dn . Hence, c = d and the solution to the linear system is unique. To prove the sufficiency, we will prove the contrapositive statement. Let v be a solution to the linear system Ax = b, and assume that the column vectors of A are linearly dependent. Then there are scalars c1 , c2 , . . . , cn , not all 0, such that   that is, if c =  



c 1 A1 + c 2 A2 + · · · + c n An = 0

c1 c2  ..  , then Ac = 0. Since matrix multiplication satisfies the dis. 

cn tributive property, A(v + c) = Av + Ac = b + 0 = b Therefore, the vector v + c is another solution to the linear system, and the solution is not unique. This completes the proof of the contrapositive statement. Therefore, we have shown that if the solution is unique, then the column vectors of A are linearly independent. Theorem 8 provides another way of establishing Theorem 11 of Sec. 1.5 that a linear system has no solutions, one solution, or infinitely many solutions.

Linear Independence and Determinants In Chap. 1, we established that a square matrix A is invertible if and only if det(A) = 0. (See Theorem 16 of Sec. 1.6.) This is equivalent to the statement that the linear system Ax = b has a unique solution for every b if and only if det(A) = 0. This gives an alternative method for showing that a set of vectors is linearly independent. Specifically, if A is a square matrix, then by Theorem 8, the column vectors of A are linearly independent if and only if det(A) = 0.

EXAMPLE 5

Solution

     1  1  1 S =  0 ,  2 ,  4    5 4 3 Determine whether the set S is linearly independent. Let

Let A be the matrix whose column vectors  1 A= 0 3

are the vectors of S; that is,  1 1 2 4  4 5

Confirming Pages

2.3 Linear Independence

The determinant of A can be found by expanding along the first       1 1  1 1   2 4      + 3  − 0 det(A) = 1    2 4 4 5 4 5 = −6 − 0 + 3(2) = 0

119

column, so that    

Therefore, by the previous remarks S is linearly dependent.

The final theorem summarizes the connections that have thus far been established concerning solutions to a linear system, linear independence, invertibility of matrices, and determinants. THEOREM 9

Let A be a square matrix. Then the following statements are equivalent. 1. 2. 3. 4. 5. 6.

The The The The The The

matrix A is invertible. linear system Ax = b has a unique solution for every vector b. homogeneous linear system Ax = 0 has only the trivial solution. matrix A is row equivalent to the identity matrix. determinant of the matrix A is nonzero. column vectors of A are linearly independent.

Fact Summary Let S be a set of m vectors in ⺢n . 1. If m > n, then S is linearly dependent. 2. If the zero vector is in S, then S is linearly dependent. 3. If u and v are in S and there is a scalar c such that u = cv, then S is linearly dependent. 4. If any vector in S is a linear combination of other vectors in S, then S is linearly dependent. 5. If S is linearly independent and T is a subset of S, then T is linearly independent. 6. If T is linearly dependent and T is a subset of S, then S is linearly dependent. 7. If S = {v1 , . . . , vm } is linearly independent and v = c1 v1 + · · · + cm vm , then the set of scalars c1 , . . . , cm is uniquely determined. 8. The linear system Ax = b has a unique solution if and only if the column vectors of A are linearly independent. 9. If A is a square matrix, then the column vectors of A are linearly independent if and only if det(A) = 0.

Confirming Pages

120

Chapter 2 Linear Combinations and Linear Independence

Exercise Set 2.3 In Exercises 1–10, determine whether the given vectors are linearly independent.     −1 2 v2 = 1. v1 = 1 −3  2. v1 =  3. v1 =  4. v1 =  v3 =

2 −4 1 −4 1 0



 v2 =





v3

−2 8





v2 =

0 −1

 10. v1



   −1 −2 5. v1 =  2  v2 =  2  1 3 

   4 −2 6. v1 =  2  v2 =  −1  −6 3 

   −4 −5 7. v1 =  4  v2 =  3  −1 3 

 3 v3 =  −5  5 

   3 −1 8. v1 =  −3  v2 =  2  −1 −2 

 −1 v3 =  3  1 

9. v1



v2 =



1 −1

1 2

   3 1  −1   0     =  −1  v2 =  2  2 1   3  −1   =  0  1     −2 3  −4   −4     =  1  v2 =  0  1 4   −1  −12   =  2  6 

v3

In Exercises 11–14, determine whether the matrices are linearly independent.     3 3 0 1 11. M1 = M2 = 2 1 0 0   1 −1 M3 = −1 −2     −1 2 1 4 12. M1 = M2 = 1 1 0 1   2 2 M3 = −1 0     1 −2 0 −1 M2 = 13. M1 = −2 −2 2 2     −1 1 1 1 M4 = M3 = −2 2 −1 −2     0 −1 −2 −1 M2 = 14. M1 = −1 1 1 −1     2 0 −2 2 M4 = M3 = −1 2 2 −1

Confirming Pages

121

2.3 Linear Independence

In Exercises 15–18, explain, without solving a linear system, why the set of vectors is linearly dependent.    1  −1 2 v2 = 15. v1 = 4 −2    2 −1 v2 = = −1 −2   1 = 3     1 0 =  −6  v2 =  0  2 0   4 = 7  1     1 1 =  0  v2 =  −2  −2 1   2 =  −2  −1 

16. v1 v3

17. v1

v3

18. v1

v3

20. a. A =    b. A =  

are linearly independent. 22. Determine the values of a such that the matrices       1 −4 1 0 1 2 a −2 1 0 0 1 are linearly independent. 23. Let

 1 v1 =  1  1 

 1 v2 =  2  3

 1 v3 =  1  2





a. Show that the vectors are linearly independent.

In Exercises 19 and 20, explain, without solving a linear system, why the column vectors of the matrix A are linearly dependent.   −1 2 5 19. a. A =  3 −6 3  2 −4 3   2 1 3 b. A =  1 0 1  −1 1 0 

21. Determine the values of a such that the vectors       2 −1 1  a   0   2  4 1 1



4 −1 2 6 5 −2 0 2  −2 4 −3 −2  −1 2 3 2 3 1   −1 −1 0  3 5 2

b. Find the unique scalars c1 , c2 , c3 such that the vector   2 v= 1  3 can be written as v = c1 v1 + c2 v2 + c3 v3 24. Let

 M1 =

1 0 −1 0



M3 =

 M2 =



0 1 1 1

1 1 1 0





a. Show that the matrices are linearly independent. b. Find the unique scalars c1 , c2 , c3 such that the matrix   3 5 M= 4 3 can be written as M = c1 M1 + c2 M2 + c3 M3

Confirming Pages

122

Chapter 2 Linear Combinations and Linear Independence

c. Show that the matrix



M=

0 3 3 1



cannot be written as a linear combination of M1, M2 , and M3 . In Exercises 25 and 26, for the given matrix A determine if the linear system Ax = b has a unique solution.   1 2 0 25. A =  −1 0 3  2 1 2   3 2 4 4  26. A =  1 −1 0 2 −4 In Exercises 27–30, determine whether the set of polynomials is linearly independent or linearly dependent. A set of polynomials S = {p1 (x), p2 (x), . . . , pn (x)} is linearly independent provided c1 p1 (x) + c2 p2 (x) + · · · + cn pn (x) = 0

31. f1 (x) = cos πx f2 (x) = sin πx 32. f1 (x) = ex f2 (x) = e−x f3 (x) = e2x 33. f1 (x) = x f2 (x) = x 2 f3 (x) = ex 34. f1 (x) = x f2 (x) = ex f3 (x) = sin πx 35. Verify that two vectors u and v in ⺢n are linearly dependent if and only if one is a scalar multiple of the other. 36. Suppose that S = {v1 , v2 , v3 } is linearly independent and w1 = v1 + v2 + v3 and w3 = v3 Show that T = {w1 , w2 , w3 } is linearly independent. 37. Suppose that S = {v1 , v2 , v3 } is linearly independent and w1 = v1 + v2

for all x implies that c 1 = c2 = · · · = cn = 0 27. p1 (x) = 1 p2 (x) = −2 + p3 (x) = 2x p4 (x) = −12x + 8x 3 4x 2

28. p1 (x) = 1 p2 (x) = x p3 (x) = 5 + 2x − x 2 29. p1 (x) = 2 p2 (x) = x p3 (x) = x 2 p4 (x) = 3x − 1 30. p1 (x) = x 3 − 2x 2 + 1 p2 (x) = 5x p3 (x) = x 2 − 4 p4 (x) = x 3 + 2x In Exercises 31–34, show that the set of functions is linearly independent on the interval [0, 1]. A set of functions S = {f1 (x), f2 (x), . . . , fn (x)} is linearly independent on the interval [a, b] provided c1 f1 (x) + c2 f2 (x) + · · · + cn fn (x) = 0 for all x ∈ [a, b] implies that c1 = c2 = · · · = cn = 0

w2 = v2 + v3

w2 = v2 − v3

and w3 = v2 + v3 Show that T = {w1 , w2 , w3 } is linearly independent. 38. Suppose that S = {v1 , v2 , v3 } is linearly independent and w1 = v2

w2 = v1 + v3

and w3 = v1 + v2 + v3 Determine whether the set T = {w1 , w2 , w3 } is linearly independent or linearly dependent. 39. Suppose that the set S = {v1 , v2 } is linearly independent. Show that if v3 cannot be written as a linear combination of v1 and v2 , then {v1 , v2 , v3 } is linearly independent. 40. Let S = {v1 , v2 , v3 }, where v3 = v1 + v2 . a. Write v1 as a linear combination of the vectors in S in three different ways.

Confirming Pages

2.3 Linear Independence

b. Find all scalars c1 , c2 , and c3 such that v1 = c1 v1 + c2 v2 + c3 v3 . 41. Show that if the column vectors of an m × n matrix A1 , . . . , An are linearly independent, then {x ∈ ⺢n | Ax = 0} = {0} .

123

42. Let v1 , . . . , vk be linearly independent vectors in ⺢n , and suppose A is an invertible n × n matrix. Define vectors wi = Avi , for i = 1, . . . , k. Show that the vectors w1 , . . . , wk are linearly independent. Show, using a 2 × 2 matrix, that the requirement of invertibility is necessary.

Review Exercises for Chapter 2 1. If ad − bc = 0, show that the vectors     c a and d b are linearly independent. Suppose that ad − bc = 0. What can you say about the two vectors? 2. Suppose that S = {v1 , v2 , v3 } is a linearly independent set of vectors in ⺢n . Show that T = {v1 , v2 , v1 + v2 + v3 } is also linearly independent. 3. Determine for which nonzero values of a the vectors     2   1 0 a  0   0   a  and 1 2 1 are linearly independent. 4. Let

 2s − t    s S=   t    s

         s, t ∈ ⺢      

a. Find two vectors in ⺢4 so that all vectors in S can be written as a linear combination of the two vectors. b. Are the vectors found in part (a) linearly independent? 5. Let

 1 v1 =  0  2 

 1 v2 =  1  1 

and

a. Is S = {v1 , v2 } linearly independent?   a b. Find a vector  b  that cannot be written c as a linear combination of v1 and v2 . c. Describe all vectors in ⺢3 that can be written as a linear combination of v1 and v2 . d. Let

 1 v3 =  0  0 

Is T = {v1 , v2 , v3 } linearly independent or linearly dependent? e. Describe all vectors in ⺢3 that can be written as a linear combination of v1 , v2 , and v3 . 6. Let  1 v1 =  −1  1   0 v3 =  2  1 

 2 v2 =  1  1 

 −2 v4 =  2  1 

and

a. Show that S = {v1 , v2 , v3 , v4 } is linearly dependent. b. Show that T = {v1 , v2 , v3 } is linearly independent. c. Show that v4 can be written as a linear combination of v1 , v2 , and v3 .

Confirming Pages

124

Chapter 2 Linear Combinations and Linear Independence

d. How does the set of all linear combinations of vectors in S compare with the set of all linear combinations of vectors in T ? 7. Consider the linear system  x + y + 2z + w = 3    −x + z + 2w = 1 2x + 2y + w = −2    x + y + 2z + 3w = 5

9. Let

b. Find the determinant of the coefficient matrix A. c. Are the column vectors of A linearly independent? d. Without solving the linear system, determine whether it has a unique solution. e. Solve the linear system. 

1 0 M1 = −1 1   0 1 M3 = 1 0



 M2 =

1 2

1 1



a. Show that the set {M1 , M2 , M3 } is linearly independent. b. Find the unique scalars c1 , c2 , and c3 such that   1 −1 = c1 M1 + c2 M2 + c3 M3 2 1 c. Can the matrix



1 −1 1 2



1 A= 2 1

a. Write the linear system in the matrix form Ax = b.

8. Let

 a b that can be d. Describe all matrices c d written as a linear combination of M1, M2 , and M3 . 



be written as a linear combination of M1, M2 , and M3 ?

 3 2 −1 3  1 −1

a. Write the linear system Ax = b in vector form. b. Compute det(A). What can you conclude as to whether the linear system is consistent or inconsistent? c. Are the column vectors of A linearly independent? d. Without solving the linear system, does the system have a unique solution? Give two reasons. 10. Two vectors in ⺢n are perpendicular provided their dot product is 0. Suppose S = {v1 , v2 , . . . , vn } is a set of nonzero vectors which are pairwise perpendicular. Follow the steps to show S is linearly independent. a. Show that for any vector v the dot product satisfies v · v ≥ 0. b. Show that for any vector v = 0 the dot product satisfies v · v > 0. c. Show that for all vectors u, v, and w the dot product satisfies u · (v + w) = u · v + u · w d. Consider the equation c1 v1 + c2 v2 + · · · + cn vn = 0 Use the dot product of vi , for each 1 ≤ i ≤ n, with the expression on the left of the previous equation to show that ci = 0, for each 1 ≤ i ≤ n.

Confirming Pages

125

2.3 Linear Independence

Chapter 2: Chapter Test In Exercises 1–33, determine whether the statement is true or false. 1. Every vector in ⺢3 can be written as a linear combination of       0 0 1  0   1   0  1 0 0 2. Every 2 × 2 matrix can be written as a linear combination of       0 0 0 1 1 0 1 0 0 0 0 0 3. Every 2 × 2 matrix can be written as a linear combination of       0 0 0 1 1 0 1 0 0 0 0 0 

0 0

0 1



In Exercises 4–8, use the vectors     1 2 v1 =  0  v2 =  1  1 0   4 v3 =  3  −1 4. The set S = {v1 , v2 , v3 } is linearly independent. 5. There are scalars c1 and c2 so that v3 = c1 v1 + c2 v2 . 6. The vector v2 can be written as a linear combination of v1 and v3 . 7. The vector v1 can be written as a linear combination of v2 and v3 .

8. If v1 , v2 , and v3 are the column vectors of a 3 × 3 matrix A, then the linear system Ax = b has a unique solution for all vectors b in ⺢3 . 9. The polynomial p(x) = 3 + x can be written as a linear combination of q1 (x) = 1 + x and q2 (x) = 1 − x − x 2 . 10. The set   1  S =  −1 ,   3  2  −2 6

 1 0 , 1 

 0  ,  1   0

is linearly independent. In Exercises 11–14, use the matrices     1 −1 0 0 M1 = M2 = 0 0 1 0  M3 =

0 0 0 1



 M4 =

2 1

−1 3



11. The set S = {M1 , M2 , M3 , M4 } is linearly independent. 12. The set T = {M1 , M2 , M3 } is linearly independent. 13. The set of all linear combinations of matrices in S is equal to the set of all linear combinations of matrices in T . 14. Every matrix that can be written as a linear combination of the matrices in T has the form   x −x y z

Confirming Pages

126

Chapter 2 Linear Combinations and Linear Independence

15. The vectors 

 s  0  0

 1  s  1 

 0  1  s 

are linearly independent if and only if s = 0 or s = 1. In Exercises 16–19, use the vectors     1 3 v1 = v2 = 1 −1 16. The set S = {v1 , v2 } is linearly independent. 17. Every vector in ⺢2 can be written as a linear combination of v1 and v2 . 18. If the column vectors of a matrix A are v1 and v2 , then det(A) = 0. 19. If b is in

⺢2

and c1 v1 + c2 v2 = b, then   c1 = A−1 b c2

where A is the 2 × 2 matrix with column vectors v1 and v2 . 20. The column vectors of the matrix   cos θ sin θ − sin θ cos θ are linearly independent. 21. If v1 and v2 are linearly independent vectors in ⺢n and v3 cannot be written as a scalar multiple of v1 , then v1 , v2 , and v3 are linearly independent. 22. If S = {v1 , v2 , . . . , vm } is a set of nonzero vectors in ⺢n that are linearly dependent, then every

vector in S can be written as a linear combination of the others. 23. If v1 and v2 are in ⺢3 , then the matrix with column vectors v1 , v2 , and v1 + v2 has a nonzero determinant. 24. If v1 and v2 are linearly independent, v1 , v2 , and v1 + v2 are also linearly independent. 25. If the set S contains the zero vector, then S is linearly dependent. 26. The column vectors of an n × n invertible matrix can be linearly dependent. 27. If A is an n × n matrix with linearly independent column vectors, then the row vectors of A are also linearly independent. 28. If the row vectors of a nonsquare matrix are linearly independent, then the column vectors are also linearly independent. 29. If v1 , v2 , v3 , and v4 are in ⺢4 and {v1 , v2 , v3 } is linearly dependent, then {v1 , v2 , v3 , v4 } is linearly dependent. 30. If v1 , v2 , v3 , and v4 are in ⺢4 and {v1 , v2 , v3 } is linearly independent, then {v1 , v2 , v3 , v4 } is linearly independent. 31. If v1 , v2 , v3 , and v4 are in ⺢4 and {v1 , v2 , v3 , v4 } is linearly independent, then {v1 , v2 , v3 } is linearly independent. 32. If v1 , v2 , v3 , and v4 are in ⺢4 and {v1 , v2 , v3 , v4 } is linearly dependent, then {v1 , v2 , v3 } is linearly dependent. 33. If S = {v1 , v2 , . . . , v5 } is a subset of ⺢4 , then S is linearly dependent.

Confirming Pages

CHAPTER

Vector Spaces CHAPTER OUTLINE

3.1 3.2 3.3 3.4 3.5

Definition of a Vector Space 129 Subspaces 140 Basis and Dimension 156 Coordinates and Change of Basis 173 Application: Differential Equations 185

W

hen a digital signal is sent through space (sometimes across millions of miles), errors in the signal are bound to occur. In response to the need for reliable information, mathematicians and scientists from a variety of disciplines have developed ways to improve the quality of these transmissions. One obvious method is to send messages repeatedly to increase the likelihood of receiving them correctly. This, however, is time-consuming and limits the number of messages that can be sent. An innovative methodology developed by Richard Hamming in 1947 involves embedding in the transmission a means for error detection and self-correction. One of Hamming’s coding schemes, known as Hamming’s (7,4) code, uses binary vectors (vectors consisting of 1s and 0s) with seven components. Some of these vectors are identified as codewords © Brand X Pictures/PunchStock/RF depending on the configuration of the 1s and 0s within it. To decide if the binary vector   b1  b2   b=  ...  b7

127

Confirming Pages

128

Chapter 3 Vector Spaces

is a codeword, a test using matrix multiplication is  1 1 1 0 1 C= 0 1 1 1 0 1 0 1 1 0

performed. The matrix given by  0 0 1 0  0 1

is called the check matrix. To carry out the test, we compute the product of C and b, using modulo 2 arithmetic, where an even result corresponds to a 0 and an odd result corresponds to a 1. This product produces a binary vector with three components called the syndrome vector given by Cb = s A binary vector b is a codeword if the syndrome vector s = 0. Put another way, b is a codeword if it is a solution to the homogeneous equation Cb ≡ 0 (mod 2). For example, the vector   1  1     0     u=  0   0     1  1 is a codeword since   1  1           0 2 1 1 1 0 1 0 0   0     (mod 2)   Cu =  0 1 1 1 0 1 0    0 = 2 ≡ 0  0 2 0 1 0 1 1 0 0 1     1  1 whereas the vector   1  1     1     v=  0   0     0  0

   1 3 Cv =  2  ≡  0  (mod 2) 0 2 

is not since

With this ingenious strategy the recipient of a legitimate codeword can safely assume that the vector is free from errors. On the other hand, if the vector received is not a codeword, an algorithm involving the syndrome vector can be applied to restore it to the original. In the previous example the fifth digit of v was altered during the

Confirming Pages

3.1 Definition of a Vector Space

transmission. The intended vector is given by  1  1   1  ∗ v =  0  1   0 0

129

         

Hamming’s (7,4) code is classified as a linear code since the sum of any two codewords is also a codeword. To see this, observe that if u and v are codewords, then the sum u + v is also a codeword since C(u + v) = Cu + Cv = 0 + 0 = 0 (mod 2) It also has the property that every codeword can be written as a linear combination of a few key codewords.

In this chapter we will see how the set of all linear combinations of a set of vectors forms a vector space. The set of codewords in the chapter opener is an example. ß

3.1

Definition of a Vector Space In Chap. 2 we defined a natural addition and scalar multiplication on vectors in ⺢n as generalizations of the same operations on real numbers. With respect to these operations, we saw in Theorem 1 of Sec. 2.1 that sets of vectors satisfy many of the familiar algebraic properties enjoyed by numbers. In this section we use these properties as axioms to generalize the concept of a vector still further. In particular, we consider as vectors any class of objects with definitions for addition and scalar multiplication that satisfy the properties of this theorem. In this way our new concept of a vector will include vectors in ⺢n but many new kinds as well.

DEFINITION 1

Vector Space A set V is called a vector space over the real numbers provided that there are two operations—addition, denoted by ⊕, and scalar multiplication, denoted by —that satisfy all the following axioms. The axioms must hold for all vectors u, v, and w in V and all scalars c and d in ⺢. 1. The sum u ⊕ v is in V . 2. u ⊕ v = v ⊕ u 3. (u ⊕ v) ⊕ w = u ⊕ (v ⊕ w)

Closed under addition Addition is commutative Addition is associative

Confirming Pages

130

Chapter 3 Vector Spaces

4. There exists a vector 0 ∈ V such that for every vector u ∈ V , 0 ⊕ u = u ⊕ 0 = u. 5. For every vector u ∈ V , there exists a vector, denoted by −u, such that u ⊕ (−u) = −u ⊕ u = 0. 6. The scalar product c  u is in V .

Additive identity Additive inverse

Closed under scalar multiplication

7. c  (u ⊕ v) = (c  u) ⊕ (c  v) 8. (c + d)  u = (c  u) ⊕ (d  u) 9. c  (d  u) = (cd)  u 10 1  u = u

In this section (and elsewhere when necessary) we use the special symbols ⊕ and  of the previous definition to distinguish vector addition and scalar multiplication from ordinary addition and multiplication of real numbers. We also will point out that for general vector spaces the set of scalars can be chosen from any field. In this text, unless otherwise stated, we chose scalars from the set of real numbers.

EXAMPLE 1 Solution

EXAMPLE 2 Solution

Euclidean Vector Spaces The set V = ⺢n with the standard operations of addition and scalar multiplication is a vector space. Axioms 2 through 5 and 7 through 10 are shown to hold in Theorem 1 of Sec. 2.1. The fact that ⺢n is closed under addition and scalar multiplication is a direct consequence of how these operations are defined. The Euclidean vector spaces ⺢n are the prototypical vector spaces on which the general theory of vector spaces is built.

Vector Spaces of Matrices Show that the set V = Mm×n of all m × n matrices is a vector space over the scalar field ⺢, with ⊕ and  defined componentwise. Since addition of matrices is componentwise, the sum of two m × n matrices is another m × n matrix as is a scalar times an m × n matrix. Thus, the closure axioms (axioms 1 and 6) are satisfied. We also have that 1  A = A. The other seven axioms are given in Theorem 4 of Sec. 1.3.

Confirming Pages

3.1 Definition of a Vector Space

131

When we are working with more abstract sets of objects, the operations of addition and scalar multiplication can be defined in nonstandard ways. The result is not always a vector space. This is illustrated in the next several examples.

EXAMPLE 3

Let V = ⺢. Define addition and scalar multiplication by a ⊕ b = 2a + 2b

k  a = ka

and

Show that addition is commutative but not associative. Solution

Since the usual addition of real numbers (on the right-hand side) is commutative, a ⊕ b = 2a + 2b = 2b + 2a =b⊕a Thus, the operation ⊕ is commutative. To determine whether addition is associative, we evaluate and compare the expressions (a ⊕ b) ⊕ c and a ⊕ (b ⊕ c) In this case, we have (a ⊕ b) ⊕ c = (2a + 2b) ⊕ c = 2(2a + 2b) + 2c = 4a + 4b + 2c

a ⊕ (b ⊕ c) = a ⊕ (2b + 2c) = 2a + 2(2b + 2c) = 2a + 4b + 4c

and

We see that the two final expressions are not equal for all choices of a, b, and c. Therefore, the associative property is not upheld, and V is not a vector space.

EXAMPLE 4

Let V = ⺢. Define addition and scalar multiplication by a ⊕ b = ab

and

k  a = ka

Show that V is not a vector space. Solution

In this case a ⊕ b = ab and b ⊕ a = ba Since a b = ba for all choices of a and b, the commutative property of addition is not upheld, and V is not a vector space.

In Example 5 we show that familiar sets with nonstandard definitions for addition and scalar multiplication can be vector spaces.

Confirming Pages

132

Chapter 3 Vector Spaces

EXAMPLE 5

Let V = {(a, b) | a, b ∈ ⺢}. Let v = (v1 , v2 ) and w = (w1 , w2 ). Define (v1 , v2 ) ⊕ (w1 , w2 ) = (v1 + w1 + 1, v2 + w2 + 1) c  (v1 , v2 ) = (cv1 + c − 1, cv2 + c − 1)

and

Verify that V is a vector space. Solution

First observe that since the result of addition or scalar multiplication is an ordered pair, V is closed under addition and scalar multiplication. Since addition of real numbers is commutative and associative, axioms 2 and 3 hold for the ⊕ defined here. Now an element w ∈ V is the additive identity provided that for all v ∈ V v⊕w=v

or

(v1 + w1 + 1, v2 + w2 + 1) = (v1 , v2 )

Equating components gives v1 + w1 + 1 = v1 w1 = −1

and and

v2 + w2 + 1 = v2 w2 = −1

so

This establishes the existence of an additive identity. Specifically, 0 = (−1, −1), so axiom 4 holds. To show that each element v in V has an additive inverse, we must find a vector w such that v ⊕ w = 0 = (−1, −1) Since v ⊕ w = (v1 + w1 + 1, v2 + w2 + 1), this last equation requires that v1 + w1 + 1 = −1 w1 = −v1 − 2

and and

v2 + w2 + 1= −1 w2 = −v2 − 2

so that

Thus, for any element v = (v1 , v2 ) in V, we have −v = (−v1 − 2, −v2 − 2). The remaining axioms all follow from the similar properties of the real numbers.

A polynomial of degree n is an expression of the form p(x) = a0 + a1 x + a2 x 2 + · · · + an−1 x n−1 + an x n where a0 , . . . , an are real numbers and an = 0. The degree of the zero polynomial is undefined since it can be written as p(x) = 0x n for any positive integer n. Polynomials comprise one of the most basic sets of functions and have many applications in mathematics.

EXAMPLE 6

Vector Space of Polynomials Let n be a fixed positive integer. Denote by Pn the set of all polynomials of degree n or less. Define addition by adding like terms. That is, if p(x) = a0 + a1 x + a2 x 2 + · · · + an−1 x n−1 + an x n

Confirming Pages

3.1 Definition of a Vector Space

133

and q(x) = b0 + b1 x + b2 x 2 + · · · + bn−1 x n−1 + bn x n then p(x) ⊕ q(x) = (a0 + b0 ) + (a1 + b1 )x + (a2 + b2 )x 2 + · · · + (an + bn )x n If c is a scalar, then scalar multiplication is defined by c  p(x) = ca0 + ca1 x + ca2 x 2 + · · · + can−1 x n−1 + can x n Verify that V = Pn ∪ {0} is a real vector space, where 0 is the zero polynomial. Solution

Since the sum of two polynomials of degree n or less is another polynomial of degree n or less, with the same holding for scalar multiplication, the set V is closed under addition and scalar multiplication. The zero vector is just the zero polynomial, and the additive inverse of p(x) is given by −p(x) = −a0 − a1 x − a2 x 2 − · · · − an−1 x n−1 − an x n The remaining axioms are consequences of the properties of real numbers. For example, p(x) ⊕ q(x) = (a0 + b0 ) + (a1 + b1 )x + (a2 + b2 )x 2 + · · · + (an + bn )x n = (b0 + a0 ) + (b1 + a1 )x + (b2 + a2 )x 2 + · · · + (bn + an )x n = q(x) ⊕ p(x)

In the sequel we will use Pn to denote the vector space of polynomials of degree n or less along with the zero polynomial. The condition degree n or less cannot be replaced with all polynomials of degree equal to n. The latter set is not closed under addition. For example, the polynomials x 2 − 2x + 1 and −x 2 + 3x + 4 are both polynomials of degree 2, but the sum is x + 5, which has degree equal to 1.

EXAMPLE 7

Vector Space of Real-Valued Functions Let V be the set of real-valued functions defined on a common domain given by the interval [a, b]. For all f and g in V and c ∈ ⺢, define addition and scalar multiplication, respectively, by (f ⊕ g)(x) = f (x) + g(x)

and

(c  f )(x) = cf (x)

for each x in [a, b]. Show that V is a real vector space. Solution

Since the pointwise sum of two functions with domain [a, b] is another function with domain [a, b], the set V is closed under addition. Similarly, the set V is closed under scalar multiplication.

Confirming Pages

134

Chapter 3 Vector Spaces

To show that addition in V is commutative, let f and g be functions in V . Then (f ⊕ g)(x) = f (x) + g(x) = g(x) + f (x) = (g ⊕ f )(x) Addition is also associative since for any functions f, g, and h in V , we have (f ⊕ (g ⊕ h))(x) = f (x) + (g ⊕ h)(x) = f (x) + g(x) + h(x) = (f ⊕ g)(x) + h(x) = ((f ⊕ g) ⊕ h)(x) The zero element of V , denoted by 0, is the function that is 0 for all real numbers in [a, b]. We have that 0 is the additive identity on V since (f ⊕ 0)(x) = f (x) + 0(x) = f (x) Next we let c and d be real numbers and let f be an element of V . The distributive property of real numbers gives us (c + d)  f (x) = (c + d)f (x) = cf (x) + df (x) = (c  f )(x) ⊕ (d  f )(x) so (c + d)  f = (c  f ) ⊕ (d  f ), establishing property 8. The other properties follow in a similar manner.

The set of complex numbers, denoted by ⺓, is defined by ⺓ = {a + bi | a, b ∈ ⺢} where i satisfies

√ or equivalently i = −1 i 2 = −1 The set of complex numbers is an algebraic extension of the real numbers, which it contains as a subset. For every complex number z = a + bi, the real number a is called the real part of z and the real number b the imaginary part of z. With the appropriate definitions of addition and scalar multiplication, the set of complex numbers ⺓ is a vector space.

EXAMPLE 8

Vector Space of Complex Numbers Let z = a + bi and w = c + di be elements of ⺓ and α a real number. Define vector addition on ⺓ by z ⊕ w = (a + bi) + (c + di) = (a + c) + (b + d)i and scalar multiplication by α  z = α  (a + bi) = αa + (αb)i Verify that ⺓ is a vector space.

Confirming Pages

3.1 Definition of a Vector Space

Solution

135

For each element z = a + bi in ⺓, associate the vector in ⺢2 whose components are the real and imaginary parts of z. That is, let   a z = a + bi ←→ b Observe that addition and scalar multiplication in ⺓ correspond to those same operations in ⺢2 . In this way ⺓ and ⺢2 have the same algebraic structure. Since ⺢2 is a vector space, so is ⺓.

In Example 8, we showed that ⺓ is a vector space over the real numbers. It is also possible to show that ⺓ is a vector space over the complex scalars. We leave the details to the reader. Example 9 is from analytic geometry.

EXAMPLE 9

Let a, b, and c be fixed real numbers. Let V be the set of points in three-dimensional Euclidean space that lie on the plane P given by ax + by + cz = 0 Define addition and scalar multiplication on V coordinatewise. Verify that V is a vector space.

Solution

To show that V is closed under addition, let u = (u1 , u2 , u3 ) and v = (v1 , v2 , v3 ) be points in V . The vectors u and v are in V provided that au1 + bu2 + cu3 = 0

and

av1 + bv2 + cv3 = 0

Now by definition u ⊕ v = (u1 + v1 , u2 + v2 , u3 + v3 ) We know that u ⊕ v is in V since a(u1 + v1 ) + b(u2 + v2 ) + c(u3 + v3 ) = au1 + av1 + bu2 + bv2 + cu3 + cv3 = (au1 + bu2 + cu3 ) + (av1 + bv2 + cv3 ) =0 Similarly, V is closed under scalar multiplication since for any scalar α, we have α  u = (αu1 , αu2 , αu3 ) and a(αu1 ) + b(αu2 ) + c(αu3 ) = α(au1 + bu2 + cu3 ) = α(0) = 0 In this case the zero vector is (0, 0, 0), which is also on the plane P . Since the addition and scalar multiplication defined on V are the analogous operations defined on the vector space ⺢3 , the remaining axioms are satisfied for elements of V as well.

Confirming Pages

136

Chapter 3 Vector Spaces

We conclude this section by showing that some familiar algebraic properties of ⺢n extend to abstract vector spaces. THEOREM 1

In a vector space V , additive inverses are unique. Proof Let u be an element of V . Suppose that v and w are elements of V and both are additive inverses of u. We show that v = w. Since u⊕v=0

and

u⊕w=0

axioms 4, 3, and 2 give v = v ⊕ 0 = v ⊕ (u ⊕ w) = (v ⊕ u) ⊕ w = 0 ⊕ w = w establishing the result.

THEOREM 2

Let V be a vector space, u a vector in V , and c a real number. 1. 2. 3. 4.

0u=0 c0=0 (−1)  u = −u If c  u = 0, then either c = 0 or u = 0.

Proof (1) By axiom 8, we have 0  u = (0 + 0)  u = (0  u) ⊕ (0  u) Adding the inverse −(0  u) to both sides of the preceding equation gives the result. (2) By axiom 4, we know that 0 ⊕ 0 = 0. Combining this with axiom 7 gives c  0 = c  (0 ⊕ 0) = (c  0) ⊕ (c  0) Again adding the inverse −(c  0) to both sides of the last equation gives the result. (3) By axioms 10 and 8 and part 1 of this theorem, u ⊕ (−1)  u = (1  u) ⊕ [(−1)  u] = (1 − 1)  u =0u =0 Thus, (−1)  u is an additive inverse of u. Since −u is by definition the additive inverse of u and by Theorem 1 additive inverses are unique, we have (−1)  u = −u. (4) Let c  u = 0. If c = 0, then the conclusion holds. Suppose that c = 0. Then multiply both sides of cu=0

Confirming Pages

3.1 Definition of a Vector Space

1 and apply part 2 of this theorem to obtain c 1  (c  u) = 0 so that c and hence u = 0.

137

by

1u=0

Fact Summary 1. To determine whether a set V with addition and scalar multiplication defined on V is a vector space requires verification of the 10 vector space axioms. 2. The Euclidean space ⺢n and the set of matrices Mm×n , with the standard componentwise operations, are vector spaces. The set of polynomials of degree n or less with termwise operations is a vector space. 3. In all vector spaces, additive inverses are unique. Also 0u=0

c0=0

and

(−1)  u = −u

In addition if c  u = 0, then either the scalar c is the number 0 or the vector u is the zero vector.

Exercise Set 3.1 In Exercises 1–4, let V = ⺢3 . Show that V with the given operations for ⊕ and  is not a vector space.       x1 x2 x1 − x2 1.  y1  ⊕  y2  =  y1 − y2  z1 z2 z1 − z2



     x1 x2 2x1 + 2x2 3.  y1  ⊕  y2  =  2y1 + 2y2  z1 z2 2z1 + 2z2 

   x1 cx1 c   y1  =  cy1  z1 cz1



   x1 cx1 c   y1  =  cy1  z1 cz1 

     x1 x2 x1 + x2 − 1 2.  y1  ⊕  y2  =  y1 + y2 − 1  z1 + z2 − 1 z1 z2 

   x1 cx1 c   y1  =  cy1  z1 cz1



     x1 x2 x1 + x2 4.  y1  ⊕  y2  =  y1 + y2  z1 z2 z1 + z2 

   x1 c + x1 c   y1  =  y1  z1 z1 5. Write out all 10 vector space axioms to show ⺢2 with the standard componentwise operations is a vector space.

Confirming Pages

138

Chapter 3 Vector Spaces

6. Write out all 10 vector space axioms to show that M2×2 with the standard componentwise operations is a vector space. 7. Let V = ⺢2 and define addition as the standard componentwise addition and define scalar multiplication by     x+c x = c y y Show that V is not a vector space. 8. Let

     a V =  b  a, b ∈ ⺢   1

a. With the standard componentwise operations show that V is not a vector space. b. If addition and scalar multiplication are defined componentwise only on the first two components and the third is always 1, show that V is a vector space. 9. Let V = ⺢2 and define       a + 2c c a = ⊕ b + 2d d b     ca a = c cb b Determine whether V is a vector space.   t t ∈⺢ −3t and let addition and scalar multiplication be the standard operations on vectors. Determine whether V is a vector space. 

10. Let

V =

  t + 1 V = t ∈⺢ 2t and let addition and scalar multiplication be the standard operations on vectors. Determine whether V is a vector space. 

11. Let

12. Let

 V =

a c

b 0

  a, b, c ∈ ⺢

and let addition and scalar multiplication be the standard componentwise operations. Determine whether V is a vector space.   a b a, b, c ∈ ⺢ V = c 1 a. If addition and scalar multiplication are the standard componentwise operations, show that V is not a vector space. b. Define       a+d b+e d e a b = ⊕ c+f 1 f 1 c 1 

13. Let

  ka a b = kc c 1 Show that V is a vector space. 

and

k

kb 1



In Exercises 14–19, let V be the set of 2 × 2 matrices with the standard (componentwise) definitions for vector addition and scalar multiplication. Determine whether V is a vector space. If V is not a vector space, show that at least one of the 10 axioms does not hold. 14. Let V be the set of all skew-symmetric matrices, that is, the set of all matrices such that At = −A. 15. Let V be the set of all upper triangular matrices. 16. Let V be the set of all real symmetric matrices, that is, the set of all matrices such that At = A. 17. Let V be the set of all invertible matrices. 18. Let V be the set of all idempotent matrices. 19. Let B be a fixed matrix, and let V be the set of all matrices A such that AB = 0. 20. Let

 V =

a c

b −a

  a, b, c ∈ ⺢

and define addition and scalar multiplication as the standard componentwise operations. Determine whether V is a vector space. 21. Let V denote the set of 2 × 2 invertible matrices. Define A ⊕ B = AB

c  A = cA

Confirming Pages

3.1 Definition of a Vector Space

a. Determine the additive identity and additive inverse. b. Show that V is not a vector space. 

22. Let

Define

V =

t 1+t





t1 1 + t1

  t ∈⺢

 ⊕ 

t2 1 + t2

Define ⊕ and  by v⊕v=v 

23. Let      1+ t V =  2 − t  t ∈ ⺢   3 + 2t 

   1 + t1 1 + t2  2 − t1  ⊕  2 − t2  3 + 2t1 3 + 2t2   1 + (t1 + t2 ) =  2 − (t1 + t2 )  3 + (2t1 + 2t2 )     1 + ct 1+ t c   2 − t  =  2 − ct  3 + 2ct 3 + 2t

a. Find the additive identity and inverse. b. Show that V is a vector space. c. Verify 0  v = 0 for all v. 24. Let





1 u= 0  1

25. Let v be a vector in ⺢n , and let S = {v}

a. Find the additive identity and inverse. b. Show that V is a vector space. c. Verify that 0  v = 0 for all v.

Define

Show that S with the standard componentwise operations is a vector space.



t1 + t2 = 1 + t1 + t2     ct t = c 1 + ct 1+t





2 v =  −1  1

and S = {au + bv | a, b ∈ ⺢}

139

cv=v

Show that S is a vector space. 26. Let

     x S =  y  3x − 2y + z = 0   z

Show that S with the standard componentwise operations is a vector space. 27. Let S be the set of all vectors   x  y  z in ⺢3 such that x + y − z = 0 and 2x − 3y + 2z = 0. Show that S with the standard componentwise operations is a vector space. 28. Let

 V =

and define



cos t1 sin t1

cos t sin t 

 ⊕ 

  t ∈⺢ cos t2 sin t2



cos(t1 + t2 ) sin(t1 + t2 )     cos ct cos t = c sin ct sin t



=

a. Determine the additive identity and additive inverse. b. Show that V is a vector space. c. Show that if ⊕ and  are the standard componentwise operations, then V is not a vector space.

Confirming Pages

140

Chapter 3 Vector Spaces

29. Let V be the set of all real-valued functions defined on ⺢ with the standard operations that satisfy f (0) = 1. Determine whether V is a vector space. 30. Let V be the set of all real-valued functions defined on ⺢. Define f ⊕ g by (f ⊕ g)(x) = f (x) + g(x) and define c  f by (c  f )(x) = f (x + c) ß

3.2

Determine whether V is a vector space. 31. Let f (x) = x 3 defined on ⺢ and let V = {f (x + t) | t ∈ ⺢} Define f (x + t1 ) ⊕ f (x + t2 ) = f (x + t1 + t2 ) c  f (x + t) = f (x + ct) a. Determine the additive identity and additive inverses. b. Show that V is a vector space.

Subspaces Many interesting examples of vector spaces are subsets of a given vector space V that are vector spaces in their own right. For example, the xy plane in ⺢3 given by      x  y  x, y ∈ ⺢   0 is a subset of ⺢3 . It is also a vector space with the same standard componentwise operations defined on ⺢3 . Another example of a subspace of a vector space is given in Example 9 of Sec. 3.1. The determination as to whether a subset of a vector space is itself a vector space is simplified since many of the required properties are inherited from the parent space.

DEFINITION 1

Subspace A subspace W of a vector space V is a nonempty subset that is itself a vector space with respect to the inherited operations of vector addition and scalar multiplication on V .

The first requirement for a subset W ⊆ V to be a subspace is that W be closed under the operations of V . For example, let V be the vector space ⺢2 with the standard definitions of addition and scalar multiplication. Let W ⊆ ⺢2 be the subset defined by    a a∈⺢ W = 0 Observe that the sum of any two vectors in W is another vector in W , since       a+b b a = ⊕ 0 0 0

Confirming Pages

3.2 Subspaces

141

In this way we say that W is closed under addition. The subset W is also closed under scalar multiplication since for any real number c,     ca a = c 0 0 which is again in W. On the other hand, the subset y

u⊕v

W

a 1

  a ∈ ⺢

is not closed under addition, since       a+b b a = ⊕ 2 1 1

1

u



W=

v x

W is not a subspace of V

Figure 1

which is not in W. See Fig. 1. The subset W is also not closed under scalar multiplication since     ca a = c c 1 which is not in W for all values of c = 1. Now let us suppose that a nonempty subset W is closed under both of the operations on V . To determine whether W is a subspace, we must show that each of the remaining vector space axioms hold. Fortunately, our task is simplified as most of these properties are inherited from the vector space V . For example, to show that the commutative property holds in W , let u and v be vectors in W. Since u and v are also in V , then u⊕v=v⊕u Similarly, any three vectors in W satisfy the associative property, as this property is also inherited from V . To show that W contains the zero vector, let w be any vector in W. Since W is closed under scalar multiplication, 0  w ∈ W. Now, by Theorem 2 of Sec. 3.1, we have 0  w = 0. Thus, 0 ∈ W. Similarly, for any w ∈ W , (−1)  w = −w is also in W. All the other vector space properties, axioms 7 through 10, are inherited from V . This shows that W is a subspace of V . Conversely, if W is a subspace of V , then it is necessarily closed under addition and scalar multiplication. This proves Theorem 3.

THEOREM 3

Let W be a nonempty subset of the vector space V . Then W is a subspace of V if and only if W is closed under addition and scalar multiplication.

By Theorem 3, the first of the examples above with    a a∈⺢ W= 0

Confirming Pages

142

Chapter 3 Vector Spaces

is a subspace of ⺢2 while the second subset   a W = 1

 a∈⺢

is not. For any vector space V the subset W = {0}, consisting of only the zero vector, is a subspace of V , called the trivial subspace. We also have that any vector space V , being a subset of itself, is a subspace.

EXAMPLE 1



Let W =

a a+1

  a∈⺢

be a subset of the vector space V = ⺢2 with the standard definitions of addition and scalar multiplication. Determine whether W is a subspace of V . Solution

In light of Theorem 3, we check to see if W is closed under addition and scalar multiplication. Let     v u and v= u= v+1 u+1 be vectors in W . Adding the vectors gives     v u ⊕ u⊕v= v+1 u+1   u+v = u+v+2 This last vector is not in the required form since u + v + 2 = u + v + 1 and hence we see that u ⊕ v is not in W. Thus, W is not a subspace of V .

It is sometimes easy to show that a subset W of a vector space is not a subspace. In particular, if 0 ∈ / W or the additive inverse of a vector is not in W , then W is not a subspace. In Example 1, W is not a subspace since it does not contain the zero vector.

EXAMPLE 2

The trace of a square matrix is the sum of the entries on the diagonal. Let M2×2 be the vector space of 2 × 2 matrices with the standard operations for addition and scalar multiplication, and let W be the subset of all 2 × 2 matrices with trace 0, that is,    a b a+d =0 W = c d Show that W is a subspace of M2×2 .

Confirming Pages

3.2 Subspaces

Solution



Let w1 =

a1 c1

b1 d1



 and

w2 =

a2 c2

b2 d2

143



be matrices in W , so that a1 + d1 = 0 and a2 + d2 = 0. The sum of the two matrices is       a2 b2 a1 + a2 b1 + b2 a1 b1 ⊕ = w1 ⊕ w2 = c1 d1 c2 d2 c1 + c2 d1 + d2 Since the trace of w1 ⊕ w2 is (a1 + a2 ) + (d1 + d2 ) = (a1 + d1 ) + (a2 + d2 ) = 0 then W is closed under addition. Also, for any scalar c,    ca1 a1 b1 = c  w1 = c  c1 d1 cc1

cb1 cd1



The trace of this matrix is ca1 + cd1 = c(a1 + d1 ) = 0. Thus, W is also closed under scalar multiplication. Therefore, W is a subspace of M2×2 .

EXAMPLE 3

Solution

Let W be the subset of V = Mn×n consisting of all symmetric matrices. Let the operations of addition and scalar multiplication on V be the standard operations. Show that W is a subspace of V . Recall from Sec. 1.3 that a matrix A is symmetric provided that At = A. Let A and B be matrices in W and c be a real number. By Theorem 6 of Sec. 1.3, (A ⊕ B)t = At ⊕ B t = A ⊕ B

and

(c  A)t = c  At = c  A

Thus, W is closed under addition and scalar multiplication, and consequently, by Theorem 3, W is a subspace.

EXAMPLE 4 Solution

Let V = Mn×n with the standard operations and W be the subset of V consisting of all idempotent matrices. Determine whether W is a subspace. Recall that a matrix A is idempotent provided that A2 = A (See Exercise 42 of Sec. 1.3.) Let A be an element of W , so that A2 = A. Then (c  A)2 = (cA)2 = c2 A2 = c2 A = c2  A so that (c  A)2 = c  A if and only if c2 = c Since this is not true for all values of c, then W is not closed under scalar multiplication and is not a subspace.

Confirming Pages

144

Chapter 3 Vector Spaces

The two closure criteria for a subspace can be combined into one as stated in Theorem 4. THEOREM 4

A nonempty subset W of a vector space V is a subspace of V if and only if for each pair of vectors u and v in W and each scalar c, the vector u ⊕ (c  v) is in W . Proof Let W be a nonempty subset of V , and suppose that u ⊕ (c  v) belongs to W for all vectors u and v in W and all scalars c. By Theorem 3 it suffices to show that W is closed under addition and scalar multiplication. Suppose that u and v are in W ; then u ⊕ (1  v) = u ⊕ v is in W, so that W is closed under addition. Next, since W is nonempty, let u be any vector in W. Then 0 = u ⊕ [(−1)  u], so that the zero vector is in W. Now, if c is any scalar, then c  u = 0 ⊕ (c  u) and hence is in W. Therefore, W is also closed under scalar multiplication. Conversely, if W is a subspace with u and v in W , and c a scalar, then since W is closed under addition and scalar multiplication, we know that u ⊕ (c  v) is in W .

EXAMPLE 5

Let W be the subset of ⺢3 defined by   W =  

  3t  0  t ∈ ⺢  −2t

Use Theorem 4 to show that W is a subspace. Solution

Let u and v be vectors in W and c be a real number. Then there are real numbers p and q such that      3q 3p 0  0  ⊕ c   u ⊕ (c  v) =  −2q −2p   3(p + cq)  0 = −2(p + cq) As this vector is in W , by Theorem 4, W is a subspace. Alternatively, the set W can be written as     3   W = t  0  t ∈ ⺢   −2 which is a line through the origin in ⺢3 .

Confirming Pages

3.2 Subspaces

145

We now consider what happens when subspaces are combined. In particular, let W1 and W2 be subspaces of a vector space V . Then the intersection W1 ∩ W2 is also a subspace of V . To show this, let u and v be elements of W1 ∩ W2 and let c be a scalar. Since W1 and W2 are both subspaces, then by Theorem 4, u ⊕ (c  v) is in W1 and is in W2 , and hence is in the intersection. Applying Theorem 4 again, we have that W1 ∩ W2 is a subspace. The extension to an arbitrary number of subspaces is stated in Theorem 5.

THEOREM 5

The intersection of any collection of subspaces of a vector space is a subspace of the vector space.

Example 6 shows that the union of two subspaces need not be a subspace.

EXAMPLE 6

Let W1 and W2 be the subspaces of ⺢2 with the standard operations given by       x 0 x∈⺢ and W2 = y∈⺢ W1 = 0 y Show that W1 ∪ W2 is not a subspace.

Solution

The subspaces W1 and W2 consist of all vectors that lie on the x axis and the y axis, respectively. Their union is the collection of all vectors that lie on either axis and is given by    x W1 ∪ W2 = x = 0 or y = 0 y This set is not closed under addition since       1 0 1 = ⊕ 1 1 0 which is not in W1 ∪ W2 , as shown in Fig. 2. y



0 1



W2





1 0

Figure 2



1 1

 ∈ / W1 ∪ W2

W1 x

Confirming Pages

146

Chapter 3 Vector Spaces

Span of a Set of Vectors Subspaces of a vector space can be constructed by collecting all linear combinations of a set of vectors from the space. These subspaces are used to analyze certain properties of the vector space. A linear combination is defined in abstract vector spaces exactly as it is defined in ⺢n in Chap. 2.

DEFINITION 2

Linear Combination Let S = {v1 , v2 , . . . , vk } be a set of vectors in a vector space V , and let c1 , c2 , . . . , ck be scalars. A linear combination of the vectors of S is an expression of the form (c1  v1 ) ⊕ (c2  v2 ) ⊕ · · · ⊕ (ck  vk )

When the operations of vector addition and scalar multiplication are clear, we will drop the use of the symbols ⊕ and . For example, the linear combination given in Definition 2 will be written as k  c1 v1 + c2 v2 + · · · + ck vk = ci vi i=1

Unless otherwise stated, the operations on the vector spaces ⺢n , Mm×n , Pn , and their subspaces are the standard operations. Care is still needed when interpreting expressions defining linear combinations to distinguish between vector space operations and addition and multiplication of real numbers.

DEFINITION 3

Span of a Set of Vectors Let V be a vector space and let S = {v1 , . . . , vn } be a (finite) set of vectors in V . The span of S, denoted by span(S), is the set span(S) = {c1 v1 + c2 v2 + · · · + cn vn | c1 , c2 , . . . , cn ∈ ⺢}

PROPOSITION 1

If S = {v1 , v2 , . . . , vn } is a set of vectors in a vector space V , then span(S) is a subspace. Proof Let u and w be vectors in span(S) and c a scalar. Then there are scalars c1 , . . . , cn and d1 , . . . , dn such that u + cw = (c1 v1 + · · · + cn vn ) + c(d1 v1 + · · · + dn vn ) = (c1 + cd1 )v1 + · · · + (cn + cdn )vn Therefore, u + cw is in span(S), and hence the span is a subspace.

Confirming Pages

3.2 Subspaces

EXAMPLE 7

147

Let S be the subset of the vector space ⺢3 defined by     1  1 2  S =  −1 ,  3 ,  1    4 −2 0 Show that

 −4 v= 4  −6 

is in span(S). Solution

To determine if v is in the span of S, we consider      2 1 c1  −1  + c2  3  + c3  0 −2

the equation    −4 1 1 = 4  −6 4

Solving this linear system, we obtain c1 = −2

c2 = 1

and

c3 = −1

This shows that v is a linear combination of the vectors in S and is thus in span(S).

The span of a single nonzero vector in ⺢n is a line through the origin, and the span of two linearly independent vectors is a plane through the origin as shown in Fig. 3. y

z

span{v}

y

3v v1

v x

−2v −3.5v

span{v1 , v2 }

v2 x

Figure 3

Since every line through the origin in ⺢2 and ⺢3 , and every plane through the origin in ⺢3 , can be written as the span of vectors, these sets are subspaces.

Confirming Pages

148

Chapter 3 Vector Spaces

EXAMPLE 8



Let S=

  0 0 1 1 0 , , 0 1 0 0 0

0 1



Show that the span of S is the subspace of M2×2 of all symmetric matrices. Solution

Recall that a 2 × 2 matrix is symmetric provided that it has the form   a b b c Since any matrix in span(S) has the form        a 0 0 0 1 1 0 = +c +b a b 0 1 1 0 0 0

b c



span(S) is the collection of all 2 × 2 symmetric matrices.

EXAMPLE 9

Solution

Show that

Let

    1  1  1 span  1 ,  0 ,  1  = ⺢3   0 2 1  a v= b  c 

be an arbitrary element of ⺢3 . The vector v is in span(S) provided that there are scalars c1 , c2 , and c3 such that         a 1 1 1 c1  1  + c 2  0  + c 3  1  =  b  c 1 2 0 This linear system in matrix form is given  1 1  1 0 1 2 After row-reducing, we obtain  1 0 0  0 1 0 0 0 1

by 1 1 0

 a b  c

 −2a + 2b + c  a− b 2a − b − c

Confirming Pages

3.2 Subspaces

149

From this final augmented matrix the original system is consistent, having solution c1 = −2a + 2b + c, c2 = a − b, and c3 = 2a − b − c, for all choices of a, b, and c. Thus, every vector in ⺢3 can be written as a linear combination of the three given vectors. Hence, the span of the three vectors is all of ⺢3 .

EXAMPLE 10

Solution

z

y x

7x − y + 9z = 0

Figure 4

Show that

    −6  4  −1 span  2 ,  1 ,  3  = ⺢3   5 −3 1

We approach this problem in the same manner as in Example 9. In this case, however, the resulting linear system is not always consistent. We can see this by reducing the augmented matrix     a −1 4 −6 −1 4 −6 a  2  0 9 −9  b + 2a 1 3 b  to 7 1 1 −3 5 c 0 0 0 c + 9a − 9b This last augmented matrix shows that the original system is consistent only if 7a − b + 9c = 0. This is the equation of a plane in 3-space, and hence the span is not all of ⺢3 . See Fig. 4. Notice that the solution to the equation 7a − b + 9c = 0 can be written in parametric form by letting b = s, c = t, and a = 17 s − 97 t, so that      1     9  −6   4 − 7   −1 7 span  2 ,  1 ,  3  = s  1  + t  0  s, t ∈ ⺢     5 −3 1 0 1 In this way, we see that the span is the subspace of all linear combinations of two linearly independent vectors, highlighting the geometric interpretation of the solution as a plane.

With Examples 9 and 10 we have completed the groundwork for the notion of a basis, which is central to linear algebra and is the subject of Sec. 3.3. Specifically, in Example 9, we saw that the set of vectors       1  1  1 S = {v1 , v2 , v3 } =  1 ,  0 ,  1    0 2 1

Confirming Pages

150

Chapter 3 Vector Spaces

spans ⺢3 . These vectors are also linearly matrix  1 A= 1 1

independent. To see this, observe that the  1 1 0 1  2 0

whose column vectors are the vectors of S, is row equivalent to the 3 × 3 identity matrix, as seen in the solution to Example 9. [Another way of showing that S is linearly independent is to observe that det(A) = 1 = 0.] Consequently, by Theorem 7 of Sec. 2.3, we have that every vector in ⺢3 can be written in only one way as a linear combination of the vectors of S. On the other hand, the span of the set of vectors       −6  4  −1   S = v 1 , v 2 , v 3 =  2 ,  1 ,  3    5 −3 1 of Example 10 is a plane passing through the origin. Hence, not every vector in ⺢3 can be written as a linear combination of the vectors in S . As we expect, these vectors are linearly dependent since   −1 4 −6 1 3  = 0 det  2 1 −3 5 In particular, v 3 = 2v 1 − v 2 . The vectors v 1 and v 2 are linearly independent vectors which span the plane shown in Fig. 4, but not ⺢3 . To pursue these notions a bit further, there are many sets of vectors which span ⺢3 . For example, the set         1  0 0  1 B = {e1 , e2 , e3 , v} =  0 ,  1 ,  0 ,  2    3 1 0 0 spans ⺢3 , but by Theorem 3 of Sec. 2.3 must necessarily be linearly dependent. The ideal case, in terms of minimizing the number of vectors, is illustrated in Example 9 where the three linearly independent vectors of S span ⺢3 . In Sec. 3.3 we will see that S is a basis for ⺢3 , and that every basis for ⺢3 consists of exactly three linearly independent vectors.

EXAMPLE 11

Show that the set of matrices S=



  1 1 −1 0 , 1 0 2 1

does not span M2×2 . Describe span(S).

Confirming Pages

3.2 Subspaces

Solution

The equation



−1 0 c1 2 1 is equivalent to the linear system



 + c2

1 1 1 0

 −c1 + c2     c2  2c + c 1 2    c1



 =

a c

b d

151



=a =b =c =d

From these equations we see that c1 = d

and

c2 = b

a =b−d

and

c = b + 2d

which gives 

Therefore, span(S) =

EXAMPLE 12

b− d b + 2d

b d

  b, d ∈ ⺢

Show that the set of polynomials S = {x 2 + 2x + 1, x 2 + 2, x} spans the vector space P2 .

Solution

An arbitrary vector in P2 can be written in the form ax 2 + bx + c. To determine whether the span(S) = P2 , we consider the equation c1 (x 2 + 2x + 1) + c2 (x 2 + 2) + c3 x = ax 2 + bx + c which simplifies to (c1 + c2 )x 2 + (2c1 + c3 )x + (c1 + 2c2 ) = ax 2 + bx + c Since two polynomials are equal if and only if the coefficients of like terms are equal, equating coefficients in the previous equation gives, in matrix form, the linear system   1 1 0 a  2 0 1 b  1 2 0 c This matrix reduces to   1 0 0 2a − c  0 1 0  −a + c 0 0 1 −4a + b + 2c

Confirming Pages

152

Chapter 3 Vector Spaces

Hence, the linear system has the unique solution c1 = 2a − c, c2 = −a + c, and c3 = −4a + b + 2c, for all a, b, and c. Therefore, span(S) = P2 .

The Null Space and Column Space of a Matrix Two special subspaces associated with every matrix A are the null space and column space of the matrix.

DEFINITION 4

Null Space and Column Space Let A be an m × n matrix. 1. The null space of A, denoted by N (A), is the set of all vectors in ⺢n such that Ax = 0. 2. The column space of A, denoted by col(A), is the set of all linear combinations of the column vectors of A.

Observe that N (A) is a subset of ⺢n and col(A) is a subset of ⺢m . Moreover, by Proposition 1, col(A) is a subspace of ⺢m . Using this terminology, we give a restatement of Theorem 2 of Sec. 2.2.

THEOREM 6

EXAMPLE 13

Let A be an m × n matrix. The linear system Ax = b is consistent if and only if b is in the column space of A.

Let

 1 −1 −2 2 3  A =  −1 2 −2 −2

 3 b= 1  −2 



and

a. Determine whether b is in col(A). b. Find N (A). Solution

a. By Theorem 6, the vector b is in col(A) if and only if there is a that Ax = b. The corresponding augmented matrix is given by    1 −1 −2 1 0 3  −1  0 1 2 3 1  which reduces to 2 −2 −2 −2 0 0

vector x such 0 0 1

 3 8  −4

Confirming Pages

3.2 Subspaces

153

Hence, the linear system Ax = b is consistent, and vector b is in col(A). Specifically,         −2 −1 1 3  1  = 3  −1  + 8  2  − 4  3  −2 −2 2 −2 b. To find the null space of A, we solve the homogeneous equation Ax = 0. The corresponding augmented matrix for this linear system is the same as in part (a), except for the right column that consists of three zeros. Consequently, the only solution is the trivial solution and hence N (A) = {0}.

In Theorem 7 we show that the null space of a matrix also is a subspace.

THEOREM 7

Let A be an m × n matrix. Then the null space of A is a subspace of ⺢n . Proof The null space of A is nonempty since 0 is in N (A). That is, A0 = 0. Now let u and v be vectors in N (A) and c a scalar. Then A(u + cv) = Au + A(cv) = Au + cA(v) = 0 + c0 = 0 Hence, u + cv is in N (A), and therefore by Theorem 4, N (A) is a subspace.

Fact Summary Let V be a vector space and W a nonempty subset of V . 1. To verify that W is a subspace of V , show that u ⊕ c  v is in W for any u and v in W and any scalar c. 2. The span of a set of vectors from V is a subspace. 3. The span of a single nonzero vector in ⺢2 or ⺢3 is a line that passes through the origin. The span of two linearly independent vectors in ⺢3 is a plane that passes through the origin. These sets are subspaces. 4. The intersection of subspaces is a subspace. The union of two subspaces may not be a subspace. 5. If A is an m × n matrix, the null space of A is a subspace of ⺢n and the column space of A is a subspace of ⺢m . 6. The linear system Ax = b is consistent if and only if b is in the column space of A.

Confirming Pages

154

Chapter 3 Vector Spaces

Exercise Set 3.2 In Exercises 1–6, determine whether the subset S of ⺢2 is a subspace. If S is not a subspace, find vectors u and v in S such that u + v is not in S; or a vector u and a scalar c such that cu is not in S.    0 y ∈ ⺢ 1. S = y    x xy ≥ 0 2. S = y    x xy ≤ 0 3. S = y    x 2 2 x 4. S = + y ≤ 1 y   x x∈⺢ 5. S = 2x − 1    x x ∈ ⺢ 6. S = 3x In Exercises 7–10, determine whether the subset S of ⺢3 is a subspace.      x1 7. S =  x2  x1 + x3 = −2   x3     x1  8. S =  x2  x1 x2 x3 = 0   x3      s − 2t  s, t ∈ ⺢ s 9. S =    t +s     x1  10. S =  2  x1 , x3 > 0   x3 In Exercises 11–18, determine whether the subset S of M2×2 is a subspace. 11. Let S be the set of all symmetric matrices. 12. Let S be the set of all idempotent matrices. 13. Let S be the set of all invertible matrices.

14. Let S be the set of all skew-symmetric matrices. 15. Let S be the set of all upper triangular matrices. 16. Let S be the set of all diagonal matrices. 17. Let S be the set of all matrices with a22 = 0. 18. Let S be the set of all matrices with a11 + a22 = 0. In Exercises 19–24, determine whether the subset S of P5 is a subspace. 19. Let S be the set of all polynomials with degree equal to 3. 20. Let S be the set of all polynomials with even degree. 21. Let S be the set of all polynomials such that p(0) = 0. 22. Let S be the set of all polynomials of the form p(x) = ax 2 . 23. Let S be the set of all polynomials of the form p(x) = ax 2 + 1. 24. Let S be the set of all polynomials of degree less than or equal to 4. In Exercises 25 and 26, determine if the vector v is in the span of      −1  −1  1 S =  1 ,  −1 ,  2    0 1 0   1 25. v =  −1  1   −2 26. v =  7  −3 In Exercises 27 and 28, determine if the matrix M is in the span of     1 −1 0 1 1 1 , , S= −4 −3 2 1 0 −1

Confirming Pages

3.2 Subspaces

 27. M =  28. M =

−2 1 6 5 1 1 2 −3

 

     2  0 −1  1 40. S =  2 ,  0 ,  1 ,  1    1 1 3 1 41. Let

    1 −1  1 S =  2 ,  3 ,  2 ,  −1 −1 2    −3  0  6 ,  4   5 1

In Exercises 29 and 30, determine if the polynomial p(x) is in the span of S = {1 + x, x 2 − 2, 3x} 29. p(x) = 2x 2 − 6x − 11 30. 3x 2 − x − 4 In Exercises 31–36, give an explicit description of the span of S.    1  2  31. S =  −1 ,  3    −1 −2     1  2  1 32. S =  1 ,  3 ,  2    −1 1 2    1 −1 1 2 , 33. 0 1 1 0     0 −1 1 1 1 0 , , 34. S = 1 1 −1 −1 1 0   35. S = x, (x + 1)2 , x 2 + 3x + 1   36. S = x 2 − 4, 2 − x, x 2 + x + 2 In Exercises 37–40, a subset S of ⺢3 is given. a. Find span(S). b. Is S linearly independent?    3  2  37. S =  1 ,  0    −2 −1      2  0  1 38. S =  1 ,  −1 ,  5    1 1 2     1  0  3 39. S =  3 ,  1 ,  −1    −1 0 2

155

a. Find span(S). b. Is S linearly independent? c. Let     1 −1  1 T =  2 ,  3 ,  2 ,  −1 −1 2   −3   4   5 Is span(T ) = ⺢3 ? Is T linearly independent? d. Let      1  −1  1 H =  2 ,  3 ,  2    −1 −1 2 Is span(H ) = ⺢3 ? Is H linearly independent? 42. Let

 S=

2 0

   −3 1 1 1 −3 , , 1 0 1 0 0

a. Find span(S). b. Is S linearly independent? c. Let     −3 1 1 1 2 −3 , , , T = 1 0 1 0 0 0   0 0 0 1 Is span(T ) = M2×2 ? Is T linearly independent?

Confirming Pages

156

Chapter 3 Vector Spaces

43. Let S = 1, x − 3, x 2 + 2x, 2x 2 + 3x + 5 a. Find span(S). b. Is S linearly independent? c. Show that 2x 2 + 3x + 5 is a linear combination of the other three polynomials in S.   d. Let T = 1, x − 3, x 2 + 2x, x 3 . Is T linearly independent? Is span(T ) = P3 ? 44. Let

c. Are the three matrices that generate S linearly independent?





 2s − t    s S=   t    −s

       s, t ∈ ⺢    

a. Show that S is a subspace of

⺢4 .

b. Find two vectors that span S.

47. Let A be a 2 × 3 matrix and let    1 S = x ∈ ⺢3 Ax = 2 Is S a subspace? Explain. 48. Let A be an m × n matrix and let   S = x ∈ ⺢n Ax = 0 Is S a subspace? Explain. 49. Let A be a fixed n × n matrix and let S = { B ∈ Mn×n | AB = BA} Is S a subspace? Explain. 50. Suppose S and T are subspaces of a vector space V . Define S + T = {u + v | u ∈ S, v ∈ T }

c. Are the two vectors found in part (b) linearly independent? d. Is S = ⺢4 ? 45. Let

   −s   S =  s − 5t  s, t ∈ ⺢   3t + 2s

Show that S + T is a subspace of V . 51. Let S = span({u1 , u2 , . . . um }) and T = span({v1 , v2 , . . . vn }) be subspaces of a vector space V . Show that S + T = span({u1 , . . . um , v1 , . . . vn }) (See Exercise 50.)

a. Show that S is a subspace of ⺢3 . b. Find a set of vectors that span S.

52. Let

c. Are the two vectors found in part (b) linearly independent? d. Is S = ⺢3 ?

 S=

and

46. Let     0 1 1 0 1 0 , , S = span 1 1 1 0 0 1 a. Describe the subspace S. b. Is S = M2×2 ? ß

3.3

 T =

x y a −a

−x z

  x, y, z ∈ ⺢

b c

  a, b, c ∈ ⺢

a. Show that S and T are subspaces. b. Describe all matrices in S + T . (See Exercises 50 and 51.)

Basis and Dimension In Sec. 2.3 we introduced the notion of linear independence and its connection to the minimal sets that can be used to generate or span ⺢n . In this section we explore this connection further and see how to determine whether a spanning set is minimal.

Confirming Pages

3.3 Basis and Dimension

157

This leads to the concept of a basis for an abstract vector space. As a first step, we generalize the concept of linear independence to abstract vector spaces introduced in Sec. 3.1.

DEFINITION 1

Linear Independence and Linear Dependence The set of vectors S = {v1 , v2 , . . . , vm } in a vector space V is called linearly independent provided that the only solution to the equation c1 v1 + c2 v2 + · · · + cm vm = 0 is the trivial solution c1 = c2 = · · · = cm = 0. If the equation has a nontrivial solution, then the set S is called linearly dependent.

EXAMPLE 1

Let

 1 v1 =  0  −1 

 0 v2 =  2  2 

 −3 v3 =  4  7 

and let W = span{v1 , v2 , v3 }. a. Show that v3 is a linear combination of v1 and v2 . b. Show that span{v1 , v2 } = W. c. Show that v1 and v2 are linearly independent. Solution

a. To solve the vector equation       −3 1 0 c1  0  + c 2  2  =  4  7 −1 2 we row-reduce the corresponding augmented matrix for the linear system to obtain     1 0 −3 1 0 −3  0 2 4  −→  0 1 2  −1 2 0 0 7 0 The solution to the vector equation above is c1 = −3 and c2 = 2, therefore v3 = −3v1 + 2v2 Notice that the vector v3 lies in the plane spanned by v1 and v2 , as shown in Fig. 1.

Figure 1

b. From part (a) an element of W = {c1 v1 + c2 v2 + c3 v3 | c1 , c2 , c3 ∈ ⺢} can be written in the form c1 v1 + c2 v2 + c3 v3 = c1 v1 + c2 v2 + c3 (−3v1 + 2v2 ) = (c1 − 3c3 )v1 + (c2 + 2c3 )v2

Confirming Pages

158

Chapter 3 Vector Spaces

and therefore, every vector in W is a linear combination of v1 and v2 . As a result, the vector v3 is not needed to generate W, so that span{v1 , v2 } = W. c. Since neither vector is a scalar multiple of the other, the vectors v1 and v2 are linearly independent.

In Example 1, we were able to reduce the number of linearly dependent vectors that span W to a linearly independent set of vectors which also spans W . We accomplished this by eliminating the vector v3 from the set, which, as we saw in the solution, is a linear combination of the vectors v1 and v2 and hence does not affect the span. Theorem 8 gives a general description of the process.

THEOREM 8

Let v1 , . . . , vn be vectors in a vector space V , and let W = span{v1 , . . . , vn }. If vn is a linear combination of v1 , . . . , vn−1 , then W = span{v1 , . . . , vn−1 } Proof If v is in span{v1 , . . . , vn−1 }, then there are scalars c1 , c2 , . . . , cn−1 such that v = c1 v1 + · · · + cn−1 vn−1 . Then v = c1 v1 + · · · + cn−1 vn−1 + 0vn , so that v is also in span{v1 , . . . , vn }. Therefore, span{v1 , . . . , vn−1 } ⊆ span{v1 , . . . , vn } Conversely, if v is in span{v1 , . . . , vn }, then there are scalars c1 , . . . , cn such that v = c1 v1 + · · · + cn vn . Also, since vn is a linear combination of v1 , . . . , vn−1 , there are scalars d1 , . . . , dn−1 such that vn = d1 v1 + · · · + dn−1 vn−1 . Then v = c1 v1 + · · · + cn−1 vn−1 + cn vn = c1 v1 + · · · + cn−1 vn−1 + cn (d1 v1 + · · · + dn−1 vn−1 ) = (c1 + cn d1 )v1 + · · · + (cn−1 + cn dn−1 )vn−1 so that v ∈ span{v1 , . . . , vn−1 } and span{v1 , . . . , vn } ⊆ span{v1 , . . . , vn−1 }. Therefore, W = span{v1 , . . . , vn } = span{v1 , . . . , vn−1 }

EXAMPLE 2

Compare the column spaces of the matrices 

1  2 A=  1 3

 0 −1 1 0 1 7   1 2 7  4 1 5



1  2 and B =   1 3

 0 −1 1 2 0 1 7 −1   1 2 7 1  4 1 5 −2

Confirming Pages

3.3 Basis and Dimension

Solution

159

By using the methods presented in Chap. 2 it can be shown that the the column vectors of the matrix A are linearly independent. Since the column vectors of B consist of a set of five vectors in ⺢4 , by Theorem 3 of Sec. 2.3, the vectors are linearly dependent. In addition, the first four column vectors of B are the same as the linearly independent column vectors of A, hence by Theorem 5 of Sec. 2.3 the fifth column vector of B must be a linear combination of the other four vectors. Finally by Theorem 8, we know that col(A) = col(B).

As a consequence of Theorem 8, a set of vectors {v1 , . . . , vn } such that V = span{v1 , . . . , vn } is minimal, in the sense of the number of spanning vectors, when they are linearly independent. We also saw in Chap. 2 that when a vector in ⺢n can be written as a linear combination of vectors from a linearly independent set, then the representation is unique. The same result holds for abstract vector spaces. THEOREM 9

If B = {v1 , v2 , . . . , vm } is a linearly independent set of vectors in a vector space V , then every vector in span(B) can be written uniquely as a linearly combination of vectors from B. Motivated by these ideas, we now define what we mean by a basis of a vector space.

DEFINITION 2

Basis for a Vector Space A subset B of a vector space V is a basis for V provided that 1. B is a linearly independent set of vectors in V 2. span(B) = V

As an example, the set of coordinate vectors S = {e1 , . . . , en } ⺢n

spans and is linearly independent, so that S is a basis for ⺢n . This particular basis is called the standard basis for ⺢n . In Example 3 we give a basis for ⺢3 , which is not the standard basis.

EXAMPLE 3

Show that the set

is a basis for ⺢3 .

     0  1  1 B =  1 ,  1 ,  1    −1 1 0

Confirming Pages

160

Chapter 3 Vector Spaces

Solution

First, to show that S spans ⺢3 , we must show that the equation         a 1 1 0 c1  1  + c2  1  + c3  1  =  b  c 0 1 −1 has a solution for every choice of a, b, and c in ⺢. To solve this linear system, we reduce the corresponding augmented matrix     1 1 0 a 1 0 0 2a − b − c  1 1 1 b  to AB =  0 1 0 −a + b + c  0 1 −1 c 0 0 1 −a + b Therefore, c1 =  b − c, c2 = −a + b + c, and c3 = −a + b. For example, sup 2a − 1 pose that v =  2 ; then 3 c1 = 2(1) − 2 − 3 = −3 c2 = −1 + 2 + 3 = 4 c3 = −1 + 2 = 1        1 0 1 1 −3  1  + 4  1  +  1  =  2  3 −1 1 0 Since the linear system is consistent for all choices of a, b, and c, we know that span(B) = ⺢3 . To show that B is linearly independent, we compute the determinant of the matrix whose column vectors are the vectors of B, that is, 1 1 0 1 1 1 = 1 0 1 −1 so that



Since this determinant is nonzero, by Theorem 9 of Sec. 2.3 the set B is linearly independent. Therefore, B is a basis for ⺢3 . Alternatively, to show that B is linearly independent, notice from the reduced matrix above that   1 1 0 1  A= 1 1 0 1 −1 is row equivalent to I . Again by Theorem 9 of Sec. 2.3, B is linearly independent.

As we have already illustrated in the examples above, bases for a vector space are not unique. For example, consider the standard basis B = {e1 , e2 , e3 } for ⺢3 . Another basis for ⺢3 is given by B = {2e1 , e2 , e3 }, where we have simply multiplied the first vector by 2.

Confirming Pages

3.3 Basis and Dimension

161

Theorem 10 generalizes this idea, showing that an infinite family of bases can be derived from a given basis for a vector space by scalar multiplication.

THEOREM 10

Let B = {v1 , . . . , vn } be a basis for a vector space V and c a nonzero scalar. Then Bc = {cv1 , v2 , . . . , vn } is a basis. Proof If v is an element of the vector space V , then since B is a basis there are scalars c1 , . . . , cn such that v = c1 v1 + c2 v2 + · · · + cn vn . But since c = 0, we can also write c1 v = (cv1 ) + c2 v2 + · · · + cn vn c so that v is a linear combination of the vectors in Bc . Thus, span(Bc ) = V . To show that Bc is linearly independent, consider the equation c1 (cv1 ) + c2 v2 + · · · + cn vn = 0 By vector space axiom 9 we can write this as (c1 c)(v1 ) + c2 v2 + · · · + cn vn = 0 Now, since B is linearly independent, the only solution to the previous equation is the trivial solution c1 c = 0

c2 = 0

...

cn = 0

Since c = 0, then c1 = 0. Therefore, Bc is linearly independent and hence is a basis.

EXAMPLE 4

Let W be the subspace of M2×2 of matrices with trace equal to 0, and let     0 0 0 1 1 0 , , S= 1 0 0 0 0 −1 Show that S is a basis for W.

Solution

In Example 2 of Sec. 3.2 we showed that W is a subspace of M2×2 . To show that span(S) = W, first recall that a matrix   a b A= c d has trace 0 if and only if a + d = 0, so that A has the form   a b A= c −a Since for every such matrix        0 0 1 1 0 a b +c +b =a 1 0 0 0 −1 c −a

0 0



Confirming Pages

162

Chapter 3 Vector Spaces

then span(S) = W . We also know that S is linearly independent system        0 1 0 0 1 0 0 + c2 + c3 = c1 0 0 −1 0 0 1 0 is equivalent to     c1 0 0 c2 = 0 0 c3 −c1 which has only the trivial solution c1 = c2 = c3 = 0. Thus, S is a

since the linear  0 0

basis for W.

Similar to the situation for ⺢n , there is a natural set of matrices in Mm×n that comprise a standard basis. Let eij be the matrix with a one in the ij position, and 0s elsewhere. The set S = {eij | 1 ≤ i ≤ m, 1 ≤ j ≤ n} is the standard basis for Mm×n . For example, the standard basis for M2×2 is      0 0 0 0 0 1 1 0 , , , S= 0 1 1 0 0 0 0 0

EXAMPLE 5

Determine whether

 B=

  0 −1 2 1 3 , , 0 1 0 2 1

1 −4



is a basis for M2×2 . Solution

 a b A= c d be an arbitrary matrix in M2×2 . To see if A is in the span of S, we consider the equation         a b 1 3 −1 2 0 1 + c2 + c3 = c1 c d 2 1 1 0 0 −4 Let



The augmented matrix corresponding to this equation is given by   1 −1 0 a  3 2 1 b     2 1 0 c  1 0 −4 d After row-reducing, we obtain  1 −1 0  0 5 1   0 0 −3 0 0 0

 a  −3a + b  −a − 3b + 5c  a + 4b − 7c + d

Observe that the above linear system is consistent only if a + 4b − 7c + d = 0. Hence, B does not span M2×2 , and therefore is not a basis.

Revised Confirming Pages

3.3 Basis and Dimension

163

Notice that in Example 5 the set B is linearly independent, but the three matrices do not span the set of all 2 × 2 matrices. We will see that the minimal number of matrices required to span M2×2 is four. Another vector space we have already considered is Pn , the vector space of polynomials of degree less than or equal to n. The standard basis for Pn is the set B = {1, x, x 2 , . . . , x n } Indeed, if p(x) = a0 + a1 x + a2 x 2 + · · · + an x n is any polynomial in Pn , then it is a linear combination of the vectors in B, so span(B) = Pn . To show that B is linearly independent, suppose that c 0 + c1 x + c2 x 2 + · · · + cn x n = 0 for all real numbers x. We can write this equation as c0 + c1 x + c2 x 2 + · · · + cn x n = 0 + 0x + 0x 2 + · · · + 0x n Since two polynomials are identical if and only if the coefficients of like terms are equal, then c1 = c2 = c3 = · · · = cn = 0. Another basis for P2 is given in Example 6.

EXAMPLE 6 Solution

Show that B = {x + 1, x − 1, x 2 } is a basis for P2 . Let ax 2 + bx + c be an arbitrary polynomial in P2 . To verify that B spans P2 , we must show that scalars c1 , c2 , and c3 can be found such that c1 (x + 1) + c2 (x − 1) + c3 x 2 = ax 2 + bx + c for every choice of a, b, and c. Collecting like terms on the left-hand side gives c3 x 2 + (c1 + c2 )x + (c1 − c2 ) = ax 2 + bx + c Equating coefficients on both sides, we obtain c3 = a, c1 + c2 = b and c1 − c2 = c. This linear system has the unique solution c1 = 12 (b + c)

c2 = 12 (b − c)

c3 = a

Therefore, span(B) = P2 . To show linear independence, we consider the equation c1 (x + 1) + c2 (x − 1) + c3 x 2 = 0 + 0x + 0x 2 Observe that the solution above holds for all choices of a, b, and c, so that c1 = c2 = c3 = 0. Therefore, the set B is also linearly independent and hence is a basis. Another way of showing that the set B of Example 6 is a basis for P2 is to show that the polynomials of the standard basis can be written as linear combinations of the polynomials in B. Specifically, we have 1 = 12 (x + 1) − 12 (x − 1) and x 2 is already in B.

x = 12 (x + 1) + 12 (x − 1)

Confirming Pages

164

Chapter 3 Vector Spaces

Dimension We have already seen in Theorem 3 of Sec. 2.3 that any set of m vectors from ⺢n , with m > n, must necessarily be linearly dependent. Hence, any basis of ⺢n contains at most n vectors. It can also be shown that any linearly independent set of m vectors, with m < n, does not span ⺢n . For example, as we have already seen, two linearly independent vectors in ⺢3 span a plane. Hence, any basis of ⺢n must contain exactly n vectors. The number n, an invariant of ⺢n , is called the dimension of ⺢n . Theorem 11 shows that this holds for abstract vector spaces.

THEOREM 11

If a vector space V has a basis with n vectors, then every basis has n vectors. Proof Let B = {v1 , v2 , . . . , vn } be a basis for V , and let T = {u1 , u2 , . . . , um } be a subset of V with m > n. We claim that T is linearly dependent. To establish this result, observe that since B is a basis, then every vector in T can be written as a linear combination of the vectors from B. That is, u1 = λ11 v1 + λ12 v2 + · · · + λ1n vn u2 = λ21 v1 + λ22 v2 + · · · + λ2n vn .. . um = λm1 v1 + λm2 v2 + · · · + λmn vn Now consider the equation c1 u1 + c2 u2 + · · · + cm um = 0 Using the equations above, we can write this last equation in terms of the basis vectors. After collecting like terms, we obtain (c1 λ11 + c2 λ21 + · · · + cm λm1 )v1 + (c1 λ12 + c2 λ22 + · · · + cm λm2 )v2 .. . + (c1 λ1n + c2 λ2n + · · · + cm λmn )vn = 0 Since B is a basis, it is linearly independent, hence c1 λ11 + c2 λ21 + · · · + cm λm1 = 0 c1 λ12 + c2 λ22 + · · · + cm λm2 = 0 .. . c1 λ1n + c2 λ2n + · · · + cm λmn = 0 This last linear system is not square with n equations in the m variables c1 , . . . , cm . Since m > n, by Theorem 3 of Sec. 2.3 the linear system has a nontrivial solution, and hence T is linearly dependent.

Confirming Pages

3.3 Basis and Dimension

165

Now, suppose that T = {u1 , u2 , . . . , um } is another basis for the vector space V . By the result we just established it must be the case that m ≤ n. But by the same reasoning, the number of vectors in the basis B also cannot exceed the number of vectors in T , so n ≤ m. Consequently n = m as desired.

We can now give a definition for the dimension of an abstract vector space.

DEFINITION 3

Dimension of a Vector Space The dimension of the vector space V , denoted by dim(V ), is the number of vectors in any basis of V .

For example, since the standard bases for ⺢n , M2×2 , Mm×n , and Pn are {e1 , e2 , . . . , en } {e11 , e12 , e21 , e22 } {eij | 1 ≤ i ≤ m, i ≤ j ≤ n} {1, x, x 2 , . . . , x n } respectively, we have dim(⺢n ) = n

dim(M2×2 ) = 4

dim(Mm×n ) = mn

dim(Pn ) = n + 1

We call a vector space V finite dimensional if there exists a basis for V with a finite number of vectors. If such a basis does not exist, then V is called infinite dimensional. The trivial vector space V = {0} is considered finite dimensional, with dim(V ) = 0, even though it does not have a basis. In this text our focus is on finite dimensional vector spaces, although infinite dimensional vector spaces arise naturally in many areas of science and mathematics. To determine whether a set of n vectors from a vector space of dimension n is or is not a basis, it is sufficient to verify either that the set spans the vector space or that the set is linearly independent.

THEOREM 12

Suppose that V is a vector space with dim(V ) = n. 1. If S = {v1 , v2 , . . . , vn } is linearly independent, then span(S) = V and S is a basis. 2. If S = {v1 , v2 , . . . , vn } and span(S) = V , then S is linearly independent and S is a basis. Proof (1) Suppose that S is linearly independent, and let v be any vector in V . If v is in S, then v is in span(S). Now suppose that v is not in S. As in the proof of

Confirming Pages

166

Chapter 3 Vector Spaces

Theorem 11, the set {v, v1 , v2 , . . . , vn } is linearly dependent. Thus, there are scalars c1 , . . . , cn , cn+1 , not all zero, such that c1 v1 + c2 v2 + · · · + cn vn + cn+1 v = 0 Observe that cn+1 = 0, since if it were, then S would be linearly dependent, violating the hypothesis that it is linearly independent. Solving for v gives c1 c2 cn v=− v1 − v2 − · · · − vn cn+1 cn+1 cn+1 As v was chosen arbitrarily, every vector in V is in span(S) and therefore V = span(S). (2) (Proof by contradiction) Assume that S is linearly dependent. Then by Theorem 5 of Sec. 2.3 one of the vectors in S can be written as a linear combination of the other vectors. We can eliminate this vector from S without changing the span. We continue this process until we arrive at a linearly independent spanning set with less than n elements. This contradicts the fact that the dimension of V is n.

EXAMPLE 7

Determine whether

    0  1  1 B =  0 ,  1 ,  0    1 0 1

is a basis for ⺢3 . Solution

Since dim(⺢3 ) = 3, the set B is a basis if  1 A= 0 1

it is linearly independent. Let  1 0 1 0  0 1

be the matrix whose column vectors are the vectors of B. The determinant of this matrix is 1, so that by Theorem 9 of Sec. 2.3 the set B is linearly independent and hence, by Theorem 12, is a basis. We can also show that B is a basis by showing that B spans ⺢3 .

Finding a Basis In Sec. 3.2, we saw that the span of a nonempty set of vectors S = {v1 , . . . , vm } is a subspace. We then ask whether S is a basis for this subspace (or a vector space). From Theorem 12, this is equivalent to determining whether S is linearly independent. When the vectors v1 , . . . , vm are in ⺢n , as in Example 7, form the matrix A with ith column vector equal to vi . By Theorem 2 of Sec. 1.2, if B is the row echelon matrix obtained from reducing A, then Ax = 0 if and only if Bx = 0. Now if the column vectors of A are linearly dependent, then there are scalars c1 , . . . , cm , not all 0, such

Confirming Pages

3.3 Basis and Dimension

167

that c1 v1 + · · · + cm vm = 0. To express this in matrix form, let   c1  c2   c=  ...  cm Then Ac = 0 = Bc. Hence, the column vectors of A and B are both linearly dependent or linearly independent. Observe that the column vectors of B associated with the pivots are linearly independent since none of the vectors can be a linear combination of the column vectors that come before. Therefore, by the previous remarks, the corresponding column vectors of A are also linearly independent. By Theorem 12, these same column vectors form a basis for col(A). When choosing vectors for a basis of col(A), we must select the column vectors in A corresponding to the pivot column vectors of B, and not the pivot column vectors of B. For example, the row-reduced echelon form of the matrix     1 0 1 1 0 1 is the matrix B= 0 1 1  A= 0 0 0  0 0 0 0 1 1 However, the column spaces of A and B are different. In this case col(A) is the xz plane and col(B) is the xy plane with     0   1 col(A) = span{v1 , v2 } = span  0 ,  0    1 0    0   1 col(B) = span{w1 , w2 } = span  0 ,  1    0 0 respectively. See Fig. 2. and

z

v2 xz plane

z

span{v1 , v2 }

w2

y

y

w1

v1 x

xy plane Figure 2

x

span{w1 , w2 }

Confirming Pages

168

Chapter 3 Vector Spaces

The details of these observations are made clearer by example. Let     2 1  1 S = {v1 , v2 , v3 , v4 , v5 } =  1 ,  0 ,  1 ,   2 1 0

considering a specific   3  2 1 ,  1   3 1

We begin by considering the equation             0 1 1 2 2 3 c 1  1  + c2  0  + c3  1  + c4  1  + c5  1  =  0  0 0 1 2 1 3 To solve this echelon form.  1  1 0

system, we reduce the corresponding augmented That is,   1 2 2 3 0 1 0 0 1 0 1 1 1 0  reduces to  0 1 0 1 1 2 1 3 0 0 0 1 0

matrix to reduced 0 1 1

 0 0  0

In the general solution, the variables c1 , c2 , and c3 are the dependent variables corresponding to the leading ones in the reduced matrix, while c4 and c5 are free. Thus, the solution is given by S = {(−s, −s − t, −t, s, t) | s, t ∈ ⺢} Now to find a basis for span(S), we substitute these values into equation to obtain          2 2 1 1 −s  1  + (−s − t)  0  + (−t)  1  + s  1  + t  1 2 1 0

the original vector    0 3 1 = 0  0 3

We claim that each of the vectors corresponding to a free variable is a linear combination of the others. To establish the claim in this case, let s = 1 and t = 0. The above vector equation now becomes         0 2 1 1 − 1  − 0 +  1  =  0  0 1 1 0 that is, −v1 − v2 + v4 = 0 Thus, v4 is a linear combination of v1 and v2 . Also, to see that v5 is a linear combination of v1 , v2 , and v3 , we let s = 0 and t = 1. In light of Theorem 8 we eliminate v4 and v5 from S to obtain S = {v1 , v2 , v3 }. Observe that S is linearly independent since each of these vectors corresponds to a column with a leading 1. Thus, the equation         0 1 1 2 c1  1  + c 2  0  + c 3  1  =  0  0 0 1 2 has only the trivial solution.

Confirming Pages

3.3 Basis and Dimension

169

We summarize the procedure for finding a basis for the span of a set of vectors. Given a set S = {v1 , v2 , v3 , . . . , vn } to find a basis for span(S): 1. Form a matrix A whose column vectors are v1 , v2 , . . . , vn . 2. Reduce A to row echelon form. 3. The vectors from S that correspond to the columns of the reduced matrix with the leading 1s are a basis for span(S). In Example 8 we use the process described above to show how to obtain a basis from a spanning set.

EXAMPLE 8

Let

      −1  1 1 0  1 S =  0 ,  1 ,  1 ,  2 ,  1    −2 1 2 1 1

Find a basis for the span of S. Solution

Start by constructing the reduce the matrix  1 0 1  0 1 1 1 1 2

matrix whose column vectors are the vectors in S. We  1 −1 2 1  1 −2

 1 0 1 0 −2  0 1 1 0 −1  0 0 0 1 1 

to

Observe that the leading 1s in the reduced matrix are in columns 1, 2, and 4. Therefore, a basis B for span(S) is given by {v1 , v2 , v4 }, that is,     1  0  1 B =  0 ,  1 ,  2    1 1 1

A set of vectors in a vector space that is not a basis can be expanded to a basis by using Theorem 13. THEOREM 13

Suppose that S = {v1 , v2 , . . . , vn } is a linearly independent subset of a vector space V . If v is a vector in V that is not in span(S), then T = {v, v1 , v2 , . . . , vn } is linearly independent. Proof To show that T is linearly independent, we consider the equation c1 v1 + c2 v2 + · · · + cn vn + cn+1 v = 0 If cn+1 = 0, then we can solve for the vector v in terms of the vectors of S, contrary to the hypothesis that v is not in the span of S. Thus, cn+1 = 0 and the starting equation is equivalent to c1 v1 + c2 v2 + · · · + cn vn = 0

Confirming Pages

170

Chapter 3 Vector Spaces

Since S is linearly independent, then c1 = 0

c2 = 0

...

cn = 0

Hence, T is linearly independent.

An alternative method for expanding a set of vectors in ⺢n to a basis is to add the coordinate vectors to the set and then trim the resulting set to a basis. This technique is illustrated in Example 9.

EXAMPLE 9

Solution

Find a basis for ⺢4 that contains the vectors   1  0   and v1 =   1  0

 −1  1   v2 =   −1  0 

Notice that the set {v1 , v2 } is linearly independent. However, it cannot span ⺢4 since dim(⺢4 ) = 4. To find a basis, form the set S = {v1 , v2 , e1 , e2 , e3 , e4 }. Since span{e1 , e2 , e3 , e4 } = ⺢4 , we know that span(S) = ⺢4 . Now proceed as in Example 8 by reducing the matrix   1 −1 1 0 0 0  0 1 0 1 0 0     1 −1 0 0 1 0  0 0 0 0 0 1 to reduced row echelon form 

1  0   0 0

0 1 0 0

0 0 1 0

1 1 1 0 0 −1 0 0

 0 0   0  1

Observe that the pivot columns are 1, 2, 3, and 6. A basis is therefore given by the set of vectors {v1 , v2 , e1 , e4 }.

The following useful corollary results from repeated application of Theorem 13.

COROLLARY 1

Let S = {v1 , v2 , . . . , vr } be a linearly independent set of vectors in an n-dimensional vector space V with r < n. Then S can be expanded to a basis for V . That is, there are vectors {vr+1 , vr+2 , . . . , vn } so that {v1 , v2 , . . . , vr , vr+1 , . . . , vn } is a basis for V .

Confirming Pages

3.3 Basis and Dimension

171

Fact Summary Let V be a vector space with dim(V ) = n. 1. There are finite sets of vectors that span V . The set of vectors can be linearly independent or linearly dependent. A basis is a linearly independent set of vectors that spans V . 2. Every nontrivial vector space has infinitely many bases. 3. Every basis of V has n elements. 4. The standard basis for ⺢n is the n coordinate vectors e1 , e2 , . . . , en . 5. The standard basis for M2×2 consists of the four matrices         0 0 0 0 0 1 1 0 0 1 1 0 0 0 0 0 6. The standard basis for Pn is {1, x, x 2 , . . . , x n }. 7. If vn is a linear combination of v1 , v2 , . . . , vn−1 , then span{v1 , v2 , . . . , vn−1 } = span{v1 , v2 , . . . , vn−1 , vn } 8. 9. 10. 11. 12.

dim(⺢ ) = n, dim(Mm×n ) = mn, dim(Pn ) = n + 1 If a set B of n vectors of V is linearly independent, then B is a basis for V . If the span of a set B of n vectors is V , then B is a basis for V . Every linearly independent subset of V can be expanded to a basis for V . If S is a set of vectors in ⺢n , a basis can always be found for span(S) from the vectors of S. n

Exercise Set 3.3 In Exercises 1–6, explain why the set S is not a basis for the vector space V .    0   2 1. S =  1 ,  −1  V = ⺢3   1 3     8 1 2 V = ⺢2 , , 2. S = −3 0 1      0  −1  1 3. S =  0 ,  1 ,  1  V = ⺢3   1 0 1 4. S = {2, x, x 3 + 2x 2 − 1}V = P3

5. S = {x, x 2 , x 2 + 2x, x 3 − x + 1} V = P3    0 1 1 0 , , 6. S = 0 0 0 1    2 −3 0 0 V = M2×2 , 1 2 1 0 In Exercises 7–12, show that S is basis for the vector space V .    −1 1 V = ⺢2 , 7. S = 2 1    1 −1 V = ⺢2 , 8. S = −1 3

Confirming Pages

172

Chapter 3 Vector Spaces

    0  0 1  9. S =  −1 ,  −2 ,  2    −2 −3 1 V = ⺢3     1  2  −1 10. S =  −1 ,  −1 ,  1    2 −3 0 V = ⺢3   1 1 1 0 , 11. S = −1 0 1 0   0 1 , −1 2

21. Let S be the subspace of V = M2×2 consisting of all 2 × 2 symmetric matrices.

 , 1 0 0 1



12. S = {x 2 + 1, x + 2, −x 2 + x} V = P2

V = ⺢4   2 −1     1 1  16. S =   0 ,  −1    2 1

whether S is a basis  1  1  V = ⺢3  1  3  1  V = ⺢3  1   −1  2    4  ,  2  2   0    3 5



1  3 ,   1 −1

 2     1   ,    1    2 

V = ⺢4 17. S = {1, 2x 2 + x + 2, −x 2 + x} V = P2    1 1 1 0 , , 18. S = 0 0 0 0    0 0 −2 1 , 0 2 1 1 V = M2×2

22. Let S be the subspace of V = M2×2 consisting of all 2 × 2 skew-symmetric matrices. 23. S = {p(x) | p(0) = 0} V = P2

V = M2×2 In Exercises 13–18, determine for the vector space V .    1  −1 13. S =  2 ,  0 ,   1 1    5 2  14. S =  −2 ,  1 ,   2 1    2 1     1  1   15. S =   −1 ,  3 ,     1 1

In Exercises 19–24, find a basis for the subspace S of the vector space V . Specify the dimension of S.   s + 2t   − s + t s, t ∈ ⺢ V = ⺢3 19. S =   t    a a + d a, d ∈ ⺢ V = M2×2 20. S = a+d d

24. S = {p(x) | p(0) = 0, p(1) = 0} V = P3 In Exercises 25–30, find a basis for the span(S) as a subspace of ⺢3 .     1  2 2  25. S =  2 ,  0 ,  2    1 −2 −1     2  4  −2 26. S =  1 ,  −1 ,  0    5 2 3      2  −1 0 2  27. S =  −3 ,  2 ,  −1 ,  3    −1 0 2 0      1  −3 1  −2 28. S =  0 ,  0 ,  −3 ,  2    −2 −2 −3 2       4  2 0 2  29. S =  −3 ,  2 ,  −1 ,  0    4 2 2 0      2  0 1  2 30. S =  2 ,  −1 ,  2 ,  3    1 2 0 0 In Exercises 31–36, find a basis for the vector space V that contains the given vectors.    1  2  31. S =  −1 ,  0  V = ⺢3   2 3

Confirming Pages

3.4 Coordinates and Change of Basis

    1   −1 32. S =  1 ,  1  V = ⺢3   1 3   1     −1  ,  33. S =   2     4

 3    1   V = ⺢4 1    2

  1 −1      −3 1  34. S =   1 ,  −1    2 −1

 1     −2   V = ⺢4 ,    −1    3 

    1   −1 35. S =  1 ,  1  V = ⺢3   1 3   36. S =  

  −1  2 2 ,  −1  V = ⺢3  3 −1

37. Find a basis for the subspace of Mn×n consisting of all diagonal matrices. 38. Show that if S = {v1 , v2 , . . . , vn } is a basis for the vector space V and c is a nonzero scalar, then S = {cv1 , cv2 , . . . , cvn } is also a basis for V . 39. Show that if S = {v1 , v2 , . . . , vn } is a basis for ⺢n and A is an n × n invertible matrix, then S = {Av1 , Av2 , . . . , Avn } is also a basis. ß

3.4

173

40. Find a basis for the subspace S = {x ∈ ⺢4 | Ax = 0} of ⺢4 where   3 3 1 3 A =  −1 0 −1 −1  2 0 2 1 41. Suppose that V is a vector space with dim(V ) = n. Show that if H is a subspace of V and dim(H ) = n, then H = V . 42. Let S and T be the subspaces of P3 defined by S = {p(x) | p(0) = 0} and T = {q(x) | q(1) = 0} Find dim(S), dim(T ), and dim(S ∩ T ). 43. Let

     2s + t + 3r W =  3s − t + 2r  s, t, r ∈ ⺢   s + t + 2r

Find dim(W ). 44. Let S and T be the subspaces of ⺢4 defined by    s        t  s, t ∈ ⺢ S=   0        0    0        s  s, t ∈ ⺢ T =    t       0 Find dim(S), dim(T ), and dim(S ∩ T ).

and

Coordinates and Change of Basis From our earliest experiences with Euclidean space we have used rectangular coordinates, (or xy coordinates), to specify the location of a point in the plane. Equivalently, these coordinates describe a vector in standard position which terminates at the point. Equipped with our knowledge of linear combinations, we now understand these xy coordinates to be the scalar multiples required to express the vector as a linear com 2 bination of the standard basis vectors e1 and e2 . For example, the vector v = , 3 with xy coordinates (2, 3), can be written as v = 2e1 + 3e2

Confirming Pages

Chapter 3 Vector Spaces

as shown in Fig. 1(a). This point (or vector) can also be specified relative to another pair of linearly independent vectors, describing an x y coordinate system. For example, since       2 1 −1 5 1 =2 +2 3 1 1   5 1

the x y coordinates of v are given by 2 , 2 . See Fig. 1(b). y

x'

y y'

174

x

x

(a)

(b)

Figure 1

In this section we generalize this concept to abstract vector spaces. Let V be a vector space with basis B = {v1 , v2 , . . . , vn }. From Theorem 7 of Sec. 2.3, every vector v in V can be written uniquely as a linear combination of the vectors of B. That is, there are unique scalars c1 , c2 , . . . , cn such that v = c1 v1 + c2 v2 + · · · + cn vn It is tempting to associate the list of scalars {c1 , c2 , . . . , cn } with the coordinates of the vector v. However, changing the order of the basis vectors in B will change the order of the scalars. For example, the sets     0 1 , B= 1 0 and

B =



   1 0 , 0 1

 1 is 2 {1, 2} relative to B but is {2, 1} relative to B . To remove this ambiguity, we introduce the notion of an ordered basis. 

are both bases for ⺢2 . Then the list of scalars associated with the vector

DEFINITION 1

Ordered Basis An ordered basis of a vector space V is a fixed sequence of linearly independent vectors that span V .

Confirming Pages

3.4 Coordinates and Change of Basis

DEFINITION 2

175

Coordinates Let B = {v1 , v2 , . . . , vn } be an ordered basis for the vector space V . Let v be a vector in V , and let c1 , c2 , . . . , cn be the unique scalars such that v = c1 v1 + c2 v2 + · · · + cn vn Then c1 , c2 , . . . , cn are called the coordinates of v relative to B. In this case we write   c1  c2   [v]B =   ...  cn and refer to the vector [v]B as the coordinate vector of v relative to B.

In ⺢n the coordinates of a vector relative to the standard basis B = {e1 , e2 , . . . , en } are simply the components of the vector. Similarly, the coordinates of a polynomial p(x) = a0 + a1 x + a2 x 2 + · · · + an x n in Pn relative to the standard basis {1, x, x 2 , . . . , x n } are the coefficients of the polynomial.

EXAMPLE 1

Solution

Let V = ⺢2 and B be the ordered basis    −1 1 , B= 1 1   1 relative to B. Find the coordinates of the vector v = 5 The coordinates c1 and c2 are found by writing v as a linear combination of the two vectors in B. That is, we solve the equation       1 1 −1 + c2 = c1 5 1 1 In this  c1 = 3 and c2 = 2. We therefore have that the coordinate vector of  case 1 relative to B is v= 5   3 [v]B = 2

EXAMPLE 2

Let V = P2 and B be the ordered basis   B = 1, x − 1, (x − 1)2

Confirming Pages

176

Chapter 3 Vector Spaces

Find the coordinates of p(x) = 2x 2 − 2x + 1 relative to B. Solution

We must find c1 , c2 , and c3 such that c1 (1) + c2 (x − 1) + c3 (x − 1)2 = 2x 2 − 2x + 1 Expanding the left-hand side and collecting like terms give c3 x 2 + (c2 − 2c3 )x + (c1 − c2 + c3 ) = 2x 2 − 2x + 1 Equating the coefficients of like terms gives the linear system    c1 − c2 + c3 = 1 c2 − 2c3 = −2   c3 = 2 The unique solution to this system is c1 = 1, c2 = 2, and c3 = 2, so that   1 [v]B =  2  2

EXAMPLE 3

Let W be the subspace of all symmetric matrices in the vector space M2×2 . Let     0 0 0 1 1 0 , , B= 0 1 1 0 0 0 Show that B is a basis for W and find the coordinates of   2 3 v= 3 5 relative to B.

Solution

In Example 8 of Sec. 3.2, we showed that B spans W. The matrices in B are also linearly independent and hence are a basis for W. Observe that v can be written as         2 3 0 0 0 1 1 0 = +5 +3 2 3 5 0 1 1 0 0 0 Then relative to the ordered basis B, the coordinate vector of v is   2 [v]B =  3  5

Confirming Pages

3.4 Coordinates and Change of Basis

177

Change of Basis Many problems in applied mathematics are made easier by changing from one basis of a vector space to another. To simplify our explanation of the process, we will consider a vector space V with dim(V ) = 2 and show how to change from coordinates relative to one basis for V to another basis for V . Let V be a vector space of dimension 2 and let B = {v1 , v2 } and B = {v 1 , v 2 } be ordered bases for V . Now let v be a vector in V , and suppose that the coordinates of v relative to B are given by   x1 that is v = x1 v1 + x2 v2 [v]B = x2 To determine the coordinates of v relative to B , we first write v1 and v2 in terms of the vectors v 1 and v 2 . Since B is a basis, there are scalars a1 , a2 , b1 , and b2 such that v1 = a1 v 1 + a2 v 2 v2 = b1 v 1 + b2 v 2 Then v can be written as v = x1 (a1 v 1 + a2 v 2 ) + x2 (b1 v 1 + b2 v 2 ) Collecting the coefficients of v 1 and v 2 gives v = (x1 a1 + x2 b1 )v 1 + (x1 a2 + x2 b2 )v 2 so that the coordinates of v relative to the basis B are given by   x1 a1 + x2 b1 [v]B = x1 a2 + x2 b2 Now by rewriting the vector on the right-hand side as a matrix product, we have      x1 a1 b1 a1 b1 = [v]B [v]B = a2 b2 x2 a2 b2 Notice that the column vectors of the matrix are the coordinate vectors [v1 ]B and [v2 ]B . The matrix   a1 b1 a2 b2

is called the transition matrix from B to B and is denoted by [I ]BB , so that

[v]B = [I ]BB [v]B

Confirming Pages

178

Chapter 3 Vector Spaces

EXAMPLE 4

Let V = ⺢2 with bases    1 1 , B= −1 1



B =

and

  −1 2 , 1 −1

a. Find the transition matrix from B to B .  3 and find [v]B . b. Let [v]B = −2

Solution

a. By denoting the vectors in B by v1 and v2 and those in B by v 1 and v 2 , the column vectors of the transition matrix are [v1 ]B and [v2 ]B . These coordinate vectors are found from the equations             1 1 2 −1 2 −1 and d1 + c2 = + d2 = c1 1 −1 −1 1 −1 1 Solving these equations gives c1 = 2 and c2 = 3, and d1 = 0 and d2 = −1, so that     2 0 and [v2 ]B = [v1 ]B = 3 −1 Therefore, the transition matrix is  

2 0 [I ]BB = 3 −1 b. Since

[v]B = [I ]BB [v]B     6 3 2 0 = 11 −2 3 −1 Observe that the same vector, relative to the different bases, is obtained from the coordinates [v]B and [v]B . That is,           −1 2 1 1 1 + 11 =6 = −2 3 1 −1 5 −1 1 

then

[v]B =

The procedure to find the transition matrix between two bases of a vector space of dimension 2 can be generalized to ⺢n and other vector spaces of finite dimension. The result is stated in Theorem 14.

THEOREM 14

Let V be a vector space of dimension n with ordered bases B = {v1 , v2 , . . . , vn }

and

B = {v 1 , v 2 , . . . , v n }

Then the transition matrix from B to B is given by      B

[I ]B =  v1 

 

 v2  · · ·  vn   B

B

B

Confirming Pages

3.4 Coordinates and Change of Basis

179

Moreover, a change of coordinates is carried out by

[v]B = [I ]BB [v]B

In Example 5 we use the result of Theorem 14 to change from one basis of P2 to another.

EXAMPLE 5

Let V = P2 with bases B = {1, x, x 2 }

and

B = {1, x + 1, x 2 + x + 1}

a. Find the transition matrix [I ]BB . b. Let p(x) = 3 − x + 2x 2 and find [p(x)]B . Solution

a. To find the first column vector of the transition matrix, we must find scalars a1 , a2 , and a3 such that a1 (1) + a2 (x + 1) + a3 (x 2 + x + 1) = 1 By inspection we see that the solution is a1 = 1, a2 = 0, and a3 = 0. Therefore,   1 [1]B =  0  0 The second and third column vectors of the transition matrix can be found by solving the equations b1 (1) + b2 (x + 1) + b3 (x 2 + x + 1) = x and c1 (1) + c2 (x + 1) + c3 (x 2 + x + 1) = x 2 respectively. The solutions are given by b1 = −1, b2 = 1, and b3 = 0, and c1 = 0, c2 = −1, and c3 = 1. Hence, the transition matrix is   1 −1 0

1 −1  [I ]BB =  0 0 0 1 b. The basis B is the standard basis for P2 , so the coordinate vector of p(x) = 3 − x + 2x 2 relative to B is given by   3 [p(x)]B =  −1  2

Confirming Pages

180

Chapter 3 Vector Spaces

    4 3 1 −1 0 1 −1   −1  =  −3  = 0 2 2 0 0 1 

Hence, [p(x)]B

Notice that 3 − x + 2x 2 = 4(1) − 3(x + 1) + 2(x 2 + x + 1).

EXAMPLE 6

Let B = {e1 , e2 } be the standard ordered basis for ⺢2 , B be the ordered basis given by     1 −1 , B = {v 1 , v 2 } = 1 1   3 . and let v = 4 a. Find the transition matrix from B to B . b. Find [v]B . c. Write the vector v as a linear combination of e1 and e2 and also as a linear combination of v 1 and v 2 . d. Show the results of part (c) graphically.

Solution

a. The transition matrix from B to B is computed by solving the equations             1 0 −1 1 −1 1 and d1 + c2 = + d2 = c1 0 1 1 1 1 1 That is, we must solve the linear systems  −c1 + c2 = 1 and c 1 + c2 = 0



−d1 + d2 d1 + d2

The unique solutions are given by c1 = − 12 , c2 = transition matrix is then given by  1 1  −2 2 B [I ]B = 1 1 2

1 2

=0 =1

and d1 = 12 , d2 = 12 . The

2

b.  Since B is the standard basis, the coordinates of v relative to B are [v]B = 3 . By Theorem 14, the coordinates of v relative to B are given by 4   1 1    1 −2 2 3 2 = 7 [v]B = 1 1 4 2

2

2

Revised Confirming Pages

3.4 Coordinates and Change of Basis

181

c. Using the coordinates of v relative to the two bases, we have

 1 7 0 1 = v = v
1 + v
2 +4 3 1 0 2 2 d. The picture given in Fig. 2 shows the location of the terminal point (3, 4) of the vector v relative to the e1 e2 axes and the v
1 v
2 axes. 5

v

v2 v
1

v
2

⫺5

5

v1

⫺5

Figure 2

Inverse of a Transition Matrix


A fact that will be useful in Chap. 4 is that the transition matrix [I ]BB between bases B and B
of a finite dimensional vector space is invertible. Moreover, the transition
matrix from B
to B is the inverse of [I ]BB . To see this, suppose that V is a vector space of dimension n with ordered bases B = {v1 , v2 , . . . , vn }

B
= {v
1 , v
2 , . . . , v
n }

and




To show that [I ]BB is invertible, let x ∈ ⺢n be such that


[I ]BB x = 0 Observe that the left-hand side of this equation in vector form is x1 [v1 ]B
+ · · · + xn [vn ]B
. Since B is a basis, then the vectors vi for 1 ≤ i ≤ n are linearly independent. Hence, so are the vectors [v1 ]B
, · · · , [vn ]B
. Therefore, x1 = x2 = · · · = xn = 0. Since
the only solution to the homogeneous equation [I ]BB x = 0 is the trivial solution, then B
by Theorem 17 of Sec. 1.6, the matrix [I ]B is invertible. Moreover, by Theorem 14, since

[v]B
= [I ]BB [v]B we know that ([I ]BB )−1 [v]B
= [v]B and therefore




[I ]BB
= ([I ]BB )−1

The previous observations are summarized in Theorem 15.

Confirming Pages

182

Chapter 3 Vector Spaces

THEOREM 15

Let V be a vector space of dimension n with ordered bases B = {v1 , v2 , . . . , vn }

B = {v 1 , v 2 , . . . , v n }

and

Then the transition matrix [I ]BB from B to B is invertible and

[I ]BB = ([I ]BB )−1

Fact Summary Let V be a vector space with dim(V ) = n. 1. In ⺢n , the coordinates of a vector with respect to the standard basis are the components of the vector. 2. Given any two ordered bases for V , a transition matrix can be used to change the coordinates of a vector relative to one basis to the coordinates relative to the other basis. 3. If B and B are two ordered bases for V , the transition matrix from B to B

is the matrix [I ]BB whose column vectors are the coordinates of the basis vectors of B relative to the basis B . Also

[v]B = [I ]BB [v]B 4. If B and B are two ordered bases for V , the transition matrix from B to B is invertible and the inverse matrix is the transition matrix from B to B.

That is, ([I ]BB )−1 = [I ]BB

Exercise Set 3.4 In Exercises 1–8, find the coordinates of the vector v relative to the ordered basis B.       8 −2 3 v= , 1. B = 0 2 1      −2 −1 −2 v= , 2. B = 1 1 4      1  3 1  3. B =  −1 ,  −1 ,  0    2 1 2

 2 v =  −1  9       0  1  2 4. B =  2 ,  0 ,  0    1 2 1   0 v= 1  

1 2

Confirming Pages

3.4 Coordinates and Change of Basis

5. B = {1, x − 1, x 2 } v = p(x) = −2x 2 + 2x + 3 6. B = {x 2 + 2x + 2, 2x + 3, −x 2 + x + 1} v = p(x) = −3x 2 + 6x + 8    0 1 1 −1 , , 7. B = 1 0 0 0    1 0 1 0 , −1 0 0 −1   1 3 v= −2 2    0 1 1 −1 , , 8. B = 0 2 0 1    1 1 1 −1 , 0 3 1 0   2 −2 v= 1 3 In Exercises 9–12, find the coordinates of the vector v relative to the two ordered bases B1 and B2 .     2 −3 , 9. B1 = 2 1       1 0 2 v= B2 = , 0 1 1     −1  1  −2 10. B1 =  1 ,  0 ,  1    0 2 0      −1  0  1 B2 =  0 ,  1 ,  −1    0 2 1   −2 v= 2  2 11. B1 = {x 2 − x + 1, x 2 + x + 1, 2x 2 } B2 = {2x 2 + 1, −x 2 + x + 2, x + 3} v = p(x) = x 2 + x + 3

183

  1 −1 1 0 , 12. B1 = 1 0 0 2    2 0 0 1 , −1 1 1 0    −1 0 3 −1 B2 = , , 1 1 0 1    0 −1 0 0 , 1 1 1 0   0 0 v= 3 1 

In Exercises 13–18, find the transition matrix between the ordered bases B1 and B2 ; then given [v]B1 , find [v]B2 .     −1 1 , 13. B1 = 1 1       0 1 2 B2 = [v]B1 = , 1 0 3     1 −2 14. B1 = , 2 1       2 1 1 B2 = [v]B1 = , 0 3 −1       0  0  1 15. B1 =  2 ,  1 ,  1    0 1 1      2  −1  0 B2 =  1 ,  1 ,  1    −1 −1 0   −1 [v]B1 =  0  2     −1  1  −2 16. B1 =  1 ,  0 ,  1    0 2 0      −1  0  1 B2 =  0 ,  1 ,  −1    0 2 1

Confirming Pages

184

Chapter 3 Vector Spaces

 2 = 1  1 

[v]B1

Find the coordinates of the vector   a  b   v=  c  d

17. B1 = {1, x, x 2 } B2 = {x 2 , 1, x},   2 [v]B1 =  3  5

relative to the ordered basis B.

18. B1 = {x 2 − 1, 2x 2 + x + 1, −x + 1}

21. Let

B2 = {(x − 1)2 , x + 2, (x + 1)2 }   1 [v]B1 =  1  2 19. Let B = {v1 , v2 , v3 } be the ordered basis of ⺢3 consisting of     −1 1 v2 =  0  v1 =  1  1 1   −1 v3 =  1  0 Find the coordinates of the vector   a v= b  c

      0  0  1 B1 =  0 ,  1 ,  0    1 0 0

be the standard ordered basis for ⺢3 and let       0  1  0 B2 =  1 ,  0 ,  0    1 0 0 be a second ordered basis. a. Find the transition matrix from the ordered basis B1 to the ordered basis B2 . b. Find the coordinates of the vector   1 v= 2  3 relative to the ordered basis B2 . 22. Let

relative to the ordered basis B. 20. Let B = {v1 , v2 , v3 , v4 } be the basis of consisting of     1 0  0   −1    v2 =  v1 =   1   1  0 −1     0 −1  −1   0    v3 =  v4 =   −1   0  0 −1



  1 2 , −2 2



   3 −1 , 0 2

B1 =

⺢4 and

B2 =

be two ordered bases for ⺢2 . B

a. Find [I ]B21 B

b. Find [I ]B12

 B c. Show that [I ]B21

−1

B

= [I ]B12

Confirming Pages

3.5 Application: Differential Equations

23. Let

 S=

   0 1 , 1 0

be the standard ordered basis for ⺢2 and let    1 ! −2 1 , B= 1 0 2

be a second ordered basis. a. Find [I ]BS b. Find the coordinates of      4 4 1 1 4 2 4 2 relative to the ordered basis B. c. Draw the rectangle in the plane with vertices (1, 2), (1, 4), (4, 1), and (4, 4). d. Draw the polygon in the plane with vertices given by the coordinates found in part (b). 24. Fix a real number θ and define the transition matrix from the standard ordered basis S on ⺢2 to ß

3.5

185

a second ordered basis B by   cos θ − sin θ B [I ]S = sin θ cos θ   x a. If [v]S = , then find [v]B . y b. Draw the rectangle in the plane with vertices      1 1 0 0 1 0 1 0 c. Let θ = π2 . Draw the rectangle in the plane with vertices the coordinates of the vectors, given in part (b), relative to the ordered basis B. 25. Suppose that B1 = {u1 , u2 , u3 } and B2 = {v1 , v2 , v3 } are ordered bases for a vector space V such that u1 = −v1 + 2v2 , u2 = −v1 + 2v2 − v3 , and u3 = −v2 + v3 . B a. Find the transition matrix [I ]B21

b. Find [2u1 − 3u2 + u3 ]B2

Application: Differential Equations Differential equations arise naturally in virtually every branch of science and technology. They are used extensively by scientists and engineers to solve problems concerning growth, motion, vibrations, forces, or any problem involving the rates of change of variable quantities. Not surprisingly, mathematicians have devoted a great deal of effort to developing methods for solving differential equations. As it turns out, linear algebra is highly useful to these efforts. However, linear algebra also makes it possible to attain a deeper understanding of the theoretical foundations of these equations and their solutions. In this section and in Sec. 5.3 we give a brief introduction to the connection between linear algebra and differential equations. As a first step, let y be a function of a single variable x. An equation that involves x, y, y , y

, . . . , y (n) , where n is a fixed positive integer, is called an ordinary differential equation of order n. We will henceforth drop the qualifier ordinary since none of the equations we investigate will involve partial derivatives. Also, for obvious reasons we will narrow the scope of our discussion and consider only equations of a certain type.

Confirming Pages

186

Chapter 3 Vector Spaces

The Exponential Model One of the simplest kinds of differential equations is the first-order equation given by y = ky where k is a real number. This equation is used to model quantities that exhibit exponential growth or decay and is based on the assumption that the rate of change of the quantity present at any time t is directly proportional to the quantity present at time t. A solution to a differential equation is a function y = f (t) that satisfies the equation, that is, results in an identity when substituted for y in the original equation. To solve this equation, we write it as y =k y and integrate both sides of the equation with respect to the independent variable to obtain " " y ln y = dt = k dt = kt + A y Solving for y gives y = eln y = ekt+A = eA ekt = Cekt where C is an arbitrary constant. As an illustration, consider the differential equation y = 3y. Then any function of the form y(t) = Ce3t is a solution. Since the parameter C in the solution is arbitrary, the solution produces a family of functions all of which satisfy the differential equation. For this reason y(t) = Ce3t is called the general solution to y = 3y. In certain cases a physical constraint imposes a condition on the solution that allows for the identification of a particular solution. If, for example, in the previous problem it is required that y = 2 when t = 0, then 2 = Ce3(0) , so that C = 2. This is called an initial condition. A differential equation together with an initial condition is called an initial-value problem. The solution to the previous initial-value problem is given by y(t) = 2e3t From a linear algebra perspective we can think of the general solution to the differential equation y = ky as the span, over ⺢, of the vector ekt which describes a one-dimensional subspace of the vector space of differentiable functions on the real line.

Second-Order Differential Equations with Constant Coefficients We now extend the differential equation of the previous subsection to second-order and consider equations of the form y

+ ay + by = 0 Motivated by the solution to the exponential model, we check to see if there are any solutions of the form y = erx , for some real number r. After computing the first and

Confirming Pages

3.5 Application: Differential Equations

187

second derivatives y = rerx and y

= r 2 erx , we see that y = erx is a solution of the second-order equation if and only if r 2 erx + arerx + berx = 0 that is, erx (r 2 + ar + b) = 0 Since erx > 0 for every choice of r and x, we know erx is a solution of y

+ ay + by = 0 if and only if r 2 + ar + b = 0 This equation is called the auxiliary equation. As this equation is quadratic there are three possibilities for the roots r1 and r2 . This in turn yields three possible variations for the solution of the differential equation. The auxiliary equation can have two distinct real roots, one real root, or two distinct complex roots. These cases are considered in order. Case 1 The roots r1 and r2 are real and distinct. In this case there are two solutions, given by y1 (x) = er1 x

EXAMPLE 1 Solution

and

y2 (x) = er2 x

Find two distinct solutions to the differential equation y

− 3y + 2y = 0. Let y = erx . Since the auxiliary equation r 2 − 3r + 2 = (r − 1)(r − 2) = 0 has the distinct real roots r1 = 1 and r2 = 2, two distinct solutions for the differential equation are y1 (x) = ex and y2 (x) = e2x

Case 2 There is one repeated root r. Although the auxiliary equation has only one root, there are still two distinct solutions, given by y1 (x) = erx

EXAMPLE 2 Solution

and

y2 (x) = xerx

Find two distinct solutions to the differential equation y

− 2y + y = 0. Let y = erx . Since the auxiliary equation r 2 − 2r + 1 = (r − 1)2 = 0 has the repeated root r = 1, two distinct solutions of the differential equation are y1 (x) = ex

and

y2 (x) = xex

Confirming Pages

188

Chapter 3 Vector Spaces

Case 3 The auxiliary equation has distinct complex (conjugate) roots given by r1 = α + βi and r2 = α − βi. In this case the solutions are y1 (x) = eαx cos βx

EXAMPLE 3 Solution

and

y2 (x) = eαx sin βx

Find two distinct solutions to the differential equation y

− 2y + 5y = 0. Let y = erx , so the auxiliary equation corresponding to y

− 2y + 5y = 0 is given by r 2 − 2r + 5 = 0. Applying the quadratic formula gives the complex roots r1 = 1 + 2i and r2 = 1 − 2i. The two solutions to the differential equation are then given by y1 (x) = ex cos 2x and y2 (x) = ex sin 2x

In what follows we require Theorem 16 on existence and uniqueness for secondorder linear differential equations. A proof can be found in any text on ordinary differential equations.

THEOREM 16

Let p(x), q(x), and f (x) be continuous functions on the interval I. If x0 is in I, then the initial-value problem y

+ p(x)y + q(x)y = f (x)

y(x0 ) = y0

y (x0 ) = y0

has a unique solution on I.

Fundamental Sets of Solutions With solutions in hand for each one of these cases, we now consider the question as to whether there are other solutions to equations of this type, and if so, how they can be described. The simple (but elegant) answer, to which the remainder of this section is devoted, is found by using linear algebra. We will see that in each case the functions y1 (x) and y2 (x) form a basis for the vector space of solutions to the equation y

+ ay + by = 0. Accordingly, every solution y(x) to this equation can be written as a linear combination y(x) = c1 y1 (x) + c2 y2 (x). Toward this end, for a positive integer n ≥ 0, let V = C (n) (I ) be the vector space of all functions that are n times differentiable on the real interval I. If n = 0, then C (0) (I ) denotes the set of all continuous functions on I. We first show that the solution set to the differential equation y

+ ay + by = 0 is a subspace of V = C (2) (I ).

THEOREM 17

Superposition Principle Suppose that y1 (x) and y2 (x) are functions in C (2) (I ). If y1 (x) and y2 (x) are solutions to the differential equation y

+ ay + by = 0 and c is any scalar, then y1 (x) + cy2 (x) is also a solution.

Confirming Pages

3.5 Application: Differential Equations

189

Proof Since y1 (x) and y2 (x) are both solutions, then y1

(x) + ay1 (x) + by1 (x) = 0

and

y2

(x) + ay2 (x) + by2 (x) = 0

Now to show that y(x) = y1 (x) + cy2 (x) is a solution to the differential equation, observe that y (x) = y1 (x) + cy2 (x)

and

y

(x) = y1

(x) + cy2

(x)

Substituting the values for y, y , and y

in the differential equation and rearranging the terms gives y1

(x) + cy2

(x) + a[y1 (x) + cy2 (x)] + b[y1 (x) + cy2 (x)] = y1

(x) + cy2

(x) + ay1 (x) + acy2 (x) + by1 (x) + bcy2 (x) = [y1

(x) + ay1 (x) + by1 (x)] + c[y2

(x) + ay2 (x) + by2 (x)] =0+0=0

Let S be the set of solutions to the differential equation y

+ ay + by = 0. By the superposition principle above and by Theorem 4 of Sec. 3.2, we know that S is a subspace of C (2) (I ). To analyze the algebraic structure of S, we recall from Exercise 31 of Sec. 2.3 that a set of functions U = {f1 (x), f2 (x), . . . , fn (x)} is linearly independent on an interval I if and only if c1 f1 (x) + c2 f2 (x) + · · · + cn fn (x) = 0 for all x ∈ I implies that c1 = c2 = · · · = cn = 0. Theorem 18 provides a useful test to decide whether two functions are linearly independent on an interval.

THEOREM 18

Wronskian Let f (x) and g(x) be differentiable functions on an interval I . Define the function W [f, g] on I by f (x) g(x) = f (x)g (x) − f (x)g(x) W [f, g](x) = f (x) g (x) If W [f, g](x0 ) is nonzero for some x0 in I , then f (x) and g(x) are linearly independent on I . Proof Consider the equation c1 f (x) + c2 g(x) = 0 Taking derivatives of both sides, we obtain c1 f (x) + c2 g (x) = 0 Taken together, these equations form a linear system of two equations in the two variables c1 #and c$2 . Observe that the determinant of the corresponding coefficient matrix is W f, g (x). Hence, if W [f, g](x) is nonzero for some x0 ∈ I , then by

Confirming Pages

190

Chapter 3 Vector Spaces

Theorem 17 of Sec. 1.6 we know that c1 = c2 = 0. Accordingly, f (x) and g(x) are linearly independent.

The function W [f, g] of Theorem 18 is called the Wronskian of f and g. The Wronskian, and the result of Theorem 18, can be extended to any finite set of functions that have continuous derivatives up to order n. If y1 and y2 are solutions to the differential equation y

+ ay + by = 0, then Abel’s formula for the Wronskian gives the next result.

THEOREM 19

Let y1 (x) and y2 (x) be solutions to the differential equation y

+ ay + by = 0. The functions y1 and y2 are linearly independent if and only if W [y1 , y2 ](x) = 0 for all x in I .

At this point we are now ready to show that any two linearly independent solutions to the differential equation y

+ ay + by = 0 span the subspace of solutions.

THEOREM 20

Fundamental Set of Solutions Suppose that y1 (x) and y2 (x) are two linearly independent solutions, on the interval I , to the differential equation y

+ ay + by = 0 Then every solution can be written as a linear combination of y1 (x) and y2 (x). Proof Let y(x) be a particular solution to the initial-value problem y

+ ay + by = 0

with

y(x0 ) = y0

and

y (x0 ) = y0

for some x0 in I . We claim that there exist real numbers c1 and c2 such that y(x) = c1 y1 (x) + c2 y2 (x) Differentiating both sides of this equation gives y (x) = c1 y1 (x) + c2 y2 (x) Now substituting x0 into both of these equations and using the initial conditions above, we obtain the linear system of two equations in the two variables c1 and c2 given by  c1 y1 (x0 ) + c2 y2 (x0 ) = y0 c1 y1 (x0 ) + c2 y2 (x0 ) = y0 Observe that the determinant of the coefficient matrix is the Wronskian W [y1 , y2 ](x0 ). Since y1 (x) and y2 (x) are linearly independent, then by Theorem 19, the determinant of the coefficient matrix is nonzero. Consequently, by Theorem 17

Confirming Pages

3.5 Application: Differential Equations

191

of Sec. 1.6, there exist unique numbers c1 and c2 that provide a solution for the linear system. Define the function g by g(x) = c1 y1 (x) + c2 y2 (x) Then g(x) is also a solution to the original initial-value problem. By the uniqueness part of Theorem 16, y(x) = g(x) = c1 y1 (x) + c2 y2 (x) as claimed.

The linearly independent solutions y1 (x) and y2 (x) of Theorem 20 are called a fundamental set of solutions. In light of this theorem, the fundamental set {y1 (x), y2 (x)} is a basis for the subspace S of solutions to y

+ ay + by = 0. As there are two of them, dim(S) = 2. We now return to the specific cases for the solutions to y

+ ay + by = 0. Recall that for case 1 we obtained the two solutions y1 (x) = er1 x

and

y2 (x) = er2 x

with r1 = r2 . To show that these functions form a fundamental set, we compute the Wronskian, so that rx e1 er2 x W [y1 , y2 ](x) = r 1 e r1 x r 2 e r2 x = r2 (er1 x er2 x ) − r1 (er1 x er2 x ) = r2 e(r1 +r2 )x − r1 e(r1 +r2 )x = e(r1 +r2 )x (r2 − r1 ) Since the exponential function is always greater than 0 and r1 and r2 are distinct, the Wronskian is nonzero for all x, and therefore the functions are linearly independent. Hence, {er1 x , er2 x } is a fundamental set, and every solution y(x) to a problem of this type has the form y(x) = c1 er1 x + c2 er2 x for scalars c1 and c2 . For case 2, the Wronskian is given by W [erx , xerx ] = e2rx Since e2rx is never zero, {erx , xerx } is a fundamental set of solutions for problems of this type. Finally, for case 3 the Wronskian is given by W [eαx cos βx, eαx sin βx] = βe2αx so that {eαx cos βx, eαx sin βx} is a fundamental set as long as β is not zero. If β = 0, then the differential equation becomes y

+ ay = 0 which reduces to case 1. There are many physical applications of second-order differential equations with constant coefficients. Two important areas are in mechanical and electrical oscillations.

Confirming Pages

192

Chapter 3 Vector Spaces

A fundamental problem in mechanics is the motion of an object on a vibrating spring. The motion of the object is described by the solution of an initial-value problem of the form my

+ cy + ky = f (x) y(0) = A y (0) = B where m is the mass of the object attached to the spring, c is the damping coefficient, k is the stiffness of the spring, and f (x) represents some external force. If there are no external forces acting on the system, then f (x) = 0.

EXAMPLE 4

Solution

Let the mass of an object attached to a spring be m = 1, and the spring constant k = 4. Solve the three initial-value problems describing the position of the object attached to the spring with no external forces; initial conditions y(0) = 2, y (0) = 0; and damping coefficients c equaling 2, 4, and 5. The differential equation describing the position of the object is given by y

+ cy + 4y = 0 When c = 2, the auxiliary equation for y

+ 2y + 4y = 0 is r 2 + 2r + 4 = 0

√ √ Since the roots are the complex values r1 = −1 + 3i and r2 = −1 − 3i, the general solution for the differential equation is % √ √ & y(x) = e−x c1 cos( 3x) + c2 sin( 3x) From the initial conditions, we have  y(x) = 2e−x

√ cos( 3x) +

 √ 3 sin( 3x) 3

√

When c = 4, the auxiliary equation for y

+ 4y + 4y = 0 is r 2 + 4r + 4 = (r + 2)2 = 0 Since there is one repeated real root, the general solution for the differential equation is y(x) = c1 e−2x + c2 xe−2x From the initial conditions, y(x) = 2e−2x (2x + 1) When c = 5, the auxiliary equation for y

+ 5y + 4y = 0 is r 2 + 5r + 4 = (r + 1)(r + 4) = 0 Since there are two distinct real roots, the general solution for the differential equation is y(x) = c1 e−x + c2 e−4x

Confirming Pages

3.5 Application: Differential Equations

193

From the initial conditions, y(x) = 23 (4e−x − e−4x ) The graphs of the solutions are shown in Fig. 1. c=2 c=4

c=5

Figure 1

Exercise Set 3.5 In Exercises 1–4, find the general solution to the differential equation. a. Find two distinct solutions to the homogeneous differential equation. b. Show that the two solutions from part (a) are linearly independent. c. Write the general solution. 1. y

− 5y + 6y = 0 2. y

+ 3y + 2y = 0 3. y

+ 4y + 4y = 0 4. y

− 4y + 5y = 0 In Exercises 5 and 6, find the solution to the initial-value problem. 5. y

− 2y + y = 0 y(0) = 1 y (0) = 3 6. y

− 3y + 2y = 0 y(1) = 0 y (1) = 1 7. Consider the the nonhomogeneous differential equation given by where y

− 4y + 3y = g(x) g(x) = 3x 2 + x + 2

a. Find the general solution to the associated homogeneous differential equation for which g(x) = 0. This is called the complementary solution and is denoted by yc (x). b. Assume there exists a particular solution denoted yp (x) to the nonhomogeneous equation of the form yp (x) = ax 2 + bx + c Substitute yp (x) into the differential equation to find conditions on the coefficients a, b, and c. c. Verify that y(x) = yc (x) + yp (x) is a solution to the differential equation. 8. Consider the nonhomogeneous differential equation given by y

+ 4y + 3y = g(x) where g(x) = 3 sin 2x a. Find the general solution to the associated homogeneous differential equation for which g(x) = 0.

Confirming Pages

194

Chapter 3 Vector Spaces

b. Assume there exists a particular solution to the nonhomogeneous equation of the form yp (x) = A cos 2x + B sin 2x Substitute yp (x) into the differential equation to find conditions on the coefficients A and B. c. Verify that yc (x) + yp (x) is a solution to the differential equation. 9. Let w be the weight of an object attached to a spring, g the constant acceleration due to gravity of 32 ft/s2 , k the spring constant, and d the distance in feet that the spring is stretched by the

weight. Then the mass of the object is m = wg and k = wd . Suppose that a 2-lb weight stretches a spring by 6-in. Find the equation of the motion of the weight if the object is pulled down by 3-in and then released. Notice that this system is undamped ; that is, the damping coefficient is 0. 10. Suppose an 8-lb object is attached to a spring with a spring constant of 4 lb/ft and that the damping force on the system is twice the velocity. Find the equation of the motion if the object is pulled down 1-ft and given an upward velocity of 2 ft/s.

Review Exercises for Chapter 3 1. Determine for   1  −2     0  2

which values   0  1     −1  3

of k the vectors    2 0  3  0      4  1  k 4



5. Suppose that S = {v1 , v2 , v3 } is a basis for a vector space V .

  

a. Determine whether the set T = {v1 , v1 + v2 , v1 + v2 + v3 } is a basis for V .

form a basis for ⺢4 . 2. For which values of a, b, c, d, e, and f are the vectors       d b a  e   c   0  f 0 0 a basis for ⺢3 ? 3. Let

 S=

a−b b+c

a a−c

  a, b, c ∈ ⺢

a. Show is a subspace of M2×2 .  that S  5 3 in S? b. Is −2 3 c. Find a basis B for S. d. Give a 2 × 2 matrix that is not in S. 4. Let S = {p(x) = a + bx + cx 2 | a + b + c = 0}. a. Show that S is a subspace of P2 . b. Find a basis for S. Specify the dimension of S.

b. Determine whether the set W = {−v2 + v3 , 3v1 + 2v2 + v3 , v1 − v2 + 2v3 } is a basis for V . 6. Let S = {v1 , v2 , v3 }, where     1 2  −3   −1    v2 =  v1 =   1   1  1 1   4  −7   v3 =   3  3 a. Explain why the set S is not a basis for ⺢4 . b. Show that v3 is a linear combination of v1 and v2 . c. Find the dimension of the span of the set S. d. Find a basis B for ⺢4 that contains the vectors v1 and v2 .

Confirming Pages

195

3.5 Application: Differential Equations

e. Show that  1    2 T =   −1    1



1  0 ,   1 0



1  0 ,   1 −1

 1     2   ,    1    −1 

⺢4

is a basis for . f. Find the transition matrix from the ordered basis B to the ordered basis T . g. Use the matrix found in part (f) to find the transition matrix from the ordered basis T to the ordered basis B.  1  3   [v]B =   −2  5 

h. If

a. Give a basis for V and find its dimension. Let S be the set of all matrices of the form   a b c a and let T be the set of all matrices of the form   x y y z b. Show that S and T are subspaces of the vector space V . c. Give bases for S and T and specify their dimensions. d. Give a description of the matrices in S ∩ T . Find a basis for S ∩ T and give its dimension. 

9. Let u=

u1 u2



 and

v=

v1 v2



' ' u21 + u22 = 1 = v12 + v22

find the coordinates of v relative to the ordered basis T .

such that u·v=0

 −2  13   [v]T =   −5  −1 find the coordinates of v relative to the ordered basis B.

a. Show that B = {u, v} is a basis for ⺢2 .   x b. Find the coordinates of the vector w = y relative to the ordered basis B.



i. If

10. Let c be a fixed scalar and let p1 (x) = 1 p2 (x) = x + c p3 (x) = (x + c)2

7. Suppose span{v1 , . . . , vn } = V and c1 v1 + c2 v2 + · · · + cn vn = 0 with c1 = 0. Show that span{v2 , . . . , vn } = V . 8. Let V = M2×2 .

and

a. Show that B = {p1 (x), p2 (x), p3 (x)} is a basis for P2 . b. Find the coordinates of f (x) = a0 + a1 x + a2 x 2 relative to the ordered basis B.

Chapter 3: Chapter Test In Exercises 1–35, determine whether the statement is true or false. 1. If V = ⺢ and addition and scalar multiplication are defined as x ⊕ y = x + 2y then V is a vector space.

cx =x+c

2. The set

     0  2  1 S =  3 ,  1 ,  4    3 −1 1

is a basis for ⺢3 . 3. A line in ⺢3 is a subspace of dimension 1.

Confirming Pages

196

Chapter 3 Vector Spaces

4. The set



S=

2 0

   1 0 3 0 1 , , 2 0 2 1 1

is a basis for M2×2 . 5. The set

    −2  0  1  2 ,  1 ,  0    1 2 3

is a basis for ⺢3 if and only if   1 0 −2 0  = 0 det  2 1 3 2 1 

6. The set S=

x y

  y ≤ 0

is a subspace of ⺢2 . 7. The set S = {A ∈ M2×2 | det(A) = 0} is a subspace of M2×2 . 8. The set {2, 1 + x, 2 − 3x 2 , x 2 − x + 1} is a basis for P3 . 9. The set {x 3 − 2x 2 + 1, x 2 − 4, x 3 + 2x, 5x} is a basis for P3 . 10. The dimension of the subspace      s + 2t S =  t − s  s, t ∈ ⺢   s of ⺢3 is 2. 11. If and

   2 1 , S= 1 4      3 2 1 , , T = 5 1 4 

then span(S) = span(T ).

12. The set

     2a S =  a  a ∈ ⺢   0

is a subspace of ⺢3 of dimension 1. 13. If S = {v1 , v2 , v3 } and T = {v1 , v2 , v3 , v1 + v2 }, then span(S) = span(T ). 14. If S = {v1 , v2 , v3 }, T = {v1 , v2 , v3 , v1 + v2 }, and S is a basis, then T is not a basis. 15. If {v1 , v2 , v3 } is a basis for a vector space V and w1 = v1 + 2v2 + v3 , w2 = v1 + v2 + v3 , w3 = v1 − v2 − v3 , then W = {w1 , w2 , w3 } is also a basis for V . 16. If V is a vector space of dimension n and S is a set of vectors that span V , then the number of vectors in S is less than or equal to n. 17. If V is a vector space of dimension n, then any set of n − 1 vectors is linearly dependent. 18. If S and T are subspaces of a vector space V , then S ∪ T is a subspace of V . 19. If S = {v1 , . . . , vn } is a linearly independent set of vectors in ⺢n , then S is a basis. 20. If A is a 3 × 3 matrix and for every vector   a b =  b  the linear system Ax = b has a c solution, then the column vectors of A span ⺢3 . 21. If an n × n matrix is invertible, then the column vectors form a basis for ⺢n . 22. If a vector space has bases S and T and the number of elements of S is n, then the number of elements of T is also n. 23. In a vector space V , if span{v1 , v2 , . . . , vn } = V and w1 , w2 , . . . , wm are any elements of V , then span{v1 , v2 , . . . , vn , w1 , w2 , . . . , wm } = V .

Confirming Pages

3.5 Application: Differential Equations

24. If V is a vector space of dimension n and H is a subspace of dimension n, then H = V . 25. If B1 and B2 are bases for the vector space V , then the transition matrix from B1 to B2 is the inverse of the transition matrix from B2 to B1 . In Exercises 26–29, use the bases of ⺢2     0 1 , B1 = 2 −1

and

30.

31.

and   3 1 B2 = , −1 1   1 , relative to B1 , are 26. The coordinates of 0   1 . 1   1 relative to B2 are 27. The coordinates of 0   1 1 . 4 1 

28. The transition matrix from B1 to B2 is   1 −1 3 B2 [I ]B1 = 1 −1 2 29. The transition matrix from B2 to B1 is   1 3 B1 [I ]B2 = 1 1 In Exercises 30–35, use the bases of P3 , B1 = {1, x, x 2 , x 3 }

32.

33.

B2 = {x, x 2 , 1, x 3 }   1 [x 3 + 2x 2 − x]B1 =  2  −1   0  −1   [x 3 + 2x 2 − x]B1 =   2  1   0  −1   [x 3 + 2x 2 − x]B2 =   2  1   −1  2   [x 3 + 2x 2 − x]B2 =   0  1

34. [(1 + x)2 − 3(x 2 + x − 1) + x 3 ]B2   4  −1   =  −2  1 35. The transition matrix from B1 to  0 1 0  0 0 1 B2 [I ]B1 =   1 0 0 0 0 0

B2 is  0 0   0  1

197

Confirming Pages

Confirming Pages

CHAPTER

Linear Transformations CHAPTER OUTLINE

4.1 4.2 4.3 4.4 4.5 4.6

Linear Transformations 200 The Null Space and Range 214 Isomorphisms 226 Matrix Representation of a Linear Transformation 235 Similarity 249 Application: Computer Graphics 255

critical component in the design of an airplane is the airflow over the wing. Four forces that act on an aircraft, and need to be considered in its design, are lift, the force of gravity, thrust, and drag. Lift and drag are aerodynamic forces that are generated by the Yaw movement of the aircraft through the air. During takez off, thrust from the engines must overcome drag. Lift, created by the rush of air over the wing, must overcome the force of gravity before the airplane can fly. Mathematical models developed by aeronautical engineers simulate the behavior of an aircraft in flight. These models involve linear systems with millions of x y equations and variables. As we saw in Chap. 1, linear algebra provides systematic methods for solv- Roll Pitch ing these equations. Another use of linear algebra in the design process of an airplane is in modeling the movement of the aircraft through space. To check the feasibility of their designs, aeronautical engineers use computer graphics to visualize simulations of the aircraft in flight. Three control parameters which affect the position of an aircraft are pitch, roll, and yaw. The pitch measures the fore and aft tilt of an airplane, relative to the earth, while the roll measures the tilt from side to side. Together these give the attitude of the aircraft. Using the figure above, the pitch is a rotation about the y axis, while a roll is a rotation about the x axis. The yaw measures the rotation about the z axis, and when combined with the pitch, gives the heading. During a simulation, the attitude and heading of the aircraft can be changed by applying a transformation to its coordinates relative to a predefined center of equilibrium. As we shall see in this chapter, such a transformation can be represented by matrix multiplication. Specifically, if the angles of

A

199

Confirming Pages

Chapter 4 Linear Transformations

rotation for pitch, roll, and yaw are given by θ, ϕ, and ψ, respectively, then the matrix representations for these transformations are given by       cos ψ sin ψ 0 1 0 0 cos θ 0 − sin θ   0 cos ϕ − sin ϕ  and  − sin ψ cos ψ 0   0 1 0 0 0 1 0 sin ϕ cos ϕ sin θ 0 cos θ This type of transformation is a linear map between vector spaces, in this case from ⺢3 to ⺢3 . The generation and manipulation of computer graphics are one of many applications that require the linear transformations which are introduced in this chapter. Due to their wide applicability linear transformations on vector spaces are of general interest and are the subject of this chapter. As functions between vector spaces, they are special since they preserve the additive structure of linear combinations. That is, the image of a linear combination under a linear transformation is also a linear combination in the range. In this chapter we investigate the connection between linear transformations and matrices, showing that every linear transformation between finite dimensional vector spaces can be written as a matrix multiplication.

4.1

ß

200

Linear Transformations In mathematics every line of inquiry ultimately leads to a description of some set and functions on that set. One may metaphorically refer to elements of the set as nouns and functions that operate on elements as verbs. In linear algebra the sets are vector spaces, which we discussed in Chap. 3, and linear transformations on vector spaces are the functions. If V and W are vector spaces, then a mapping T from V to W is a function that assigns to each vector v in V a unique vector w in W . In this case we say that T maps V into W , and we write T: V −→ W . For each v in V the vector w = T (v) in W is the image of v under T.

EXAMPLE 1

Define a mapping T: ⺢2 −→ ⺢2 by     x+y x = T x−y y a. Find the image of the coordinate vectors e1 and e2 under the mapping T. b. Give a description of all vectors in ⺢2 that are mapped to the zero vector. c. Show that the mapping T satisfies T (u + v) = T (u) + T (v)

(preserves vector space addition)

and T (cv) = cT (v) (preserves scalar multiplication) for all vectors u and v in V and all scalars c in ⺢.

Confirming Pages

4.1 Linear Transformations

 0 and e2 = , we have a. Since e1 = 1     1 1+0 and T (e1 ) = = 1 1−0 

Solution

1 0



201



 T (e2 ) =

1 −1



b. To answer this question, we solve       0 x+y x = = T 0 x−y y This leads to the linear system



x+y x−y

=0 =0

where the unique solution is x = y = 0. Thus, the only vector that is mapped     0 0 . is the zero vector by T to 0 0 c. To show that the mapping T preserves vector space addition, let     u1 v1 u= and v= u2 v2 Then

   v1 u1 + T (u + v) = T u v 2  2 u 1 + v1 =T u 2 + v2   (u1 + v1 ) + (u2 + v2 ) = (u + v1 ) − (u2 + v2 )  1    u 1 + u2 v1 + v 2 = + u1 − u2 v 1 − v2     v1 u1 +T =T u2 v2 = T (u) + T (v)

We also have

  cu1 T (cu) = T cu2     u 1 + u2 cu1 + cu2 =c = cu1 − cu2 u 1 − u2 = cT (u)



A mapping T between vector spaces V and W that satisfies the two properties, as in Example 1, T (u + v) = T (u) + T (v)

and

T (cu) = cT (u)

Confirming Pages

202

Chapter 4 Linear Transformations

is called a linear transformation from V into W . Notice that the operations of addition and scalar multiplication on the left-hand side of each equation refer to operations in the vector space V, and on the right-hand side refer to operations in the vector space W . Definition 1 combines the two requirements for the linearity of T into one statement. DEFINITION 1

Linear Transformation Let V and W be vector spaces. The mapping T: V → W is called a linear transformation if and only if T (cu + v) = cT (u) + T (v) for every choice of u and v in V and scalars c in ⺢. In the case for which V = W , then T is called a linear operator. The mapping T defined in Example 1 is a linear operator on ⺢2 . In Example 2 we show how matrices can be used to define linear transformations.

EXAMPLE 2

Let A be an m × n matrix. Define a mapping T: ⺢n → ⺢m by T (x) = Ax a. Show that T is a linear transformation. b. Let A be the 2 × 3 matrix 
1 2 −1 A= −1 3 2 Find the images of

 1  1  1

 7  −1  5 



and

under the mapping T: ⺢3 → ⺢2 with T (x) = Ax. Solution

a. By Theorem 5 of Sec. 1.3, for all vectors u and v in ⺢n and all scalars c in ⺢, A(cu + v) = cAu + Av Therefore, T (cu + v) = cT (u) + T (v) b. Since T is defined by matrix multiplication, we have       1

1 2 1 2 −1       1 = 1 = T 4 −1 3 2 1 1

Confirming Pages

4.1 Linear Transformations

and

203

       7 7 0 1 2 −1  −1  = T  −1  = 0 −1 3 2 5 5 

Later in this chapter, in Sec. 4.4, we show that every linear transformation between finite dimensional vector spaces can be represented by a matrix. In Examples 1 and 2, we have discussed some of the algebraic properties of linear transformations. In Example 3 we consider the action of a linear transformation from a geometric perspective.

EXAMPLE 3

Define a linear transformation T: ⺢3 −→ ⺢2 by     x x T  y  = y z a. Discuss the action of T on of the equation    1 T  0  +  1 b. Find the image of the set

c. Find the image of the set

d. Describe the set

a vector in ⺢3 , and give a geometric interpretation      0 1 0 1  = T  0  + T  1  1 1 1

    1    S1 = t  2  t ∈ ⺢   1       x  S2 =  y  x, y ∈ ⺢   3       x  S3 =  0  x, z ∈ ⺢   z 

and find its image. Solution

a. The linear transformation T gives the projection, or shadow, of a vector in 3-space to its image in the xy plane. Let       1 0 1 v1 =  0  v2 =  1  and v3 = v1 + v2 =  1  1 1 2

Confirming Pages

204

Chapter 4 Linear Transformations

The images of these vectors are shown in Fig. 1. We see from the figure that       0 1 1 , is equal to the vector sum T (v1 ) + T (v2 ) = + T (v3 ) = 1 1 0 as desired. z

y T

v3 v1

v2

T (v2 )

T (v3 )

y

T (v1 )

x

x

Figure 1

 1 b. The set S1 is a line in 3-space with direction vector  2 . By the definition 1 of T we have     1  t ∈ ⺢ T (S1 ) = t 2  

which is a line in ⺢2 through the origin with slope 2. c. The set S2 is a plane in 3-space 3 units above and parallel to the xy plane. In this case,    x  x, y ∈ ⺢ T (S2 ) = y  Thus, the image of S2 is the entire xy plane, which from the description of T as a projection is the result we expect. d. The set S3 is the xz plane. Here we have    x  x ∈ ⺢ T (S3 ) = 0  which is just the x axis. Again, this is the expected result, given our description of T.

In Example 4 we use the derivative of a function to define a linear transformation between vector spaces of polynomials.

Confirming Pages

4.1 Linear Transformations

EXAMPLE 4

205

Define a mapping T: P3 −→ P2 by T (p(x)) = p (x) where p (x) is the derivative of p(x). a. Show that T is a linear transformation. b. Find the image of the polynomial p(x) = 3x 3 + 2x 2 − x + 2. c. Describe the polynomials in P3 that are mapped to the zero vector of P2 .

Solution

First observe that if p(x) is in P3 , then it has the form p(x) = ax 3 + bx 2 + cx + d so that

T (p(x)) = p (x) = 3ax 2 + 2bx + c

Since p (x) is in P2 , then T is a map from P3 into P2 . a. To show that T is linear, let p(x) and q(x) be polynomials of degree 3 or less, and let k be a scalar. Recall from calculus that the derivative of a sum is the sum of the derivatives, and that the derivative of a scalar times a function is the scalar times the derivative of the function. Consequently, d T (kp(x) + q(x)) = (kp(x) + q(x)) dx d d (kp(x)) + (q(x)) = dx dx = kp (x) + q  (x) = kT (p(x)) + T (q(x)) Therefore, the mapping T is a linear transformation. b. The image of the polynomial p(x) = 3x 3 + 2x 2 − x + 2 is T (p(x)) =

d (3x 3 + 2x 2 − x + 2) = 9x 2 + 4x − 1 dx

c. The only functions in P3 with derivative equal to zero are the constant polynomials p(x) = c, where c is a real number.

PROPOSITION 1

Let V and W be vector spaces, and let T: V → W be a linear transformation. Then T (0) = 0. Proof Since T (0) = T (0 + 0) and T is a linear transformation, we know that T (0) = T (0 + 0) = T (0) + T (0). Subtracting T (0) from both sides of the last equation gives T (0) = 0.

Confirming Pages

206

Chapter 4 Linear Transformations

EXAMPLE 5

Define a mapping T: ⺢2 −→ ⺢2 by   x   e x = T ey y Determine whether T is a linear transformation.

Solution



Since T (0) = T

0 0



 =

e0 e0



 =

1 1



by the contrapositive of Proposition 1, we know that T is not a linear transformation.

EXAMPLE 6

Define a mapping T: Mm×n −→ Mn×m by T (A) = At Show that the mapping is a linear transformation.

Solution

By Theorem 6 of Sec. 1.3, we have T (A + B) = (A + B)t = At + B t = T (A) + T (B) Also by this same theorem, T (cA) = (cA)t = cAt = cT (A) Thus, T is a linear transformation.

EXAMPLE 7

Coordinates Let V be a vector space with dim(V ) = n, and B = {v1 , v2 , . . . , vn } an ordered basis for V . Let T: V −→ ⺢n be the map that sends a vector v in V to its coordinate vector in ⺢n relative to B. That is, T (v) = [v]B It was shown in Sec. 3.4 that this map is well defined, that is, the coordinate vector of v relative to B is unique. Show that the map T is also a linear transformation.

Solution

Let u and v be vectors in V and let k be a scalar. Since B is a basis, there are unique sets of scalars c1 , . . . , cn and d1 , . . . , dn such that u = c1 v1 + · · · + cn vn

and

v = d1 v1 + · · · + dn vn

Confirming Pages

4.1 Linear Transformations

207

Applying T to the vector ku + v gives T (ku + v) = T ((kc1 + d1 )v1 + · · · + (kcn + dn )vn )       kc1 + d1 c1 d1  kc2 + d2   c   d   = k  .2  +  .2  .. =    ..   ..  . kcn + dn

cn

dn

= kT (u) + T (v) Therefore, we have shown that the mapping T is a linear transformation.

As mentioned earlier, when T: V −→ W is a linear transformation, then the structure of V is preserved when it is mapped into W . Specifically, the image of a linear combination of vectors, under a linear map, is equal to a linear combination of the image vectors with the same coefficients. To see this, let V and W be vector spaces and T: V → W be a linear transformation. Then by repeated application of Definition 1, we have T (c1 v1 + c2 v2 + · · · + cn vn ) = T (c1 v1 ) + · · · + T (cn vn ) = c1 T (v1 ) + c2 T (v2 ) + · · · + cn T (vn ) The fact that a linear transformation T between vector spaces V and W preserves linear combinations is useful in evaluating T when its action on the vectors of a basis for V is known. This is illustrated in Example 8.

EXAMPLE 8

Solution

Let T: ⺢3 → ⺢2 be a linear transformation, and let B be the standard basis for ⺢3 . If       1 −1 0 T (e2 ) = and T (e3 ) = T (e1 ) = 1 2 1 find T (v), where   1 v= 3  2 To find the image of the vector v, we first write the vector as a linear combination of the basis vectors. In this case v = e1 + 3e2 + 2e3

Confirming Pages

208

Chapter 4 Linear Transformations

Applying T to this linear combination and using the linearity properties of T, we have T (v) = T (e1 + 3e2 + 2e3 ) = T (e1 ) + 3T (e2 ) + 2T (e3 )       0 −1 1 +2 +3 = 1 2 1   −2 = 9

EXAMPLE 9

Let T: ⺢3 −→ ⺢3 be a linear operator   1 B=  1  1 If      1 1 1 T  2 T  1  =  1  3 1 1 find

Solution

and B a basis for ⺢3 given    1  1 ,  2 ,  1   2 3     −1  =  −2  T  −3

by

   2 1 1  =  2  4 2

 2 T  3  6 

Since B is a basis for ⺢3 , there are (unique) scalars c1 , c2 , and c3 such that         2 1 1 1 c1  1  + c 2  2  + c 3  1  =  3  6 1 3 2 Solving this equation, we obtain c1 = −1, c2 = 1, and c3 = 2. Hence,          1 1 1 2 T  3  = T −1  1  +  2  + 2  1  2 3 1 6 By the linearity of T, we have      1 2 T  3  = (−1)T  1  + T  1 6      −1 1 = −  1  +  −2  + 2  −3 1   2 = 1  4

   1 1 2  + 2T  1  2 3  2 2  4

Confirming Pages

209

4.1 Linear Transformations

Operations with Linear Transformations Linear transformations can be combined by using a natural addition and scalar multiplication to produce new linear transformations. For example, let S, T: ⺢2 → ⺢2 be defined by  S

x y



 =

x+y −x



 and

x y

T



 =

2x − y x + 3y



We then define (S + T )(v) = S(v) + T (v)  To illustrate this definition, let v =  (S + T )(v) = S(v) + T (v) =

2 −1

(cS)(v) = c(S(v))

and  ; then

2 + (−1) −2



 +

2(2) − (−1) 2 + 3(−1)



 =

6 −3



For scalar multiplication let c = 3. Then  (3T )(v) = 3T (v) = 3

5 −1



 =

15 −3



In Theorem 1 we show that these operations on linear transformations produce linear transformations.

THEOREM 1

Let V and W be vector spaces and let S, T: V → W be linear transformations. The function S + T defined by (S + T )(v) = S(v) + T (v) is a linear transformation from V into W . If c is any scalar, the function cS defined by (cS)(v) = cS(v) is a linear transformation from V into W . Proof Let u, v ∈ V and let d be any scalar. Then (S + T )(du + v) = S(du + v) + T (du + v) = S(du) + S(v) + T (du) + T (v) = dS(u) + S(v) + dT (u) + T (v) = d(S(u) + T (u)) + S(v) + T (v) = d(S + T )(u) + (S + T )(v)

Confirming Pages

210

Chapter 4 Linear Transformations

so that S + T is a linear transformation. Also (cS)(du + v) = c(S(du + v)) = c(S(du) + S(v)) = c(dS(u) + S(v)) = (cd)S(u) + cS(v) = d(cS)(u) + (cS)(v) so that cS is a linear transformation.

Using the sum of two linear transformations and the scalar product defined above, the set of all linear transformations between two given vector spaces is itself a vector space, denoted by £(U, V ). Verification of this is left to Exercise 45 at the end of this section. As we saw in Example 2, every m × n matrix A defines a linear map from ⺢n to m ⺢ . Also, if B is an n × p matrix, then B defines a linear map from ⺢p to ⺢n . The product matrix AB, which is an m × p matrix, then defines a linear transformation from ⺢p to ⺢m . As we shall see (in Sec. 4.4), this map corresponds to the composition of the maps defined by A and B. The desire for this correspondence is what motivated the definition of matrix multiplication given in Sec. 1.3.

THEOREM 2

Let U, V, and W be vector spaces. If T: V → U and S : U → W are linear transformations, then the composition map S ◦T: V → W , defined by (S ◦T )(v) = S(T (v)) is a linear transformation. (See Fig. 2.) Proof To show that S ◦T is a linear transformation, let v1 and v2 be vectors in V and c a scalar. Applying S ◦T to cv1 + v2 , we obtain

V

T

U

W S

v T (v) S ◦T (v)

Figure 2

(S ◦T )(cv1 + v2 ) = S(T (cv1 + v2 )) = S(cT (v1 ) + T (v2 )) = S(cT (v1 )) + S(T (v2 )) = cS(T (v1 )) + S(T (v2 )) = c(S ◦T )(v1 ) + (S ◦T )(v2 ) This shows that S ◦T is a linear transformation.

In the case of all linear operators on a vector space V , denoted by £(V , V ), the operations of addition and scalar multiplication make £(V , V ) a vector space. If, in addition, we define a product on £(V , V ) by ST (v) = (S ◦T )(v) then the product satisfies the necessary properties making £(V , V ) a linear algebra.

Revised Confirming Pages

4.1 Linear Transformations

211

Fact Summary Let V , W , and Z be vector spaces and S and T functions from V into W . 1. The function T is a linear transformation provided that for all u, v in V and all scalars c, T (cu + v) = cT (u) + T (v). 2. If A is an m × n matrix and T is defined by T (x) = Ax, then T is a linear transformation from ⺢n into ⺢m . 3. If T is a linear transformation, then the zero vector in V is mapped to the zero vector in W , that is, T (0) = 0. 4. If B = {v1 , v2 , . . . , vn } is an ordered basis for V and W = ⺢n , then the coordinate mapping T (v) = [v]B is a linear transformation. 5. If {v1 , v2 , . . . , vn } is a set of vectors in V and T is a linear transformation, then T (c1 v1 + c2 v2 + · · · + cn vn ) = c1 T (v1 ) + c2 T (v2 ) + · · · + cn T (vn ) for all scalars c1 , . . . , cn . 6. If S and T are linear transformations and c is a scalar, then S + T and cT are linear transformations. 7. If T: V −→ W is a linear transformation and L : W −→ Z is a linear transformation, then L◦T: V −→ Z is a linear transformation.

Exercise Set 4.1 In Exercises 1–6, determine whether the function T: ⺢2 → ⺢2 is a linear transformation.     y x = 1. T x y     x+y x = 2. T x−y+2 y     x x = 3. T y y2     2x − y x = 4. T x + 3y y     x x = 5. T 0 y   x+y   x 2 = x+y 6. T y 2

In Exercises 7–16, determine whether the function is a linear transformation between vector spaces. 7. T: ⺢ → ⺢, T (x) = x 2 8. T: ⺢ → ⺢, T (x) = −2x   x 2 = x2 + y2 9. T: ⺢ → ⺢, T y 10. T: ⺢3 → ⺢2 ,

   x x T   y  = y z

11. T: ⺢3 → ⺢3 , 



   x+y−z x  2xy T  y  =  x+z+1 z

Confirming Pages

212

Chapter 4 Linear Transformations

12. T: ⺢3 → ⺢3 , 



Let





cos x x  sin y T   y  =  sin x + sin z z

T (p(x)) = 2p (x) − 3p (x) + p(x) 14. T: Pn → Pn , T (p(x)) = p(x) + x 15. T: M2×2 → ⺢, T (A) = det(A) 16. T: M2×2 → M2×2 , T (A) = A + At In Exercises 17–20, a function T: V → W between vector spaces and two vectors u and v in V are given. a. Find T (u) and T (v). b. Is T (u + v) = T (u) + T (v)? c. Is T a linear transformation? 17. Define T: ⺢2 → ⺢2 by     −x x = T y y  u=

−2 3



 v=

2 −2



18. Define T: P2 → P2 by



T (ax + bx + cx + d) =

u = −x 3 + 2x 2 − x + 1

v = x2 − 1

2

Let

20. Define T: ⺢3 → ⺢2 by    2  x x −1     y = T y+z z

T (x) = 2 + x 2

T (x 2 ) = x − 3x 2 then find T (−3 + x − x 2 ).

v = −x − 1

−a − b + 1 c+d

3

22. If T: ⺢3 → ⺢3 is a linear operator and     1 1 T  0  =  −1  0 0     2 0 T  1  =  0  1 0     1 0 T  0  =  −1  1 1   1 then find T  7 . 5 T (1) = 1 + x

Let 19. Define T: P3 → ⺢2 by

 − 12 v =  −1  1

23. If T: P2 → P2 is a linear operator and

T (p(x)) = p (x) − 2p (x) + p(x) u = x 2 − 3x + 1



21. If T: ⺢2 → ⺢2 is a linear operator and     2 1 = T 3 0     −1 0 = T 4 1   1 . then find T −3

13. T: P3 → P3 ,

Let

 1 u= 2  3 



24. If T: M2×2 → M2×2 is a linear operator and   0 1 T (e11 ) = 0 0   1 0 T (e12 ) = 0 −1   1 1 T (e21 ) = 0 0   0 0 T (e22 ) = 2 0

Confirming Pages

4.1 Linear Transformations



then find T

2 1 −1 3

→ ⺢3 by  3 3  2

27. Suppose that T: P2 → P2 is a linear operator such that T (x ) = 2x − 1

a. Find

b. Is it possible to determine T (v) for all vectors v in ⺢3 ? Explain. 29. Define a linear operator T: ⺢2 → ⺢2 by     −x x = T −y y a. Find a matrix A such that T (v) = Av. b. Find T (e1 ) and T (e2 ). 30. Define a linear transformation T: ⺢2 → ⺢3 by     x − 2y x =  3x + y  T y 2y

a. Find T (e1 ), T (e2 ), and T (e3 ). b. Find T (3e1 − 4e2 + 6e3 ).

2

   8 1 T  3  =  −10  −3 0   2 T  −5  0 



25. Suppose that T: ⺢2 → ⺢2 is a linear operator such that     2 1 = T −1 1     2 −1 = T −1 0   3 ? If so, find Is it possible to determine T 7 it; and if not, explain why. 26. Define a linear operator T: ⺢3 T (u) = Au, where  1 2 A= 2 1 1 3

213

T (−3x) = x − 1 2

T (−x 2 + 3x) = 2x 2 − 2x + 1 a. Is it possible to determine T (2x 2 − 3x + 2)? If so, find it; and if not, explain why. b. Is it possible to determine T (3x 2 − 4x)? If so, find it; and if not, explain why. 28. Suppose that T: ⺢3 → ⺢3 is a linear operator such that     −1 1 T  0  =  2  3 0     2 1 T  1  =  −2  1 0

a. Find a matrix A such that T (v) = Av. b. Find T (e1 ) and T (e2 ). 31. Define T: ⺢3 → ⺢2 by     x x+y T  y  = x−y z Find all vectors that are mapped to 0. 32. Define T: ⺢3 → ⺢2 by     x x + 2y + z     y = T −x + 5y + z z Find all vectors that are mapped to 0. 33. Define T: ⺢3 → ⺢3 by     x − y + 2z x T  y  =  2x + 3y − z  − x + 2y − 2z z a. Find all vectors in ⺢3 that are mapped to the zero vector.

Confirming Pages

214

Chapter 4 Linear Transformations

 7 b. Let w =  −6 . Determine whether there is −9 

a vector v in ⺢3 such that T (v) = w. 34. Define T: P2 → P2 by T (p(x)) = p (x) − p(0) a. Find all vectors that are mapped to 0. b. Find two polynomials p(x) and q(x) such that T (p(x)) = T (q(x)) = 6x − 3. c. Is T a linear operator? 35. Suppose T1: V → ⺢ and T2: V → ⺢ are linear transformations. Define T: V → ⺢2 by   T1 (v) T (v) = T2 (v) Show that T is a linear transformation. 36. Define T: Mn×n → ⺢ by T (A) = tr(A). Show that T is a linear transformation. 37. Suppose that B is a fixed n × n matrix. Define T: Mn×n → Mn×n by T (A) = AB − BA. Show that T is a linear operator. 38. Define T: ⺢ → ⺢ by T (x) = mx + b. Determine when T is a linear operator. ß

4.2

39. Define T: C (0) [0, 1] → ⺢ by  1 f (x) dx T (f ) = 0

for each function f in C (0) [0, 1]. a. Show that T is a linear operator. b. Find T (2x 2 − x + 3). 40. Suppose that T: V → W is a linear transformation and T (u) = w. If T (v) = 0, then find T (u + v). 41. Suppose that T: ⺢n → ⺢m is a linear transformation and {v, w} is a linearly independent subset of ⺢n . If {T (v), T (w)} is linearly dependent, show that T (u) = 0 has a nontrivial solution. 42. Suppose that T: V → V is a linear operator and {v1 , . . . , vn } is linearly dependent. Show that {T (v1 ), . . . , T (vn )} is linearly dependent. 43. Let S = {v1 , v2 , v3 } be a linearly independent subset of ⺢3 . Find a linear operator T: ⺢3 → ⺢3 , such that {T (v1 ), T (v2 ), T (v3 )} is linearly dependent. 44. Suppose that T1: V → V and T2: V → V are linear operators and {v1 , . . . , vn } is a basis for V . If T1 (vi ) = T2 (vi ), for each i = 1, 2, . . . , n, show that T1 (v) = T2 (v) for all v in V . 45. Verify that £(U, V ) is a vector space.

The Null Space and Range In Sec. 3.2, we defined the null space of an m × n matrix to be the subspace of ⺢n of all vectors x with Ax = 0. We also defined the column space of A as the subspace of ⺢m of all linear combinations of the column vectors of A. In this section we extend these ideas to linear transformations.

DEFINITION 1

Null Space and Range Let V and W be vector spaces. For a linear transformation T: V −→ W the null space of T , denoted by N (T ), is defined by N (T ) = {v ∈ V | T (v) = 0} The range of T, denoted by R(T ), is defined by R(T ) = {T (v) | v ∈ V }

Confirming Pages

4.2 The Null Space and Range

215

The null space of a linear transformation is then the set of all vectors in V that are mapped to the zero vector, with the range being the set of all images of the mapping, as shown in Fig. 1.

T V

T U

N (T )

V

0

U

R(T )

Figure 1

In Theorem 3 we see that the null space and the range of a linear transformation are both subspaces.

THEOREM 3

Let V and W be vector spaces and T: V −→ W a linear transformation. 1. The null space of T is a subspace of V . 2. The range of T is a subspace of W . Proof (1) Let v1 and v2 be in N (T ), so that T (v1 ) = 0 and T (v2 ) = 0. If c is a scalar, then using the linearity of T , we have T (cv1 + v2 ) = cT (v1 ) + T (v2 ) = c0 + 0 = 0 Thus, cv1 + v2 is in N (T ), and by Theorem 4 of Sec. 3.2, N (T ) is a subspace of V . (2) Let w1 and w2 be in R(T ). Then there are vectors v1 and v2 in V such that T (v1 ) = w1 and T (v2 ) = w2 . Then for any scalar c, T (cv1 + v2 ) = cT (v1 ) + T (v2 ) = cw1 + w2 so that cw1 + w2 is in R(T ) and hence R(T ) is a subspace of W .

EXAMPLE 1

Define the linear transformation T:  a  b  T  c d

⺢4 −→ ⺢3 by    a+b   =  b − c   a+d

a. Find a basis for the null space of T and its dimension.

Revised Confirming Pages

216

Chapter 4 Linear Transformations

b. Give a description of the range of T. c. Find a basis for the range of T and its dimension. Solution

a. The null space of T is found by setting each component of the image vector equal to 0. This yields the linear system   =0 a + b b−c =0   a +d =0 This linear system has infinitely many solutions, given by    −t         t    t ∈ ⺢ S=  t        t    −1       1  N (T ) = span   1       1 A basis for N (T ) consists of the one vector   −1  1     1  1 Hence,

Consequently, dim(N (T )) = 1. b. Observe that any vector in the range can be written as         0 0 1 1 a  0  + b  1  + c  −1  + d  0  1 0 0 1 for some real numbers a, b, c, and d. Therefore,      0  0 1  1 R(T ) = span  0 ,  1 ,  −1 ,  0    1 0 0 1 c. Since the range is a subspace of ⺢3 , it has dimension less than or equal to 3. Consequently, the four vectors found to span the range in part (b) are linearly dependent and do not form a basis. To find a basis for R(T ), we use the trimming procedure given in Sec. 3.3 and reduce the matrix     1 0 0 1 1 1 0 0  0 1 0 −1   0 1 −1 0  to 0 0 1 −1 1 0 0 1

Confirming Pages

4.2 The Null Space and Range

Since the reduced matrix has pivots in range of T is    1 B =  0 ,   1

217

the first three columns, a basis for the   0  1 1 ,  −1   0 0

Therefore, dim(R(T )) = 3. Observe that B also spans ⺢3 , so that R(T ) = ⺢3 .

EXAMPLE 2

Define the linear transformation T: P4 −→ P3 , by T (p(x)) = p (x) Find the null space and range of T.

Solution

Recall that the derivative of a constant polynomial is 0. Since these are the only polynomials for which the derivative is 0, we know that N (T ) is the set of constant polynomials in P4 . We claim that the range of T is all of P3 . To see this, let q(x) = ax 3 + bx 2 + cx + d be an arbitrary element of P3 . A polynomial p(x) whose derivative is q(x) is found by using the antiderivative. That is, to find p(x), we integrate q(x) to obtain   a b c p(x) = q(x) dx = (ax 3 + bx 2 + cx + d) dx = x 4 + x 3 + x 2 + dx + e 4 3 2 which is an element of P4 , with p (x) = q(x). This shows that for every polynomial q(x) in P3 there is a polynomial p(x) in P4 such that T (p(x)) = q(x), giving that the range of T is all of P3 .

In Sec. 4.1, we saw that the image of an arbitrary vector v ∈ V can be computed if the image T (vi ) is known for each vector vi in a basis for V . This leads to Theorem 4. THEOREM 4

Let V and W be finite dimensional vector spaces and B = {v1 , v2 , . . . , vn } a basis for V . If T: V → W is a linear transformation, then R(T ) = span{T (v1 ), T (v2 ), . . . , T (vn )} Proof To show that the two sets are equal, we will show that each is a subset of the other. First, if w is in R(T ), then there is a vector v in V such that T (v) = w. Now, since B is a basis for V , there are scalars c1 , . . . , cn with v = c1 v1 + c2 v2 + · · · + cn vn so that T (v) = T (c1 v1 + c2 v2 + · · · + cn vn )

Confirming Pages

218

Chapter 4 Linear Transformations

From the linearity of T, we have w = T (v) = c1 T (v1 ) + c2 T (v2 ) + · · · + cn T (vn ) As w is a linear combination of T (v1 ), T (v2 ), . . . , T (vn ), then w ∈ span{T (v1 ), T (v2 ), . . . , T (vn )}. Since this is true for all w in R(T ), then R(T ) ⊂ span{T (v1 ), T (v2 ), . . . , T (vn )} On the other hand, suppose that w ∈ span{T (v1 ), T (v2 ), . . . , T (vn )}. Then there are scalars c1 , . . . , cn with w = c1 T (v1 ) + c2 T (v2 ) + · · · + cn T (vn ) = T (c1 v1 + c2 v2 + · · · + cn vn ) Therefore, w is the image under T of c1 v1 + c2 v2 + · · · + cn vn , which is an element of V . Therefore, span{T (v1 ), T (v2 ), . . . , T (vn )} ⊂ R(T ).

EXAMPLE 3

Let T: ⺢3 → ⺢3 be a linear operator and B = {v1 , v2 , v3 } a basis for ⺢3 . Suppose that       1 1 2 T (v2 ) =  0  T (v3 ) =  1  T (v1 ) =  1  0 −1 −1   1 a. Is  2  in R(T )? 1 b. Find a basis for R(T ). c. Find the null space N (T ).

Solution

 1 a. From Theorem 4, the vector w =  2  is in R(T ) if there are scalars c1 , c2 , 1 and c3 such that   1 c1 T (v1 ) + c2 T (v2 ) + c3 T (v3 ) =  2  1 that is,



       1 1 1 2 c 1  1  + c2  0  + c2  1  =  2  1 0 −1 −1 The set of solutions to this linear system is given by S = {(2 − t, −1 − t, t) | 

t ∈ ⺢}. In particular, if t = 0, then a solution is c1 = 2, c2 = −1, and c3 = 0. Thus, w ∈ R(T ).

Confirming Pages

4.2 The Null Space and Range

b. To find a basis for R(T ), we row-reduce the matrix    1 0 1 1 2  0 1  1 0 1  to obtain 0 0 0 −1 −1

219

 1 1  0

Since the leading 1s are in columns 1 and 2, a basis for R(T ) is given by    1   1 R(T ) = span  1 ,  0    −1 0 Observe that since the range is spanned by two linearly independent vectors, R(T ) is a plane in ⺢3 , as shown in Fig. 2. c. Since B is a basis for ⺢3 , the null space is the set of all vectors c1 v1 + c2 v2 + c3 v3 such that   0 c1 T (v1 ) + c2 T (v2 ) + c3 T (v3 ) =  0  0 By using the reduced matrix

 1 0 1  0 1 1  0 0 0 

from part (b), the null space consists of all vectors such that c1 = −c3 , c2 = −c3 , and c3 is any real number. That is, N (T ) = span {−v1 − v2 + v3 } which is a line in ⺢ . See Fig. 2. 3

z

T

z

N(T) x

y

R(T)

y

x

Figure 2

Notice that in Example 3 we have dim(⺢3 ) = dim(R(T )) + dim(N (T )) In Theorem 5 we establish this fundamental result for all linear transformations between finite dimensional vector spaces.

Confirming Pages

220

Chapter 4 Linear Transformations

THEOREM 5

Let V and W be finite dimensional vector spaces. If T: V −→ W is a linear transformation, then dim(V ) = dim(R(T )) + dim(N (T )) Proof Suppose that dim(V ) = n. To establish the result, we consider three cases. First, suppose that dim(N (T )) = dim(V ) = n. In this case, the image of every vector in V is the zero vector (in W ), so that R(T ) = {0}. Since the dimension of the vector space containing only the zero element is 0, the result holds. Now suppose 1 ≤ r = dim(N (T )) < n. Let {v1 , v2 , . . . , vr } be a basis for N (T ). By Corollary 1 of Sec. 3.3, there are n − r vectors {vr+1 , vr+2 , . . . , vn }, such that {v1 , v2 , . . . , vr , vr+1 , . . . , vn } is a basis for V . We claim that S = {T (vr+1 ), T (vr+2 ), . . . , T (vn )} is a basis for R(T ). By Theorem 4, we have R(T ) = span{T (v1 ), T (v2 ), . . . , T (vr ), T (vr+1 ), . . . , T (vn )} Since T (v1 ) = T (v2 ) = · · · = T (vr ) = 0, each vector in R(T ) is a linear combination of T (vr+1 ), . . . , T (vn ) and hence R(T ) = span(S). To show that S is linearly independent, we consider the equation cr+1 T (vr+1 ) + cr+2 T (vr+2 ) + · · · + cn T (vn ) = 0 We need to show that cr+1 = cr+2 = · · · = cn = 0. Since T is linear, the previous equation can be written as T (cr+1 vr+1 + cr+2 vr+2 + · · · + cn vn ) = 0 From this last equation, we have cr+1 vr+1 + cr+2 vr+2 + · · · + cn vn is in N (T ). However, since {v1 , v2 , . . . , vr } is a basis for N (T ), there are scalars c1 , c2 , . . . , cr such that cr+1 vr+1 + cr+2 vr+2 + · · · + cn vn = c1 v1 + c2 v2 + · · · + cr vr that is, −c1 v1 − c2 v2 − · · · − cr vr + cr+1 vr+1 + cr+2 vr+2 + · · · + cn vn = 0 Now, since {v1 , v2 , . . . , vr , vr+1 , . . . , vn } is a basis for V and hence linearly independent, the coefficients of the last equation must all be 0, that is, c1 = c2 = · · · = cr = cr+1 = · · · = cn = 0. In particular, cr+1 = cr+2 = · · · = cn = 0. Hence, the n − r vectors T (vr+1 ), . . . , T (vn ) are a basis for R(T ). Consequently, n = dim(V ) = (n − r) + r = dim(R(T )) + dim(N (T )) Finally, suppose that N (T ) = {0}, so that dim(N (T )) = 0. If {v1 , . . . , vn } is a basis for V , then by Theorem 4 we have R(T ) = span{T (v1 ), . . . , T (vn )} A similar argument to the one above shows that {T (v1 ), . . . , T (vn )} is linearly independent. Thus, dim(R(T )) = n = dim(V ), and the result also holds in this case.

Confirming Pages

4.2 The Null Space and Range

EXAMPLE 4

221

Define a linear transformation T: P4 −→ P2 by T (p(x)) = p (x) Find the dimension of the range of T, and give a description of the range.

Solution

Let B = {1, x, x 2 , x 3 , x 4 } be the standard basis for P4 . Since p(x) is in N (T ) if and only if its degree is 0 or 1, the null space is the subspace of P4 consisting of polynomials with degree 1 or less. Hence, {1, x} is a basis for N (T ), and dim(N (T )) = 2. Since dim(P4 ) = 5, by Theorem 5 we have 2 + dim(R(T )) = 5

so

dim(R(T )) = 3

Then as in the proof of Theorem 5, we have   T (x 2 ), T (x 3 ), T (x 4 ) = 2, 6x, 12x 2 is a basis for R(T ). Observe that R(T ) is just the subspace P2 of P4 .

Matrices In Sec. 3.2 we defined the column space of a matrix A, denoted by col(A), as the span of its column vectors. We also defined the null space of the m × n matrix A as the set of all vectors x in ⺢n such that Ax = 0. We further examine these notions here in the context of linear transformations. In particular, let A be an m × n matrix and let T: ⺢n −→ ⺢m be the linear transformation defined by T (v) = Av This last equation can be written in vector form as T (v) = v1 A1 + v2 A2 + · · · + vn An where Ai are the column vectors of A, and vi are the components of v for 1 ≤ i ≤ n. In this way we see that the range of T, which is a subspace of ⺢m , is equal to the column space of A, that is, R(T ) = col(A) The dimension of the column space of A is called the column rank of A. We also have N (T ) = {v ∈ ⺢n | Av = 0} = N (A) The dimension of N (A) is called the nullity of A. Applying Theorem 5, we have column rank(A) + nullity(A) = n Another subspace of ⺢n associated with the matrix A is the row space of A, denoted by row(A), and is the span of the row vectors of A. Since the transpose operation maps the row vectors of A to the column vectors of At , the row space of A is the same as the column space of At , that is, row(A) = col(At )

Confirming Pages

222

Chapter 4 Linear Transformations

By using the algorithm for finding a basis, given in Sec. 3.3, a basis for col(A) can be found by row reduction. In particular, the columns with the leading 1s in the rowreduced form of A correspond to the column vectors of A needed for a basis of col(A). Hence, the column rank of A is equal to the number of leading 1s in the row-reduced form of A. On the other hand, row-reducing A eliminates row vectors that are linear combinations of the others, so that the nonzero row vectors of the reduced form of A form a basis for row(A). Hence, the row rank is equal to the number of leading 1s in the reduced form of A. We have now established Theorem 6.

THEOREM 6

The row rank and the column rank of a matrix A are equal.

We can now define the rank of a matrix A as dim(row(A)) or dim(col(A)). Again by Theorem 5, we have rank(A) + nullity(A) = n

Linear Systems When the nullity of a matrix A is known, the above formula can sometimes be used to determine whether the linear system Ax = b is consistent. For example, suppose that a linear system consists of 20 equations each with 22 variables. Further suppose that a basis for the null space of the 20 × 22 coefficient matrix consists of two vectors. That is, every solution to the homogeneous linear system Ax = 0 is a linear combination of two linearly independent vectors in ⺢22 . Then nullity(A) = 2, so that dim(col(A)) = rank(A) = 22 − nullity(A) = 20 But the only subspace of ⺢20 with dimension 20 is ⺢20 itself. Hence, col(A) = ⺢20 , and consequently every vector b in ⺢20 is a linear combination of the columns of A. That is, the linear system Ax = b is consistent for every vector b in ⺢20 . In general, if A is an m × n matrix, nullity(A) = r, and dim(col) = n − r = m, then the linear system Ax = b is consistent for every vector b in ⺢m . We now add several more items to the list of equivalences given in Theorem 9 of Sec. 2.3, connecting solutions of the linear system Ax = b and properties of the coefficient matrix A.

THEOREM 7

Let A be an n × n matrix. Then the following statements are equivalent. 1. 2. 3. 4.

The The The The

matrix A is invertible. linear system Ax = b has a unique solution for every vector b. homogeneous linear system Ax = 0 has only the trivial solution. matrix A is row equivalent to the identity matrix.

Confirming Pages

4.2 The Null Space and Range

5. 6. 7. 8. 9. 10. 11. 12. 13.

223

The determinant of the matrix A is nonzero. The column vectors of A are linearly independent. The column vectors of A span ⺢n . The column vectors of A are a basis for ⺢n . rank(A) = n R(A) = col(A) = ⺢n N (A) = {0} row(A) = ⺢n The number of pivot columns of the reduced row echelon form of A is n.

Fact Summary Let V and W be vector spaces and T a linear transformation from V into W . 1. The null space N (T ) is a subspace of V, and the the range R(T ) is a subspace of W . 2. If B = {v1 , . . . , vn } is a basis for V , then R(T ) = span{T (v1 ), . . . , T(vn )} 3. If V and W are finite dimensional vector spaces, then dim(V ) = dim(R(T )) + dim(N (T )) 4. If A is an m × n matrix, then rank(A) + nullity(A) = n 5. If A is an m × n matrix, then the rank of A is the number of leading 1s in the row-reduced form of A. 6. If A is an n × n invertible matrix, in addition to Theorem 9 of Sec. 2.3, we know that rank(A) = n, R(A) = col(A) = ⺢n , N (A) = {0}, and the number of leading 1s in the row echelon form of A is n.

Exercise Set 4.2 In Exercises 1–4, define a linear operator T: ⺢2 → ⺢2 by     x − 2y x = T −2x + 4y y

Determine whether the vector v is in N (T ).   0 1. v = 0

Confirming Pages

224

Chapter 4 Linear Transformations

 2. v =  3. v =  4. v =

2 1 1 3 1 2 1 4

  

In Exercises 5–8, define a linear operator T: P3 → P3 by T (p(x)) = xp (x) Determine whether the polynomial p(x) is in N (T ). 5. p(x) = x 2 − 3x + 1 6. p(x) = 5x + 2 7. p(x) = 1 − x 2 8. p(x) = 3 In Exercises 9–12, define a linear operator T: ⺢3 → ⺢3 by     x+ 2z x T  y  =  2x + y + 3z  x − y + 3z z Determine whether the vector v is in R(T ).   1 9. v =  3  0   2 10. v =  3  4   −1 11. v =  1  −2   −2 12. v =  −5  −1 In Exercises 13–16, define a linear transformation T: M2×2 → M3×2 by     a+ c b+ d a b =  −a + 2c −b + 2d  T c d 2a 2b

Determine whether the matrix A is in R(T ).   −1 −1 13. A =  −5 −2  2 0   1 2 14. A =  3 −3  −2 2   1 0 15. A =  2 1  4 0   4 1 5  16. A =  −1 6 −2 In Exercises 17–24, find a basis for the null space of the linear transformation T. 17. T: ⺢2 → ⺢2 ,



T 18. T: ⺢2 → ⺢2 ,



T

x y

x y



 =



 =

3x + y y

−x + y x−y





19. T: ⺢3 → ⺢3 , 

   x + 2z x T  y  =  2x + y + 3z  x − y + 3z z

20. T: ⺢3 → ⺢3 , 

   −2x + 2y + 2z x T  y  =  3x + 5y + z  2y + z z

21. T: ⺢3 → ⺢3 , 

   x − 2y − z x T  y  =  −x + 2y + z  2x − 4y − 2z z

22. T: ⺢4 → ⺢3 ,  x  y  T  z w



  x+y−z+w   =  2x + y + 4z + w   3x + y + 9z

Confirming Pages

4.2 The Null Space and Range

23. T: P2 → ⺢,

a. Determine whether

 −6 w= 5  0

T (p(x)) = p(0)

24. T: P2 → P2 , T (p(x)) = p (x) In Exercises 25–30, find a basis for the range of the linear transformation T . 25. T: ⺢3 → ⺢3 ,





1 1 2 T (v) =  0 1 −1  v 2 0 1 26. T: ⺢5 → ⺢3 ,

 1 −2 −3 1 5 1 0 4 v T (v) =  3 −1 1 1 3 1 2

27. T: ⺢3 → ⺢3 ,











x x T   y  =  y  0 z 28. T: ⺢3 → ⺢3 , 

   x − y + 3z x T   y  =  x + y + z  −x + 3y − 5z z

29. T: P3 → P3 , T (p(x)) = p (x) + p (x) + p(0) 30. T: P2 → P2 , T (ax 2 + bx + c) = (a + b)x 2 + cx + (a + b) ⺢3

⺢3

225

31. Let T: → be a linear operator and B = {v1 , v2 , v3 } a basis for ⺢3 . Suppose     −2 0 T (v2 ) =  1  T (v1 ) =  1  1 −1   −2 T (v3 ) =  2  0



is in the range of T. b. Find a basis for R(T ). c. Find dim(N (T )). 32. Let T: ⺢3 → ⺢3 be a linear operator and B = {v1 , v2 , v3 } a basis for ⺢3 . Suppose     −1 0 T (v2 ) =  5  T (v1 ) =  2  1 0   −1 T (v3 ) =  −1  2 a. Determine whether

 −2 w= 1  2 

is in the range of T. b. Find a basis for R(T ). c. Find dim(N (T )). 33. Let T: P2 → P2 be defined by T (ax 2 + bx + c) = ax 2 + (a − 2b)x + b a. Determine whether p(x) = 2x 2 − 4x + 6 is in the range of T. b. Find a basis for R(T ). 34. Let T: P2 → P2 be defined by T (ax 2 + bx + c) = cx 2 + bx − b a. Determine whether p(x) = x 2 − x − 2 is in the range of T. b. Find a basis for R(T ). 35. Find a linear transformation T: ⺢3 → ⺢2 such that R(T ) = ⺢2 . 36. Find a linear operator T: ⺢2 → ⺢2 such that R(T ) = N (T ).

Confirming Pages

226

Chapter 4 Linear Transformations

37. Define a linear operator T: Pn → Pn by 

T (p(x)) = p (x) a. Describe the range of T. b. Find dim(R(T )). c. Find dim(N (T )). 38. Define a linear operator T: Pn → Pn by dk (p(x)) dx k where 1 ≤ k ≤ n. Show dim(N (T )) = k. T (p(x)) =

39. Suppose T: ⺢4 → ⺢6 is a linear transformation. a. If dim(N (T )) = 2, then find dim(R(T )). b. If dim(R(T )) = 3, then find dim(N (T )). 40. Show that if T: V → V is a linear operator such that R(T ) = N (T ), then dim(V ) is even. 

41. Let A=

1 0

Define T: M2×2 → M2×2

0 −1 by



T (B) = AB − BA ß

4.3

Find a basis for the null space of T. 42. Define T: Mn×n → Mn×n by T (A) = At . Show that R(T ) = Mn×n . 43. Define T: Mn×n → Mn×n by T (A) = A + At . a. Find R(T ). b. Find N (T ). 44. Define T: Mn×n → Mn×n by T (A) = A − At . a. Find R(T ). b. Find N (T ). 45. Let A be a fixed n × n matrix, and define T: Mn×n → Mn×n by T (B) = AB. When does R(T ) = Mn×n ? 46. Let A be a fixed n × n diagonal matrix, and define T: ⺢n → ⺢n by T (v) = Av. a. Show dim(R(T )) is the number of nonzero entries on the diagonal of A. b. Find dim(N (T )). How is it related to the diagonal terms of the matrix A?

Isomorphisms Many of the vector spaces that we have discussed are, from an algebraic perspective, the same. In this section we show how an isomorphism, which is a special kind of linear transformation, can be used to establish a correspondence between two vector spaces. Essential to this discussion are the concepts of one-to-one and onto mappings. For a more detailed description see App. A, Sec. A.2.

DEFINITION 1

One-to-One and Onto mapping.

Let V and W be vector spaces and T: V → W a

1. The mapping T is called one-to-one (or injective) if u = v implies that T (u) = T (v). That is, distinct elements of V must have distinct images in W . 2. The mapping T is called onto (or surjective) if T (V ) = W . That is, the range of T is W . A mapping is called bijective if it is both injective and surjective. When we are trying to show that a mapping is one-to-one, a useful equivalent formulation comes from the contrapositive statement. That is, T is one-to-one if T (u) = T (v) implies that u = v. To show that a mapping is onto, we must show that if w is an arbitrary element of W , then there is some element v ∈ V with T (v) = w.

Confirming Pages

4.3 Isomorphisms

EXAMPLE 1

227

Let T: ⺢2 → ⺢2 be the mapping defined by T (v) = Av, with 
1 1 A= −1 0 Show that T is one-to-one and onto.

Solution

First, to show that T is one-to-one, let 
u1 u= u2 Then


T (u) =

and


T (v) =

Now if T (u) = T (v), then




1 1 −1 0 1 1 −1 0 u 1 + u2 −u1


v=

and 


u1 u2






v1 v2


=




=




=

v1 + v2 −v1

v1 v2



u1 + u2 −u1 v1 + v2 −v1

 



Equating the second components gives u1 = v1 , and using this when equating the first components gives u2 = v2 . Thus, u = v, establishing that the mapping is oneto-one. 
a be an arbitrary vector in ⺢2 . We Next, to show that T is onto, let w = b 
v1 in ⺢2 such that must show that there is a vector v = v2  


v1 a 1 1 = T (v) = b −1 0 v2 Applying the inverse of A to both sides of this equation, we have 




v1 a v1 0 −1 −b that is = = b 1 1 a+b v2 v2 
1 ; then using the above formula for the Thus, T is onto. For example, let w = 2  

−2 −2 . As verification, observe that = preimage, we have v = 3 1+2 
 

1 −2 1 1 = T (v) = 2 3 −1 0 An alternative argument is to observe that the column vectors of A are linearly independent and hence are a basis for ⺢2 . Therefore, the range of T being the column space of A is all of ⺢2 .

Confirming Pages

228

Chapter 4 Linear Transformations

Theorem 8 gives a useful way to determine whether a linear transforation is one-to-one. THEOREM 8

The linear transformation T: V −→ W is one-to-one if and only if the null space of T consists of only the zero vector of V . Proof First suppose that T is one-to-one. We claim that N (T ) = {0}. To show this, let v be any vector in the null space of T, so that T (v) = 0. We also have, by Proposition 1 of Sec. 4.1, T (0) = 0. Since T is one-to-one, then v = 0, so only the zero vector is mapped to the zero vector. Now suppose that N (T ) = {0} and T (u) = T (v) Subtracting T (v) from both sides of the last equation and using the linearity of T, we obtain T (u) − T (v) = 0 so that T (u − v) = 0 Thus, u − v ∈ N (T ). Since the null space consists of only the zero vector, u − v = 0, that is, u = v.

EXAMPLE 2

Solution

Define a linear operator T: ⺢2 −→ ⺢2 by     2x − 3y x = T 5x + 2y y Use Theorem 8 to show that T is one-to-one.   x is in the null space of T if and only if The vector y  2x − 3y = 0 5x + 2y = 0 This linear system has the unique solution x = y = 0. Thus, N (T ) = {0} and hence by Theorem 8, T is one-to-one.

The mapping of Example 2 can alternatively be defined by using the matrix   2 −3 A= 5 2 so that T (x) = Ax. Since det(A) = 0, then A is invertible. This allows us to show that the map is also onto. Indeed, if b is any vector in ⺢2 , then x = A−1 b is the vector in the domain that is the preimage of b, so that T is onto. In Theorem 4 of Sec. 4.2, we showed that if T: V −→ W is a linear transformation between vector spaces, and B = {v1 , . . . , vn } is a basis for V , then R(T ) = span{T (v1 ), . . . , T (vn )}. If, in addition, the transformation is one-to-one, then the spanning vectors are also a basis for the range, as given in Theorem 9.

Confirming Pages

4.3 Isomorphisms

THEOREM 9

229

Suppose that T: V → W is a linear transformation and B = {v1 , . . . , vn } is a basis for V . If T is one-to-one, then {T (v1 ), . . . , T (vn )} is a basis for R(T ). Proof By Theorem 4 of Sec. 4.2, we know that the span{T (v1 ), . . . , T (vn )} = R(T ), so it suffices to show that {T (v1 ), . . . , T (vn )} is linearly independent. To do so, we consider the equation c1 T (v1 ) + c2 T (v2 ) + · · · + cn T (vn ) = 0 which is equivalent to T (c1 v1 + c2 v2 + · · · + cn vn ) = 0 Since T is one-to-one, the null space consists of only the zero vector of V , so that c1 v1 + c2 v2 + · · · + cn vn = 0 Finally, since B is a basis for V , it is linearly independent; hence c1 = c 2 = · · · = c n = 0 Therefore, {T (v1 ), . . . , T (vn )} is linearly independent. We note that in Theorem 9 if T is also onto, then {T (v1 ), . . . , T (vn )} is a basis for W . We are now ready to define an isomorphism on vector spaces.

DEFINITION 2

Isomorphism Let V and W be vector spaces. A linear transformation T: V −→ W that is both one-to-one and onto is called an isomorphism. In this case the vector spaces V and W are said to be isomorphic.

Proposition 2 builds on the remarks following Example 2 and gives a useful characterization of linear transformations defined by a matrix that are isomorphisms.

PROPOSITION 2

Let A be an n × n matrix and T: ⺢n −→ ⺢n be the mapping defined by T (x) = Ax. Then T is an isomorphism if and only if A is invertible. Proof Let A be invertible and b be any vector in ⺢n . Then x = A−1 b is the preimage of b. Thus, the mapping T is onto. To show that T is one-to-one, observe that by Theorem 10 of Sec. 1.5 the equation Ax = 0 has only the solution x = A−1 0 = 0. Thus, by Theorem 8, the mapping T is one-to-one and hence is an isomorphism from ⺢n onto ⺢n . Conversely, suppose that T is an isomorphism. Then T: ⺢n −→ ⺢n is onto, with the column space of A being ⺢n . Hence, by Theorem 7 of Sec. 4.2 the matrix A is invertible.

Revised Confirming Pages

230

Chapter 4 Linear Transformations

Theorem 10 is of fundamental importance to the study of finite dimensional vector spaces and is the main result of the section.

THEOREM 10

If V is a vector space with dim(V ) = n, then V and ⺢n are isomorphic. Proof Let B = {v1 , . . . , vn } be an ordered basis for V . Let T: V −→ ⺢n be the coordinate transformation defined by T (v) = [v]B , first introduced in Example 7 of Sec. 4.1. We claim that T is an isomorphism. First, to show that T is one-to-one, suppose that T (v) = 0. Since B is a basis, there are unique scalars c1 , . . . , cn such that v = c1 v1 + · · · + cn vn Thus,



   c1 0  c2   0     T (v) = [v]B =   ...  =  ...  0 cn so that c1 = c2 = · · · = cn = 0 and v = 0. Therefore, N (T ) = {0}, and by Theorem 8, T is one-to-one. Now, to show that T is onto, let   k1  k2   w=  ...  kn ⺢n .

be a vector in Define v in V by v = k1 v1 + · · · + kn vn . Observe that T (v) = w and hence T is onto. Therefore, the linear transformation T is an isomorphism, and V and ⺢n are isomorphic vector spaces.

So far in our experience we have seen that dim(P2 ) = 3 and dim(S2×2 ) = 3, where S2×2 is the vector space of 2 × 2 symmetric matrices. Consequently, by Theorem 10, the vector spaces P2 and S2×2 are both isomorphic to ⺢3 , where the isomorphism is the coordinate map between the standard bases. Next we show that in fact all vector spaces of dimension n are isomorphic to one another. To do so, we first require the notion of the inverse of a linear transformation.

DEFINITION 3

Inverse of a Linear Transformation Let V and W be vector spaces and T: V −→ W a one-to-one linear transformation. The mapping T −1: R(T ) −→ V , defined by T −1 (w) = v if and only if T (v) = w is called the inverse of T. If T is onto, then T −1 is defined on all of W .

Confirming Pages

4.3 Isomorphisms

231

By Theorem 4 of Sec. A.2, if T is one-to-one, then the inverse map is well defined. Indeed, let u and v be vectors in V such that T −1 (w) = u and T −1 (w) = v. Applying T gives T (T −1 (w)) = T (u) and T (T −1 (w)) = T (v), so that T (u) = T (v). Since T is one-to-one, we have u = v. The inverse map of a one-to-one linear transformation is also a linear transformation, as we now show.

PROPOSITION 3

Let V and W be vector spaces and T: V −→ W a one-to-one linear transformation. Then the mapping T −1: R(T ) −→ V is also a linear transformation. Proof Let w1 and w2 be vectors in R(T ), and let c be a scalar. Also let v1 and v2 be vectors in V with T −1 (w1 ) = v1 and T −1 (w2 ) = v2 . Since T is linear, T (cv1 + v2 ) = cT (v1 ) + T (v2 ) = cw1 + w2 Hence, T −1 (cw1 + w2 ) = cv1 + v2 = cT −1 (w1 ) + T −1 (w2 ) Consequently, T −1 is a linear transformation.

Proposition 4 shows that the inverse transformation of an isomorphism defined by matrix multiplication can be written using the inverse of the matrix. The proof is left as an exercise.

PROPOSITION 4

EXAMPLE 3

Solution

Let A be an n × n invertible matrix and T: ⺢n −→ ⺢n the linear transformation defined by T (x) = Ax. Then T −1 (x) = A−1 x.

Let T: ⺢2 → ⺢2 be the mapping of Example 1 with T (v) = Av, where   1 1 A= −1 0 Verify that the inverse map T −1: ⺢2 −→ ⺢2 is given by T −1 (w) = A−1 w, where   0 −1 −1 A = 1 1   v1 Let v = be a vector in ⺢2 . Then v2   v1 + v 2 w = T (v) = −v1

Confirming Pages

232

Chapter 4 Linear Transformations

Applying A−1 to w, we obtain      v 1 + v2 0 −1 v1 = = T −1 (w) 1 1 −v1 v2

THEOREM 11

If V and W are vector spaces of dimension n, then V and W are isomorphic. Proof By Theorem 10, there are isomorphisms T1: V −→ ⺢n and T2: W −→ ⺢n , as shown in Fig. 1. Let φ = T2−1 ◦T1: V −→ W . To show that φ is linear, we first note that T2−1 is linear by Proposition 3. Next by Theorem 2 of Sec. 4.1, the composition T2−1 ◦T1 is linear. Finally, by Theorem 4 of Sec. A.2, the mapping φ is one-to-one and onto and is therefore a vector space isomorphism.

V

T1

⺢n

φ T2 W φ = T2−1 ◦T1: V −→ W Figure 1

EXAMPLE 4 Solution

Find an explicit isomorphism from P2 onto the vector space of 2 × 2 symmetric matrices S2×2 . To use the method given in the proof of Theorem 11, first let       1 0 0 1 0 0 , , and B2 = B1 = {1, x, x 2 } 0 0 1 0 0 1 be ordered bases for P2 and S2×2 , respectively. Let T1 and T2 be the coordinate maps from P2 and S2×2 into ⺢3 . Then      c a b = and T2 T1 (ax 2 + bx + c) =  b  b c a

respective  c b  a

Confirming Pages

233

4.3 Isomorphisms

Observe that T2−1: ⺢3 −→ S2×2 maps the vector   c  b  to the symmetric matrix a



a b

b c



Thus, the desired isomorphism is given by (T2−1 ◦T1 ): P2 −→ S2×2 with   a b −1 ◦ 2 (T2 T1 )(ax + bx + c) = b c For example,    2 1 (T2−1 ◦T1 )(x 2 − x + 2) = T2−1 (T1 (x 2 − x + 2)) = T2−1  −1  = −1 1

−1 2

Fact Summary Let V and W be vector spaces and T a linear transformation from V into W . 1. The mapping T is one-to-one if and only if the null space of T consists of only the zero vector. 2. If {v1 , . . . , vn } is a basis for V and T is one-to-one, then S = {T (v1 ), . . . , T (vn )} is a basis for the range of T. If T is also onto, then S is a basis for W . 3. Every vector space of dimension n is isomorphic to the Euclidean space ⺢n . 4. If T is one-to-one, then T −1 is also a linear transformation. 5. If V and W are both of dimension n, then they are isomorphic. 6. Let A be an n × n matrix and T (x) = Ax. Then the mapping T is an isomorphism if and only if A is invertible. 7. If A is an invertible matrix and T (x) = Ax, then T −1 (x) = A−1 x.

Exercise Set 4.3 In Exercises 1–6, determine whether the linear transformation is one-to-one. 1. T: ⺢2 → ⺢2 ,  T

x y



 =

2. T: ⺢2 → ⺢2 ,    x = T y

4x − y x 1 2x 1 2x

+ 12 y + 12 y





3, 3. T: ⺢3 → ⺢ 

   x+y−z x  y T  y  =  y−z z

4. T: ⺢3 → ⺢3 , 

  2 x − 2 y − 2 z  x 3 3 3   T  y  =  − 23 x − 13 y − 13 z  z − 23 x − 43 y − 13 z



Confirming Pages

234

Chapter 4 Linear Transformations

5. T: P2 → P2 ,

14. T: ⺢2 → ⺢2 ,    x = T y

T (p(x)) = p (x) − p(x) 6. T: P2 → P3 , T (p(x)) = xp(x) In Exercises 7–10, determine whether the linear transformation is onto. 7. T: ⺢2 → ⺢2 ,





x y

T





3x − y x+y

=

8. T: ⺢2 → ⺢2 ,     −2x + y x = T y x − 12 y 9. T: ⺢3 → ⺢3 , 

   x − y + 2z x  y−z T   y  =  2z z

10. T: ⺢3 → ⺢3 , 

   2x + 3y − z x T  y  =  −x + y + 3z  x + 4y + 2z z

In Exercises 11–14, T: ⺢2 → ⺢2 is a linear operator. Determine whether the set {T (e1 ), T (e2 )} is a basis for ⺢2 . 11. T: ⺢2 → ⺢2 ,     −x − 2y x = T 3x y 12. T: ⺢2 → ⺢2 ,



T

13. T:

⺢2

→

⺢2 ,



T

x y

x y



 =



 =

x −3x



3x − y −3x − y

1 10 x 1 5x

+ 15 y + 25 y



In Exercises 15–18, T: ⺢3 → ⺢3 is a linear operator. Determine whether the set {T (e1 ), T (e2 ), T (e3 )} is a basis for ⺢3 . 15. T: ⺢3 → ⺢3 , 

   −x − y + 2z x  y−z T   y  =  5z z

16. T: ⺢3 → ⺢3 , 

   2x + 3y − z x T  y  =  2x + 6y + 3z  4x + 9y + 2z z

17. T: ⺢3 → ⺢3 , 

   4x − 2y + z x  2x + z T  y  =  z 2x − y + 32 z

18. T: ⺢3 → ⺢3 , 

   x − y + 2z x T  y  =  −x + 2y − z  −y + 5z z

In Exercises 19 and 20, T: P2 → P2 is a linear operator. Determine whether the set {T (1), T (x), T (x 2 )} is a basis for P2 . 19. T (ax 2 + bx + c) = (a + b + c)x 2 + (a + b)x + a 20. T (p(x)) = xp (x) In Exercises 21–24, let T: V → V be the linear operator defined by T (v) = Av. a. Show that T is an isomorphism. b. Find A−1 .



c. Show directly that T −1 (w) = A−1 w for all w ∈ V.      x 1 0 x = 21. T y −2 −3 y

Confirming Pages

4.4 Matrix Representation of a Linear Transformation

 22. T 

x y

x 23. T  y z  x 24. T  y z

 x −2 3 = y −1 −1   −2 0 1  =  1 −1 −1 0 1 0   2 −1 1  =  −1 1 −1 0 1 0





 

30. Show that T: P3 → P3 defined by T (p(x)) = p (x) + p (x) + p (x) + p(x)



x  y  z   x  y  z

In Exercises 25–28, determine whether the matrix mapping T: V → V is an isomorphism.      x −3 1 x = 25. T y 1 −3 y      x −3 1 x = 26. T y −3 1 y      x 0 −1 −1 x 0 2  y  27. T  y  =  2 z 1 1 −3 z      x 1 3 0 x 28. T  y  =  −1 −2 −3   y  z 0 −1 3 z 29. Show that T: Mn×n → Mn×n defined by T (A) = At is an isomorphism. ß

4.4

235

is an isomorphism. 31. Let A be an n × n invertible matrix. Show that T: Mn×n → Mn×n defined by T (B) = ABA−1 is an isomorphism. 32. Find an isomorphism from M2×2 onto ⺢4 . 33. Find an isomorphism from ⺢4 onto P3 . 34. Find an isomorphism from M2×2 onto P3 . 35. Let

     x  V =  y  x + 2y − z = 0   z 

Find an isomorphism from V onto ⺢2 .   a b  a, b, c ∈ ⺢ c −a  Find an isomorphism from P2 onto V . 

36. Let

V =

37. Suppose T: ⺢3 → ⺢3 is an isomorphism. Show that T takes lines through the origin to lines through the origin and planes through the origin to planes through the origin.

Matrix Representation of a Linear Transformation Matrices have played an important role in our study of linear algebra. In this section we establish the connection between matrices and linear transformations. To illustrate the idea, recall from Sec. 4.1 that given any m × n matrix A, we can define a linear transformation T: ⺢n −→ ⺢m by T (v) = Av In Example 8 of Sec. 4.1, we showed how a linear transformation T: ⺢3 −→ ⺢2 is completely determined by the images of the coordinate vectors e1 , e2 , and e3 of ⺢3 .   v1 The key was to recognize that a vector v =  v2  can be written as v3 v = v1 e1 + v2 e2 + v3 e3

so that

T (v) = v1 T (e1 ) + v2 T (e2 ) + v3 T (e3 )

Confirming Pages

236

Chapter 4 Linear Transformations

In that example, T was defined so that     1 −1 , T (e2 ) = T (e1 ) = 1 2

 T (e3 ) =

0 1



Now let A be the 2 × 3 matrix whose column vectors are T (e1 ), T (e2 ), and T (e3 ). Then   1 −1 0 v = Av T (v) = 1 2 1 That is, the linear transformation T is given by a matrix product. In general, if T: ⺢n −→ ⺢m is a linear transformation, then it is possible to write T (v) = Av where A is the m × n matrix whose j th column vector is T (ej ) for j = 1, 2, . . . , n. The matrix A is called the matrix representation of T relative to the standard bases of ⺢n and ⺢m . In this section we show that every linear transformation between finite dimensional vector spaces can be written as a matrix multiplication. Specifically, let V and W be finite dimensional vector spaces with fixed ordered bases B and B  , respectively. If T: V −→ W is a linear transformation, then there exists a matrix A such that [T (v)]B  = A[v]B In the case for which V = ⺢n , W = ⺢m , and B and B  are, respectively, the standard bases, the last equation is equivalent to T (v) = Av as above. We now present the details. Let V and W be vector spaces with ordered bases B = {v1 , v2 , . . . , vn } and B  = {w1 , w2 , . . . , wm }, respectively, and let T: V −→ W be a linear transformation. Now let v be any vector in V and let   c1  c2   [v]B =   ...  cn be the coordinate vector of v relative to the basis B. Thus, v = c1 v1 + c2 v2 + · · · + cn vn Applying T to both sides of this last equation gives T (v) = T (c1 v1 + c2 v2 + · · · + cn vn ) = c1 T (v1 ) + c2 T (v2 ) + · · · + cn T (vn ) Note that for each i = 1, 2, . . . , n the vector T (vi ) is in W . Thus, there are unique scalars aij with 1 ≤ i ≤ m and 1 ≤ j ≤ n such that

Revised Confirming Pages

4.4 Matrix Representation of a Linear Transformation

237

T (v1 ) = a11 w1 + a21 w2 + · · · + am1 wm T (v2 ) = a12 w1 + a22 w2 + · · · + am2 wm .. . T (vn ) = a1n w1 + a2n w2 + · · · + amn wm Thus, the coordinate vectors relative to the ordered basis B
are given by   a1i  a2i   [T (vi )]B
=  for i = 1, 2, . . . , n  ...  ami Recall from Example 7 of Sec. 4.1 that the coordinate map defines a linear transformation. Thus, the coordinate vector of T (v) relative to B
can be written in vector form as       a11 a12 a1n  a21   a22   a2n       [T (v)]B
= c1   ...  + c2  ...  + · · · + cn  ...  am1

am2

amn

or in matrix form as    [T (v)]B
=  

a11 a21 .. .

a12 a22 .. .

... ... .. .

am1

am2

. . . amn

a1n a2n .. .



 c1  c   2    ..   . cn


The matrix on the right-hand side of the last equation is denoted by [T ]BB , with 
[T ]BB







=  T (v1 ) 



 

 T (v2 )  · · ·  T (vn )   B


B


B


B


We call [T ]B the matrix of T relative to B and B
. In the case for which T: V → V is a linear operator and B is a fixed ordered basis for V , the matrix representation for the mapping T is denoted by [T ]B . The preceding discussion is summarized in Theorem 12.

THEOREM 12

Let V and W be finite dimensional vector spaces with ordered bases B = {v1 , v2 , . . . vn } and B
= {w1 , w2 , . . . wm }, respectively, and let T: V −→ W be a linear
transformation. Then the matrix [T ]BB is the matrix representation for T relative to the bases B and B
. Moreover, the coordinates of T (v) relative to B
are given by


[T (v)]B
= [T ]BB [v]B

Confirming Pages

238

Chapter 4 Linear Transformations

Suppose that in Theorem 12 the vector spaces V and W are the same, B and B  are two different ordered bases for V , and T: V −→ V is the identity operator, that   is, T (v) = v for all v in V . Then [T ]BB is the change of bases matrix [I ]BB , given in Sec. 3.4.

EXAMPLE 1

Define the linear operator T: ⺢3 −→ ⺢3 by     x x T  y  =  −y  z z a. Find the matrix of T relative to the standard basis for ⺢3 . b. Use the result of part (a) to find   1 T  1  2

Solution

a. Let B = {e1 , e2 , e3 } be the standard basis for ⺢3 . Since       1 0 0 [T (e2 )]B =  −1  [T (e3 )]B =  0  [T (e1 )]B =  0  0 0 1 then

z

T (v)

v y



1  [T ]B = 0 0

b. Since B is the standard basis for ⺢3 , the by its components. In this case, with   1 then v= 1  2 Thus, by Theorem 12,  1 0 T (v) = [T (v)]B =  0 −1 0 0

 0 0 −1 0  0 1 coordinates of any vector are given  1 [v]B =  1  2 

    1 1 0 0   1  =  −1  2 2 1

Notice that the action of T is a reflection through the xz plane, as shown in Fig. 1.

x

Figure 1

The following steps summarize the process for finding the matrix representation of a linear transformation T: V −→ W relative to the ordered bases B and B  .

Confirming Pages

4.4 Matrix Representation of a Linear Transformation

239

1. For the given basis B = {v1 , v2 , . . . , vn }, find T (v1 ), T (v2 ), . . . , T (vn ). 2. Find the coordinates of T (v1 ), T (v2 ), . . . , T (vn ) relative to the basis B  = {w1 , w2 , . . . , wm } of W . That is, find [T (v1 )]B  , [T (v2 )]B  , . . . , [T (vn )]B  .  3. Define the m × n matrix [T ]BB with ith column vector equal to [T (vi )]B  . 4. Compute [v]B . 5. Compute the coordinates of T (v) relative to B  by   c1  c2    [T (v)]B  = [T ]BB [v]B =   ...  cm 6. Then T (v) = c1 w1 + c2 w2 + · · · + cm wm .

EXAMPLE 2

Let T: ⺢2 −→ ⺢3 be the linear transformation defined by     x2 x1 =  x1 + x2  T (v) = T x2 x1 − x2 and let           1  1  1 3 1 B  =  0 ,  1 ,  1  , B= 1 2   1 0 0 2 3 be ordered bases for ⺢ and ⺢ , respectively. 

a. Find the matrix [T ]BB .   −3 . Find T (v) directly and then use the matrix found in part (a). b. Let v = −2 Solution

a. We first apply T to the basis vectors of B, which gives         1 2 3 1 = 4  and T = 3  T 1 2 2 −1 Next we find the coordinates of each of these vectors relative to the basis B  . That is, we find scalars such that         2 1 1 1 a1  0  + a2  1  + a3  1  =  3  −1 0 0 1 and

       1 1 1 1 b 1  0  + b2  1  + b3  1  =  4  2 0 0 1 

Confirming Pages

240

Chapter 4 Linear Transformations

The solution to the first linear system is a1 = −1

a2 = 4

a3 = −1

and the solution to the second system is b1 = −3

b2 = 2

b3 = 2

 −1 −3 2  = 4 −1 2 

Thus, B

[T ]B

b. Using the definition of T directly, we have       −2 −2 −3 =  −3 − 2  =  −5  T −2 −1 −3 + 2 Now, to use the matrix found in part (a), we need to find the coordinates of v relative to B. Observe that the solution to the equation       3 −3 1 3 a1 is a1 = − + a2 = −2 2 1 5   −3 relative to B is Thus, the coordinate vector of −2  3    −5 −3 = −2 B − 45

a2 = −

4 5

We can now evaluate T, using matrix multiplication, so that     3  −1 −3 − 35 2  =  −4  [T (v)]B  =  4 4 − −1 −1 2 5        −2 1 1 1 T (v) = 3  0  − 4  1  −  1  =  −5  −1 1 0 0 which agrees with the direct computation. Hence,

EXAMPLE 3



Define a linear transformation T: P2 −→ P3 by T (f (x)) = x 2 f  (x) − 2f  (x) + xf (x) Find the matrix representation of T relative to the standard bases for P2 and P3 .

Confirming Pages

4.4 Matrix Representation of a Linear Transformation

Solution

241

Since the standard basis for P2 is B = {1, x, x 2 }, we first compute T (1) = x

T (x) = x 2 − 2

T (x 2 ) = x 2 (2) − 2(2x) + x(x 2 ) = x 3 + 2x 2 − 4x

Since the standard basis for P3 is B  = {1, x, x 2 , x 3 }, the coordinates relative to B  are [T (1)]B 

 0  1   =  0  0 

 −2  0   =  1  0 

[T (x)]B 

 0  −4   =  2  1 

and

[T (x 2 )]B 

Hence, the matrix of the transformation is given by   0 −2 0  1  0 −4   [T ]BB =   0 1 2  0 0 1 As an example, let f (x) = x 2 − 3x + 1. Since f  (x) = 2x − 3 and f  (x) = 2, we have T (f (x)) = x 2 (2) − 2(2x − 3) + x(x 2 − 3x + 1) = x 3 − x 2 − 3x + 6 Using the matrix representation of T to find the same image, we observe that   1 [f (x)]B =  −3  1 The coordinates of the image of f (x) under the mapping T relative to B  are then given by      6 0 −2 0  1    1  0 −4    −3  =  −3  [T (f (x))]B  = [T ]BB [f (x)]B =     0 −1  1 2 1 1 0 0 1 The image T (f (x)) is the linear combination of the monomials in B  with coefficients the components of [T (f (x))]B  , that is, T (f (x)) = 6(1) − 3(x) − x 2 + x 3 = x 3 − x 2 − 3x + 6 This agrees with the direct calculation.

In Sec. 4.1 we discussed the addition, scalar multiplication, and composition of linear maps. The matrix representations for these combinations are given in a natural way, as described by Theorems 13 and 14. The proofs are omitted.

Confirming Pages

242

Chapter 4 Linear Transformations

THEOREM 13

Let V and W be finite dimensional vector spaces with ordered bases B and B  , respectively. If S and T are linear transformations from V to W , then 





1. [S + T ]BB = [S]BB + [T ]BB   2. [kT ]BB = k[T ]BB for any scalar k As before in the special case for which S and T are linear operators on a finite dimensional vector space V , and B is a fixed ordered basis for V , the notation becomes [S + T ]B = [S]B + [T ]B and [kT ]B = k[T ]B .

EXAMPLE 4

Let S and T be linear operators on ⺢2 with     x + 2y x and = S −y y

 T

x y



 =

−x + y 3x



If B is the standard basis for ⺢2 , find [S + T ]B and [3S]B . Solution

The matrix representations for the linear operators S and T are, respectively,        1 2 [S]B =  S(e1 )   S(e2 )   = 0 −1 B

B





and

 



[T ]B =  T (e1 ) 

 T (e2 )   =

B

Then by Theorem 13,



[S + T ]B =

1 2 0 −1 

and [3S]B = 3



−1 1 3 0



B



 +

1 0

2 −1



−1 1 3 0 

 =

 =

3 0

6 −3

0 3

3 −1





As we mentioned in Sec. 4.1, the matrix of the composition is the product of the matrices of the individual maps, as given in Theorem 14.

THEOREM 14

Let U, V , and W be finite dimensional vector spaces with ordered bases B, B  , and B  , respectively. If T: U → V and S: V → W are linear transformations, then 



[S ◦T ]BB = [S]BB  [T ]BB



Confirming Pages

4.4 Matrix Representation of a Linear Transformation

243

Again, if S and T are linear operators on a finite dimensional vector space V , and B is a fixed ordered basis for V , then [S ◦T ]B = [S]B [T ]B Repeated application of Theorem 14 gives the following result.

COROLLARY 1

EXAMPLE 5

Let V be a finite dimensional vector space with ordered basis B. If T is a linear operator on V , then [T n ]B = ([T ]B )n

Let D: P3 → P3 be the linear operator defined by D(p(x)) = p (x) a. Find the matrix of D relative to the standard basis B = {1, x, x 2 , x 3 }. Use the matrix to find the derivative of p(x) = 1 − x + 2x 3 . b. Find the matrix needed to compute the second derivative of a polynomial in P3 . Use this matrix to find the second derivative of p(x) = 1 − x + 2x 3 .

Solution

a. By Theorem 12, we have  



[D]B =  D(1)  

0 1  0 0 =  0 0 0 0



B





 D(x 2 ) 

 D(x)  B

0 2 0 0





 

 D(x 3 )  

B

B

0 0   3  0

Since the coordinate vector of p(x) = 1 − x + 2x 3 , relative to B, is given by   1  −1   [p(x)]B =   0  2 then



0  0 [D(p(x))]B =   0 0

1 0 0 0

0 2 0 0

    −1 1 0     0    −1  =  0      6  0 3 0 2 0

Confirming Pages

244

Chapter 4 Linear Transformations

Therefore, as expected, D(p(x)) = −1 + 6x 2 . b. By Corollary 1, the matrix we need is given by  0 0  0 0 [D 2 ]B = ([D]B )2 =   0 0 0 0 If p(x) = 1 − x + 2x 3 , then 

0  0 2 [D (p(x))]B =   0 0

0 0 0 0

2 0 0 0

2 0 0 0

 0 6   0  0

    0 1 0     6    −1  =  12      0  0 0 0 2 0

so that p (x) = 12x.

The final result of this section describes how to find the matrix representation of the inverse map of an invertible linear operator. COROLLARY 2

Let T be an invertible linear operator on a finite dimensional vector space V and B an ordered basis for V . Then [T −1 ]B = ([T ]B )−1 Proof Since T −1 ◦T is the identity map, by Theorem 14, we have [I ]B = [T −1 ◦T ]B = [T −1 ]B [T ]B Since [I ]B is the identity matrix, [T −1 ]B = ([T ]B )−1 .

Fact Summary Let V and W be vector spaces, B = {v1 , . . . , vn } and B  = {w1 , . . . , wm } ordered bases of V and W , respectively, and T a linear transformation from V into W . 1. The matrix of T relative to B and B  is given by 

[T ]BB = [ [T (v1 )]B  [T (v2 )]B  · · · [T (vn )]B  ] 2. If v is a vector in V , the coordinates of T (v) relative to the basis B  can be computed by  [T (v)]B  = [T ]BB [v]B

Confirming Pages

4.4 Matrix Representation of a Linear Transformation

245

3. To find T (v) multiply each basis vector in B  by the corresponding  t component of [T (v)]B  . That is, if [T (v)]B  = [b1 b2 . . . bm ] , then  wm T (v) = b1 w1 + b2 w2 + · · · + bm

4. If S is another linear transformation from V into W , then the matrix representation of S + T relative to B and B  is the sum of the matrix    representations for S and T. That is, [S + T ]BB = [S]BB + [T ]BB . 5. If c is a scalar, then to find the matrix representation of cT relative to B and   B  , multiply the matrix representation for T by c. That is, [cT ]BB = c[T ]BB . 6. If S is a linear transformation from W into Z and B  is an ordered basis    for Z, then [S ◦T ]BB = [S]BB  [T ]BB . 7. [T n ]B = ([T ]B )n 8. If T is invertible, then [T −1 ]B = ([T ]B )−1 .

Exercise Set 4.4 In Exercises 1–4, T: ⺢n → ⺢n is a linear operator. a. Find the matrix representation for T relative to the standard basis for ⺢n . b. Find T (v), using a direct computation and using the matrix representation. 1. T: ⺢2 → ⺢2 ,       2 5x − y x v= = T 1 −x + y y 2. T: ⺢2 → ⺢2 ,     −x x = T y y   −1 v= 3 3. T: ⺢3 →  ⺢3,

   −x + y + 2z x  3y + z T  y  =  x−z z   1 v =  −2  3

4. T: ⺢3 → ⺢3 ,

   x x T   y  =  y  −z z   2 v =  −5  1 

In Exercises 5–12, T: V → V is a linear operator with B and B  ordered bases for V . a. Find the matrix representation for T relative to the ordered bases B and B  . b. Find T (v), using a direct computation and using the matrix representation. 5. T: ⺢2 → ⺢2 ,     −x + 2y x = T 3x y     2 1 , B= 0 −1

Confirming Pages

246

Chapter 4 Linear Transformations

   0 1 , B = 1 0   −1 v= −2 



9. T: P2 → P2 , T (ax 2 + bx + c) = ax 2 + bx + c B = {1, 1 − x, (1 − x)2 }

6. T: ⺢3 → ⺢3 ,     2x − z x T  y  =  −x + y + z  2z z      1  1  −1 B =  0 ,  2 ,  2    1 0 1      0  0  1 B  =  0 ,  1 ,  0    1 0 0   1 v =  −1  1

v = x 2 − 3x + 3 10. T: P2 → P2 , T (p(x)) = p (x) + p(x) B = {1 − x − x 2 , 1, 1 + x 2 } B  = {−1 + x, −1 + x + x 2 , x} v=1−x 

11. Let H =

1 0 0 −1



and let T be the linear operator on all 2 × 2 matrices with trace 0, defined by T (A) = AH − H A       0 0 0 1 1 0 , , B= 1 0 0 0 0 −1 B = B   2 1 v= 3 −2

7. T: ⺢2 → ⺢2 ,     2x x = T x+y y    1 −1 , B= 1 −2    0 3  B = , −2 −2   −1 v= −3 8. T: ⺢3 → ⺢3 ,     x+z x T  y  =  2y − x  y+z z    0 −1  −1 B =  1 ,  −1 ,  1  1 1 1    −1 1  0 B  =  0 ,  0 ,  −1  0 −1 1   −2 v= 1  3

B  = {1, x, x 2 }

12. T: M2×2 → M2×2 , T (A) = 2At + A B and B  the standard basis on M2×2   1 3 v= −1 2        

13. Let T: ⺢2 → ⺢2 be the linear operator defined by     x + 2y x = T x− y y Let B be the standard ordered basis for ⺢2 and B  the ordered basis for ⺢2 defined by     4 1 , B = −1 2 a. Find [T ]B . b. Find [T ]B  .

Confirming Pages

4.4 Matrix Representation of a Linear Transformation 

c. Find [T ]BB . d. Find [T ]BB  . e. Let C be the ordered basis obtained by switching the order of the vectors in B. Find  [T ]BC . f. Let C  be the ordered basis obtained by switching the order of the vectors in B  . Find  [T ]BC  . 14. Let T: ⺢2 → ⺢3 be the linear transformation defined by     x−y x  x = T y x + 2y Let B and B  be ordered bases for ordered basis for ⺢3 defined by     1 1 , B= 3 1     0 1 B = , 1 0     0 1  B  =  1 ,  1 ,   1 0

247

b. Let C be the ordered basis obtained by switching the order of the vectors in B. Find  [T ]BC . c. Let C  be the ordered basis obtained by switching the first and second vectors in B  .  Find [T ]C C . d. Define S: P2 → P1 by S(a + bx + cx 2 ) = b + 2cx Find [S]BB  .  e. Verify that [S]BB  [T ]BB = I , but that  [T ]BB [S]BB  = I . f. Interpret the statement 

[S]BB  [T ]BB = I

⺢2 and B  the

in terms of the functions T and S.

 1  1   2



a. Find [T ]BB .  b. Find [T ]BB  . c. Let C be the ordered basis obtained by switching the order of the vectors in B. Find  [T ]BC . d. Let C  be the ordered basis obtained by switching the order of the vectors in B  . Find  [T ]BC  . e. Let C  be the ordered basis obtained by switching the order of the first and third  vectors in B  . Find [T ]C B . 15. Let T: P1 → P2 be the linear transformation defined by b T (a + bx) = ax + x 2 2  Let B and B be the standard ordered bases for P1 and P2 , respectively.  a. Find [T ]BB .

16. Define a linear operator T: M2×2 → M2×2 by     2a c − b a b = T −d d − a c d Let B be the standard ordered basis for M2×2 and B  the ordered basis     0 1 1 0  , B = 1 0 0 1     −1 −1 −1 1 , 1 1 −1 1 a. b. c. d. e.



Find [T ]BB . Find [T ]BB  . Find [T ]B  .  Find [I ]BB and [I ]BB  . Verify that 



[T ]BB = [T ]B  [I ]BB [T ]BB  = [I ]BB  [T ]B  17. Define a linear operator T: ⺢2 → ⺢2 by     x x = T −y y Find the matrix for T relative to the standard basis for ⺢2 . Describe geometrically the action of T on a vector in ⺢2 .

Confirming Pages

248

Chapter 4 Linear Transformations

18. Define a linear operator T: ⺢2 → ⺢2 by      x cos θ − sin θ x = T y sin θ cos θ y

In Exercises 26–29, let S, T: ⺢3 → ⺢3 be defined by     x−y−z x T  y  =  2y + 2z  −x + y + z z

Describe geometrically the action of T on a vector in ⺢2 . 19. Let c be a fixed scalar and define T:    x1 x1  x2  x2     T  ...  = c  ... xn

⺢n 

→

and ⺢n

  

xn

Find the matrix for T relative to the standard basis for ⺢n . 20. Define T: M2×2 → M2×2 by T (A) = A − At

21. Define T: M2×2 → ⺢ by

In Exercises 22–25, let S, T: ⺢2 → ⺢2 be defined by     2x + y x = T − x + 3y y and 

x x = x+y y a. Find the matrix representation for the given linear operator relative to the  basis.  standard −2 directly b. Compute the image of v = 3 and using the matrix found in part (a). S

22. −3S 23. 2T + S 24. T ◦S 25. S ◦T

27. −3T + 2S 29. S ◦T



Find the matrix [T ]BB , where B is the standard basis for M2×2 and B  = {1}.



26. 2T 28. T ◦S

T (A) = tr(A)



a. Find the matrix representation for the given linear operator relative to the standard basis. b. Compute the image of   −1 v= 1  3 directly and using the matrix found in part (a).

Find the matrix for T relative to the standard basis for M2×2 .



   3x − z x  x S  y  =  z z 

by

30. Let B be the basis for ⺢2 defined by     1 1 , B= 3 1 If T: ⺢2 → ⺢2 is the linear operator defined by     9x − 5y x = T 15x − 11y y find the matrix for T k , for k ≥ 1, relative to the basis B. 31. Define T: P4 → P4 by T (p(x)) = p (x) Find the matrix for T relative to the standard basis for P4 . Use the matrix to find the third derivative of p(x) = −2x 4 − 2x 3 + x 2 − 2x − 3. 32. Let T: P2 → P2 be defined by T (p(x)) = p(x) + xp (x) Find the matrix [T ]B where B is the standard basis for P2 .

Confirming Pages

4.5 Similarity

33. Let S: P2 → P3 and D: P3 → P2 be defined by S(p(x)) = xp(x) and D(p(x)) = p (x)

reflects a vector v through the line perpendicular to v1 . 35. Let A be a fixed 2 × 2 matrix and define T: M2×2 → M2×2 by T (B) = AB − BA



Find the matrices [S]BB and [D]BB  , where B = {1, x, x 2 } and B  = {1, x, x 2 , x 3 }. Observe that the operator T in Exercise 32 satisfies T = D ◦S. Verify Theorem 14 by showing that  [T ]B = [D]BB  [S]BB . 34. a. Define a basis for ⺢2 by     0 1 , B= 1 1 Find [T ]B where T: ⺢2 → ⺢2 is the linear operator that reflects a vector  v through the 1 . line perpendicular to 1 b. Let B = {v1 , v2 } be a basis for ⺢2 . Find [T ]B where T: ⺢2 → ⺢2 is the linear operator that

ß

4.5

249

Find the matrix for T relative to the standard basis for M2×2 . 36. Let B = {v1 , v2 , v3 } and B  = {v2 , v1 , v3 } be ordered bases for the vector space V . If T: V → V is defined by T (v) = v, then find  [T ]BB . Describe the relationship between [v]B and [v]B  and the relationship between the identity  matrix I and [T ]BB . 37. Let V be a vector space and B = {v1 , v2 , . . . , vn } be an ordered basis for V. Define v0 = 0 and T: V → V by T (vi ) = vi + vi−1

for i = 1, . . . , n

Find [T ]B .

Similarity We have just seen in Sec. 4.4 that if T: V −→ V is a linear operator on the vector space V , and B is an ordered basis for V , then T has a matrix representation relative to B. The specific matrix for T depends on the particular basis; consequently, the matrix associated with a linear operator is not unique. However, the action of the operator T on V is always the same regardless of the particular matrix representation, as illustrated in Example 1.

EXAMPLE 1

Let T: ⺢2 −→ ⺢2 be the linear operator defined by     x+ y x = T − 2x + 4y y

   1 Also let B1 = {e1 , e2 } be the standard basis for , and let B2 = 2  2 be a second basis for ⺢2 . Verify that the action on the vector v = by the 3 operator T is the same regardless of the matrix representation used for T. ⺢2



1 1 

Confirming Pages

250

Chapter 4 Linear Transformations

Solution

The matrix representations for T relative to B1 and B2 are     1 1 2 0 and [T ]B2 = [T ]B1 = −2 4 0 3 respectively. Next, observe that   2 [v]B1 = 3

 and

[v]B2 =

1 1



Applying the matrix representations of the operator T relative to B1 and B2 , we obtain      5 2 1 1 = [T (v)]B1 = [T ]B1 [v]B1 = 8 3 −2 4 and      2 1 2 0 [T (v)]B2 = [T ]B2 [v]B2 = = 3 1 0 3 To see that the result is the same, observe that             5 1 1 5 0 1 = +3 and T (v) = 2 = +8 T (v) = 5 8 2 1 8 1 0

Theorem 15 gives the relationship between the matrices for a linear operator relative to two distinct bases.

THEOREM 15

Let V be a finite dimensional vector space, B1 and B2 two ordered bases for V, B and T: V −→ V a linear operator. Let P = [I ]B12 be the transition matrix from B2 to B1 . Then [T ]B2 = P −1 [T ]B1 P Proof Let v be any vector in V. By Theorem 12 of Sec. 4.4, we have [T (v)]B2 = [T ]B2 [v]B2 Alternatively, we can compute [T (v)]B2 as follows: First, since P is the transition matrix from B2 to B1 , [v]B1 = P [v]B2 Thus, the coordinates of T (v) relative to B1 are given by [T (v)]B1 = [T ]B1 [v]B1 = [T ]B1 P [v]B2 Now, to find the coordinates of T (v) relative to B2 , we multiply on the left by P −1 , which is the transition matrix from B1 to B2 , to obtain [T (v)]B2 = P −1 [T ]B1 P [v]B2 Since both representations for [T (v)]B2 hold for all vectors v in V , then [T ]B2 = P −1 [T ]B1 P . See Fig. 1.

Confirming Pages

4.5 Similarity

[v]B1

251

[T (v)]B1 [T ]B1

P −1

P

[T ]B2

[v]B2

[T (v)]B2

Figure 1

EXAMPLE 2

Let T, B1 , and B2 be the linear operator and bases of Example 1. Then   1 1 [T ]B1 = −2 4 Use Theorem 15 to verify that

 [T ]B2 =

Solution

EXAMPLE 3

2 0 0 3



Since B1 is the standard basis for ⺢2 , by Theorem 14 of Sec. matrix from B2 to B1 is       1 1 1 1 B1 = P = [I ]B2 = 1 2 2 B 1 B 1 1 and hence   2 −1 P −1 = −1 1 Then      2 1 1 1 1 2 −1 = P −1 [T ]B1 P = 0 1 2 −2 4 −1 1

3.4 the transition 

0 3

 = [T ]B2

Let T: ⺢2 −→ ⺢2 be the linear operator given by     −x + 2y x = T 3x + y y and let

 B1 =

2 −1

   1 , 0

 and

B2 =

1 −1

   0 , 1

Confirming Pages

252

Chapter 4 Linear Transformations

be ordered bases for ⺢2 . Find the matrix of T relative to B1 , and then use Theorem 15 to find the matrix of T relative to B2 . Solution



Since T

2 −1



 = 

we have [T ]B1 =

−4 5 −4 5



 and



 B1

−1 3

T  

 =

B1

1 0



−5 6

The transition matrix from B2 to B1 is       1 0 1 B1 = P = [I ]B2 = −1 1 B −1 B 1

Therefore, by Theorem 15 [T ]B2 = P

−1

1

 =

−3 5

−1 3





−1 2



   1 −1 −5 −3 2 1 [T ]B1 P = −1 2 6 5 1 1   −3 2 = −1 3 

In general, if the square matrices A and B are matrix representations for the same linear operator, then the matrices are called similar. Using Theorem 15, we can define similarity for square matrices without reference to a linear operator.

DEFINITION 1

Similar Matrices Let A and B be n × n matrices. We say that A is similar to B if there is an invertible matrix P such that B = P −1 AP .

The notion of similarity establishes a relation between matrices. This relation is symmetric; that is, if the matrix A is similar to the matrix B, then B is similar to A. To see this, let A be similar to B; that is, there is an invertible matrix P such that B = P −1 AP Now let Q = P −1 , so that B can be written as B = QAQ−1 Hence, A = Q−1 BQ, establishing that B is similar to A. For this reason we say that A and B are similar if either A is similar to B or B is similar to A. In addition, the relation is reflexive since any matrix is similar to itself with P being the identity matrix. This relation is also transitive; that is, if A is similar to B and B is similar to C, then A is similar to C. See Exercise 17. Any relation satisfying these three properties is called an equivalence relation.

Confirming Pages

4.5 Similarity

253

Fact Summary Let V be a finite dimensional vector space, B1 and B2 ordered bases of V, and T a linear operator on V. 1. The matrix representations [T ]B1 and [T ]B2 are similar. That is, there is an invertible matrix P such that [T ]B2 = P −1 [T ]B1 P . In addition, the matrix P is the transition matrix from B2 to B1 . 2. A matrix is similar to itself. If A is similar to B, then B is similar to A. If A is similar to B and B is similar to C, then A is similar to C.

Exercise Set 4.5 In Exercises 1 and 2, [T ]B1 is the matrix representation of a linear operator relative to the basis B1 , and [T ]B2 is the matrix representation of the same operator relative to the basis B2 . Show that the action of the operator on the vector v is the same whether using [T ]B1 or [T ]B2 .     1 2 2 1 , [T ]B2 = , 1. [T ]B1 = −1 3 −1 2     0 1 B1 = , 1 0     −1 1 B2 = , 0 1   4 v= −1     0 1 −2 0 2. [T ]B1 = , [T ]B2 = , 2 −1 4 1     0 1 B1 = , 1 0     1 1 B2 = , 1 2   5 v= 2 In Exercises 3–6, a linear operator T and bases B1 and B2 are given. a. Find [T ]B1 and [T ]B2 . b. Verify that the action on v of the linear operator T is the same when using the matrix

representation of T relative to the bases B1 and B2 .     x+y x , = 3. T x+y y B1 = {e1 , e2 }     −1 1 , B2 = 1 1   3 v= −2     −x x = 4. T y y B1 = {e1 , e2 }    −1 2 , B2 = 2 −1   2 v= −2     x x 5. T  y  =  0  z z B1 = {e1 , e2 , e3 }      0  −1  1 B2 =  0 ,  1 ,  0    1 0 1   1 v= 2  −1

Revised Confirming Pages

254

Chapter 4 Linear Transformations

   x+y x 6. T  y  =  x − y + z  y−z z

   1 1 , B1 = 1 −1     0 −1 B2 = , 1 2





B1 = {e1 , e2 , e3 }      1  0  −1 B2 =  1 ,  0 ,  0    1 1 0  2 v =  −1  −1 

B1 = {e1 , e2 }     1 2 , B2 = 2 3

In Exercises 7–10, [T ]B1 and [T ]B2 are, respectively, the matrix representations of a linear operator relative to the bases B1 and B2 . Find the transition matrix B P = [I ]B12 , and use Theorem 15 to show directly that the matrices are similar.  7. [T ]B1 =

1 1 3 2





In Exercises 11–14, find the matrix representation of the linear operator T relative to B1 . Then use Theorem 15 to find [T ]B2 .     2x x = 11. T 3y y

[T ]B2 =

9 2 23 2

 12. T

x y



− 32



B1 = {e1 , e2 }    −1 3 , B2 = 1 −1

x− y x + 2y



B1 = {e1 , e2 }     1 3 , B2 = 2 5



− 12

 =

13. T

x y



 =

y −x 



   1 1 , 0 −1     0 1 B2 = , 1 1 B1 =

 8. [T ]B1 =

0 2

2 −3



 [T ]B2 =

−4 1 0 1



B1 = {e1 , e2 }     1 −1 , B2 = 0 2  9. [T ]B1 =

1 0

0 −1



 [T ]B2 =

0 1 3

3 0

 14. T 

  2 1 , B1 = −1 1     −1 1 B2 = , 2 0 

 10. [T ]B1 =

−1 0 0 1



 [T ]B2 =

−2 −1 3 2



x y



 −2x + y = 2y     −1 2 B1 = , 1 0     0 2 B2 = , −1 2 

15. Let T: P2 −→ P2 be the linear operator defined by T (p(x)) = p (x). Find the matrix representation [T ]B1 relative to the basis B1 = {1, x, x 2 } and the matrix representation [T ]B2 relative to B2 = {1, 2x, x 2 − 2}. Find the B transition matrix P = [I ]B12 , and use Theorem 15

Confirming Pages

4.6 Application: Computer Graphics

to show directly that the matrices [T ]B1 and [T ]B2 are similar. 16. Let T: P2 −→ P2 be the linear operator defined by T (p(x)) = xp (x) + p (x). Find the matrix representation [T ]B1 relative to the basis B1 = {1, x, x 2 } and the matrix representation [T ]B2 relative to B2 = {1, x, 1 + x 2 }. Find the B transition matrix P = [I ]B12 , and use Theorem 15 to show directly that the matrices [T ]B1 and [T ]B2 are similar. 17. Show that if A and B are similar matrices and B and C are similar matrices, then A and C are similar matrices. ß

4.6

255

18. Show that if A and B are similar matrices, then det(A) = det(B). 19. Show that if A and B are similar matrices, then tr(A) = tr(B). 20. Show that if A and B are similar matrices, then At and B t are similar matrices. 21. Show that if A and B are similar matrices, then An and B n are similar matrices for each positive integer n. 22. Show that if A and B are similar matrices and λ is any scalar, then det(A − λI ) = det(B − λI ).

Application: Computer Graphics The rapid development of increasingly more powerful computers has led to the explosive growth of digital media. Computer-generated visual content is ubiquitous, found in almost every arena from advertising and entertainment to science and medicine. The branch of computer science known as computer graphics is devoted to the study of the generation and manipulation of digital images. Computer graphics are based on displaying two- or three-dimensional objects in two-dimensional space. Images displayed on a computer screen are stored in memory using data items called pixels, which is short for picture elements. A single picture can be comprised of millions of pixels, which collectively determine the image. Each pixel contains information on how to color the corresponding point on a computer screen, as shown in Fig. 1. If an image contains curves or lines, the pixels which describe the object may be connected by a mathematical formula. The saddle shown in Fig. 1 is an example.

Figure 1

Confirming Pages

256

Chapter 4 Linear Transformations

Graphics Operations in ⺢2 To manipulate images, computer programmers use linear transformations. Most of the examples we consider in this section use linear operators on ⺢2 . One of the properties of linear transformations that is especially useful to our work here is that linear transformations map lines to lines, and hence polygons to polygons. (See Exercise 10 of the Review Exercises for Chapter 4.) Therefore, to visualize the result of a linear transformation on a polygon, we only need to transform the vertices. Connecting the images of the vertices then gives the transformed polygon.

Scaling and Shearing

y 5 T

6

x

To find a matrix representation of S, let B = {e1 , e2 } be the standard basis for ⺢2 . Then by Theorem 12 of Sec. 4.4, we have   3 0 [S]B = [ [S(e1 )] [S(e2 )] ] = 0 1

Figure 2

y 5 T

S

'

T

6

Figure 3

A transformation on an object that results in a horizontal contraction or dilation (stretching) is called a horizontal scaling. !For example, let T be the triangle shown " in Fig. 2 with vertices (1, 1), (2, 1), and 32 , 3 . Suppose that we wish to perform a horizontal scaling of T by a factor of 3. The transformed triangle T  is obtained by multiplying the x coordinate of each vertex by 3. Joining the new vertices with straight lines produces the result shown in Fig. 3. The linear operator S: ⺢2 −→ ⺢2 that accomplishes this is given by     3x x = S y y

x

Let vi and vi , for i = 1, 2, and 3, be, respectively, the vertices (in vector form) of T and T . Since the coordinates of the vertices of T are given relative to B, the vertices of T  can be found by matrix multiplication. Specifically,           3 6 1 2 3 0 3 0 v2 = = = v1 = 1 1 1 1 0 1 0 1 and v3

 =

3 0 0 1



3 2

3



 =

9 2



3

These results are consistent with the transformed triangle T , as shown in Fig. 3. In general, a horizontal scaling by a factor k is given by the linear transformation Sh defined by     kx x = Sh y y

Confirming Pages

4.6 Application: Computer Graphics

257

The matrix representation of Sh , relative to the standard basis for ⺢2 , is given by   k 0 [Sh ]B = 0 1 Similarly, a vertical scaling is given by the linear operator     x x = Sv ky y The matrix representation of Sv , relative to the standard basis for ⺢2 , is given by   1 0 [Sv ]B = 0 k If both components are multiplied by the same number k, then the result is called a uniform scaling. In all the above cases, if k > 1, then the transformation is called a dilation, or stretching; and if 0 < k < 1, then the operator is a contraction.

EXAMPLE 1

Let T denote the triangle with vertices given by the vectors       0 2 1 v2 = v3 = v1 = 1 1 3 as shown in Fig. 4. a. Stretch the triangle horizontally by a factor of 2. b. Contract the triangle vertically by a factor of 3. c. Stretch the triangle horizontally by a factor of 2, and contract the triangle vertically by a factor of 3.

Solution y 5 T

⫺5

5

⫺5

Figure 4

x

a. To stretch the triangle horizontally by a factor of 2, we apply the matrix   2 0 0 1 to each vertex to obtain   0 v1 = 1

v2 =



4 1



v3 =



2 3



Connecting the new vertices by straight-line segments gives the triangle T  shown in Fig. 5(a). b. To contract the triangle vertically by a factor of 3, we apply the matrix   1 0 0 13 to each vertex to obtain       0 2 1 v2 = 1 v3 = v1 = 1 1 3 3

Confirming Pages

258

Chapter 4 Linear Transformations

The contracted triangle T  is shown in Fig. 5(b). y

y

5

5

'

T

''

T ⫺5

5

x

⫺5

⫺5

5

x

⫺5 (a)

(b)

Figure 5

c. This operator is the composition of the linear operators of parts (a) and (b). By Theorem 14 of Sec. 4.4, the matrix of the operator, relative to the standard basis for ⺢2 , is given by the product      2 0 1 0 2 0 = 0 1 0 13 0 13

y 5

'''

T ⫺5

5

⫺5

Figure 6

x

Applying this matrix to the vertices of the original triangle gives       0 4 2    v2 = 1 v3 = v1 = 1 1 3 3 as shown in Fig. 6.

Another type of transformation, called shearing, produces the visual effect of slanting. The linear operator S: ⺢2 → ⺢2 used to produce a horizontal shear has the form     x + ky x = S y y where k is a real number. Relative to the standard basis B, the matrix representation of S is given by   1 k [S]B = 0 1   0 , As an illustration, let T be the triangle of Fig. 7(a) with vertices v1 = 0     2 1 , and v3 = , and let k = 2. After applying the matrix v2 = 0 1   1 2 [S]B = 0 1

Confirming Pages

259

4.6 Application: Computer Graphics

 to each of the vertices of T, we obtain The resulting triangle

T

v1

=

0 0

     2 3   , v2 = , and v 3 = . 0 1

is shown in Fig. 7(b).

y

y

5

5

'

T

T

⫺5

5

x

⫺5

5

⫺5

x

⫺5 (a)

(b)

Figure 7

A vertical shear is defined similarly by     x x = S y + kx y In this case the matrix for S, relative to the standard basis B, is given by   1 0 [S]B = k 1

EXAMPLE 2 Solution y 5

⫺5

5

x

Perform a vertical shear, with k = 2, on the triangle of Fig. 2. The matrix of this operator, relative to the standard basis for ⺢2 , is given by   1 0 2 1 Applying this matrix to the vertices     1 2 v2 = v1 = 1 1 we obtain

⫺5

Figure 8

v1 =



1 3



v2 =



2 5



 v3 = v3 =

3 2



3 

3 2



6

The images of the original triangle and the sheared triangle are shown in Fig. 8.

Confirming Pages

260

Chapter 4 Linear Transformations

Reflection The reflection of a geometric object through a line produces the mirror image of the object across the line. The linear operator that reflects a vector through the x axis is given by     x x = Rx −y y A reflection through the y axis is given by     −x x Ry = y y and a reflection through the line y = x is given by     y x Ry=x = x y The matrix representations, relative to the standard basis B, for each of these are given by       1 0 −1 0 0 1 [Rx ]B = [Ry ]B = [Ry=x ]B = 0 −1 0 1 1 0

EXAMPLE 3

Perform the following reflections on the triangle T of Fig. 4. a. Reflection through the x axis. b. Reflection through the y axis. c. Reflection through the line y = x.

Solution

a. The vertices of the triangle in Fig. 4 are given by       0 2 1 v2 = v3 = v1 = 1 1 3 Applying the matrix [Rx ]B to the vertices of the original triangle, we obtain       0 2 1 v2 = v3 = v1 = −1 −1 −3 The image of the triangle is shown in Fig. 9(a). b. Applying the matrix [Ry ]B to the vertices of the original triangle, we obtain       0 −2 −1 v2 = v3 = v1 = 1 1 3 The image of the triangle with this reflection is shown in Fig. 9(b). c. Finally, applying the matrix [Rx=y ]B to the vertices of the original triangle, we obtain       1 1 3    v2 = v3 = v1 = 0 2 1 The image of the triangle is shown in Fig. 9(c).

Revised Confirming Pages

4.6 Application: Computer Graphics

y

y

5

⫺5

y

5

5

x

⫺5

⫺5

5

5

x

⫺5

(a)

261

(b)

⫺5

5

x

⫺5

(c)

Figure 9

Reversing Graphics Operations The operations of scaling, shearing, and reflection are all reversible, and hence the matrix representations for each of these operators are invertible. By Corollary 2 of Sec. 4.4, to reverse one of these operations, we apply the inverse matrix to the transformed image.

EXAMPLE 4

Let S be the linear operator that performs a reflection through the line y = x, followed by a horizontal stretching by a factor of 2. a. Find the matrix representation of S, relative to the standard basis B. b. Find the matrix representation of the reverse operator, again relative to B.

Solution

a. Using the matrices given above for these operations, and by Theorem 14 of Sec. 4.4, the matrix of the transformation, relative to the standard basis for ⺢2 , is given by the product      0 2 0 1 2 0 = [S]B = 1 0 1 0 0 1 b. By Corollary 2 of Sec. 4.4, the matrix which reverses the operation of part (a) is given by     1 0 1 0 −2 [S −1 ]B = ([S]B )−1 = − = 1 0 0 2 −1 2 As we noted in Example 4(a), if a graphics operation S is given by a sequence of linear operators S1 , S2 , . . . , Sn , then S = Sn ◦Sn−1 ◦ · · · ◦S1 The matrix representation, relative to the basis B, is then given by the matrix product [S]B = [Sn ]B [Sn−1 ]B · · · [S1 ]B The reverse process is given by −1 −1 −1 [S]−1 B = [S1 ]B [S2 ]B · · · [Sn ]B

Confirming Pages

262

Chapter 4 Linear Transformations −1 −1 Thus, applying the matrices [S1 ]−1 B , [S2 ]B , . . . , [Sn ]B in succession reverses the process one transformation at a time.

Translation A translation of a point in the plane moves the point vertically, horizontally, or both. For example, to translate the point (1, 3) three units to the right and two units up, add 3 to the x coordinate and 2 to the y coordinate to obtain the point (4, 5).     v1 b1 2 Now let v = be any vector in ⺢ and b = some fixed vector. An v2 b2 operation S: ⺢2 → ⺢2 of the form S(v) = v + b =



v1 + b1 v2 + b2



is called a translation by the vector b. This transformation is a linear operator if and only if b = 0. Consequently, when b = 0, then S cannot be accomplished by means of a 2 × 2 matrix. However, by using homogeneous coordinates, translation of a vector in ⺢2 can be represented by a 3 × 3 matrix. The homogeneous coordinates of a vector in ⺢2 are obtained by adding a third component whose value is 1. Thus,   x are given by the homogeneous coordinates for the vector v = y   x w= y  1   b1 Now, to translate w by the vector b = , we let b2   1 0 b1 A =  0 1 b2  0 0 1     x + b1 1 0 b1 x Aw =  0 1 b2   y  =  y + b2  1 0 0 1 1 To return to ⺢2 , we select the first two components of Aw so that   x + b1 S(v) = y + b2 as desired.   1 . Using homogeneous coordinates, the As an illustration of this, let b = −2 3 × 3 matrix to perform the translation is   1 0 1 A =  0 1 −2  0 0 1 so that



Confirming Pages

4.6 Application: Computer Graphics

 Now let v =

263

 3 . Then 2 

    4 3 1 0 1  0 1 −2   2  =  0  1 1 0 0 1

 4 is the translation of v by the vector b. 0 In the previous illustration the translation can be accomplished with less work by simply adding the vector b to v. The benefits of using a matrix representation are realized when we combine translation with other types of transformations. To do this, we note that all the previous linear operators can be represented by 3 × 3 matrices. For example, the 3 × 3 matrix for reflecting a point (in homogeneous coordinates) through the x axis is   1 0 0  0 −1 0  0 0 1 

The vector S(v) =

EXAMPLE 5

Solution

The matrix for the composition of these    1.5 0 0 1 0 0  0 −1 0   0 1 0  0 0 1 0 0 1

operations is given   1 0 −5 0 1 3 = 0 0 1

by the product 1.5 0 0

 0 −7.5 −1 −3  0 1

The vertices of the original triangle in homogeneous coordinates are given by       0 2 1 v1 =  1  v2 =  1  v3 =  3  1 1 1

y 10 Q P R ⫺10

Find the image of the triangle T of Fig. 4 under a translation by the vector  −5 , followed by a horizontal scaling by a factor of 1.5, followed by b= 3 a reflection through the x axis.

10 x

⫺10

T = RQP

After applying the above matrix to each of these vectors, we obtain       −7.5 −4.5 −6 v1 =  −4  v2 =  −4  v3 =  −6  1 1 1

Figure 10

The resulting triangle, along with the intermediate steps, are shown in Fig. 10.

EXAMPLE 6

Find a 3 × 3 matrix that will transform the triangle shown in Fig. 11(a) to the triangle shown in Fig. 11(b).

Confirming Pages

264

Chapter 4 Linear Transformations

y

y 6

6

9

9

x

x

(b) Triangle T'

(a) Triangle T

Figure 11

Solution

y

Triangle T  is obtained from triangle T through a horizontal scaling by a factor of 3, followed by a vertical scaling by a factor of 2, without changing the left vertex (1, 1). The scalings alone will move the point (1, 1) to (3, 2). One way to correct this is to first translate the triangle so that the left vertex is located at the origin, perform the scaling, and then translate back. The matrix to perform all these operations is the product of the matrices for each transformation. The matrix is given by        3 0 −2 1 0 −1 3 0 0 1 0 0 1 0 1  0 1 1   0 2 0   0 1 0   0 1 −1  =  0 2 −1  0 0 1 0 0 1 0 0 1 0 0 1 0 0 1 Notice that

x

Rotation by 45◦ y

x

Figure 12

  1 0 1 0 1  0 1 1 = 0 1 0 0 0 0 1  that is, the matrix representation for translation by   −1 . representation for translation by −1 

−1 −1 −1  1  1 is the inverse of the matrix 1

Rotation Another common graphics operation is a rotation through an angle θ. See Fig. 12. To describe how a point is rotated, let (x, y) be the coordinates of a point in ⺢2 and θ a real number. From trigonometry, the new coordinates (x  , y  ) of a point (x, y) rotated by θ rad about the origin are given by x  = x cos θ − y sin θ y  = x sin θ + y cos θ If θ > 0, then v is revolved about the origin in a counterclockwise direction. If θ < 0, the direction is clockwise. These equations define a linear operator Sθ : ⺢2 → ⺢2

Confirming Pages

4.6 Application: Computer Graphics

given by

 Sθ

x y



 =

x cos θ − y sin θ x sin θ + y cos θ

265



The matrix of Sθ relative to the standard basis B = {e1 , e2 } for ⺢2 is given by   cos θ − sin θ [Sθ ]B = sin θ cos θ When using homogeneous coordinates, we apply  cos θ − sin θ  sin θ cos θ 0 0

EXAMPLE 7

Solution

y

5

⫺5

Figure 13

Find the image of the triangle of Fig. 4 under a translation by the vector b =   1 , followed by a rotation of 30◦ , or π/6 rad, in the counterclockwise −1 direction. The matrix for the combined operations is given by  √      3 1 − 0 1 0 1 cos π6 − sin π6 0 1 0 1 2 2   √  0 1 −1   sin π 1 3 cos π6 0   0 1 −1  =   0   2 6  2 0 0 1 0 0 1 0 0 1 0 0 1  √  √ 3 1 3 1 − + 2 2 2  2 √ √  1 1 3 3  = −  2 2 2 2  0 0 1 The vertices of the triangle in homogeneous coordinates are given by       0 2 1 v2 =  1  and v3 =  3  v1 =  1  1 1 1

5

⫺5

the matrix  0 0  1

x

After applying the above matrix to each of these vectors, we obtain  √3   3√3   √  3−1 2 2 √     v1 =  12  v2 =  32  and v3 =  3 + 1  1 1 1 The resulting triangle is shown in Fig. 13.

Projection Rendering a picture of a three-dimensional object on a flat computer screen requires projecting points in 3-space to points in 2-space. We discuss only one of many methods to project points in ⺢3 to points in ⺢2 that preserve the natural appearance of an object.

Confirming Pages

266

Chapter 4 Linear Transformations

Parallel projection simulates the shadow that is cast onto a flat surface by a far away light source, such as the sun. Shown in Fig. 14 are rays intersecting an object in 3-space and the projection into 2-space. The orientation of the axes in Fig. 14 is such that the xy plane represents the computer screen. To show how to find the xy coordinates of a projected point, let the vector

y



 xd vd =  yd  zd

x

z Figure 14

represent the direction of the rays. If (x0 , y0 , z0 ) is a point in ⺢3 , then the parametric equations of the line going through the point and in the direction of vd are given by   x(t) = x0 + txd y(t) = y0 + tyd   z(t) = z0 + tzd for all t ∈ ⺢. The coordinates of the projection of (x0 , y0 , z0 ) onto the xy plane are found by letting z(t) = 0. Solving for t, we obtain t =−

z0 zd

Now, substituting this value of t into the first two equations above, we find the coordinates of the projected point, which are given by xp = x0 −

z0 xd zd

yp = y0 −

z0 yd zd

and

zp = 0

The components of vd can also be used to find the angles that the rays make with the z axis and the xz plane. In particular, we have # yd tan ψ = xd

and

tan ϕ =

xd2 + yd2 zd

where ψ is the angle vd makes with the xz plane and φ is the angle made with the z axis. On the other hand, if the angles ψ and φ are given, then these equations can be used to find the components of the projection vector vd .

EXAMPLE 8

Let ψ = 30◦ and φ = 26.6◦ . a. Find the direction vector vd and project the cube, shown in Fig. 15, into ⺢2 . The vertices of the cube are located at the points (0, 0, 1), (1, 0, 1), (1, 0, 0), (0, 0, 0), (0, 1, 1), (1, 1, 1), (1, 1, 0), and (0, 1, 0).

Confirming Pages

4.6 Application: Computer Graphics

267

y

x

z

Figure 15 ◦ b. Find a 3 × 3 matrix that will rotate the (projected) vertices of the cube by 30  2 . and another that will translate the cube by the vector 1

Solution

a. We can arbitrarily set zd = −1. Then yd ◦ ◦ tan ψ = tan 30 ≈ 0.577 = and (tan φ)2 = (tan 26.6 )2 ≈ (0.5)2 = xd2 + yd2 xd so that yd = 0.577xd and xd2 + yd2 = 14 Solving the last two equations gives xd ≈ 0.433 and yd ≈ 0.25, so that the direction vector is   0.433 vd =  0.25  −1 Using the formulas for a projected point given above, we can project each vertex of the cube into ⺢2 . Connecting the images by line segments gives the picture shown in Fig. 16. The projected points are given in Table 1. Table 1 Vertex

Projected Point

(0,0,1)

(0.433, 0.25)

(1,0,1)

(1.433, 0.25)

(1,0,0)

(1, 0)

(0,0,0)

(0, 0)

(0,1,1)

(0.433, 1.25)

(1,1,1)

(1.433, 1.25)

(1,1,0)

(1, 1)

(0,1,0)

(0, 1)

Confirming Pages

268

Chapter 4 Linear Transformations

b. Using homogeneous coordinates, we find the matrices to rotate the cube coun 2 ◦ are given by terclockwise by 30 and translate the cube by the vector 1     cos( π6 ) − sin( π6 ) 0 1 0 2    0 1 1  and  sin( π6 ) cos( π6 ) 0  0 0 1 0 0 1 respectively. Depictions of the results when the original cube is rotated and then the result is translated are shown in Figs. 17 and 18. y

y

y

x

x

Figure 16

x

Figure 17

Figure 18

Exercise Set 4.6 1. Find the matrix representation relative to the standard basis for the linear transformation T: ⺢2 −→ ⺢2 that transforms the triangle with vertices at the points (0, 0), (1, 1), and (2, 0) to the triangle shown in the figure. y a.

c.

y 5

⫺5

5

x

5 ⫺5

⫺5

5

x

⫺5

b.

y 5

⫺5

5

5

⫺5

2. Find the matrix representation relative to the standard basis for the linear transformation T: ⺢2 −→ ⺢2 that transforms the square with vertices at the points (0, 0), (1, 0), (1, 1), and (0, 1) to the polygon shown in the figure. y a.

x

⫺5

5

⫺5

x

Confirming Pages

4.6 Application: Computer Graphics

b.

y 5

⫺5

5

x

⫺5

c.

y 5

⫺5

5

x

⫺5

3. Let T: ⺢2 −→ ⺢2 be the transformation that performs a horizontal stretching by a factor of 3, followed by a vertical contraction by a factor of 2, followed by a reflection through the x axis. a. Find the matrix of T relative to the standard basis. b. Apply the transformation to the triangle with vertices (1, 0), (3, 0), and (2, 2), and give a sketch of the result. c. Find the matrix relative to the standard basis that reverses T. 4. Let T: ⺢2 −→ ⺢2 be the transformation that performs a reflection through the y axis, followed by a horizontal shear by a factor of 3. a. Find the matrix of T relative to the standard basis. b. Apply the transformation to the rectangle with vertices (1, 0), (2, 0), (2, 3), and (1, 3), and give a sketch of the result. c. Find the matrix relative to the standard basis that reverses T. 5. Let T: ⺢2 −→ ⺢2 be the transformation that performs a rotation by 45◦, followed by a reflection through the origin. a. Find the matrix of T relative to the standard basis.

269

b. Apply the transformation to the square with vertices (0, 0), (1, 0), (1, 1), and (0, 1), and give a sketch of the result. c. Find the matrix relative to the standard basis that reverses T. 6. Let T: ⺢2 −→ ⺢2 be the transformation that performs a reflection through the line y = x, followed by a rotation of 90◦ . a. Find the matrix of T relative to the standard basis. b. Apply the transformation to the triangle with vertices (0, 0), (2, 0), and (1, 3), and give a sketch of the result. c. Find the matrix relative to the standard basis that reverses T. d. Describe this transformation in another way. Verify your answer. 7. Let T: ⺢2 −→ ⺢2 be the (nonlinear) transformation  that performs a translation by the  1 , followed by a rotation of 30◦ . vector 1 a. Using homogeneous coordinates, find the 3 × 3 matrix that performs the translation and rotation. b. Apply the transformation to the parallelogram with vertices (0, 0), (2, 0), (3, 1), and (1, 1), and give a sketch of the result. c. Find the matrix that reverses T. 8. Let T: ⺢2 −→ ⺢2 be the (nonlinear) transformation  that performs a translation by the  −4 , followed by a reflection through vector 2 the y axis. a. Using homogeneous coordinates, find the 3 × 3 matrix that performs the translation and reflection. b. Apply the transformation to the trapezoid with vertices (0, 0), (3, 0), (2, 1), and (1, 1), and give a sketch of the result. c. Find the matrix that reverses T. 

9. Let B=

1 1

   −1 , 1

Confirming Pages

270

Chapter 4 Linear Transformations

be a basis for ⺢2 , and let A be the triangle in the xy coordinate system with vertices (0, 0), (2, 2), and (0, 2). a. Find the coordinates of the vertices of A relative to B. b. Let T be the transformation that performs a reflection through the line y = x. Find [T ]SB , where S is the standard basis for ⺢2 . c. Apply the matrix found in part (b) to the coordinates found in part (a). Sketch the result. d. Show the same result is obtained by applying  0 1 to the original coordinates. 1 0 10. Let  B=

1 0

   1 , 1

be a basis for ⺢2 , and let A be the parallelogram in the xy coordinate system with vertices (0, 0), (1, 1), (1, 0), and (2, 1). a. Find the coordinates of the vertices of A relative to B. b. Find the matrix representation relative to B of the transformation T that performs a reflection through the horizontal axis. c. Apply the matrix found in part (b) to the coordinates found in part (a). Write the resulting vectors relative to the standard basis, and sketch the result. d. Find the matrix representation relative to the standard basis for the transformation that performs the same operation on the parallelogram. Apply this matrix to the original coordinates, and verify the result agrees with part (c).

Review Exercises for Chapter 4 1. Let T: ⺢2 −→ ⺢4 be a linear transformation. a. Verify that     3 1 , S= −1 1 is a basis for ⺢2 . b. If  T

 T determine   x T y

1 1

3 −1

 1  2   =  0  and 2   3   2   =  4  −2 



 for all

x y

 ∈ ⺢2

 (Hint: Find the coordinates of c. d. e. f. g.

x y

 relative

to S.) Describe all vectors in N (T ). Is the linear map T one-to-one? Explain. Find a basis for R(T ). Is T onto? Give two reasons. Find a basis for ⺢4 that contains the vectors     −1 1  1   0      and  0   1  1 1 

h. Use the basis B =

1 2

  ,

1 −1

 of ⺢2

and the basis found in part (g) for ⺢4 , call it C, and find a matrix representation for T with respect to the bases B and C.

Confirming Pages

4.6 Application: Computer Graphics

i. Apply the matrix A found in part (h) to an  x . arbitrary vector y 2. Define linear transformations S, T: P3 → P4 and H: P4 → P4 by S(p(x)) = p (0) T (p(x)) = (x + 1)p(x) H (p(x)) = p (x) + p(0) a. Compute H ◦T and S ◦(H ◦T ). b. Find the matrix for S, T, and H relative to the standard bases for P3 and P4 . c. Show that T is one-to-one. d. Find R(T ). 3. Let S, T: ⺢2 → ⺢2 be transformations so S reflects every vector through the x axis and T reflects every vector through the y axis. a. Give definitions for S and T. Show the mappings are linear transformations. b. Find the matrix for S and for T relative to the standard basis for ⺢2 . c. Find the matrix for the linear transformations T ◦S and S ◦T . Describe geometrically the action of T ◦S and S ◦T . 4. a. Let T: M2×2 → M2×2 be defined by   1 3 A T (A) = −1 1 Is T a linear transformation? Is T one-to-one? Is T an isomorphism? b. Let T: M2×2 → M2×2 be defined by   1 0 A T (A) = 1 0 Is T a linear transformation? Is T one-to-one? Show R(T ) is isomorphic to ⺢2 . 5. Let v1 and v2 be linearly independent vectors in ⺢2 and T: ⺢2 → ⺢2 a linear operator such that T (v1 ) = v2 Let B = {v1 , v2 } and

and B

T (v2 ) = v1

= {v2 , v1 }.

271

a. Find [T ]B .  b. Find [T ]BB . 6. Let T: ⺢2 → ⺢2 be the linear operator that   1 projects a vector across the line span −1 and S: ⺢2 → ⺢2 the linear operator that reflects a   1 . Let B vector across the line span 0 denote the standard basis for ⺢2 . a. Find [T ]B and [S]B .     2 −2 . and S b. Find T 3 1 c. Find the matrix representation for the linear operator H : ⺢2 → ⺢2 that reflects a vector 1 and across the subspace span −1   1 . across the subspace span 0   −2 . d. Find H −1 e. Find N (T ) and N (S). f. Find all vectors v such that T (v) = v and all vectors v such that S(v) = v. 7. Let T: ⺢3 → ⺢3 be the linear operator that reflects a vector across the plane    0   1 span  0 ,  1    1 0 The projection of a vector u onto a vector v is the vector u·v projv u = v v·v and the reflection of v across the plane with normal vector n is v − 2 projn v Let B denote the standard basis for ⺢3 . a. Find [T ]B.  −1 b. Find T  2 . 1

Confirming Pages

272

Chapter 4 Linear Transformations

c. Find N (T ). d. Find R(T ). e. Find the matrix relative to B for T n , n ≥ 2. 8. Define a transformation T: P2 → ⺢ by  1 p(x) dx T (p(x)) =

10. Let T: ⺢2 → ⺢2 be a linear operator. a. Show that the line segment between two vectors u and v in ⺢2 can be described by tu + (1 − t)v

0

Show that T is a linear transformation. Compute T (−x 2 − 3x + 2). Describe N (T ). Is T one-to-one? Find a basis for N (T ). Show that T is onto. Let B be the standard basis for P2 and  B  = {1}, a basis for ⺢. Find [T ]BB . g. Compute T (−x 2 − 3x + 2), using the matrix found in part (f). h. Define linear operators T: C (1) [0, 1] → C (1) [0, 1] and S: C (1) [0, 1] → C (1) [0, 1] by d f (x) T (f ) = dx and  x f (t) dt S(f ) = F, where F (x) = a. b. c. d. e. f.

0

Find T (xex ) and S(xex ). Describe S ◦T and T ◦S. 9. Let T: V → V be a linear operator such that T 2 − T + I = 0, where I denotes the identity mapping. Show that T −1 exists and is equal to I − T.

for

0≤t ≤1

b. Show that the image of a line segment under the map T is another line segment. c. A set in ⺢2 is called convex if for every pair of vectors in the set, the line segment between the vectors is in the set. See the figure.

Convex set

Not a convex set

Suppose T: ⺢2 → ⺢2 is an isomorphism and S is a convex set in ⺢2 . Show that T (S) is a convex set. d. Define T: ⺢2 −→ ⺢2 by     2x x = T y y Show that T is an isomorphism. Let    x  2 2 x S= + y = 1 y  Describe the image of the set S under the transformation T.

Chapter 4: Chapter Test In Exercises 1–40, determine whether the statement is true or false. 1. The transformation T: ⺢2 → ⺢2 defined by  T

x y



 =

is a linear transformation.

2x − 3y x+y+2



2. The transformation T: ⺢ → ⺢ defined by T (x) = 2x − 1 is a linear transformation. 3. If b = 0, then the transformation T: ⺢ → ⺢ defined by T (x) = mx + b is a linear transformation. 4. If A is an m × n matrix, then T defined by T (v) = Av is a linear transformation from ⺢n into ⺢m .

Confirming Pages

4.6 Application: Computer Graphics

5. Let A be a fixed matrix in Mn×n . Define a transformation T: Mn×n → Mn×n by T (B) = (B + A)2 − (B + 2A)(B − 3A) If A2 = 0, then T is a linear transformation.     0 1 . If T: ⺢2 −→ ⺢2 is and v = 6. Let u = 1 0 a linear operator and T (u + v) = v

T (2u − v) = u + v

and 

then T (u) =

2 3 1 3



7. If T: ⺢2 → ⺢2 is defined by      x 2 −4 x = T y 1 −2 y then T is an isomorphism. 8. If T: V −→ W is a linear transformation and {v1 , . . . , vn } is a linearly independent set in V, then {T (v1 ), . . . , T (vn )} is a linearly independent subset of W . 9. The vector spaces P8 and M3×3 are isomorphic. 10. If a linear map T: P4 −→ P3 is defined by T (p(x)) = p (x), then T is a one-to-one map. 11. If A is an n × n invertible matrix, then as a mapping from ⺢n into ⺢n the null space of A consists of only the zero vector. 12. The linear operator T: ⺢2 → ⺢2 defined by     x−y x = T 0 y is one-to-one. 13. If T: ⺢2 → ⺢2 is the transformation that reflects each vector through the origin, then the matrix for T relative to the standard basis for ⺢2 is   −1 0 0 −1 14. A linear transformation preserves the operations of vector addition and scalar multiplication.

273

15. Every linear transformation between finite dimensional vector spaces can be defined using a matrix product. 16. A transformation T: V → W is a linear transformation if and only if T (c1 v1 + c2 v2 ) = c1 T (v1 ) + c2 T (v2 ) for all vectors v1 and v2 in V and scalars c1 and c2 . 17. If f: ⺢ −→ ⺢ is a linear operator and φ: ⺢2 −→ ⺢2 is defined by φ(x, y) = (x, y − f (x)) then the mapping φ is an isomorphism. 18. Let U, V , and W be finite dimensional vector spaces. If U is isomorphic to V and V is isomorphic to W , then U is isomorphic to W . 19. If T: V → V is a linear operator and u ∈ N (T ), then T (cu + v) = T (v) for all v ∈ V and scalars c. 20. If P : ⺢3 → ⺢3 is the projection defined by     x x P   y  =  y  0 z then P 2 = P . 21. If T: V → W is a linear transformation between vector spaces such that T assigns each element of a basis for V to the same element of W , then T is the identity mapping. 22. If T: ⺢4 → ⺢5 and dim(N (T )) = 2, then dim(R(T )) = 3. 23. If T: ⺢4 → ⺢5 and dim(R(T )) = 2, then dim(N (T )) = 2. 24. If T: ⺢3 → ⺢3 is defined by     2x − y + z x  x T  y  =  y−x z

Confirming Pages

274

Chapter 4 Linear Transformations

then the matrix for T −1 relative to the standard basis for ⺢3 is   0 1 0  0 0 1  −1 −2 1 25. If T: ⺢2 → ⺢2 is defined by     2x + 3y x = T −x + y y B = {e1 , e2 }, and B  = {e2 , e1 }, then   −1 1 B [T ]B = 2 3 26. There exists a linear transformation T between vector spaces such that T (0) = 0. 27. The linear transformation T: ⺢3 → ⺢3 defined by     x x T   y  =  0  y z projects each vector in ⺢3 onto the xy plane. 28. The linear operator T: ⺢2 → ⺢2 defined by      x 0 1 x = T y 1 0 y reflects each vector in ⺢2 across the line y = x. 29. Let T: V → W be a linear transformation and B = {v1 , . . . , vn } a basis for V. If T is onto, then {T (v1 ), . . . , T (vn )} is a basis for W. 30. The vector space P2 is isomorphic to the subspace    of ⺢5   a            b     W =  0  a, b, c ∈ ⺢        c      0  31. If T: V → V is the identity transformation, then the matrix for T relative to any pair of bases B and B  for V is the identity matrix. 32. If T: ⺢3 → ⺢3 is defined by     x+y+z x T  y  =  y − x  y z then dim(N (T )) = 1.

33. If T: P2 → P2 is defined by T (ax 2 + bx + c) = 2ax + b then a basis for N (T ) is {−3}. 34. If T: M2×2 → M2×2 is defined by T (A) = A2 − A then N (T ) = {0}. 35. If T: P3 → P3 is defined by T (p(x)) = p (x) − xp (x) then T is onto. 36. If T: P3 → P3 is defined by T (p(x)) = p (x) − xp (x) then q(x) = x 2 is in R(T ). 37. The linear operator T: ⺢3 → ⺢3    3 −3 x 2 T   y  =  1 3 −1 z

defined  0 1  1

by  x y  z

is an isomorphism. 38. If A is an m × n matrix and T: ⺢n → ⺢m is defined by T (v) = Av then the range of T is the set of all linear combinations of the column vectors of A. 39. If A is an m × n matrix with m > n and T: ⺢n → ⺢m is defined by T (v) = Av then T cannot be one-to-one. 40. If A is an m × n matrix with m > n and T: ⺢n → ⺢m is defined by T (v) = Av then T cannot be onto.

Confirming Pages

CHAPTER

Eigenvalues and Eigenvectors CHAPTER OUTLINE

5.1 5.2 5.3 5.4

Eigenvalues and Eigenvectors 276 Diagonalization 287 Application: Systems of Linear Differential Equations Application: Markov Chains 310

300

Markov chain is a mathematical model used to describe a random process that, at any given time t = 1, 2, 3, . . . , is in one of a finite number of states. Between the times t and t + 1 the process moves from state j to state i with a probability pij . Markov processes are also memoryless; that is, the next state of the system depends only on the current state. As an example, consider a city C with surrounding residential areas N, S, E, and W . Residents can move between any two locations or stay in their current location, with fixed probabilities. In this case a state is the location of a resident at any given time. The state diagram shown in Fig. 1 describes the situation with the probabilities of moving from one location to another shown in the corresponding transition matrix A = (pij ). For example, entry p34 = 0.2 is the probability that U.S. Geological Survery/DAL a resident in region E moves to region S. Since a resident is assumed to be living in one of the five regions, the probability of being in one of these regions is 1, and hence each column sum of A is equal to 1. A square matrix with each entry between 0 and 1 and column sums all equal to 1 is called a stochastic matrix. The initial distribution of the population is called the initial probability vector. Assume that the initial population distribution is given by the vector   0.3  0.2     v=  0.1   0.2  0.2

A

C

N

W

S

E

Figure 1

C

N

S

0.3 C 0.2 N   0.1 S  0.2 E 0.2 W

0.2 0.4 0.2 0.1 0.1

0.4 0.1 0.2 0.2 0.1



E 0.1 0.2 0.2 0.3 0.2

W  0.1 0.1  0.1   0.2 0.5

275

Confirming Pages

Chapter 5 Eigenvalues and Eigenvectors

Then the population distribution after one time step is Av, after two time steps is A(Av) = A2 v, and so on. For example, after 10 time steps, the population distribution (rounded to two decimal places) is   0.21  0.20    10  A v=  0.16   0.20  0.23 Notice that the sum of the entries of the population distribution vector is equal to 1. Starting with some initial distribution vector, the long-term behavior of the Markov chain, that is, An v as n tends to infinity, gives the limiting population distribution in the five regions into the future. When An v approaches a distribution vector s as n tends toward infinity, we say that s is the steady-state vector. If a transition matrix for a Markov chain is a stochastic matrix with positive terms, then for any initial probability vector v, there is a unique steady-state vector s. Moreover, if s is the steady-state vector, then As = s. Finding the steady-state vector is equivalent to solving the matrix equation Ax = λx with λ = 1. In general, if there is a scalar λ and a nonzero vector v such that Av = λv, then λ is called an eigenvalue for the matrix A and v is an eigenvector corresponding to the eigenvalue λ. In our Markov chain example, the steady-state vector corresponds to the eigenvalue λ = 1 for the transition matrix A. In the last decade the growth in the power of modern computers has, quite miraculously, made it possible to compute the eigenvalues of a matrix with rows and columns in the billions. Google’s page rank algorithm is essentially a Markov chain with transition matrix consisting of numerical weights for each site on the World Wide Web used as a measure of its relative importance within the set. The algorithm was developed by Larry Page and Sergey Brin, the founders of Google. For any n × n matrix A, there exists at least one number-vector pair λ, v such that Av = λv (although λ may be a complex number). That is, the product of A and v is a scaling of the vector v. Many applications require finding such number-vector pairs.

5.1

ß

276

Eigenvalues and Eigenvectors One of the most important problems in linear algebra is the eigenvalue problem. It can be stated thus: If A is an n × n matrix, does there exist a nonzero vector v such that Av is a scalar multiple of v?

DEFINITION 1

Eigenvalue and Eigenvector Let A be an n × n matrix. A number λ is called an eigenvalue of A provided that there exists a nonzero vector v in ⺢n such that Av = λv

Confirming Pages

5.1 Eigenvalues and Eigenvectors

277

Every nonzero vector satisfying this equation is called an eigenvector of A corresponding to the eigenvalue λ. The zero vector is a trivial solution to the eigenvalue equation for any number λ and is not considered as an eigenvector. As an illustration, let   1 2 A= 0 −1 Observe that      1 1 1 2 = 0 0 0 −1   1 so v1 = is an eigenvector of A corresponding to the eigenvalue λ1 = 1. We 0 also have        1 −1 1 1 2 = −1 = −1 1 −1 0 −1   1 so v2 = is another eigenvector of A corresponding to the eigenvalue λ2 = −1. −1 In Example 1 we show how to find eigenvalues and eigenvectors for a 2 × 2 matrix.

EXAMPLE 1



Let A=

Solution

0 1 1 0



a. Find the eigenvalues of A. b. Find the eigenvectors corresponding to each of the eigenvalues found in part (a).   x a. The number λ is an eigenvalue of A if there is a nonzero vector v = y such that          λx y x x 0 1 = which is equivalent to =λ λy x y y 1 0 This matrix equation is equivalent to the homogeneous linear system −λx + y = 0 x − λy = 0 By Theorem 17 of Sec. 1.6, the linear system has a nontrivial solution if and only if



−λ 1



1 −λ = 0

Confirming Pages

278

Chapter 5 Eigenvalues and Eigenvectors

Consequently, λ is an eigenvalue of A if and only if λ2 − 1 = 0

so that λ1 = 1 and λ2 = −1  x is an eigenvector if b. For λ1 = 1, a vector v1 = y      x x 0 1 = y y 1 0 

This yields the linear system    

−x + y = 0 t

t ∈ ⺢ with solution set S= x−y=0 t

  t Thus, any vector of the form v1 = , for t = 0, is an eigenvector corret sponding to the eigenvalue λ1 = 1. In a similar way, we find that any vector   t , for t = 0, is an eigenvector of A corresponding to of the form v2 = −t the eigenvalue λ2 = −1. Specific eigenvectors of A can be found by choosing any value for t so that neither v1 nor v2 is the zero vector. For example, letting t = 1, we know that   1 v1 = 1 is an eigenvector corresponding to λ1 = 1 and   1 v2 = −1 is an eigenvector corresponding to λ2 = −1.

Geometric Interpretation of Eigenvalues and Eigenvectors A nonzero vector v is an eigenvector of a matrix A only when Av is a scaling of   1 −1 . Using the techniques just introduced, the vector v. For example, let A = 2 4 the eigenvalues of A are λ1 = 2 and λ2 = 3 with corresponding eigenvectors     1 1 and v2 = , respectively. Observe that v1 = −1 −2      2 1 1 −1 = 2v1 = Av1 = −2 −1 2 4 

and Av2 =

1 2

−1 4



1 −2



 =

3 −6

 = 3v2

Confirming Pages

5.1 Eigenvalues and Eigenvectors

y 5

v1

⫺5

v2

⫺6

5

x

Av1

Av2

Figure 2

279

In Fig. 2, we provide sketches of the vectors v1 , v2 , Av1 , and Av2 to underscore that the action of A on each of its eigenvectors is a scaling. Observe that this is not the   1 , then case for an arbitrary vector. For example, if v = 1      0 1 1 −1 = Av = 6 1 2 4

Eigenspaces Notice that in Example 1, for each of the eigenvalues there are infinitely many eigenvectors. This is the case in general. To show this, let v be an eigenvector of the matrix A corresponding to the eigenvalue λ. If c is any nonzero real number, then A(cv) = cA(v) = c(λv) = λ(cv) so cv is another eigenvector associated with the eigenvalue λ. Notice that all eigenvectors corresponding to an eigenvalue are parallel but can have opposite directions. Building on the procedure used in Example 1, we now describe a general method for finding eigenvalues and eigenvectors. If A is an n × n matrix, then Av = λv for some number λ if and only if Av − λv = 0

that is

(A − λI )v = Av − λI v = 0

Again by Theorem 17, of Sec. 1.6, this equation has a nontrivial solution if and only if det(A − λI ) = 0 We summarize this result in Theorem 1. THEOREM 1

The number λ is an eigenvalue of the matrix A if and only if det(A − λI ) = 0 The equation det(A − λI ) = 0 is called the characteristic equation of the matrix A, and the expression det(A − λI ) is called the characteristic polynomial of A. If A is an n × n matrix and λ is an eigenvalue of A, then the set Vλ = {v ∈ ⺢n | Av = λv} is called the eigenspace of A corresponding to λ. Notice that Vλ is the union of the set of eigenvectors corresponding to λ and the zero vector. We have already shown that Vλ is closed under scalar multiplication. Therefore, to show that Vλ is a subspace of ⺢n , we need to show that it is also closed under addition. To see this, let u and v be vectors in Vλ ; that is, Au = λu and Av = λv for a particular eigenvalue λ. Then A(u + v) = Au + Av = λu + λv = λ(u + v)

Confirming Pages

280

Chapter 5 Eigenvalues and Eigenvectors

Alternatively, the set Vλ = {v ∈ ⺢n | Av = λv} = {v ∈ ⺢n | (A − λI )v = 0} = N (A − λI ) Since Vλ is the null space of the matrix A − λI , by Theorem 3 of Sec. 4.2 it is a subspace of ⺢n .

EXAMPLE 2

Find the eigenvalues and corresponding eigenvectors of   2 −12 A= 1 −5 Give a description of the eigenspace corresponding to each eigenvalue.

Solution

By Theorem 1 to find the eigenvalues, we solve the characteristic equation



2−λ −12

det(A − λI ) =

1 −5 − λ

= (2 − λ)(−5 − λ) − (1)(−12) = λ2 + 3λ + 2 = (λ + 1)(λ + 2) = 0 Thus, the eigenvalues are λ1 = −1 and λ2 = −2. To find the eigenvectors, we need to find all nonzero vectors in the null spaces of A − λ1 I and A − λ2 I . First, for λ1 = −1,       3 −12 1 0 2 −12 = + A − λ1 I = A + I = 1 −4 0 1 1 −5 The null space of A + I is found by row-reducing the augmented matrix     3 −12 0 1 −4 0 to 1 −4 0 0 0 0 

 4t

t ∈ ⺢ . Choosing The solution set for this linear system is given by S = t

  4 t = 1, we obtain the eigenvector v1 = . Hence, the eigenspace corresponding 1 to λ1 = −1 is 

  4

t is any real number Vλ1 = t 1

For λ2 = −2,   4 −12 A − λ2 I = 1 −3   3 In a similar way we find that the vector v2 = is an eigenvector corresponding 1 to λ2 = −2. The corresponding eigenspace is

Confirming Pages

5.1 Eigenvalues and Eigenvectors

Vλ2

281



  3

t is any real number = t 1

  4 The eigenspaces Vλ1 and Vλ2 are lines in the direction of the eigenvectors 1   3 , respectively. The images of the eigenspaces, after multiplication by and 1     3 4 are scalar and A A, are the same lines, since the direction vectors A 1 1     3 4 , respectively. and multiples of 1 1

In Example 3 we illustrate how the eigenspace associated with a single eigenvalue can have dimension greater than 1.

EXAMPLE 3

Find the eigenvalues of



1  0 A=  1 1

0 1 0 0

 0 0 5 −10   2 0  0 3

and find a basis for each of the corresponding eigenspaces. Solution

The characteristic equation of A is

1−λ 0 0

0 1 − λ 5 det(A − λI ) =

1 0 2 − λ

1 0 0

0 −10 0 3−λ







= (λ − 1)2 (λ − 2)(λ − 3) = 0





Thus, the eigenvalues are λ1 = 1

λ2 = 2

and

λ3 = 3

Since the exponent of the factor λ − 1 is 2, we say that the eigenvalue λ1 = 1 has algebraic multiplicity 2. To find the eigenspace for λ1 = 1, we reduce the matrix     1 0 0 2 0 0 0 0  0 0 1 −2   0 0 5 −10     to A − (1)I =   0 0 0  1 0 1 0  0  0 0 0 0 1 0 0 2

Confirming Pages

282

Chapter 5 Eigenvalues and Eigenvectors

Hence, the eigenspace corresponding to λ1 = 1 is    

  −2

0          0 

1  s, t ∈ ⺢ +t V1 = s   2 

 0    

   1

0 Observe that the two vectors 

 0  1     0  0

 −2  0     2  1 

and

are linearly independent and hence form a basis for Vλ1 . Since dim(Vλ1 ) = 2, we say that λ1 has geometric multiplicity equal to 2. Alternatively, we can write     −2  0       1  ,  0  Vλ1 = span   0   2       1 0 Similarly, the eigenspaces corresponding to λ2 = 2 and λ3 = 3 are, respectively,     0  0            −5  5     and V = span Vλ2 = span  λ3  1   0            0 1

In Example 3 the algebraic and geometric multiplicities of each eigenvalue are equal. This is not the case in general. For  if  example, 1 1 A= 0 1 then the characteristic equation is (λ − 1)2 = 0. Thus, λ = 1 has algebraic multiplicity 2. However, 

 t

t ∈ ⺢ Vλ = 0

so λ = 1 has geometric multiplicity 1. Although eigenvectors are always nonzero, an eigenvalue can be zero. Also, as mentioned at the beginning of this section, eigenvalues can be complex numbers. These cases are illustrated in Example 4.

EXAMPLE 4

Find the eigenvalues of

 0 0 0 A =  0 0 −1  0 1 0 

Confirming Pages

5.1 Eigenvalues and Eigenvectors

Solution

The characteristic equation is

−λ 0 0

det(A − λI ) =

0 −λ −1

0 1 −λ

283





= −λ3 − λ = −λ(λ2 + 1) = 0



Thus, the eigenvalues are λ1 = 0, λ2 = i, and λ3 = −i. The corresponding eigenvectors are       0 0 1  1   1   0  and i −i 0

A fact that will be useful in the next section has to do with the eigenvalues of a square triangular matrix. For example, let   2 4 A= 0 −3 Since det(A − λI ) = 0 if and only if (2 − λ)(−3 − λ) = 0, we see that the eigenvalues of A are precisely the diagonal entries of A. In general, we have the following result. PROPOSITION 1

The eigenvalues of an n × n triangular matrix are the numbers on the diagonal. Proof Let A be an n × n triangular matrix. By Theorem 13 of Sec. 1.6, the characteristic polynomial is given by det(A − λI ) = (a11 − λ)(a22 − λ) · · · (ann − λ) Hence, det(A − λI ) = 0 if and only if λ1 = a11 , λ2 = a22 , . . . , λn = ann .

Eigenvalues and Eigenvectors of Linear Operators The definitions of eigenvalues and eigenvectors can be extended to linear operators.

DEFINITION 2

Eigenvalue and Eigenvector of a Linear Operator Let V be a vector space and T:V −→ V be a linear operator. A number λ is an eigenvalue of T provided that there is a nonzero vector v in V such that T (v) = λv. Every nonzero vector that satisfies this equation is an eigenvector of T corresponding to the eigenvalue λ. As an illustration define T: P2 → P2 by T (ax 2 + bx + c) = (−a + b + c)x 2 + (−b − 2c)x − 2b − c Observe that T (−x 2 + x + 1) = 3x 2 − 3x − 3 = −3(−x 2 + x + 1)

Confirming Pages

284

Chapter 5 Eigenvalues and Eigenvectors

so p(x) = −x 2 + x + 1 is an eigenvector of T corresponding to the eigenvalue λ = −3. Example 5 is from ordinary differential equations.

EXAMPLE 5

Interpret the solutions to the equation f  (x) = kf (x) as an eigenvalue problem of a linear operator.

Solution

Let D denote the collection of all real-valued functions of one variable that have derivatives of all orders. Examples of such functions are polynomials, the trigonometric functions sin(x) and cos(x), and the natural exponential function ex on ⺢. Define a linear operator T: D −→ D by T (f (x)) = f  (x) Then λ is an eigenvalue of T if there is a function f (x), not identically zero, such that T (f (x)) = λf (x). That is, f (x) satisfies the differential equation f  (x) = λf (x) Nonzero solutions to this differential equation are eigenvectors of the operator T, called eigenfunctions, corresponding to the eigenvalue λ. The general solution to this equation is given by f (x) = keλx where k is an arbitrary constant. This class of functions is a model for exponential growth and decay with extensive applications.

Fact Summary Let A be an n × n matrix. 1. The number λ is an eigenvalue of A if and only if det(A − λI ) = 0. 2. The expression det(A − λI ) is a polynomial of degree n. 3. If λ is an eigenvalue of A and c is a nonzero scalar, then cλ is another eigenvalue of A. 4. If λ is an eigenvalue of A, then the eigenspace Vλ = {v ∈ ⺢n | Av = λv} is a subspace of ⺢n .

Confirming Pages

5.1 Eigenvalues and Eigenvectors

285

5. The eigenspace corresponding to λ is the null space of the matrix A − λI . 6. The eigenvalues of a square triangular matrix are the diagonal entries.

Exercise Set 5.1 In Exercises 1–6, a matrix A and an eigenvector v are given. Find the corresponding eigenvalue directly by solving Av = λv.     0 3 0 v= 1. A = 1 1 3     −1 −1 1 v= 2. A = 1 0 −2     1 −3 2 3 3. A =  −1 −2 1  v =  0  1 −3 2 3   1 0 1 0  4. A =  3 2 3 0 −1  4  −3 v= 1  4   1 0 1 1  0 1 0 0   5. A =   1 1 0 0  0 1 0 1   −1  0   v=  −1  1   1 1 1 0  −1 −1 0 −1   6. A =   −1 1 0 1  0 −1 −1 0   0  1   v=  −1  0

In Exercises 7–16, a matrix A is given. a. Find the characteristic equation for A. b. Find the eigenvalues of A. c. Find the eigenvectors corresponding to each eigenvalue. d. Verify the result of part (c) by showing that Avi = λi vi .   −2 2 7. A = 3 −3   −2 −1 8. A = −1 −2   1 −2 9. A = 0 1   0 2 10. A = −1 −3   −1 0 1 0  11. A =  0 1 0 2 −1   0 2 0 12. A =  0 −1 1  0 0 1   2 1 2 13. A =  0 2 −1  0 1 0   1 1 1 14. A =  0 1 0  0 0 1   −1 0 0 0  0 2 0 0   15. A =   0 0 −2 0  0 0 0 4

Confirming Pages

286

Chapter 5 Eigenvalues and Eigenvectors



3  0 16. A =   0 0

2 1 0 0

 3 −1 2 1   2 0  0 −1

17. Show that if λ2 + bλ + c is the characteristic polynomial of the 2 × 2 matrix A, then b = −tr(A) and c = det(A). 18. Let A be an invertible matrix. Show that if λ is an eigenvalue of A, then 1/λ is an eigenvalue of A−1 . 19. Let A be an n × n matrix. Show that A is not invertible if and only if λ = 0 is an eigenvalue of A. 20. Let V be a vector space with dim(V ) = n and T: V −→ V a linear operator. If λ is an eigenvalue of T with geometric multiplicity n, then show that every nonzero vector of V is an eigenvector. 21. Let A be an idempotent matrix. Show that if λ is an eigenvalue of A, then λ = 0 or λ = 1. 22. Show that A and At have the same eigenvalues. Give an example to show A and At can have different eigenvectors. 23. Show that if there is a positive integer n such that An = 0, then λ = 0 is the only eigenvalue of A.   1 0 . Define an operator 24. Let A = 0 −1 T: M2×2 → M2×2 by T (B) = AB − BA   0 1 is an eigenvector a. Show that e = 0 0 corresponding to the eigenvalue λ = 2.   0 0 is an eigenvector b. Show that f = 1 0 corresponding to the eigenvalue λ = −2. 25. Let A and B be n × n matrices with A invertible. Show that AB and BA have the same eigenvalues. 26. Show that no such matrices A and B exist such that AB − BA = I

27. Show that the eigenvalues of a square triangular matrix are the diagonal entries of the matrix. 28. Let λ be an eigenvalue of A. Use mathematical induction to show that for all n in the set of all natural numbers ⺞, if λ is an eigenvalue of A, then λn is an eigenvalue of An . What can be said about corresponding eigenvectors? 29. Let C = B −1 AB. Show that if v is an eigenvector of C corresponding to the eigenvalue λ, then Bv is an eigenvector of A corresponding to λ. 30. Let A be an n × n matrix and suppose v1 , . . . , vm are eigenvectors of A. If S = span{v1 , . . . , vm }, show that if v ∈ S, then Av ∈ S. 31. Let T: ⺢2 → ⺢2 be the linear operator that reflects a vector through the x axis. Find the eigenvalues and corresponding eigenvectors for T. 32. Define a linear operator T: ⺢2 → ⺢2 by     y x = T x y Show that the only eigenvalues of T are λ = ±1. Find the corresponding eigenvectors. 33. Define a linear operator T: ⺢2 → ⺢2 by      x cos θ − sin θ x = T y sin θ cos θ y That is, the action of T is a counterclockwise rotation of a vector by a nonnegative angle θ. Argue that if θ = 0, π, then T has no real eigenvalues; if θ = 0, then λ = 1 is an eigenvalue; and if θ = π, then λ = −1 is an eigenvalue. 34. Let D denote the function space of all real-valued functions that have two derivatives, and define a linear operator T on D by T (f ) = f  − 2f  − 3f a. Show that for each k, the function f (x) = ekx is an eigenfunction for the operator T. b. Find the corresponding eigenvalues for each eigenfunction f (x) = ekx .

Confirming Pages

5.2 Diagonalization

c. Find two nonzero functions f such that 

a. Find the matrix representation for T relative to the basis B. b. Find the matrix representation for T relative to the basis B  . c. Show that the eigenvalues for the matrices found in parts (a) and (b) are the same.



f (x) − 2f (x) − 3f (x) = 0 35. Define a linear operator T: P2 → P2 by T (ax 2 + bx + c) = (a − b)x 2 + cx Define two ordered bases for P2 by B = {x − 1, x + 1, x 2 } and B  = {x + 1, 1, x 2 }. ß

5.2

287

Diagonalization Many applications of linear algebra involve factoring a matrix and writing it as the product of other matrices with special properties. For example, in Sec. 1.7, we saw how the LU factorization of a matrix can be used to develop efficient algorithms for solving a linear system with multiple input vectors. In this section, we determine if a matrix A has a factorization of the form A = P DP −1 where P is an invertible matrix and D is a diagonal matrix. The ideas presented here build on the concept of similarity of matrices, which we discussed in Sec. 4.5. Recall that if A and B are n × n matrices, then A is similar to B if there exists an invertible matrix P such that B = P −1 AP If B is a diagonal matrix, then the matrix A is called diagonalizable. Observe that if D is a diagonal matrix, then A is diagonalizable if either D = P −1 AP

or

A = PDP−1

for some invertible matrix P . One of the immediate benefits of diagonalizing a matrix A is realized when computing powers of A. This is often necessary when one is solving systems of differential equations. To see this, suppose that A is diagonalizable with A = P DP −1 Then

A2 = (P DP −1 )(P DP −1 ) = P D(P −1 P )DP −1 = P D 2 P −1 Continuing in this way (see Exercise 27), we see that Ak = P D k P −1 for any positive whole number k. Since D is a diagonal matrix, the entries of D k are simply the diagonal entries of D raised to the k power. As we shall soon see, diagonalization of a matrix A depends on the number of linearly independent eigenvectors, and fails when A is deficient in this way. We note that a connection does not exist between a matrix being diagonalizable and the matrix having an inverse. A square matrix has an inverse if and only if the matrix has only nonzero eigenvalues (see Exercise 19 of Sec. 5.1).

Confirming Pages

288

Chapter 5 Eigenvalues and Eigenvectors

EXAMPLE 1

Solution

Let

 1 2 0 0  A= 2 1 0 0 −3 Show that A is diagonalizable with   1 1 0 P =  −1 1 0  0 0 1 

The inverse matrix is given by   P −1 = 

1 2 1 2

0 so that

 0  1 0  2 0 1

− 12

 −1 0 0  0  P −1 AP =  0 3  0 0 −3 

Therefore, the matrix A is diagonalizable.

The diagonal entries of the matrix P −1 AP, in Example 1, are the eigenvalues of the matrix A, and the column vectors of P are the corresponding eigenvectors. For example, the product of A and the first column vector of P is given by        1 1 1 2 0 1 0   −1  = −1  −1  A  −1  =  2 1 0 0 0 0 −3 0 Similarly, the second and third diagonal entries of P −1 AP are the eigenvalues of A with corresponding eigenvectors the second and third column vectors of P, respectively. With Theorem 2 this idea is extended to n × n matrices.

THEOREM 2

An n × n matrix A is diagonalizable if and only if A has n linearly independent eigenvectors. Moreover, if D = P −1 AP , with D a diagonal matrix, then the diagonal entries of D are the eigenvalues of A and the column vectors of P are the corresponding eigenvectors. Proof First suppose that A has n linearly independent eigenvectors v1 , v2 , . . . , vn , corresponding to the eigenvalues λ1 , λ2 , . . . , λn . Note that the

Confirming Pages

5.2 Diagonalization

289

eigenvalues may not all be distinct. Let       p11 p12 p1n  p21   p22   p2n     v1 =  v2 =  ... vn =   ...   ...   ...  pn1 pn2 pnn and define the n × n matrix P so that the ith column vector is vi . Since the column vectors of P are linearly independent, by Theorem 9 of Sec. 2.3 the matrix P is invertible. Next, since the ith column vector of the product AP is APi = Avi = λi vi we have

   AP =   

λ1 p11 λ1 p21 .. .

λ2 p12 λ2 p22 .. .

λ1 pn1

λ2 pn2

p11 p21 .. .

p12 p22 .. .

pn1 = PD

pn2

  = 

... ... .. .

λn p1n λn p2n .. .

    

. . . λn pnn  λ1 . . . p1n  0 . . . p2n   ..   .. .. . .  . . . . pnn 0

0 λ2 .. .

... ... .. .

0 0 .. .

0

. . . λn

    

where D is a diagonal matrix with diagonal entries the eigenvalues of A. So AP = P D and multiplying both sides on the left by P −1 gives P −1 AP = D The matrix A is similar to a diagonal matrix and is therefore diagonalizable. Conversely, suppose that A is diagonalizable, that is, a diagonal matrix D and an invertible matrix P exist such that D = P −1 AP As above, denote the column vectors of the matrix P by v1 , v2 , . . . , vn and the diagonal entries of D by λ1 , λ2 , . . . , λn . Since AP = P D, for each i = 1, . . . , n, we have Avi = λi vi Hence, v1 , v2 , . . . , vn are eigenvectors of A. Since P is invertible, then by Theorem 9 of Sec. 2.3 the vectors v1 , v2 , . . . , vn are linearly independent.

EXAMPLE 2

Use Theorem 2 to diagonalize the matrix   1 0 0 A =  6 −2 0  7 −4 2

Confirming Pages

290

Chapter 5 Eigenvalues and Eigenvectors

Solution

Since A is a triangular matrix, by Proposition 1 of Sec. 5.1, the eigenvalues of the matrix A are the diagonal entries λ1 = 1

λ2 = −2

and

λ3 = 2

The corresponding eigenvectors, which are linearly independent, are given, respectively, by       1 0 0 v2 =  1  and v3 =  0  v1 =  2  1 1 1 Therefore, by Theorem 2, D = P −1 AP , where   1 0 0 and D =  0 −2 0  0 0 2



1 0 P = 2 1 1 1

 0 0  1

To verify that D = P −1 AP , we can avoid finding P −1 by showing that P D = AP In this case,

 1 1 0 0 PD =  2 1 0  0 0 1 1 1

EXAMPLE 3

Solution



  1 0 0 −2 0  =  2 1 0 2  1 = 6 7 = AP

 0 0 −2 0  −2 2   1 0 0 0 0 −2 0   2 1 0  1 1 1 −4 2

   −1 1 0 0 1 1 and B =  0 −1 1  A= 1 0 1  0 0 2 1 1 0 Show that A is diagonalizable but that B is not diagonalizable.

Let



To find the eigenvalues of A, we solve the characteristic equation   −λ 1 1 1  det(A − λI ) = det  1 −λ 1 1 −λ = −(λ + 1)2 (λ − 2) = 0 Thus, the eigenvalues of A are λ1 = −1, with algebraic multiplicity 2, and λ2 = 2, with algebraic multiplicity 1. To find the eigenvectors, we find the null space of

Confirming Pages

5.2 Diagonalization

291

A − λI for each eigenvalue. For λ1 = −1 we reduce the matrix     1 1 1 1 1 1  0 0 0   1 1 1  to 0 0 0 1 1 1 Hence,

   −1   −1 N (A + I ) = span  1 ,  0    1 0 In a similar manner we find    1  N (A − 2I ) = span  1    1      1 −1 −1 Since the three vectors  1 ,  0 , and  1  are linearly independent, 1 1 0 by Theorem 2 the matrix A is diagonalizable. Using the same approach, we find that B has the same characteristic polynomial and hence the same eigenvalues. However, in this case      1   1  and N (B − 2I ) = span  3  N (B + I ) = span  0      9 0 Since B does not have three linearly independent eigenvectors, by Theorem 2, B is not diagonalizable.

The matrix P that diagonalizes an n × n matrix A is not unique. For example, if the columns of P are permuted, then the resulting matrix also diagonalizes A. As an illustration, the matrix A of Example 3 is diagonalized by     −1 −1 1 −1 0 0 0 1  with P −1 AP =  0 −1 0  P = 1 0 1 1 0 0 2 However, if Q is the matrix obtained from interchanging columns 2 and 3 of P , then Q also diagonalizes A, with   −1 0 0 0  Q−1 AQ =  0 2 0 0 −1 Notice, in this case, that the second and third diagonal entries are also interchanged. Theorem 3 gives sufficient conditions for a matrix to be diagonalizable.

Confirming Pages

292

Chapter 5 Eigenvalues and Eigenvectors

THEOREM 3

Let A be an n × n matrix, and let λ1 , λ2 , . . . , λn be distinct eigenvalues with corresponding eigenvectors v1 , v2 , . . . , vn . Then the set {v1 , v2 , . . . , vn } is linearly independent. Proof The proof is by contradiction. Assume that λ1 , λ2 , . . . , λn are distinct eigenvalues of A with corresponding eigenvectors v1 , v2 , . . . , vn , and assume that the set of eigenvectors is linearly dependent. Then by Theorem 5 of Sec. 2.3, at least one of the vectors can be written as a linear combination of the others. Moreover, the eigenvectors can be reordered so that v1 , v2 , . . . , vm , with m < n, are linearly independent, but v1 , v2 , . . . , vm+1 are linearly dependent with vm+1 a nontrivial linear combination of the first m vectors. Therefore, there are scalars c1 , . . . , cm , not all 0, such that vm+1 = c1 v1 + · · · + cm vm This is the statement that will result in a contradiction. We multiply the last equation by A to obtain Avm+1 = A(c1 v1 + · · · + cm vm ) = c1 A(v1 ) + · · · + cm A(vm ) Further, since vi is an eigenvector corresponding to the eigenvalue λi , then Avi = λi vi , and after substitution in the previous equation, we have λm+1 vm+1 = c1 λ1 v1 + · · · + cm λm vm Now multiplying both sides of vm+1 = c1 v1 + · · · + cm vm by λm+1 , we also have λm+1 vm+1 = c1 λm+1 v1 + · · · + cm λm+1 vm By equating the last two expressions for λm+1 vm+1 we obtain c1 λ1 v1 + · · · + cm λm vm = c1 λm+1 v1 + · · · + cm λm+1 vm or equivalently, c1 (λ1 − λm+1 )v1 + · · · + cm (λm − λm+1 )vm = 0 Since the vectors v1 , v2 , . . . , vm are linearly independent, the only solution to the previous equation is the trivial solution, that is, c1 (λ1 − λm+1 ) = 0

c2 (λ2 − λm+1 ) = 0

...

cm (λm − λm+1 ) = 0

Since all the eigenvalues are distinct, we have λ1 − λm+1 = 0

λ2 − λm+1 = 0

...

λm − λm+1 = 0

and consequently c1 = 0 c2 = 0 ... cm = 0 This contradicts the assumption that the nonzero vector vm+1 is a nontrivial linear combination of v1 , v2 , . . . , vm .

Confirming Pages

5.2 Diagonalization

COROLLARY 1

EXAMPLE 4 Solution

293

If A is an n × n matrix with n distinct eigenvalues, then A is diagonalizable.

Show that every 2 × 2 real symmetric matrix is diagonalizable. Recall that the matrix A is symmetric if and only if A = At . Every 2 × 2 symmetric matrix has the form   a b A= b d See Example 5 of Sec. 1.3. The eigenvalues are found by solving the characteristic equation



a−λ b

= λ2 − (a + d)λ + ad − b2 = 0 det(A − λI ) =

b d −λ

By the quadratic formula, the eigenvalues are  a + d ± (a − d)2 + 4b2 λ= 2 2 2 Since the discriminant (a − d) + 4b ≥ 0, the characteristic equation has either one or two real roots. If (a − d)2 + 4b2 = 0, then (a − d)2 = 0 and b2 = 0, which holds if and only if a = d and b = 0. Hence, the matrix A is diagonal. If (a − d)2 + 4b2 > 0, then A has two distinct eigenvalues; so by Corollary 1, the matrix A is diagonalizable.

By Theorem 2, if A is diagonalizable, then A is similar to a diagonal matrix whose eigenvalues are the same as the eigenvalues of A. In Theorem 4 we show that the same can be said about any two similar matrices.

THEOREM 4

Let A and B be similar n × n matrices. Then A and B have the same eigenvalues. Proof Since A and B are similar matrices, there is an invertible matrix P such that B = P −1 AP . Now det(B − λI ) = det(P −1 AP − λI ) = det(P −1 (AP − P (λI ))) = det(P −1 (AP − λI P )) = det(P −1 (A − λI )P )

Confirming Pages

294

Chapter 5 Eigenvalues and Eigenvectors

Applying Theorem 15 and Corollary 1 of Sec. 1.6, we have det(B − λI ) = det(P −1 ) det(A − λI ) det(P ) = det(P −1 ) det(P ) det(A − λI ) = det(A − λI ) Since the characteristic polynomials of A and B are the same, their eigenvalues are equal.

EXAMPLE 5



Let A=

1 2 0 3



 and

P =

1 1 1 2



Verify that the matrices A and B = P −1 AP have the same eigenvalues. Solution

The characteristic equation for A is det(A − λI ) = (1 − λ)(3 − λ) = 0 so the eigenvalues of A are λ1 = 1 and λ2 = 3. Since       3 4 1 1 1 2 2 −1 = B = P −1 AP = 0 1 1 2 0 3 −1 1 the characteristic equation for B is det(B − λI ) = (1 − λ)(3 − λ) = 0 and hence, the eigenvalues of B are also λ1 = 1 and λ2 = 3.

In Sec. 4.5, we saw that a linear operator on a finite dimensional vector space can have different matrix representations depending on the basis used to construct the matrix. However, in every case the action of the linear operator on a vector remains the same. These matrix representations also have the same eigenvalues.

COROLLARY 2

Let V be a finite dimensional vector space, T: V −→ V a linear operator, and B1 and B2 ordered bases for V . Then [T ]B1 and [T ]B2 have the same eigenvalues. Proof Let P be the transition matrix from B2 to B1 . Then by Theorem 15 of Sec. 4.5, P is invertible and [T ]B2 = P −1 [T ]B1 P . Therefore, by Theorem 4, [T ]B1 and [T ]B2 have the same eigenvalues.

Confirming Pages

5.2 Diagonalization

295

Recall that in Example 3, the characteristic polynomial for A and B is −(λ + 1)2 (λ − 2). For the matrix A the eigenspaces corresponding to λ1 = −1 and λ2 = 2 are      −1   −1  1  and Vλ2 = span  1  Vλ1 = span  1 ,  0      1 0 1 whereas the eigenspaces for B are    1  Vλ1 = span  0    0

and

Vλ2

   1  = span  3    9

Notice that for the matrix A, we have dim(Vλ1 ) = 2 and dim(Vλ2 ) = 1, which, respectively, are equal to the corresponding algebraic multiplicities in the characteristic polynomial. This is not the case for the matrix B, since dim(Vλ1 ) = 1 and the corresponding algebraic multiplicity is 2. Moreover, for A, we have dim(Vλ1 ) + dim(Vλ2 ) = 3 = n. The general result describing this situation is given, without proof, in Theorem 5.

THEOREM 5

Let A be an n × n matrix, and suppose that the characteristic polynomial is c(x − λ1 )d1 (x − λ2 )d2 · · · (x − λk )dk . The matrix A is diagonalizable if and only if di = dim(Vλi ), for each i = 1, . . . , n, and d1 + d2 + · · · + dk = dim(Vλ1 ) + dim(Vλ2 ) + · · · + dim(Vλk ) = n To summarize Theorem 5, an n × n matrix A is diagonalizable if and only if the algebraic multiplicity for each eigenvalue is equal to the dimension of the corresponding eigenspace, which is the corresponding geometric multiplicity, and the common sum of these multiplicities is n.

Diagonalizable Linear Operators In Theorem 12 of Sec. 4.4, we established that every linear operator on a finite dimensional vector space has a matrix representation. The particular matrix for the operator depends on the ordered basis used. From Corollary 2, we know that all matrix representations for a given linear operator are similar. This allows us to make the following definition.

DEFINITION 1

Diagonalizable Linear Operator Let V be a finite dimensional vector space and T: V −→ V a linear operator. The operator T is called diagonalizable if there is a basis B for V such that the matrix for T relative to B is a diagonal matrix.

Confirming Pages

296

Chapter 5 Eigenvalues and Eigenvectors

Now suppose that V is a vector space of dimension n, T: V −→ V a linear operator, and B = {v1 , v2 , . . . , vn } a basis for V consisting of n eigenvectors. Then 







[T ]B =  T (v1 ) 

 



 T (v2 )  · · ·  T (vn )   B

B

B

Since for each i = 1, . . . , n the vector vi is an eigenvector, then T (vi ) = λi vi , where λi is the corresponding eigenvalue. Since for each i, the basis vector vi can be written uniquely as a linear combination of v1 , . . . , vn , we have vi = 0v1 + · · · + 0vi−1 + vi + 0vi+1 + · · · + 0vn Then the coordinate vector of T (vi ) relative to B is 

 0  ..   .     0     [T (vi )]B =   λi   0     .   ..  0 Therefore, [T ]B is a diagonal matrix. Alternatively, we can say that T is diagonalizable if there is a basis for V consisting of eigenvectors of T. As an illustration, define the linear operator T: ⺢2 −→ ⺢2 by  T Observe that

 v1 =

1 1

x y



 =

2x x+y





 and

v2 =

0 1



are eigenvectors of T with corresponding eigenvalues λ1 = 2 and λ2 = 1, respectively. Let B = {v1 , v2 }, so that   2 0 [T ]B = [ [T (v1 )]B [T (v2 )]B ] = 0 1 is a diagonal matrix. In practice it is not always so easy to determine the eigenvalues and eigenvectors of T. However, if B is any basis for V such that [T ]B is diagonalizable with diagonalizing matrix P , then T is diagonalizable. That is, if B  is the basis consisting of the column vectors of P , then [T ]B  = P −1 [T ]B P is a diagonal matrix. This procedure is illustrated in Example 6.

Confirming Pages

5.2 Diagonalization

EXAMPLE 6

297

Define the linear operator T: ⺢3 −→ ⺢3 by     x1 3x1 − x2 + 2x3  2x1 + 2x3 T  x2  =  x3 x1 + 3x2 Show that T is diagonalizable.

Solution

Let B = {e1 , e2 , e3 } be the standard basis for ⺢3 . Then the matrix for T relative to B is   3 −1 2 0 2  [T ]B =  2 1 3 0 Observe that the eigenvalues of [T ]B are λ1 = −2, λ2 = 4, and λ3 = 1 with corresponding eigenvectors, respectively,       1 1 −5 v2 =  1  and v3 =  4  v1 =  1  −2 1 7 Now let B  = {v1 , v2 , v3 } and

Then [T ]B 

 1 1 −5 4  P = 1 1 −2 1 7

 3 −1 4 −3 1 3  2 =  5 1 9 −1 1 1 0 



    −2 0 0 1 1 −5 −1 2 4 = 0 4 0  0 2  1 1 0 0 1 −2 1 7 3 0

Fact Summary Let A be an n × n matrix. 1. If A is diagonalizable, then A = P DP −1 or equivalently D = P −1 AP . The matrix D is a diagonal matrix with diagonal entries the eigenvalues of A. The matrix P is invertible whose column vectors are the corresponding eigenvectors. 2. If A is diagonalizable, then the diagonalizing matrix P is not unique. If the columns of P are permuted, then the diagonal entries of D are permuted in the same way. 3. The matrix A is diagonalizable if and only if A has n linearly independent eigenvectors. 4. If A has n distinct eigenvalues, then A is diagonalizable.

Revised Confirming Pages

298

Chapter 5 Eigenvalues and Eigenvectors

5. Every 2 × 2 real symmetric matrix is diagonalizable and has real eigenvalues. 6. Similar matrices have the same eigenvalues. 7. If A is diagonalizable, then the algebraic multiplicity for each eigenvalue is equal to the dimension of the corresponding eigenspace (the geometric multiplicity). The common sum of these multiplicities is n. 8. Let T: V −→ V be a linear operator on a finite dimensional vector space V . If V has an ordered basis B consisting of eigenvectors of T, then [T ]B is a diagonal matrix. 9. Let T: V −→ V be a linear operator and B1 and B2 ordered bases for V. Then [T ]B1 and [T ]B2 have the same eigenvalues.

Exercise Set 5.2 In Exercises 1–4, show that A is diagonalizable, using the matrix P .     −2 0 1 0 P = 1. A = 1 1 −2 −3   −1 1 2. A = −3 −5   1 −1 P = −3 1   1 0 0 3. A =  2 −2 0  0 2 0   0 0 32 P =  0 −1 1  1 1 2   −1 2 2 2 0  4. A =  0 2 −1 2   1 −2 −2 0  P =  0 −4 2 1 1 In Exercises 5–18, find the eigenvalues, and if necessary the corresponding eigenvectors, of A and determine whether A is diagonalizable.

 5. A =  6. A =  7. A =  8. A =  9. A =  10. A =

−1 0

1 −2

−2 −2

−3 −2

−1 0

−1 −1

−3 2

−2 1 

0 0 2 −2

1 1

2 −2

   



 2 2 0 11. A =  2 2 2  0 0 3   −1 3 2 12. A =  −1 2 3  −1 2 3   2 1 1 13. A =  2 −1 −1  −1 1 2 

Confirming Pages

5.2 Diagonalization

 −1 0 0 2 −1  A =  −1 0 −1 2   1 0 0  −1 0 0  A= −1 0 0   0 1 0 0 −1  A= 1 0 −1 0   0 0 0 0  1 0 1 0   A=  0 1 0 1  1 1 1 1   1 0 0 1  0 0 1 1   A=  1 0 0 1  0 1 0 1 

14.

15.

16.

17.

18.

In Exercises 19–26, diagonalize the matrix A.   2 0 19. A = −1 −1   −2 1 20. A = 1 2   1 0 0 21. A =  0 −2 1  1 −2 1   0 −1 2 2 2  22. A =  0 0 0 1   −1 0 0 23. A =  −1 1 0  0 0 1   1 0 0 0 0  24. A =  0 0 −1 1   1 0 1 0  0 1 0 0   25. A =   1 0 1 1  0 0 0 1



1  0 26. A =   1 1

0 1 1 1

1 0 1 1

299

 1 0   1  1

27. Suppose A is diagonalizable with D = P −1 AP. Show that for any positive integer k, Ak = P D k P −1 

28. Let A=

2 1 2 1



Factor A in the form A = P DP −1 , where D is a diagonal matrix. Then find A6 . See Exercise 27. 29. Let

 3 −1 −2 0 −2  A= 2 2 −1 −1 

Factor A in the form A = P DP −1 , where D is a diagonal matrix. Then find Ak , for any positive integer k. See Exercise 27. 30. Suppose A is an n × n matrix that is diagonalized by P. Find a matrix that diagonalizes At . 31. Suppose A is an n × n matrix that is diagonalizable. Show that if B is a matrix similar to A, then B is diagonalizable. 32. Show that if A is invertible and diagonalizable, then A−1 is diagonalizable. Find a 2 × 2 matrix that is not a diagonal matrix, is not invertible, but is diagonalizable. 33. Suppose A is an n × n matrix and λ is an eigenvalue of multiplicity n. Show that A is diagonalizable if and only if A = λI. 34. An n × n matrix A is called nilpotent if there is a positive integer k such that Ak = 0. Show that a nonzero nilpotent matrix is not diagonalizable. 35. Define a linear operator T: P2 → P2 by T (p(x)) = p (x) a. Find the matrix A for T relative to the standard basis {1, x, x 2 }.

Confirming Pages

300

Chapter 5 Eigenvalues and Eigenvectors

b. Find the matrix B for T relative to the basis {x, x − 1, x 2 }. c. Show the eigenvalues of A and B are the same. d. Explain why T is not diagonalizable. 36. Define a vector space V = span{sin x, cos x} and a linear operator T: V → V by T (f (x)) = f  (x). Show that T is diagonalizable. 37. Define a linear operator T: ⺢3 → ⺢3 by     x1 2x1 + 2x2 + 2x3 T  x2  =  −x1 + 2x2 + x3  x3 x1 − x2 ß

5.3

Show that T is not diagonalizable. 38. Define a linear operator T: ⺢3 → ⺢3 by     x1 4x1 + 2x2 + 4x3 T  x2  =  4x1 + 2x2 + 4x3  x3 4x3 Show that T is diagonalizable. 39. Let T be a linear operator on a finite dimensional vector space, A the matrix for T relative to a basis B1 , and B the matrix for T relative to a basis B2 . Show that A is diagonalizable if and only if B is diagonalizable.

Application: Systems of Linear Differential Equations In Sec. 3.5 we considered only a single differential equation where the solution involved a single function. However, in many modeling applications, an equation that involves the derivatives of only one function is not sufficient. It is more likely that the rate of change of a variable quantity will be linked to other functions outside itself. This is the fundamental idea behind the notion of a dynamical system. One of the most familiar examples of this is the predator-prey model. For example, suppose we wish to create a model to predict the number of foxes and rabbits in some habitat. The growth rate of the foxes is dependent on not only the number of foxes but also the number of rabbits in their territory. Likewise, the growth rate of the rabbit population in part is dependent on their current number, but is obviously mitigated by the number of foxes in their midst. The mathematical model required to describe this relationship is a system of differential equations of the form y1 (t) = f (t, y1 , y2 ) y2 (t) = g(t, y1 , y2 ) In this section we consider systems of linear differential equations. Problems such as predator-prey problems involve systems of nonlinear differential equations.

Uncoupled Systems At the beginning of Sec. 3.5 we saw that the differential equation given by y  = ay has the solution y(t) = Ceat , where C = y(0). An extension of this to two dimensions is the system of differential equations y1 = ay1 y2 = by2

Confirming Pages

5.3 Application: Systems of Linear Differential Equations

301

where a and b are constants and y1 and y2 are functions of a common variable t. This system is called uncoupled since y1 and y2 depend only on y1 and y2 , respectively. The general solution of the system is found by solving each equation separately and is given by y1 (t) = C1 eat

y2 (t) = C2 ebt

and

where C1 = y1 (0) and C2 = y2 (0). The previous system of two differential equations can also be written in matrix form. To do this, define        y y1 a 0 1  and y= y = A= 0 b y2 y2 Then the uncoupled system above is equivalent to the matrix equation y = Ay The matrix form of the solution is given by   at 0 e y(0) y(t) = 0 ebt   y1 (0) where y(0) = y2 (0) As an illustration, consider the system of differential equations y1 = −y1 y2 = 2y2 In matrix form the system is written as



y = Ay = The solution to the system is

 y=

e−t 0

−1 0 0 2 0 e2t

 y

 y(0)

that is, y1 (t) = y1 (0)e−t

and

y2 (t) = y2 (0)e2t

The Phase Plane In the case of a single differential equation, it is possible to sketch particular solutions in the plane to see explicitly how y(t) depends on the independent variable t. However, for a system of two differential equations, the solutions are vectors which depend on a common parameter t, which is usually time. A particular solution can be viewed as a parameterized curve or trajectory in the plane, called the phase plane. Shown in Fig. 1 are trajectories for several particular solutions of the system y1 = −y1 y2 = 2y2

Confirming Pages

302

Chapter 5 Eigenvalues and Eigenvectors

Figure 1

The vectors shown in Fig. 1 comprise the direction field for the system and describe the motion along a trajectory for increasing t. This sketch is called the phase portrait for the system. Phase portraits are usually drawn without the direction field. We have done so here to give a more complete picture of the system and its solutions.

Diagonalization In the previous example, the matrix A is diagonal, as this is the case for any uncoupled system of differential equations. We now consider more general systems of the form y = Ay for which A is not a diagonal matrix, but is diagonalizable with real distinct eigenvalues. To solve problems of this type, our strategy is to reduce the system y = Ay to one that is uncoupled. To develop this idea, let A be a 2 × 2 diagonalizable matrix with distinct real eigenvalues. Consider the system of differential equations given by y = Ay Since A is diagonalizable, then by Theorem 2 of Sec. 5.2 there is a diagonal matrix D and an invertible matrix P such that D = P −1 AP The diagonal matrix D is given by  D=

λ1 0

0 λ2



where λ1 and λ2 are the eigenvalues of A. The column vectors of P are the corresponding eigenvectors. To uncouple the system y = Ay, let w = P −1 y

Confirming Pages

5.3 Application: Systems of Linear Differential Equations

303

Differentiating both sides of the last equation gives w = (P −1 y) = P −1 y = P −1 Ay = P −1 (P DP −1 )y = (P −1 P )(DP −1 )y = DP −1 y = Dw Since D is a diagonal matrix, the original linear system y = Ay is transformed into the uncoupled linear system w = P −1 AP w = Dw The general solution of this new system is given by   λt 0 e 1 w(0) w(t) = 0 eλ2 t Now, to find the solution to the original system, we again use the substitution w = P −1 y to obtain   λt 0 e 1 P −1 y(0) P −1 y(t) = 0 eλ2 t Hence, the solution to the original system is   λt 0 e 1 P −1 y(0) y(t) = P 0 eλ2 t

EXAMPLE 1

Find the general solution to the system of differential equations y1 = −y1 y2 = 3y1 + 2y2 Sketch several trajectories in the phase plane.

Solution

The differential equation is given in matrix form by   −1 0  y y = Ay = 3 2 After solving the characteristic equation det(A − λI ) = 0, we know that the eigenvalues of A are λ1 = −1 and λ2 = 2 with corresponding eigenvectors     1 0 and v2 = v1 = −1 1 Hence, the matrix P which diagonalizes A (see Theorem 2 of Sec. 5.2) is     1 0 1 0 −1 with P = P = 1 1 −1 1

Confirming Pages

304

Chapter 5 Eigenvalues and Eigenvectors

The related uncoupled system is then given by w = P −1 AP w     1 0 −1 0 1 0 w = −1 1 3 2 1 1   −1 0 w = 0 2 whose general solution is

 w(t) =

e−t 0

0 e2t

 w(0)

Hence, the solution to the original system is given by   −t    e 0 1 0 1 0 y(0) y(t) = −1 1 1 1 0 e2t   e−t 0 = y(0) −t −e + e2t e2t The general solution can also be written in the form y1 (t) = y1 (0)e−t

and

  y2 (t) = −y1 (0)e−t + y1 (0) + y2 (0) e2t

The phase portrait is shown in Fig. 2. The signs of the eigenvalues and the direction of the corresponding eigenvectors help to provide qualitative information about the trajectories in the phase portrait. In particular, notice in Fig. 2 that along  1 the line spanned by the eigenvector v1 = the flow is directed toward the −1 origin. This is so because the sign 1 = −1 is negative. On the other hand, flow  of λ 0 is away from the origin, since in this case along the line spanned by v2 = 1 λ2 = 2 is positive.

Figure 2

Confirming Pages

5.3 Application: Systems of Linear Differential Equations

305

In Example 2 we describe the solution for a system when the eigenvalues have the same sign.

EXAMPLE 2

Solution

Find the general solution to the system of differential equations y1 = y1 + 3y2 y2 = 2y2 The system of differential equations is given in matrix form by   1 3  y y = Ay = 0 2 The eigenvalues of A are λ1 = 1 and λ2 = 2 with corresponding eigenvectors     1 3 and v2 = v1 = 0 1 The matrix that diagonalizes A is then   1 3 with P = 0 1

P

−1

 =

1 0

−3 1



The uncoupled system is given by     1 3 1 3 1 −3  w w = 0 1 0 2 0 1   1 0 w = 0 2 with general solution



 et 0 w(t) = w(0) 0 e2t Hence, the solution to the original system is given by   t   e 0 1 −3 1 3 y(0) y(t) = 0 1 0 1 0 e2t  t  e −3et + 3e2t = y(0) 0 e2t

The general solution can also be written in the form   y1 (t) = y1 (0) − 3y2 (0) et + 3y2 (0)e2t and

y2 (t) = y2 (0)e2t

Confirming Pages

306

Chapter 5 Eigenvalues and Eigenvectors

The phase portrait is shown in Fig. 3. For this example, since λ1 and λ2 are both positive, the flow is oriented outward along the lines spanned by v1 and v2 .

Figure 3

The process described for solving a system of two equations can be extended to higher dimensions provided that the matrix A of the system is diagonalizable.

EXAMPLE 3

Solution

Find the general solution to the system of differential equations    y1 = −y1 y2 = 2y1 + y2    y3 = 4y1 + y2 + 2y3 The system of differential equations in matrix  −1 y = Ay =  2 4

form is  0 0 1 0 y 1 2

Since A is triangular, the eigenvalues of A are the diagonal entries λ1 and λ3 = 2 with corresponding eigenvectors      −1 0 v2 =  1  and v3 =  v1 =  1  1 −1

= −1, λ2 = 1,  0 0  2

respectively. Since A is a 3 × 3 matrix with three distinct eigenvalues, by Corollary 1 of Sec. 5.2, A is diagonalizable. Now, by Theorem 2 of Sec. 5.2, the diagonalizing matrix is given by     −1 0 0 −1 0 0 1 0  with P −1 =  1 1 0  P = 1 1 −1 2 1 12 21

Confirming Pages

5.3 Application: Systems of Linear Differential Equations

307

The related uncoupled system then becomes     −1 0 0 −1 0 0 −1 0 0 1 0 w w =  1 1 0   2 1 0   1 1 1 1 −1 2 4 1 2 1 2 2   −1 0 0 =  0 1 0 w 0 0 2 with general solution



 e−t 0 0 0  w(0) w(t) =  0 et 2t 0 0 e Hence, the solution to the original system is given by   −t   −1 0 e 0 0 −1 0 0 0  1 1 1 0   0 et y(t) =  1 1 −1 2 0 0 e2t 1 12   0 0 e−t −e−t + et et 0  y(0) = −t t 2t t −e + e2t e2t −e − e + 2e

 0 0  y(0) 1 2

The general solution can also be written in the form y2 (t) = −y1 (0)e−t + [y1 (0) + y2 (0)]et and y1 (t) = y1 (0)e−t −t y3 (t) = −y1 (0)e − [y1 (0) + y2 (0)]et + [2y1 (0) + y2 (0) + y3 (0)]e2t .

Example 4 gives an illustration of how a linear system of differential equations can be used to model the concentration of salt in two interconnected tanks.

EXAMPLE 4

1

2

Figure 4

Suppose that two brine storage tanks are connected with two pipes used to exchange solutions between them. The first pipe allows water from tank 1 to enter tank 2 at a rate of 5 gal/min. The second pipe reverses the process allowing water to flow from tank 2 to tank 1, also at a rate of 5 gal/min. Initially, the first tank contains a well-mixed solution of 8 lb of salt in 50 gal of water, while the second tank contains 100 gal of pure water. a. Find the linear system of differential equations to describe the amount of salt in each tank at time t. b. Solve the system of equations by reducing it to an uncoupled system.

Confirming Pages

308

Chapter 5 Eigenvalues and Eigenvectors

c. Determine the amount of salt in each tank as t increases to infinity and explain the result. Solution

a. Let y1 (t) and y2 (t) be the amount of salt (in pounds) in each tank after t min. Thus, y1 (t) and y2 (t) are, respectively, the rates of change for the amount of salt in tank 1 and tank 2. To develop a system of equations, note that for each tank Rate of change of salt = rate in − rate out Since the volume of brine in each tank remains constant, for tank 1, the rate in 5 5 5 is 100 y2 (t) while the rate out is 50 y1 (t). For tank 2, the rate in is 50 y1 (t) while 5 the rate out is 100 y2 (t). The system of differential equations is then given by   5 5 1 1 y1 (t) = 100 y2 (t) − 50 y1 (t) y1 (t) + 20 y2 (t) y1 (t) = − 10 that is, 5 5 1 1   y2 (t) = 50 y1 (t) − 100 y2 (t) y2 (t) = 10 y1 (t) − 20 y2 (t) Since the initial amounts of salt in tank 1 and tank 2 are 8 and 0 lb, respectively, the initial conditions on the system are y1 (0) = 8 and y2 (0) = 0. b. The system of equations in matrix form is given by     1 1 − 10 8 20  y with y(0) = y = 1 1 0 − 20 10 3 The eigenvalues the matrix  λ1 = − 20 and λ2 = 0 with corresponding  are  of  1 −1 . Thus, the matrix that uncouples the system and eigenvectors 2 1 is     − 23 31 −1 1 −1 with P = P = 1 1 1 2 3

The uncoupled system is then given by   1 − 10 − 23 31  w = 1 1 1  =

3

3

3 − 20 0 0 0

10



1 20 1 − 20



−1 1 1 2

w

The solution to the uncoupled system is   3 t − 20 0 w(0) w(t) = e 0 1

3

 w

Confirming Pages

5.3 Application: Systems of Linear Differential Equations

309

Hence, the solution to the original system is given by  2 1    3 t − 20 −3 3 −1 1 e 0 y(0) y(t) = 1 1 1 2 0 1 3 3    3 3 1 8 2e− 20 t + 1 −e− 20 t + 1 = 3 3 0 3 −2e− 20 t + 2 e− 20 t + 2   3 8 2e− 20 t + 1 = 3 3 −2e− 20 t + 2 c. The solution to the system in equation form is given by 3 8 −3t 8 y1 (t) = 2e 20 + 1 −2e− 20 t + 2 and y2 (t) = 3 3 To find the amount of salt in each tank as t goes to infinity, we compute the limits 8 8 8 −3t 2e 20 + 1 = (0 + 1) = lim t→∞ 3 3 3 and  3 16 8 8 −2e− 20 + 2 = (0 + 2) = lim t→∞ 3 3 3 These values make sense intuitively as we expect that the 8 lb of salt should eventually be thoroughly mixed, and divided proportionally between the two tanks in a ratio of 1 : 2.

Exercise Set 5.3 In Exercises 1–6, find the general solution to the system of differential equations. y1 = −y1 + y2 1. y2 = − 2y2 y1 = −y1 + 2y2 2. y2 = y1 y1 − 3y2 y1 = 3.  y2 = −3y1 + y2 y1 = y1 − y2 4. y2 = −y1 + y2

    y1 5. y2    y3     y1 6. y2    y3

= −4y1 − 3y2 − 3y3 = 2y1 + 3y2 + 2y3 = 4y1 + 2y2 + 3y3 = −3y1 − 4y2 − 4y3 = 7y1 + 11y2 + 13y3 = −5y1 − 8y2 − 10y3

In Exercises 7 and 8, solve the initial-value problem. y1 (0) = 1 y2 (0) = −1 y1 = −y1 7.  y2 = 2y1 + y2

Confirming Pages

310    y1 8. y2    y3

Chapter 5 Eigenvalues and Eigenvectors

= 5y1 − 12y2 + 20y3 = 4y1 − 9y2 + 16y3 = 2y1 − 4y2 + 7y3

y1 (0) = 2 y2 (0) = −1 y3 (0) = 0 9. Suppose that two brine storage tanks are connected with two pipes used to exchange solutions between them. The first pipe allows water from tank 1 to enter tank 2 at a rate of 1 gal/min. The second pipe reverses the process, allowing water to flow from tank 2 to tank 1, also at a rate of 1 gal/min. Initially, the first tank contains a well-mixed solution of 12 lb of salt in 60 gal of water, while the second tank contains 120 gal of pure water. a. Find the linear system of differential equations to describe the amount of salt in each tank at time t. b. Solve the system of equations by reducing it to an uncoupled system. c. Determine the amount of salt in each tank as t increases to infinity and explain the result. 10. On a cold winter night when the outside temperature is 0 degrees Fahrenheit (0o F) at 9:00 p.m. the furnace in a two-story home fails. ß

5.4

Suppose the rates of heat flow between the upstairs, downstairs, and outside are as shown in the figure. Further suppose the temperature of the first floor is 70◦ F and that of the second floor is 60◦ F when the furnace fails.

0

0.2 0.2

0.1 0.1

0.5

1

2 0.5

a. Use the balance law Net rate of change = rate in − rate out to set up an initial-value problem to model the heat flow. b. Solve the initial-value problem found in part (a). c. Compute how long it takes for each floor to reach 32◦ F.

Application: Markov Chains In probability theory a Markov process refers to a type of mathematical model used to analyze a sequence of random events. A critical factor when computing the probabilities of a succession of events is whether the events are dependent on one another. For example, each toss of a fair coin is an independent event as the coin has no memory of a previous toss. A Markov process is useful in describing the tendencies of conditionally dependent random events, where the likelihood of each event depends on what happened previously. As an illustration, we consider a simple weather model based on the two observations that

Table 1 Sunny Cloudy Sunny

0.7

0.5

Cloudy

0.3

0.5

1. If today is sunny, then there is a 70 percent chance that tomorrow will be sunny. 2. If today is cloudy, then there is a 50 percent chance that tomorrow will be cloudy. The conditional probabilities for the weather tomorrow, given the weather for today, are given in Table 1. The column headings in Table 1 describe today’s weather, and the row headings the weather for tomorrow. For example, the probability that a sunny day today is

Confirming Pages

5.4 Application: Markov Chains

311

followed by another sunny day tomorrow is 0.7, and the probability that a sunny day today is followed by a cloudy day tomorrow is 0.3. Notice that the column sums are both 1, since, for example, it is certain that a sunny day today is followed by either a sunny day or a cloudy day tomorrow. In a Markov process, these observations are applied iteratively, giving us the ability to entertain questions such as, If today is sunny, what is the probability that it will be sunny one week from today?

State Vectors and Transition Matrices To develop the Markov process required to make predictions about the weather using   v1 the observations above, we start with a vector v = whose components are the v2 probabilities for the current weather conditions. In particular, let v1 be the probability that today is sunny and v2 the probability that today is cloudy. Each day the components of v change in accordance with the probabilities, listed in Table 1, giving us the current state of the weather. In a Markov process, the vector v is called a state vector, and a sequence of state vectors a Markov chain. Using Table 1, the state    v1  vector v = for the weather tomorrow has components v2 v1 = 0.7v1 + 0.5v2

and

v2 = 0.3v1 + 0.5v2

That is, the probability v1 of a sunny day tomorrow is 0.7 times the probability of a sunny day today plus 0.5 times the probability of a cloudy day today. Likewise, the probability v2 of a cloudy day tomorrow is 0.3 times the probability of a sunny day today plus 0.5 times the probability of a cloudy day today. For example, if today is sunny, then v1 = 1 and v2 = 0 so that v1 = 0.7(1) + 0.5(0) = 0.7

and

v2 = 0.3(1) + 0.5(0) = 0.3

which is in agreement with the observations above. Observe that if we let T be the matrix   0.7 0.5 T = 0.3 0.5 then the relationship above between v and v can be written using matrix multiplication as       v1 v1 0.7 0.5 = 0.3 0.5 v2 v2 In a Markov chain, the matrix used to move from one state to the next is called the transition matrix. If n is the number of possible states, then the transition matrix T is an n × n matrix where the ij entry is the probability of moving from state j to state i. In the above example t12 = 0.5 gives the probability that a cloudy day is followed by one that is sunny. A vector with positive entries whose sum is 1 is called a probability vector. A matrix whose column vectors are probability vectors is called a stochastic matrix. The transition matrix T given above is an example of a stochastic matrix.

Confirming Pages

312

Chapter 5 Eigenvalues and Eigenvectors

Returning to the weather example, to predict the weather 2 days forward, we apply the transition matrix T to the vector v so that        v1 0.7 0.5 v1 = 0.3 0.5 v2 v2  2      v1 v1 0.64 0.60 0.7 0.5 = = 0.36 0.40 v2 v2 0.3 0.5 Thus, for example, if today is sunny, the state vector for the weather 2 days from now is given by        v1 0.64 1 0.64 0.60 = = 0.36 0 0.36 0.40 v2 In general, after n days the state vector for the weather is given by n    v1 0.7 0.5 T nv = 0.3 0.5 v2 To answer the question posed earlier about the weather one week after a sunny day, we compute     7     0.625 1 0.625 0.625 0.7 0.5 1 = = 0.375 0 0.375 0.375 0.3 0.5 0 That is, if today is sunny, then the probability that it will be sunny one week after today is 0.625, and the probability it will be cloudy is 0.375.

Diagonalizing the Transition Matrix As we have just seen, determining future states in a Markov process involves computing powers of the transition matrix. To facilitate the computations, we use the methods of Sec. 5.2 to diagonalize the transition matrix. To illustrate, we again consider the transition matrix  7  5 T =

10

10

3 10

5 10

of the weather example above. Observe that T has distinct eigenvalues given by λ1 = 1

and

λ2 =

and

v2 =

with corresponding eigenvectors   v1 =

5 3

1

2 10



−1 1



Confirming Pages

5.4 Application: Markov Chains

313

For reasons that will soon be clear, we scale v1 (by the reciprocal of the sum of its components) so that it becomes a probability vector. Observe that this new vector  5  ! v1 =

8 3 8

is also an eigenvector since it is in the eigenspace Vλ1. Since the 2 × 2 transition matrix has two distinct eigenvalues, by Corollary 1 of Sec. 5.2, T is diagonalizable and, by Theorem 2 of Sec. 5.2, can be written as T = P DP −1   5 −1  1 8 = 3 0 1 8

0 2 10



1

1

− 38

5 8



By Exercise 27 of Sec. 5.2, the powers of T are given by   n 1 " 0# n n −1 P −1 T = PD P = P 2 n 0 10 As mentioned above, this gives us an easier way to compute the state vector for large values of n. Another benefit from this representation is that the matrix D n approaches   1 0 0 0 as n gets large. This suggests that the eigenvector corresponding to λ = 1 is useful in determining the limiting proportion of sunny days to cloudy days far into the future.

Steady-State Vector Given an initial state vector v, of interest is the long-run behavior of this vector in a Markov chain, that is, the tendency of the vector T n v for large n. If for any initial state vector v there is some vector s such that T n v approaches s, then s is called a steady-state vector for the Markov process. In our weather model we saw that the transition matrix T has an eigenvalue λ = 1 and a corresponding probability eigenvector given by  5    0.625 ! v1 = 83 = 0.375 8

Confirming Pages

314

Chapter 5 Eigenvalues and Eigenvectors

We claim that this vector is a steady-state vectorfor theweather model. As verification, 0.4 . We then compute let u be an initial probability vector, say, u = 0.6     0.6249999954 0.6250000002 T 10 u = and T 20 u = 0.3750000046 0.3750000002 which suggests that T n u approaches ! v1 . That this is in fact the case is stated in Theorem 6. Before doing so, we note that a regular transition matrix T is a transition matrix such that for some n, all the entries of T n are positive. THEOREM 6

If a Markov chain has a regular stochastic transition matrix T, then there is a unique probability vector s with T s = s. Moreover, s is the steady-state vector for any initial probability vector.

EXAMPLE 1

A group insurance plan allows three different options for participants, plan A, B, or C. Suppose that the percentages of the total number of participants enrolled in each plan are 25 percent, 30 percent, and 45 percent, respectively. Also, from past experience assume that participants change plans as shown in the table. A B C A 0.75 0.25 0.2 B 0.15 0.45 0.4 C 0.1 0.3 0.4 a. Find the percent of participants enrolled in each plan after 5 years. b. Find the steady-state vector for the system.

Solution

Let T be the matrix given by

 0.75 0.25 0.2 T =  0.15 0.45 0.4  0.1 0.3 0.4 

a. The number of participants enrolled in each plan after 5 years is approximated by the vector      0.47 0.25 0.49776 0.46048 0.45608 T 5 v =  0.28464 0.30432 0.30664   0.30  =  0.30  0.22 0.45 0.21760 0.23520 0.23728 so approximately 47 percent will be enrolled in plan A, 30 percent in plan B, and 22 percent in plan C. b. The steady-state vector for the system is the probability eigenvector corresponding to the eigenvalue λ = 1, that is,   0.48 s =  0.30  0.22

Revised Confirming Pages

5.4 Application: Markov Chains

315

Exercise Set 5.4 1. Each year it is estimated that 15 percent of the population in a city moves to the surrounding suburbs and 8 percent of people living in the suburbs move to the city. Currently, the total population of the city and surrounding suburbs is 2 million people with 1.4 million living in the city. a. Write the transition matrix for the Markov chain describing the migration pattern. b. Compute the expected population after 10 years. c. Find the steady-state probability vector. 2. After opening a new mass transit system, the transit authority studied the user patterns to try to determine the number of people who switched from using an automobile to the system. They estimated that each year 30 percent of those who tried the mass transit system decided to go back to driving and 20 percent switched from driving to using mass transit. Suppose that the population remains constant and that initially 35 percent of the commuters use mass transit. a. Write the transition matrix for the Markov chain describing the system. b. Compute the expected number of commuters who will be using the mass transit system in 2 years. In 5 years. c. Find the steady-state probability vector. 3. A plant blooms with red, pink, or white flowers. When a variety with red flowers is cross-bred with another variety, the probabilities of the new plant having red, pink, or white flowers are given in the table. R

P

W

R

0.5

0.4

0.1

P

0.4

0.4

0.2

W

0.1

0.2

0.7

Suppose initially there are only plants with pink flowers which are bred with other varieties with

the same likelihood. Find the probabilities of each variety occurring after three generations. After 10 generations. 4. A fleet of taxis picks up and delivers commuters between two nearby cities A and B and the surrounding suburbs S. The probability of a driver picking up a passenger in location X and delivering the passenger to location Y is given in the table. The taxi company is interested in knowing on average where the taxis are. A

B

S

A

0.6

0.3

0.4

B

0.1

0.4

0.3

S

0.3

0.3

0.3

a. If a taxi is in city A, what is the probability it will be in location S after three fares? b. Suppose 30 percent of the taxis are in city A, 35 percent are in city B, and 35 percent are in the suburbs. Calculate the probability of a taxi being in location A, B, or S after five fares. c. Find the steady-state probability vector. 5. An endemic disease that has reached epidemic proportions takes the lives of one-quarter of those who are ill each month while one-half of those who are healthy become ill. Determine whether the epidemic will be eradicated. If so, estimate how long it will take. 6. A regional study of smokers revealed that from one year to the next 55 percent of smokers quit while 20 percent of nonsmokers either became new smokers or started smoking again. If 70 percent of the population are smokers, what fraction will be smoking in 5 years? In 10 years? In the long run? 7. A frog is confined to sitting on one of four lily pads. The pads are arranged in a square. Label the corners of the square A, B, C, and D clockwise. Each time the frog jumps, the probability of

Confirming Pages

316

Chapter 5 Eigenvalues and Eigenvectors

jumping to an adjacent pad is 1/4, the probability of jumping to the diagonal pad is 1/6, and the probability of landing on the same pad is 1/3. a. Write the transition matrix for the Markov process. b. Find the probability state vector after the frog has made n jumps starting at pad A. c. Find the steady-state vector. 8. Let the transition matrix for a Markov process be   0 1 T = 1 0 a. Find the eigenvalues of T . b. Find T n for n ≥ 1. Use T n to explain why the Markov process does have a steady-state vector. c. Suppose T is the transition matrix describing the population distribution at any time for a

constant population where residents can move between two locations. Describe the interaction in the population. 9. Show that for all p and q such that 0 < p < 1 and 0 < q < 1, the transition matrix   1−p q T = p 1−q has steady-state probability vector  q  p+q p p+q

10. Suppose the transition matrix T for a Markov process is a 2 × 2 stochastic matrix that is also symmetric. a. Find the eigenvalues for the matrix T. b. Find the steady-state probability vector for the Markov process.

Review Exercises for Chapter 5  a b A= b a for some real numbers  a and b. 1 is an eigenvector of A. a. Show that 1 b. Find the eigenvalues of A. c. Find the eigenvectors corresponding to each eigenvalue found in part (b). d. Diagonalize the matrix A, using the eigenvectors found in part (b). That is, find the matrix P such that P −1 AP is a diagonal matrix. Specify the diagonal matrix. 

1. Let

2. Let

 0 0 2 0  A= 0 2 0 0 −1 

a. Find the eigenvalues of A. b. From your result in part (a) can you conclude whether A is diagonalizable? Explain. c. Find the eigenvectors corresponding to each eigenvalue.

d. Are the eigenvectors found in part (c) linearly independent? Explain. e. From your result in part (c) can you conclude whether A is diagonalizable? Explain. f. If your answer to part (e) is yes, find a matrix P that diagonalizes A. Specify the diagonal matrix D such that D = P −1 AP . 3. Repeat Exercise 2 with  1  1 A=  0 1

0 1 0 0

1 1 0 1

 0 0   0  0

4. Let T be a linear operator on a finite dimensional vector space with a matrix representation   1 0 0 3 2  A= 6 −3 −1 0 a. Find the characteristic polynomial for A. b. Find the eigenvalues of A. c. Find the dimension of each eigenspace of A.

Confirming Pages

5.4 Application: Markov Chains

d. Using part (c), explain why the operator T is diagonalizable. e. Find a matrix P and diagonal matrix D such that D = P −1 AP . f. Find two other matrices P1 and P2 and corresponding diagonal matrices D1 and D2 such that D1 = P1−1 AP1 and D2 = P2−1 AP2 .  0 1 0 A= 0 0 1  −k 3 0 a. Show the characteristic equation of A is λ3 − 3λ + k = 0. b. Sketch the graph of y(λ) = λ3 − 3λ + k for k < −2, k = 0, and k > 2. c. Determine the values of k for which the matrix A has three distinct real eigenvalues.

5. Let



6. Suppose that B = P −1 AP and v is an eigenvector of B corresponding to the eigenvalue λ. Show that P v is an eigenvector of A corresponding to the eigenvalue λ. 7. Suppose that A is an n × n matrix such that every row of A has the same sum λ. a. Show that λ is an eigenvalue of A. b. Does the same result hold if the sum of every column of A is equal to λ? 8. Let V be a vector space and T: V −→ V a linear operator. A subspace W of V is invariant under T if for each vector w in W , the vector T (w) is in W . a. Explain why V and {0} are invariant subspaces of every linear operator on the vector space. b. Show that if there is a one-dimensional subspace of V that is invariant under T, then T has a nonzero eigenvector. c. Let T be a linear operator on ⺢2 with matrix representation relative to the standard basis given by   0 −1 A= 1 0

317

Show that the only invariant subspaces of T are ⺢2 and {0}. 9. a. Two linear operators S and T on a vector space V are said to commute if S(T (v)) = T (S(v)) for every vector v in V. If S and T are commuting linear operators on V and λ0 is an eigenvalue of T, show that Vλ0 is invariant under S, that is, S(Vλ0 ) ⊆ Vλ0 . b. Let S and T be commuting linear operators on an n-dimensional vector space V. Suppose that T has n distinct eigenvalues. Show that S and T have a common eigenvector. c. A pair of linear operators T and S on a vector space V is called simultaneously diagonalizable if there is an ordered basis B for V such that [T ]B and [S]B are both diagonal. Show that if S and T are simultaneously diagonalizable linear operators on an n-dimensional vector space V, then S and T commute. d. Show directly that the matrices 

3 0 A= 0 2 1 0

 1 0  3

 1 0 −2 0  B= 0 1 −2 0 1 are simultaneously diagonalizable. and



10. The Taylor series expansion (about x = 0) for the natural exponential function is ∞

ex = 1 + x +

$ 1 1 2 1 x + x3 + · · · = xk 2! 3! n! k=0

If A is an n × n matrix, we can define the matrix exponential as 1 2 1 A + A3 + · · · 2! 3! 1 1 1 m A ) = lim (I + A + A2 + A3 + · · · + m→∞ 2! 3! m!

eA = I + A +

Confirming Pages

318

Chapter 5 Eigenvalues and Eigenvectors

a. Let D be the diagonal matrix  λ1 0 0 ... 0  0 λ2 0 . . . 0  D= . .. .. ..  .. . . . 0 . . . . . . . . . λn

    

and find eD .

b. Suppose A is diagonalizable and D = P −1 AP . Show that eA = P eD P −1 . c. Use parts (a) and (b) to compute eA for the matrix   6 −1 A= 3 2

Chapter 5: Chapter Test In Exercises 1–40, determine whether the statement is true or false. 

1. The matrix P =

1 1 0 1

diagonalizes the matrix  −1 A= 0 

2. The matrix A=

 are

1 −2

−1 1 0 −2



 A=

A=



is λ3 + 2λ2 + λ − 4.

and

1 3



0 −5



. 



1

−2 A =

√

23

√ 3 2 1 2









then AAt = I .



9. If A is a 2 × 2 matrix with det(A) < 0, then A has two real eigenvalues. 10. If A is a 2 × 2 matrix that has two distinct eigenvalues λ1 and λ2 , then tr(A) = λ1 + λ2 .   a b , then the eigenvalues of A are 11. If A = b a λ1 = a + b and λ2 = b − a. 12. For all integers k the matrix A =

are λ1 = −3 and λ2 = −1. 5. The characteristic polynomial  −1 −1 0 A= 0 2 −2



8. If

−1 0 0 0 1 0  −1 −1 1

−1 0 −4 −3



−4 3

 3 −2 A= 2 −1 has an eigenvalue λ1 = 1 and Vλ1 has dimension 1.





0 1

7. The matrix

is diagonalizable. 4. The eigenvalues of



A=



is similar to the matrix   −1 0 D= 0 −1 3. The matrix

6. The eigenvectors of



1 1

k 1

 has

only one eigenvalue. of



−1 −1  −1

13. If A is a 2 × 2 invertible matrix, then A and A−1 have the same eigenvalues. 14. If A is similar to B, then tr(A) = tr(B).   1 1 is diagonalizable. 15. The matrix A = 0 1

Confirming Pages

5.4 Application: Markov Chains

 16. If A =

a c

b d



27. If an n × n matrix A is diagonalizable, then A has n linearly independent eigenvectors.

and a+c =b+d =λ

then λ is an eigenvalue of A. In Exercises 17–19, let   1 0 0 0  A= 0 2 0 0 −1 and



−1 0 B= 0 1 0 0

 0 0  2

17. The matrices A and B have the same eigenvalues. 18. The matrices A and B are similar.  0 1 0 P = 0 0 1  1 0 0 then B = P −1 AP .   −1 and 20. If a 2 × 2 matrix has eigenvectors 1   1 , then it has the form −2   2α − β α−β β − 2α 2β − α 19. If

319



21. The only matrix similar to the identity matrix is the identity matrix. 22. If λ = 0 is an eigenvalue of A, then the matrix A is not invertible. 23. If A is diagonalizable, then A is similar to a unique diagonal matrix.

28. If A and B are n × n matrices, then AB and BA have the same eigenvalues. 29. If D is a diagonal matrix and A = P DP −1 , then A is diagonalizable. 30. If A is invertible, then A is diagonalizable. 31. If A and B are n × n invertible matrices, then AB −1 and B −1 A have the same eigenvalues. 32. A 3 × 3 matrix of the form   a 1 0  0 a 1  0 0 b always has fewer than three distinct eigenvalues. 33. If A and B are n × n diagonalizable matrices with the same diagonalizing matrix, then AB = BA. 34. If λ is an eigenvalue of the n × n matrix A, then the set of all eigenvectors corresponding to λ is a subspace of ⺢n . 35. If each column sum of an n × n matrix A is a constant c, then c is an eigenvalue of A. 36. If A and B are similar, then they have the same characteristic equation. 37. If λ is an eigenvalue of A, then λ2 is an eigenvalue of A2 . 38. If A is a 2 × 2 matrix with characteristic polynomial λ2 + λ − 6, then the eigenvalues of A2 are λ1 = 4 and λ2 = 9.

25. If an n × n matrix A has n distinct eigenvalues, then A is diagonalizable.

39. Define a linear operator T: P1 → P1 , by T (a + bx) = a + (a + b)x. Then the matrix representation for A relative to the standard basis is   1 0 A= 1 1 and so T is not diagonalizable.

26. If an n × n matrix A has a set of eigenvectors that is a basis for ⺢n , then A is diagonalizable.

40. If V = span{ex , e−x } and T: V → V is defined by T (f (x)) = f  (x), then T is diagonalizable.

24. If an n × n matrix A has only m distinct eigenvalues with m < n, then A is not diagonalizable.

Confirming Pages

Confirming Pages

CHAPTER

Inner Product Spaces CHAPTER OUTLINE

6.1 6.2 6.3 6.4 6.5 6.6 6.7 6.8

The Dot Product on ⺢n 323 Inner Product Spaces 333 Orthonormal Bases 342 Orthogonal Complements 355 Application: Least Squares Approximation 366 Diagonalization of Symmetric Matrices 377 Application: Quadratic Forms 385 Application: Singular Value Decomposition 392

ccording to a growing number of scientists, a contributing factor in the rise in global temperatures is the emission of greenhouse gases such as carbon dioxide. The primary source of carbon dioxide in the atmosphere is from the burning of fossil fuels. Table 1∗ gives the global carbon emissions, in billions of tons, from burning fossil fuels during the period from 1950 through 2000. A scatterplot of the data, shown in Fig. 1, exhibits an increasing trend which can be approximated with a straight line, also shown in Fig. 1, which best fits the data even though

A

Billion Tons

8

2000 1995 1990 1985 1980 1975 1970 1965 1960 1955 1950

Figure 1

Table 1 Global Carbon Emissions 1950–2000 1950

1.63

1980

5.32

1955

2.04

1985

5.43

1960

2.58

1990

6.14

1965

3.14

1995

6.40

1970

4.08

2000

6.64

1975

4.62

∗ Worldwatch Institute, Vital Signs 2006–2007. The trends that are shaping our future, W. W. Norton and Company, New York London, 2006.

321

Confirming Pages

322

Chapter 6 Inner Product Spaces

there is no one line that passes through all the points. To find this line, let (xi , yi ), for i = 1, 2, . . . , 11, denote the data points where xi is the year, starting with x1 = 1950, and yi is the amount of greenhouse gas being released into the atmosphere. The linear equation y = mx + b will best fit these data if we can find values for m and b such that the sum of the square errors 11
 2 yi − (mxi + b) = [1.63 − (1950m − b)]2 + · · · + [6.64 − (2000m − b)]2 i=1

is minimized. One method for finding the numbers m and b uses results from multivariable calculus. An alternative approach, using linear algebra, is derived from the ideas developed in this chapter. To use this approach, we attempt to look for numbers m and b such that the linear system  m(1950) + b = 1.63     m(1955) + b = 2.04 ..   .    m(2000) + b = 6.64 is satisfied. In matrix form, this system is given by Ax = b, where    1950 1   1955 1       1960 1       1965 1       1970 1       m   1975 1 and b = x = A=    b   1980 1       1985 1       1990 1       1995 1  2000 1

1.63 2.04 2.58 3.14 4.08 4.62 5.32 5.43 6.14 6.40 6.64

                 

Now, since there is no one line going through each of the data points, an exact solution to the previous linear system does not exist! However, as we will see, the best-fit line comes from finding a vector x so that Ax is as close as possible to b. In this case, the equation of the best-fit line, shown in Fig. 1, is given by y = 0.107x − 207.462

In the last several chapters we have focused our attention on algebraic properties of abstract vector spaces derived from our knowledge of Euclidean space. For example, the observations made in Sec. 2.1 regarding the behavior of vectors in ⺢n provided us with a model for the axiomatic development of general vector spaces given in

Confirming Pages

6.1 The Dot Product on ⺢n

323

Sec. 3.1. In this chapter we follow a similar approach as we describe the additional structures required to generalize the geometric notions of length, distance, and angle from ⺢2 and ⺢3 to abstract vector spaces. These geometric ideas are developed from a generalization of the dot product of two vectors in ⺢n , called the inner product, which we define in Sec. 6.2. We begin with a description of the properties of the dot product on ⺢n and its relation to the geometry in Euclidean space. ß

6.1

The Dot Product on ⺢n In Definition 2 of Sec. 1.3, we defined the dot product of two vectors     v1 u1  v2   u2      and v= .  u= .   ..   ..  un vn in ⺢n as

u · v = u1 v 1 + u 2 v 2 + · · · + u n v n To forge a connection between the dot product and the geometry of Euclidean space, recall that in ⺢3 the distance from a point (x1 , x2 , x3 ) to the origin is given by  d = x12 + x22 + x32 Now let



 v1 v =  v2  v3

be a vector in ⺢3 in standard position. Using the distance formula, the length (or norm) of v, which we denote by || v ||, is defined as the distance from the terminal point of v to the origin and is given by  || v || = v12 + v22 + v32 Observe that the quantity under the square root symbol can be written as the dot product of v with itself. So the length of v can be written equivalently as √ || v || = v · v Generalizing this idea to ⺢n , we have the following definition.

DEFINITION 1

Length of a Vector in ⺢n

The length (or norm) of a vector   v1  v2    v= .   ..  vn

Confirming Pages

324

Chapter 6 Inner Product Spaces

in ⺢n , denoted by || v ||, is defined as  || v || = v12 + v22 + · · · + vn2 √ = v·v  1 As an illustration, let v =  2 . Then −1  √ √ || v || = v · v = 12 + 22 + (−1)2 = 6 

y

u−v u

v x

In Sec. 2.1, it was shown that the difference u − v, of two vectors u and v in standard position, is a vector from the terminal point of v to the terminal point of u, as shown in Fig. 2. This provides the rationale for the following definition. Figure 2

DEFINITION 2

Distance Between Vectors in ⺢n   u1  u2    u= .   ..  un be vectors in

⺢n .

Let  and

  v= 

v1 v2 .. .

    

vn

The distance between u and v is defined by  || u − v || = (u − v) · (u − v)

Since the orientation of a vector does not affect its length, the distance from u to v is equal to the distance from v to u, so that || u − v || = || v − u ||

EXAMPLE 1

Show that if v is a vector in ⺢n and c is a real number, then || cv || = | c | || v ||

Solution

Let

 v1  v2   v=  ...  

vn

Confirming Pages

6.1 The Dot Product on ⺢n

Then

325

 (cv) · (cv) = (cv1 )2 + (cv2 )2 + · · · + (cvn )2   = c2 v12 + c2 v22 + · · · + c2 vn2 = c2 (v12 + v22 + · · · + vn2 )  = |c| v12 + v22 + · · · + vn2 = | c | || v ||

|| cv || =



The result of Example 1 provides verification of the remarks following Definition 2 of Sec. 2.1 on the effect of multiplying a vector v by a real number c. Indeed, as a consequence of Example 1, if |c| > 1, then cv is a stretching or dilation of v; and is a shrinking or contraction of v if |c| < 1. If, in addition, c < 0, then the direction of cv is reversed. As an illustration, let v be a vector in ⺢n with || v || = 10. Then 2v has length 20. The vector −3v has length 30 and points in the opposite direction of v. If the length of a vector in ⺢n is 1, then v is called a unit vector.

PROPOSITION 1

Let v be a nonzero vector in ⺢n . Then uv =

1 v || v ||

is a unit vector in the direction of v. Proof Using Definition 1 and the result of Example 1, we have      1   1  || v ||      || v || v =  || v ||  || v || = || v || = 1 Since 1/ || v || > 0, then the vector uv has the same direction as v.

EXAMPLE 2

Solution

 1 v= 2  −2 Find the unit vector uv in the direction of v.  Observe that || v || = 12 + 22 + (−2)2 = 3. Then by Proposition 1, we have   1 1 1 2  uv = v = 3 3 −2 Let



Revised Confirming Pages

326

Chapter 6 Inner Product Spaces

Theorem 1 gives useful properties of the dot product. The proofs are straightforward and are left to the reader. THEOREM 1

EXAMPLE 3 Solution

Let u, v, and w be vectors in ⺢n and c a scalar. 1. u · u ≥ 0 2. u · u = 0 if and only if u = 0 3. u · v = v · u 4. u · (v + w) = u · v + u · w and (u + v) · w = u · w + v · w 5. (cu) · v = c(u · v)

Let u and v be vectors in ⺢n . Use Theorem 1 to expand (u + v) · (u + v). By repeated use of part 4, we have (u + v) · (u + v) = (u + v) · u + (u + v) · v = u·u + v·u + u·v + v·v Now, by part 3, v · u = u · v, so that (u + v) · (u + v) = u · u + 2u · v + v · v or equivalently, (u + v) · (u + v) = || u ||2 + 2u · v + || v ||2 The next result, know as the Cauchy-Schwartz inequality, is fundamental in developing a geometry on ⺢n . In particular, this inequality makes it possible to define the angle between vectors.

THEOREM 2

Cauchy-Schwartz Inequality

If u and v are in vectors in ⺢n , then |u · v| ≤ || u || || v ||

Proof If u = 0, then u · v = 0. We also know, in this case, that || u || || v || = 0 || v || = 0 so that equality holds. Now suppose that u = 0 and k is a real number. Consider the dot product of the vector ku + v with itself. By Theorem 1, part 1, we have (ku + v) · (ku + v) ≥ 0 Now, by Theorem 1, part 4, the left-hand side can be expanded to obtain k 2 (u · u) + 2k(u · v) + v · v ≥ 0 Observe that the expression on the left-hand side is quadratic in the variable k with real coefficients. Letting a = u · u, b = u · v, and c = v · v, we rewrite this inequality as ak 2 + 2bk + c ≥ 0

Revised Confirming Pages

6.1 The Dot Product on ⺢n

327

This inequality imposes conditions on the coefficients a, b, and c. Specifically, the equation ak 2 + 2bk + c = 0 must have at most one real zero. Thus, by the quadratic formula, the discriminant (2b)2 − 4ac ≤ 0, or equivalently, (u · v)2 ≤ (u · u)(v · v) After taking the square root of both sides, we obtain |u · u| ≤ || v || || v || as desired.

The Angle between Two Vectors

y

|| u − v || || v ||

θ

|| u || x

With the Cauchy-Schwartz inequality in hand, we are now in a position to define the angle between two vectors. To motivate this idea, let u and v be nonzero vectors in ⺢2 with u − v the vector connecting the terminal point of v to the terminal point of u, as shown in Fig. 3. As these three vectors form a triangle in ⺢2 , we apply the law of cosines to obtain || u − v ||2 = || u ||2 + || v ||2 − 2 || u || || v || cos θ

Figure 3

Using Theorem 1, we rewrite this equation as u · u − 2u · v + v · v = u · u + v · v − 2 || u || || v || cos θ After simplifying and solving for cos θ, we obtain u·v || u || || v || Our aim now is to extend this result and use it as the definition of the cosine of the angle between vectors in n-dimensional Euclidean space. To do so, we need | cos θ| ≤ 1 for every angle θ, that is, cos θ =

−1 ≤

u·v ≤1 || u || || v ||

for all vectors u and v in ⺢n . But this fact follows immediately from the CauchySchwartz inequality. Indeed, dividing both sides of |u · v| ≤ || u || || v || by || u || || v ||, we obtain |u · v| ≤1 || u || || v || so that −1 ≤

u·v ≤1 || u || || v ||

This permits us to make the following definition.

Revised Confirming Pages

328

Chapter 6 Inner Product Spaces

DEFINITION 3

EXAMPLE 4

Solution

Angle Between Vectors in ⺢n If u and v are vectors in ⺢n , then the cosine of the angle θ between the vectors is defined by u·v cos θ = || u || || v ||

Find the angle between the two vectors   2 and u =  −2  3 The lengths of the vectors are  √ || u || = 22 + (−2)2 + 32 = 17

and

 −1 v= 2  2 

|| v || =



(−1)2 + 22 + 22 = 3

and the dot product of the vectors is u · v = 2(−1) + (−2)2 + 3(2) = 0 By Definition 3, the cosine of the angle between u and v is given by u·v =0 cos θ = || u || || v || Hence, θ = π/2 and the vectors are perpendicular. Such vectors are also called orthogonal.

DEFINITION 4

Orthogonal Vectors The vectors u and v are called orthogonal if the angle between them is π/2.

As a direct consequence of Definition 3, we see that if u and v are nonzero vectors in ⺢n with u · v = 0, then cos θ = 0, so that θ = π/2. On the other hand, if u and v are orthogonal, then cos θ = 0, so that u·v =0 therefore u·v = 0 || u || || v || The zero vector is orthogonal to every vector in ⺢n since 0 · v = 0, for every vector v. These results are given in Proposition 2.

Revised Confirming Pages

6.1 The Dot Product on ⺢n

PROPOSITION 2

329

Two nonzero vectors u and v in ⺢n are orthogonal if and only if u · v = 0. The zero vector is orthogonal to every vector in ⺢n .

One consequence of Proposition 2 is that if u and v are orthogonal, then || u + v ||2 = (u + v) · (u + v) = || u ||2 + 2u · v + || v ||2 = || u ||2 + || v ||2 This is a generalization of the Pythagorean theorem to ⺢n . Theorem 3 gives several useful properties of the norm in ⺢n .

THEOREM 3

Properties of the Norm in ⺢n 1. 2. 3. 4.

Let v be a vector in ⺢n and c a scalar.

|| v || ≥ 0 || v || = 0 if and only if v = 0 || cv || = |c| || v || (Triangle inequality) || u + v || ≤ || u || + || v ||

Proof Parts 1 and 2 follow immediately from Definition 1 and Theorem 1. Part 3 is established in Example 1. To establish part 4, we have || u + v ||2 = (u + v) · (u + v) = (u · u) + 2(u · v) + (v · v) = || u ||2 + 2(u · v) + || v ||2 ≤ || u ||2 + 2|u · v| + || v ||2 Now, by the Cauchy-Schwartz inequality, |u · v| ≤ || u || || v ||, so that

y

|| u + v || || v || || v ||

|| u ||

= (|| u || + || v ||)2 After taking square roots of both sides of this equation, we obtain

x

Figure 4

|| u + v ||2 ≤ || u ||2 + 2 || u || || v || + || v ||2

|| u + v || ≤ || u || + || v ||

Geometrically, part 4 of Theorem 3 confirms our intuition that the shortest distance between two points is a straight line, as seen in Fig. 4.

Revised Confirming Pages

330

Chapter 6 Inner Product Spaces

PROPOSITION 3

Let u and v be vectors in ⺢n . Then || u + v || = || u || + || v || if and only if the vectors have the same direction. Proof First suppose that the vectors have the same direction. Then the angle between the vectors is 0, so that cos θ = 1 and u · v = || u || || v ||. Therefore, || u + v ||2 = (u + v) · (u + v) = || u ||2 + 2(u · v) + || v ||2 = || u ||2 + 2 || u || || v || + || v ||2 = (|| u || + || v ||)2 Taking square roots of both sides of the previous equation gives || u + v || = || u || + || v ||. Conversely, suppose that || u + v || = || u || + || v ||. After squaring both sides, we obtain || u + v ||2 = || u ||2 + 2 || u || || v || + || v ||2 However, we also have || u + v ||2 = (u + v) · (u + v) = || u ||2 + 2u · v + || v ||2 Equating both expressions for || u + v ||2 gives || u ||2 + 2 || u || || v || + || v ||2 = || u ||2 + 2u · v + || v ||2 Simplifying the last equation, we obtain u · v = || u || || v || and hence u·v =1 || u || || v || Therefore, cos θ = 1, so that θ = 0 and the vectors have the same direction.

Fact Summary All vectors are in ⺢n . 1. The length of a vector and the distance between two vectors are natural extensions of the same geometric notions in ⺢2 and ⺢3 . 2. The dot product of a vector with itself gives the square of its length and is 0 only when the vector is the zero vector. The dot product of two vectors is commutative and distributes through vector addition. 3. By using the Cauchy-Schwartz inequality |u · v| ≤ || u || || v ||, the angle between vectors is defined by u·v cos θ = || u || || v || 4. Two vectors are orthogonal if and only if the dot product of the vectors is 0.

Confirming Pages

6.1 The Dot Product on ⺢n

331

5. The norm of a vector is nonnegative, is 0 only when the vector is the zero vector, and satisfies || cu || = |c| || u ||

and

|| u + v || ≤ || u || + || v ||

Equality holds in the last inequality only when the vectors are in the same direction. 6. If u and v are orthogonal vectors, then the Pythagorean theorem || u + v ||2 = || u ||2 + || v ||2 holds.

Exercise Set 6.1 In Exercises 1–4, let   0 u= 1  3   1 w= 1  −3





1 v =  −1  2

10. Find a vector w that is orthogonal to both u and v. In Exercises 11–16, let   −3 u =  −2  3

 −1 v =  −1  −3 

11. Find || u ||.

Compute the quantity.

12. Find the distance between u and v.

1. u · v u·v 2. v·v 3. u · (v + 2w) u·w w 4. w·w In Exercises 5–10, let   1 u= 5

13. Find a unit vector in the direction of u. 14. Find the cosine of the angle between the two vectors. Are the vectors orthogonal? Explain. 15. Find a vector in the opposite direction of v with length 3.  v=

2 1



5. Find || u ||. 6. Find the distance between u and v. 7. Find a unit vector in the direction of u. 8. Find the cosine of the angle between the two vectors. Are the vectors orthogonal? Explain. 9. Find a vector in the direction of v with length 10.

16. Find a vector w that is orthogonal to both u and v.   c is orthogonal to 17. Find a scalar c, so that 3   −1 . 2   −1 18. Find a scalar c, so that  c  is orthogonal to 2   0  2 . −1

Confirming Pages

332

Chapter 6 Inner Product Spaces

 1 v1 =  2  −1   −1 v3 =  −2  1   3 v5 =  −1  1 

   0 2 28. u =  3  v =  2  3 −1 

In Exercises 19–22, let  6 v2 =  −2  2 √   −1/√3 v4 =  1/√3  1/ 3 

19. Determine which of the vectors are orthogonal. 20. Determine which of the vectors are in the same direction. 21. Determine which of the vectors are in the opposite direction. 22. Determine which of the vectors are unit vectors. In Exercises 23–28, find the projection of u onto v given by u·v w= v v·v The vector w is called the orthogonal projection of u onto v. Sketch the three vectors u, v, and w.     4 2 v= 23. u = 0 3     4 −2 v= 24. u = 0 3     3 4 v= 25. u = 1 3     1 5 26. u =  2  v =  0  0 1     5 1 27. u =  0  v =  2  1 0

29. Let S = {u1 , u2 , . . . , un } and suppose v · ui = 0 for each i = 1, . . . , n. Show that v is orthogonal to every vector in span(S). 30. Let v be a fixed vector in ⺢n and define S = {u | u · v = 0}. Show that S is a subspace of ⺢n . 31. Let S = {v1 , v2 , . . . , vn } be a set of nonzero vectors which are pairwise orthogonal. That is, if i = j , then vi · vj = 0. Show that S is linearly independent. 32. Let A be an n × n invertible matrix. Show that if i = j , then row vector i of A and column vector j of A−1 are orthogonal. 33. Show that for all vectors u and v in ⺢n , || u + v ||2 + || u − v ||2 = 2 || u ||2 + 2 || v ||2 34. a. Find a vector that is orthogonal to every vector in the plane P : x + 2y − z = 0. b. Find a matrix A such that the null space N (A) is the plane x + 2y − z = 0. 35. Suppose that the column vectors of an n × n matrix A are pairwise orthogonal. Find At A. 36. Let A be an n × n matrix and u and v vectors in ⺢n . Show that u · (Av) = (At u) · v 37. Let A be an n × n matrix. Show that A is symmetric if and only if (Au) · v = u · (Av) for all u and v in ⺢n . Hint: See Exercise 36.

Confirming Pages

6.2 Inner Product Spaces

ß

6.2

333

Inner Product Spaces In Sec. 6.1 we introduced the concepts of the length of a vector and the angle between vectors in Euclidean space. Both of these notions are defined in terms of the dot product and provide a geometry on ⺢n . Notice that the dot product on ⺢n defines a function from ⺢n × ⺢n into ⺢. That is, the dot product operates on two vectors in ⺢n , producing a real number. To extend these ideas to an abstract vector space V , we require a function from V × V into ⺢ that generalizes the properties of the dot product given in Theorem 1 of Sec. 6.1.

DEFINITION 1

Inner Product Let V be a vector space over ⺢. An inner product on V is a function that associates with each pair of vectors u and v in V a real number, denoted by u, v, that satisfies the following axioms: 1. 2. 3. 4.

u, u ≥ 0 and u, u = 0 if and only if u = 0 (positive definite) u, v = v, u (symmetry) u + v, w = u, w + v, w cu, v = c u, v

The last two properties make the inner product linear in the first variable. Using the symmetry axiom, it can also be shown that the inner product is linear in the second variable, that is, 

3 . u, v + w = u, v + u, w  4 . u, cv = c u, v With these additional properties, the inner product is said to be bilinear. A vector space V with an inner product is called an inner product space. By Theorem 1 of Sec. 6.1, the dot product is an inner product on Euclidean n-space. Thus, ⺢n with the dot product is an inner product space.  1 . Find all vectors u in ⺢2 such that u, v = 0, where the inner 3 product is the dot product.   x , then If u = y u, v = u · v = x + 3y 

EXAMPLE 1

Solution

Let v =

Confirming Pages

334

Chapter 6 Inner Product Spaces y 5

v ⫺5

5

x

S ⫺5

Figure 1

so that u, v = 0 if and only if y = − 13 x. Therefore, the set of all vectors such   1 u, . The vector v and the set S are that v = 0 is given by S = span − 13 shown in Fig. 1. Notice that each vector in S is perpendicular to v. For another example, consider the vector space of polynomials P2 . To define an inner product on P2 , let p(x) = a0 + a1 x + a2 x 2 and q(x) = b0 + b1 x + b2 x 2 . Now let  · , ·  : P2 × P2 → ⺢ be the function defined by p, q = a0 b0 + a1 b1 + a2 b2 Notice that this function is similar to the dot product on ⺢3 . The proof that P2 is an inner product space follows along the same lines as the proof of Theorem 1 of Sec. 6.1. Another way to define an inner product on P2 is to use the definite integral. Specifically, let p(x) and q(x) be polynomials in P2 , and let  · , ·  be the function defined by  1

p, q =

p(x)q(x) dx 0

This function is also an inner product on P2 . The justification, in this case, is based on the fundamental properties of the Riemann integral which can be found in any text on real analysis.

EXAMPLE 2

Let V = P2 with inner product defined by  1 p, q = p(x)q(x) dx 0

a. Let p(x) = 1 − x and q(x) = 1 − x + 2x 2 . Find p, q. b. Let p(x) = 1 − x 2 . Verify that p, p > 0. 2

Solution

a. Using the definition given for the inner product, we have  1 p, q = (1 − x 2 )(1 − x + 2x 2 ) dx 0 1 = (1 − x + x 2 + x 3 − 2x 4 ) dx 0  1  1 1 1 2 = x − x 2 + x 3 + x 4 − x 5  2 3 4 5 0 41 = 60

Confirming Pages

6.2 Inner Product Spaces

335

b. The inner product of p with itself is given by  1 p, p = (1 − x 2 )(1 − x 2 ) dx 0 1 = (1 − 2x 2 + x 4 ) dx 0  1  2 1 = x − x 3 + x 5  3 5 0 8 >0 = 15

Example 3 gives an illustration of an inner product on ⺢n that is not the dot product.

EXAMPLE 3

Solution

   v1 u1 and v= u= u2 v2 be vectors in V . Let k be a fixed positive real number, and define the function  · , · : ⺢2 × ⺢2 → ⺢ by u, v = u1 v1 + ku2 v2 Show that V is an inner product space. Let V = ⺢2 and



First we show that  · , ·  is nonnegative. From the definition above, we have u, u = u21 + ku22 Since k > 0, then u21 + ku22 ≥ 0 for every vector u. In addition, u21 + ku22 = 0

if and only if

u1 = u2 = 0

or equivalently if u = 0. The property of symmetry also holds since  Next, let w =

u, v = u1 v1 + ku2 v2 = v1 u1 + kv2 u2 = v, u  w1 be another vector in ⺢2 . Then w2 u + v, w = (u1 + v1 )w1 + k(u2 + v2 )w2 = (u1 w1 + ku2 w2 ) + (v1 w1 + kv2 w2 ) = u, w + v, w

Revised Confirming Pages

336

Chapter 6 Inner Product Spaces

Finally, if c is a scalar, then cu, v = (cu1 )v1 + k(cu2 )v2 = c(u1 v1 + ku2 v2 ) = c u, v Therefore, ⺢2 with this definition for  · , ·  is an inner product space.

Notice that in Example 3 the requirement that k > 0 is necessary. For example,   1 , then the inner product of u with itself is given by u, u = if k = −1 and u = 2 (1)2 + (−1)(2)2 = −3, which violates the first axiom of Definition 1. Again using ⺢n as our model, we now define the length (or norm) of a vector v in an inner product space V as  || v || = v, v The distance between two vectors u and v in V is then defined by  || u − v || = u − v, u − v The norm in an inner product space satisfies the same properties as the norm in ⺢n . The results are summarized in Theorem 4.

THEOREM 4

Properties of the Norm in an Inner Product Space Let u and v be vectors in an inner product space V and c a scalar. 1. 2. 3. 4. 5.

EXAMPLE 4

|| v || ≥ 0 || v || = 0 if and only if v = 0 || cv || = |c| || v || | u, v | ≤ || u || || v || (Cauchy-Schwartz inequality) || u + v || ≤ || u || + || v || (Triangle inequality)

Let V = ⺢2 with inner product defined by u, v = u1 v1 + 3u2 v2 

Let u=

2 −2



 and

v=

1 4

a. Verify that the Cauchy-Schwartz inequality is upheld. b. Verify that the Triangle Inequality is upheld.



Revised Confirming Pages

6.2 Inner Product Spaces

Solution

337

a. Using the given definition for the inner product, we have |u, v| = |(2)(1) + 3(−2)(4)| = | − 22| = 22 The norms of u and v are given, respectively, by   √ || u || = u, u = (2)2 + 3(−2)2 = 16 = 4 and || v || =



v, v =



(1)2 + 3(4)2 =

√ 49 = 7

Since 22 = |u, v| < || u || || v || = 28 the Cauchy-Schwartz inequality is satisfied for the vectors u and v. b. To verify the triangle inequality, observe that       3 1 2 = + u+v= 2 4 −2 so that || u + v || = Since

 √ (3)2 + 3(2)2 = 21

√

21 = || u + v || < || u || + || v || = 4 + 7 = 11 the triangle inequality holds for u and v.

Orthogonal Sets Taking the same approach as in Sec. 6.1, we define the cosine of the angle between the vectors u and v in an inner product space V by u, v cos θ = || u || || v || As before, the vectors u and v in V are orthogonal provided that u, v = 0.

EXAMPLE 5

Let V = P2 with inner product defined by  1 p, q = p(x)q(x) dx −1

a. Show that the vectors in

  S = 1, x, 12 (3x 2 − 1)

are mutually orthogonal. b. Find the length of each vector in S.

Confirming Pages

338

Chapter 6 Inner Product Spaces

Solution

a. The inner product of each pair of vectors in S is  1 1 1, x = x dx = 12 x 2 −1 = 0 −1 1    1  1 1 1 1, (3x 2 − 1) = (3x 2 − 1) dx = (x 3 − x) = 0 2 2 −1 2 −1     1  1 1 2 1 1 1 3 2 3 4 (3x − x) dx = x − x  =0 x, (3x − 1) = 2 2 4 2 −1 2 −1 Since each pair of distinct vectors is orthogonal, the vectors in S are mutually orthogonal. b. For the lengths of the vectors in S, we have   1  √ || 1 || = 1, 1 = dx = 2  −1  1   || x || = x, x = x 2 dx = 23 −1  1   1  1 1 1 2 2 2 2 − 1) = 2 − 1)2 dx =  (3x − 1) = (3x − 1), (3x (3x 2 2 2 4 −1 5

DEFINITION 2

Orthogonal Set A set of vectors {v1 , v2 , . . . , vn } in an inner product space is called  orthogonal if the vectors are mutually orthogonal; that is, if i = j , then vi , vj = 0. If in addition, || vi || = 1, for all i = 1, . . . n, then the set of vectors is called orthonormal. Observe that the vectors of Example 5 form an orthogonal set. They do not, however, form an orthonormal set. Proposition 4 shows that the zero vector is orthogonal to every vector in an inner product space.

PROPOSITION 4

Let V be an inner product space. Then v, 0 = 0 for every vector v in V . Proof Let v be a vector in V . Then v, 0 = v, 0 + 0 = v, 0 + v, 0 After subtracting v, 0 from both sides of the previous equation, we have v, 0 = 0 as desired.

A useful property of orthogonal sets of nonzero vectors is that they are linearly independent. For example, the set of coordinate vectors {e1 , e2 , e3 } in ⺢3 is orthogonal

Confirming Pages

6.2 Inner Product Spaces

339

and linearly independent. Theorem 5 relates the notions of orthogonality and linear independence in an inner product space. THEOREM 5

If S = {v1 , v2 , . . . , vn } is an orthogonal set of nonzero vectors in an inner product space V , then S is linearly independent. Proof Since the set S is an orthogonal set of nonzero vectors,   vi , vi  = || vi ||2 = 0 for i = j and vi , vj = 0

for all i

Now suppose that c1 v1 + c2 v2 + · · · + cn vn = 0 The vectors are linearly independent if and only if the only solution to the previous equation is the trivial solution c1 = c2 = · · · = cn = 0. Now let vj be an element of S. Take the inner product on both sides of the previous equation with vj so that     vj , (c1 v1 + c2 v2 + · · · + cj −1 vj −1 + cj vj + cj +1 vj +1 + · · · + cn vn ) = vj , 0 By the linearity of the inner product and the fact that S is orthogonal, this equation reduces to     cj vj , vj = vj , 0   Now, by Proposition 4 and the fact that  vj  = 0, we have  2 so that cj = 0 cj  vj  = 0 Since this holds for each j = 1, . . . , n, then c1 = c2 = · · · = cn = 0 and therefore S is linearly independent.

COROLLARY 1

If V is an inner product space of dimension n, then any orthogonal set of n nonzero vectors is a basis for V . The proof of this corollary is a direct result of Theorem 12 of Sec. 3.3. Theorem 6 provides us with an easy way to find the coordinates of a vector relative to an orthonormal basis. This property underscores the usefulness and desirability of orthonormal bases.

THEOREM 6

If B = {v1 , v2 , . . . , vn } is an ordered orthonormal basis for an inner product space V and v = c1 v1 + c2 v2 + · · · + cn vn , then the coordinates of v relative to B are given by ci = vi , v for each i = 1, 2, . . . , n. Proof Let vi be a vector in B. Taking the inner product on both sides of v = c1 v1 + c2 v2 + · · · + ci−1 vi−1 + ci vi + ci+1 vi+1 + · · · + cn vn

Confirming Pages

340

Chapter 6 Inner Product Spaces

with vi on the right gives v, vi  = (c1 v1 + c2 v2 + · · · + ci−1 vi−1 + ci vi + ci+1 vi+1 + · · · + cn vn ), vi  = c1 v1 , vi  + · · · + ci vi , vi  + · · · + cn vn , vi  Since B is an orthonormal set, this reduces to v, vi  = ci vi , vi  = ci As this argument can be carried out for any vector in B, then ci = v, vi  for all i = 1, 2, . . . , n.

In Theorem 6, if the ordered basis B is orthogonal and v is any vector in V , then the coordinates relative to B are given by ci =

v, vi  vi , vi 

for each i = 1, . . . , n

so that v=

v, v1  v, v2  v, vn  v1 + v2 + · · · + vn v1 , v1  v2 , v2  vn , vn 

Fact Summary All vectors are in an inner product space. 1. An inner product on a vector space is a function that assigns to each pair of vectors a real number and generalizes the properties of the dot product on ⺢n . n 2. The norm √ of a vector is defined analogously to the definition in ⺢ by || v || = v, v. 3. An orthogonal set of vectors is linearly independent. Thus, any set of n orthogonal vectors is a basis for an inner product space of dimension n. 4. When an arbitrary vector is written in terms of the vectors in an orthogonal basis, the coefficients are given explicitly by an expression in terms of the inner product. If {v1 , . . . , vn } is the orthogonal basis and v is an arbitrary vector, then v, v1  v, v2  v, vn  v1 + v2 + · · · + vn v1 , v1  v2 , v2  vn , vn  If case B is an orthonormal basis, then v=

v = v, v1  v1 + · · · + v, vn  vn

Confirming Pages

6.2 Inner Product Spaces

341

Exercise Set 6.2 In Exercises 1–10, determine whether V is an inner product space. 1. V = ⺢2 u, v = u1 v1 − 2u1 v2 − 2u2 v1 + 3u2 v2 2. V = ⺢2 u, v = −u1 v1 + 2u1 v2 3. V = ⺢2 u, v = u21 v12 + u22 v22 4. V = ⺢3 u, v = u1 v1 + 2u2 v2 + 3u3 v3 5. V = ⺢n u, v = u · v 6. V = Mm×n A, B = tr(B t A) 7. V = Mm×n m
n
A, B = aij bij 8. V = Pn p, q =

i=1 j =1 n


 f, g =

b

f (x)g(x) dx a

a. Find the distance between the vectors f and g. b. Find the cosine of the angle between the vectors f and g. 15. f (x) = 3x − 2, g(x) = x 2 + 1; a = 0, b = 1 16. f (x) = cos x, g(x) = sin x; a = −π, b = π 17. f (x) = x, g(x) = ex ; a = 0, b = 1 18. f (x) = ex , g(x) = e−x ; a = −1, b = 1 In Exercises 19 and 20, let V = P2 with inner product p, q =

2


pi qi

i=0

a. Find the distance between the vectors p and q. b. Find the cosine of the angle between the vectors p and q. 19. p(x) = x 2 + x − 2, q(x) = −x 2 + x + 2 20. p(x) = x − 3, q(x) = 2x − 6

pi qi

i=0

9. V = C (0) [−1, 1] 1 f, g = −1 f (x)g(x)e−x dx

10. V = C (0) [−1, 1] 1 f, g = −1 f (x)g(x)x dx In Exercises 11–14, let V = C (0) [a, b] with inner product  b f, g = f (x)g(x) dx a

Verify that the set of vectors is orthogonal. 11. {1, cos x, sin x}; a = −π, b = π   12. 1, x, 12 (5x 3 − 3x) ; a = −1, b = 1   13. 1, 2x − 1, −x 2 + x − 16 ; a = 0, b = 1 14. {1, cos x, sin x, cos 2x, sin 2x}; a = −π, b = π In Exercises 15–18, let V = C (0) [a, b] with inner product

In Exercises 21–24, let V = Mn×n with inner product A, B = tr(B t A) a. Find the distance between the vectors A and B. b. Find the cosine of the angle between the vectors A and B.     2 1 1 2 B= 21. A = 1 3 2 −1     0 2 3 1 B= 22. A = 1 −2 0 −1   1 0 −2 1 1  23. A =  −3 −3 −3 −2   3 −1 −1 2 3  B =  −3 −1 −2 1

Confirming Pages

342

Chapter 6 Inner Product Spaces

 2 1 2 24. A =  3 1 0  3 2 1   0 0 1 B= 3 3 2  1 0 2 

30. Let A be a fixed 2 × 2 matrix, and define a function on ⺢2 × ⺢2 by u, v = ut Av a. Verify that if A = I , then the function defines an inner product.   2 −1 , then the b. Show that if A = −1 2 function defines an inner product.  3 2 , then the function c. Show that if A = 2 0 does not define an inner product.

25. Describe the set of all vectors in ⺢2 that are   2 . orthogonal to 3 26. Describe the set of all vectors in ⺢2 that are   1 . orthogonal to −b 27. Describe the set of all vectors in ⺢3 that are   2 orthogonal to  −3 . 1 28. Describe the set of all vectors in ⺢3 that are   1 orthogonal to  1 . 0

31. Define an inner product on C (0) [−a, a] by  a f, g = f (x)g(x) dx −a

Show that if f is an even function and g is an odd function, then f and g are orthogonal. 32. Define an inner product on C (0) [−π, π] by  π f, g = f (x)g(x) dx −π

Show {1, cos x, sin x, cos 2x, sin 2x, . . .}

29. For f and g in C (0) [0, 1] define the inner product by 

is an orthogonal set. (See Exercise 31.)

1

e.

6.3

ß

a. b. c. d.

f, g = f (x)g(x) dx 0  2 3 Find x , x .   Find ex , e−x . Find || 1 || and || x ||. Find the angle between f (x) = 1 and g(x) = x. Find the distance between f (x) = 1 and g(x) = x.

33. In an inner product space, show that if the set {u1 , u2 } is orthogonal, then for scalars c1 and c2 the set {c1 u1 , c2 u2 } is also orthogonal. 34. Show that if u, v and u, v are two different inner products on V , then their sum u, v = u, v + u, v defines another inner product.

Orthonormal Bases In Theorem 6 of Sec. 6.2 we saw that if B = {v1 , v2 , . . . , vn } is an ordered orthonormal basis of an inner product space V , then the coordinates of any vector v in V are given by an explicit formula using the inner product on the space. In particular, these coordinates relative to B are given by ci = v, vi  for i = 1, 2, . . . n. For this reason, an orthonormal basis for an inner product space is desirable. As we have already seen,

Revised Confirming Pages

6.3 Orthonormal Bases

343

the set of coordinate vectors S = {e1 , e2 , . . . , en } is an orthonormal basis for ⺢n . In this section we develop a method for constructing an orthonormal basis for any finite dimensional inner product space.

Orthogonal Projections Of course, most of the bases we encounter are not orthonormal, or even orthogonal. We can, however, in a finite dimensional inner product space, transform any basis to an orthonormal basis. The method, called the Gram-Schmidt process, involves projections of vectors onto other vectors. To motivate this topic, let u and v be vectors in ⺢2 , as shown in Fig. 1(a). u

u

θ

θ v (a)

w

v

(b)

Figure 1

Our aim is to find a vector w that results from an orthogonal projection of u onto v, as shown in Fig. 1(b). To do this, recall from trigonometry that || w || || w || = || u || cos θ so that || u || Moreover, using the expression for cos θ, established at the beginning of Sec. 6.1, we have u·v u·v || w || = || u || cos θ = || u || = || u || || v || || v || This quantity is called the scalar projection of u onto v. Now, to find w, we take the product of the scalar projection with a unit vector in the direction of v, so that   u·v u·v v w= = v || v || || v || || v ||2 cos θ =

u

u − projv u

projv u Figure 2

v

Moreover, since || v ||2 = v · v, the vector w can be written in the form !u · v" v w= v·v This vector is called the orthogonal projection of u onto v and is denoted by projv u, so that !u · v" projv u = v v·v Another useful vector, shown in Fig. 2, is the vector u − projv u

Confirming Pages

344

Chapter 6 Inner Product Spaces

from u to projv u. From the manner in which projv u is defined, the vector u − projv u is orthogonal to projv u, as shown in Fig. 2. To verify algebraically that projv u and u − projv u are orthogonal, we show that the dot product of these two vectors is zero. That is, # $ # $ !u · v " ! u·v " v · u− v projv u · u − projv u = v·v v·v " u·v ! u·v = v·u − v·v v·v v·v u·v u·v = (v · u − u · v) = (u · v − u · v) = 0 v·v v·v

u θ w

v

In Fig. 1, the angle θ shown is an acute angle. If θ is an obtuse angle, then projv u gives the orthogonal projection of u onto the negative of v, as shown in Fig. 3. If θ = 90◦ , then projv u = 0.

Figure 3

EXAMPLE 1



Let u=

1 3



 and

v=

1 1



a. Find projv u. b. Find u − projv u, and verify that projv u is orthogonal to u − projv u. Solution

a. From the formula given above, we have        !u · v" (1)(1) + (3)(1) 2 1 1 = =2 v= projv u = 2 1 1 v·v (1)(1) + (1)(1) b. Using the result of part (a), we have       −1 2 1 u − projv u = = − 1 2 3

y

u − projv u

u

projv u v x

Figure 4

To show that projv u is orthogonal to u − projv u, we compute the dot product. Here we have     # $ −1 2 = (2)(−1) + (2)(1) = 0 · projv u · u − projv u = 1 2 See Fig. 4.

Definition 1 provides an extension of this idea to general inner product spaces.

Confirming Pages

6.3 Orthonormal Bases

DEFINITION 1

345

Orthogonal Projection Let u and v be vectors in an inner product space. The orthogonal projection of u onto v, denoted by projv u, is defined by u, v v v, v The vector u − projv u is orthogonal to projv u. projv u =

EXAMPLE 2

Define an inner product on P3 by



p, q =

1

p(x)q(x) dx 0

Let p(x) = x and q(x) = x 2 . a. Find projq p. b. Find p − projq p and verify that projq p and p − projq p are orthogonal. Solution

a. In this case



1

1 q, q = and 4 0 Now the projection of p onto q is given by p, q 5 projq p = q = x2 q, q 4 p, q =



1

x 3 dx =

0

x 4 dx =

1 5

b. From part (a), we have 5 p − projq p = x − x 2 4 To show that the vectors p and p − projq p are orthogonal, we show that the inner product is zero. Here we have       1  1 5 2 5 5 1 5 2 5 3 25 4 5 4 x x − x dx = x − x dx = x − x  =0 4 4 16 16 16 0 4 0 0

We now turn our attention to the construction of an orthonormal basis for an inner product space. The key to this construction is the projection of one vector onto another. As a preliminary step, let V = ⺢2 and let B = {v1 , v2 } be a basis, as shown in Fig. 5. Now, define the vectors w1 and w2 by w1 = v1

and

w2 = v2 − projv1 v2

Confirming Pages

346

Chapter 6 Inner Product Spaces

Recall from Example 1 that w2 defined in this way is orthogonal to w1 as shown in Fig. 6. We normalize these vectors by dividing each by its length, so that 

B =



w2 w1 , || w1 || || w2 ||



is an orthonormal basis for ⺢2 .

v2

v2 v1

v1 = w1

w2

projv1 v2

B = {v1 , v2 } is a basis for ⺢2 Figure 5

w2 = v2 − projv1 v2 Figure 6

To construct an orthonormal basis for ⺢n , we first need to extend this idea to general inner product spaces.

THEOREM 7

Every finite dimensional inner product space has an orthogonal basis. Proof The proof is by induction on the dimension n of the inner product space. First if n = 1, then any basis {v1 } is orthogonal. Now assume that every inner product space of dimension n has an orthogonal basis. Let V be an inner product space with dim(V ) = n + 1, and suppose that {v1 , v2 , . . . , vn , vn+1 } is a basis. Let W = span{v1 , v2 , . . . , vn }. Observe that dim(W ) = n. By the inductive hypothesis, W has an orthogonal basis B. Let B = {w1 , w2 , . . . , wn }. Notice that B  = {w1 , w2 , . . . , wn , vn+1 } is another basis for V . By Theorem 5 of Sec. 6.2, it suffices to find a nonzero vector w that is orthogonal to each vector in B. (Here is where we extend the idea presented just prior to the theorem.) Let w = vn+1 − projw1 vn+1 − projw2 vn+1 − · · · − projwn vn+1 vn+1 , w1  vn+1 , w2  vn+1 , wn  w1 − w2 − · · · − wn = vn+1 − w1 , w1  w2 , w2  wn , wn  Observe that w = 0 since if w = 0, then B  will be linearly dependent and therefore not a basis for V . To complete the proof, we must show that w is orthogonal to

Confirming Pages

6.3 Orthonormal Bases

347

each vector in B. To see this, let wi be a vector in B. Then   w, wi  = vn+1 − projw1 vn+1 − projw2 vn+1 − · · · − projwn vn+1 , wi vn+1 , w1  vn+1 , w2  w1 , wi  − w2 , wi  = vn+1 , wi  − w1 , w1  w2 , w2  vn+1 , wi  vn+1 , wn  wi , wi  − · · · − wn , wi  − ··· − wi , wi  wn , wn  Now, as each vector in B = {w1 , w2 , . . . , wn } is mutually orthogonal, the previous equation reduces to w, wi  = vn+1 , wi  − 0 − 0 − · · · − = vn+1 , wi  − vn+1 , wi  = 0

vn+1 , wi  wi , wi  − 0 − · · · − 0 wi , wi 

B

= {w1 , w2 , . . . , wn , w} is an orthogonal set of n + 1 vectors in V . Therefore That B  is a basis for V is due to Corollary 1 of Sec. 6.2. From Theorem 7, we also know that every finite dimensional vector space has an orthonormal basis. That is, if B = {w1 , w2 , . . . , wn } is an orthogonal basis, then dividing each vector by its length gives the orthonormal basis   w2 wn w1 B = , ,..., || w1 || || w2 || || wn ||

Gram-Schmidt Process Theorem 7 guarantees the existence of an orthogonal basis in a finite dimensional inner product space. The proof of Theorem 7 also provides a procedure for constructing an orthogonal basis from any basis of the vector space. The algorithm, called the Gram-Schmidt process, is summarized here. 1. Let B = {v1 , v2 , . . . , vn } be any basis for the inner product space V . 2. Use B to define a set of n vectors as follows: w1 = v1 v2 , w1  w1 w1 , w1  w3 = v3 − projw1 v3 − projw2 v3 v3 , w1  v3 , w2  w1 − w2 = v3 − w1 , w1  w2 , w2  .. . w2 = v2 − projw1 v2 = v2 −

wn = vn − projw1 vn − projw2 vn − · · · − projwn−1 vn = vn −

vn , w1  vn , w2  vn , wn−1  w1 − w2 − · · · − wn−1 w1 , w1  w2 , w2  wn−1 , wn−1 

Confirming Pages

348

Chapter 6 Inner Product Spaces

3. The set B  = {w1 , w2 , . . . , wn } is an orthogonal basis for V . 4. Dividing each of the vectors in B  by its length gives an orthonormal basis for the vector space V   w2 wn w1 , ,..., B  = || w1 || || w2 || || wn ||

A Geometric Interpretation of the Gram-Schmidt Process The orthogonal projection projv u of the vector u onto the vector v, in ⺢n , is the projection of u onto the one-dimensional subspace W = span{v}. See Figs. 2 and 3. As seen above, to construct an orthogonal basis from the basis B = {v1 , v2 , v3 }, the first step in the Gram-Schmidt process is to let w1 = v1 , and then perform an orthogonal projection of v2 onto span{v1 }. As a result, w1 is orthogonal to w2 = v2 − projw1 v2 . Our aim in the next step is to find a vector w3 that is orthogonal to the two-dimensional subspace span{w1 , w2 }. This is accomplished by projecting v3 separately onto the onedimensional subspaces span{w1 } and span{w2 }, as shown in Fig. 7. The orthogonal projections are projw1 v3 =

v3 , w1  w1 w1 , w1 

and

projw2 v3 =

v3 , w2  w2 w2 , w2 

Hence, the orthogonal projection of v3 onto span{w1 , w2 } is the sum of the projections projw1 v3 + projw2 v3 also shown in Fig. 7. Finally, the required vector is $ # w3 = v3 − projw1 v3 + projw2 v3 = v3 − projw1 v3 − projw2 v3 which is orthogonal to both w1 and w2 , as shown in Fig. 7. In general, when dim(W ) = n > 1, then the Gram-Schmidt process describes projecting the vector vn+1 onto n one-dimensional subspaces span{w1 }, span{w2 }, . . . , span{wn }. Then the vector wn+1 that is orthogonal to each of the vectors w1 , w2 , . . . , wn is obtained by subtracting each projection from the vector vn .

v3

w3

w2

projw1 v3 + projw2 v3 projw2 v3

span{w1 , w2 }

projw1 v3

w1

Figure 7

Confirming Pages

6.3 Orthonormal Bases

EXAMPLE 3

349

Let B be the basis for ⺢3 given by

      −1  −1  1 B = {v1 , v2 , v3 } =  1 ,  1 ,  0    1 0 1

Apply the Gram-Schmidt process to B to find an orthonormal basis for ⺢3 . Solution

In this case the inner product on ⺢3 is the dot product. Notice that v1 · v2 = 0, so that the vectors v1 and v2 are already orthogonal. Applying the Gram-Schmidt process results in w1 = v1 and w2 = v2 . Following the steps outlined above, we have   1 w1 = v1 =  1  1     −1 v2 · w1 w2 = v2 − w1 = v2 − 03 w1 = v2 =  1  w 1 · w1 0 Next note that v1 and v3 are also orthogonal, so that in this case only one projection is required. That is,     v3 · w1 v3 · w2 w1 − w2 w3 = v3 − w ·w w 2 · w2 1 1   1    −2 −1 −1 =  0  − 0w1 − 12  1  =  − 12  1 0 1 Then

   1    −2   −1  1   1       1 ,  −2  1 , B = {w1 , w2 , w3 } =    1  0 1

is an orthogonal basis for ⺢3 . See Fig. 8. An orthonormal basis is then given          1 −1 −1 1 1 w w 1 w 1 2 3 , , = √  1 , √  1 , √  −1 B  =  3 || w1 || || w2 || || w3 || 6 2 1 2 0 w3

v3

z

v3 − projw2 v3

w1

w2 x

y

Figure 8

by    

Confirming Pages

350

Chapter 6 Inner Product Spaces

Example 4 illustrates the use of the Gram-Schmidt process on a space of polynomials.

EXAMPLE 4

Define an inner product on P3 by



p, q =

Solution

1

p(x)q(x) dx −1

Use the standard basis B = {v1 , v2 , v3 , v4 } = {1, x, x 2 , x 3 } to construct an orthogonal basis for P3 . First note that B is not orthogonal, since  1   2 3 x, x = x 4 dx = 5 −1 We can simplify some of the work by noting that since the interval [−1, 1] is a symmetric interval,  1 When p is an odd function, then p(x) dx = 0 −1  1  1 When p is an even function, then p(x) dx = 2 p(x) dx −1

0

Now, since f (x) = x, g(x) = and h(x) = are all odd functions,  1  1 v2 , v3  = v1 , v2  = x dx = 0 x 3 dx = 0 x3,

 v4 , v1  =

x5

−1 1

−1

−1

 x 3 dx = 0

v4 , v3  =

1

−1

x 5 dx = 0

Since v1 and v2 are orthogonal, proceeding with the Gram-Schmidt process, we have w1 = v1 and w2 = v2 Next, to find w3 , the required computation is v3 , w1  v3 , w2  w1 − w2 w3 = v3 − w1 , w1  w2 , w2  v3 , v1  v3 , v2  v1 − v2 = v3 − v1 , v1  v2 , v2  v3 , v1  v2 , v3  v1 − v2 = v3 − v1 , v1  v2 , v2  But we have already noted above that 0 = v2 , v3  and since  1  1  1 2 v1 , v1  = v3 , v1  = and x 2 dx = 2 x 2 dx = dx = 2 3 −1 0 −1 then w3 = x 2 −

1 3

Confirming Pages

351

6.3 Orthonormal Bases

To find w4 , we first note that since v4 , v1  = 0, w1 = v1 , and w2 = v2 , then v4 , w1  v4 , w2  v4 , w3  w1 − w2 − w3 w4 = v4 − w1 , w1  w2 , w2  w3 , w3  v4 , v2  v4 , w3  v2 − w3 = v4 − v2 , v2  w3 , w3  Next, observe that p(x) = x 5 − 13 x 3 is an odd function. Hence,  1 # 5 1 3$ v4 , w3  = x − 3 x dx = 0 −1

Consequently,

v4 , v2  v2 = x 3 − 35 x v2 , v2  An orthogonal basis for P3 is therefore given by   B  = 1, x, x 2 − 13 , x 3 − 35 x w4 = v4 −

By normalizing each of these vectors, we obtain the orthonormal basis (√ √ ) √ √ 2 6 3 10 # 2 1 $ 5 14 # 3 3 $  , x, x −3 , x − 5x B = 2 2 4 4

EXAMPLE 5

Let U be the subspace of ⺢4 with basis      1 −1 −1      0   0 1     B = {u1 , u2 , u3 } =   1 ,  1 ,  0    1 0 0

       

where the inner product is the dot product. Find an orthonormal basis for U . Solution

Following the Gram-Schmidt process, we let w1 = u1 . Next we have  1       −3 −1 −1 1  2   0  2  1   −3  1 u2 · w1 2        − =−  w2 = u2 − = w1 =  1  1  3 1   −1 w 1 · w1 3  3  0 0 0 0 To facilitate the computations, we replace w2 with   1  2   w2 =   −1  0

   

Confirming Pages

352

Chapter 6 Inner Product Spaces

To justify this substitution, note that w1 · w2 = 0; that is, multiplying w2 by a scalar does not change the fact that it is orthogonal to w1 . To find w3 , we use the computation u3 · w1 u3 · w2 w3 = u3 − w1 − w2 w · w w 1 1     2 · w2   1 1 1   −1     0    1    1  1 2  1 0  =  0  − − 3  1  − 6  −1  = 2  1  1 0 0 2 As before we replace w3 with

 1  0   w3 =   1  2 An orthogonal basis for U is then given by      1 −1       1  ,  2 ,  B =       −1 1    0 0 

 1    0   1    2

Normalizing each of the vectors of B  produces the orthonormal basis        −1 1 1      1      1  , √1  2 , √1  0  B  = √   1  6  −1  6  1       3 0 0 2

Fact Summary 1. Every finite dimensional inner product space has an orthonormal basis. 2. The Gram-Schmidt process is an algorithm to construct an orthonormal basis from any basis of the vector space.

Exercise Set 6.3 In Exercises 1–8, use the standard inner product on ⺢n . a. Find projv u. b. Find the vector u − projv u and verify this vector is orthogonal to v.

 1. u =  2. u =

−1 2 3 −2



 v=



 v=

−1 1 1 −2

 

Confirming Pages

6.3 Orthonormal Bases

 3. u =  4. u =  5. u =   6. u =   7. u =   8. u = 

1 −2 1 −1



 v=



 v=

1 2



−2 −2



   1 −1 3  v =  −1  −1 0    3 1 0 v =  2  −1 1    0 1 −1  v =  0  1 −1    1 3 2 v =  0  −1 0

In Exercises 9–12, use the inner product on P2 defined by  1 p, q = p(x)q(x) dx 0

a. Find projq p. b. Find the vector p − projq p and verify that this vector is orthogonal to q. 9. p(x) = x 2 − x + 1, q(x) = 3x − 1 10. p(x) = x 2 − x + 1, q(x) = 2x − 1 11. p(x) = 2x 2 + 1, q(x) = x 2 − 1 12. p(x) = −4x + 1, q(x) = x In Exercises 13–16, use the standard inner product on ⺢n . Use the basis B and the Gram-Schmidt process to find an orthonormal basis for ⺢n .     1 1 , 13. B = −2 −1     3 2 , 14. B = −2 −1       0  0  1 15. B =  0 ,  −1 ,  −1    −1 1 1

  16. B =  

353

     1  0 1 0 ,  1 ,  1   1 1 −1

In Exercises 17 and 18, use the inner product on P2 defined by  1 p, q = p(x)q(x) dx 0

Use the given basis B and the Gram-Schmidt process to find an orthonormal basis for P2 . 17. B = {x − 1, x + 2, x 2 } 18. B = {x 2 − x, x, 2x + 1} In Exercises 19–22, use the standard inner product on ⺢n to find an orthonormal basis for the subspace span(W ).     1   1 19. W =  1 ,  −1    −1 1     −1   0 20. W =  1 ,  −1    1 1       1  −1    −1   −2   3   −2   ,  ,  21. W =   0   −1   0       1 −1 1       0  −1 1         −2  ,  3 ,  −1  22. W =   0   1   0       −1 −1 0 In Exercises 23 and 24, use the inner product on P3 defined by  1 p, q = p(x)q(x) dx 0

to find an orthonormal basis for the subspace span(W ). 23. W = {x, 2x + 1} 24. W = {1, x + 2, x 3 − 1}

Confirming Pages

354

Chapter 6 Inner Product Spaces

25. In ⺢4 with the standard orthonormal basis for    0 1      1 0   span   1 ,  −1    1 1

inner product find an   −1 2   −3   2   ,    5 ,  −3 1 −1  

       

26. In ⺢3 with the standard inner product find an orthonormal basis for         1  3 3  2 span  0 ,  1 ,  −1 ,  1    0 2 1 1 27. Let {u1 , u2 , . . . , un } be an orthonormal basis for ⺢n . Show that || v ||2 = |v · u1 |2 + · · · + |v · un |2 for every vector v in ⺢n . 28. Let A be an n × n matrix. Show that the following conditions are equivalent. a. A−1 = At b. The row vectors of A form an orthonormal basis for ⺢n . c. The column vectors of A form an orthonormal basis for ⺢n . 29. Show that an n × n matrix A has orthonormal column vectors if and only if At A = I . 30. Let A be an m × n matrix, x a vector in ⺢m , and y a vector in ⺢n . Show that x · (Ay) = At x · y. 31. Show that if A is an m × n matrix with orthonormal column vectors, then || Ax || = || x ||. 32. Show that if A is an m × n matrix with orthonormal column vectors and x and y are in ⺢n , then (Ax) · (Ay) = x · y. 33. Show that if A is an m × n matrix with orthonormal column vectors and x and y are in ⺢n , then (Ax) · (Ay) = 0 if and only if x · y = 0. 34. In ⺢4 with the standard inner product show that   1  0   the set of all vectors orthogonal to both   −1  1

 2  3   and   −1  is a subspace. Find a basis for the 2 

subspace. 35. Let S = {u1 , . . . , um } be a set of vectors in ⺢n . Show that the set of all vectors orthogonal to every ui is a subspace of ⺢n . In Exercises 36–41, a (real) n × n matrix A is called positive semidefinite if A is symmetric and ut Au ≥ 0 for every nonzero vector u in ⺢n . If the inequality is strict, then A is positive definite. 36. Let A be a positive definite matrix. Show that the function u, v = ut Av defines an inner product on ⺢n . (Note that when A = I this function corresponds to the dot product.)   3 1 . Show that A is positive 37. Let A = 1 3 definite. 38. Show that if A is positive definite, then the diagonal entries are positive. 39. Let A be an m × n matrix. Show that At A is positive semidefinite. 40. Show that a positive definite matrix is invertible. 41. Show that the eigenvalues of a positive definite matrix are positive. 42. Let     −2 2 v1 = v2 = −1 −4   −2 and a. Are the vectors v1 = −1   2 orthogonal? v2 = −4   −2 −1 . Let A = 2 −4 b. Find det(At A). c. Showthat the  area of the rectangle  spanned by √ −2 2 and v2 = is det(At A). v1 = −1 −4 d. Show that the area of the rectangle is | det(A)|. e. If v1 and v2 are any two orthogonal vectors in ⺢2 , show that the area of the rectangle spanned

Confirming Pages

6.4 Orthogonal Complements

by the vectors is | det(A)|, where A is the matrix with row vectors v1 and v2 . f. Let v1 and v2 be two vectors in ⺢2 that span a parallelogram, as shown in the figure. Show that the area of the parallelogram is | det(A)|, where A is the matrix with row vectors v1 and v2 . g. If v1 , v2 , and v3 are mutually orthogonal vectors in ⺢3 , show that the volume of the box spanned by the three vectors is | det(A)|, where A is the matrix with row vectors v1 , v2 , and v3 . ß

6.4

355

y

v2 p

v1 x

Orthogonal Complements Throughout this chapter we have seen the importance of orthogonal vectors and bases in inner product spaces. Recall that two vectors u and v in an inner product space V are orthogonal if and only if u, v = 0 A collection of vectors forms an orthogonal basis if the vectors are a basis and are pairwise orthogonal. In this section we extend the notion of orthogonality to subspaces of inner product spaces. As a first step, let v be a vector in an inner product space V and W a subspace of V . We say that v is orthogonal to W if and only if v, w = 0

for each vector w ∈ W

As an illustration, let W be the yz plane in the Euclidean space ⺢3 . Observe that W is closed under addition and scalar multiplication, so that by Theorem 3 of Sec. 3.2 it is a subspace. Using the dot product as the inner product on ⺢3 , the coordinate vector   1  e1 = 0  0 is orthogonal to W since

  0 1  0 · y  = 0 z 0 

for every y, z ∈ ⺢. Note that any scalar multiple of e1 is also orthogonal to W . Example 1 gives an illustration of how to find vectors orthogonal to a subspace.

EXAMPLE 1

Let V = ⺢3 , with the dot product as the inner product, and let W be the subspace defined by   1   W = span  −2    3

Confirming Pages

356

Chapter 6 Inner Product Spaces

Describe all vectors in ⺢3 that are orthogonal to W . Solution

Let

 1 w =  −2  3 

Thus, any vector in W has the form cw, for some real number c. Consequently, a vector   x v= y  z in ⺢3 is orthogonal to W if and only if v · w = 0. This last equation is equivalent to the equation x − 2y + 3z = 0 whose solution set is given by S = {(2s − 3t, s, t) | s, t ∈ ⺢} Therefore, the set of vectors orthogonal to W is given by          −3  2     2s − 3t   s, t ∈ ⺢ = s  1  + t  0  s, t ∈ ⺢ s S =         1  0 t Letting s = t = 1 gives the particular vector   −1 v= 1  1

w

which is orthogonal to W since     1 −1 v· w =  1 ·  −2  = (−1)(1) + (1)(−2) + (1)(3) = 0 3 1

S

5

z

0 ⫺5 0x

⫺5

y 0

If the vectors in S  are placed in standard position, then the solution set describes a plane in ⺢3 , as shown in Fig. 1. This is in support of our intuition as the set of vectors orthogonal to a single vector in ⺢3 should all lie in a plane perpendicular to that vector, which is called the normal vector.

5

Figure 1

The set of vectors found in Example 1, orthogonal to the subspace W, is called the orthogonal complement of W . The following definition generalizes this idea to inner product spaces.

Confirming Pages

6.4 Orthogonal Complements

DEFINITION 1

357

Orthogonal Complement Let W be a subspace of an inner product space V . The orthogonal complement of W, denoted by W ⊥ , is the set of all vectors in V that are orthogonal to W. That is, W ⊥ = {v ∈ V | v, w = 0 for all w ∈ W }

EXAMPLE 2

Let V = P3 and define an inner product on V by  1 p, q = p(x)q(x) dx 0

Find W ⊥ if W is the subspace of constant polynomials. Solution

Let f (x) = a + bx + cx 2 + dx 3 be an arbitrary polynomial in P3 and p(x) = k be an arbitrary constant polynomial. Then f is in W ⊥ if and only if    1 d b c f, p = =0 k(a + bx + cx 2 + dx 3 ) dx = k a + + + 2 3 4 0 Since this equation must hold for all k ∈ ⺢,     b c d ⊥ 2 3 W = a + bx + cx + dx  a + + + = 0 2 3 4

Notice in Examples 1 and 2 that the zero vector is an element of the orthogonal complement W ⊥ . It can also be shown for these examples that W ⊥ is closed under vector space addition and scalar multiplication. This leads to Theorem 8.

THEOREM 8

Let W be a subspace of an inner product space V . 1. The orthogonal complement W ⊥ is a subspace of V . 2. The only vector in W and W ⊥ is the zero vector; that is, W ∩ W ⊥ = {0}. Proof (1) Let u and v be vectors in W ⊥ , and w a vector in W, so that u, w = 0

and

v, w = 0

Now for any scalar c, we have u + cv, w = u, w + cv, w = u, w + c v, w =0+0=0 Thus, u + cv is in W ⊥ , and therefore by Theorem 4 of Sec. 3.2, W ⊥ is a subspace of V .

Confirming Pages

358

Chapter 6 Inner Product Spaces

(2) Let w be any vector in W ∩ W ⊥. Then w, w = 0 and hence w = 0 (see Definition 1 of Sec. 6.2). Thus, W ∩ W ⊥ = {0}. To determine whether a vector v is in the orthogonal complement of a subspace, it suffices to show that v is orthogonal to each one of the vectors in a basis for the subspace.

PROPOSITION 5

Let W be a subspace of an inner product space V and B = {w1 , . . . , wm } a basis for W. The vector v is in W ⊥ if and only if v is orthogonal to each vector in B. Proof First suppose that v is orthogonal to each vector in B. Let w be a vector in W . Then there are scalars c1 , . . . , cm such that w = c 1 w1 + · · · + c m wm To show that v is in W ⊥ , take the inner product of both sides of the previous equation with v, so that v, w = c1 v, w1  + c2 v, w2  + · · · + cm v, wm    Since v, wj = 0 for all j = 1, 2, . . . , m, we have v, w = 0 and hence v ∈ W ⊥ . On the other hand, if v ∈ W ⊥ , then v is orthogonal to each vector in W. In particular, v is orthogonal to wj , for all j = 1, 2, . . . , m.

EXAMPLE 3

Let V = ⺢4 with the dot product as the inner product,   0 1     1 0  W = span   −1 ,  −1    1 −1 a. b. c. d.

Find a basis for W . Find a basis for W ⊥ . Find an orthonormal basis for ⺢4 . Let



1  0  v0 =  0 0 Show that v0 can be written as the sum from W ⊥ .

and let        

    of a vector from W and a vector

Confirming Pages

6.4 Orthogonal Complements

Solution

a. Let

 1  0   w1 =   −1  −1

359

 0  1   w2 =   −1  1





and

Notice that w1 and w2 are orthogonal and hence by Theorem 5 of Sec. 6.2 are linearly independent. Thus, {w1 , w2 } is a basis for W . b. Now by Proposition 5, the vector

 x  y   v=  z  w 

is in W ⊥ if and only if v · w1 = 0 and v · w2 = 0. This requirement leads to the linear system ( x −z−w =0 y−z+w =0 The two-parameter solution set for this linear system is    s + t         s − t  s, t ∈ ⺢ S=   s          t The solution to this system, in vector form, provides a description of the orthogonal complement of W and is given by     1  1       1  ,  −1  W ⊥ = span   1   0       1 0 Let

 1  1   v1 =   1  0 

 1  −1   v2 =   0  1 

and

Since v1 and v2 are orthogonal, by Theorem 5 of Sec. 6.2 they are linearly independent and hence a basis for W ⊥ . c. Let B be the set of vectors B = {w1 , w2 , v1 , v2 }. Since B is an orthogonal set of four vectors in ⺢4 , then by Corollary 1 of Sec. 6.2, B is a basis for ⺢4 . Dividing each of these vectors by its length, we obtain the (ordered) orthonormal basis for ⺢4 given by

Confirming Pages

360

Chapter 6 Inner Product Spaces

   

  1 0  0  1  1 1     ,√ B = {b1 , b2 , b3 , b4 } = √3  −1  3  −1    −1 1 

   1 1    1  1  1  −1  , √  , √    3  1  3  0    0 1 



d. By Theorem 6 of Sec. 6.2 the coordinates of v0 relative to B  are given by ci = v0 · bi , for 1 ≤ i ≤ 4. So 1 c1 = √ 3

c2 = 0

1 c3 = √ 3

1 c4 = √ 3

Now, observe that the first two vectors of B  are an orthonormal basis for W while the second two vectors are an orthonormal basis for W ⊥ . Let w be the vector in W given by   1 1 0   w = c1 b1 + c2 b2 =  3  −1  −1 and u be the vector in W ⊥ given by

 2 1 0   u = c3 b3 + c4 b4 =  3 1  1

Then

  1  1 0  + 1  w+u=    −1 3 3 −1 



  1 2  0 0  = 1   0 0 1

   = v0 

The vector w in Example 3 is called the orthogonal projection of v onto the subspace W , and the vector u is called the component of v orthogonal to W . The situation, in general, is the content of Definition 2 and Theorem 9.

DEFINITION 2

Direct Sum Let W1 and W2 be subspaces of a vector space V . If each vector in V can be written uniquely as the sum of a vector from W1 and a vector from W2 , then V is called the direct sum of W1 and W2 . In this case we write, V = W 1 ⊕ W2 .

Confirming Pages

6.4 Orthogonal Complements

PROPOSITION 6

361

Let W1 and W2 be subspaces of a vector space V with V = W1 ⊕ W2 . Then W1 ∩ W2 = {0}. Proof Let v ∈ W1 ∩ W2 . Then v = w1 + 0

and

v = 0 + w2

with w1 ∈ W1 and w2 ∈ W2 . Hence, by the uniqueness of direct sum representations, we have w1 = w2 = 0.

THEOREM 9

Projection Theorem If W is a finite dimensional subspace of an inner product space V , then V = W ⊕ W⊥ Proof The proof of this theorem has two parts. First we must show that for any vector v ∈ V there exist vectors w ∈ W and u ∈ W ⊥ such that w + u = v. Then we must show that this representation is unique. For the first part, let B = {w1 , . . . , wn } be a basis for W. By Theorem 7 of Sec. 6.3, we can take B to be an orthonormal basis for W. Now, let v be a vector in V , and let the vectors w and u be defined by w = v, w1  w1 + v, w2  w2 + · · · + v, wn  wn

and

u=v−w

Since w is a linear combination of the vectors in B, then w ∈ W . To show that u is in W ⊥ , we show that u, wi  = 0 for each i = 1, 2, . . . , n and invoke Proposition 5. To this end, let wi be a vector in B. Then u, wi  = v − w, wi  = v, wi  − w, wi  n
   = v, wi  − v, wj wj , wi j =1

Since B is an orthonormal basis, wi , wi  = 1

and



 wj , wi = 0

for i = j

Hence, u, wi  = v, wi  − v, wi  wi , wi  = 0 Since this holds for each i = 1, . . . , n, then by Proposition 5 the vector u ∈ W ⊥ as claimed. For the second part of the proof, let v=w+u

and

v = w + u

with w and w in W and u and u in W ⊥ . Subtracting the previous equations gives (w − w ) + (u − u ) = 0

Confirming Pages

362

Chapter 6 Inner Product Spaces

or equivalently,

w − w = u − u

Now, from this last equation we know that the vector u − u is in W , as it is a linear combination of the vectors w and w , which are in W . However, u − u is also in W ⊥ since it is the difference of two vectors in W ⊥ . Therefore, by Theorem 8, part 2, u − u = 0 and hence u = u . This being the case, we now have w − w = 0, so that w = w. Thus, we have shown that for every v in V , there are unique vectors w in W and u in W ⊥ such that v = w + u and hence V = W ⊕ W ⊥ .

Motivated by the terminology of Example 3, we call the vector w, of Theorem 9, the orthogonal projection of v onto W , which we denote by projW v, and call u the component of v orthogonal to W .

Matrices In Definition 4 of Sec. 3.2 we defined the null space of an m × n matrix A, denoted by N (A), as the set of all vectors x in ⺢n such that Ax = 0. The column space of A, denoted by col(A), is the subspace of ⺢m spanned by the column vectors of A. In a similar way, the left null space of A, denoted by N (At ), is the set of vectors y in ⺢m such that At y = 0. Finally, the row space of A, which we discussed in Sec. 4.2, denoted by row(A), is the subspace of ⺢n spanned by the row vectors of A. Since the rows of A are the columns of At , then row(A) = col(At ). These four subspaces N (A) N (At ) col(A) and col(At ) are referred to as the four fundamental subspaces associated with the matrix A. Theorem 10 gives relationships among them.

THEOREM 10

Let A be an m × n matrix. 1. N (A) = col(At )⊥ 2. N (At ) = col(A)⊥ Proof (1) Let v1 , . . . , vm denote the row vectors of A. So that   v1 · x  v2 · x   .. Ax =    . vm · x First let x be a vector in N (A) so that Ax = 0. Then vi · x = 0 for i = 1, 2, . . . , m. Thus, x ∈ row(A)⊥ = col(At )⊥ and N (A) ⊆ col(At )⊥ . On the other hand, let x be a vector in col(At )⊥ = row(A)⊥ . Then x · vi = 0, for i = 1, . . . , m, so that Ax = 0. Therefore, col(At )⊥ ⊆ N (A). Hence, N (A) = col(At )⊥ . For part 2, substitute At for A in part 1.

Confirming Pages

6.4 Orthogonal Complements

363

Linear Systems Let A be an m × n matrix. In light of Theorem 10, we are now in a position to provide an analysis of the linear system Ax = b in terms of the geometric structure of Euclidean space and the fundamental subspaces of A. As a first step, we describe the action of A on a vector x in ⺢n . Since row(A) = col(At ), by Theorem 10, N (A) is the orthogonal complement of row(A). Thus, by Theorem 9, a vector x in ⺢n can be uniquely written as x = xrow + xnull where xrow is in the row space of A and xnull is in the null space of A. Now, multiplying x by A, we have Ax = A(xrow + xnull ) = Axrow + Axnull Since Axnull = 0, the mapping T : ⺢n −→ ⺢m defined by T (x) = Ax maps the row space of A to the column space of A. Observe that no vector in ⺢n is mapped to a nonzero vector in N (At ), which by Theorem 10 is the orthogonal complement of the column space of A. See Fig. 2. xrow

A

col(A)

row(A)

x

Ax

dim r

dim n − r

dim r

A xnull

N (At )

A

dim m − r

N (A) ⺢n

⺢m Figure 2

We now consider, again from a geometric point of view, the consistency of the linear system Ax = b for an m × n matrix A and a given vector b in ⺢m . We have already observed in Sec. 3.2 that Ax = b is consistent if and only if b is in the column space of A. By Theorem 10, this system is consistent if and only if b is perpendicular to the left null space of A, or equivalently, if and only if b is orthogonal to every vector in ⺢m , which is orthogonal to the column vectors of A. This sounds a bit awkward. However, in cases where a basis for the null space of At consists of only a few vectors, we can perform an easy check to see if Ax = b is consistent. As an illustration, let     2 1 0 0 1 1  and b= 1  A= 0 −3 −1 −1 −1 Since

 1 0 −1 At =  0 1 −1  0 1 −1 

then

    1  N (At ) = span  1    1

Confirming Pages

364

Chapter 6 Inner Product Spaces

 1 Now as b is orthogonal to  1 , by Proposition 5, b is orthogonal to N (At ) and 1 hence in col(A). Therefore, the linear system Ax = b is consistent. 

Fact Summary Let W be a subspace of an inner product space V . 1. The orthogonal complement of the span of a single nonzero vector v in 3-space is the plane with normal vector v. 2. The orthogonal complement of W is a subspace of V . 3. The only vector common to both W and its orthogonal complement is the zero vector. 4. If W is finite dimensional, then the vector space is the direct sum of W and its orthogonal complement. That is, V = W ⊕ W ⊥ . 5. If B is a basis for W, then v is in W ⊥ if and only if v is orthogonal to each vector in B. 6. If A is an m × n matrix, then N (A) = col(At )⊥ and N (At ) = col(A)⊥ .

Exercise Set 6.4 In Exercises 1–8, find the orthogonal complement of W in ⺢n with the standard inner product.   1 1. W = span −2   1 2. W = span 0   2   3. W = span  1    −1    1  4. W = span  0    2     1  2  5. W = span  1 ,  2    0 −1     0   −3 6. W = span  1 ,  1    1 −1

    7. W = span          8. W = span     

   0  3   2  1  ,   1   1    2 −1     0 1 1  0   1 1  ,  ,  0   1   1 1 1 1

       

In Exercises 9–12, find a basis for the orthogonal complement of W in ⺢n with the standard inner product.     −1   2 9. W = span  1 ,  1    0 1     −2  1  10. W = span  −1 ,  2    −2 1

Confirming Pages

6.4 Orthogonal Complements

    11. W = span          12. W = span     

  1 3  1 1  ,  −1   4 0 2   2 1  0 1  ,  1   −1 1 1

         0     2  ,     3    1  

In Exercises 13 and 14, find a basis for the orthogonal complement of W in P2 with the inner product  1 p, q = p(x)q(x) dx 0

13. W = span{x − 1, x 2 } 14. W = span{1, x 2 } 15. Let W be the subspace of ⺢4 , with the standard inner product, consisting of all vectors w such that w1 + w2 + w3 + w4 = 0. Find a basis for W ⊥ . In Exercises 16–21, W is a subspace of ⺢n with the standard inner product. If v is in ⺢n and {w1 , . . . , wm } is an orthogonal basis for W, then the orthogonal projection of v onto W is given by m
v, wi  projW v = wi wi , wi  i=1

Find the orthogonal projection of v onto W. If necessary, first find an orthogonal basis for W.     2  1  16. W = span  0 ,  1    1 −1   1 v =  −2  2     0   2 17. W = span  0 ,  −1    0 0   1 v= 2  −3

365

    −2  3  18. W = span  −1 ,  2    0 1   5 v =  −3  1     −1   1 19. W = span  2 ,  3    2 1   1 v =  −3  5  1    2 20. W = span   −1    1

  3 1   3   0   ,    −1 ,  1 −1 0  

       

 0  0   v=  1  0 

 3    0 21. W = span   −1    2

 −6     0   ,    2    4  

 1  2   v=  1  −1 

In Exercises 22–25, W is a subspace of ⺢n with the standard inner product. a. Find W ⊥ . b. Find the orthogonal projection of v onto W. (See Exercises 16–21.) c. Compute u = v − projW v. d. Show u is in W ⊥ so v is a sum of a vector in W and one in W ⊥ . e. Make a sketch of W, W ⊥ , v, projW v, and u.

Confirming Pages

366

Chapter 6 Inner Product Spaces



1 2

22. W = span



 v=

 3 v= −1    1  24. W = span  1    0 

23. W = span



1 1 



0 1



29. Let V = M2×2 with the inner product A, B = tr(B t A)



1 v= 1  1   25. W = span    2 v= 1  1 

c. Verify that W ∩ W ⊥ = {0}. d. Let g(x) = 12 [f (x) + f (−x)] and h(x) = 12 [f (x) − f (−x)]. Verify g(−x) = g(x) and h(−x) = −h(x), so every f can be written as the sum of a function in W and a function in W ⊥ .

   −1  1 1 ,  2   4 −1

26. Show that if V is an inner product space, then V ⊥ = {0} and {0}⊥ = V . 27. Show that if W1 and W2 are finite dimensional subspaces of an inner product space and W1 ⊂ W2 , then W2⊥ ⊂ W1⊥ . 28. Let V = C (0) [−1, 1] with the inner product  1 f, g = f (x)g(x) dx −1

and W = {f ∈ V | f (−x) = f (x)}. a. Show that W is a subspace of V . b. Show W ⊥ = {f ∈ V | f (−x) = −f (x)}. ß

6.5

Let W = {A ∈ V | A is symmetric}. a. Show that W ⊥ = {A ∈ V | A is skew symmetric} b. Show that every A in V can be written as the sum of matrices from W and W ⊥ . 30. In ⺢2 with the standard inner product, the transformation that sends a vector to the orthogonal projection onto a subspace W is a   2 . linear transformation. Let W = span 1 a. Find the matrix representation P relative to the standard basis for the orthogonal projection of ⺢2 onto W.  1 . Find projW v and verify the b. Let v = 1 result is the same by applying the matrix P found in part (a). c. Show P 2 = P . 31. If W is a finite dimensional subspace of an inner product space, show that (W ⊥ )⊥ = W.

Application: Least Squares Approximation There are many applications in mathematics and science where an exact solution to a problem cannot be found, but an approximate solution exists that is sufficient to satisfy the demands of the application. Consider the problem of finding the equation of a line going through the points (1, 2), (2, 1), and (3, 3). Observe from Fig. 1 that this problem has no solution as the three points are noncollinear. This leads to the problem of finding the line that is the best fit for these three points based on some criteria for measuring goodness of fit. There are different ways of solving this new problem. One way, which uses calculus, is based on the idea

Confirming Pages

6.5 Application: Least Squares Approximation

367

y 5

⫺5

5

x

⫺5

Figure 1

of finding the line that minimizes the sum of the square distances between itself and each of the points. Another quite elegant method uses the concepts of linear algebra to produce the same result. To illustrate the technique, we consider the original problem of finding an equation of the form y = mx + b that is satisfied by the points (1, 2), (2, 1), and (3, 3). Substitution of these points into the equation y = mx + b yields the linear system    m+b =2 2m + b = 1   3m + b = 3 As noted above, this system is inconsistent. As a first step toward finding an optimal approximate solution, we let       2 1 1 m and b= 1  x= A= 2 1  b 3 3 1 and we write the linear system as Ax = b. From this perspective, we see that the linear system is inconsistent as b is not in col(A). Thus, the best we can do is to look * in col(A) that is as close as possible to b, as shown in Fig. 2. for a vector w

b

* −b w

* w col(A) Figure 2

Confirming Pages

368

Chapter 6 Inner Product Spaces

* be the orthogonal projection of b We will soon see that the optimal choice is to let w * we let onto col(A). In this case, to find w      1   1 W = col(A) = span  2 ,  1    1 3 By Theorem 9 of Sec. 6.4, the vector b can be written uniquely as * +y b=w where y is in W ⊥ . By Theorem 10 of Sec. 6.4, we have W ⊥ = N (At ). By row reducing At , the orthogonal complement of W is   1   W ⊥ = span  −2    1 As this space is one-dimensional, the computations are simplified by finding y first, which in the terminology of Sec. 6.4 is the component of b orthogonal to W . To find y, we use Definition 1 of Sec. 6.3 and compute the orthogonal projection of b onto   1 W ⊥ . Let v =  −2 , so that 1   1 b·v 1 y= v =  −2  v·v 2 1 Hence,

y 5

⫺5

⫺5

     2 1 3 1 1 * = b − y =  1  −  −2  =  4  w 2 2 5 3 1 

* , that is, the Finally, to find values for m and b, we solve the linear system Ax = w system      1 1  3  2 1  m = 1 4  b 2 5 3 1 By Theorem 6 of Sec. 3.2, this last linear system is consistent since the vector on the right-hand side is in col(A). Solving the linear system, we obtain m = 12 and b = 1, 1 y = 2 x + 1 giving us the slope and the y intercept, respectively, for the best-fit line y = 12 x + 1. The vector x 5  + 1 ,  m = 2 b 1

Figure 3

is the least squares solution to the system Ax = b since it produces a line whose total squared distances from the given points are minimal, as shown in Fig. 3. Finding the line that best fits a set of data points is called linear regression.

Confirming Pages

6.5 Application: Least Squares Approximation

369

Least Squares Solutions We now consider the general problem of finding a least squares solution to the m × n linear system Ax = b. An exact solution exists if b is in col(A); moreover, the solution is unique if the columns of A are linearly independent. In the case where b is not in col(A), we look for a vector x in ⺢n that makes the error term || b − Ax || as small as possible. Using the standard inner product on ⺢m to define the length of a vector, we have || b − Ax ||2 = (b − Ax) · (b − Ax) = [b1 − (Ax)1 ]2 + [b2 − (Ax)2 ]2 + · · · + [bm − (Ax)m ]2 This equation gives the rationale for the term least squares solution. To find the least squares solution * x to the linear system Ax = b, we let W = col(A). As W is a finite dimensional subspace of ⺢m , by Theorem 9 of Sec. 6.4, the vector b can be written uniquely as

W⊥ b w2

b = w 1 + w2

w1 where w1 is the orthogonal projection of b onto W and w2 is the component of b

col(A) orthogonal to W , as shown in Fig. 4. Figure 4

We now show that the orthogonal projection minimizes the error term || b − Ax ||, for all x in col(A). First, we have || b − Ax ||2 = || w1 + w2 − Ax ||2 = w2 + (w1 − Ax), w2 + (w1 − Ax) = w2 , w2  + 2 w2 , w1 − Ax + w1 − Ax, w1 − Ax Since w1 and Ax are in W and w2 is in W ⊥ , the middle term vanishes, giving || b − Ax ||2 = w2 , w2  + w1 − Ax, w1 − Ax = || w2 ||2 + || w1 − Ax ||2 The quantity on the right-hand side is minimized if x is any solution to Ax = w1 Since w1 is in col(A), this linear system is consistent. We call any vector * x in ⺢n such that Ax = w1 a least squares solution of Ax = b. Moreover, the solution is unique if the columns of A are linearly independent. Occasionally, as was the case for the example at the beginning of this section, it is possible to find w1 directly. The least squares solution can then be found by solving Ax = w1 . In most cases, however, the vector w1 is hard to obtain. Solving the normal equation At Ax = At b circumvents this difficulty.

Confirming Pages

370

Chapter 6 Inner Product Spaces

THEOREM 11

Let A be an m × n matrix and b a vector in ⺢m . A vector * x in ⺢n is a solution to the normal equation At Ax = At b if and only if it is a least squares solution to Ax = b. Proof From the discussion just before the theorem, we know that a least squares solution * x to Ax = b exists. By Theorem 9 of Sec. 6.4, there are unique vectors w1 in W = col(A) and w2 in W ⊥ such that b = w1 + w2 . First assume that * x is a least squares solution. Since w2 is orthogonal to the columns of A, then At w2 = 0. Moreover, since * x is a least squares solution, A* x = w1 . Therefore, x = At w1 = At (b − w2 ) = At b At A* so that * x is a solution to the normal equation. Conversely, we now show that if * x is a solution to At Ax = At b, then it is also a least squares solution to Ax = b. Suppose that At A* x = At b, or equivalently, x) = 0 At (b − A* Consequently, the vector b − A* x is orthogonal to each row of At and hence to each column of A. Since the columns of A span W, the vector b − A* x is in W ⊥ . Hence, b can be written as b = A* x + (b − A* x) where A* x is in W = col(A) and b − A* x is in W ⊥ . Again, by Theorem 9 of Sec. 6.4, this decomposition of the vector b is unique and hence A* x = w1 . Therefore, * x is a least squares solution.

EXAMPLE 1

Solution

   1 −2 3 and b =  −1  A =  1 −2  2 1 −1 a. Find the least squares solution to Ax = b. b. Find the orthogonal projection of b onto W = col(A) and the decomposition b = w1 + w2 , where w1 is in W and w2 is in W ⊥ .

Let



a. Since the linear system Ax = b is inconsistent, the least squares solution is the best approximation we can find. By Theorem 11, the least squares solution can be found by solving the normal equation At Ax = At b In this case the normal equation becomes      −2  3  −2 1 x −2 1 1   1 −2 = 3 −2 y 3 −2 −1 1 −1

1 −1



 1  −1  2 

Confirming Pages

6.5 Application: Least Squares Approximation

which simplifies to



6 −9

−9 14



x y



 =

−1 3

371



The matrix on the left-hand side is invertible, so that      1 14 9 x −1 = y 3 9 6 3 The least squares solution is then given by  + 13 ,  x 3 * = x= y 3 b. To find the orthogonal projection w1 of b onto col(A), we use the fact that w1 = A* x. So +    , 13 −2 3 1 1 3 =  −5  w1 =  1 −2  3 3 1 −1 4 We now find w2 from the equation w2 = b − w1 , so that       1 1 1 1 2 w2 = b − w1 =  −1  −  −5  =  1  3 3 1 2 4 The decomposition of b is then given by     1 1 2 1 b = w1 + w2 =  −5  +  1  3 3 1 4 Note that w2 is orthogonal to each of the columns of A.

Linear Regression Example 2 illustrates the use of least squares approximation to find trends in data sets.

EXAMPLE 2

The data in Table 1, which are also shown in the scatter plot in Fig. 5, give the average temperature, in degree celsius (o C), of the earth’s surface from 1975 through 2002.∗ Find the equation of the line that best fits these data points.

∗ Worldwatch Institute, Vital Signs 2006–2007. The trends that are shaping our future, W. W. Norton and Company, New York London, 2006.

Confirming Pages

Chapter 6 Inner Product Spaces

Table 1 Average Global Temperatures 1975–2002 1975 13.94 1985 14.03 1994 14.25 1976 13.86 1986 14.12 1995 14.37 1977 14.11 1987 14.27 1996 14.23 1978 14.02 1988 14.29 1997 14.40 1979 14.09 1989 14.19 1998 14.56 1980 14.16 1990 14.37 1999 14.32 1981 14.22 1991 14.32 2000 14.31

Temperature

372

1982 14.04 1992 14.14 2001 14.46 1983 14.25 1993 14.14 2002 14.52 1984 14.07

Year

Figure 5 Solution

Denote the data points by (xi , yi ), for i = 1, 2, . . . , 28, where xi is the year starting with x1 = 1975 and yi is the average global temperature for that year. A line with equation y = mx + b will pass through all the data points if the linear system  m(1975) + b = 13.94     m(1976) + b = 13.86 ..   .    m(2002) + b = 14.52 has a solution. In matrix form, this linear system becomes     13.94 1975 1    13.86   1976 1     m  = .    .. . b     . . 2002

1

14.52

Since the linear system is inconsistent, to obtain the best fit of the data we seek   m is a least squares solution. The normal values for m and b such that x = b equation for this system is given by      13.94  1975 1     1975 · · · 2002  1975 · · · 2002  .   m ..   ..   b = . 1 ··· 1 1 ··· 1 14.52 2002 1

Confirming Pages

6.5 Application: Least Squares Approximation

373

which simplifies to      791,553.23 m 110,717,530 55,678 = 398.05 b 55,678 28 The least squares solution is 

m b



 =

0.0168609742 −19.31197592



Temperature

The line that best fits the data is then given by y = 0.0168609742x − 19.31197592, as shown in Fig. 6.

Year

Figure 6

The procedure used in Example 2 can be extended to fit data with a polynomial of any degree n ≥ 1. For example, if n = 2, to find the best-fit parabola of the form y = ax 2 + bx + c for a set of data points requires finding the least squares solution to an n × 3 linear system. See Exercise 6.

Fourier Polynomials A trigonometric polynomial of degree n is an expression in cosines and sines of the form a0 + a1 cos x + b1 sin x + a2 cos 2x + b2 sin 2x + · · · + an cos nx + bn sin nx where the coefficients a0 , a1 , b1 , a2 , b2 , . . . , an , bn are real numbers. Let PC[−π, π] denote the vector space of piecewise continuous functions on the interval [−π, π]. The vector space PC[−π, π] is an inner product space with inner product defined by  f, g =

π

f (x)g(x) dx −π

Suppose now that given a piecewise continuous function f defined on [−π, π], which may or may not be a trigonometric polynomial, we wish to find the trigonometric polynomial of degree n that best approximates the function.

Confirming Pages

374

Chapter 6 Inner Product Spaces

To solve this problem using linear algebra, √ let W be the subspace of PC[−π, π] of trigonometric polynomials. Let f0 (x) = 1/ 2π, and for k ≥ 1, let 1 fk (x) = √ cos kx π

and

1 gk (x) = √ sin kx π

Define the set B by B = {f0 , f1 , f2 , . . . , fn , g1 , g2 , . . . , gn } . = √12π , √1π cos x, √1π cos 2x, . . . , √1π cos nx, √1π sin x, √1π sin 2x, . . . , √1π sin nx It can be verified that relative to the inner product above, B is an orthonormal basis for W . Now let f be a function in PC[−π, π]. Since W is finite dimensional, f has the unique decomposition f = fW + fW ⊥ W ⊥.

with fW in W and fW ⊥ in Since B is already an orthonormal basis for W , then fW can be found by using the formula for the orthogonal projection given in the proof of Theorem 9 in Sec. 6.4. In this case we have fW = f, f0  f0 + f, f1  f1 + · · · + f, fn  fn + f, g1  g1 + · · · + f, gn  gn We now claim that fW , defined in this way, is the best approximation for f in W . That is, || f − fW || ≤ || f − w || for all w ∈ W To establish the claim, observe that  2 || f − w ||2 =  fW + fW ⊥ − w   2 =  fW ⊥ + (fW − w)    = fW ⊥ + (fW − w), fW ⊥ + (fW − w)     = fW ⊥ , fW ⊥ + 2 fW ⊥ , fW − w + fW − w, fW − w The middle term of the last equation is zero since fW ⊥ and fW − w are orthogonal. So 2  || f − w ||2 =  fW ⊥  + || fW − w ||2 Observe that the right-hand side is minimized if w = fW , that is, if we choose w to be the orthogonal projection of f onto W . The function fW is called the Fourier polynomial of degree n for f.

EXAMPLE 3



Let

−1 −π ≤ x < 0 1 0<x≤π Find the Fourier polynomial for f of degree n = 5. f (x) =

Confirming Pages

6.5 Application: Least Squares Approximation

The graph of y = f (x) is shown in Fig. 7. Since f (x) is an odd function and fk (x) is an even function for k ≥ 0, the product f (x)fk (x) is also an odd function. Hence, the integral on any symmetric interval about the origin is 0, and we have

Solution

f, fk  = 0 Now for k ≥ 1, we have  f, gk  =

π

−π

1

−1

for k ≥ 0

f (x)gk (x) dx  0  π 1 1 = −√ sin kx dx + √ sin kx dx π −π π 0 1 = √ [2 − 2 cos kπ] k π ( 0 if k is even = 4 √ if k is odd k π

y

−π

375

π

x

Therefore, the Fourier polynomial of degree 5 that best approximates the function f on the interval [−π, π] is 4 4 4 p(x) = sin x + sin 3x + sin 5x π 3π 5π In Fig. 8 we see the function and its Fourier approximations for n = 1, 3, and 5.

Figure 7

y

y

y

x

p1 (x) =

4 π

x

p3 (x) =

sin x

4 π

sin x +

4 3π

sin 3x

x

p3 (x) =

4 π

sin x +

4 3π

sin 3x +

4 5π

sin 5x

Figure 8

Exercise Set 6.5 1. Let





1 3 A= 1 3  2 3



and



4 b= 1  5

a. Find the least squares solution to Ax = b. b. Find the orthogonal projection of b onto W = col(A) and the decomposition of the vector b = w1 + w2 , where w1 is in W and w2 is in W ⊥ .

Confirming Pages

376

Chapter 6 Inner Product Spaces

2. Let

 2 2 A= 1 2  1 1

 −2 b= 0  1 



and

5. The table gives world population in billions.

a. Find the least squares solution to Ax = b. b. Find the orthogonal projection of b onto W = col(A) and the decomposition of the vector b = w1 + w2 , where w1 is in W and w2 is in W ⊥ . 3. The table gives world hydroelectricity use in thousands of terawatthours. 1965

927

1990

2185

1970

1187

1995

2513

1975

1449

2000

2713

1980

1710

2004

2803

1985

2004

1955

157

1985

78

1960

141

1990

70

1965

119

1995

66

1970

104

2000

62

1975

93

2005

57

1980

87

a. Sketch a scatter plot of the data. b. Find the linear function that is the best fit to the data.

2.56

1960

3.04

1970

3.71

1980

4.46

1990

5.28

2000

6.08

a. Sketch a scatter plot of the data. b. Find the linear function that is the best fit to the data. 6. The table gives the worldwide cumulative HIV infections in millions.

a. Sketch a scatter plot of the data. b. Find the linear function that is the best fit to the data. 4. The table gives world infant mortality rates in deaths per 1000 live births.

1950

1980

0.1

1995

29.8

1982

0.7

1997

40.9

1985

2.4

2000

57.9

1987

4.5

2002

67.9

1990

10

2005

82.7

1992

16.1

a. Sketch a scatter plot of the data. b. Find a curve of the form y = ax 2 + bx + c that best fits the data. 7. Let f (x) = x on the interval −π ≤ x ≤ π. a. Find the Fourier polynomials for f of degrees n = 2, 3, 4, and 5. b. Sketch y = f (x) along with the polynomials found in part (a). 8. Let

 f (x) =

x x+π

if 0 ≤ x ≤ π if −π ≤ x < 0

Confirming Pages

6.6 Diagonalization of Symmetric Matrices

a. Find the Fourier polynomials for f of degrees n = 2, 3, 4, and 5. b. Sketch y = f (x) along with the polynomials found in part (a). 9. Let f (x) = x 2 on the interval −π ≤ x ≤ π. a. Find the Fourier polynomials for f of degrees n = 2, 3, 4, and 5. b. Sketch y = f (x) along with the polynomials found in part (a). ß

6.6

377

10. Let A be an m × n matrix with rank(A) = n, and suppose A = QR is a QR factorization of A. (See Exercise 9, Review Exercises for Chapter 6.) Show that the best least squares solution to the linear system Ax = b can be found by back substitution on the upper triangular system Rx = Qt b.

Diagonalization of Symmetric Matrices In Sec. 5.2 a methodology was given for diagonalizing a square matrix. A characterization was also provided to determine which n × n matrices were diagonalizable. Recall, specifically, from Theorem 2 of Sec. 5.2 that an n × n matrix is diagonalizable if and only if it has n linearly independent eigenvectors. As we have seen, the application of this theorem requires finding all eigenvectors of a matrix. In certain cases, however, we can tell by inspection if a matrix is diagonalizable. An example of such a case was given in Example 4 of Sec. 5.2, where it was shown that any 2 × 2 real symmetric matrix is diagonalizable with real eigenvalues. That this is the case in general is the subject of this section. In the remarks preceding Example 8 of Sec. 3.1, we defined the set of complex numbers ⺓. The proof of our main result requires that the reader be familiar with some of the terminology and notation from complex variables. In particular, if z = a + bi is a complex number, then the conjugate of z, denoted by z, is given by z = a − bi. Two complex numbers are equal if and only if their real and imaginary parts are equal. From this we know that a complex number z = z if and only if z is a real number. To see this, first suppose that z = z. Then bi = −bi or 2bi = 0 and hence b = 0. We therefore have z = a + 0i and z is a real number. Conversely, if z is a real number, then z = a + 0i = a and z = a − 0i = a so that z = z. We can also define this bar notation for vectors and matrices. So if v is a vector with complex components and M is a matrix with complex entries, then     a11 a12 . . . a1n v1  a21 a22 . . . a2n   v2     . v= M = and  .. .. ..  ..  ..   . . . .  vn am1 a12 . . . amn We are now ready to state our main result.

Confirming Pages

378

Chapter 6 Inner Product Spaces

THEOREM 12

The eigenvalues of an n × n real symmetric matrix A are all real numbers. Proof Let v be an eigenvector of A corresponding to the eigenvalue λ. To show that λ is a real number, we will show that λ = λ. We first consider the matrix product (vt Av)t , which by Theorem 6 of Sec. 1.3 can be written as (vt Av)t = vt At v Since A is symmetric, At = A. Also since A has real entries, then A = A. Therefore, vt At v = vt Av = vt Av = vt Av Now as v is an eigenvector of A corresponding to the eigenvalue λ, then Av = λv, so that vt Av = vt λv = vt λv = λvt v Alternatively, the original expression can be evaluated by (vt Av)t = (vt λv)t = vt λv = λvt v Equating these results gives λvt v = λvt v

that is

(λ − λ)vt v = 0

Since v is an eigenvector of A, and therefore nonzero, so is vt v. By an extension to the complex numbers of Theorem 2 (part 4) of Sec. 3.1, we have λ − λ = 0; hence λ = λ, establishing that λ is a real number.

One consequence of Theorem 12 is that the eigenvectors of a real symmetric matrix have real components. To see this, let A be a symmetric matrix with real entries and v an eigenvector corresponding to the real eigenvalue λ = a. Observe that v is a vector in the null space of the real n × n matrix A − aI . By Theorem 7 of Sec. 3.2, N (A − aI ) is a subspace of ⺢n . Thus, v being a vector in ⺢n , has real components as claimed.

EXAMPLE 1

Let A be the symmetric matrix defined by   2 0 2 0 −2  A= 0 2 −2 1 Verify that the eigenvalues and corresponding eigenvectors of A are real.

Confirming Pages

6.6 Diagonalization of Symmetric Matrices

Solution

379

The characteristic equation of A is det(A − λI ) = −λ3 + 3λ2 + 6λ − 8 = 0 After factoring the characteristic polynomial we obtain (λ − 1)(λ + 2)(λ − 4) = 0 Thus, the eigenvalues of A are λ1 = 1, λ2 = −2, and λ3 = 4. To find eigenvectors corresponding to λ = 1, we find the null space of A − I . To do this, we see that     1 0 2 1 0 2  0 1 2  reduces to A − I =  0 −1 −2  0 0 0 2 −2 0   −2 Thus, an eigenvector corresponding to λ1 = 1 is v1 =  −2 . In a similar way 1 we have that eigenvectors corresponding to λ2 = −2 and λ3 = 4 are, respectively,     1 −2 v2 =  −2  and v3 =  1 . −2 −2

Orthogonal Diagonalization In Sec. 6.1 we showed that two vectors u and v in ⺢n are orthogonal if and only if their dot product u · v = 0. An equivalent formulation of this condition can be developed by using matrix multiplication. To do this, observe that if u and v are vectors in ⺢n , then vt u is a matrix with a single entry equal to u · v. Hence, we know that u and v are orthogonal if and only if vt u = 0. Theorem 13 shows that eigenvectors which correspond to distinct eigenvalues of a real symmetric matrix are orthogonal.

THEOREM 13

Let A be a real symmetric matrix and v1 and v2 be eigenvectors corresponding, respectively, to the distinct eigenvalues λ1 and λ2 . Then v1 and v2 are orthogonal. Proof We have already shown that v1 and v2 are vectors in ⺢n . To show that they are orthogonal, we show that vt1 v2 = 0. Now, since λ2 is an eigenvalue of A, then Av2 = λ2 v2 , so that vt1 Av2 = vt1 λ2 v2 = λ2 vt1 v2

Confirming Pages

380

Chapter 6 Inner Product Spaces

Also since At = A, vt1 Av2 = vt1 At v2 = (Av1 )t v2 = λ1 vt1 v2 Equating the two expressions for vt1 Av2 , we obtain (λ1 − λ2 )vt1 v2 = 0 Since λ1 = λ2 then λ1 − λ2 = 0. Hence, by Theorem 2, part 4, of Sec. 3.1, we have vt1 v2 = 0, which, by the remarks preceding this theorem, gives that v1 is orthogonal to v2 .

EXAMPLE 2

Let A be the real symmetric matrix given by   1 0 0 A= 0 0 1  0 1 0 Show that the eigenvectors corresponding to distinct eigenvalues of A are orthogonal.

Solution

The characteristic equation of A is det(A − λI ) = −(λ − 1)2 (λ + 1) = 0 so the eigenvalues are λ1 = 1 and λ2 = −1. Then the eigenspaces (see Sec. 5.1) are given by        0  0   1  and Vλ2 = span  −1  Vλ1 = span  0 ,  1      1 0 1     0 1 Since every vector in Vλ1 is a linear combination of u =  0  and v =  1 , 1 0   0 and w =  −1  is orthogonal to both u and v, then by Proposition 5 of Sec. 6.4, 1 w is orthogonal to every eigenvector in Vλ1 . Hence, every eigenvector in Vλ2 is orthogonal to every eigenvector in Vλ1 . In Example 2, we showed that every vector in the eigenspace Vλ1 is orthogonal to every vector in the eigenspace Vλ2 . Notice, moreover, that the vectors within Vλ1 are orthogonal to one another. In this case, the matrix has a special factorization. We normalize the spanning vectors of the eigenspaces to obtain     0   0 1 1 √    − √1   2   0  and 2    1  1 √ 0 √ 2 2

Confirming Pages

6.6 Diagonalization of Symmetric Matrices

Using these vectors, we construct the matrix  1 0  0 √1 2 P =  1 √ 0 2 which is then used to diagonalize A.    1 0 0 1 1 1  √ √  0 2 2  0 P −1 AP =    0 0 − √12 √12

381

 0 − √12    1 √

2

That is,  1 0 0  0 1  0 1 0 0

0 √1 2 1 √ 2

  0 1 0 − √12  =  0 1  0 0 √1 2

 0 0  −1

Observe in this case that the diagonalizing matrix P has the special property that P P t = I, so that P −1 = P t . This leads to Definition 1.

DEFINITION 1

Orthogonal Matrix A square matrix P is called an orthogonal matrix if it is invertible and P −1 = P t .

One important property of orthogonal matrices is that the column (and row) vectors of an n × n orthogonal matrix are an orthonormal basis for ⺢n . That is, the vectors of this basis are all mutually orthogonal and have unit length. As we mentioned at the beginning of this section, one particularly nice fact about symmetric matrices is that they are diagonalizable. So by Theorem 2 of Sec. 5.2 a real symmetric matrix has n linearly independent eigenvectors. For the matrix A of Example 2, the eigenvectors are all mutually orthogonal. Producing an orthogonal matrix P to diagonalize A required only that we normalize the eigenvectors. In many cases there is more to do. Specifically, by Theorem 2, eigenvectors corresponding to distinct eigenvalues are orthogonal. However, if the geometric multiplicity of an eigenvalue λ is greater than 1, then the vectors within Vλ (while linearly independent), might not be mutually orthogonal. In this case we can use the Gram-Schmidt process, given in Sec. 6.3, to find an orthonormal basis from the linearly independent eigenvectors. The previous discussion is summarized in Theorem 14.

THEOREM 14

Let A be an n × n real symmetric matrix. Then there is an orthogonal matrix P and a diagonal matrix D such that P −1AP = P tAP = D. The eigenvalues are the diagonal entries of D.

The following steps can be used to diagonalize an n × n real symmetric matrix A.

Confirming Pages

382

Chapter 6 Inner Product Spaces

1. Find the eigenvalues and corresponding eigenvectors of A. 2. Since A is diagonalizable, there are n linearly independent eigenvectors. If necessary, use the Gram-Schmidt process to find an orthonormal set of eigenvectors. 3. Form the orthogonal matrix P with column vectors determined in Step 2. 4. The matrix P −1 AP = P t AP = D is a diagonal matrix.

EXAMPLE 3

Let

 0 1 1 A= 1 0 1  1 1 0 

Find an orthogonal matrix P such that P −1 AP is a diagonal matrix. Solution

The characteristic equation for A is given by det(A − λI ) = −λ3 + 3λ + 2 = −(λ − 2)(λ + 1)2 = 0 Thus, the eigenvalues are λ1 = −1 and λ2 = 2. The corresponding eigenspaces are       −1   −1  1  and Vλ2 = span  1  Vλ1 = span  1 ,  0      1 0 1 Let B be the set of vectors       −1  −1  1 B = {v1 , v2 , v3 } =  1 ,  1 ,  0    1 0 1 Since B is a linearly independent set of three vectors, by Theorem 2 of Sec. 5.2, A is diagonalizable. To find an orthogonal matrix P which diagonalizes A, we use the Gram-Schmidt process on B. This was done in Example 3 of Sec. 6.3, yielding the orthonormal basis        1 −1  −1  1 1 1 B  = √  1 , √  1 , √  −1    3 6 2 1 2 0 Now let P be the matrix given by    P = 

√

3 3 √ 3 3 √ 3 3

−

√ 2 2 √ 2 2

0

−

√

6 6 √ − 66 √ 6 3

    

Observe that P is an orthogonal matrix with P −1 = P t . Morevover,   2 0 0 0  P t AP =  0 −1 0 0 −1

Confirming Pages

6.6 Diagonalization of Symmetric Matrices

383

Fact Summary Let A be an n × n real symmetric matrix. 1. 2. 3. 4.

The eigenvalues of A are all real numbers. The eigenvectors corresponding to distinct eigenvalues of A are orthogonal. The matrix A is diagonalizable. There is an orthogonal matrix P such that D = P −1 AP = P t AP , where D is a diagonal matrix with diagonal entries the eigenvalues of A.

Exercise Set 6.6 In Exercises 1–4, verify that the eigenvalues of the symmetric matrix are all real numbers.   1 2 1. A = 2 1   −1 3 2. A = 3 −1   1 2 0 3. A =  2 −1 2  0 2 1   1 1 −2 2  4. A =  1 −1 −2 2 1 In Exercises 5–8, verify that the eigenvectors of the symmetric matrix corresponding to distinct eigenvalues are orthogonal.   1 2 5. A = 2 −2   −3 2 6. A = 2 −3   1 2 0 7. A =  2 −1 −2  0 −2 1   1 0 −2 0  8. A =  0 −1 −2 0 1

In Exercises 9–12, find the eigenspaces of the n × n symmetric matrix, and verify that the sum of the dimensions of the eigenspaces is n.   1 0 2 9. A =  0 −1 0  2 0 1  1 0 1 10. A =  0 −1 0  1 0 1  2 1 1  1 −2 1 11. A =   1 1 −1 1 1 0  1 0 0  0 −2 0 12. A =   0 0 −1 0 0 0 

 1 1   0  −1  0 0   0  1

In Exercises 13–16, determine whether the matrix is orthogonal. + √ , 1 3 13. A =

2

− 12 +

14. A =

−1 0

2 3 2

√

−

√

5 5 √ −255

,

Confirming Pages

384

Chapter 6 Inner Product Spaces

  15. A = 

√

√

2 2 √ − 22

2 2 √ 2 2

0

  16. A = 

2 3 − 23 1 3

0



 0  1

0

2 3 2 3

1 3 1 3

0

1

  

In Exercises 17–22, for the given matrix A find an orthogonal matrix P and a diagonal matrix D such that D = P −1 AP .  17. A =  18. A =  19. A =  20. A =

3 4 4 3 5 2 2 5

 

−1 3 3 −1 1 2

2 −2





 1 −1 1 21. A =  −1 −1 1  1 1 1 

 1 0 −1 0  22. A =  0 −1 −1 0 1 

23. Show that if A and B are orthogonal matrices, then AB and BA are orthogonal matrices. 24. Show that if A is an orthogonal matrix, then det(A) = ±1. 25. Show that if A is an orthogonal matrix, then At is an orthogonal matrix. 26. Show that if A is an orthogonal matrix, then A−1 is an orthogonal matrix.

27. a. Show that the matrix   cos θ − sin θ A= sin θ cos θ is orthogonal. b. Suppose that A is a 2 × 2 orthogonal matrix. Show that there is a real number θ such that   cos θ − sin θ A= sin θ cos θ  cos θ sin θ sin θ − cos θ (Hint: Consider the equation At A = I.) 

or

A=

c. Suppose that A is an orthogonal 2 × 2 matrix and T : ⺢2 → ⺢2 is a linear operator defined by T (v) = Av. Show that if det(A) = 1, then T is a rotation and if det(A) = −1, then T is a reflection about the x axis followed by a rotation. 28. Matrices A are B are orthogonally similar if there is an orthogonal matrix P such that B = P t AP . Suppose that A and B are orthogonally similar. a. Show that A is symmetric if and only if B is symmetric. b. Show that A is orthogonal if and only if B is orthogonal. 29. Suppose that A is an n × n matrix such that there exists a diagonal matrix D and an orthogonal matrix P such that D = P t AP . (Matrix A is called orthogonally diagonalizable.) Show that A is symmetric. 30. Suppose A is invertible and orthogonally diagonalizable. Show that A−1 is orthogonally diagonalizable. (See Exercise 29.) 31. Let A be an n × n skew-symmetric matrix. a. If v is in ⺢n , expand vt v in terms of the components of the vector. b. Show that the only possible real eigenvalue of A is λ = 0. [Hint: Consider the quantity vt (λv).]

Confirming Pages

6.7 Application: Quadratic Forms

ß

6.7

385

Application: Quadratic Forms When working with complicated algebraic expressions, mathematicians will often attempt to simplify problems by applying transformations designed to make these expressions easier to interpret, or at least better suited to the task at hand. In this section we show how certain transformations of the coordinate axes in ⺢2 can be used to simplify equations that describe conic sections, that is, equations in x and y whose graphs are parabolas, hyperbolas, circles, and ellipses. As an illustration, consider the equation x 2 − 4x + y 2 − 6y − 3 = 0 To simplify this equation, we complete the square on x 2 − 4x and y 2 − 6y to obtain (x 2 − 4x + 4) + (y 2 − 6y + 9) = 3 + 4 + 9 that is, (x − 2)2 + (y − 3)2 = 16 y' y

10

10

⫺10

x'

10

⫺10

10

x

⫺10 ⫺10

Figure 1

This last equation describes a circle of radius 4 centered at the point (2, 3). The graph is shown in Fig. 1. To further simplify this equation, we can translate the coordinate axes by means of the equations x = x − 2

and

y = y − 3

The equation of the circle then becomes #  $2 #  $2 x + y = 16 This is the equation of the circle in standard position in the x  y  plane with center at the origin.

Rotation of Axes The most general quadratic equation in two variables has the form ax 2 + bxy + cy 2 + dx + ey + f = 0

Confirming Pages

386

Chapter 6 Inner Product Spaces

where a, b, c, d, e, and f are real numbers such that at least one of a, b, or c is not zero. The graph of a quadratic equation in x and y is a conic section (including possible degenerate cases), the particular one being dependent on the values of the coefficients. When b = 0, the conic section is rotated from standard position. The expression ax 2 + bxy + cy 2 is called the associated quadratic form. For example, the quadratic equation 2x 2 + 5xy − 7y 2 + 2x − 4y + 1 = 0 has an associated quadratic form given by 2x 2 + 5xy − 7y 2 The quadratic equation ax 2 + bxy + cy 2 + dx + ey + f = 0 is also given in matrix form by setting +   a x A= b x= y

,

b 2

 and

c

2

b=

d e



Then the quadratic equation above is equivalent to xt Ax + bt x + f = 0 The quadratic form (in matrix form) is then given by xt Ax As an illustration, the quadratic equation 2x 2 + 5xy + y 2 + 3x − y + 1 = 0 in matrix form is given by + ,    2 52 x x +1=0 + [3 − 1] [x y] 5 y y 1 2

The associated quadratic form is [x y]

+

2

5 2

5 2

1

,

x y



Observe that the matrix A, for any quadratic equation in two variables, is symmetric; that is, At = A. This fact enables us to develop a transformation that we can use to simplify the equation. Specifically, the map we desire will rotate the coordinate axes by the precise angle needed to situate the conic section in standard position with respect to a new coordinate system. To produce such a mapping, first recall from Theorem 14 of Sec. 6.6 that if A is a real symmetric matrix, then there exists an orthogonal matrix P and a diagonal matrix D such that A = P DP −1 = P DP t . Next, we need to examine which orthogonal 2 × 2 matrices are rotations.

Confirming Pages

6.7 Application: Quadratic Forms

387

Now by Exercise 27(b) of Sec. 6.6, a real orthogonal 2 × 2 matrix has the form     cos θ sin θ cos θ − sin θ or B = B= sin θ − cos θ sin θ cos θ Next, recall from Sec. 4.6 that B is the matrix representation, relative to the standard basis for ⺢2 , of a linear operator which rotates a vector in the plane by θ rad. The matrix B  is not a rotation (relative to any basis). To see this, let Q = {v1 , v2 } be a basis for ⺢2 and θ = 0. Then   1 0  B = 0 −1 Relative to the basis Q, this matrix produces a reflection through the line spanned by v1 . For example, if Q is the standard basis for ⺢2 , then B  is a reflection through the x axis. These results are summarized in Theorem 15. THEOREM 15

Let B be a real orthogonal 2 × 2 matrix. The change of coordinates given by      x x = B y y is a rotation if and only if det(B) = 1. We are now in a position to analyze quadratic equations in two variables. Start with C a conic section with equation xt Ax + bt x + f = 0 Let P be the orthogonal matrix that diagonalizes A, so that   0 λ1 where D = A = P DP t 0 λ2 with λ1 and λ2 being the eigenvalues of A. As P is orthogonal, by the above remarks on the form of P , its determinant is either +1 or −1. If det(P ) = −1, then interchange the column vectors of P , along with the diagonal entries of D. Since  + cos # π − θ$ sin # π − θ$ ,  2 2 sin θ cos θ = #π $ #π $ − cos θ sin θ − sin 2 − θ cos 2 − θ a rearrangement of the column vectors of P is a rotation. To obtain the equation for C in the x  y  coordinate system, substitute x = P x into xt Ax + bt x + f = 0 to obtain #  $t #  $ P x A P x + bt P x + f = 0 By Theorem 6 of Sec. 1.3, if the product of A and B is defined, then (AB)t = B t At , and since matrix multiplication is associative, we have (x )t P t AP x + bt P x + f = 0 that is, (x )t Dx + bt P x + f = 0    d Let bt P = . The last equation can now be written as e λ1 (x  )2 + λ2 (y  )2 + d  x  + e y  + f = 0

Confirming Pages

388

Chapter 6 Inner Product Spaces

This equation gives the conic section C in standard position in the x  y  coordinate system. The type of conic section depends on the eigenvalues. Specifically, C is 1. An ellipse if λ1 and λ2 have the same sign 2. A hyperbola if λ1 and λ2 have opposite signs 3. A parabola if either λ1 or λ2 is zero

EXAMPLE 1

Let C be the conic section whose equation is x 2 − xy + y 2 − 8 = 0. a. Transform the equation to x  y  coordinates so that C is in standard position with no x  y  term. b. Find the angle of rotation between the standard coordinate axes and the x  y  coordinate system.

Solution

a. The matrix form of this equation is given by x Ax − 8 = 0 t

with

A=



1 − 12

− 12 1



The eigenvalues of A are λ1 = 12 and λ2 = 32 , with corresponding (unit) eigenvectors     1 1 1 −1 v1 = √ and v2 = √ 1 2 1 2 Then the orthogonal matrix   1 1 −1 P =√ 1 2 1 diagonalizes A. Moreover, since det(P ) = 1, then by Theorem 15, the coordinate transformation is a rotation. Making the substitution x = P x in the matrix equation above gives #  $t t x P AP x − 8 = 0 that is,   1 0 where D= 2 3 (x )t Dx − 8 = 0 0 2 This last equation can now be written as     1 0 x −8=0 [x  y  ] 2 3 y 0 2 so that the standard form for the equation of the ellipse in the x  y  coordinate system is 3(y  )2 (x  )2 + =1 16 16

Confirming Pages

6.7 Application: Quadratic Forms

389

This is the equation of an ellipse with x  as the major axis and y  as the minor axis, as shown in Fig. 2.

y'

y

5

5

x'

5

45◦ ⫺5

⫺5

⫺

5 ⫺

x

5

5

(x  )2 + 3(y  )2 = 16

Figure 2

b. To find the angle between the original axes and the x  y  coordinate system, observe that the eigenvector v1 points in the direction of the x  axis. Now using Definition 3 of Sec. 6.1, the cosine of the angle between e1 and v1 is given by e1 · v1 1 π cos θ = = √ so that θ= || e1 || || v1 || 4 2 An alternative way to find the angle between the axes is to note that the matrix P , which is the transition matrix from x  y  coordinates to xy coordinates, can be written as + 1 ,   √ √1 − π cos θ − sin θ 2 2 with θ = P = = sin θ cos θ √1 √1 4 2

2

Example 2 involves a rotation and a translation.

EXAMPLE 2

Describe the conic section C whose equation is 2x 2 − 4xy − y 2 − 4x − 8y + 14 = 0

Solution

The equation for C has the form xt Ax + bt x + f = 0 given by      x x 2 −2 + 14 = 0 + [−4 − 8] [x y] y y −2 −1   2 −2 are λ1 = −2 and λ2 = 3, with corresponding The eigenvalues of A = −2 −1 (unit) eigenvectors     1 1 1 −2 √ √ and v2 = v1 = 1 5 2 5

Confirming Pages

Chapter 6 Inner Product Spaces

Since the eigenvalues have opposite sign, the conic section C is a hyperbola. To describe the hyperbola, we first diagonalize A. Using the unit eigenvectors, the orthogonal matrix that diagonalizes A is     1 −2 0 1 −2 = P t AP with P =√ 0 3 1 5 2 Making the substitution x = P x in the equation xt Ax + bt x + f = 0 gives , + 1      −2 √ √ x x −2 0 5 5   + [−4 − 8] + 14 = 0 [x y ] 0 3 y y √2 √1 5

5

After simplification of this equation we obtain √ −2(x  )2 − 4 5x  + 3(y  )2 + 14 = 0 that is,

√ −2[(x  )2 + 2 5(x  )] + 3(y  )2 + 14 = 0 After completing the square on x  , we obtain √ −2[(x  )2 + 2 5(x  ) + 5] + 3(y  )2 = −14 − 10 √ (y  )2 (x  + 5)2 − =1 12 8 This last equation describes a hyperbola with x  as the major axis. An additional transformation translating the x  axis allows us to simplify the result even further. If we let √ x  = x  + 5 and y  = y  then the equation now becomes that is,

(y  )2 (x  )2 − =1 12 8 The graph is shown in Fig. 3. y

10

x'

10

10

y'

⫺10

0

⫺10

Figure 3

⫺1

0

10

⫺1

390

x

Confirming Pages

6.7 Application: Quadratic Forms

391

Quadric Surfaces The graph of a quadratic equation in three variables of the form ax 2 + bxy + cxz + dy 2 + eyz + f z2 + gx + hy + iz + j = 0 is an ellipsoid, a hyperboloid, a paraboloid, or a cone. As in the two-dimensional case, the terms gx, hy, and iz produce translations from standard form, while the mixed terms xy, xz, and yz produce rotations. The quadratic form ax 2 + bxy + cxz + dy 2 + eyz + f z2 can be written in matrix form as



 x Ax = [x y z]  t

a

b 2

b 2 c 2

d e 2



 x   y  z f c 2 e 2

As before, a rotation developed from the eigenvectors of A can be used to transform the quadric surface to one in standard form λ1 (x  )2 + λ2 (y  )2 + λ3 (z )2 + j = 0 where λ1 , λ2 , and λ3 are the eigenvalues of A. We omit the details.

EXAMPLE 3

Write the quadratic equation 5x 2 + 4y 2 − 5z2 + 8xz = 36 in standard form by eliminating the xz term.

Solution

 5 0 4 0  A= 0 4 4 0 −5 Then the quadratic equation can be written as    x 5 0 4 0   y  = 36 [x y z]  0 4 z 4 0 −5 Let



The eigenvalues of the matrix A are √ √ λ1 = 41 λ2 = − 41

λ3 = 4

Hence, the quadric surface, in standard position, has the equation √ √ 41(x  )2 − 41(y  )2 + 4(z )2 = 36 Figure 4

The graph of the surface, which is a hyperboloid of one sheet, is shown in Fig. 4.

Confirming Pages

392

Chapter 6 Inner Product Spaces

Exercise Set 6.7 In Exercises 1–6, let C denote the conic section given by the equation. Transform the equation to x  y  coordinates so that C is in standard position with no x  y  term. 1. 27x 2 − 18xy + 3y 2 + x + 3y = 0 2.

2x 2

− 8xy +

8y 2

+ 2x + y = 0

3. 12x 2 + 8xy + 12y 2 − 8 = 0 4. 11x 2 − 6xy + 19y 2 + 2x + 4y − 12 = 0 5. −x 2 − 6xy − y 2 + 8 = 0 6. xy = 1 7. Let C denote the conic section in standard position given by the equation 4x 2 + 16y 2 = 16. a. Write the quadratic equation in matrix form. b. Find the quadratic equation that describes the conic C rotated by 45◦ . 8. Let C denote the conic section in standard position given by the equation x 2 − y 2 = 1. ß

6.8

a. Write the quadratic equation in matrix form. b. Find the quadratic equation that describes the conic C rotated by −30◦ . 9. Let C denote the conic section in standard position given by the equation 16x 2 + 4y 2 = 16. a. Find the quadratic equation for the conic section obtained by rotating C by 60◦ . b. Find the quadratic equation that describes the conic found in part (a) after a translation 3 units to the right and 2 units upward. 10. Let C denote the conic section in standard position given by the equation x 2 − y = 0. a. Find the quadratic equation for the conic section obtained by rotating C by 30◦ . b. Find the quadratic equation that describes the conic found in part (a) after a translation 2 units to the right and 1 unit downward.

Application: Singular Value Decomposition In earlier sections we have examined various ways to write a given matrix as a product of other matrices with special properties. For example, with the LU factorization of Sec. 1.7, we saw that an m × n matrix A could be written as A = LU with L being an invertible lower triangular matrix and U an upper triangular matrix. Also in Sec. 1.7, we showed that if A is invertible, then it could be written as the product of elementary matrices. In Sec. 5.2 it was shown that an n × n matrix A with n linearly independent eigenvectors can be written as A = P DP −1 where D is a diagonal matrix of eigenvalues of A. As a special case, if A is symmetric, then A has the factorization A = QDQt where Q is an orthogonal matrix. In this section we consider a generalization of this last result for m × n matrices. Specifically, we introduce the singular value decomposition, abbreviated as SVD, which enables us to write any m × n matrix as A = U
V t where U is an m × m orthogonal matrix, V is an n × n orthogonal matrix, and
is an m × n matrix with numbers, called singular values, on its diagonal.

Confirming Pages

6.8 Application: Singular Value Decomposition

393

Singular Values of an m × n Matrix To define the singular values of an m × n matrix A, we consider the matrix At A. Observe that since A is an m × n matrix, At is an n × m matrix, so the product At A is a square n × n matrix. This new matrix is symmetric since (At A)t = At Att = At A. Hence, by Theorem 14 of Sec. 6.6, there is an orthogonal matrix P such that P t (At A)P = D where D is a diagonal matrix of the eigenvalues of  λ1 0 ···  0 λ2 · · ·  D= . .. ..  .. . . 0

···

···

At A given by  0 0   ..  .  λn

Since by Exercise 39 of Sec. 6.3 the matrix At A is positive semidefinite, we also have, by Exercise 41 of Sec. 6.3, that λi ≥ 0 for 1 ≤ i ≤ n. This permits us to make the following definition.

DEFINITION 1

Singular Values Let A be an m × n matrix. The singular values of A, denoted by σi for 1 ≤ i ≤ n, are the positive square roots of the eigenvalues λ1 , . . . , λn of At A. That is,  σi = λi for 1 ≤ i ≤ n

It is customary to write the singular values of A in decreasing order σ1 ≥ σ2 ≥ · · · ≥ σn As mentioned in Sec. 5.2, this can be accomplished by permuting the columns of the diagonalizing matrix P .

EXAMPLE 1

Let A be the matrix given by

 1 1 A= 0 1  1 0 

Find the singular values of A. Solution

The singular values of A are found by first computing the eigenvalues of the square matrix     1 1   2 1 1 0 1  0 1 = At A = 1 2 1 1 0 1 0

Confirming Pages

394

Chapter 6 Inner Product Spaces

The characteristic equation, in this case, is given by det(At A − λI ) = (λ − 3)(λ − 1) = 0 The eigenvalues of At A are then λ1 = 3 and λ2 = 1, so that the singular values √ are σ1 = 3 and σ2 = 1.

We have already seen that orthogonal bases are desirable and the Gram-Schmidt process can be used to construct an orthogonal basis from any basis. If A is an m × n matrix and v1 , . . . , vr are the eigenvectors of At A, then we will see that {Av1 , . . . , Avr } is an orthogonal basis for col(A). We begin with the connection between the singular values of A and the vectors Av1 , . . . , Avr .

THEOREM 16

Let A be an m × n matrix and let B = {v1 , v2 , . . . , vn } be an orthonormal basis of ⺢n consisting of eigenvectors of At A, with corresponding eigenvectors λ1 , λ2 , . . . , λn . Then 1. || Avi || = σi for each i = 1, 2, . . . , n. 2. Avi is orthogonal to Avj for i = j. Proof For the first statement recall from Sec. 6.6 that the length of a vector v in √ Euclidean space can be given by the matrix product || v || = vt v. Therefore, || Avi ||2 = (Avi )t (Avi ) = v ti (At A)vi = v ti λi vi = λi || vi || = λi The last equality is √ due to the fact that vi is a unit vector. Part 1 is established by noting that σi = λi = || Avi ||. For part 2 of the theorem, we know that (as in Sec. 6.6) the dot product of two vectors u and v in Euclidean space can be given by the matrix product u · v = ut v. Thus, since B is an orthonormal basis of ⺢n , if i = j , then (Avi ) · (Avj ) = (Avi )t (Avj ) = vti (At A)vj = vti λj vi = λj vti vj = 0

In Theorem 16, the set of vectors {Av1 , Av2 , . . . , Avn } is shown to be orthogonal. In Theorem 17 we establish that the eigenvectors of At A, after multiplication by A, are an orthogonal basis for col(A).

THEOREM 17

Let A be an m × n matrix and B = {v1 , v2 , . . . , vn } an orthonormal basis of ⺢n consisting of eigenvectors of At A. Suppose that the corresponding eigenvalues satisfy λ1 ≥ λ2 ≥ · · · ≥ λr > λr+1 = · · · λn = 0, that is, At A has r nonzero eigenvalues. Then B  = {Av1 , Av2 , . . . , Avr } is an orthogonal basis for the column space of A and rank(A) = r.

Revised Confirming Pages

6.8 Application: Singular Value Decomposition

395

√ Proof First observe that since σi = λi are all nonzero for 1 ≤ i ≤ r; then by Theorem 16, part 1, we have Av1 , Av2 , . . . , Avr are all nonzero vectors in col(A). By part 2 of Theorem 16, we have B  = {Av1 , Av2 , . . . , Avr } is an orthogonal set of vectors in ⺢m . Hence, by Theorem 5 of Sec. 6.2, B  is linearly independent. Now to show that these vectors span the column space of A, let w be a vector in col(A). Thus, there exists a vector v in ⺢n such that Av = w. Since B = {v1 , v2 , . . . , vn } is a basis for ⺢n , there are scalars c1 , c2 , . . . , cn such that v = c1 v1 + c2 v2 + · · · + cn vn Multiplying both sides of the last equation by A, we obtain Av = c1 Av1 + c2 Av2 + · · · + cn Avn Now, using the fact that Avr+1 = Avr+2 = · · · = Avn = 0, then Av = c1 Av1 + c2 Av2 + · · · + cr Avr so that w = Av is in span{Av1 , Av2 , . . . , Avr }. Consequently, B  = {Av1 , Av2 , . . . , Avr } is an orthogonal basis for the column space of A, and the rank of A is equal to the number of its nonzero singular values.

EXAMPLE 2

Solution

Let A be the matrix given by

 1 1 A= 0 1  1 0 

Find the image of the unit circle under the linear transformation T : ⺢2 → ⺢3 defined by T (v) = Av. From Example 1, the eigenvalues of At A are λ1 = 3 and λ2 = 1, with eigenvectors √   √   1/√2 −1/√ 2 and v2 = v1 = 1/ 2 1/ 2 √ respectively. The singular values of A are then σ1 = 3 and σ2 = 1. Let C(t) be the unit circle given by cos(t)v1 + sin(t)v2 for 0 ≤ t ≤ 2π. The image of C(t) under T is given by T (C(t)) = cos (t)Av1 + sin (t)Av2  By Theorem 17, B  = σ11 Av1 , σ12 Av2 is a basis for the range of T . Hence, the √ coordinates of T (C(t)) relative to B  are x  = σ1 cos t = 3 cos t and y  = σ2 sin t = sin t. Observe that  2 (x  )2 x √ + (y  )2 = cos2 t + sin2 t = 1 + (y  )2 = 3 3 which is an ellipse with the length of the semimajor axis equal to σ1 and length of the semiminor axis equal to σ2 , as shown in Fig. 1. 

Confirming Pages

396

Chapter 6 Inner Product Spaces y

z

2

σ2 ⫺2

y

σ1

Multiplication by

A

2

x x

⫺2

x y

Figure 1

For certain matrices, some of the singular values may be zero. As an illustration,     1 1 2 . . For this matrix, we have col(A) = span consider the matrix A = 3 3 6   1 2 , which has only one The reduced row echelon form for A is the matrix 0 0 pivot column. Hence, the rank of A is equal to 1. The eigenvalues of At A are λ1 = 50 and λ2 = 0 with corresponding unit eigenvectors √   √   1/√5 −2/√ 5 and v2 = v1 = 2/ 5 1/ 5 √ The singular values of A are given by σ1 = 5 2 and σ2 = 0. Now, multiplying v1 and v2 by A gives  √    0 √5 and Av2 = Av1 = 0 3 5 Observe that Av1 spans the one dimensional column space of A. In this case, the linear transformation T : ⺢2 −→ ⺢2 defined by T (x) = Ax maps the unit circle to the line segment   √    √5  − 1 ≤ t ≤ 1 t 3 5  as shown in Fig. 2.

Singular Value Decomposition (SVD) We now turn our attention to the problem of finding a singular value decomposition of an m × n matrix A. THEOREM 18

SVD Let A be an m × n matrix of rank r, with r nonzero singular values σ1 , σ2 , . . . , σr . Then there exists an m × n matrix
, an m × m orthogonal matrix

Confirming Pages

6.8 Application: Singular Value Decomposition

y

397

y

T

2

x

x

Figure 2

U, and an n × n orthogonal matrix V such that A = U
V t Proof Since At A is an n × n symmetric matrix, by Theorem 14 of Sec. 6.6 there is an orthonormal basis {v1 , . . . , vn } of ⺢n , consisting of eigenvectors of At A. Now by Theorem 17, {Av1 , . . . , Avr } is an orthogonal basis for col(A). Let {u1 , . . . , ur } be the orthonormal basis for col(A), given by 1 1 ui = Avi = Avi for i = 1, . . . , r || Avi || σi Next, extend {u1 , . . . , ur } to the orthonormal basis {u1 , . . . , um } of ⺢m . We can now define the orthogonal matrices V and U , using the vectors {v1 , . . . , vn } and {u1 , . . . , um }, respectively, as column vectors, so that V = [ v1

v2

· · · vn ]

and

U = [ u1

u2

· · · um ]

Moreover, since Avi = σi ui , for i = 1, . . . , r, then    AV =  Av1

· · · Avr

0



· · · 0  =  σ1 u1

Now let
be the m × n matrix given by  σ1 0 ... 0  0 σ2 . . . 0   .. .. . ..  . . .. . 
=  0 · · · · · · σr  0 ... ... 0   . ..  .. . 0 ... ... 0

0 ... 0 ... .. .. . . 0 ... 0 ... .. . 0

· · · σr ur

0 0 .. . 0 0 .. .

... 0

           

0

··· 0 

Confirming Pages

398

Chapter 6 Inner Product Spaces

Then

 U
=  u1

 u2

· · · um 


 =  σ1 u1

· · · σr ur

0

··· 0 

= AV Since V is orthogonal, then V t = V −1 , and hence, A = U
V t .

EXAMPLE 3

Solution

Find a singular value decomposition of the matrix   −1 1 1  A =  −1 2 −2 A procedure for finding an SVD of A is included in the proof of Theorem 18. We present the solution as a sequence of steps. Step 1. Find the eigenvalues and corresponding orthonormal eigenvectors of At A and define the matrix V . The eigenvalues of the matrix   6 −6 At A = −6 6 in decreasing order are given by λ1 = 12 and λ2 = 0. The corresponding orthonormal eigenvectors are √    √  −1/√2 1/√2 and v2 = v1 = 1/ 2 1/ 2 Since the column vectors of V are given by the orthonormal eigenvectors of At A, the matrix V is given by √ √   −1/√2 1/√2 V = 1/ 2 1/ 2 Step 2. Find the singular values of A and define the matrix
. The singular values of A are the square roots of the eigenvalues of At A, so that √ √ √ and σ2 = λ2 = 0 σ1 = λ1 = 2 3 Since
has the same dimensions as A, then
is 3 × 2. In this case,   √ 2 3 0
= 0 0  0 0

Confirming Pages

6.8 Application: Singular Value Decomposition

399

Step 3. Define the matrix U. The matrix A has one nonzero singular value, so by Theorem 17 the rank of A is 1. Therefore, the first column of U is √   1/√6 1 u1 = Av1 =  1/√6  σ1 −2/ 6 Next we extend the set {u1 } to an orthonormal basis for ⺢3 by adding to it the vectors √   √   2/ 5 −1/√2 u2 =  √0  and u3 =  1/ 2  1/ 5 0 so that √  √ √  1/√6 2/ 5 −1/√2 U =  1/√6 0√ 1/ 2  0 −2/ 6 1/ 5 The singular value decomposition of A is then given by √  √ √ √   √ √  1/√6 2/ 5 −1/√ 2 2 3 0  −1/√2 1/√2 A = U
V t =  1/ √6 0  0√ 1/ 2   0 1/ 2 1/ 2 0 0 0 −2/ 6 1/ 5   −1 1 1  =  −1 2 −2

In Example 3, the process of finding a singular value decomposition of A was complicated by the task of extending the set {u1 , . . . , ur } to an orthogonal basis for ⺢m . Alternatively, we can use At A to find V and AAt to find U. To see this, note that if A = U
V t is an SVD of A, then At = V
t U t . After multiplying A on the left by its transpose, we obtain At A = V
t U t U
V t = V D1 V t where D1 is an n × n diagonal matrix with diagonal entries the eigenvalues of At A. Hence, V is an orthogonal matrix that diagonalizes At A. On the other hand, AAt = U
V t V
t U t = U D2 U t where D2 is an m × m diagonal matrix with diagonal entries the eigenvalues of AAt and U is an orthogonal matrix that diagonalizes AAt . Note that the matrices At A and AAt have the same eigenvalues. (See Exercise 22 of Sec. 5.1.) Therefore, the nonzero diagonal entries of D1 and D2 are the same. The matrices U and V found using this

Confirming Pages

400

Chapter 6 Inner Product Spaces

procedure are not unique. We also note that changing the signs of the column vectors in U and V also produces orthogonal matrices that diagonalize AAt and At A. As a result, finding an SVD of A may require changing the signs of certain columns of U or V . In Example 4 we use this idea to find an SVD for a matrix.

EXAMPLE 4

Solution

Find a singular value decomposition of the matrix   1 1 A= 3 −3 First observe that

   10 −8 1 1 = AA= −8 10 3 −3   1 By inspection we see that v1 = √12 is a unit eigenvector of At A with −1   1 is a unit eigenvector of corresponding eigenvalue λ1 = 18, and v2 = √12 1 At A with corresponding eigenvalue λ2 = 2. Hence,   1 1 1 V =√ 2 −1 1 

t

1 1

3 −3



√ √ The singular values of A are σ1 = 3 2 and σ2 = 2 so that   √ 3 2 √0
= 2 0 To find U , we compute AAt =



1 3

1 −3



1 1

3 −3



 =

2 0 0 18

 

Observe that a unit eigenvector corresponding to λ1 = 18 is u1 =   1 . Thus, eigenvector corresponding to λ2 = 2 is u2 = 0   0 1 U= 1 0 A singular value decomposition of A is then given by   √  √1  3 2 0 0 1  2 √ A = U
V t = 1 0 √1 2 0 2

−1 √ 2 1 √ 2

 =



0 1

1 3

 and a unit

1 −3



Confirming Pages

6.8 Application: Singular Value Decomposition

401

The Four Fundamental Subspaces In this subsection we show how the matrices U and V , which give the singular value decomposition of A, provide orthonormal bases for the four fundamental subspaces of A, introduced in Sec. 6.4. To develop this idea, let A be an m × n matrix of rank r ≤ n and B = {v1 , . . . , vn } be an orthonormal basis of eigenvectors of At A with corresponding eigenvalues λ1 ≥ λ2 ≥ · · · ≥ λr > λr+1 = · · · = λn = 0. First, from the proof of Theorem 17 if σ1 , · · · , σr are the nonzero singular values of A, then   1 1 C = Av1 , . . . , Avr = {u1 , . . . , ur } σ1 σr is a basis for col(A). Next, the remaining columns of U are defined by extending C  to an orthonormal basis C = {u1 , . . . , ur , ur+1 , . . . , um } for ⺢m . We claim that C  = {ur+1 , . . . , um } is an orthonormal basis for N (At ). To see this, observe that each vector of C  is orthogonal to each vector of C  . Hence, by Proposition 5 of Sec. 6.4 and the fact that dim(⺢m ) = m, we have span{ur+1 , . . . , um } = col(A)⊥ . By Theorem 10, part 2, of Sec. 6.4, span{ur+1 , . . . , um } = N (At ), so that C  = {ur+1 , . . . , um } is a basis for N (At ) as claimed. We now turn our attention to the matrix V . From the proof of Theorem 16, we have Avr+1 = · · · = Avn = 0. Consequently, span{vr+1 , . . . , vn } is contained in N (A). Now by Theorem 5 of Sec. 4.2, dim(N (A)) + dim(col(A)) = n

N(A)

v1

row(A) A

col(A) u2

N(At ) u1 Av1

u3

so that dim(N (A)) = n − r. Since B  = {vr+1 , . . . , vn } is an orthogonal, and hence linearly independent, set of n − r vectors in N (A), by Theorem 12, part (1), of Sec. 3.3, B  is a basis for N (A). Finally, since B = {v1 , . . . , vn } is an orthonormal basis for ⺢n , each vector of B  is orthogonal to every vector in B  = {v1 , . . . , vr }. Hence, span{v1 , . . . , vr } = N (A)⊥ = col(At ) = row(A) so that B  is a basis for row(A). To illustrate the ideas of this discussion, consider the matrix A of Example 3 and its SVD. By the above discussion, we have     1   −1 col(A) = span  1  row(A) = span 1   −2       −1   2 1 N (At ) = span  0 ,  1  N (A) = span 1   0 1 The four fundamental subspaces are shown in Fig. 3.

Figure 3

Data Compression An important application that involves the singular value decomposition is data compression. As a preliminary step, suppose that a matrix A of rank r (with r nonzero

Confirming Pages

402

Chapter 6 Inner Product Spaces

singular values) has the SVD A = U
V t . That is,  A = U
V t =  σ1 u1

· · · σr ur

0



··· 0 Vt

= σ1 u1 vt1 + σ2 u2 vt2 + · · · + σr ur vtr       1 1 1 = σ1 Av1 vt1 + σ2 Av2 vt2 + · · · + σr Avr vtr σ1 σ2 σr = (Av1 )vt1 + (Av2 )vt2 + · · · + (Avr )vtr Observe that each of the terms Avi vti is a matrix of rank 1. Consequently, the sum of the first k terms of the last equation is a matrix of rank k ≤ r, which gives an approximation to the matrix A. This factorization of a matrix has application in many areas. As an illustration of the utility of such an approximation, suppose that A is the 356 × 500 matrix, where each entry is a numeric value for a pixel, of the gray scale image of the surface of Mars shown in Fig. 4. A simple algorithm using the method above for approximating the image stored in the matrix A is given by the following: Figure 4

1. Find the eigenvectors of the n × n symmetric matrix At A. 2. Compute Avi , for i = 1, . . . , k, with k ≤ r = rank(A). 3. The matrix (Av1 )vt1 + (Av2 )vt2 + · · · + (Avk )vtk is an approximation of the original image. To transmit the kth approximation of the image and reproduce it back on earth requires the eigenvectors v1 , . . . , vk of At A and the vectors Av1 , . . . , Avk . The images in Fig. 5 are produced using matrices of ranks 1, 4, 10, 40, 80, and 100, respectively.

Figure 5

The storage requirements for each of the images are given in Table 1.

Confirming Pages

6.8 Application: Singular Value Decomposition

403

Table 1 Image

Storage Requirement

Percent of Original

Rank

Original

356 × 500 = 178, 000

100%

Approximation 1

2 × 500 = 1, 000

0.6

1

Approximation 2

8 × 500 = 4, 000

1

4

Approximation 3

20 × 500 = 10, 000

1

10

Approximation 4

80 × 500 = 40, 000

22

40

Approximation 5

160 × 500 = 80, 000

45

80

Approximation 6

200 × 500 = 100, 000

56

100

Exercise Set 6.8 In Exercises 1–4, find the singular values for the matrix.   −2 −2 1. A = 1 1  2. A =

−1 −2 1 −2



 1 0 2 3. A =  2 −1 −1  −2 1 1 

 1 −1 0 0 1  4. A =  0 −1 1 0 

In Exercises 5–8, find a singular value decomposition for the matrix.   5 3 5. A = 3 5  6. A =  7. A =

2 2 4 −1



1 0 0 0 1 1



 8. A =

−2 1 −1 0 1 1



In Exercises 9 and 10, the condition number of a matrix A is the ratio σ1 /σr , of the largest to the smallest singular value. The condition number provides a measure of the sensitivity of the linear system Ax = b to perturbations to A or b. A linear system is ill-conditioned when the condition number is too large and called singular when the condition number is infinite (the matrix is not invertible).   1 1 . 9. Let A = 1 1.000000001 a. Solve the linear system Ax =



2 2



b. Solve the linear system   2 Ax = 2.000000001 c. Find the condition number for A.   1 10. Let b =  3 . −4

Confirming Pages

404

Chapter 6 Inner Product Spaces

a. Let

b. Let 



−2 −1 0 A =  −2 −1 −2  0 −2 1 Solve the linear system Ax = b.

 −2.00001 −1.001 0 −0.87 −2  B =  −2.01 0 −2 1 

Solve the linear system Bx = b. c. Find the condition number for A.

Review Exercises for Chapter 6 1. Let V be the inner product space ⺢3 with the standard inner product and let       2  1  1 B =  0 ,  0 ,  1    0 0 1 a. Verify that B is a basis for ⺢3 . 3 b. Use B to find an orthonormal  for ⺢ .   basis 1   1 c. Let W = span  0 ,  0  and   0 1   −2 v =  1 . Find projW v. (Hint: First use −1 the Gram-Schmidt process to find an orthogonal basis for W ; then refer to Exercise 16 of Sec. 6.4.) 2. Let

    −3 −1      2  ,  0 , W = span     0  2    0 −2     0  3   −2   0   ,    −1   −1    1 1

be a subspace of ⺢4 with the standard inner product. a. Find a basis for W. b. Find W ⊥ . c. Find an orthonormal basis for W.

d. Find an orthonormal basis for W ⊥ . ⊥ e. Verify that dim(⺢4 ) = dim(W ) + dim(W  ). −2  0 f. Find the orthogonal projection of v =   3 −1 onto W.

   

3. Let a, b, and c be real numbers and        x  W =  y  ∈ ⺢3  ax + by + cz = 0    z where ⺢3 is given  standard inner product.  the a a. Show that  b  is in W ⊥ . c b. DescribeW ⊥ .  x1 c. Let v =  x2 . Find projW ⊥ v. x3   d. Find  projW ⊥ v  . 4. Define on P2 an inner product by  1 p, q = p(x)q(x) dx −1

Let p(x) = x and q(x) = x 2 − x + 1. a. Find p, q . b. Find the distance between p and q. c. Are p and q orthogonal? Explain. d. Find the cosine of the angle between p and q. e. Find projq p. f. Let W = span{p}. Find W ⊥ .

Confirming Pages

6.8 Application: Singular Value Decomposition

5. Let V be the inner product space C (0) [−π, π] with inner product defined by  π f, g = f (x)g(x) dx −π

Let W = span{1, cos x, sin x}. a. Verify that the set {1, cos x, sin x} is orthogonal. b. Find an orthonormal basis for W. c. Find projW x 2 .   d. Find  projW x 2 . 6. Let B = {v1 , . . . , vn } be an orthonormal basis for an inner product space V , and let v be a vector in V .   c1  c2   a. Find the coordinate   ...  of v relative to B. cn b. Show that ci vi = projvi v for each i = 1, 2, . . . , n. c. Let        1 1 1   B = √12  1 , √12  −1 , √16  1    0 0 −2 be an orthonormal basis for ⺢3 , with the standard inner product, and let  1  √ + √1 2 3   v =  √1 − √13 . Find the coordinate 2    



√1 3

c1 c2  ..  of v relative to B. . 

cn 7. Show that if B is an orthonormal basis for ⺢n , with thestandard  inner product, and c1  c2   [v]B =   ...  , then cn || v || =



c12 + c22 + · · · + cn2

Give a similar formula for || v || if B is an orthogonal basis, not necessarily orthonormal.

405

8. Let {v1 , . . . , vm } be an orthonormal subset of ⺢n , with the standard inner product, and let v be any vector in ⺢n . Show that || v ||2 ≥

m


(v · vi )2

i=1

2  /  .) (Hint: Expand  v − m i=1 v, vi  vi 9. (QR factorization) Let A be an m × n matrix with linearly independent column vectors. In this exercise we will describe a process to write A = QR, where Q is an m × n matrix whose column vectors form an orthonormal basis for col(A) and R is an n × n upper triangular matrix that is invertible. Let   1 0 −1  1 −1 2   A=  1 0 1  1 −1 2 a. Let B = {v1 , v2 , v3 } be the set of column vectors of the matrix A. Verify that B is linearly independent and hence forms a basis for col(A). b. Use the Gram-Schmidt process on B to find an orthogonal basis B1 = {w1 , w2 , w3 }. c. Use B1 to find an orthonormal basis B2 = {q1 , q2 , q3 }. d. Define the matrix Q = [q1 q2 q3 ]. Define the upper triangle matrix R for i = 1, 2, 3 by ( 0 if i > j rij = if i ≤ j, j = i, . . . , 3 vj · qi e. Verify that A = QR. 10. Let B = {v1 , v2 , . . . , vn } be an orthogonal basis for an inner product space V and c1 , c2 , . . . , cn arbitrary nonzero scalars. Show that B1 = {c1 v1 , c2 v2 , . . . , cn vn } is an orthogonal basis for V . How can the scalars be chosen so that B1 is an orthonormal basis?

Confirming Pages

406

Chapter 6 Inner Product Spaces

Chapter 6: Chapter Test In Exercises 1–40, determine whether the statement is true or false. 1. If u is orthogonal to both v1 and v2 , then u is orthogonal span{v1 , v2 }. 2. If W is a subspace of an inner product space V and v ∈ V , then v − projW v ∈ W ⊥ . 3. If W is a subspace of an inner product space V , then W ∩ W ⊥ contains a nonzero vector. 4. Not every orthogonal set in an inner product space is linearly independent. In Exercises 5–10, let 

 2  1   v1 =   −4  3

 −2  1   v2 =   2  1 

be vectors in ⺢4 with inner product the standard dot product.

be vectors in ⺢3 with inner product the standard dot product. 11. The set {v1 , v2 , v3 } is orthogonal. 12. The set {v1 , v2 , v3 } is a basis for ⺢3 .   1 13. If W = span{v1 , v2 } and u =  1 , then 1 projW u = v1 + 13 v2 . 14. If W = span{v1 , v2 , v3 }, then W ⊥ = {0}. 15. If W = span{v1 , v2 }, then W ⊥ = {0}. 16. W = span{v1 , v2 , v3 }, then projW v = v for any vector v ∈ ⺢3 . In Exercises 17–23, use the inner product defined on P2 defined by  1 p, q = p(x)q(x) dx −1

5. || v1 || = 30

  17.  x 2 − x  =

6. The √ distance between the vectors v1 and v2 is 2 14.

18. The polynomials p(x) = x and q(x) = x 2 − 1 are orthogonal.

7. The vector u =

19. The polynomials p(x) = 1 and q(x) = x 2 − 1 are orthogonal.

√1 30

v1 is a unit vector.

8. The vectors v1 and v2 are orthogonal. 9. The cosine of the√angle between the vectors 4 v1 and v2 is − 15 10.   −8/15  −4/15   10. projv1 v2 =   −16/15  −12/15 In Exercises 11–16, let   1 v1 =  0  1

 v2 = 

 2 v3 =  4  −2 



−1 1  1

√4 15

20. The set {1, x, x 2 − 13 } is orthogonal. 21. The vector p(x) =

1 2

is a unit vector.

22. If W = span{1, x}, then dim(W ⊥ ) = 1. 23. If W = span{1, x 2 }, then a basis for W ⊥ is {1, x}. 24. An n × n symmetric matrix has n distinct real eigenvalues. 25. If u and v are vectors in ⺢2 , then u, v = 3u1 v2 − u2 v1 defines an inner product. 26. For any inner product 2u, 2v + 2w = 2 u, v + 2 u, w

Confirming Pages

6.8 Application: Singular Value Decomposition

27. If W = span{1, x 2 } is a subspace of P2 with inner product  1 p, q = p(x)q(x) dx 0

then a basis for

W⊥

is {x}.

28. If {u1 , . . . , uk } is a basis for a subspace W of an inner product space V and {v1 , . . . , vm } is a basis for W ⊥ , then {u1 , . . . , uk , v1 , . . . , vm } is a basis for V . 29. If A is an n × n matrix whose column vectors form an orthogonal set in ⺢n with the standard inner product, then col(A) = ⺢n . 30. In ⺢2 with the standard inner product, the orthogonal complement of y = 2x is y = 12 x. 31. In ⺢3 with the standard inner product, the orthogonal of −3x + 3z = 0 is  complement   −3  span  0  .   3 32. Every finite dimensional inner product space has an orthonormal basis.     0   1 W = span  2 ,  1    −1 1 then a basis for W ⊥ is also a basis for the null space of   1 0 1  A= 2 1 −1

33. If

34. If

407

    −1   1 W = span  0 ,  1    0 1

then dim(W ⊥ ) = 2. 35. If

    1   0 W = span  1 ,  0    1 1

then W⊥

   −1  = span  −1    1

36. In ⺢5 with the standard inner product there exists a subspace W such that dim(W ) = dim(W ⊥ ). 37. If A is an n × n matrix whose column vectors are orthonormal, then AAt v is the orthogonal projection of v onto col(A). 38. If u and v are orthogonal, w1 is a unit vector in the direction of u, and w2 is a unit vector in the opposite direction of v, then w1 and w2 are orthogonal. 39. If u and v are vectors in ⺢n and the vector projection of u onto v is equal to the vector projection of v onto u, then u and v are linearly independent. 40. If A is an m × n matrix, then AAt and At A have the same rank.

Confirming Pages

Confirming Pages

APPENDIX

Preliminaries

ß

A.1

Algebra of Sets The notion of a set is a fundamental concept in mathematics allowing for the grouping and analysis of objects with common attributes. For example, we can consider the collection of all even numbers, or the collection of all polynomials of degree 3. A set is any well-defined collection of objects. By this we mean that a clear process exists for deciding whether an object is contained in the set. The colors of the rainbow—red, yellow, green, blue, and purple—can be grouped in the set C = {red, yellow, green, blue, purple} The objects contained in a set are called members, or elements, of the set. To indicate that x is an element of a set S, we write x ∈ S. Since green is one of the colors of the rainbow, we have that green ∈ C. The color orange, however, is not one of the colors of the rainbow and therefore is not an element of C. In this case we write orange ∈ / C. There are several ways to write a set. If the number of elements is finite and small, then all the elements can be listed, as we did with the set C, separated by commas and enclosed in braces. Another example is S = {−3, −2, 0, 1, 4, 7} If a pattern exists among its elements, a set can be described by specifying only a few of them. For example, S = {2, 4, 6, . . . , 36} is the set of all even numbers between 2 and 36, inclusive. The set of all even whole numbers can be written T = {2, 4, 6, . . .} Special sets of numbers are often given special symbols. Several common ones are described here. The set of natural numbers, denoted by ⺞, is the set ⺞ = {1, 2, 3, . . .} The set of integers, denoted by ⺪, is given by ⺪ = {. . . , −3, −2, −1, 0, 1, 2, 3, . . .}

409

Confirming Pages

410

Appendix A

Preliminaries

We use the symbol ⺡ to denote the set of rational numbers, which can be described as 
 p  p, q ∈ ⺪, q =  0 ⺡= q Finally, the set of real numbers, denoted by ⺢, √ consists of all rational and irrational numbers. Examples of irrational numbers are 2 and π. In many cases, the set we wish to consider is taken from a larger one. For example, S = {x ∈ ⺢ | −1 ≤ x < 4} is the set of all real numbers greater than or equal to −1 and less than 4. In general, the notation {x ∈ L | restriction on x} translates to “the set of all x in L such that x satisfies the restriction.” In some cases L is omitted if a universal set is implied or understood. Sets can be compared using the notion of containment. Denote two sets by A and B. The set A is contained in B if each element of A is also in B. When this happens, we say that A is a subset of B and write A ⊆ B. For example, let A = {1, 2}

B = {1, 2, 3}

and

C = {2, 3, 4}

Since every element of A is also in B, we have A ⊆ B. However, A is not a subset of C since 1 ∈ A but 1 ∈ / C. In this case we write A
C. For the sets of natural numbers, integers, rational numbers, and real numbers, we have ⺞⊆⺪⊆⺡⊆⺢ The set with no elements is called the empty set, or null set, and is denoted by φ. One special property of the empty set φ is that it is a subset of every set. Two sets A and B are equal if they have the same elements. Alternatively, A and B are equal if A ⊆ B and B ⊆ A. In this case we write A = B.

Operations on Sets Elements can be extracted from several sets and placed in one set by using the operations of intersection and union. The intersection of two sets A and B, denoted by A ∩ B, is the set of all elements that are in both A and B, that is, A ∩ B = {x | x ∈ A and x ∈ B} The union of two sets A and B, denoted by A ∪ B, is the set of all elements that are in A or B, that is, A ∪ B = {x | x ∈ A or x ∈ B} As an illustration, let A = {1, 3, 5} and B = {1, 2, 4}. Then A ∩ B = {1}

and A ∪ B = {1, 2, 3, 4, 5}

A graphical device, called a Venn diagram, is helpful for visualizing set operations. The Venn diagrams for the intersection and union of two sets are shown in Fig. 1.

Confirming Pages

A.1 Algebra of Sets

A

B

A∩B

A

411

B

A∪B Figure 1

EXAMPLE 1 Solution

Define two intervals of real numbers by A = [−3, 2) and B = [−7, 1). Find A ∩ B and A ∪ B. Since the intervals overlap with −7 < −3 < 1 < 2, the intersection is the interval A ∩ B = [−3, 1) and the union is the interval A ∪ B = [−7, 2)

Notice that x ∈ / A ∩ B if and only if x ∈ / A or x ∈ / B, and x ∈ / A ∪ B if and only if x ∈ / A and x ∈ / B. The complement of the set A relative to the set B, denoted by B\A, consists of all elements of B that are not elements of A. In set notation this complement is given by B\A = {x ∈ B | x ∈ / A} For example, let A and B be the intervals given by A = [1, 2] and B = [0, 5]. Then B\A = [0, 1) ∪ (2, 5] If A is taken from a known universal set, then the complement of A is denoted by Ac . To illustrate, let A = [1, 2] as before. Then the complement of A relative to the set of real numbers is ⺢\A = Ac = (−∞, 1) ∪ (2, ∞) Another operation on sets is the Cartesian product. Specifically, the Cartesian product of two sets A and B, denoted by A × B, is the set of all ordered pairs whose first component comes from A and whose second component comes from B. So A × B = {(x, y) | x ∈ A and y ∈ B} For example, if A = {1, 2} and B = {10, 20}, then A × B = {(1, 10), (1, 20), (2, 10), (2, 20)} This last set is a subset of the Euclidean plane, which can be written as the Cartesian product of ⺢ with itself, so that ⺢2 = ⺢ × ⺢ = {(x, y) | x, y ∈ ⺢}

Confirming Pages

412

Appendix A

Preliminaries

EXAMPLE 2 y 5

Solution

[−3, 2) × (−2, 1] ⫺5

5

x

Let A = [−3, 2) and B = (−2, 1]. Describe the set A × B. Since A × B consists of all ordered pairs whose first component comes from A and second from B, we have −3 ≤ x < 2 and − 2 < y ≤ 1 The points that satisfy these two conditions lie in the rectangular region shown in Fig. 2.

⫺5

Figure 2

EXAMPLE 3 Solution

Example 3 shows that operations on sets can be combined to produce results similar to the arithmetic properties of real numbers.

Verify that if A, B, and C are sets, then A ∩ (B ∪ C) = (A ∩ B) ∪ (A ∩ C). The Venn diagrams in Fig. 3 show that although the two sets are computed in different ways, the result is the same. The quantities inside the parentheses are carried out first. Of course, the picture alone does not constitute a proof. To establish the fact, we must show that the set on the left-hand side of the equation above is a subset of the set on the right, and vice versa. A

A

B C A ∩ (B ∪ C)

B C (A ∩ B) ∪ (A ∩ C)

Figure 3

Indeed, if x ∈ A ∩ (B ∪ C), then x ∈ A and x ∈ B ∪ C. This is equivalent to the statement x ∈ A and (x ∈ B or x ∈ C), which in turn is also equivalent to (x ∈ A and x ∈ B)

or

(x ∈ A and x ∈ C)

Hence, x ∈ (A ∩ B) ∪ (A ∩ C), and we have shown that A ∩ (B ∪ C) ⊆ (A ∩ B) ∪ (A ∩ C) On the other hand, let x ∈ (A ∩ B) ∪ (A ∩ C), which can also be written as x ∈ (A ∩ B)

or

x ∈ (A ∩ C)

x∈B

or

x∈A

This gives x∈A

and

and

In either case, x ∈ A and, in addition, x ∈ B or x ∈ C, so that x ∈ A ∩ (B ∪ C)

x∈C

Confirming Pages

A.1 Algebra of Sets

413

Therefore, (A ∩ B) ∪ (A ∩ C) ⊆ A ∩ (B ∪ C) Since each set is a subset of the other, we have A ∩ (B ∪ C) = (A ∩ B) ∪ (A ∩ C) Theorem 1 includes the result given in Example 3 along with other properties of set operations. The verifications of the remaining properties are left as exercises. THEOREM 1

Let A, B, and C be sets contained in a universal set U. 1. 2. 3. 4. 5. 6.

THEOREM 2

A ∩ A = A, A ∪ A = A (Ac )c = A A ∩ Ac = φ, A ∪ Ac = U A ∩ B = B ∩ A, A ∪ B = B ∪ A (A ∩ B) ∩ C = A ∩ (B ∩ C), (A ∪ B) ∪ C = A ∪ (B ∪ C) A ∩ (B ∪ C) = (A ∩ B) ∪ (A ∩ C) A ∪ (B ∩ C) = (A ∪ B) ∩ (A ∪ C)

DeMorgan’s Laws Let A, B, and C be sets. Then 1. A\(B ∪ C) = (A\B) ∩ (A\C) 2. A\(B ∩ C) = (A\B) ∪ (A\C) Proof (1) We need to verify that the set on the left-hand side of the equation is a subset of the set on the right, and vice versa. We begin by letting x ∈ A\(B ∪ C). This means that x ∈ A and x ∈ / B ∪ C. This is equivalent to the statement x∈A

(x ∈ / B and x ∈ / C)

and

which is then equivalent to x∈A

and

x∈ /B

and

x∈A

and

x∈ /C

This last pair of statements gives x ∈ (A\B) ∩ (A\C)

so that

A\(B ∪ C) ⊆ (A\B) ∩ (A\C)

To show containment in the other direction, we let x ∈ (A\B) ∩ (A\C). Rewriting this in equivalent forms, we have x∈A

and

x ∈ (A\B) x∈ /B x∈A x∈A

and and and and

x x x x

∈ (A\C) ∈ A and ∈ / B and ∈ / (B ∪ C)

x∈ /C x∈ /C

Confirming Pages

414

Appendix A

Preliminaries

Therefore, (A\B) ∩ (A\C) ⊆ A\(B ∪ C) (2) The proof is similar to the one given in part 1 and is left as an exercise.

Exercise Set A.1 In Exercises 1–6, let the universal set be ⺪ and let A = {−4, −2, 0, 1, 2, 3, 5, 7, 9} B = {−3, −2, −1, 2, 4, 6, 8, 9, 10}

17. C × B 18. (A × B)\[C × (B ∩ C)] 19. A × (B ∩ C) 20. (A × B) ∩ (A × C) In Exercises 21–26, let

Compute the set. 1. A ∩ B

A = {1, 2, 3, 5, 7, 9, 11} B = {2, 5, 10, 14, 20} C = {1, 5, 7, 14, 30, 37}

2. A ∪ B 3. A × B 4. (A ∪ B)c

Verify that the statement holds.

5. A\B

21. (A ∩ B) ∩ C = A ∩ (B ∩ C)

6. B\A

22. (A ∪ B) ∪ C = A ∪ (B ∪ C)

In Exercises 7–14, use the sets A = (−11, 3]

B = [0, 8]

C = [−9, ∞)

Compute the set.

23. A ∩ (B ∪ C) = (A ∩ B) ∪ (A ∩ C) 24. A ∪ (B ∩ C) = (A ∪ B) ∩ (A ∪ C) 25. A\(B ∪ C) = (A\B) ∩ (A\C)

7. A ∩ B

26. A\(B ∩ C) = (A\B) ∪ (A\C)

8. (A ∪ B)c

In Exercises 27–34, show that the statement holds for all sets A, B, and C.

9. A\B 10. C\A

27. (Ac )c = A

11. A\C

28. The set A ∪ Ac is the universal set.

12. (A ∪ B)c ∩ C

29. A ∩ B = B ∩ A

13. (A ∪ B)\C

30. A ∪ B = B ∪ A

14. B\(A ∩ C)

31. (A ∩ B) ∩ C = A ∩ (B ∩ C)

In Exercises 15–20, use the sets

32. (A ∪ B) ∪ C = A ∪ (B ∪ C)

A = (−2, 3]

B = [1, 4]

C = [0, 2]

33. A ∪ (B ∩ C) = (A ∪ B) ∩ (A ∪ C)

to sketch the specified set in the plane.

34. A\(B ∩ C) = (A\B) ∪ (A\C)

15. A × B

35. If A and B are sets, show that

16. B × C

A\B = A ∩ B c

Confirming Pages

A.2 Functions

36. If A and B are sets, show that (A ∪ B) ∩ A = B\A c

37. If A and B are sets, show that (A ∪ B)\(A ∩ B) = (A\B) ∪ (B\A)

415

39. If A, B, and C are sets, show that A × (B ∩ C) = (A × B) ∩ (A × C) 40. The symmetric difference operation
on two sets A and B is defined by A
B = (A\B) ∪ (B\A)

38. If A and B are sets, show that (A ∩ B) = A\(A\B)

ß

A.2

Show that A
B = (A ∪ B)\(A ∩ B)

Functions The sets we described in Sec. A.1 along with functions are two of the fundamental objects of modern mathematics. Sets act as nouns defining objects and functions as verbs describing actions to be performed on the elements of a set. Functions connect each element of one set to a unique element of another set. The functions that are studied in calculus are defined on sets of real numbers. Other branches of mathematics require functions that are defined on other types of sets. The following definition is general enough for a wide variety of abstract settings.

DEFINITION 1

Function A function f from a set ⺨ to a set ⺩ is a rule of correspondence that associates with each element of ⺨ exactly one element of ⺩. Before continuing with a description of functions, we note that there are other ways of associating the elements of two sets. A relation is a rule of correspondence that does not (necessarily) assign a unique element of ⺩ for each element of ⺨. A function, then, is a relation that is well defined with a clear procedure that associates a unique element of ⺩ with each element of ⺨. A common metaphor for a function is a machine that produces a unique output for each input. A function f is also called a mapping from ⺨ to ⺩ and is written f : ⺨ −→ ⺩. If x ∈ ⺨ is associated with y ∈ ⺩ via the function f , then we call y the image of x under f and write y = f (x). The set ⺨ is called the domain of f and is denoted by dom(f ). The range of f , denoted by range(f ), is the set of all images of f . That is, range(f ) = {f (x) | x ∈ dom(f )} If A is a subset of the domain, then the image of A is defined by f (A) = {f (x) | x ∈ A} Using this notation, we have range(f ) = f (⺨). There are many ways of describing functions. The pictures shown in Fig. 1, give us one way while providing an illustration of the key idea distinguishing relations from functions.

Confirming Pages

416

Appendix A

Preliminaries f

g

x1

⺨

y1

x2

y2

x3

⺨

⺩

x1

y1

x2 y4

y2 y3

x3

y3

x4

⺩ y4

A function

Not a function

Figure 1

y 5

⫺5

5

x

The relation f , shown in Fig. 1, is a function since it is well defined with each element of the set ⺨ corresponding to a unique element of the set ⺩. Notice that more than one element in the domain of the function f can be associated with the same element in the range. In this case x3 and x4 both map to y3 . However, the relation g, also shown in Fig. 1, is not a function since x3 corresponds to both y3 and y4 . Notice in this example that f (⺨) is not equal to ⺩, since y4 is not in the range of f . In general, for the mapping f : ⺨ −→ ⺩, the set ⺨ is always the domain, but range(f ) ⊆ ⺩. The graph of a function f : ⺨ −→ ⺩ is a subset of the Cartesian product ⺨ × ⺩ and is defined by graph(f ) = {(x, y) | x ∈ ⺨ and y = f (x) ∈ range(f )} For a function f : ⺢ −→ ⺢ the graph is a subset of ⺢2 , the Cartesian plane. A familiar function is f : ⺢ −→ ⺢ defined by the rule

⫺5

Figure 2

f (x) = x 2 − 4x + 3 = (x − 2)2 − 1

y

1 −1

π

2π x

Since the rule describing the function is defined for all real numbers, we have dom(f ) = ⺢. For the range, since the vertex of the parabola is (2, −1), then range(f ) = [−1, ∞). These sets are also evident from the graph of the function, as shown in Fig. 2. Also the image of x = 0 is f (0) = 3. Notice that in this example it is also the case that f (4) = 3, so {0, 4} is the set of all real numbers with image equal to 3. The set {0, 4} is called the inverse image of the set {3}. This motivates the next concept. If f : ⺨ −→ ⺩ is a function and B ⊆ ⺩, then the inverse image of B, denoted by f −1 (B), is the set of all elements of the domain that are mapped to B. That is, f −1 (B) = {x ∈ ⺨ | f (x) ∈ B}

Figure 3

The set f −1 (B) is also called the set of preimages of the set B. As another illustration let f : [0, 2π] −→ [−1, 1] be defined by f (x) = sin x. The graph is shown in Fig. 3. We see from the graph that f −1 ([0, 1]) = [0, π]

and

f −1 ([−1, 0]) = [π, 2π]

Confirming Pages

A.2 Functions

EXAMPLE 1

Let f : ⺢ −→ ⺢ be the function defined by f (x) = x 2 − 4x + 3. Define the sets A = [0, 3] a. b. c. d.

Solution

417

Compare Compare Compare Compare

the the the the

sets sets sets sets

B = [1, 4]

C = [−1, 3]

D = [0, 3]

f (A ∩ B) and f (A) ∩ f (B). f (A ∪ B) and f (A) ∪ f (B). f −1 (C ∩ D) and f −1 (C) ∩ f −1 (D). f −1 (C ∪ D) and f −1 (C) ∪ f −1 (D).

a. Since A ∩ B = [1, 3], we see from Fig. 2 that f (A ∩ B) = [−1, 0]. Again using the graph of f , we have f (A) = f ([0, 3]) = [−1, 3] and f (B) = f ([1, 4]) = [−1, 3], so that f (A) ∩ f (B) = [−1, 3]. Hence, we have shown that f (A ∩ B) ⊆ f (A) ∩ f (B) with f (A ∩ B) = f (A) ∩ f (B) b. Since A ∪ B = [0, 4], we have f (A ∪ B) = [−1, 3]. Also f (A) = [−1, 3] = f (B), so that f (A) ∪ f (B) = [−1, 3]. Therefore, f (A ∪ B) = f (A) ∪ f (B) c. Since C ∩ D = [0, 3], we have f −1 (C ∩ D) = {x ∈ ⺢ | f (x) ∈ [0, 3]} = {x ∈ ⺢ | 0 ≤ f (x) ≤ 3} = {x ∈ ⺢ | 0 ≤ (x − 2)2 − 1 ≤ 3} We see from Fig. 2 f −1 (C ∩ D) = [0, 1] ∪ [3, 4] On the other hand,

f −1 (C) = [0, 4]

The inverse image of the set D is f −1 (D) = [0, 1] ∪ [3, 4] Finally,

f −1 (C) ∩ f −1 (D) = f −1 (C ∩ D)

d. Since C ∪ D = [−1, 3], we have from the results in part (c) f −1 (C ∪ D) = f −1 (C) ∪ f −1 (D)

Theorem 3 summarizes several results about images of sets and inverse images of sets including the observations made in Example 1.

Confirming Pages

418

Appendix A

THEOREM 3

Preliminaries

Let f : ⺨ −→ ⺩ be a function, and suppose A and B are subsets of ⺨ and C and D are subsets of ⺩. Then 1. 2. 3. 4. 5. 6.

f (A ∩ B) ⊆ f (A) ∩ f (B) f (A ∪ B) = f (A) ∪ f (B) f −1 (C ∩ D) = f −1 (C) ∩ f −1 (D) f −1 (C ∪ D) = f −1 (C) ∪ f −1 (D) A ⊆ f −1 (f (A)) f (f −1 (C)) ⊆ C

Proof (1) Let y ∈ f (A ∩ B). Then there is some x ∈ A ∩ B such that y = f (x). This means that y ∈ f (A) and y ∈ f (B), and hence y ∈ f (A) ∩ f (B). Therefore, f (A ∩ B) ⊆ f (A) ∩ f (B). (3) To show that the sets are equal, we show that each set is a subset of the other. Let x ∈ f −1 (C ∩ D), so that f (x) ∈ C ∩ D, which is equivalent to the statement f (x) ∈ C and f (x) ∈ D. Therefore, x ∈ f −1 (C) and x ∈ f −1 (D), and we have f −1 (C ∩ D) ⊆ f −1 (C) ∩ f −1 (D). Now let x ∈ f −1 (C) ∩ f −1 (D), which is equivalent to the statement x ∈ −1 f (C) and x ∈ f −1 (D). Then f (x) ∈ C and f (x) ∈ D, so that f (x) ∈ C ∩ D. Therefore, x ∈ f −1 (C ∩ D) and hence f −1 (C) ∩ f −1 (D) ⊆ f −1 (C ∩ D). (5) If x ∈ A, then f (x) ∈ f (A), and hence x ∈ f −1 (f (A)). This gives A ⊆ f −1 (f (A)). The proofs of parts 2, 4, and 6 are left as exercises.

Example 1(a) provides a counterexample to show that the result in Theorem 3, part 1, cannot be replaced with equality.

Inverse Functions An inverse function of a function f , when it exists, is a function that reverses the action of f . Observe that if g is an inverse function of f and f (a) = b, then g(b) = a. For example, if f (x) = 3x − 1 and g(x) = (x + 1)/3, then f (2) = 5 and g(5) = 2. One of the most important function-inverse pairs in mathematics and science is f (x) = ex and g(x) = ln x. For a function to have an inverse function, the inverse image for each element of the range of the function must be well defined. This is often not the case. For example, the function f : ⺢ −→ ⺢ defined by f (x) = x 2 cannot be reversed as a function since the inverse image of the set {4} is the set {−2, 2}. Notice that the inverse image of a set in the range of a function is always defined, but the function may not have an inverse function. A function that has an inverse is called invertible. Later in this section we show that if a function is invertible, then it has a unique inverse. This will justify the use of the definite article and the symbol f −1 when referring to the inverse of the function f . Functions that have inverses are characterized by the

Revised Confirming Pages

A.2 Functions

419

property called one-to-one. The function described in Fig. 1 is not one-to-one, since both x3 and x4 are sent to the same element of ⺩. This cannot occur for a one-to-one function. DEFINITION 2

One-to-One Function Let f : ⺨ −→ ⺩ be a function. Then f is called oneto-one, or injective, if for all x1 and x2 with x1 = x2 , then f (x1 ) = f (x2 ). Alternatively, f is one-to-one if whenever f (x1 ) = f (x2 ), then x1 = x2 . For a function f : ⺢ −→ ⺢, this condition is met if every horizontal line intersects the graph of f in at most one point. When this happens, f passes the horizontal line test and is thus invertible. This test is similar to the vertical line test used to determine if f is a function. The inverse of f is denoted by f −1 with f −1 : range(f ) −→ ⺨. Theorem 4 gives a characterization of functions that are invertible. We omit the proof.

THEOREM 4

Let ⺨ and ⺩ be nonempty sets and f : ⺨ −→ ⺩ be a function. The function f has an inverse function if and only if f is one-to-one. As an illustration, let f : ⺢ −→ ⺢ be defined by y = f (x) = 3x + 1. Since the graph, which is a straight line, satisfies the horizontal line test, the function is oneto-one and hence has an inverse function. To find the inverse in this case is an easy matter. We can solve for x in terms of y to obtain y−1 x= 3 The inverse function is then written using the same independent variable, so that x−1 3 It is also possible to show that a function has an inverse even when it is difficult to find the inverse. f −1 (x) =

EXAMPLE 2 Solution

Show the function f : ⺢ −→ ⺢ defined by f (x) = x 3 + x is invertible. By Theorem 4, to show that f is invertible, we show that f is one-to-one. Suppose that x1 = x2 with x1 < x2 . We wish to show that f (x1 ) = f (x2 ). Since the cubing function is strictly increasing for all x, we have x1 < x2

and

x13 < x23

Therefore, f (x1 ) = x13 + x1 < x23 + x2 = f (x2 )

so that

f (x1 ) = f (x2 )

Revised Confirming Pages

420

Appendix A

Preliminaries

The graph of an invertible function can be used to describe the graph of the inverse function. To see how, suppose that (a, b) is a point on the graph of y = f (x). Then b = f (a) and a = f −1 (b). Consequently, the point (b, a) is on the graph of y = f −1 (x). Since the point (b, a) is the reflection of (a, b) through the line y = x, the graphs of f and f −1 are also reflections through y = x. The graph of the function and its inverse in Example 2 are shown in Fig. 4. y 5

f (x) = x 3 + x y = f −1 (x)

⫺5

5

x

⫺5

Figure 4

When f : ⺨ −→ ⺩ is a function such that the set of images is all of ⺩, that is, f (⺨) = ⺩, we call the function onto. DEFINITION 3

Onto Function range(f ) = ⺩.

The function f : ⺨ −→ ⺩ is called onto, or surjective, if

For example, the function of Example 2 is onto since the range of f is all of ⺢. See Fig. 4. A function is called bijective if it is both one-to-one and onto. Notice that the function f : ⺢ −→ ⺢ with f (x) = x 2 is not onto since range (f ) = [0, ∞). Of course, every function is a mapping onto its range. So the function f : ⺢ −→ [0, ∞) defined by f (x) = x 2 is onto. This new version of the original function is not one-to-one, but by restricting the domain to [0, ∞), we can define a version that is one-to-one and onto. That is, the function f : [0, ∞) −→ [0, ∞) defined by f (x) = x 2 is a bijection. The function defined in Example 2 is also bijective. Notice also that a function has an inverse if and only if it is bijective.

Composition of Functions Functions can be combined in a variety of ways to create new functions. For example, if f : ⺨1 −→ ⺩1 and g: ⺨2 −→ ⺩2 are real-valued functions of a real variable, then the standard arithmetic operations on functions are defined by (f + g)(x) = f (x) + g(x) (f − g)(x) = f (x) − g(x) (f g)(x) = f (x)g(x)   f (x) f (x) = g g(x)

Confirming Pages

A.2 Functions

421

The domains of these functions are given by dom(f + g) = dom(f − g) = dom(f g) = ⺨1 ∩ ⺨2 and dom(f/g) = (⺨1 ∩ ⺨2 )\{x | g(x) = 0}. Another method of combining functions is through the composition of two functions. In the composition of two functions f and g, the √ output of one function is used as the input to the other. For example, if f (x) = x and g(x) = x 2 − x − 2, then f (g(3)) = f (4) = 2 is the composition of f with g evaluated at the number 3 and is denoted by (f ◦g)(3).

DEFINITION 4

Composition Let A, B, and C be nonempty sets and f : B −→ C and g: A −→ B be functions. The composition f ◦g: A −→ C is defined by (f ◦g)(x) = f (g(x)) The domain of the composition is dom(f ◦g) = {x ∈ dom(g) | g(x) ∈ dom(f )}. A function and its inverse undo each other relative to composition. For example, let f (x) = 2x − 1. Since f is one-to-one, it is invertible with f −1 (x) = (x + 1)/2. Notice that 2x − 1 + 1 f (x) + 1 (f −1 ◦f )(x) = f −1 (f (x)) = = =x 2 2 and   x+1 −1 −1 ◦ (f f )(x) = f (f (x)) = 2 −1=x+1−1=x 2

THEOREM 5

Suppose that f : ⺨ −→ ⺩ is a bijection. Then 1. (f −1 ◦f )(x) = x for all x ∈ ⺨ 2. (f ◦f −1 )(x) = x for all x ∈ ⺩ As mentioned earlier, when an inverse function exists, it is unique. To see this, let f : ⺨ −→ ⺩ be an invertible function and f −1 an inverse function. Suppose that g: ⺩ −→ ⺨ is another inverse function for f . Let I⺨ be the identity function on ⺨ and I⺩ the identity function on ⺩. That is, I⺨ (x) = x for all x ∈ ⺨ and I⺩ (y) = y for all y ∈ ⺩. If y is in ⺩, then g(y) = g ◦I⺩ (y) = g ◦(f ◦f −1 )(y) = g(f (f −1 (y))) = (g ◦f )◦f −1 (y) = I⺨ (f −1 (y)) = f −1 (y) Since this holds for all y in ⺩, then g = f −1 . Consequently, when it exists, the inverse function is unique. This justifies the use of the symbol f −1 for the inverse of f , when it exists.

Confirming Pages

422

Appendix A

THEOREM 6

Preliminaries

Let f : ⺨ −→ ⺩ be a bijection. Then 1. f −1 : ⺩ −→ ⺨ is also a bijection, 2. (f −1 )−1 = f

THEOREM 7

Let A, B, and C be nonempty sets and f : B −→ C and g: A −→ B be functions. 1. 2. 3. 4. 5.

If If If If If

f and g are injections, then f ◦g is an injection. f and g are surjections, then f ◦g is a surjection. f and g are bijections, then f ◦g is a bijection. f ◦g is an injection, then g is an injection. f ◦g is a surjection, then f is a surjection.

Proof (1) Suppose that x1 and x2 are in A and (f ◦g)(x1 ) = (f ◦g)(x2 ). Then by the definition of composition, we have f (g(x1 )) = f (g(x2 )) Since f is an injection, g(x1 ) = g(x2 ). But since g is also an injection, we have x1 = x2 . Therefore, f ◦g is an injection. (5) Let c ∈ C. Since f ◦g: A −→ C is a surjection, there is some a ∈ A such that (f ◦g)(a) = c. That is, f (g(a)) = c. But g(a) ∈ B, so there is an element of B with image under f equal to c. Since c was chosen arbitrarily, we know that f is a surjection. The proofs of parts 2, 3, and 4 are left as exercises.

THEOREM 8

Let A, B, and C be nonempty sets and f : B −→ C and g: A −→ B be functions. If f and g are bijections, then the function f ◦g has an inverse function and (f ◦g)−1 = g −1 ◦f −1 . Proof By Theorem 7, the composition f ◦g: A −→ C is a bijection; hence by Theorem 4, the inverse function (f ◦g)−1 : C −→ A exists. Moreover, the function g −1 ◦f −1 also maps C to A. For each c ∈ C we will show that (f ◦g)−1 (c) = (g −1 ◦f −1 )(c). Let c ∈ C. Since f is onto, there is b ∈ B such that f (b) = c, so that b = f −1 (c). Next, since g is onto, there is an a ∈ A such that g(a) = b, which is equivalent to a = g −1 (b). Taking compositions gives (f ◦g)(a) = f (g(a)) = c, and hence (f ◦g)−1 (c) = a. We also have g −1 (f −1 (c)) = (g −1 ◦f −1 )(c) = a. Since this holds for all c ∈ C, the functions (f ◦g)−1 and g −1 ◦f −1 are identical, that is, (f ◦g)−1 = g −1 ◦f −1 .

Confirming Pages

A.2 Functions

423

Exercise Set A.2 15. Define a function f : ⺢ → ⺢ by f (x) = ax + b, where a and b are real numbers with a = 0. Find the inverse function of f .

In Exercises 1–10, let ⺨ = {1, 2, 3, 4, 5, 6} ⺩ = {−2, −1, 3, 5, 9, 11, 14} and define f : ⺨ −→ ⺩ by the set of ordered pairs {(1, −2), (2, 3), (3, 9), (4, −2), (5, 11), (6, −1)} 1. Explain why f is a function.

16. Define a function f : ⺢ → ⺢ by f (x) = x 5 + 2x. Show that the inverse function of f exists. 17. Given a function f , define for each positive integer n f n (x) = (f ◦f ◦ · · · ◦f )(x)

2. Is f a one-to-one function? Explain. 3. Is f an onto function? Specify range(f ). 4. Let A = {1, 2, 4}. Find f (A). 5. Find f −1 ({−2}). 6. Find f −1 (f ({1})). 7. Does f have an inverse function? Explain. 8. Is it possible to define a function with domain ⺨ that is onto ⺩? Explain. 9. Define a function g: ⺨ −→ ⺩ that is one-to-one. 10. Is it possible to define a function g: ⺩ −→ ⺨ that is onto? Explain. In Exercises 11–14, use the function f : ⺢ → ⺢ defined by f (x) = x 2 11. Let A = (−3, 5) and B = [0, 7). Verify that f (A ∪ B) = f (A) ∪ f (B) 12. Let C = [1, ∞) and D = [3, 5]. Verify that f −1 (C ∪ D) = f −1 (C) ∪ f −1 (D) 13. Let A = [−2, 0] and B = [0, 2]. Verify that

where the composition is taken n − 1 times. If c is a fixed real number and f (x) = −x + c, find f n (c) for all n. 18. Define a function f : ⺢ → ⺢ by
2x if 0 ≤ x ≤ 12 f (x) = 2 − 2x if 12 < x ≤ 1 Sketch the graphs of y = f (x) and y = (f ◦f )(x). 19. Define a function f : ⺢ → ⺢ by f (x) = e2x−1 a. Show that f is one-to-one. b. Is f onto? Justify your answer. c. Define a function g with the same rule and domain as f but that is onto. d. Find the inverse function for the g defined in part (c). 20. Define a function f : ⺢ → ⺢ by f (x) = e−x

2

Show the function is not one-to-one. 21. Define a function f : ⺞ → ⺞ by

f (A ∩ B) ⊂ f (A) ∩ f (B)

f (n) = 2n

and the sets are not equal. 14. Define a function g by the rule g(x) = x , but with the domain restricted to the interval [0, ∞). If A = [0, 5) and B = [2, 7), verify that 2

g(A ∩ B) = g(A) ∩ g(B) What property does g have that f does not?

a. Show that f is one-to-one. b. Is f onto? Explain. c. If E denotes the set of even positive integers and O the odd positive integers, find f −1 (E) and f −1 (O).

Revised Confirming Pages

424

Appendix A

Preliminaries

22. Define a function f : ⺪ → ⺪ by
n+1 if n is even f (n) = n−3 if n is odd Let E denote the set of even integers and O the set of odd integers. Find f (E) and f (O). 23. Define a function f : ⺪ × ⺪ → ⺪ by f ((m, n)) = 2m + n a. Let A = {(p, q) | p and q are odd}. Find f (A). b. Let B = {(p, q) | q is odd}. Find f (B). c. Find f −1 ({0}). d. Let E denote the set of even integers. Find f −1 (E). e. Let O denote the set of odd integers. Find f −1 (O). f. Show that f is not one-to-one. g. Show that f is onto.

c. Let A be the set of all points that lie on the line y = x + 1. Find f (A). In Exercises 25–27, f : ⺨ → ⺩ is a function, A and B are subsets of ⺨, and C and D are subsets of ⺩. Prove the statements. 25. f (A ∪ B) = f (A) ∪ f (B) 26. f −1 (C ∪ D) = f −1 (C) ∪ f −1 (D) 27. f (f −1 (C)) ⊆ C In Exercises 28–30, f : B → C and g: A → B are functions. Prove the statements. 28. If f and g are surjections, then f ◦g is a surjection. 29. If f and g are bijections, then f ◦g is a bijection. 30. If f ◦g is an injection, then g is an injection. 31. If f : ⺨ → ⺩ is a function and A and B are subsets of ⺨, show that f (A)\f (B) ⊆ f (A\B)

24. Define a function f : ⺢2 → ⺢2 by f ((x, y)) = (2x, 2x + 3y) a. Show that f is one-to-one. b. Is f onto? Justify your answer.

ß

A.3

32. If f : ⺨ → ⺩ is a function and C and D are subsets of ⺩, show that f −1 (C\D) = f −1 (C)\f −1 (D)

Techniques of Proof Mathematics is built on facts. A few of these, called axioms, are accepted as selfevident and do not require justification. Every other statement of fact requires proof. A proof is the process of establishing the validity of a statement. Results in mathematics that require proof are called theorems and are made up of two parts. The first part, called the hypothesis, is a set of assumptions. The second part, called the conclusion, is the statement that requires proof. It is customary to use the letter P to denote the hypotheses (or hypothesis if there is only one) and the letter Q to denote the conclusion. A theorem is symbolized by P ⇒ Q which we read as “if P , then Q” or “P implies Q” or “P is sufficient for Q.” The converse of a theorem is symbolized by Q ⇒ P

Revised Confirming Pages

A.3 Techniques of Proof

425

read as “Q implies P ” or “P is necessary for Q.” For example, let P be the statement Mary lives in Iowa and Q the statement that Mary lives in the United States. Then certainly P
⇒ Q is a theorem since every resident of Iowa is a resident of the United States. But Q
⇒ P is not a theorem since, for example, if Mary is a resident of California, then she is a resident of the United States but not a resident of Iowa. So the statement Q
⇒ P is not always true given that Q is true. In terms of sets, if A is the set of residents of Iowa and B is the set of residents of the United States, then the statement P is Mary is in A and Q is Mary is in B. Then Mary is in A implies Mary is in B. It is also clear that if Mary is in B\A, then Mary is in B does not imply that Mary is in A. A statement that is equivalent to the theorem P
⇒ Q is the contrapositive statement ∼Q
⇒ ∼P , that is, not Q implies not P. In the example above, if Mary is not a resident of the United States, then Mary is not a resident of Iowa. An equivalent formulation of the statement, in the terminology of sets, is that if Mary ∈ / B, then it implies Mary ∈ / A. There are other statements in mathematics that require proof. Lemmas are preliminary results used to prove theorems, propositions are results not as important as theorems, and corollaries are special cases of a theorem. A statement that is not yet proven is called a conjecture. One of the most famous conjectures is the celebrated Riemann hypothesis. A single counterexample is enough to refute a false conjecture. For example, the statement All lions have green eyes is rendered invalid by the discovery of a single blue-eyed lion. In this section we briefly introduce three main types of proof. A fourth type, called mathematical induction, is discussed in Sec. A.4.

Direct Argument In a direct argument, a sequence of logical steps links the hypotheses P to the conclusion Q. Example 1 provides an illustration of this technique.

EXAMPLE 1 Solution

Prove that if p and q are odd integers, then p + q is an even integer. To prove this statement with a direct argument, we assume that p and q are odd integers. Then there are integers m and n such that p = 2m + 1

and

q = 2n + 1

Adding p and q gives p + q = 2m + 1 + 2n + 1 = 2(m + n) + 2 = 2(m + n + 1) Since p + q is a multiple of 2, it is an even integer.

Confirming Pages

426

Appendix A

Preliminaries

Contrapositive Argument The contrapositive statement of the statement P ⇒ Q is the statement ∼Q ⇒ ∼P . The notation ∼Q denotes the negation of the statement Q. A statement and the contrapositive statement are equivalent, so that if one holds, then the other also holds. In a contrapositive argument the hypothesis is ∼Q, and we proceed with a direct argument to show that ∼P holds.

EXAMPLE 2 Solution

If p2 is an even integer, then p is an even integer. In a direct argument we assume that p2 is even, so that we can write p2 = 2k for some integer k. Then √ √ √ p = 2k = 2 k which does not allow us to conclude that p is even. To use a contrapositive argument, we assume that p is not an even integer. That is, we assume that p is an odd integer. Then there is an integer k such that p = 2k + 1. Squaring both sides of equation p = 2k + 1 gives p2 = (2k + 1)2 = 4k 2 + 4k + 1 = 2(2k 2 + 2k) + 1 and hence p2 is an odd integer. Therefore, the original statement holds.

Contradiction Argument In a contradiction argument to show that a statement holds, we assume the contrary and use this assumption to arrive at some contradiction. For example, to prove that the set of natural numbers ⺞ is infinite, we would assume the set of natural numbers is finite and argue that this leads to a contradiction. A contrapositive argument is a form of contradiction where to prove P ⇒ Q, we assume that P holds and ∼Q holds and arrive at the conclusion that ∼P holds. Since both P and ∼P cannot be true, we have a contradiction. In certain cases the contradiction may be hard to recognize.

EXAMPLE 3 Solution

Prove that

√ 2 is an irrational number.

To use a contradiction argument, we assume that assume that there are integers p and q such that √ p 2= q

√ 2 is not irrational. That is, we

Confirming Pages

A.3 Techniques of Proof

427

where p and q√have no common factors. We will arrive at a contradiction by showing that if 2 = p/q, then p and q do have a common factor. Squaring both sides of the last equation gives 2=

p2 q2

so that

p2 = 2q 2

Hence, p2 is even. Since p2 is an even integer, then by Example 2 so is p. Thus, there is an integer k such that p = 2k. Substituting 2k for p in the equation 2q 2 = p2 gives so that q 2 = 2k 2 2q 2 = p2 = (2k)2 = 4k 2 Hence, q is also an even integer. Since p and q are both even, they have a common factor of 2, which contradicts the assumption that p and q are chosen to have no common factors.

Quantifiers Often statements in mathematics are quantified using the universal quantifier for all, denoted by the symbol ∀, or by the existential quantifier there exists, denoted by the symbol ∃. If P (x) is a statement that depends on the parameter x, then the symbols ∀x, P (x) are read for all x, P (x). To prove that the statement is true, we have to verify that the statement P (x) holds for every choice of x. To prove that the statement is false, we need to find only one x such that P (x) is false, that is, we need to find a counterexample. To prove that a statement of the form ∃x, P (x) holds requires finding at least one x such that P (x) holds. The statement is false if the statement ∼(∃x, P (x)) holds. When we negate a statement involving quantifiers, ∼∃ becomes ∀ and ∼∀ becomes ∃. So the statement ∼(∃x, P (x))

is equivalent to

∀x, ∼P (x)

∼(∀x, P (x))

is equivalent to

∃x, ∼P (x)

and the statement

Confirming Pages

428

Appendix A

Preliminaries

Exercise Set A.3 1. Prove that in an √ isosceles right triangle, the hypotenuse is 2 times the length of one of the equal sides. 2. Prove that if ABC is an isosceles right triangle with C the vertex of the right angle and sides opposite the vertices a, b, and c, respectively, then the area of the triangle is c2 /4. 3. Prove that √ in an equilateral triangle the area of the triangle is 3/4 times the square of the length of a side. 4. Prove that if s and t are rational numbers with t = 0, then s/t is a rational number. 5. Prove that if a, b, and c are integers such that a divides b and b divides c, then a divides c.

17. Prove that if x is a nonnegative real number such that x < , for every real number  > 0, then x = 0. 18. Prove that if x is a rational number and x + y is an irrational number, then y is an irrational number. √ 19. Prove that 3 2 is irrational. 20. Prove that if n in ⺞, then n n > n+1 n+2 21. Suppose that x and y are real numbers with x < 2y. Prove that if 7xy ≤ 3x 2 + 2y 2 , then 3x ≤ y. 22. Define a function f : ⺨ → ⺩ and sets A and B in ⺨ that is a counterexample to show the statement

6. Prove that if m and n are even integers, then m + n is an even integer. 7. Prove that if n is an odd integer, then n2 is an odd integer. 8. Prove that if n is in ⺞, then

n2

+ n + 3 is odd.

9. Prove that if a and b are consecutive integers, then (a + b)2 is an odd integer. 10. Prove that if m and n are odd integers, then mn is an odd integer. 11. Show that the statement if m and n are two consecutive integers, then 4 divides m2 + n2 is false.

If f (A) ⊆ f (B), then A ⊂ B is false. 23. Define a function f : ⺨ → ⺩ and sets C and D in ⺩ that is a counterexample to show the statement If f −1 (C) ⊆ f −1 (D), then C ⊂ D is false. In Exercises 24–30, f : ⺨ → ⺩ is a function, A and B are subsets of ⺨, and C and D are subsets of ⺩. Prove the statements. 24. If A ⊆ B, then f (A) ⊆ f (B). 25. If C ⊆ D, then f −1 (C) ⊆ f −1 (D).

12. Let f (x) = (x − 1)2 and g(x) = x + 1. Prove that if x is in the set S = {x ∈ ⺢ | 0 ≤ x ≤ 3}, then f (x) ≤ g(x).

26. If f is an injection, then for all A and B

13. Prove that if n is an integer and n2 is odd, then n is odd.

27. If f is an injection, then for all A and B

14. Prove that if n is an integer and n3 is even, then n is even. 15. Prove that if p and q are positive real numbers √ such that pq = (p + q)/2, then p = q. 16. Prove that if c is an odd integer, then the equation n2 + n − c = 0 has no integer solution for n.

f (A ∩ B) = f (A) ∩ f (B)

f (A\B) = f (A)\f (B) 28. If f is an injection, then for all A f −1 (f (A)) = A 29. If f is a surjection, then for all C f (f −1 (C)) = C

Confirming Pages

A.4 Mathematical Induction

ß

A.4

429

Mathematical Induction Throughout mathematics there are statements that depend on natural numbers and where the aim is to determine whether the statement is true or false for all natural numbers. Some simple examples are the following three statements, the third being a well-known puzzle, called the Tower of Hanoi puzzle. 1. For every natural number n, the sum of the first n natural numbers is given by n(n + 1) 2 2. The expression 6n + 1 is a prime number for every natural number n. 3. Given three pegs, labeled 1, 2, and 3, and a stack of n disks of decreasing diameters on peg–1, the disks can be moved to peg–3 in 2n − 1 moves. This is under the restriction that a disk can be placed on top of another disk only when it has smaller diameter. 1 + 2 + 3 + ··· + n =

When we are considering a statement involving natural numbers to provide insight, a useful first step is to substitute specific numbers for n and determine whether the statement is true. If the statement is false, often a counterexample is found quickly, allowing us to reject the statement. For example, in the second statement above, for n = 1, 2, and 3 the expression 6n + 1 has values 7, 13, and 19, respectively, all of which are prime numbers. However, if n = 4, then 6(4) + 1 = 25, which is not a prime number, and the statement is not true for all natural numbers n. In the case of the first statement, the data in Table 1 provide more convincing evidence that the formula may indeed hold for all natural numbers. Of course, to establish the fact for all n requires a proof, which we postpone until Example 1. For the Tower of Hanoi puzzle, when n = 1, the number of steps required is 1, and when n = 2, it is also easy to see a solution requiring 3 steps. A solution for n = 3 is given by the moves D3 −→ P 3, D2 −→ P 2, D3 −→ P 2, D1 −→ P 3, D3 −→ P 1, D2 −→ P 3, D1 −→ P 3

Table 1 1 + 2 + 3 + ··· + n

n(n+1) 2

1

(1)(2) 2

=1

1+2=3

(2)(3) 2

=3

1+2+3 =6

(3)(4) 2

=6

1 + 2 + 3 + 4 = 10

(4)(5) 2

= 10

1 + 2 + 3 + 4 + 5 = 15

(5)(6) 2

= 15

1 + 2 + 3 + 4 + 5 + 6 = 21

(6)(7) 2

= 21

1 + 2 + 3 + 4 + 5 + 6 + 7 = 28

(7)(8) 2

= 28

Confirming Pages

430

Appendix A

Preliminaries

where D1, D2, and D3 represent the three disks of decreasing diameters and P 1, P 2, and P 3 represent the three pegs. So for n = 3, we have a solution with 7 = 23 − 1 moves. Again, the evidence is leading toward the result being true, but we have not given a satisfactory proof. Let’s push this example a bit further. How can we use the result for three disks to argue that this result holds for four disks? The same sequence of steps we gave for the solution of the three-disk problem can be used to move the stack from P 1 to either P 2 or P 3. Now, suppose that there are four disks on P 1. Since the bottom disk is the largest, P 1 can be used as before to move the top three disks. So as a first step, move the top three disks to P 2, which requires 23 − 1 = 7 moves. Next, move the remaining (largest) disk on P 1 to P 3, which requires 1 move. Now, using the same procedure as before, move the three-disk stack on P 2 over to P 3, requiring another 23 − 1 = 7 moves. The total number of moves is now 2(23 − 1) + 1 = 24 − 2 + 1 = 24 − 1 = 15 This approach contains the essentials of mathematical induction. We start with an initial case, called the base case, that we can argue holds. The next step, called the inductive hypothesis, provides a mechanism for advancing from one natural number to the next. In the Tower of Hanoi example, the base case is the case for n = 1, and one disk on P 1 requires only 1 = 21 − 1 move to transfer the disk to P 3 or P 2. The inductive hypothesis is to assume that the result holds when there are n disks on P 1. We are required to argue the result holds for n + 1 disks on P 1. We did this for n = 3. Theorem 9 provides a formal statement of the principle of mathematical induction. The proof of this statement, which we omit, is based on the axiomatic foundations of the natural numbers. Specifically, the proof uses the well-ordering principle, which states that every nonempty subset of ⺞ has a smallest element.

THEOREM 9

The Principle of Mathematical Induction Let P be a statement that depends on the natural number n. Suppose that 1. P is true for n = 1 and 2. When P is true for a natural number n, then P is true for the successor n + 1 Then the statement P is true for every natural number n.

The principle of mathematical induction is also referred to as mathematical induction, or simply induction. An analogy to describe the process of mathematical induction is an infinite row of dominoes that are toppled one domino at a time, starting with the first domino. If the dominoes are set up so that whenever a domino falls its successor will fall (the inductive hypothesis), then the entire row of dominoes will fall once the first domino is toppled (base case).

Confirming Pages

A.4 Mathematical Induction

431

The principle of mathematical induction is used to prove a statement holds for all natural numbers, or for all natural numbers beyond a fixed natural number. This is illustrated in the following examples.

EXAMPLE 1

Prove that for every natural number n, n 

k = 1+ 2 + 3 + ···+ n =

k=1

Solution

n(n + 1) 2

To establish the base case when n = 1, notice that (1)(2) 2 The inductive hypothesis is to assume that the statement is true for some fixed natural number n. That is, we assume 1=

n(n + 1) 2 Next, add n + 1 to both sides of the last equation to obtain 1 + 2 + 3 + ··· + n =

1 + 2 + 3 + · · · + n + (n + 1) = (1 + 2 + 3 + · · · + n) + (n + 1) and we apply the inductive hypothesis to conclude 1 + 2 + 3 + · · · + n + (n + 1) = (1 + 2 + 3 + · · · + n) + (n + 1) n(n + 1) + (n + 1) 2 (n + 1)(n + 2) = 2 The last equality agrees with the stated formula for the successor of n, that is, for n + 1. Therefore, by induction the statement holds for all natural numbers. =

EXAMPLE 2 Solution

Prove that for every natural number n, the number 3n − 1 is divisible by 2. In Table 2 we have verified that for n = 1, 2, 3, 4, and 5 the number 3n − 1 is divisible by 2. In particular, if n = 1, then 3n − 1 = 2, which is divisible by 2. Next, we assume that the statement 3 n − 1 is divisible by 2 holds. To complete the proof, we must verify that the number 3n+1 − 1 is also divisible by 2. Since 3n − 1 is

Confirming Pages

432

Appendix A

Preliminaries

divisible by 2, then there is a natural number q such that

Table 2 n

3n − 1

1

2

2

8

3

26

4

80

5

242

3n − 1 = 2q which gives 3n = 2q + 1 Next, we rewrite the expression 3n+1 − 1 to include 3n in order to use the inductive hypothesis. This gives 3n+1 − 1 = 3(3n ) − 1 = 3(2q + 1) − 1 = 6q + 2 = 2(3q + 1) Therefore, the expression 3n+1 − 1 is also divisible by 2.

Recall that factorial notation is used to express the product of consecutive natural numbers. Several examples are 1! = 1 2! = 1 · 2 = 2 3! = 1 · 2 · 3 = 6 4! = 1 · 2 · 3 · 4 = 24 .. . 20! = 2, 432, 902, 008, 176, 640, 000 For a natural number n, the definition of n factorial is the positive integer n! = n(n − 1)(n − 2) · · · 3 · 2 · 1 We also define 0! = 1.

EXAMPLE 3

Verify that for every natural number n, n! ≥ 2n−1

Solution

For n = 1 the statement is true, since n! = 1! = 1 and 2n−1 = 20 = 1. Now assume that the statement n! ≥ 2n−1 holds. Next, we consider (n + 1)! = (n + 1)n! which we need to show is greater than or equal to 2n . Applying the inductive hypothesis to n! gives the inequality (n + 1)! ≥ (n + 1)2n−1

Confirming Pages

A.4 Mathematical Induction

433

Since for every natural number n ≥ 1 it is also the case that n + 1 ≥ 2, we have (n + 1)! ≥ (n + 1)2n−1 ≥ 2 · 2n−1 = 2n Consequently, the statement n! ≥ 2n−1 is true for every natural number n.

EXAMPLE 4 Solution

For any natural number n, find the sum of the odd natural numbers from 1 to 2n − 1. The first five cases are given in Table 3. Table 3 n

2n − 1

1 + 3 + · · · + (2n − 1)

1

1

1

2

3

1+3 =4

3

5

1+3+5 =9

4

7

1 + 3 + 5 + 7 = 16

5

9

1 + 3 + 5 + 7 + 9 = 25

The data in Table 3 suggest that for each n ≥ 1, 1 + 3 + 5 + 7 + · · · + (2n − 1) = n2 Starting with the case for n = 1, we see that the left-hand side is 1 and the expression on the right is 12 = 1. Hence, the statement holds when n = 1. Next, we assume that 1 + 3 + 5 + · · · + (2n − 1) = n2 . For the next case when the index is n + 1, we consider the sum 1 + 3 + 5 + · · · + (2n − 1)+[2(n + 1)−1] = 1 + 3 + 5 + · · · + (2n − 1) + (2n + 1) Using the inductive hypothesis, we get 1 + 3 + 5 + · · · + (2n − 1) +[2(n + 1) − 1] = n2 + (2n + 1)  

n2

= n2 + 2n + 1 = (n + 1)2 Therefore, by induction the statement holds for all natural numbers.

Confirming Pages

434

Appendix A

Preliminaries

EXAMPLE 5

Let P1 , P2 , . . . , Pn be n points in a coordinate plane with no three points collinear (in a line). Verify that the number of line segments joining all pairs of points is n2 − n 2

Solution

In Fig. 1 is a picture for the case with five points. The number of line segments connecting pairs of points is 10 = (52 − 5)/2. P2

P3

P1

P5 P4

Figure 1

If one additional point is added to the graph in Fig. 1, the result is the graph shown in Fig. 2. Moreover, adding the one additional point requires adding five additional line segments, one to connect the new point to each of the five original points. In general, an additional n line segments are required to move from a graph with n points to one with n + 1 points. P2

P3

P1

P5 P4

Figure 2

These observations lead to the following proof by induction. If there is only one point, then the graph contains no line segments. Also since (12 − 1)/2 = 0, the statement holds for n = 1. Next, assume the number of line segments needed to join n points in a coordinate plane is (n2 − n)/2. If there is one additional point, that is, n + 1 points, then n additional line segments are required. Hence, by the inductive hypothesis, the total number of line segments required for

Confirming Pages

A.4 Mathematical Induction

435

n + 1 points is n2 − n + 2n n2 − n +n= 2 2 n2 + 2n + 1 − 1 − n = 2 (n + 1)2 − (n + 1) = 2 Therefore, by induction the statement holds for all natural numbers.

Binomial Coefficients and the Binomial Theorem In Fig. 3 are the first eight rows of Pascal’s triangle. Notice that each element can be obtained from the sum of the two elements to the immediate left and right in the row above. 1 1 1 1 1 1 1 1

7

3 4

5 6

1 3

6 10

15 21

1 2

1 4

10 20

35

1 5

15 35

1 6

21

1 7

1

Figure 3

In Fig. 4 are the expansions for (a + b)n for n = 0, 1, 2, 3, . . . , 7. The coefficients of the expansions are exactly the numbers in Pascal’s triangle. (a + b)0 (a + b)1 (a + b)2 (a + b)3 (a + b)4 (a + b)5 (a + b)6 (a + b)7

1 a+b a 2 + 2ab + b2 3 a + 3a 2 b + 3ab2 + b3 a 4 + 4a 3 b + 6a 2 b2 + 4ab3 + b4 a 5 + 5a 4 b + 10a 3 b2 + 10a 2 b3 + 5ab4 + b5 a 6 + 6a 5 b + 15a 4 b2 + 20a 3 b3 + 15a 2 b4 + 6ab5 + b6 7 a + 7a 6 b + 21a 5 b2 + 35a 4 b3 + 35a 3 b4 + 21a 2 b5 + 7ab6 + b7 Figure 4

The numbers in Pascal’s triangle or the coefficients of an expansion of the form (a + b)n are called the binomial coefficients. Notice that the number 20, in Fig. 3, is

Confirming Pages

436

Appendix A

Preliminaries

located in row 6 (starting with a row 0) and column 3 (starting with a column 0). In addition, using factorials, we have the formula 1·2·3·4·5·6 6! = 3!(6 − 3)! (1 · 2 · 3)(1 · 2 · 3) 4·5·6 = 1·2·3 = 20  DEFINITION 1

Binomial Coefficient For n ≥ 0 and 0 ≤ r ≤ n, the binomial coefficient is defined by



n r

 =

n r



n! r!(n − r)!

We observed above that entries in Pascal’s triangle can be obtained from the sum of the two elements to the immediate left and right in the row above. The next identity is the equivalent statement about binomial coefficients. PROPOSITION 1

If k and r are natural numbers such that 0 ≤ r ≤ k, then       k−1 k−1 k + = r r −1 r Proof First observe that r! = r(r − 1)!

and

(k − r)! = (k − r)(k − r − 1)!

Expanding the binomial coefficients using factorials gives     (k − 1)! (k − 1)! k−1 k−1 = + + r r −1 (r − 1)![(k − 1) − (r − 1)]! r!(k − 1 − r)!  1 1 + = (k − 1)! (r − 1)!(k − r)! r!(k − r − 1)!   1 (k − 1)! 1 + = (r − 1)!(k − r − 1)! k − r r  (k − 1)! r + (k − r) = (r − 1)!(k − r − 1)! r(k − r)  (k − 1)! k = (r − 1)!(k − r − 1)! r(k − r) k! = r!(k − r)!   k = r

Confirming Pages

A.4 Mathematical Induction

THEOREM 10

437

Binomial Theorem If a and b are any numbers and n is a nonnegative integer, then       n n n an + a n−1 b + a n−2 b2 (a + b)n = 0 1 2       n n n n−r r n−1 a b + ···+ ab bn + ···+ + r n−1 n Proof The proof is by induction on the exponent n. If n = 1, then (a + b)n = a + b, and         1 n n 1 b =a+b a1 + b1 = a+ 1 0 1 0 Therefore, the statement holds for the case n = 1. Next assume that the statement         n n n n (a + b)n = an + a n−1 b + · · · + abn−1 + bn 0 1 n−1 n holds. For the next case, we consider (a + b)n+1 = (a + b)(a + b)n and apply the inductive hypothesis. This gives (a + b)n+1 = (a + b)(a + b)n         n n n n an + a n−1 b + · · · + abn−1 + bn = (a + b) 0 1 n−1 n         n n n n n n−1 n−1 a b + ···+ ab bn a + + =a 1 n−1 n 0         n n n n a n−1 b + · · · + abn−1 + bn an + +b 1 n−1 n 0         n n n n n+1 n 2 n−1 a b + ···+ a b abn a + + = 1 n−1 n 0         n n n n anb + a n−1 b2 + · · · + abn + bn+1 + 0 1 n−1 n Now, combine the terms with the same exponents on a and b to obtain           n n n n n anb + a n−1 b2 a n+1 + + + (a + b)n+1 = 1 2 0 0 1  + ··· +

n n−1



 +

n n



 abn +

n n

 bn+1

Confirming Pages

438

Appendix A

Preliminaries

Finally by repeated use of Proposition 1, we have       n+1 n+1 n+1 n+1 n+1 n a a b+ a n−1 b2 = + (a + b) 0 1 2     n+1 n+1 n ab + bn+1 + ···+ n n+1 Therefore, by induction the statement holds for all natural numbers.

Exercise Set A.4 In Exercises 1–10, use mathematical induction to show that the summation formula holds for all natural numbers. 1. 12 + 22 + 32 + · · · + n2 =

n(n+1)(2n+1) 6

2. 13 + 23 + 33 + · · · + n3 =

n2 (n+1)2 4

3. 1 + 4 + 7 + · · · + (3n − 2) =

n(3n−1) 2

4. 3 + 11 + 19 + · · · + (8n − 5) = 4n2 − n 5. 2 + 5 + 8 + · · · + (3n − 1) =

n(3n+1) 2

6. 3 + 7 + 11 + · · · + (4n − 1) = n(2n + 1) 7. 3 + 6 + 9 + · · · + 3n =

3n(n+1) 2

8. 1 · 2 + 2 · 3 + 3 · 4 + · · · + n(n + 1) = 9. 10.

n

k k=1 2

n

k=1 k

n(n+1)(n+2) 3

= 2n+1 − 2

· k! = (n + 1)! − 1

11. Find a formula for all natural numbers n for the sum 2 + 4 + 6 + 8 + · · · + 2n Verify your answer, using mathematical induction. 12. Find a formula for all natural numbers n for the sum n  (4k − 3) k=1

13. Show that for all natural numbers n ≥ 5, the inequality 2n > n2 holds. First show the inequality holds for n = 5, and then proceed to the second step when using mathematical induction. 14. Show that for all natural numbers n ≥ 3, the inequality n2 > 2n + 1 holds. First show that the inequality holds for n = 3, and then proceed to the second step when using mathematical induction. 15. Show that for all natural numbers n the expression n2 + n is divisible by 2. 16. Show that for all natural numbers n the expression x n − y n is divisible by x − y. Note that x 2 − y 2 is divisible by x − y since x 2 − y 2 = (x + y)(x − y). 17. Use mathematical induction to show that for a real number r and all natural numbers n, 1 + r + r 2 + r 3 + · · · + r n−1 =

rn − 1 r −1

18. Let fn denote the nth Fibonacci number. a. Determine the sum of the first n Fibonacci numbers for n = 2, 3, 4, and 5. That is, determine f1 + f2 , f1 + f2 + f3 , f1 + f2 + f3 + f4 , and f1 + f2 + f3 + f4 + f5 . b. Find a formula for the sum of the first n Fibonacci numbers. c. Show that the formula found in part (b) holds for all natural numbers.

Confirming Pages

439

A.4 Mathematical Induction

19. Let A, B1 , B2 , . . . be sets. Prove that for every natural number n,

21. Verify that if 0 ≤ r ≤ n, then 

A ∩ (B1 ∪ B2 ∪ · · · ∪ Bn ) = (A ∩ B1 ) ∪ · · · ∪ (A ∩ Bn ) 20. Show that for every natural number n, a 2n × 2n grid of squares with one square removed can be covered with copies of the shape

n r



 =

n n−r



22. Verify that 

n r −1

23. Show that as shown in the figure.



 +

n r



 =

n+1 r

 n   n = 2n k k=0

24. Show that n  k=0

 (−1)k

n k

 =0



Confirming Pages

Answers to Odd-Numbered Exercises



Chapter 1

c.

Section 1.1 1. x1 = 3, x2 = 8, x3 = −4 3. x1 = 2 − 3x4 , x2 = 1 − x4 , x3 = −1 − 2x4 , x4 ∈ ⺢ 5. x = 0, y = − 23 7. x = 1, y = 0
 2t+4   9. S = ,t  t ∈ ⺢

41. a. b. 43. a. b. c.

S S k k k

x 3x

+ +

y 3y

= =

2 −6

= {(3 − 2s − t , 2 + s − 2t , s, t ) | s, t ∈ ⺢} = {(7 − 2s − 5t , s, −2 + s + 2t , t ) | s, t ∈ ⺢} =3 = −3 = ±3

3

11. x = 0, y = 1, z = 0
   13. S = −1 − 5t , 6t + 12 , t  t ∈ ⺢
   15. S = −t + 23 , − 12 , t  t ∈ ⺢
   17. S = 3 − 53 t , −s − 43 t + 3, s, t  s, t ∈ ⺢ x = −2a + b, y = −3a + 2b x = 2a + 6b − c, y = a + 3b, z = −2a − 7b + c Consistent if a = −1 Consistent if b = −a Consistent for all a, b, and c such that c − a − b = 0 Inconsistent if a = 2 Inconsistent for a = 6    2 33. y = x − 32 − 2; vertex: 32 , −2 35. y = − (x − 2)2 + 3; vertex: (2, 3) 37. a. (2, 3) 19. 21. 23. 25. 27. 29. 31.

b.

y 5

⫺5

5

⫺5

 39. a.  b.

440

x x x 2x

+ − + +

= =

y y y 2y

2 0 = =

1 2

x

Section 1.2   5 2 −3 1. −1 1 −3   2 0 −1 4 1 2  3.  1 4 4 1 −1 1   2 0 −1 4 5. 1 4 1 2  2 4 2 2 7.  4 −2 −3 −2 1 3 3 −3

 −2 2  −4

x = −1, y = 12 , z = 0 x = −3 − 2z , y = 2 + z , z ∈ ⺢ x = −3 + 2y, z = 2, y ∈ ⺢ Inconsistent x = 3 + 2z − 5w , y = 2 + z − 2w , z ∈ ⺢, w ∈ ⺢ x = 1 + 3w , y = 7 + w , z = −1 − 2w , w ∈ ⺢ In reduced row echelon form Not in reduced row echelon form In reduced row echelon form Not in reduced row echelon form   1 0 29. 0 1   1 0 0 31.  0 1 0  0 0 1   1 0 −1 33. 0 1 0

9. 11. 13. 15. 17. 19. 21. 23. 25. 27.

Confirming Pages

441

Answers to Odd-Numbered Exercises 

 35. 37. 39. 41. 43.

 1 0 0 −2  0 1 0 −1  0 0 1 0 x = −1, y = 2 x = 1, y = 0, z = 13 Inconsistent x1 = − 12 − 2x3 , x2 = − 34 + 32 x3 , x3 ∈ ⺢

45. x1 = 1 − 12 x4 , x2 = 1 − 12 x4 , x3 = 1 − 12 x4 , x4 ∈ ⺢ 47. x1 x4 49. a. b. c. d. 51. a. b. c. d.

= 1 + 13 x3 + 12 x4 , x2 = 2 + 23 x3 + 32 x4 , x3 ∈ ⺢, ∈⺢ c−a +b =0 c − a + b = 0 Infinitely many solutions. a = 1, b = 0, c = 1; x = −2, y = 2, z = 1 a + 2b − c = 0 a + 2b − c = 0 Infinitely many solutions a = 0, b = 0, c = 0; x = 45 , y = 15 , z = 1

Section 1.3 1. A + B =



1 2

0 6 

3. (A + B) + C = 

5.

7. 9.

11.

13. 15.

17.

 =B +A 2 7

1 4

 = A + (B + C )

 −7 −3 9 5 6  (A − B) + C =  0 1 −2 10   −7 3 9 2A + B =  −3 10 6  2 2 11     7 −2 6 2 AB = ; BA = 0 −8 7 −7   −9 4 AB = −13 7   5 −6 4 6 −18  AB =  3 5 −7 6   1 3 A(B + C ) = 12 0   10 −18 2A(B − 3C ) = −24 0   7 5 3  2At − B t =  −1 −3 −2

 −4 1   −1 7 8  (At + B t )C =  6 4 12  0 20 15 t C )B =  0 0 0 (A −18 −22 −15   −5 −1 AB = AC = 5 1    1 0 1 A has the form , 0 1 0   −1 0 or 0 −1    1 1 −1 −1 A= ,B = 0 0 1 1

19. AB t =

21.

23.

25. 27.

29.

−7 −5

31. a = b = 4  1 33. A20 =  0 0

0 1 0

 

b −1

  −1 , 0

b 1

 ,



 0 0  1

35. If AB = BA, then A2 B = AAB = ABA = BAA = BA2 .   1  0    37. If x =  .. , then Ax = 0 implies the first column of  .  0   0  1    A has all 0 entries. Then let x =  ..  and so on, to  .  0 show that each column of A has all 0 entries. 39. The only matrix is the 2 × 2 zero matrix. 41. Since (AAt )t = (At )t At = AAt , the matrix AAt is symmetric. Similarly, (At A)t = At (At )t = At A. 43. If At = −A, then the diagonal entries satisfy aii = −aii and hence aii = 0 for each i . Section 1.4   1 −1 2 1. A−1 = 5 −3 1 3. The matrix is not invertible.   3 1 −2 −1 3  5. A =  −4 −1 −5 −1 3

Confirming Pages

442

Answers to Odd-Numbered Exercises

7. The matrix is not invertible.  1  1 −1 −2 3 2  0 1 2 −1    9. A−1 =  0 1  0 −1  2  0 0 0 − 12   3 0 0 0  −6 3 0 0   11. A−1 = 13   1 −2 −1 0  1 1 1 1 13. The matrix is not invertible.   0 0 −1 0  1 −1 −2 1   15. A−1 =   1 −2 −1 1  0 −1 −1 1   3 8 17. AB + A = = A(B + I ) 10 −10   2 9 AB + B = = (A + I )B 6 −3   −3 4 19. a. Since A2 = and −4 −3   −2 −4 −2A = , then A2 − 2A + 5I = 0. 4 −2   1 1 −2 1 −1 b. A = = (2I − A) 1 5 2 5 c. If A2 − 2A + 5I = 0, then A2 − 2A = −5I , so that   A 15 (2I − A) = 25 A − 15 A2 = − 51 (A2 − 2A) = − 51 (−5I ) = I . 21. If λ = −2, then the matrix is not invertible. 23. a. If λ = 1, then the matrix is invertible.   λ 1 λ − λ−1 − λ−1 λ−1   1 1 1 b.   − λ−1 λ−1 λ−1 0 0 1 25. The matrices  1 A= 0

29. a. If A is invertible and AB = 0, then A−1 (AB) = A−1 0, so that B = 0. b. If A is not invertible, then Ax = 0 has infinitely many solutions. Let x1 , . . . , xn be solutions of Ax = 0 and B be the matrix with nth column vector xn . Then AB = 0. 31. (AB)t = B t At = BA = AB 33. If AB = BA, then B −1 AB = A, so B −1 A = AB −1 . Now (AB −1 )t = (B −1 )t At = (B t )−1 At = B −1 A = AB −1 . 35. If At = A−1 and B t = B −1 , then (AB)t = B t At = B −1 A−1 = (AB)−1 . 37. a. (ABC )(C −1 B −1 A−1 ) = (AB)CC −1 (B −1 A−1 ) = ABB −1 A−1 b. Case 1, k = 2: (A1 A2 Case 2: Suppose that

0 0

 

are not invertible, but A + B = 27. (A + B)A−1 (A − B) = = = =

B=

and

1 0

0 1



= AA−1 = I −1 = A−1 2 A1

−1 −1 (A1 A2 · · · Ak )−1 = A−1 k Ak −1 · · · A1

Then (A1 A2 · · · Ak Ak +1 )−1 = ([A1 A2 · · · Ak ]Ak +1 )−1 −1 = A−1 k +1 [A1 A2 · · · Ak ] −1 −1 −1 = A−1 k +1 Ak Ak −1 · · · A1

39. If A is invertible, then the augmented matrix [A|I ] can be row-reduced to [I |A−1 ]. If A is upper triangular, then only terms on or above the main diagonal can be affected by the reduction process, and hence the inverse is upper triangular. Similarly, the inverse for an invertible lower triangle matrix is also lower triangular.     1 0 ax1 + bx3 ax2 + bx4 = 41. a. cx1 + dx3 cx2 + dx4 0 1 b. From part (a), we have the two linear systems   ax2 + bx4 = 0 ax1 + bx3 = 1 and cx1 + dx3 = 0 cx2 + dx4 = 1 so



)−1

0 0

0 1



is invertible.

(AA−1 + BA−1 )(A − B) (I + BA−1 )(A − B) A − B + B − BA−1 B A − BA−1 B

Similarly, (A − B)A−1 (A + B) = A − BA−1 B.

(ad − bc)x3 = d

and

(ad − bc)x4 = −b

If ad − bc = 0, then b = d = 0. c. From part (b), both b = 0 and d = 0. Notice that if in addition either a = 0 or c = 0, then the matrix is not invertible. Also from part(b), we have that ax1 = 1, ax2 = 0, cx1 = 0, and cx2 = 1. If a and c are not zero, then these equations are inconsistent and the matrix is not invertible. Section 1.5       2 3 x −1 1. A = ,x= , and b = −1 2 y 4

Confirming Pages

Answers to Odd-Numbered Exercises 

3.

5.

7.

9.

11.

13.

15.

17.

19.

21.

23.

25.

   2 −3 1 x 2 , x =  y , and A =  −1 −1 3 −2 −2 z   −1 b =  −1  3     x1 4 3 −2 −3  x2   1 0 , x =  A =  −3 −3  x3 , and 2 −3 4 −4 x4   −1 b= 4  3  2x − 5y = 3 2x + y = 2  − 2y = 3  2x − y − z = 1  3x − y + 2z = −1  2x1 + 5x2 − 5x3 + 3x4 = 2 x2 − 2x3 − 4x4 = 0 3x1 +   1 x= 4  −3   9  −3   x=  −8  7   1 −16 x= 9 10   −11 4  x= 12   0 1 0   x=  3 1  −1   1 7 a. x = 5 −3   1 −7 b. x = 8 5 The general solution is    −4t  S = t ∈ ⺢ t  with a particular nontrivial solution of x = −4 and y = 1.



1 27. A =  1 1

2 2 2

443

 1 1  1

29. From the fact that Au = Av, we have A(u − v) = 0. If A is invertible, then u − v = 0, that is, u = v, which contradicts the statement that u = v.   1 31. a. x = −1   1 1 −1 0 b. C = 2 0 3 1       1 1 1 −1 0  1 −2  = c. C b = 2 0 −1 3 1 −1 Section 1.6 1. The determinant is the product of the terms on the diagonal and equals 24. 3. The determinant is the product of the terms on the diagonal and equals −10. 5. Since the determinant is 2, the matrix is invertible. 7. Since the determinant is −6, the matrix is invertible. 9. a–c. det(A) = −5  −4 1 d. det  3 −1 2 0

 −2 4  = 5 1

e. Let B denote the matrix in part (d) and B  denote the new matrix. Then det(B  ) = −2 det(B) = −10. Then det(A) = 12 det(B  ).

11. 13. 15. 17. 19. 21. 23. 25. 27. 29.

f. Let B  denote the new matrix. The row operation does not change the determinant, so det(B  ) = det(B  ) = −10. g. Since det(A) = 0, the matrix A does have an inverse. Determinant: 13; invertible Determinant: −16; invertible Determinant: 0; not invertible Determinant: 30; invertible Determinant: −90; invertible Determinant: 0; not invertible Determinant: −32; invertible Determinant: 0; not invertible det(3A) = 33 det(A) = 270 1 1 1 = 3 = det((2A)−1 ) = det(2A) 2 det(A) 80

Confirming Pages

444

Answers to Odd-Numbered Exercises

31. Since the determinant of the matrix is −5x 2 + 10x = −5x (x − 2), the determinant is 0 if and only if x = 0 or x = 2. 33. y =

b2 −a2 b1 −a1 x

+



b.

y 5

b1 a2 −a1 b2 b1 −a1

−1 2 −2

1 35. a. A =  −1 2

 −2 3  −2

⫺5

b. det(A) = 2 c. Since the coefficient matrix is invertible, the linear system has a unique solution.   3 d. x =  8  −4   −1 0 −1 0 2  37. a. A =  2 1 −3 −3

x

5

⫺5

           

43. a.

 x y 1  x 2 xy y 2 1 0 0 −1 0 1  0 0 1 0 1 1  1 0 0 1 0 1  4 4 4 2 2 1  9 3 1 3 1 1  2 = −12 + 12x − 36xy + 42y 2 − 30y = 0

b.

y

b. det(A) = 0

5

c. Since the determinant of the coefficient matrix is 0, A is not invertible. Therefore, the linear system has either no solutions or infinitely many solutions.

⫺5

x

5

d. No solutions 39. a.

       

y2 4 4 9

x −2 3 4

y −2 2 −3

1 1 1 1

       

⫺5

= − 29y + 20x − 25y + 106 = 0 2

b.

45. x =

y 5

47. x =

⫺5

x

5

                 

         

x2 0 0 1 4

y2 16 16 4 9

x 0 0 1 2

y −4 4 −2 3

1 1 1 1 1

         

=136x 2 − 16y 2 − 328x + 256 = 0

−5 −3

5 2

−5 −3

49. x =

     

     

           

9 =− , y 5

3 −10

−4 5

−9 −7

−4 5

=−

⫺5

41. a.

     

7 6

111 73

4 3 −1 −8

     

−3 4 −3 4

160 51. x = − 103 ,y =

     

     

     

=

25 , 73

=−

10 103 , z

=

   5 7     2 6  16  =− =   5 −5  5    2 −3     −9 3    −7 −10   y=   −9 −4     −7 5 

  −1   −8

25 , y=   −1 28   −8

 4  3 

29  =− 28 −3  4 

42 103

53. Expansion of the determinant of A across row one equals the expansion down column one of At , so det(A) = det(At ).

Confirming Pages

445

Answers to Odd-Numbered Exercises Section 1.7



b. A = E1−1 E2−1 · · · E6−1   1 0 1 11. A = LU = −3 1 0



1 0 0 1. a. E =  2 1 0  0 0 1   1 2 1 4  b. EA =  5 5 1 1 −4 

1 3. a. E =  0 0 

0 1 −3

1 b. EA =  3 −8



1 13. A = LU =  2 −3

2 1 −2

1 15. A = LU =  1 −1

 1 2  −10

5. a. I = E3 E2 E1 A   1 1 −3 = 0 1 0

0



1 10

b. A = E1−1 E2−1 E3−1   1 0 1 = −2 1 0 7. a. I = E5 E4 E3 E2 E1 A  1 0 E1 =  −2 1 0 0  1 −2 1 E3 =  0 0 0

0 10 

0 0  1  0 0  1  1 E5 =  0 0



1 2

0 1

1 0

3 1 

E2 =  

 A



 0 0  1

 0 0  1 

1 0  1

− 21

 1 0 0  0 1 0

1 2

1 0

 −3 4  3

0 1 0 0 1 0



0 0  1  11 0  1

19. • LU factorization:   1 0 0 L =  −1 1 0  2 0 1



1 U = 0 0



 0 0  1  1 0 0 E4 =  0 1 1  0 0 1   1 0 0 0  E6 =  0 1 0 0 −1 1 E2 =  0 0 



−2 1 0

4 1 0

 −3 2  1

 x1 + 4x2 − 3x3  x2 + 2x3 • y = Ux =  x3   0 • Solve Ly =  −3 : y1 = 0, y2 = −3, y3 = 1 1

• Solve U x = y: x1 = 23, x2 = −5, x3 = 1 

1 1

• Solve U x = y: x1 = 2, x2 = 3

1 0 −1

1 E4 =  0 0  0 0 1 −5  0 1

0 1

 1 3  1

2 1 0

17. • LU factorization:    1 0 −2 L= U = −2 1 0   −2x1 + x2 • y = Ux = x2   −1 • Solve Ly = : y1 = −1, y2 = 3 5

b. A = E1−1 E2−1 E3−1 E4−1 E5−1 9. a. I = E6 · · · E1 A  0 1 E1 =  1 0 0 0  1 0 1 E3 =  0 0 −1  1 0 E5 =  0 1 0 0

 0 1 0  0 1 0

0 1 0



 0 0  1



−2 1

21. • LU factorization:



 L= 

1 1 2 −1



1  0  U = 0 0

0 1 0 −1

0 0 1 0

 0 0   0  1

−2 1 0 0

3 2 1 0

 1 2   1  1

Confirming Pages

446

Answers to Odd-Numbered Exercises 

 x1 − 2x2 + 3x3 + x4   x2 + 2x3 + 2x4  • y = Ux =    x3 + x4 x4   5  6   • Solve Ly =   14  : −8

Section 1.8 1. x1 = 2, x2 = 9, x3 = 3, x4 = 9 3. Let x5 = 3. Then x1 = x5 = 3, x2 = 13 x5 = 1, x3 = 1 3 x5 = 1, x4 = x5 = 3. 5. Let x1 , x2 , . . . , x7 be defined as in the figure.

y1 = 5, y2 = 1, y3 = 4, y4 = −2

  1 −3 2 0 0  0 1 − 54  0 0 1 − 51   1 0 1 4 25. A = LU = −3 1 0 1    1 −4 1 0 A−1 = U −1 L−1 = 0 1 3 1   −11 −4 = 3 1    1 0 0 2 1 −1 27. A = LU =  1 1 0   0 1 −1  1 1 1 0 0 3  1 0 0  2 0 0

0 5 1 

A−1 = U −1 L−1   1 − 21 0  1 2    = 1 13   0  −1 0 0 0 13   1 − 12 0  1  2 =  −1 3 3  0 29. Suppose



a b

0 c

− 13 

d 0

 0 0  1

0 1 −1

1 3

e f



 =

x2

800

• Solve U x = y: x1 = −25, x2 = −7, x3 = 6, x4 = −2 23. A =  PLU 0 1 = 1 0 0 0

0 1

300

x1

1 0



This gives the system of equations ad = 0, ae = 1, bd = 1, be + cf = 0. The first two equations are satisfied only when a = 0 and d = 0. But this is incompatible with the third equation. 31. If A is invertible, there are elementary matrices E1 , . . . , Ek such that I = Ek · · · E1 A. Similarly, there are elementary matrices D1 , . . . , D
such that I = D
· · · D1 B. Then A = Ek−1 · · · E1−1 D
· · · D1 B, so A is row equivalent to B.

x3

500

x6 x5

x4

700

300

x7

Then x1 = 1000 − x4 − x7 , x2 = 800 − x6 , x3 = 1000 − x4 + x6 − x7 , x5 = 300 + x6 − x7 Since the network consists of one-way streets, the individual flows are nonnegative. As a sample solution let x4 = 200, x6 = 300, x7 = 100; then x1 = 700, x2 = 500, x3 = 1000, x5 = 500. 7. x1 = 150 − x4 , x2 = 50 − x4 − x5 , x3 = 50 + x4 + x5 . As a sample solution let x4 = x5 = 20; then x1 = 130, x2 = 10, x3 = 90 9. x1 = 1.4, x2 = 3.2, x3 = 1.6, x4 = 6.2   0.02 0.04 0.05 11. a. A =  0.03 0.02 0.04  0.03 0.3 0.1 b. The internal demand vector is     300 22 A  150  =  20 . The total external demand 200 74 for the three sectors is 300 − 22 = 278, 150 − 20 = 130, and 200 − 74 = 126, respectively.   1.02 0.06 0.06 c. (I − A)−1 ≈  0.03 1.04 0.05  0.05 0.35 1.13 d.

X = (I − A)−1 D  1.02 0.06 =  0.03 1.04 0.05 0.35   418.2 =  454.9  832.3

  0.06 350 0.05   400  1.13 600

Confirming Pages

Answers to Odd-Numbered Exercises 13. a.

447

after 2 months by 

1400

   20, 000 24, 900 A  20, 000  =  13, 400  10, 000 11, 700

1200

2

1000 800 600

and after 1 year by     20, 000 30, 530 A12  20, 000  ≈  11, 120  10, 000 8, 350

400 200

  3, 880, 900a 3, 920, 400a b.  3, 960, 100a c. a = d.

27 20 , b

+ + +

2000

1995

1990

1985

1980

1975

1970

1965

0

1970b 1980b 1990b

+ + +

c c c

= = =

80 250 690

= − 10,631 2 , c = 5, 232, 400

1400

19. a. I1 + I3 = I2  4I1 + 3I2 b. 3I2   c.

1200



I1 4I1

− +

1000

I2 3I2 3I2

+

5I3

+

I3

+

5I3

= =

8 10

= = =

0 8 10

Solution: I1 ≈ 0.72, I2 ≈ 1.7, I3 ≈ 0.98

800 600 400 200 2000

1995

1990

1985

1980

1975

1970

1965

0

e. The model gives an estimate, in billions of dollars, for health care costs in 2010 at 10, 631 27 (2010)2 − (2010) + 5, 232, 400 = 2380 20 2   0.9 0.08 15. a. A = 0.1 0.92     1, 500, 000 1, 398, 000 b. A = 600, 000 702, 000     1, 500, 000 1, 314, 360 = c. A2 600, 000 785, 640   1, 500, 000 d. An 600, 000 17. The transition matrix is  0.9 A =  0.1 0

0.2 0.5 0.3

 0.1 0.3  0.6

so the numbers of people in each category after 1 month are given by     20, 000 23, 000 A  20, 000  =  15, 000  10, 000 12, 000

21. Denote the average temperatures of the four points by a, b, c, and d clockwise, starting with the upper left point. The resulting linear system is  − d = 50   4a − b  −a + 4b − c = 55 − b − d = 45    −a − c + 4d = 40 The solution is a ≈ 24.4, b ≈ 25.6, c ≈ 23.1, d ≈ 21.9. Review Exercises Chapter 1   1 1 2 1  −1 0 1 2   1. a. A =   2 2 0 1  1 1 2 3 b. det(A) = −8 c. Since the determinant of the coefficient matrix is not 0, the matrix is invertible and the linear system is consistent and has a unique solution. d. The only solution is the trivial solution. e. From part (b), since the determinant is not zero, the inverse exists.   −3 −8 −2 7 1 5 8 6 −9   A−1 =  0 −2 −1  8 5 −4 0 0 4

Confirming Pages

448

Answers to Odd-Numbered Exercises 

 3  1   f. x = A−1   −2  = 5



 11   1  −17  4  7  4

3. a = 0, c = 0, b = 0; a = 0, c = 1, b ∈ ⺢; a = 1, c = 0, b ∈ ⺢; a = 1, b = 0, c = 1 5. a. If

  a2 a1 b1 B= A= c1 d1 c2 then the sum of the diagonal entries is 

b2 d2



(a1 a2 + b1 c2 ) − (a1 a2 + b2 c1 ) + (b2 c1 + d1 d2 ) − (b1 c2 + d1 d2 ) = 0 b.  a c

b −a



a c

b −a



 =

a + bc 0 2

0 a 2 + bc

= (a 2 + bc)I c. Let M = AB − BA. By part (a), M 2 = kI for some k . Then (AB − BA)2 C = M 2 C = (kI )C = C (kI ) = CM 2 = C (AB − BA)2 7. a. Since det(A) = 1, then A is invertible. b. Six 1s can be added, making 21 the maximum number of entries that can be 1 and the matrix is invertible. 9. a. B t = (A + At )t = At + (At )t = At + A = B; C t = (A − At )t = At − (At )t = At − A = −C b. A = 12 (A + At ) + 12 (A − At )



23. T

24. T

25. T

26. T

27. F

28. T

29. F

30. T

31. T

32. T

33. F

34. T

35. F

36. T

37. T

38. F

39. T

40. T

41. T

42. F

43. T

44. F

45. T

Chapter 2 Section 2.1 1.

3.

5.

7.



 −1 u+v= 2 =v+u 3   11 u − 2v + 3w =  −7  0   1 −3(u + v) − w =  −7  −8   −17  −14     9  −6 

Chapter Test: Chapter 1 1. T

2. F

3. F

4. T

5. F

6. T

7. F

8. T

9. T

10. T

11. T

12. T

13. T

14. T

15. F

16. T

17. F

18. T

19. F

20. T

21. F

22. T

9.

 1  −2   (x1 + x2 )u = (x1 + x2 )   3  0   x1 + x2  −2x1 − 2x2   =  3x1 + 3x2  0    x1 x2  −2x1   −2x2   =  3x1  +  3x2 0 0 = x1 u + x2 v

   

Confirming Pages

449

Answers to Odd-Numbered Exercises 

 a  such that b 31. All vectors of the form  2a − 3b a, b ∈ ⺢.

11. v = 2e1 + 4e2 + e3 13. v = 3e2 − 2e3  7   15. w =   17.

2

 1  −1

c1 −2c1

+ −

= =

3c2 2c2

−2 −1

Solution: c1 = 74 , c2 = − 45     −2 1 The vector is a combination of and −1 −2   3 . −2  19.

c1 2c1

− −

c2 2c2

= =

87 121 , c2



= − 238 121 , c3 =

5. Yes



−2  3 4 7. Yes 

3 1

Solution: The linear system is inconsistent.   3 The vector cannot be written as a combination 1     1 −1 of and . 2 −2  − 5c3 = −3  −4c1 c3 = −3 4c1 + 3c2 + 21.  c2 − 5c3 = 4 3c1 − Solution: c1 = 

Section 2.2    1 −2 −4 −→ 1. 1 3 11    −2 3 1 3. −→ 4 −6 1

3 121

2  −2 0

9. No



1  2 −1 11. Yes 

2  −3   4 1

3 0 −3

−1 −1 3

1 6 −1 2

0 1

1

− 32

0

0

0

1

; yes  ; no

  1 −3 10  −→  0 10 0

0 1 0

 2 1  0

−2 0 −1

  1 2 8  −→  0 2 0

0 1 0

0 0 1

0 1 2

  −1 1 1  −→  0 5 0

0 1 0

0 1 0

 0 0  1

−1 −1 2 3

  3 1  −17   −→  0  0 17  7 0

0 1 0 0

0 0 1 0

 3 −1   2  0

1 4 2

−3 The vector  −3  is a combination of the three 4 vectors.   −c1 − c2 + c3 = −1 0 c2 − c3 = 23.  2 c1 + c2 − c3 = Solution: The linear system is inconsistent   −1 The vector  0  cannot be written as a 2

15. Infinitely many ways

combination of the other vectors. 25. All 2 × 2 vectors. Moreover, c1 = 13 a − 23 b, c2 = 13 a + 13 b   a 27. All vectors of the form such that a ∈ ⺢. −a 29. All 3 × 3 vectors. Moreover, c1 = 13 a − 23 b + 23 c,

19. No 

c2 = − 13 a + 23 b + 13 c, c3 = 13 a + 13 b − 13 c



2 3

1 0

−4 2 3

 

−4

13. Infinitely many ways c1 = 1 + 13 c3 , c2 = 1 + 73 c3 , c3 ∈ ⺢ c1 = 3 + 6c4 , c2 = −2 − c4 , c3 = 2 + 2c4 , c4 ∈ ⺢ 17. Yes 

1  2   1 −1

2  2   −1 3

−2 3 1 4

3 −1 2 −2

−1 3 2 1

3 −1 2 2

  1 −2  0 4   −→   0 4  0 0   1 2  1   −→  0  0 −1  2 0

0 1 0 0

0 1 0 0

0 0 1 0

0 0 1 0

 −1 −1   3  0  0 0   0  1

Confirming Pages

450

Answers to Odd-Numbered Exercises 

   1 3 − −2 1     −1 −2 +2 , 23. (AB)1 = 3 3 4     −1 −2 (AB)2 = 2 +5 3 4

11. Since

21. Ax = 2

3  3   2 1

25. Not possible. 27. x 3 − 2x + 1 = 12 (1 + x ) + 2(−x ) + 0(x 2 + 1)+ − x + 1)   a 29. All vectors  b  such that 3a − b + c = 0. c 1 3 2 (2x

31. v = 2v1 − v2 + 4v3 33. Since c1 = 0, v1 = − cc2 v2 − · · · − 1

cn c1 vn . c

35. Let v ∈ S1 . Since c = 0, then v = c1 v1 + · · · + ck (cvk ), so v ∈ S2 . If v ∈ S2 , then v = c1 v1 + · · · + (cck )vk , so v ∈ S1 . Therefore, S1 = S2 . 37. If A3 = cA1 , then det(A) = 0. Since the linear system is assumed to be consistent, it must have infinitely many solutions. Section 2.3   −1 1. Since  1

 2  = 1, the vectors are linearly −3 

independent.    1 −2   = 0, the vectors are linearly 3. Since  −4 8  dependent.     −1 2 −1 2 5. Since  2 2  −→  0 6 , the vectors are 1 3 0 0 linearly independent.    −4 −5 3   3 −5  = 0, the vectors are linearly 7. Since  4  −1 3 5  dependent. 

3  −1  9. Since  −1 2

1 0 2 1

  3 3  0 −1   −→   0 0  1 0

vectors are linearly independent.



1 1 3

0 0

 3 0  , the 1  0

0 1 0 0

  3 1  0 −1   −→   0 −1  −2 0

0 1 0 0

 1 −2   − 53  0

the matrices are linearly independent. 13. Since   1 0 −1 1  −2 −1 1 1     −2 2 −2 −1  −2 2 2 −2   1 0 −1 1  0 −1 −1 3   −→   0 0 −6 7  0 0 0 11 3 the matrices are linearly independent. 15. v2 = − 21 v1 17. Any set of vectors containing the zero vector is linearly dependent. 19. a. A2 = −2A1 b. A3 = A1 + A2 21. a = 6    1 1 1    23. a. Since  1 2 1  = 1, the vectors are linearly  1 3 2  independent. b. c1 = 0, c2 = −1, c3 = 3    1 2 0    25. Since  −1 0 3  = 13, the matrix is invertible so  2 1 2  27. 29. 31. 33. 35.

37.

Ax = b has a unique solution for every vector b. Linear independent Linearly dependent If x = 0, then c1 = 0, and if x = 12 , then c2 = 0. Let x = 0, then c3 = 0. Now letting x = 1 and x = −1, c1 = c2 = c3 = 0. If u and v are linearly dependent, then there are scalars a and b, not both 0, such that au + bv = 0. If a = 0, then u = −(b/a)v. On the other hand, if there is a scalar c such that u = cv, then u − cv = 0. Setting a linear combination of w1 , w2 , w3 to 0, we have 0 = c1 w1 + c2 w2 + c3 w3 = c1 v1 + (c1 + c2 + c3 )v2 + (−c2 + c3 )v3 if and only if c1 = 0, c1 + c2 + c3 = 0, and −c2 + c3 = 0 if and only if c1 = c2 = c3 = 0.

Confirming Pages

Answers to Odd-Numbered Exercises 39. Consider c1 v1 + c2 v2 + c3 v3 = 0, which is true if and only if c3 v3 = −c1 v1 − c2 v2 . If c3 = 0, then v3 would be a linear combination of v1 and v2 contradicting the hypothesis that it is not the case. Therefore, c3 = 0. Now since v1 and v2 are linearly independent c1 = c2 = 0. 41. Since A1 , A2 , . . . , An are linearly independent, if Ax = x1 A1 + · · · + xn An = 0 then x1 = x2 = · · · = xn = 0. Review Exercises Chapter 2    a b   = ad − bc = 0, the column vectors 1. Since  c d  are linearly independent. If ad − bc = 0, then the column vectors are linearly dependent.   2  a 0 1   3. The determinant  0 a 0  = a 3 − a = 0 if and  1 2 1  only if a = ±1, and a = 0. So the vectors are linearly independent if and only if a = ±1, and a = 0. 5. a. Since the vectors are not scalar multiples of each other, S is linearly independent. b. Since     1 1 1 1 a a  0 1 b → 0 1  b 2 1 c 0 0 −2a + b + c

c. d. e. 7. a.

the linear system is inconsistent for −2a + b+ c = 0. If a = 1, b = 1, c = 3, then the system is   1 inconsistent and v =  1  is not a linear 3 combination of the vectors.   a All vectors  b  such that −2a + b + c = 0 c Linearly independent All vectors in ⺢3     1 1 2 1 x  −1 0 1 2   y     Let A =   2 2 0 1 , x =  z , and 1 1 2 3 w   3  1   . b= −2  5

451

b. det(A) = −8 c. Yes, since the determinant of A is nonzero. d. Since the determinant of the coefficient matrix is nonzero, the matrix A is invertible, so the linear system has a unique solution. 17 7 e. x = 11 4 ,y = − 4 ,z = 4,w = 1         9. a. 1 3 2 b1 x1  2  + x2  −1  + x3  3  =  b2  b3 1 1 −1 b. Since det(A) = 19, the linear system has a unique solution equal to x = A−1 b. c. Yes d. Yes, since the determinant of A is nonzero, A−1 exists and the linear system has a unique solution. Chapter Test: Chapter 2 1. T

2. F

3. T

4. T

5. F

6. F

7. F

8. T

9. F

10. F

11. T

12. T

13. F

14. T

15. F

16. T

17. T

18. F

19. T

20. T

21. F

22. F

23. F

24. F

25. T

26. F

27. T

28. F

29. T

30. F

31. T

32. F

33. T

Chapter 3 Section 3.1 1. Since

and

  x1  y1  ⊕  z1    x2  y2  ⊕  z2 

  x2 y2  =  z2   x1 y1  =  z1

 x1 − x2 y1 − y2  z1 − z2  x2 − x1 y2 − y1  z2 − z1

Confirming Pages

452

Answers to Odd-Numbered Exercises

do not agree for all pairs of vectors, the operation ⊕ is not commutative, so V is not a vector space. 3. The operation ⊕ is not associative, so V is not a vector space. 7. Since     x x +c+d (c + d )  = y y does not equal     x x c +d  y y     x +c x +d = + y y   2x + c + d = 2y 



x , then V is not a vector space. y 9. Since the operation ⊕ is not commutative, V is not a vector space.   0 11. The zero vector is given by 0 = . Since this 0 vector is not in V , then V is not a vector space. 13. a. Since V is not closed under vector addition, V is not a vector space. b. Each of the 10 vector space axioms are satisfied with vector addition and scalar multiplication defined in this way. 15. Yes, V is a vector space. 17. No, V is not a vector space. Let A = I and B = −I . Then A + B is not invertible and hence not in V . 19. Yes, V is a vector space.   1 0 21. a. The additive identity is 0 = , and the 0 1 additive inverse of A is A−1 . b. If c = 0, then cA is not in V .   1 23. a. The additive identity is 0 =  2 . Let 3   1+a u =  2 − a . Then the additive inverse is 3 + 2a   1−a −u =  2 + a . 3 − 2a for all vectors

b. Each of the 10 vector space axioms is satisfied.       1 1 + 0t 1+t c. 0   2 − t  =  2 − 0t  =  2  3 3 + 2(0)t 3 + 2t 25. Each of the 10 vector space axioms is satisfied. 27. Each of the 10 vector space axioms is satisfied. 29. Since ( f + g)(0) = f (0) + g(0) = 1 + 1 = 2, then V is not closed under addition and hence is not a vector space. 31. a. The zero vector is given by f (x + 0) = x 3 and −f (x + t ) = f (x − t ). b. Each of the 10 vector space axioms is satisfied. Section 3.2 1. The set S is a subspace of ⺢2 .



3. The set S is not a subspace of ⺢2 . If u =     −1 1 v= , then u + v = ∈ / S. 3 2 5. The set S is not a subspace of ⺢2 .   0 c = 0, then cv = ∈ / S. 0 7. Since      y1 x1  x2  + c  y2  =  x3 y3

If u =



2 −1

0 −1

 and

 and

 x1 + cy1 x2 + cy2  x3 + cy3

and (x1 + cy1 ) + (x3 + cy3 ) = −2(c + 1) = 2 if and only if c = −2, so S is not a subspace of ⺢3 . 9. Since     s − 2t x − 2y  +c  s x t +s y +x   (s + cx ) − 2(t + cy)  s + cx = (t + cy) + (s + cx ) is in S, then S is a subspace. Yes, S is a subspace. No, S is not a subspace. Yes, S is a subspace. Yes, S is a subspace. No, S is not a subspace since x 3 − x 3 = 0, which is not a polynomial of degree 3. 21. Yes, S is a subspace. 11. 13. 15. 17. 19.

Confirming Pages

Answers to Odd-Numbered Exercises 23. No, S is not a subspace. 25. Since  1  1 0

−1 −1 1

  1 1 −1  →  0 1 0

−1 2 0

−1 1 0

−1 0 3

 1 1  −2

the vector v is in the span. 27. Since 

1  1   0 −1

0 1 2 1

1 −1 −4 −3

  −2 1  1  → 0  0 6  5 0

0 1 0 0

1 −2 0 0

 −2 3   0  0

the vector v is in the span. 29. Since c1 (1 + x ) + c2 (x 2 − 2) + c3 (3x ) = 2x 2 − 6x − 11 implies c1 = −7, c2 = 2, c3 = 13 , the polynomial is in the span.     a   31. span(S ) =  b  a + c = 0   c     a b  a, b ∈ ⺢ 33. span(S ) = 2a−b a+b  3

3

  ax 2 + bx + c  a − c = 0     a    37. a. span(S ) =  b  a, b ∈ ⺢  b−2a   35. span(S ) =




3

b. Yes, S is linearly independent. 39. a. span(S ) = ⺢3 b. Yes, S is linearly independent. 41. a. b. c. d.

span(S ) = ⺢3 No, S is linearly dependent. span(T ) = ⺢3 ; T is linearly dependent. span(H ) = ⺢3 ; H is linearly independent.

43. a. span(S ) = P2 b. No, S is linearly dependent. c. 2x 2 + 3x + 5 = 2(1) − (x − 3) + 2(x 2 + 2x ) d. T is linearly independent; span(T ) = P3 45. a–b Since       −s −1 0  s − 5t  = s  1  + t  −5  2s + 3t 2 3

453

    0   −1 then S = span  1 ,  −5  .   2 3 Therefore, S is a subspace. c. Yes, the vectors are linearly independent. d. S = ⺢3       1 1 1 47. Since A(x + cy) = +c = if and 2 2 2 only if c = 0, then S is not a subspace. 49. Let B1 , B2 ∈ S . Since A(B1 + cB2 ) = AB1 + cAB2 = B1 A + c(B2 A) = (B1 + cB2 )A then B1 + cB2 ∈ S and S is a subspace. Section 3.3 1. The set S has only two vectors, while dim(⺢3 ) = 3. 3. Since the third vector can be written as the sum of the first two, the set S is not linearly independent. 5. Since the third polynomial is a linear combination of the first two, the set S is not linearly independent. 7. The set S is a linearly independent set of two vectors in ⺢2 . 9. The set S is a linearly independent set of three vectors in ⺢3 . 11. The set S is a linearly independent set of four vectors in M2×2 . Since dim (M2×2 ) = 4, then S is a basis. 13. The set S is a linearly independent set of three vectors in ⺢3 and so is a basis. 15. The set S is linearly dependent and is therefore not a basis for ⺢4 . 17. The set S is a linearly independent set of three vectors in P2 so S is a basis.    1 2   19. A basis for S is B =  −1 ,  1  and   0 1 dim(S ) = 2. 21. A basis for S is       1 0 0 1 0 0 B= , , 0 0 1 0 0 1 and dim(S ) = 3. 
23. A basis for S is B = x , x 2 and dim(S ) = 2. 25. The set S is already a basis for ⺢3 since it is a linearly independent set of three vectors in ⺢3 .

Confirming Pages

454

Answers to Odd-Numbered Exercises

27. A basis for the span of S is given by     2 0 −1   B =  −3 ,  2 ,  −1  . Observe that   0 2 0 span(S ) = ⺢ 29. A basis for the span of S is given by     2 0 4   B =  −3 ,  2 ,  0  . Observe that   0 2 4 3.



11. [v]B1 B

13. [I ]B21 [v]B2 B

15. [I ]B21

0

span(S ) = ⺢3 . 31. A basis for ⺢3 containing S is     2 1 1   B =  −1 ,  0 ,  0    3 2 0 33. A basis for ⺢4 containing   1     −1   B=   2 ,     4

S is  3  1  ,  1  2

35. A basis for ⺢3 containing S is     1 1   −1 B =  1 ,  1 ,  0    3 1 0

        19.

21.

37. B = {eii | 1 ≤ i ≤ n} 43. dim(W ) = 2 Section 3.4   2 1. [v]B = −1   2 3. [v]B =  −1  3   5 5. [v]B =  2  −2   −1  2   7. [v]B =   −2  4    − 41 ; [v]B2 = 9. [v]B1 = 1 8

23.





0

 −1 = 1  [v]B2 = 0   0 0 1 B [I ]B21 =  1 0 0  0 1 0   5 B2 [v]B2 = [I ]B1 [v]B1 =  2  3     −a − b + c a  b  =  a +b a + 2b − c c B   0 1 0 B2 a. [I ]B1 =  1 0 0  0 0 1     1 2 B2  b. [v]B2 = [I ]B1 2  =  1  3 3   1 1 a. [I ]BS = 0 2        3 1 1 = = b. 4 4 B 2 B        6 4 4 = = 4 2 B 4 B c.

1 2 − 12

− 31

B [I ]B21 [v]B1

17.  1 0  0  ,  0 0  1 0 0

   1 1 =  2 ; [v]B2 =  1  −1 0   1 −1 = 1 1   −1 B = [I ]B21 [v]B1 = 5   3 2 1   =  −1 − 32 0 

5 8 8 8

 

Confirming Pages

Answers to Odd-Numbered Exercises 7. Since v1 can be written as   −c2 −c3 v2 + v1 = c1 c1

d.

 v3 + · · · +

−cn c1

455

vn

then V = span{v2 , v3 , . . . , vn } 9. a. The set B = {u, v} is a basis for ⺢2 since it is linearly independent. To see this, consider 

B

25. a. [I ]B21

−1 = 2 0

−1 2 −1

b. [2u1 − 2u2 + u3 ]B2 = Section 3.5 1. a. y1 = e 2x , y2 = e 3x   e 2x  b. W [y1 , y2 ](x ) =   2e 2x

au + bv = 0

 0 −1  1  B [I ]B21

   2 1  −3  =  −3  1 4

e 3x 3e 3x

c. y(x ) = C1 e 2x + C2 e 3x 3. a. y1 = e −2x , y2 =xe −2x  e −2x  b. W [y1 , y2 ](x ) =   −2e −2x

    = e 5x > 0 for all x . 

xe −2x e −2x − 2xe −2x

   = 

e −4x > 0 for all x . c. y(x ) = C1 e −2x + C2 xe −2x 5. y(x ) = e x + 2xe x 7. a. yc (x ) = C1 e 3x + C2 e x b. a = 1, b = 3, c = 4 9. y(x ) =

1 4

cos (8x )

Review Exercises Chapter 3 1. k = 69 3. a. Since S is closed under vector addition and scalar multiplication, S is a subspace of M2×2 . b. Yes, let a = 3, b = −2, c = 0.     1 1 −1 0 0 0 c. B = , , 0 1 1 0 1 −1   0 1 d. The matrix is not in S . 2 1 5. a. The set T is a basis since it is a linearly independent set of three vectors in the three-dimensional vector space V . b. The set W is not a basis for V since it is not linearly independent.

Now take the dot product of both sides with first u, then v, to show that a = b = 0.   α , then b. If [w]B = β    x v1     y v2  xv2 − yv1  = α=   u1 v1  u 1 v2 − v1 u2    u2 v2  and

    β=    

u1 u2 u1 u2

x y v1 v2

    yu1 − xu2  = .  u 1 v2 − v1 u2  

Chapter Test: Chapter 3 1. F

2. T

3. F

4. F

5. T

6. F

7. F

8. F

9. T

10. T

11. T

12. T

13. T

14. T

15. T

16. F

17. F

18. F

19. T

20. T

21. T

22. T

23. T

24. T

25. T

26. F

27. T

28. T

29. T

30. F

31. T

32. F

33. T

34. F

35. T

Confirming Pages

456

Answers to Odd-Numbered Exercises

Chapter 4 Section 4.1 1. T is linear. 3. T is not linear. 5. T is linear. 7. Since T (x + y) = T (x ) + T (y) for all real numbers x and y, T is not linear. 9. Since T (cu) = cT (u), T is not linear. 11. Since T (0) = 0, T is not linear. 13. T is linear. 15. Since T (cA) = c 2 T (A) = cT (A) for all scalars c, T is not linear.     2 −2 17. a. T (u) = ; T (v) = 3 −2 b. Yes c. Yes     0 0 19. a. T (u) = ; T (v) = 0 −1     −1 0 b. No. T (u + v) = ; T (u) + T (v) = −1 −1 c. No, by part (b).     1 5 21. T = −3 −9 23. T (−3 + x − x 2 ) = −1 − 4x + 4x 2     3 22 25. T = 7 −11 27. a. No. The polynomial 2x 2 − 3x + 2 cannot be written as a linear combination of x 2 , −3x , and −x 2 + 3x . b. Yes. T (3x 2 − 4x ) = 43 x 2 + 6x − 13 3   −1 0 29. a. A = 0 −1     −1 0 b. T (e1 ) = and T (e2 ) = . Observe 0 −1 that  these are the column vectors of A.   0 0 31. T  0  = , for all z ∈ ⺢ 0 z 33. a. The zero vector is the only vector in ⺢3 such that     x 0 T  y  = 0 z     1 7 b. T  −2  =  −6  2 −9

 cT1 (v) + T1 (w) cT2 (v) + T2 (w)     T1 (w) T1 (v) + =c T2 (v) T2 (w) 

35. T (cv + w) =

= cT (v) + T (w) 37. T (kA + C ) = (kA + C )B − B(kA + C ) = kAB − kBA + CB − BC = kT (A) ! 1 + T (C )   39. a. T (cf + g) = cf (x ) + g(x ) dx 0

!

1

=

!

1

cf (x ) dx +

0

!

=c

g(x ) dx 0

1

! f (x ) dx +

0

1

g(x ) dx 0

= cT ( f ) + T (g) b. − x + 3) = 19 6 41. Since neither v nor w is the zero vector, if either T (v) = 0 or T (w) = 0, then the conclusion holds. Now assume that T (v) and T (w) are linearly dependent and not zero; then there exist scalars a0 and b0 , not both 0, such that a0 T (v) + b0 T (w) = 0. Since v and w are linearly independent, then a0 v + b0 w = 0 and since T is linear, then T (a0 v + b0 w) = 0. 43. Let T (v) = 0 for all v in ⺢3 . T (2x 2

Section 4.2



1. Since T (v) =

0 0



, v is in N (T ).  −5 3. Since T (v) = , v is not in N (T ). 10 5. Since T (p(x )) = 2x , p(x ) is not in N (T ). 

7. Since T (p(x )) = −2x , p(x ) is not in N (T ).   −1 9. Since T  2  = v, v is in R(T ). 1 11. The vector v is not in R(T ). 13. The matrix A is in R(T ) with a = 1, b = 0, c = −2, d = −1. 15. The matrix A is not in R(T ).   0 17. 0    −2  19.  1    1

Confirming Pages

Answers to Odd-Numbered Exercises

21. 23. 25.

27. 29. 31.

   1   2  1 ,  0    0 1 
x, x2     1 2   1  0 ,  1 ,  −1    2 0 1    0   1  0 ,  1    0 0 
1, x , x 2   −6 a. No,  5  is not in R(T ). 0     0   −2 b.  1 ,  1    1 −1

c. Since dim(N (T )) + dim(R(T )) = dim(⺢3 ) = 3 and dim(R(T )) = 2, then dim(N (T )) = 1. 33. a. The polynomial 2x 2 − 4x + 6 is not in R(T ).

 b. −2x + 1, x 2 + x = T (x ), T (x 2 )     x x 35. T  y  = y z 37. a. The range R(T ) is the subspace of Pn consisting of all polynomials of degree n − 1 or less. b. dim(R(T )) = n c. dim(N (T )) = 1 39. a. dim(R(T )) = 2 b. dim(N (T )) = 1    1 0 0 0 41. , 0 0 0 1 43. a. The range of T is the set of symmetric matrices. b. The null space of T is the set of skew-symmetric matrices. 45. If the matrix A is invertible, then R(T ) = Mn×n . Section 4.3 1. T is one-to-one. 3. T is one-to-one. 5. T is one-to-one. 7. T is onto ⺢2 . 9. T is onto ⺢3 .

11. 13. 15. 17. 19. 21.

Is Is Is Is Is a.

457

a basis a basis a basis a basis  a basis Since det(A) = det

 1 0 = −3 = 0, −2 −3 then T is an isomorphism.   −3 0 b. A−1 = − 13 2 1   x c. A−1 T y    1 0 x = −2x − 3y − 23 − 31   x = y 23. a. Since   −2 0 1 det(A) = det  1 −1 −1  0 1 0 = −1 = 0 then T is an isomorphism.   −1 −1 −1 −1 =  0 0 1  b. A −1 −2 −2   x −1  y  c. A T z    −1 −1 −1 −2x + z 0 1  x − y − z  = 0 −1 −2 −2 y   x = y  z 25. T is an isomorphism. 27. T is an isomorphism. 29. Since T (cA + B) = (cA + B)t = cAt + B t = cT (A) +T (B), T is linear. Since T (A) = 0 implies that A = 0, T is one-to-one. If B is a matrix in Mn×n and A = B t , then T (A) = T (B t ) = (B t )t = B, so T is onto. Hence, T is an isomorphism. 31. Since T (kB + C ) = A(kB + C )A−1 = kABA−1 + ACA−1 = kT (B) + T (C ), T is linear. Since T (B) = ABA−1 = 0 implies that B = 0, T is one-to-one. If C is

Confirming Pages

458

Answers to Odd-Numbered Exercises

a matrix in Mn×n and B = A−1 CA, then T (B) = T (A−1 CA) = A(A−1 CA)A−1 = C , so T is onto. Hence, T is an isomorphism.   33. a  b  3 2   T  c  = ax + bx + cx + d d   V =  

35. Since

 T

b.

 T

define T : V → ⺢ by 

7. a. 

x  = y x + 2y



x y



 b. T

Now, let T : ⺢3 −→ ⺢3 be an isomorphism. Since T is linear, T (t v) = tT (v). Also, by Theorem 8, T (v) is nonzero. Hence, the set

 T

L = {tT (v)| t ∈ ⺢} is also a line in ⺢3 through the origin. The proof for a plane is similar with the plane being given by P = {su + t v| s, t ∈ ⺢} for two linearly independent vectors u and v in ⺢3 . 

5 −1

 b. T  T

2 1 2 1



 =



 =



−1 3. a. [T ]B =  0 1

−1 1 9 −1 5 −1 1 3 0



 ; −1 1



2 1



 =

9 −1

 2 1  −1

     1 1 3 b. T  −2  =  −3  = [T ]B  −2  3 3 −2 



=

−1 −3



 −3 −2 3 6    −3 = ; −3     −3 −2 −1 = 3 6 −2 B      2 −3 −2 −3 = = 3 3 6 −3 −2 − 32

2 3 − 53

13 6



−2 −4

=



 ;

         −1 2 −1 = [T ]BB = [T ]BB T −3 B 1 −3 B  2  −3 = 8

L = {t v| t ∈ ⺢}

1. a. [T ]B =

−1 −2

  [T ]BB

37. Let v be a nonzero vector in ⺢3 . Then a line L through the origin can be given by

Section 4.4

−1 −2

   x    x , y ∈ ⺢ y   x + 2y 

2

T 





5. a. [T ]BB =

−1 −3

B

9. a. [T ]B

 =− 

1 = 0 0

2 3



1 −1 0

3

3 −2

 +

 1 −2  1

8 3



0 −2



 =

−2 −4



b. T (x 2 − 3x + 3) = x 2 − 3x + 3;    T (x 2 − 3x + 3) B  = [T ]BB [x 2 − 3x + 3]B     1 3 B  1  =  −3  = [T ]B 1 1 T (x 2 − 3x + 3) = 3 − 3x + x 2     a b 0 −2b 11. If A = , then T (A) = . c −a 2c 0   0 0 0 a. [T ]B =  0 −2 0  0 0 2     2 1 0 −2 b. T = ; 3 −2 6 0         2 0 2 1 = [T ]B  1  =  −2  T 3 −2 B 3 6

Confirming Pages

Answers to Odd-Numbered Exercises  T

2 3

1 −2



 =0

1 0 

+6  = 



0 6

0 −1 0 1 −2 0

0 0 



 −2



0 0

1 0



1 2 1 −1   1 1 22 [T ]B  = 9 11 −1   1 5 −2  [T ]BB = 5 9 1   1 5 2 [T ]BB  = 3 −1 5   1 −2 5  [T ]BC = 5 1 9   1  22 1 [T ]BC  = 9 −1 11   0 0  [T ]BB =  1 0  0 12   0 0 B [T ]C =  0 1  1 0 2   0 1 C [T ]C =  0 0  1 0 2   0 1 0 [S ]BB  = 0 0 2    1 0 [S ]BB  [T ]BB = 0 1   0 0 0  [T ]BB [S ]BB  =  0 1 0  0 0 1



21. [T ]BB = [1 0 0 1] 23. a. [2T + S ]B = 2[T ]B + [S ]B =  b. 25. a. [S

13. a. [T ]B = b. c. d. e. f.

15. a.

b.

c.

d. e.

f. The function S ◦ T is the identity map; that is, (S ◦ T )(ax + b) = ax + b so S reverses the action of T .   1 0 17. [T ]B = 0 −1 The transformation T reflects a vector across the x -axis. 19. [T ]B = cI



 b.

−4 23 ◦

 

T ]B = [S ]B [T ]B =

−1 10



2 1

5 −1

1 4





 3 3 1 27. a. [−3T + 2S ]B =  2 −6 −6  3 −3 −1   3 b.  −26  −9   4 −4 −4 29. a. [S ◦ T ]B =  1 −1 −1  −1 1 1   −20 b.  −5  5   0 0 0 6 0  0 0 0 0 24    0  31. [T ]B =   0 0 0 0   0 0 0 0 0  0 0 0 0 0   −12  −48    0  [T (p(x )]B =     0  0 T (p(x )) = p  (x ) = −12 − 48x   0 0 0   1 0 0   33. [S ]BB =   0 1 0  0 0 1   0 1 0 0  [D]BB =  0 0 2 0  0 0 0 3   1 0 0   [D]BB [S ]BB =  0 2 0  = [T ]B 0 0 3

2 7



459

Confirming Pages

460

Answers to Odd-Numbered Exercises 

a c

35. If A =



b d

T is



, then the matrix representation for 

−c a −d 0 c

0  −b [T ]S =   c 0       37. [T ]B =     

1 0 0 .. . 0 0 0

1 1 0 .. . 0 0 0

0 1 1 .. . 0 0 0

0 0 1 .. . 0 0 0

... ... ... .. . 0 0 0

Section 4.5  1. 1 [T ]B1 [v]B1 = −1  2 [T ]B2 [v]B2 = −1

 0 b   −c  0

b 0 d −a −b ... ... ... .. . ... ... ...

0 0 0 .. . 1 1 0

0 0 0 .. . 0 1 1

 [T ]B1 [v]B1 =

          

    2 4 2 = 3 −1 −7     1 −1 −7 = 2 −5 9

 [T ]B2 [v]B2 =

1 1

1 1

2 0

0 0

 

3 −2 1 2 − 52

 

 =  =

1 0

 

To show the results are the same, observe that       1 −1 1 1 +0 = 1 1 1 

5. a. [T ]B1

[T ]B2

1 = 0 0  1 = 0 0

0 0 0 −1 0 1 

b. [T ]B1 [v]B1

1 = 0 0

 0 0  1

[T ]B2 = P −1 [T ]B1 P  



B

    0 1 1 0  2  =  0  1 −1 −1

3 2





1 2

1 0

[T ]B2 = P −1 [T ]B1 P   2 −1 2 = −3 2 0   −1 −2 = 6 6

P = [I ]B12



3 2

− 12   0 3 = 1 0 3   2 1 B 11. P = [I ]B12 = 3 2

13. [T ]B1 =

0 0  1 0 0 0

[T ]B2 = P −1 [T ]B1 P   1 1 1 1 = 3 2 1 3   9 1 −2 2  = 3 23 − 2 2  1  1 B1 3 9. P = [I ]B2 = 1 −1 3

=

1 1

    0 3 1 0  2  =  0  1 −4 −2

To show that the results are the same, observe that         1 −1 0 1 1  0  + 0  1  + (−2)  0  =  0  1 0 1 −1   3 −1 B 7. P = [I ]B12 = −1 1

To show the results are the same, observe that       1 −1 2 −7 + (−9) = . 1 0 −7     1 1 2 0 ; [T ]B2 = 3. a. [T ]B1 = 1 1 0 0 b.

[T ]B2 [v]B2

−1 0 1

1 = 0 0

1 2



0 −1

0 3

3 −1







−1 1

1 3 1 3

−1

2 3

1 2

1





 1 −1 −2 1   −1 −1 = 2 1

[T ]B2 = P −1 [T ]B1 P   1 1 1 = −2 −1 −2   1 1 = 2 1

−1 1



−1 2

−1 1



Confirming Pages

Answers to Odd-Numbered Exercises 

15. [T ]B1

[T ]B2

0 = 0 0  0 = 0 0

1 0 0 2 0 0

 0 2  0  0 1  0

c. [T ]−1 S =

B

P = [I ]B12

1 3

5. a. [T ]S = b.

1 = 0 0

0 2 0

tr(B) = tr(P −1 AP) = tr(APP −1 ) = tr(A)

 √  √2/2 − 2/2

y

 −2 0  1

then [T ]B2 = P −1 [T ]B1 P. 17. Since A and B are similar, there is an invertible matrix P such that B = P −1 AP. Also since B and C are similar, there is an invertible matrix Q such that C = Q −1 BQ. Therefore, C = Q −1 P −1 APQ = (PQ)−1 A(PQ) so that A and C are also similar. 19. For any square matrices A and B the trace function satisfies the property tr(AB) = tr(BA). Now, since A and B are similar matrices, there exists an invertible matrix P such that B = P −1 AP. Hence,

0 −2

0 √ −√2/2 − 2/2





If



5

⫺5

5

x

⫺5

c.

[T ]−1 S  √

√ −√2/2 2/2

 =

3/2 7. a.  1/2 0

−1/2 √ 3/2 0

b.

√  −√2/2 − 2/2

√  √3/2 − 1/2 3/2 + 1/2  1

y 5

21. Since A and B are similar matrices, there exists an invertible matrix P such that B = P −1 AP. Hence, B n = (P −1 AP)n = P −1 An P ⫺5

Thus, An and B n are similar. Section 4.6   1 0 1. a. 0 −1   −1 0 b. 0 1   1 0 c. 0 3   3 0 3. a. [T ]S = 0 − 21 b.

c.

9. a.

y

b. 10

⫺10

x

⫺5

 √

10

⫺10

5

x

c.

 3/2 √1/2 −1  −1/2 3/2 −1  0 0 1     0 0 = 0 0 B     2 2 = 0 2 B     1 0 = 1 2 B   1 1 [T ]SB = 1 −1      1 1 0 0 = 1 −1 0 0      1 1 2 2 = 1 −1 0 2

461

Confirming Pages

462

Answers to Odd-Numbered Exercises 

1 1

1 −1



1 1



 =

2 0





y 5

⫺5

x

5

⫺5

The original y = x.   0 1 d. 1 0   0 1 1 0   0 1 1 0

triangle is reflected across the line 0 0 2 2 0 2



 =



 =



 =

0 0 2 2 2 0

 2 −4   −5  4  −1 2     5 −4 x  = A i. 7 −5 y C −2 4  −1 2  5 −4 =  7 −5 −2 4  x −y  −x + 3y  = −x + 4y 2x − 2y     x This implies that A =  y

−1  5 C  h. [T ]B =  7 −2

  



Review Exercises Chapter 4 1. a. The vectors are not scalar multiples, so S is a basis.   x    x x +y   b. T =  y x −y  2y

b. [S ]B = c. [T

c. N (T ) = {0} d. Since N (T ) = {0}, T is one-to-one.    1 0        1  ,  1  e.   1   −1       2 2 f. No, T is not onto since dim(R(T )) = 2 and   a  b   dim(⺢4 ) = 4. Also   c  is in R(T ) if and only d if c + b − 2a = 0.    −1 1      0  ,  1 ,  g.     0  1    1 1

 0 1  1 0  ,  0  0 0 0

       

x y

3. a. S

5. a. b.

7. a.

b.

c.

◦

 

 =

x −y

1 0

0 −1

S ]B =



−1 0

       





x y

1 3x 2 3x

 B

+ 13 y



− 13 y

   

 x x +y  . x −y  2y      x −x ;T = y y    −1 0 ; [T ]B = 0 1  0 = [S ◦ T ]B −1

The linear operators S ◦ T and T ◦ S reflect a vector through the origin.   0 1 [T ]B = 1 0    1 0 [T ]BB = 0 1   1 0 0 [T ]B =  0 0 1  0 1 0        −1 −1 −1 T  2  =  1  = T  2  1 2 1 B    0  N (T ) =  0    0

d. R(T ) = ⺢3

Confirming Pages

463

Answers to Odd-Numbered Exercises 

1 e. [T n ]B =  0 0

n 0 1  0

0 0 1

9. Since T 2 − T + I = 0, T − T 2 = I . Then (T ◦ (I − T ))(v) = T ((I − T )(v)) = T (v − T (v)) = T (v) − T 2 (v) = I (v) = v Chapter Test: Chapter 4 1. F

2. F

3. T

4. T

5. T

6. F

7. F

8. F

9. T

10. F

11. T

12. F

13. T

14. T

15. T

16. T

17. T

18. T

19. T

20. T

21. F

22. F

23. T

24. F

25. T

26. F

27. F

28. T

29. F

30. T

31. F

32. F

33. T

34. F

35. F

36. T

37. T

38. T

39. F

40. T

Chapter 5 Section 5.1 1. λ = 3 3. λ = 0 5. λ = 1 7. a. λ2 + 5λ = 0 b. λ1 = 0, λ2 = −5     1 −2 , v2 = c. v1 = 1 3



d. 

−2 3

−2 3 2 −3

2 −3 



−2 3

1 1 









 1 ; 1     10 −2 = = −5 −15 3 =

0 0

=0

9. a. (λ − 1)2 = 0 b. λ1 = 1   1 c. v1 = 0        d. 1 −2 1 1 1 = = 1 0 1 0 0 0 11. a. (λ + 1)2 (λ − 1) = 0 b. λ1 = −1, λ2 = 1     1 1 c. v1 =  0 , v2 =  2  0 2     d. −1 0 1 1 −1  0 1 0  0  =  0 0 2 −1 0 0     −1 0 1 1 1  0 1 0  2  =  2 0 2 −1 2 2





 1  = −1  0  0    1  = 1 2  2

13. a. (λ − 2)(λ − 1)2 = 0 b. λ1 = 2, λ2 = 1     1 −3 c. v1 =  0 , v2 =  1  0 1        d. 2 1 2 1 2 1  0 2 −1   0  =  0  = 2  0  0 1 0 0 0 0        2 1 2 −3 −3 −3  0 2 −1   1  =  1  = 1  1  0 1 0 1 1 1 15. a. (λ + 1)(λ − 2)(λ + 2)(λ − 4) = 0 b. λ1 = −1, λ2 = 2, λ3 = −2, λ4 = 4      1 0  0   1       c. v1 =   0 , v2 =  0 , v3 =  0 0   0  0   v4 =   0  1

 0 0  , 1  0

Confirming Pages

464

Answers to Odd-Numbered Exercises d.



−1  0   0 0

0 2 0 0 

0 0 −2 0 

 0 1  0 0   0  0 4 0





 −1   0  =    0  0

1  0    = −1  0  0

The other cases are similar.   a b 17. Let A = . The characteristic equation is c d (a − λ)(d − λ) − bc = 0, which simplifies to λ2 − (a + d )λ + (ad − bc) = 0. Observe that the coefficient of λ is −(a + d ), which is equal to −tr(A). Also, the constant term ad − bc is equal to det(A). 19. Suppose A is not invertible. Then the homogeneous equation Ax = 0 has a nontrivial solution x0 . Observe that x0 is an eigenvector of A corresponding to the eigenvalue λ = 0 since Ax0 = 0 = 0x0 . On the other hand, suppose that λ = 0 is an eigenvalue of A. Then there exists a nonzero vector x0 such that Ax0 = 0, so A is not invertible. 21. Let A be such that A2 = A, and let λ be an eigenvalue of A with corresponding eigenvector v so that Av = λv. Then A2 v = λAv, so Av = λ2 v. The two equations Av = λv

and

Av = λ2 v

imply that λ2 v = λv, so that (λ2 − λ)v = 0. Since v = 0, then λ(λ − 1) = 0, so that either λ = 0 or λ = 1. 23. Let A be such that An = 0 for some n, and let λ be an eigenvalue of A with corresponding eigenvector v, so that Av = λv. Then A2 v = λAv = λ2 v. Continuing in this way, we see that An v = λn v. Since An = 0, then λn v = 0. Since v = 0, then λn = 0, so that λ = 0. 25. If A is invertible, then det(AB − λI ) = det(A−1 (AB − λI )A) = det(BA − λI ) 27. Since det(A − λI ) = (λ − a11 )(λ − a22 ) · · · (λ − ann ) the eigenvalues are the diagonal entries. 29. Let λ be an eigenvalue of C with corresponding eigenvector v. Let C = B −1 AB. Since C v = λv, then B −1 ABv = λv. Then A(Bv) = λ(Bv). Therefore, Bv is an eigenvector of A corresponding to λ.



x y







x −y

=

. The eigenvalues are λ = 1   1 and λ = −1 with corresponding eigenvectors 0   0 and , respectively. 1

31. Let T

33. If θ = 0 or θ = π, then T can only be described as a   x cannot be expressed by scalar rotation. Hence, T y multiplication as this only performs a contraction or a dilation. When θ = 0, then T is the identity map     x x T = . In this case every vector in ⺢2 is an y y eigenvector with corresponding eigenvalue equal to 1.      x −1 0 x Also, if θ = π, then T = . y 0 −1 y In this case every vector in ⺢2 is an eigenvector with eigenvalue equal to −1.   1 1 −2 0 2   1 35. a. [T ]B =  − 21 0  2 −1

−1

1 =  −1 −1

1 −1 0



b. [T ]B 

1

 0 0  1

c. The characteristic polynomial for the matrices in parts (a) and (b) is given by p(x ) = x 3 − x 2 . Hence, the eigenvalues are the same. Section 5.2



1 0

0 −3

0 = 0 0

0 −2 0

1. P −1 AP = 

3. P

−1 AP

  0 0  1

5. Eigenvalues: −2, −1; A is diagonalizable since there are two distinct eigenvalues. 7. Eigenvalues: −1 with multiplicity 2; eigenvectors:   1 ; A is not diagonalizable. 0 9. Eigenvalues: 1,0; A is diagonalizable since there are two distinct eigenvalues. 11. Eigenvalues: 3, 4, 0; A is diagonalizable since there are three distinct eigenvalues.

Confirming Pages

Answers to Odd-Numbered Exercises 13. Eigenvalues: −1 and 2 with multiplicity 2;    1 −1 eigenvectors:  −5 ,  −1 ; A is not 2 1 diagonalizable since there are only two linearly independent eigenvectors. 15. Eigenvalues: 1 and 0 with multiplicity 2; eigenvectors:     −1 0 0  1 ,  1 ,  0 ; A is diagonalizable since 1 0 1 there are three linearly independent eigenvectors. 17. Eigenvalues: −1, 2, 0 with multiplicity 2;      0 0 0 −1  −1   1   −1   0      eigenvectors:   1 ,  2 ,  0 ,  1 ; 0 3 1 0

19.

21.

23.

25.

A is diagonalizable since there are four linearly independent eigenvectors.     2 0 −3 0 −1 P= ; P AP = 0 −1 1 1   0 2 0 P = 1 1 1  1 3 2   −1 0 0 P −1 AP =  0 1 0  0 0 0   2 0 0 P = 1 1 0  0 0 1   −1 0 0 P −1 AP =  0 1 0  0 0 1   −1 0 −1 1  0 1 0 0   P =  0 0 1 1  1 0 0 0   1 0 0 0  0 1 0 0   P −1 AP =   0 0 0 0  0 0 0 2

27. By induction. If k = 1, then Ak = A = PDP −1 = PD k P −1 . Suppose the result holds for a natural number k . Then

465

Ak +1 = (PDP −1 )k +1 = (PDP −1 )k (PDP −1 ) = (PD k P −1 )(PDP −1 ) = (PD k )(P −1 P)(DP −1 ) = PD k +1 P −1   1 0 1 0 29. P =  1 −2 2 ; D =  0 1 1 0 0  3 −1 −2 0 −2 Ak = PD k P −1 =  2 2 −1 −1 

0 1 0 

 0 0 ; 1



31. Since A is diagonalizable, there is an invertible P and diagonal D such that A = PDP −1 . Since B is similar to A, there is an invertible Q such that B = Q −1 AQ. Then D = P −1 QBQ −1 P = (Q −1 P)−1 B(Q −1 P) 33. If A is diagonalizable with an eigenvalue of multiplicity n, then A = P(λI )P −1 = (λI )PP −1 = λI . On the other hand, if A = λI , then A is a diagonal matrix.   0 1 0 35. a. [T ]B1 =  0 0 2  0 0 0   1 1 2 b. [T ]B2 =  −1 −1 0  0 0 0 c. The only eigenvalue of A and B is λ = 0, of multiplicity 3. d. The only eigenvector corresponding to λ = 0 is   −1  1 , so T is not diagonalizable. 0 37. If B is the standard basis for ⺢3 , then   2 2 2 2 1  [T ]B =  −1 1 −1 0 The eigenvalues are λ1 = 1, multiplicity 2, and λ2 = 2   0 with corresponding eigenvectors  −1  and 1   1  −1 , respectively. Since there are only two 1 linearly independent eigenvectors, T is not diagonalizable.

Confirming Pages

466

Answers to Odd-Numbered Exercises

39. Since A and B are matrix representations for the same linear operator, they are similar. Let A = Q −1 BQ. The matrix A is diagonalizable if and only if D = P −1 AP for some invertible matrix P and diagonal matrix D. Then D = P −1 (Q −1 BQ)P = (QP)−1 B(QP) so B is diagonalizable. The proof of the converse is identical. Section 5.3 y1 (t ) = [y1 (0) + y2 (0)]e −t − y2 (0)e −2t 1. y2 (t ) = y2 (0)e −2t 1 [y1 (0) − y2 (0)]e 4t 2 1 + [y1 (0) + y2 (0)]e −2t 2 1 y2 (t ) = [−y1 (0) + y2 (0)]e 4t 2 1 + [y1 (0) + y2 (0)]e −2t 2

y1 (t ) =

3.

y1 (t ) = [2y1 (0) + y2 (0) + y3 (0)]e −t

5.

+ [−y1 (0) − y2 (0) − y3 (0)]e 2t y2 (t ) = [−2y1 (0) − y2 (0) − 2y3 (0)]e t + 2[y1 (0) + y2 (0) + y3 (0)]e 2t y3 (t ) = [−2y1 (0) − y2 (0) − y3 (0)]e −t + [2y1 (0) + y2 (0) + 2y3 (0)]e t

1 60 y1

−

1 120 y2

y1 (0) = 12, y2 (0) = 0 1

1

b. y1 (t ) = 4 + 8e − 40 t , y2 (t ) = 8 − 8e − 40 t c. limt→∞ y1 (t ) = 4, limt→∞ y2 (t ) = 8 The 12 lb of salt will be evenly distributed in a ratio of 1:2 between the two tanks. Section 5.4



 0.85 0.08 0.15 0.92     0.7 0.37 10 ≈ b. T 0.3 0.63   0.35 c. 0.65

1. a. T =

 0.5 0.4 0.1 3. T =  0.4 0.4 0.2  0.1 0.2 0.7     0 0.36 3 1  ≈  0.35  T 0 0.29     0 0.33 T 10  1  ≈  0.33  0 0.33   0.5 0 0 5. T =  0.5 0.75 0  0 0.25 1



 0 The steady-state probability vector is  0 , and 1

hence the disease will not be eradicated.   0.33 0.25 0.17 0.25  0.25 0.33 0.25 0.17   7. a. T =   0.17 0.25 0.33 0.25  0.25 0.17 0.25 0.33     1 0.5(0.16)n + 0.25   0   0.25    b. T   0  =  −0.5(0.16)n + 0.25  0.25 0   0.25  0.25    c.  0.25  0.25 9. Eigenvalues of T : λ1 = −q + p + 1, λ2 = 1, with     −1 q/p corresponding eigenvectors and . 1 1

7. y1 (t ) = e −t , y2 (t ) = −e −t 1 1 9. a. y1 (t ) = − 60 y1 + 120 y2 , y2 (t ) =



The steady-state probability vector is    q  1 q/p p+q . = p 1 1 + q/p p+q Review Exercises Chapter 5      a b 1 a +b 1. a. = b a 1 a +b   1 = (a + b) 1 b. λ1 = a + b, λ2 = a − b     1 −1 , v2 = c. v1 = 1 1

Confirming Pages

Answers to Odd-Numbered Exercises 

  1 −1 a +b ,D = 1 1 0 3. a. λ1 = 0, λ2 = 1

0 a −b

d. P =



b. No conclusion can be drawn from part (a) about the diagonalizability of A.     0 −1  0   0    c. λ1 = 0: v1 =  v2 =   0   1  1 0   0  1    λ2 = 1: v3 =  0  0 d. The eigenvectors {v1 , v2 , v3 } are linearly independent. e. A is not diagonalizable as it is a 4 × 4 matrix with only three linearly independent eigenvectors.    −λ 1 0   1  5. a. det(A − λI ) =  0 −λ  −k 3 −λ  = λ − 3λ + k = 0 3

b.

k=4 k=3 k = 2.5 k=0

Chapter Test: Chapter 5 1. F

2. F

3. F

4. T

5. T

6. T

7. T

8. T

9. T

10. T

11. F

12. F

13. F

14. T

15. F

16. T

17. T

18. T

19. T

20. F

21. T

22. T

23. F

24. F

25. T

26. T

27. T

28. T

29. T

30. F

31. T

32. T

33. T

34. F

35. T

36. T

37. T

38. T

39. T

40. T

Chapter 6 Section 6.1

k = −2.5 k = −3 k = −4 c. −2 < k < 2  1  1  7. a. Let v =  .  .. 1

1. 5 3. −11 √ 5. 26

   . Then     Av =  

λ λ .. . λ





     = λ  

1 1 .. . 1

    

so λ is an eigenvalue of A corresponding to the eigenvector v. b. Yes, since A and At have the same eigenvalues.

  1 1 7. √ 26 5   10 2 9. √ 5 1 √ 11. 22   −3 1  −2  13. √ 22 3   1 3  1  15. √ 11 3 17. c = 6

467

Confirming Pages

468

Answers to Odd-Numbered Exercises

19. v1 ⊥v2 ; v1 ⊥v4 ; v1 ⊥v5 ; v2 ⊥v3 ; v3 ⊥v4 ; v3 ⊥v5

Then v·u = v·(c1 u1 + c2 u2 + · · · + cn un )

21. Since v3 = −v1 , the vectors v1 and v3 are in opposite directions.   2 23. w = 0

= c1 v·u1 + c2 v·u2 + · · · + cn v·un = c1 (0) + c2 (0) + · · · + cn (0) = 0 31. Consider the equation c1 v1 + c2 v2 + · · · + cn vn = 0

y 5

Since u

⫺5

w

v1 ·(c1 v1 + c2 v2 + · · · + cn vn ) = v1 · 0 v

5

x

⫺5

25. w =

3 2



3 1



so c1 v1 ·v1 + c2 v1 ·v2 + · · · + cn v1 ·vn = 0 Since S is an orthogonal set of vectors, this equation reduces to c1 ||v1 ||2 = 0 and since ||v1 || = 0, then c1 = 0. In a similar way we have c2 = c3 = · · · = cn = 0. Hence, S is linearly independent. 33. Since ||u||2 = u·u,

y

||u + v||2 + ||u − v||2 = (u + v)·(u + v)

5

u

v ⫺5

+ (u − v)·(u − v) = u·u + 2u·v + v·v

w 5

+ u·u − 2u·v + v·v

x

= 2||u||2 + 2||v||2 ⫺5

35. If the column vectors of A form an orthogonal set, then the row vectors of At are orthogonal to the column vectors of A. Consequently,

  5 1 2  27. w = 6 1

(At A)ij = 0

if i = j

On the other hand, if i = j , then (At A)ii = ||Ai ||2 . Thus,  u w

  t AA=  

||A1 ||2

0

···

0 .. . 0

||A2 ||2

0 ..

0 ···

0

.

0 .. . 0 ||An ||2

     

v

29. Let u be a vector in span{u1 , u2 , · · · , un }. Then there exist scalars c1 , c2 , · · · , cn such that u = c1 u1 + c2 u2 + · · · + cn un

37. Suppose that (Au)·v = u·(Av) for all u and v in ⺢n . By Exercise 36, u·(Av) = (At u)·v and by hypothesis u·(Av) = (Au)·v

Confirming Pages

Answers to Odd-Numbered Exercises 

for all u and v in ⺢n . Thus,

for all u and v in ⺢n . Let u = ei and v = ej , so (At )ij = Aij . Hence At = A, so A is symmetric. For the converse, suppose that A = At . Then by Exercise 36, u·(Av) = (At u)·v = (Au)·v Section 6.2 1. Since u, u = 0 when u1 = 3u2 or u1 = u2 , V is not an inner product space. 3. Since u + v, w and u, v + v, w are not equal for all u, v, and w, V is not an inner product space. 5. Yes, V is an inner product space. 7. Yes, V is an inner product space. 9. Yes, V is an inner product space. ! π ! π 11. sin x dx = cos x dx −π

! =

$1

−π π −π

cos x sin x dx = 0

√

31. If f is an even function and g is an odd function, then fg is an odd function. Then ! a f (x )g(x ) dx = 0 so f and g are orthogonal.





− 23 3 2

1 2 1 2

b. u − projv u =



v·(u − projv u) =  3. a. projv u =

b. cos θ = − 32

−5 17

2 −5

 =

√ 19

'  ( 8 ( ( 23. a.  A − B = )tr  0 8

− 56 

b. u − projv u =

0 3 4



8 √ 4  = 25 = 5 14

 5. a. projv u = 

− 43 4 3 4 3

 −1 · 1

8 5 − 54

v·(u − projv u) = 





− 53



3 √ 5 6

√ 26√ 38 39

√ 3 3



√ 19. a.  2x 2 − 4 = 2 5

b. cos θ =

e.  1 − x =

1. a. projv u =

2e 2 −2

b. cos θ =

3 √ 2 3

Section 6.3

3

&  21. a.  A − B = tr

d. cos θ =

33. c1 u1 , c2 u2  = c1 u1 , c2 u2  = c1 c2 u1 , u2  =0

0

b. cos θ = √2

x y

−a

(2x − 1) dx = 0  $1 2 1 0 −x + x − 6 dx = 0  $1 4 1 3 2 0 −2x + 3x − 3 x + 6 dx = 0 % 15. a.  −3 + 3x − x 2 = 370 10 √ 5 b. cos θ = − 168 105 % 17. a.  x − e x = 12 e 2 − 13 6

13.

    2x + 3y = 0      x    y  2x − 3y + z = 0 27.    z  + $1 * 29. a. x 2 , x 3 = 0 x 5 dx = 16 * + $1 b. e x , e −x = 0 dx = 1 %$ 1 c.  1 = 0 dx = 1 %$ √ 1 2 3  x = 0 x dx = 3 25.

(At u)·v = (Au)·v

469

  

1 2 1 2

 =0

  8  1 5 =0 · 2 − 54

Confirming Pages

470

Answers to Odd-Numbered Exercises   b. u − projv u = 

1 3 5 3 − 43

  

  1  3 1   v·(u − projv u) =  −1  ·  53  = 0 −1 − 43   0 7. a. projv u =  0  −1   1 b. u − projv u =  −1  0     0 1 v·(u − projv u) =  0 ·  −1  = 0 1 0 

9. a. projq p = 54 x −

11. a. projq p = − 74 x 2 +

!

1 0

1 √ 2   1 √ 15.  2

13.

17.

       

27. Let v = c1 u1 + c2 u2 + · · · + cn un Then || v ||2 = v · v = c12 (u1 ·u1 ) + c22 (u2 ·u2 ) + · · · + cn2 (un ·un ) = c12 + c22 + · · · + cn2

17 12

29. Since

−

/ 0 aki akj = 1 k =1

n .

dx = 0

if i = j if i = j

  = At A ij

then At A = I . √ 31. Since ||Ax|| = Ax·Ax and

7 4

Ax·Ax = xt ·(At Ax) = x·x 3 4

q, p − projq p

= 

15 2 4 x

       

= |v·u1 |2 + · · · + |v·un |2

+ * q, p − projq p  ! 1 9 17 = (3x − 1) x 2 − x + 4 12 0

*

    1 0    1  1  0  1    √  25. , √  1  −1  3 3   1 1

5 12

b. p − projq p = x 2 − 94 x +

b. p − projq p =

      −1 −2 1    1  1  1  −2  1  0      √  21. , √  , √  0  −1  −2  6 6 6   1 0 1 ,√ 3x , −3x + 2 23.

+ 

(x 2 − 1)

15 2 3 x − 4 4

dx = 0

   1 1 −1 ,√ −1 2 −1       1 1  1 1 1  0 , √  2 , √  −1   6 3 −1 −1 1

then ||Ax||2 = x·x = ||x||2 so ||Ax|| = ||x||. 33. By Exercise 32, Ax·Ay = x·y. Then Ax·Ay = 0 if and only if x·y = 0 35. Let W = {v | v·ui = 0 for all i = 1, 2, . . . , m}



,√ √ 3(x − 1), 3x − 1, 6 5(x 2 − x + 16 )

     1 2   1 1 √  1 , √  −1  19.  3  6 1 −1

If c is a real number and x and y are vectors in W , then (x + cy)·ui = x·ui + cy·ui = 0 + c(0) = 0 for all i = 1, 2, . . . , n. 37.

vt Av = [x y]



3 1

1 3



x y



= 3x 2 + 2xy + 3y 2 ≥ (x + y)2 ≥ 0 39.

xt At Ax = (Ax)t Ax = (Ax)·(Ax) = ||Ax||2 ≥ 0

Confirming Pages

Answers to Odd-Numbered Exercises 41. Since Ax = λx, then xt Ax = λ||x||2 . Since A is positive definite and x is not the zero vector, then xt Ax > 0, so λ > 0. Section 6.4



1. W ⊥ = span

3.

W⊥

5. W ⊥

1 1 2



 1  = span  −2  0  2    31 = span  − 3   1

 0  ,  1   1        

 1    2  −6    3    1     − 2   −1  ⊥  7. W = span  ,    0   1      1  0  1     −3     9.  − 13      1  1   1   −2   2     3   1     −2   2  11.  ,     1   0        0 1 
52 2 13. 50 9 x − 9 x +1   1       1    15.  1       1 17. An orthogonal basis is    0   2 B =  0 ,  −1    0 0 

 1 projW v =  2  0   0 19. projW v =  0  0

471

21. An orthogonal basis is    3 −5        0  ,  21  B=   −1   −3       2 6   −5   4  21  projW v = 73  −3  6   1 23. a. W ⊥ = span 3   1 −3 b. projW v = 1 10   1 3 c. u = v − projW v = 10 9     1 3 −3 · d. =0 1 10 9 e.

y 5

W⊥

v

⫺5

projW v

5

x

W

⫺5

25. Notice that the vectors v1 and v2 are not orthogonal. Using the Gram-Schmidt process orthogonal vectors with the same span are     1 0  1   3  −1 3   2   a. W ⊥ = span  −1    1   2 1 5  b. projW v = 3 1   4 1 −2  c. u = v − projW v = 3 2

Confirming Pages

472

Answers to Odd-Numbered Exercises 

 2 d. Since u is a scalar multiple of  −1 , then u is 1 in W ⊥ . e.

W

3. a.

projW v v

b. y =

W⊥

653,089 13,148 x

−

317,689,173 3287

5. a.

27. Let w ∈ W2⊥ , so w, u = 0 for all u ∈ W2 . Since W1 ⊆ W2 , then w, u = 0 for all u ∈ W1 . Hence w ∈ W1⊥ , so W2⊥ ⊆ W1⊥ .  29. a. Let A =

d f



e g

 and B =



A, B = tr  = tr

a b

b c

a b



ad + bf bd + cf

b c d f

 . Then e g

ae + bg be + cg

 

So A ∈ W ⊥ if and only if ad + bf + be + cg = 0 for all real numbers a, b, and c. This implies   0 e A= . That is, A is skew-symmetric. −e 0  b.

a c

Section 6.5  1. a. 0 x=

b d

5 2



 =

b+c 2

a b+c 2

d



 +

0 − b−c 2

b−c 2



b. y = 0.07162857143x − 137.2780952 7. a. p2 (x ) = 2 sin x − sin 2x 2 p3 (x ) = 2 sin x − sin 2x + sin 3x 3 2 p4 (x ) = 2 sin x − sin 2x + sin 3x 3 1 − sin 4x 2 2 p5 (x ) = 2 sin x − sin 2x + sin 3x 3 2 1 − sin 4x + sin 5x 2 5 b.

0



0 

 x= b. w1 = A0

5 2 5 2



9. a.

 

5   w2 = b − w1 = 

1 2 π − 4 cos x + cos 2x 3 1 4 p3 (x ) = π2 − 4 cos x + cos 2x − cos 3x 3 9 1 4 p4 (x ) = π2 − 4 cos x + cos 2x − cos 3x 3 9 1 + cos 4x 4 p2 (x ) =

3 2 − 32

0

  

Confirming Pages

473

Answers to Odd-Numbered Exercises

p5 (x ) =

V−3

1 2 4 π − 4 cos x + cos 2x − cos 3x 3 9 1 4 + cos 4x − cos 5x 4 25

V−1 b.

  0       −2   = span   1       1    −1 0      0 1   = span   2 ,  −1    0 1

       

dim (V3 ) + dim (V−3 ) + dim (V−1 ) =1+1+2=4 13. Yes.  √

3/2 −1/2

Section 6.6 1. λ1 = 3, λ2 = −1 3. λ1 = 1, λ2 = −3, λ3 = 3

15. Yes.  

5. λ1 = −3 with eigenvector v1 = 

−1 2

√ √2/2  − 2/2 0

 ; λ2 = 2 with

 2 . Observe that v1 ·v2 = 0. 1   1 7. λ1 = 1 with eigenvector v1 =  0 ; λ2 = −3 with 1   −1 eigenvector v2 =  2 ; λ3 = 3 with eigenvector 1   −1 v3 =  −1 . Observe that v1 ·v2 = v1 ·v3 = 1

dim (V3 ) + dim (V−1 ) = 1 + 2 = 3   3       1   11. V3 = span      1     1

 √

√ √2/2 2/2 0

3/2 1/2

−1/2 √ 3/2

 √ 0 √2/2 0   2/2 1 0 

eigenvector v2 =

v2 ·v3 = 0.    1  9. V3 = span  0    1     0   −1 V−1 = span  0 ,  1    1 0

√1/2 3/2

1 = 0 0

0 1 0

19.

21.

23.

 =

1 0

√ −√2/2 2/2 0

0 1



 0 0  1

 0 0  1

√  √  −1 −1/√2 1/√2 ;D = P= 0 1/ 2 1/ 2 √ √    −4 −1/√2 1/√2 ;D = P= 0 1/ 2 1/ 2 √ √ √   −1/√3 1/ 2 −1/√6  P =  1/√3 √0 −2/√6 1/ 3 1/ 2 1/ 6   1 0 0 0  D = 0 2 0 0 −2 Since AAt = BB t = I , then 

17.



0 7 0 2

 

(AB)(AB)t = AB(B t At ) = A(BB t )At = AIAt = AAt = I Similarly, (BA)(BA)t = I . 25. Since AAt = I , At is the inverse of A so At A = I and hence At is also orthogonal.

Confirming Pages

474

Answers to Odd-Numbered Exercises

27. a. Since cos2 θ + sin2 θ = 1, then     cos θ − sin θ cos θ sin θ 1 = sin θ cos θ − sin θ cos θ 0

0 1



29. If D = P t AP, then D t = (P t AP)t = P t At P Since D is a diagonal matrix, then D t = D, so D = P t AP and hence P t AP = P t At P. Then P(P t AP)P t = P(P t At P)P t , so A = At . 31. a. vt v = v12 + · · · + vn2 b. The transpose of both sides of the equation Av = λv gives vt At = λvt . Since A is skew-symmetric, vt (−A) = λvt . Now, right multiplication of both sides by v gives vt (−Av) = λvt v, so vt (−λv) = λvt v. Then 2λvt v = 0 so 2λ(v12 + · · · + vn2 ) = 0 and this gives λ = 0. Section 6.7 1.

30(y  )2

3.

2(x  )2

5.

(x  )2 2

+

+

−

√

4



7. a. [x y]

=0

=1

=1 4 0

0 16



x y

 − 16 = 0

Section 6.8 √ 1. σ1 = 10, σ2 = 0 √ √ 3. σ1 = 2 3, σ2 = 5, σ3 = 0   1  √ √1 8 0 2 2 5. A = 0 2 √1 − √12 2 7. A =

0 1



  x a  y ·  b  = ax + by + cz = 0 z c  a so  b  is in W ⊥ . c    a  ⊥ b. W = span  b    c That is, W ⊥

b. 10x 2 − 12xy + 10y 2 − 16 = 0 √ 9. a. 7x 2 + 6 3xy + 13y 2 − 16 = 0 √ b. 7(x − 3)2 + 6 3(x − 3)(y − 2) + 13(y − 2)2 − 16 = 0



 0 0 1 0  0 1 √  2/2   √0  2/2



10x 

(y  )2

(y  )2

Review Exercises Chapter 6    1 1 2 1 1. a.  0 0 1  −→  0 1 0 0 0    √ 2/2  0 ,  b.  1 ,  √0  0 − 2/2   −2 c. projW v =  0  −1   x 3. a. If  y  ∈ W , then z

 √ 1 2 0 0

0 1

0 0





 a is the line in the direction of  b  c

and which is perpendicular (the normal vector) to the plane ax + by + cz = 0.   a ax1 +bx2 +cx3  b  c. projW ⊥ v = a 2 +b 2 +c 2 c |ax +bx +cx | d.  projW ⊥ v = √1 2 22 32 a +b +c

√1 2 √1 2

√1 2 − √12

0

√1 2

  1 0

9. a. x1 = 2, x2 = 0 b. x1 = 1, x2 = 1 c. σ1 /σ2 ≈ 6, 324, 555



0 − √12



√1 2



 0 

√1 2

Note: This gives the distance from the point (x1 , x2 , x3 ) to the plane. $ π 5. a. 1, cos x  = −π cos x dx = 0 $π

1, sin x  = −π sin x dx = 0 $π

cos x , sin x  = −π cos x sin x dx = 0 , b. √12π , √1π cos x , √1π sin x c. projW x 2 = 13 π2 − 4 cos x √ d.  projW x 2 = 13 2π5 + 144π

Confirming Pages

Answers to Odd-Numbered Exercises 7. Using the properties of an inner product and the fact that the vectors are orthonormal, √  v  = v·v % = c12 v1 , v1  + · · · + cn2 vn , vn  % = c12 + · · · + cn2 If the basis is orthogonal, then %  v = c12 v1 , v1  + · · · + cn2 vn , vn  

1  1  9. a.  1 1

  −1 1  2   −→  0  0 1  2 0

0 −1 0 −1

       b. B1 =               c. B2 =            d. Q =    



 1   1  ,  1   1 1 2 1 2 1 2 1 2

   ,   

1 2 1 2

1 2 − 21

1 2 1 2

1 2 − 21

2 R= 0 0

−1 1 0



  −1  0 ,   1  0

1 2 − 21 1 2 − 21



1 2 − 12 1 2 − 12

  ,   

√ − 22



 0   √  2  2  0 

2  −2 √ 2

 0 0   1  0

0 1 0 0

−

           

       0 √  2  2     0 √ 2 2

7. T

8. F

9. F

10. F

11. T

12. T

13. T

14. T

15. F

16. T

17. T

18. T

19. F

20. T

21. F

22. T

23. F

24. F

25. F

26. F

27. T

28. T

29. T

30. F

31. T

32. T

33. F

34. F

35. T

36. F

37. T

38. T

39. F

40. T

Appendix A Section A.1 1. A ∩ B = {−2, 2, 9} 3. A × B = {(a, b) | a ∈ A, b ∈ B} There are 9 × 9 = 81 ordered pairs in A × B. 5. A\B = {−4, 0, 1, 3, 5, 7} 7. A ∩ B = [0, 3] 9. A\B = (−11, 0) 11. A\C = (−11, −9) 13. (A ∪ B)\C = (−11, −9) 15.

e. A = QR

y 5

Chapter Test: Chapter 6 1. T

2. T

3. F

4. F

5. F

6. T

⫺5

5

⫺5

x

475

Confirming Pages

476

Answers to Odd-Numbered Exercises

17.

y 5

⫺5

5

x

5

x

⫺5

19.

y 5

⫺5

⫺5

21. (A ∩ B) ∩ C = {5} = A ∩ (B ∩ C ) 23. A ∩ (B ∪ C ) = {1, 2, 5, 7} = (A ∩ B) ∪ (A ∩ C ) 25. A\(B ∪ C ) = {3, 9, 11} = (A\B) ∩ (A\C ) Section A.2 1. Since for each first coordinate there is a unique second coordinate, f is a function. 3. Since there is no x such that f (x ) = 14, the function is not onto. The range of f is the set {−2, −1, 3, 9, 11}. 5. f −1 ({−2}) = {1, 4} 7. Since f is not one-to-one, f does not have an inverse. 9. {(1, −2), (2, −1), (3, 3), (4, 5), (5, 9), (6, 11)} 11. f (A ∪ B) = f ((−3, 7)) = [0, 49) f (A) ∪ f (B) = [0, 25] ∪ [0, 49] = [0, 49] 13. f (A ∩ B) = f ({0}) = {0} f (A) ∩ f (B) = [0, 4] ∩ [0, 4] = [0, 4] Therefore, f (A ∩ B) ⊂ f (A) ∩ f (B), but f (A ∩ B) = f (A) ∩ f (B). 15. f −1 (x ) = x −b a 17. If n is odd, then f (n) (x ) = −x + c. If n is even, then f (n) (x ) = x . 19. a. To show that f is one-to-one, we have e 2x1 −1 = e 2x2 −1 ⇔ 2x1 − 1 = 2x2 − 1 ⇔ x1 = x2 b. Since the exponential function is always positive, f is not onto ⺢.

c. Define g : ⺢ → (0, ∞) by g(x ) = e 2x −1 . d. g −1 (x ) = 12 (1 + ln x ). 21. a. To show that f is one-to-one, we have 2n1 = 2n2 if and only if n1 = n2 . b. Since every image is an even number, the range of f is a proper subset of ⺞. c. f −1 (E ) = ⺞; f −1 (O) = φ 23. a. f (A) = {2k + 1 | k ∈ ⺪} b. f (B) = {2k + 1 | k ∈ ⺪} c. f −1 ({0}) = {(m, n) | n = −2m} d. f −1 (E ) = {(m, n) | n is even} e. f −1 (O) = {(m, n) | n is odd} f. Since f ((1, −2)) = 0 = f ((0, 0)), then f is not one-to-one. g. If z ∈ ⺪, let m = 0 and n = z , so that f (m, n) = z . Section A.3 1. If the side is x , then h 2 = x 2 + x 2 = 2x 2 , so √ h = 2x . 3. If the side √is x , then the height is h = √ 1 3 3 2 is A = 2 x 2 x = 4 x .

√

3 2 x,

so the area

5. If a divides b, there is some k such that ak = b; and if b divides c, there is some
such that b
= c. Then c = b
= (ak )
= (k
)a, so a divides c. 7. If n is odd, there is some k such that n = 2k + 1. Then n 2 = (2k + 1)2 = 2(2k 2 + k ) + 1, so n 2 is odd. 9. If b = a + 1, then (a + b)2 = (2a + 1)2 = 2(2a 2 + 2a) +1, so (a + b)2 is odd. 11. Let m = 2 and n = 3. Then m 2 + n 2 = 13, which is not divisible by 4. 13. Contrapositive: Suppose n is even, so there is some k such that n = 2k . Then n 2 = 4k 2 , so n 2 is even. 15. Contrapositive: Suppose p = q. Then 1 √ pq = p 2 = p = (p + q)/2. 17. Contrapositive: Suppose x > 0. If  = x /2 > 0, then x > . √ 19. Contradiction: Suppose 3 2 = p/q such that p and q have no common factors. Then 2q 3 = p 3 , so p 3 is even and hence p is even. This gives that q is also even, which contradicts the assumption that p and q have no common factors. 21. If 7xy ≤ 3x 2 + 2y 2 , then 3x 2 − 7xy + 2y 2 = (3x − y)(x − 2y) ≥ 0. There are two cases: either both factors are greater than or equal to 0, or both are less

Confirming Pages

Answers to Odd-Numbered Exercises than or equal to 0. The first case is not possible since the assumption is that x < 2y. Therefore, 3x ≤ y. 23. Define f :⺢ → ⺢ by f (x ) = x 2 . Let C = [−4, 4], D = [0, 4]. Then f −1 (C ) = [−2, 2] = f −1 (D) but C
D. 25. If x ∈ f −1 (C ), then f (x ) ∈ C . Since C ⊂ D, then f (x ) ∈ D. Hence, x ∈ f −1 (D). 27. If y ∈ f (A\B), there is some x such that y = f (x ) with x ∈ A and x ∈ / B. So y ∈ f (A)\ f (B), and f (A\B) ⊂ f (A)\ f (B). Now suppose y ∈ f (A)\ f (B). So there is some x ∈ A such that y = f (x ). Since f is one-to-one, this is the only preimage for y, so x ∈ A\B. Therefore, f (A)\ f (B) ⊂ f (A\B). 29. By Theorem 3 of Sec. A.2, f (f −1 (C )) ⊂ C . Let y ∈ C . Since f is onto, there is some x such that y = f (x ). So x ∈ f −1 (C ), and hence y = f (x ) ∈ f (f −1 (C )). Therefore, C ⊂ f (f −1 (C )). Section A.4 1. Base case: n = 1 : 12 = 1(2)(3) 6 Inductive hypothesis: Assume the summation formula holds for the natural number n. Consider 12 + 22 + 32 + · · · + n 2 + (n + 1)2 n(n + 1)(2n + 1) + (n + 1)2 = 6 n +1 = (2n 2 + 7n + 6) 6 n +1 = (2n + 3)(n + 2) 6 (n + 1)(n + 2)(2n + 3) = 6 3. Base case: n = 1 : 1 = 1(3−1) 2 Inductive hypothesis: Assume the summation formula holds for the natural number n. Consider 1 + 4 + 7 + · · · + (3n − 2) + [3(n + 1) − 2] n(3n − 1) + (3n + 1) = 2 3n 2 + 5n + 2 = 2 (n + 1)(3n + 2) = 2

477

5. Base case: n = 1 : 2 = 1(4) 2 Inductive hypothesis: Assume the summation formula holds for the natural number n. Consider 2 + 5 + 8 + · · · + (3n − 1) + [3(n + 1) − 1] 1 = (3n 2 + 7n + 4) 2 (n + 1)(3n + 4) = 2 (n + 1)(3(n + 1) + 1) = 2 7. Base case: n = 1 : 3 = 3(2) 2 Inductive hypothesis: Assume the summation formula holds for the natural number n. Consider 3 + 6 + 9 + · · · + 3n + 3(n + 1) 1 = (3n 2 + 9n + 6) 2 3 = (n 2 + 3n + 2) 2 3(n + 1)(n + 2) = 2 9. Base case: n = 1 : 21 = 22 − 2 Inductive hypothesis: Assume the summation formula holds for the natural number n. Consider n+1 . k =1

2k =

n .

2k + 2n+1

k =1

= 2n+1 − 2 + 2n+1 = 2n+2 − 2 11. From the data in the table n

2 + 4 + · · · + 2n

1 2 3 4 5

2 = 1(2) 6 = 2(3) 12 = 3(4) 40 = 4(5) 30 = 5(6)

we make the conjecture that 2 + 4 + 6 + · · · + (2n) = n(n + 1)

Confirming Pages

478

Answers to Odd-Numbered Exercises Base case: n = 1 : 2 = 1(2) Inductive hypothesis: Assume the summation formula holds for the natural number n. Consider 2 + 4 + 6 + · · · + 2n + 2(n + 1) = n(n + 1) + 2(n + 1) = (n + 1)(n + 2)

13. Base case: n = 5 : 32 = 25 > 25 = 52 Inductive hypothesis: Assume 2n > n 2 holds for the natural number n. Consider 2n+1 = 2(2n ) > 2n 2 . But since 2n 2 − (n + 1)2 = n 2 − 2n − 1 = (n − 1)2 − 2 > 0, for all n ≥ 5, we have 2n+1 > (n + 1)2 . 15. Base case: n = 1 : 12 + 1 = 2, which is divisible by 2. Inductive hypothesis: Assume n 2 + n is divisible by 2. Consider (n + 1)2 + (n + 1) = n 2 + n + 2n + 2. By the inductive hypothesis, n 2 + n is divisible by 2, so since both terms on the right are divisible by 2, then (n + 1)2 + (n + 1) is divisible by 2. Alternatively, observe that n 2 + n = n(n + 1), which is the product of consecutive integers and is therefore even. 17. Base case: n = 1 : 1 = r−1 r−1 Inductive hypothesis: Assume the formula holds for the natural number n. Consider 1 + r + r 2 + · · · + r n−1 + r n rn − 1 = + rn r −1 r n − 1 + r n (r − 1) = r −1 r n+1 − 1 = r −1

19. Base case: n = 2 : A ∩ (B1 ∪ B2 ) = (A ∩ B1 ) ∪ (A ∩ B2 ), by Theorem 1 of Sec. A.1 Inductive hypothesis: Assume the formula holds for the natural number n. Consider A∩(B1 ∪ B2 ∪ · · · ∪ Bn ∪ Bn+1 ) = A ∩ [(B1 ∪ B2 ∪ · · · ∪ Bn ) ∪ Bn+1 ] = [A ∩ (B1 ∪ B2 ∪ · · · ∪ Bn )] ∪ (A ∩ Bn+1 ) = (A ∩ B1 ) ∪ (A ∩ B2 ) ∪ · · · ∪ (A ∩ Bn ) ∪ (A ∩ Bn+1 )  21. n n! = r!(n − r)! r n! = (n − r)!(n − (n − r))!  n = n −r 23. By the binomial theorem, 2n = (1 + 1)n =

n  . n k =0

k

Confirming Pages

Index A Abel’s formula, 190 Addition linear transformations and, 209–210 matrix, 27–29 of real numbers, 28 of vectors, 95–99, 129 Additive identity, 98 Additive inverse, 97, 98, 136 Aerodynamic forces, 199 Algebra, matrix, 26–37. See also Matrix algebra Angles, between vectors, 327–330 Arguments contradiction, 426–427 contrapositive, 426 direct, 425 Associated quadratic form, 386 Associative property, 97, 98 Augmented matrix for consistent linear systems, 22 explanation of, 15, 16 facts about, 23 as solution to linear systems, 16–17 Axioms, 424

B Back substitution, 4 Balance law, 310 Balancing chemical equations, 79 Basis change of, 177–181 explanation of, 149 facts about, 171 method for finding, 166–170 ordered, 174–176 orthogonal, 339–340 orthonormal, 342–352 standard, 159, 162, 163 for vector space, 159–164

Best-fit line, 322 Bijective functions, 420, 422 Bijective mapping, 226 Bilinear inner product, 333 Binary vectors, 127–128 Binomial coefficients, 435–436 Binomial theorem, 437–438 Brin, Sergey, 276

C Cartesian product, of two sets, 411 Cauchy-Schwartz inequality, 326–327, 336, 337 Characteristic equation, 279 Characteristic polynomials, 279 Check matrix, 128 Chemical equation balancing application, 1, 79 Circle, equation of, 385 Codewords, 127–129 Coefficient matrix, 15 Cofactors, of matrices, 56 Column rank, 221, 222 Column space, 152 Column vectors, 27 Commutative operation, 28 Commutative property, 97, 129 Commute, matrices that, 32, 33 Complement, of sets, 411 Complementary solution, 193 Complex numbers conjugate of, 377 equality and, 377 imaginary part, 144 real part, 144 set of, 134 Components, of vectors, 27, 95 Composition, of functions, 420–422 Computer graphics applications explanation of, 255 projection, 265–268

479

Confirming Pages

480

Index Computer graphics applications (continued) reflection, 260, 261 reversing graphics operations, 261–262 rotation, 264–265 scaling and shearing, 256–259 translation, 262–264 types of, 199 Conclusions, 424 Condition number, 403 Conic sections eigenvalues and, 388–390 explanation of, 61, 386 simplifying equations that describe, 385 Conservation of mass law, 1 Contained sets, 410 Continuous signals, 93 Contraction, 257 Contradiction argument, 426–427 Contrapositive argument, 426 Contrapositive statement, 425, 426 Converse, of theorems, 424–425 Convex, set on ⺢2 as, 272 Corollaries, 425 Cosine, of angle between vectors, 328, 337 Counter example, 425 Cramer’s rule, 62–64

D Damping coefficient, 192 Data compression, 401–403 Data sets, least squares approximation to find trends in, 371–373 Demand vectors, 83 DeMorgan’s laws, 413–414 Determinants facts about, 64–65 to find equation of conic sections, 61 linear independence and, 118–119 method to find, 55–56 properties of, 57–62 to solve linear systems, 62–64 of square matrices, 56 of 3 x 3 matrix, 54–55 of triangular matrices, 56–57 of 2 x 2 matrix, 54, 55 Diagonalization conditions for matrix, 291–292 eigenvalues and, 293

eigenvectors and, 282 examples of, 289–291, 293 explanation of, 287–289, 377 facts about, 297–298 linear operators and, 295–297 orthogonal, 379–382 similar matrices and, 293–294 of symmetric matrices, 377–383 symmetric matrices and, 293 systems of linear differential equations and, 302–309 of transition matrices, 312–313 Diagonal matrix, 56. See also Diagonalization Differential equations applications for, 185, 191–193 explanation of, 185 first-order, 186 fundamental sets of solutions and, 188–193 general solution of, 186 second-order with constant coefficients, 186–188, 191–193 Digraphs, 79–80 Dilation, 257 Dimension of column space, 221 explanation of, 164–165 of vector space, 165–166 Direct argument, 425 Directed graphs, 79–80 Direction field, of systems of differential equations, 302 Direct sum, 360–361 Discrete signals, 93 Distance formula, 323 Domain, 415 Dot product on Euclidean n-space (⺢n ), 323–331, 333 inner product that is not, 335 properties of, 326 of vectors, 29–32, 323 Dynamical systems, 300

E Echelon form definition of, 19 of matrices, 17–21 Economic input-output models, 82–84 Eigenfunctions, 284

Revised Confirming Pages

Index Eigenspaces algebraic multiplicity of, 281 corresponding to eigenvalues, 280–281 dimension of, 281–282 explanation of, 279–280 geometric multiplicity of, 282 Eigenvalues as complex numbers, 282–283 diagonalization and, 287–298 explanation of, 276 facts about, 284–285 geometric interpretation of, 278–279 of linear operators, 283–284 Markov chains and method to find, 310–314 of square triangular matrices, 283 systems of linear differential equations and, 302–309 transition matrix and, 312, 313 of triangular matrices, 283 for 2x2 matrices, 277–278 Eigenvectors definition of, 276–277 diagonalization and, 287–298 explanation of, 276 facts about, 284–285 geometric interpretation of, 278–279 of linear operators, 283–284 Markov chains and method to find, 310–314 orthogonal, 379–380 probability, 313 of real symmetric matrices, 378 systems of linear differential equations and, 300–309 transition matrix and, 312, 313 for 2x2 matrices, 277–278 Elementary matrix definition of, 69 explanation of, 69–71 inverse of, 71–72 Elements, of sets, 409 Empty sets, 410 Equal matrix, 27 Equations, matrix, 48–51 Equivalent linear systems, 4, 5 Equivalent triangular systems, 6–7, 10, 14

Euclidean n-space (⺢n ) definition of, 95 dot product on, 323–331, 333 properties of norm in, 329 vectors in, 94–99, 108, 119, 323–331 Euclidean space dot product and, 323–331 geometry of, 7 Euclidean vector spaces, 130 Existential quantifiers, 427 Exponential model, 186

F Factorials, 432 Finite dimensional vector spaces explanation of, 165 isomorphisms and, 230 linear operators and, 243–244 linear transformations between, 236, 237 Forward substitution, 68–69 Fourier polynomials, 373–375 Free variables, 9 Frequency, of wave, 93 Functions composition of, 420–422 explanation of, 415–418 inverse, 418–421 one-to-one, 419 onto, 420 relations vs., 415, 416 vector space of real-valued, 133–134 Fundamental frequency, 93–94 Fundamental sets of solutions superposition principle and, 188–189 theorem of, 190–191 Wronskian and, 189–191

G Gaussian elimination explanation of, 4 to solve linear systems, 6–11, 14, 15, 68 Gauss-Jordan elimination, 19 General solution, 3 Goodness of fit, measurement of, 366 Google, 276

481

Confirming Pages

482

Index Gram-Schmidt process examples of, 349–352 explanation of, 344, 347–348, 394 geometric interpretation of, 348–349 Graphics operations in ⺢2 reflection, 260, 261 reversing, 261–262 rotation, 264–265 scaling and shearing, 256–259 translation, 262–264 Graphs of conic sections, 61 of functions, 416

H Hamming, Richard, 127 Hamming’s code, 127, 129 Homogeneous coordinates, 262–264 Homogeneous linear systems, 49–51, 113 Horizontal line test, 419 Horizontal scaling, 256, 257 Horizontal shear, 258 Hypothesis explanation of, 424 inductive, 430

I Identity matrix, 39 Images explanation of, 415, 418 inverse, 416, 418 Imaginary part, complex numbers, 134 Inconsistent linear systems explanation of, 2, 10 reduced matrix for, 21–22 Independence, linear. See Linear independence Inequality, Cauchy-Schwartz, 326–327 Infinite dimensional vector space, 165 Initial point, vector, 95 Initial probability vectors, 275 Initial-value problems, 186, 192–193 Injective mapping. See One-to-one mapping Injective functions, 419 Inner product examples of, 334–336

explanation of, 333 that is not dot product, 335 Inner product spaces diagonalization of symmetric matrices and, 377–383 explanation of, 333–334 facts about, 340 least squares approximation and, 366–375 orthogonal complements and, 355–364 orthogonal sets and, 338–340 orthonormal bases and, 342–352 properties of norm in, 336–337 quadratic forms and, 385–391 singular value decomposition and, 392–403 subspaces of, 355 Input-output matrix, 83 Integers, set of, 409 Internal demand, 83 Intersection, of sets, 410, 411 Inverse functions explanation of, 418–420 unique nature of, 421 Inverse images, 416, 418 Inverse of elementary matrix, 71–72 Inverse of square matrix definition of, 40 explanation of, 40–45 facts about, 45 Inverse transformations, 230–231 Invertible functions, 418–420 Invertible matrix elementary matrices and, 72 explanation of, 41, 54 inverse of product of, 44–45 square, 60–61 Isomorphisms definition of, 229 explanation of, 226 inverse and, 230–231 linear transformations as, 229–231 one-to-one and onto mappings and, 226–230 vector space, 232–233

K Kepler, Johannes, 61 Kirchhoff’s laws, 88

Confirming Pages

Index

L Law of conservation of mass, 1 Leading variables, 10 Least squares approximation background of, 366–368 to find trends in data sets, 371–373 Fourier polynomials and, 373–375 linear regression and, 371–373 Least squares solutions, 369–371 Lemmas, 425 Length, of vectors, 95 Leontief input-output model, 82 Linear codes, 129 Linear combinations definition of, 102, 146 of elements of fundamental set, 94 matrix multiplication and, 107 of vectors, 102–106, 146 Linear dependence definition of, 111, 157 explanation of, 111, 157 of vectors, 112, 158 Linear equations, in n variables, 3 Linear independence definition of, 111, 157 determinants and, 118–119 explanation of, 111–112 of vectors, 112–117, 158 Linear operators diagonalizable, 295–297 eigenvalues and eigenvectors of, 283–284 explanation of, 202, 237 similarity and, 249–252 Linear regression, 368, 371–373 Linear systems augmented matrices to solve, 16–17, 22 consistent, 2, 117 converted to equivalent triangular systems, 6–7, 10, 14 Cramer’s rule to solve, 62–64 definition of, 3 discussion of, 3–4 elimination method to solve, 4–11, 14, 15 equivalent, 4, 5 explanation of, 2–3 facts about, 11–12 with four variables, 2, 8–9 homogeneous, 49–51, 113

ill-conditioned, 403 inconsistent, 2, 10, 21–22 linear independence and, 117–118 LU factorization to solve, 75–76 matrix form of, 48 nullity of matrices and, 222–223 in terms of geometric structure of Euclidean space, 363–364 3 x, 3, 7–8 triangular form of, 4, 6, 10 with two variables, 2–3 vector form of, 106–107 vector form of solution to, 48–50 Linear systems applications balancing chemical equations, 79 economic input-output models, 82–84 network flow, 79–81 nutrition, 81–82 Linear transformations computer graphics and, 199, 255–268 definition of, 202, 235 explanation of, 200–202, 235–236 from geometric perspective, 203–204 inverse of, 230 as isomorphisms, 229–231 isomorphisms as, 226–233 matrices and, 202–203, 221–222, 235–245 null space and range and, 214–223 operations with, 209–210 similarity and, 249–253 Lower triangular matrix examples of, 57 explanation of, 56, 73 LU factorization facts about, 77 of matrices, 69, 72–75, 392 solving linear systems using, 75–76

M Mapping bijective, 226 explanation of, 200–201, 415 linear transformations and, 201–202, 205–207, 241–242 one-to-one, 226–230 onto, 226, 227

483

Revised Confirming Pages

484

Index Markov chains applications of, 310–314 explanation of, 275–276 Markov process, 310 Mathematical induction base case, 430 binomial coefficients and binomial theorem and, 435–438 examples of, 431–435 inductive hypothesis, 430 introduction to, 429–430 principle of, 430–431 Matrices addition of, 27–29 augmented, 15–17, 22, 23 check, 128 coefficient, 15 condition number of, 403 definition of, 14 determinants of, 54–65 diagonal, 56 discussion of, 14–15 echelon form of, 17–21 elementary, 69–72 finding singular value decomposition of, 398–402 identity, 39 input-output, 83 inverse of product of invertible, 44–45 inverse of square, 39–45 linear independence of, 114 linear transformations and, 202–203, 221–222, 235–245 LU factorization of, 69, 72–75 minors and cofactors of, 56 nullity of, 221–223 null space of, 152–153 orthogonal, 381–382 permutation, 76–77 positive definite, 354 positive semidefinite, 354 rank of, 222 scalar multiplication, 27 singular values of, 393–396 stochastic, 275, 311, 314 subspaces and, 362 symmetric, 36 that commute, 32, 33 transition, 177–182, 275, 276, 311–313

transpose of, 35–36 triangular, 15, 56–57, 283 vector spaces of, 130 Matrix addition, 27–29 Matrix algebra addition and scalar multiplication, 27–29 explanation of, 26–27 facts about, 36–37 matrix multiplication, 29–35 symmetric matrix, 36 transpose of matrix, 35–36 Matrix equations, 48–51 Matrix form, of linear systems, 48 Matrix multiplication definition of, 32 explanation of, 29–35, 210 linear combinations and, 107 linear transformations between finite dimensional vector spaces and, 236–237 properties of, 35 to write linear systems in terms of matrices and vectors, 48–51 Members, of sets, 409 Minors, of matrices, 56 Multiplication. See Matrix multiplication; Scalar multiplication, Multiplicative identity, 39 Multivariate calculus, 322

N Natural numbers. See also Mathematical induction, set of, 409 statements involving, 429–434 Network flow application, 79–81 Newton, Isaac, 61 Nilpotent, 299 Noninvertible matrix, 41 Normal equation, least squares solution to, 369–370 Nullity, of matrices, 221–223 Null sets, 410 Null space, of linear transformations, 214–221 of matrices, 152–153, 221 Nutrition application, 81–82

Confirming Pages

Index

O

P

One-parameter family, of solutions, 9 One-to-one functions, 419 One-to-one mapping, 226–230 Onto functions, 420 Onto mapping, 226, 227 Ordered basis, 174–176 Ordinary differential equation, 185 Orthogonal basis construction of, 347–348 of finite dimensional inner product space, 346–347 singular values and, 394 vectors that form, 355 Orthogonal complement definition of, 357 examples of, 358–360 explanation of, 355–358 facts about, 364 inner product spaces and, 356 linear systems and, 363–364 matrices and, 362 projection theorem and, 361–362 subspaces and, 358 Orthogonal diagonalization, 379–382 Orthogonal matrix, 381–382 Orthogonal projection explanation of, 343–345, 360, 362 Gram-Schmidt process and, 347, 348 Orthogonal sets explanation of, 338 properties of, 338–340 Orthogonal vectors explanation of, 328–329, 337 in inner product spaces, 338 (See also Inner product spaces) subspaces of inner product spaces and, 355–360 Orthonormal basis Gram-Schmidt process and, 347–352 for inner product space, 345–347 ordered, 339–340, 342 orthogonal matrices and, 381 orthogonal projections and, 343–345 Orthonormal vectors, 338

Page, Larry, 276 Page range algorithm (Google), 276 Parabolas, general form of, 11 Parallelogram rule, 96 Parallel projection, 266 Parametric equations, 266 Pascal’s triangle, 435 Past plane, 301–302 Period, of wave, 93 Periodic motion, 93 Periodic signals, 93 Permutation matrix, 76–77 Phase portrait, 302–304, 306 Photosynthesis application, 1–2 Pitch, 199 Pivot, 18, 19 Pixels, 255 PLU factorization, 76–77 Polynomials characteristic, 279 of degree n, 132 derivative of constant, 217 Fourier, 373–375 trigonometric, 373–374 use of Gram-Schmidt process on space of, 350–351 vector space of, 132–133, 163, 334 zero, 132 Positive definite matrix, 354 Positive semidefinite matrix, 354 Predator-prey model, 300 Preimages, 416 Principle of mathematical induction. See Mathematical induction, Probability vectors, 311 Production vectors, 83 Product matrix, 30–31 Projection example of, 266–268 explanation of, 265 orthogonal, 343 parallel, 266 Projection theorem, 361–362 Proofs contradiction argument, 426–427 contrapositive argument, 426 direct argument, 425

485

Confirming Pages

486

Index Proofs (continued) explanation of, 424–425 quantifiers, 427 Propositions, 425

Q Quadratic equations in three variables, 391 in two variables, 385–388 Quadratic forms applications for, 385 associated, 386 rotation of axes and, 385–390 Quadratic surfaces, 391 Quantifiers, 427

R Range explanation of, 415 of linear transformations, 214–221 Rank of a matrix, 222 Rational numbers, set of, 410 Real numbers addition of, 28 set of, 410 signals representing, 93 Real part, complex numbers, 134 Real-valued fuctions, 133–134 Reduced matrix, for inconsistent linear systems, 21–22 Reduced row echelon form explanation of, 18, 19, 23 transforming matrices to equivalent, 22–23 Reflection, 260, 261 Relations explanation of, 415 functions vs., 415, 416 Riemann integral, 334 Roll, 199 Rotation, 264–265 Rotation of axes, 385–390 Row echelon form explanation of, 17, 19 reduced, 18–21 Row equivalent, 16 elementary matrices and, 72 Row operations, 16, 58

Row rank, of matrices, 222 Row vectors, 27

S Scalar multiplication explanation of, 27–29 linear transformations and, 209–210 of vectors, 95–99, 129, 161 Scalar product of matrices, 27 of vectors, 96 Scalar projection, 343 Scaling, 96, 256–258 Scatterplots, 321 Second-order differential equations, with constant coefficients, 186–188, 191–193 Sets, empty, 410 explanation of, 409–410 null, 410 operations on, 410–414 orthogonal, 338–340 solution, 3 Shearing, 258–259 Signals, 93 Similar matrix background of, 249–251 explanation of, 252, 253 Singular value decomposition (SVD) data compression and, 401–403 explanation of, 392 four fundamental subspaces and, 401 method for, 398–400 theorem of, 396–398 Singular values, 392 definition of, 393 of m x n matrix, 393–396 Solutions, to linear systems with n variables, 3 Solution set, 3 Span, of set of vectors, 146–152 Square matrix determinant of, 56 inverse of, 39–45 invertibility of, 60 trace of, 142–143 Standard basis explanation of, 162, 163

Confirming Pages

Index matrix representation relative to, 235–237 polynomials of, 163 Standard position, of vectors, 95 State vectors, Markov chains and, 311–312 Steady-state vectors explanation of, 276 Markov chain and, 313–314 Stochastic matrix, 275, 311, 314 Subsets, 410, 412 Subspaces closure criteria for, 144 definition of, 140 examples of, 142–143 explanation of, 140–142 facts about, 153 four fundamental, 401 of inner product spaces, 355–360, 362 null space and column space of matrix and, 152–153 span of set of vectors and, 146–152 trivial, 142 of vector spaces, 140, 145, 146 Substitution back, 4 forward, 68–69 Superposition principle, 188–189 Surjective functions, 420 Surjective mapping. See Onto mapping, Symmetric matrix diagonalization of, 377–383 explanation of, 36 Syndrome vectors , 128 Systems of linear differential equations diagonalization and, 302–309 explanation of, 300 to model concentration of salt in interconnected tanks, 307–309 phase plane and, 301–302 uncoupled, 300–301 Systems of linear equations. See Linear systems

T Terminal point, vector, 95 Theorems converse of, 424–425 explanation of, 424

487

Tower of Hanoi puzzle, 429–430 Trace, of square matrices, 142–143 Trajectories, 301–302 Transformation, 199–200. See also Linear transformations Transition matrix diagonalizing the, 312–313 example of, 275, 276 explanation of, 177–180 inverse of, 181–182 Markov chains and, 311–312 Translation, 262–264 Transpose, of matrices, 35–36 Triangular form of linear systems, 4, 6–7, 10 matrices in, 15 Triangular matrix determinant of, 57, 58 eigenvalues of, 283 explanation of, 56–57 Trigonometric polynomials, 373–374 Trivial solution, to homogeneous systems, 49, 50 Trivial subspaces, 142

U Uncoupled systems, 300–301 Uniform scaling, 257 Union, of sets, 410 Unit vectors, 325 Universal quantifiers, 427 Universal set, 410 Upper triangular matrix examples of, 57 explanation of, 56, 68, 74

V Vector addition, 95–99, 129 Vector form of linear systems, 106–107 of solution to linear systems, 48–50 Vectors addition and scalar multiplication of, 95–99 algebraic properties of, 97–98 angle between, 327–330 applications for, 94

Confirming Pages

488

Index Vectors (continued) binary, 127–128 components of, 27, 95 demand, 83 distance between, 324, 336 dot product of, 29–32, 323 equal, 95 in Euclidean n-space (⺢n ), 94–99, 108, 119, 323–331 explanation of, 27, 94 length of, 95, 323–325, 336 linear combinations of, 102–106, 146 linear independence of, 112–117 orthogonal, 328–329, 337 orthonormal, 338 probability, 311 production, 83 span of set of, 146–152 standard position of, 95 state, 311–312 steady-state, 276, 313–314 syndrome, 128 unit, 325 zero, 97 Vector space isomorphism, 229, 232–233 Vector spaces, abstract, 136–137 basis for, 159–164, 166–170, 174–182 of complex numbers, 134–135 definition of, 129–130 dimension of, 165–166

Euclidean, 130 examples of, 130–135 expanding set of vectors in, 169–170 facts about, 137 finite dimensional, 165, 230, 236, 237 infinite dimensional, 165 isomorphic, 229, 232–233 linear transformations and, 204–208, 211, 220 of matrices, 130 or real-valued functions, 133–134 of polynomials, 132–133, 163, 334 subspaces of, 140, 145, 146 Venn diagrams example of, 411, 412 explanation of, 410–411 Vertical line test, 419 Vertical scaling, 257 Vertical shear, 259

W Waveform, of signal, 93 Waves, period of, 93 Wronskian, 189–191

Y Yaw, 199

Z Zero polynomials, 132 Zero vectors, 97

Confirming Pages

DeFranza

Linear Algebra Introduction to Linear Algebra with Applications by Jim DeFranza and Daniel Gagliardi provides the proper balance between computation, problem solving, and abstraction that will equip students with the necessary skills and problem solving strategies to allow for a greater understanding and appreciation of linear algebra and its numerous applications.

Abstract theory is essential to understanding how linear algebra is applied. Each concept is fully developed presenting natural connections between topics giving students a working knowledge of the theory and techniques for each module covered. Applications have been carefully chosen to highlight the utility of linear algebra in order to see the relevancy of the subject matter in other areas of science as well as in mathematics.

MD DALIM 976667 7/29/08 CYAN MAG YELO BLACK

Ranging from routine to more challenging, each exercise set extends the concepts or techniques by asking the student to construct complete arguments. End of chapter True/False questions help students connect concepts and facts presented in the chapter. Examples are designed to develop intuition and prepare students to think more conceptually about new topics as they are introduced. Students are introduced to the study of linear algebra in a sequential and thorough manner through an engaging writing style gaining a clear understanding of the theory essential for applying linear algebra to mathematics or other fields of science. Summaries conclude each section with important facts and techniques providing students with easy access to the material needed to master the exercise sets.

www.mhhe.com/defranza

ISBN 978-0-07-353235-6 MHID 0-07-353235-5

www.mhhe.com

Linear Algebra

Introduction to Linear Algebra with Applications provides students with the necessary tools for success:

Linear Algebra