A-PDF Split DEMO : Purchase from www.A-PDF.com to remove the watermark
Example: P(tQq< 1.697)= .95 P(/(30) > 1.697)=.05
Table 2 df I 2 3 4 5 o
7 8 o 10 ll t2 13 t4 15 16 t7 l8 19 20 2l 22 24 25 zo
2'7 28 29 30 31 -)z JJ
34 35 36 37 38 39 40 50 oo
Percentiles of thef-distribution /(.eo,df)
3.078 1.886 1.638 1.533 1.476 1.440 1.4t5 1.397 r.383 1.372 t.363 1.356 1.350 t.345 1.341 1.337 1.333 1.330 1.328 1.325 1.323 1.321 1.3t9 1 . 31 8 1 . 31 6 1 . 31 5 t.3t4 1.313 1 . 3 I1 1.310 1.309 1.309 1.308 1.307 1.306 1.306 1.305 r.304 t.304 1.303 r.299 t.282
/(.esdf)
6.314 2.920 2.353 2.132 2.0t5 1.943 1.895 1.860 1.833 1.8t2 r.796 1.182 t.7'71 l.t6r 1.753 1.746 t-740 1.734 1.729 1.725 1.121 t ; 7t 7 1 . 7t 4 1 . 7t l 1.708 1;706 t;703 1.701 r.699 1.697 1.696 1.694 r.692 1.691 1.690 1.688 1.687 1.686 1.685 1.684 1.676 1.645
t(.975,df)
t2.706 4.303 3.182 2.7',76 2.571 2.447 2.365 2.306 2.262 2.228 2.201 2.179 2.160 2.145 2.131 2.120 2.110 2-r01 2.093 2.086 2.080 2.0'74 2.069 2.064 2.060 2.056 2.052 2.048 2.045 2.042 2.O40 2.03'7 2.035 2.032 2.030 2.028 2.026 2.024 2.023 2.021 2.009 1.960
Source: This table was generatedusing the SAS@ function TINV
t(.99,df)
3r.821 6.965 4.541 3.74'7 3.365 3.143 2.998 2.896 2.821 2.764 2.718 2.681 2.650 2.624 2.602 2.583 2.567 2.552 2.539 2.528 2.5t8 2.508 2.500 2.492 2.485 2.479 2.473 2.46'7 2.462 2.457 2.453 2.449 2.445 2.44r 2.438 2.434 2.431 2.429 2.426 2.423 2.403 2.326
r1.WS,ar1 63.657 9.925 5.841 4.604 4.032 3.707 3.499 3.355 3.250 3.169 3.106 3.055 3.012 2.97'7 2.94'7 2.921 2.898 2.878 2.861 2.845 2.831 2.8t9 2.807 2.797 2.787 2.779 2.77r 2.763 2.756 2.750 2.744 2.738 Z.IJJ
2.728 2.724 2.7t9 2.715 2.712 2.708 2.704 2.678 2.576
The Rules of Summation
Expectations, Variances & Covariances
cov(x, Y): El(x- EItDV EIYDI : IIk - E(x)lb- n(v))f(x,y) '
2xi:xr+x2+"'+xn
-"
na
la:
n:-
s-*.--s..
/var(X)var(Y)
i: I i:l nnn s / - . r - . \ - S - .- r S , , . a\
t
I
i:t
E(cfi-t I
L^t
ft)
cov(x, r)
'
i:i
Ll l
c 2 Y ): c t E ( x ) + c 2 E ( Y )
E(x+Y):E(x)+E(Y)
i:l
nnn
L(^,+byi):aLxi+bLyi
var(aX t bY + cZ) : a2var(X) + b2va\Y) + c2var(Z)+ 2 a b c o v( X , Y ) + 2 a c c o v ( X , Z ) - t 2 b c c o v ( Y , Z )
2 (a + bxi) : na I bLxi i:l
i-
s..
|
If X, Y and Z are independent,or uncorrelated,random variables, then the covariance terms are zero and:
x1+x2+ .+xn
n
var(aX j bY + cZ)
: azvar(X)
L(r;-r):o
+b2va\v) -l c2var(Z)
23
2
2 . 2 . f ( n . y ) : I . t , f ( r i . y r) + f ( x i , y 2 )+ , f ( r i , y : ) l t:tl:l
: " f ( . r l, y r ) + f ( u , y z l + f ( x , , y t ) + f(xz,yr) + f (xz,yz)+ f(xz,yt) ExpectedValues& Variances E(X) : n f(xr) + x2f(x2)*..
I x^f(x")
Normal Probabilities Y .-
If X-N(p.o2). thenZ: "
If x-Ni*,r';
P -,v(0, t ')
anda is a.olro*,,n"r' P(x>a\-P(z>o- P\ \(r/
If X - N(p, o2) anda and, areconstants, then - P - F\ t z tb P ( a< x ' n't\-o oP/( o
: 2x, f (xi) - Lx f (x) i:l r
Ets(x)l: Isk).f(") slsr(x)+ gr(x)l : Ilg' (') + sz(x)l f Q) : Isr (x)/("r)+Lgr(x) f (x) ElcJ :
- E[e'(')l E[gz(x)] "
c
Assumptionsof the SimpleLinear RegressionModel SRI Thevalueofy, for eachvalueof.r, is y: 9' 1 B2x*e SR2 The averagevalue of the randomerror e is : 9r + Bu.r E(e) - 6s1n""*eassumethatE(y) SR3 The varianceof the randomerrore is var(e): o2 : var()) The covariance between any pair of random e l r o r s ,e r a n d e ; i s c o v ( e i , e . y ) : c o v ( ) i , y ; ) : 0 The variable x is not random and must take at least two different values. (optional) The values of e are normally distributedabout their mean a-N(0,o2)
E(cx) :68(Y1
SR4
E(a+cX):a+cE(X)
SR5
E(x)12 : E62) - lE(x)]'? var(a * cX) : E[(a + cX) - E(a + cx)]z -- c2var(X)
SR6
Marginal and Conditional Distributions
Least Squares Estimation
f (t) :2
If bl and bz Ne the least squaresestimates,then
var(x) : o2 : Elx
f G,y)
f(y) : L f(r,y)
for each valueXcan take for each value fcan take
, / . 1 . , \ -- " r f r . - . l v - . , ,
J \
'f('r'Y)
9i: bt * bzxi ?i:li-ii:y1-b1-b2x;
VI
The Normal Equations If X and r are independe", :Yl.variables, then .f(r,y) : f(x) f(y) for each and every pair of values .r and y. The converseis also true. If X and lare independentrandom variables, then the conditional probability density function ofX given that rt"\ I Q , v ) - ' !I ,t 't\! ' ' : / ( , r ) I-yrsl\xty): for ^
J\v)
J\v)
each and every pair of values "r and y. The converseis also true.
Nbt +>rib2:}yi lxibl ,'Lf
tr:2r'r'
Least Squares Estimators ,
O -) : -
bt :t
I('i
- I)(yi - 7) Z(xi - i)'
- b2t
Brief Contents Chapter 1 Chapter 2 Chapter 3 Chapter 4 Chapter 5 Chapter 6 Chapter 7 Chapter 8 Chapter 9 Chapter 10 Chapter l1 Chapter 12 Chapter 13 Chapter 14 Chapter 15 Chapter 16 Chapter 17 Appendix A Appendix B Appendix C Appendix D Appendix E
An Introduction to Econometrics The Simple Linear RegressionModel Interval Estimation and Hypothesis Testing Prediction, Goodness-of-Fitand Modeling Issues The Multiple RegressionModel Further Inference in the Multiple RegressionModel Nonlinear Relationships Heteroskedasticity Dynamic Models, Autoconelation and Forecasting Random Regressorsand Moment Based Estimation SimultaneousEquations Models Nonstationary Time-Series Data and Cointegration Vector Error Correction and Vector AutoregressiveModels: An Introduction to Macroeconometrics Time-Varying Volatility and ARCH Models: An Introduction to Financial Econometrics Panel Data Models Qualitative and Limited Dependent Variable Models Writing an Empirical ResearchReport, and Sourcesof Economic Review of Math Essentials Review of Probability Concepts Review of Statistical Inference Answers to SelectedExercises Tables
ltrt
Contents Preface Chapter I
lx
An Introduction to Econometrics
1.1 Why StudyEconometrics? I.2 What is Econometrics About? 1.2.1 SomeExamples 1.3 The EconometricModel 1.4 How Do We ObtainData? 1.4.L ExperimentalData 1.4.2 Nonexperimental Data 1.5 StatisticalInference 1.6 A ResearchFormat
Chapter 2
The Simple Linear Regression Model
LearningObjectives Keywords 2.1 An EconomicModel 2.2 An EconometricModel 2.2.1 Introducingthe Error Term 2.3 Estimatingthe RegressionParameters 2.3.1 The LeastSquaresPrinciple 2.3.2 Estimatesfor the FoodExpenditureFunction 2.3.3 Interpretingthe Estimates 2.3.3a Elasticities 2.3.3b Prediction 2.3.3c ComputerOutput 2.3.4 OtherEconomicModels 2.4 Assessingthe LeastSquaresEstimators 2.4.I The Estimatorb2 2.4.2 The ExpectedValuesof bl andb2 2.4.3 RepeatedSampling 2.4.4 The Variancesand Covarianceof b1 andb2 2.5 The Gauss-Markov Theorem 2.6 The ProbabilityDistributionsof the LeastSquaresEstimators 2.7 Estimatingthe Varianceof the Error Term of the LeastSquares 2.1J Estimatingthe Variancesand Covariances Estimators
I I 2 J i
5 5 f
6
8 8 9 9 t2 15 18 20 22 22 23 24 24 24 26 27 27 28 29 3I 32 33
34
CONTENTS
2.'7.2 Calculations for the Food Expenditure Data 2.8 Exercises 2.8.1 Problems 2.8.2 Computer Exercises Appendix 2,A'Derivation of the Least SquaresEstimates Appendix 28 Deviation from the Mean Form of b2 Appendix 2C bz is a Linear Estimator Appendix 2D Derivation of Theoretical Expression for bz Appendix 2E Deriving the Variance of b2 Appendix 2F Proof of the Gauss-Markov Theorem
Ch
35 36 36 39 42 43 44 44 45 46
Lr K A
4.
4. Chapter 3
Interval Estimation and Hypothesis Testing
Learning Objectives Keywords 3.1 Interval Estimation 3.1.1 The l-distribution 3.1.2 Obtaining Interval Estimates 3.1.3 An Illustration 3.1.4 The RepeatedSampling Context 3.2 HypothesisTests 3.2.1 The Null Hypothesis 3.2.2 The Alternative Hypothesis 3.2.3 The Test Statistic 3.2.4 The Rejection Region 3.2.5 A Conclusion 3.3 Rejection Regions for Specific Alternatives 3.3.I One-Tail Tests with Alternative "Greater Than" (>) 3.3.2 One-Tail Tests with Alternative "Less Than" (() 3.3.3 Two-Tail Tests with Alternative "Not Equal To" (l) 3.4 Examples of Hypothesis Tests 3.4.1 Right-Tail Tests 3.4.1a One-Tail Test of Significance 3.4.Ib One-Tail Test of an Economic Hypothesis 3.4.2 Left-Tail Tests 3.4.3 Two-Tail Tests 3.4.3a Two-Tail Test of an Economic Hypothesis 3.4.3b Two-Tail Test of Significance 3.5 The p-value 3.5.1 p-value for a Right-Tail Test 3.5.2 p-value for a Left-Tail Test 3.5.3 p-value for a Two-Tail Test 3.5.4 p-value for a Two-Tail Test of Significance 3.6 Exercises 3.6.1 Problems 3.6.2 ComputerExercises Appendix 3,{ Derivation of the r-Distribution Appendix 38 Distribution of the /-Statistic Under FI1
48 48 48 49 49 51 52 53 54 55 55 55 55 56 56 56 57 58 59 59 59 60 6l 62 62 63 64 65 66 66 67 68 68 69 '72 t3
4.
i I
CI
L K 5
CONTENTS
35 36 36 39 42 43 44 44 45 46
48 48 48 49 49 51 52 53 54 55 55 55 55 56 56 56
Chapter 4 Prediction, Goodness-of-Fit,and Modeling Issues
75
Learning Objectives Keywords 4.1 Least SquaresPrediction 4.1.1 Prediction in the Food Expenditure Model 4.2 Measuring Goodness-of-Fit 4.2.1 CorrelationAnalysis 4.2.2 Correlation Analysis and R2 4.2.3 The Food Expenditure Example 4.2.4 Reporting the Results 4.3 Modeling Issues 4.3.1 The Effects of Scaling the Data 4.3.2 Choosing a Functional Form 4.3.3 The Food Expenditure Model 4.3.4 Are the RegressionErrors Normally Distributed? 4.3.5 Another Empirical Example 4.4 Log-Linear Models 4.4.1 A Growth Model 4.4.2 A Wage Equation 4.4.3 Prediction in the Log-Linear Model 4.4.4 A GeneralizedR2 Measure 4.4.5 Prediction Intervals in the Loe-Linear Model 4.5 Exercises 4.5.1 Problems 4.5.2 Computer Exercises Appendix 4A Development of a Prediction Interval Appendix 48 The Sum of SquaresDecomposition Appendix 4C The Log-Normal Distribution
75 76 76 79 80 81 82 82 83 84 84 86 87 89 90 93 94 94 95 96 96 97 97 98
10r 103 r03
il
58 59 59 59 60 61 62 62 63 64 65 66 66 67 68 68 69 72 73
Chapter 5
The Multiple
Regression Model
105
Learning Objectives Keywords 5.1 Introduction 5.1.1 The Economic Model 5.I.2 The EconometricModel 5.1.2a The GeneralModel 5.1.2b The Assumptionsof the Model 5.2 Estimating the Parametersof the Multiple RegressionModel 5.2.1 Least SquaresEstimation Procedure 5.2.2 Least SquaresEstimates Using Hamburger Chain Data 5.2.3 Estimation of the Error Variance o2 5.3 Sampling Properties of the Least SquaresEstimator 5.3.1 The Variancesand Covariancesof the Least SquaresEstimators 5.3.2 The Properties of the Least SquaresEstimators Assuming Normally Distributed Errors 5.4 Interval Estimation 5.5 Hypothesis Testing for a Single Coefficient 5.5.1 Testing the Significance of a Single Coefficient
105 105 106 106 l0g 109 110 l 11 111 ll2 ll4 115 115
rt7 118 120 r20
CONTENTS
5.5.2 One-TailHypothesisTestingfor a SingleCoefficient 5.5.2a TestingFor ElasticDemand 5.5.2b TestingAdvertisingEffectiveness 5.6 MeasuringGoodness-of-Fit Results 5.6.1 Reportingthe Regression 5.7 Exercises 5.7.1 Problems 5.7.2 ComputerExercises Appendix5,A'Derivationof LeastSquaresEstimators Chapter 6
Further Inference in the Multiple Regression Model
LearningObjectives Keywords 6.1 The F-Test 6.I.I The RelationshipBetweenr- and F-Tests 6.2 Testingthe Significanceof a Model 6.3 An ExtendedModel 6.4 TestingSomeEconomicHypotheses 6.4.1 The Significanceof Advertising 6.4.2 The OptimalLevel of Advertising 6.4.2aA One-TailTestwith More than One Parameter 6.4.3 Using ComputerSoftware 6.5 The Use of NonsampleInformation 6.6 Model Specification 6.6.1 OmittedVariables 6.6.2 IrrelevantVariables 6.6.3 Choosingthe Model 6.6.3aThe RESETTest 6.7 Poor Data,Collinearity,and Insignificance 6.7.L The Consequences of Collinearity 6.7.2 An Example 6.7.3 Identifyingand Mitigating Collinearity 6.8 Prediction 6.9 Exercises 6.9.I Problems 6.9.2 ComputerExercises Appendix6,4'Chi-Squareand F-Tests:More Details Appendix68 Omitted-Variable Bias: A Proof Chapter 7 Nonlinear Relationships LearningObjectives Keywords 7.1 Polynomials 7.1.1 CostandProductCurves 7.1.2 AWage Equation 7.2 Dummy Variables 7.2.1 InterceptDummy Variables 7.2.1a Choosingthe ReferenceGroup
122 122
r23 t24 t26 127 127 129 133
7.3
134
7.4 7.5
134 135 135 138 138 140 t42 142 t42 144
7.6
r45 r46 148 149 150 151 151 153 153 154 155 156 1,57 t57 160
r63 165
Cha
Lea Ke) 8.1 8.2 8.3
8.4
8.5
166 t66
r66 r67 r67 169 170
t7r 172
Cha
Lea Ke) 9.1 9.2
CONTENTS
122 122 123 r24 126 127 127 129 t33 134 134 135 135 138 138 140 t42
r42 142 IM t45 r46 148 149 150 151 151
r53 153 154 155 156 157 157 160 r63 165 166
r66 t66 167 167 r69
r70 tll 172
7.2.2 7.2.3 '7.3 Applying 7.3.1 '7.3.2
Slope Dummy Variables An Example: The University Effect on House Prices Dummy Variables InteractionsbetweenQualitativeFactors Qualitative Factors with Several Categories 7.3.3 Testing the Equivalenceof Two Regressions 1.3.4 Controlling for Time 7.3.4a SeasonalDummies 7.3.4b Annual Dummies 7.3.4c Regime Effects 7.4 Interactions Between Continuous Variables 7.5 Log-Linear Models 7.5.1 Dummy Variables 7.5.1a A Rough Calculation 7.5.1b An Exact Calculation 7.5.2 Interaction and Quadratic Terms 7.6 Exercises 7.6.1 Problems 7.6.2 ComputerExercises Appendix 7A Details of Log-Linear Model Interpretation Chapter 8
xix
r72 t74 175
r75 177 r79 181 181 r82 r82 r82 184 185 185 185 186 186 186 190 195
r97
Heteroskedasticity
Learning Objectives Keywords 8.1 The Nature of Heteroskedasticity 8.2 Using the Least SquaresEstimator 8.3 The Generalized Least SquaresEstimator 8.3.1 Transforming The Model 8.3.2 Estimating the Variance Function 8.3.3 A HeteroskedasticPartition 8.4 Detecting Heteroskedasticity 8.4.1 ResidualPlots 8.4.2 The Goldfeld-Quandt Test 8.4.3 Testing the Variance Function 8.4.3a The White Test 8.4.3b Testing the Food Expenditure Example 8.5 Exercises 8.5.1 Problems 8.5.2 Computer Exercises Appendix 8A Properties of the Least SquaresEstimator Appendix 8B Variance Function Tests for Heteroskedasticity
197 197 l9'7 201 202 203 205 208 2tl 2tl 2tl 212 215 215 216 2t6 219 222 224
Dynamic Models, Autocorrelation
226
Chapter 9
and Forecasting
Learning Objectives Keywords 9.1 Introduction 9.2 Lags in the Error Term: Autocorrelation 9.2.1 Area ResponseModel for SugarCane
226 226 227 230 230
CONTENTS
9.2.2 First Order AutoregressiveErrors Estimating an AR(l) Error Model 9.3.1 Least SquaresEstimation 9.3.2 Nonlinear Least SquaresEstimation 9.3.2a Generalized Least SquaresEstimation 9.3.3 Estimating a More General Model 9.4 Testing for Autocorrelation 9.4.1 Residual Correlogram 9.4.2 A Lagrange Multiplier Test 9.4.3 Recapping and Looking Forward 9.5 An Introduction to Forecasting: AutoregressiveModels 9.6 Finite Distributed Lags 9.7 AutoregressiveDistributed Lag Models 9.8 Exercises 9.8.1 Problems 9.8.2 Computer Exercises Appendix 9A Generalized Least SquaresEstimation Appendix 9B The Durbin-Watson Test 9B.l The Durbin-Watson Bounds Test Appendix 9C Deriving ARDL Lag Weights 9C.1 The Geometric Lag 9C.2Lag Weights for More General ARDL Models Appendix 9D Forecasting:ExponentialSmoothing 9.3
Chapter
10
Random Regressors and Moment
Based Estimation
Learning Objectives Keywords 10.1 Linear Regressionwith Random x's 10.1.1 The Small Sample Properties of the Least SquaresEstimator l0.l.2 Asymptotic Properties of the Least SquaresEstimator: x Not Random 10.1.3 Asymptotic Properties of the Least SquaresEstimator: x Random 10.1.4 Why Least SquaresFails 10.2 Casesin Which x and e are Correlated 10.2.1 MeasurementError 10.2.2 Omitted Variables 10.2.3 SimultaneousEquations Bias 10.2.4 Lagged Dependent Variable Models with Serial Correlation 10.3 Estimators Based on the Method of Moments 10.3.1 Method of Moments Estimation of a Population Mean and Variance 10.3.2 Method of Moments Estimation in the Simple Linear RegressionModel 10.3.3 Instrumental Variables Estimation in the Simple Linear RegressionModel 10.3.3a The Importance of Using Strong Instruments 10.3.3b An Illustration Using Simulated Data 10.3.3c An Illustration Using a Wage Equation
231 235 235 236 237 237 239 239 242 243 244 248 250 253 253 255 259 26r 263 264 264 265 266
10
10
268 268 269 270 270
Chs Lei
271 272 273 274 274 275 276 276 276
278 278 279 280 281
Ke ll. I l. 11 11 I l.
I l.
ll
ll
CONTENTS
231 235 235 236 z)t zJt
239 239 242 244 248 250 253 253 255 259 261 263 264 264 265 266 268
:or ot
268 269 270 270
27r 272 zt3
2',74 a1A Lta
t
275 276 276 276 27',7 278 278 219 280 281
10.3.4 Instrumental Variables Estimation with Surplus Instruments 10.3.4a An Illustration Using SimulatedData 10.3.4b An Illustration Using a Wage Equation 10.3.5 Instrumental Variables Estimation in a General Model 10.3.5a HypothesisTestingwith InstrumentalVariables Estimates 10.3.5b Goodness-of-Fit with Instrumental Variables Estimates 10.4 Specification Tests 10.4.1 The Hausman Test for Endogeneity 10.4.2 Testing for Weak Instruments 10.4.3 Testing Instrument Validity 10.4.4 Numerical ExamplesUsing SimulatedData 10.4.4a The HausmanTest 10.4.4b Test for Weak Instruments 10.4.4c Testing Surplus Moment Conditions 10.4.5 Specification Tests for the Wage Equation 10.5 Exercises 10.5.1 Problems 10.5.2 Computer Exercises Appendix l0A Conditional and Iterated Expectations 10A.1 Conditional Expectations 10,4'.2Iterated Expectations 104,.3RegressionModel Applications Appendix 10B The Inconsistency of Least Squares Appendix 10C The Consistency of the 1V Estimator Appendix 10D The Logic of the Hausman Test
Chapter11 SimultaneousEquationsModels Learning Objectives Keywords l1.l A Supply and Demand Model lI.2 The Reduced Form Equations 11.3 The Failure of Least Squares ll.4 The Identification Problem ll.5 Two-StageLeast SquaresEstimation 11.5.1 The GeneralTwo-StageLeast SquaresEstimation Procedure 11.5.2 The Properties of the Two-Stage Least SquaresEstimator I1.6 An Example of Two-StageLeast SquaresEstimation 11.6.1 Identification II.6.2 The ReducedForm Equations 11.6.3 The Structural Equations ll.7 Supply and Demand at the Fulton Fish Market ll.7.I Identification 11.7.2 The ReducedForm Equations 11.7.3 Two-StageLeast SquaresEstimationof Fish Demand 11.8 Exercises 11.8.1 Problems
xxl
282 284 284 285 286 286 286 287 288 289 290 290 290 29r 291 292 292 293 297 297 298 298 299 300 301 303 303 303 304 306 307 307 309 310 311 311
3r2 312 313 314 3r5 315 3r7 318 318
CONTENTS
11.8.2 ComputerExercises Appendix 11A An AlgebraicExplanationof the Failureof LeastSquares
Chapter 12 NonstationaryTime-SeriesData and Cointegration LearningObjectives Keywords I2.l Stationaryand NonstationaryVariables l2.l.I The First-OrderAutoregressive Model 12.1.2 RandomWalk Models L2.2 SpuriousRegressions 12.3 Unit Root Testsfor Stationarity 12.3.1 Dickey-FullerTest I (No Constaatand No Trend) 12.3.2 Dickey-FullerTest2 (With ConstantBut No Trend) 12.3.3 Dickey-FullerTest3 (With ConstantandWith Trend) 12.3.4 The Dickey-FullerTestingProcedure 12.3.5 The Dickey-FullerTests:An Example 12.3.6 Orderof Intecration 12.4 Cointegration 12.4.1 An Exampleof a CointegrationTest 12.5 Regression WhenThereis No Cointegration 12.5.1 First DifferenceStationary 12.5.2 TrendStationarv 12.6 Exercises 12.6.I hoblems 12.6.2 ComputerExercises
Chapter 13
VEC and VAR Models: An Introduction to Macroeconometrics
LearningObjectives Keywords 13.1 VEC and VAR Models 13.2 Estimatinga VectorError CorrectionModel 13.2.1 Example 13.3 Estimatinga VAR Model 13.4 ImpulseResponses and VarianceDecompositions 13.4.1 ImpulseResponse Functions 13.4.laThe UnivariateCase l3.4.lb The BivariateCase 13.4.2 ForecastError VarianceDecompositions 13.4.2aUnivariateAnalysis 13.4.2bBivariateAnalysis 13.4.2cThe GeneralCase 13.5 Exercises 13.5.1 Problems 13.5.2 ComputerExercises Appendix 13AThe IdentificationProblem
319
323
32s 325 325 326 328
Cha
Lea Ke) 14 t4. t4.
33r 333 335 335
t4.
33s 336 336 337 338 339 340 340
34r 342 342 342 344
14
Cha
Lea Ke) 15 15 15
15 346 346 346 347 349 349 351 352 352 352 353 355 355 356 357 35'7 357 358 36r
15
15
CONTE NTS
319
Chapter 14
Time-Varying Volatility and ARCH Models: An Introduction to Financial Econometrics
3ZJ
32s 325 325 326 328
33r JJJ
335 335 335 336 336 337 338 339 340 340 341 342 342 342 344
346 346 346 34'l 349 349 351 352 352 352 353 35s 355 356 357 357 3)l
358
36r
Learning Objectives Keywords l4.l The ARCH Model I4.I.l Conditional and UnconditionalForecasts 14.2 Time-Varying Volatility 14.3 Testing, Estimating and Forecasting 14.3.1 Testing for ARCH Effects 14.3.2 EstimatingARCH Models 14.3.3 ForecastingVolatility 14.4 Extensions 14.4.1 The GARCH Model-Generalized ARCH 14.4.2 Allowing for an Asymmetric Effect 14.4.3 GARCH-in-Mean and Time-Varying Risk Premium 14.5 Exercises 14.5.1 Problems 14.5.2 Computer Exercises
Chapter 15
Panel Data Models
Learning Objectives Keywords 15.1 Grunfeld's InvestmentData 15.2 Setsof RegressionEquations 15.3 Seemingly Unrelated Regressions 15.3.1 Separateor Joint Estimation? 15.3.2 Testing Cross-EquationHypotheses 15.4 The Fixed Effects Model 15.4.1 A Dummy Variable Model 15.4.2 The Fixed Effects Estimator 15.4.3 Fixed Effects Estimation Usins a Microeconomic Panel 15.5 The Random Effects Model 15.5.1 Error Term Assumptions 15.5.2 Testing for Random Effects 15.5.3 Estimation of the Random Effects Model 15.5.4 An Example Using the NLS Data 15.5.5 Comparing Fixed and Random Effects Estimators 15.5.5a Endogeneityin the Random Effects Model 15.5.5b The Fixed Effects Estimator in a Random Effects Model 15.5.5c A HausmanTest 15.6 Exercises 15.6.1 Problems 15.6.2 ComputerExercises Appendix l5A Estimation of Error Components
363 363 363 364 365 365 369 369 369 370 371 371 3tz Jt+
375 375 376
382 382 382 384 385 387 389 390 391 391 393 396 398 399 400 401 402 403 403 404 404 406 406 408 4t5
CONTENTS
Chapter 1.6 Qualitative and Limited DependentVariable Models LearningObjectives Keywords 16.1 Modelswith Binary DependentVariables 16.1.1 The Linear ProbabilityModel 16.1.2 The ProbitModel 16.1.3 Interpretationof the ProbitModel 16.1.4 MaximumLikelihoodEstimationof the ProbitModel 16.1.5 An Example 16.2 The Logit Model for Binary Choice 16.3 Multinomial Logit 16.3.1 Multinomial Logit ChoiceProbabilities 16.3.2 MaximumLikelihoodEstimation 16.3.3 Post-Estimation Analysis 16.3.4 An Example 16.4 ConditionalLogit 16.4.1 ConditionalLogit ChoiceProbabilities 16.4.2 Post-Estimation Analysis 16.4.3 An Example 16.5 OrderedChoiceModels 16.5.1 OrdinalProbit ChoiceProbabilities 16.5.2 Estimationand Interpretation 16.5.3 An Example 16.6 Modelsfor CountData 16.6.I MaximumLikelihoodEstimation 16.6.2 Interpretationin the PoissonRegression Model 16.6.3 An Example 16.7 Limited DependentVariables 16.7.1 Censored Data 16.7.2 A Monte Carlo Experiment 16.7.3 MaximumLikelihoodEstimation L6.7.4 Tobit Model Interpretation 16.7.5 An Example 16.7.6 SampleSelection 16.7.6aThe EconometricModel 16.7.6bHeckit Example:Wagesof Manied Women 16.8 Exercises Chapter 17 l7.l
Writing an Empirical Research Report, and Sources of Economic Data
Selectinga Topic for an EconomicsProject 17.I.I Choosinga Topic l7.L2 Writing an Abstract l'7.2 A Format for Writing a ResearchReport 17.3 Sourcesof EconomicData I7.3.1 Links to EconomicData on the Internet 17.3.2 TraditionalSourcesof EconomicData 17.3.3 InterpretingEconomicData 17.4 Exercises
417 417 4t7 418 419 421 422 423 424 425 426 427 427 428 429
App
Lea Ke) A.1 A.2
A.3
A.4
43r 431 432 433 433 434 435 437 437 438 439 440 441 44r M2 444 445 446 447 448 449 450
A
-{pp
l*a Key B.l 8.2 B.3
8.4
B.5
457 457 457 458 458 460 460 461 461 462
8.6
App l*a Kel
CONTENTS
417 477 417 418 419 421 422 423 424 425 426 427 427 428 429 431 431 +32
433 433 434 435 437 437 438 439 440 441
44r 442 444 445 446 447 448 449 450
457 451 457 458 458 460 460 461 461 462
Appendix
A Review of Math Essentials
Learning Objectives Keywords A.1 Summation A.2 Some Basics 4.2.1 Numbers 4.2.2 Exponents A.2.3 Scientific Notation 4.2.4 Logarithms and the Number e A.3 LinearRelationships A.3.1 Elasticity 4.4 Nonlinear Relationships 4.4.1 QuadraticFunction 4.4.2 Cubic Function 4.4.3 Reciprocal Function 4.4.4 Log-Log Function 4.4.5 Log-Linear Function 4.4.6 Approximating Logarithms A.4.7 Approximating Logarithms in the Log-Linear Model A.4.8 Linear-LoeFunction A.5 Exercises
Appendix B Review of Probability Concepts Learning Objectives Keywords B.1 Random Variables 8.2 Probability Distributions 8.3 Joint, Marginal and Conditional Probability Distributions B.3.1 Marginal Distributions 8.3.2 Conditional Probability B.3.3 A Simple Experiment 8.4 Properties of Probability Distributions 8.4.1 Mean, Median and Mode 8.4.2 Expected Values of Functions of a Random Variable 8.4.3 Expected Values of Several Random Variables 8.4.4 The Simple Experiment Again B.5 Some Important Probability Distributions 8.5.1 The Normal Distribution 8.5.2 The Chi-Square Distribution B.5.3 The r-Distribution 8.5.4 The F-Distribution 8.6 Exercises \ppendix
C Review of Statistical Inference
xxr)
463 463 463 464 465 465 466 466 466 468 469 470 471 471 472 473 473 Aaa
+tJ
474 475 476 478 478 479 479 480 483 484 484 486 487 +6 I
488 490 492 493 493 495 495 496 497
501
Learning Objectives
s0l
Keywords
502
CONTENTS
c.1 c.2 c.3
c.4
c.5
c.6
c.7
c.8
A Sample of Data An Econometric Model Estimating the Mean of a Population C.3.1 The ExpectedValue of Y C.3.2 The Varianceof Y C.3.3 The Sampling Distribution of Y C.3.4 The Central Limit Theorem C.3.5 Best Linear UnbiasedEstimation Estimating the Population Variance and Other Moments C.4.I Estimating the Population Variance C.4.2 Estimating Higher Moments C.4.3 The Hip Data C.4.4 Using the Estimates lnterval Estimation C.5.1 Interval Estimation: o2 Known C.5.2 A Simulation C.5.3 Interval Estimation: o2 Unkno*n C.5.4 A Simulation (Continued) C.5.5 Interval Estimation Using the Hip Data Hypothesis Tests About a Population Mean C.6.1 Componentsof HypothesisTests C.6.la The Null Hypothesis C.6.lb The Alternative Hypothesis C.6.1c The Test Statistic C.6.ld The Rejection Region C.6.le A Conclusion C.6.2 One-Tail Tests with Alternative "Greater Than" (>) C.6.3 One-Tail Tests with Alternative "Less Than" (<) C.6.4 Two-Tail Tests with Alternative "Not Equal To" (l) C.6.5 Example of a One-Tail Test Using the Hip Data C.6.6 Example of a Two-Tail Test Using Hip Data C.6.7 The p-Value C.6.8 A Comment on Stating Null and Alternative Hypotheses C.6.9 Type I and Type II Errors C.6.10 A Relationship Between Hypothesis Testing and Confidence Intervals Some Other Useful Tests C.7.I Testing the Population Variance C.7.2 Testing the Equality of Two Population Means C.7.3 Testing the Ratio of Two Population Variances C.7.4 Testing the Normality of a Population Introduction to Maximum Likelihood Estimation C.8.1 Inference with Maximum Likelihood Estimators C.8.2 The Variance of the Maximum Likelihood Estimator C.8.3 The Distribution of the Sample Proportion C.8.4 Asymptotic Test Procedures C.8.4a The Likelihood Ratio (ZR) Test C.8.4b The Wald Test C.8.4c The Lagrange Multiplier (LI[) Test
502 504 504 506 506 507 508 509 509 510 511 511 5t2 5t2 513 5t4 515 5t7 5r7 5r'1
5r7 518 518 518 519 519 519 519 520 520 521 522 523 524 525 525 525 526 527 527 528 532 533 534 536 536 538 539
c.9
c.l0
-{ppe
Appe
Tabl Tabk Tabk Tablt Tabl lnder
CONTENTS
502 504 504 506 506 507 508 509 509 510 51i 511 512 512 513 514 515 5t7 511 517
5r7 518 518 518 519 519 519 519 520 520 521 522 523 524 525 525 525 526 52',7 527 528 s32 533 534 536 536 538 539
Algebraic Supplements(Optional) C.9.1 Derivation of Least SquaresEstimator C.9.2 Best Linear UnbiasedEstimation C.l0 Exercises C.9
xtcuii
541 541 543 544
AppendixD Answersto SelectedExercises
s48
AppendixE Tables
572
Table I Table 2 Table 3 Table 4 Table 5 lndex
Cumulative Probabilities for the Standard Normal Distribution Percentilesfor the r-Distribution Percentilesfor the Chi-square Distribution 95th Percentile for the F-Distribution 99th Percentile for the F-Distribution
572 573 574 575 576 3t I
Chapter
An Introduction to Econornetrics
7,7 Why Study Econometrics? The importance of econometricsextendsfar beyond the discipline of economics. Econometrics is a set of researchtools also employedin the businessdisciplinesof accounting, finance,marketing, and management.It is usedby social scientists,specifically researchers in history, political science,and sociology. Econometrics plays an important role in such diverse fields as forestry and in agricultural economics. This breadth of interest in econometricsarises in part becauseeconomicsis the foundation of businessanalysis. Thus researchmethods employed by economists,which include the field of econometrics, are useful to a broad spectrum of individuals. Econometrics plays a special role in the training of economists. As a student of economics, you are learning to "think like an economist." You are learning economic conceptssuch as opportunity cost, scarcity, and comparative advantage.You are working with economic models of supply and demand,macroeconomicbehavior, and international trade.Through this training you becomea personwho betterunderstandsthe world in which we live; you become someonewho understandshow markets work, and the way in which government policies affect the marketplace. If economicsis yourmajororminorfield of study,a wide rangeof opportunitiesis opento you upon graduation.If you wish to enter the businessworld, your employer will want to know the answerto the question,"What can you do for me?" Studentstaking a traditional economicscurriculumanswer,"I can think like an economist."While we may view sucha responseto be powerful, it is not very specificand may not be very satisfying to an employer who does not understandeconomics. The problem is that a gap existsbetweenwhat you have learnedas an economicsstudent and what economists actually do. Very few economists make their livings by studying economic theory alone, and those who do are usually employed by universities. Most economists,whether they work in the businessworld or for the government, or teach in universities,engagein economicanalysisthat is in part "empirical." By that,we meanthat they useeconomicdatato estimateeconomicrelationships,testeconomichypotheses,and predict economicoutcomes. Studyingeconometricsfills a gapbetweenbeing "a studentof economics"andbeing "a practicingeconomist."With the econometricskills you will learnfrom this book, including how to work with econometricsoftware,you will be able to elaborateon your answerto the employer'squestionabove by saying "I can predict the salesof your product." "I can estimatethe effect on your salesif your competitionlowersits price by $l per unit." "I can
AN INTRODUCTION
TO ECONOMETRICS
test whether your new ad campaign is actually increasing your sales." These answersare music to an employer's ears,becausethey reflect your ability to think like aneconomistand to analyzeeconomic data. Such pieces of information are keys to good businessdecisions. Being able to provide your employer with useful information will make you a valuable employee and increaseyour odds of getting a desirablejob. Ifyou plan to continueyour educationby enrolling in graduateschool,or law school,you will find that this introduction to econometricsis invaluable.If your goal is to earna master's or Ph.D. degree in economics, finance, accounting, marketing, agricultural economics, sociology, political science, or forestry, you will encounter more econometrics in your future. The graduate coursestend to be quite technical and mathematical, and the forest often gets lost in studying the trees. By taking this introduction to econometricsyou will gain an overview of what econometricsis about and develop some "intuition" about how things work, before entering a technically oriented course.
7.2 Vhat Is Econornetrics About? At this point we needto describethe nafureof econometrics.It all begins with a theory from your field of study,whether it is accounting,sociology, or economics,about how important variablesare related to one another.In economicswe expressour ideas about relationships betweeneconomic variablesusing the mathematicalconcept of a function. For example,to expressa relationship between income and consumption, we may write CoNSUMPTION : f (INCOME) which says that the level of consumption is some function,/(.), of income. The demandfor an individual commodity, say the Honda Accord, might be expressedas Qd:f(P,
tr, P',INC)
whichsaysthatthequantityofHondaAccordsdemanded,Ec,isafunction/(P,F,P,INC) of the price of Honda Accords P, the price of cars that are substitutesP", the price of items that are complementsP', like gasoline, and the level of income /NC. The supply of an agricultural commodity such as beef might be written as
7.2.1
Asac the Fe (FRB) mustr so by discou Increa in othr funds autom consu aggre! are su Thr discou answe ratesa and mr and m policie Ecc "Gooc
as the conse Eve Bernar a)
a ol
\
a : f l p , P , ,P r ) where Q" is the quantity supplied, P is the price of teef, P' is the price of competitive productsin production (e.g., the price of hogs), and It is the price of factors or inputs (e.g., the price of corn) used in the production process. Each of the aboveequationsis a generaleconomicmodel that describeshow we visualize the way in which economic variables are interrelated.Economic models of this type guide our economicanalysis, For most economic decision or choice problems, it is not enough to know that certain economic variablesare interrelated,or eventhe direction of the relationship.In addition, we must understandthe magnitudes involved. That is, we must be able to say how much a chanse in one variable affects another.
I ol t o!
) .l
i i ol
i I Econometricsis abouthow we canusetheoryand datafrom economics,business,andthe I social sciences,along with tools from statistics,to answer"how much" type questions.
a
.l
1.2 WHAT
answersare ofuonistand ls decisions. r a valuable / school,you rn a master's economics, .rics in your rd the forest ics you will " abouthow
Ltheory from rw important relationships : example,to
expressedas
,Pt,P',INC) rriceof items IS
'competitive r inputs (e.g., 'we visualize
1.2.1
IS ECONOMETRICS
ABOUT?
SovrE Examprns
As a casein point, considerthe problem faced by a central bank. In the United States,this is the FederalReserveSystem,with Ben Bernankeas Chairman of the FederalReserveBoard (FRB). When prices areobservedto rise, suggestingan increasein the inflation rate,the FRB must make a decision about whether to dampenthe rate of growth of the economy.It can do so by raising the interest rate it chargesits member banks when they borrow money (the discount rate) or the rate on overnight loans between banks (the federal funds rate). Increasingtheserates will then senda ripple effect through the economy causing increases in other interestrates,such asthose faced by would-be investors,who may be firms seeking tunds for capital expansion or individuals who wish to buy consumer durables like automobiles and refrigerators. This has the economic effect of increasing costs, and consumerswould react by reducing the quantity of the durable goods demanded.Overall, aggregatedemandfalls and this is expectedto slow the rate of inflation. Theserelationships are suggestedby economic theory. The real question facing Chairman Bernanke is "How muchshotld we increase the discount rate to slow inflation, and yet maintain a stable and growing economy?" The answerwill dependon the responsivenessof firms and individuals to increasesin the interest ratesand to the effectsof reducedinvestmenton GrossNational Product.The key elasticities and multipliers are called parameters. The values of economic parametersare unknown and must be estimated using a sample of economic data when formulating economic policies. Econometricsis abouthow to bestestimateeconomicparametersgiven the datawe have. "Good" econometricsis important, sinceerrorsin the estimatesusedby policy makerssuch a: the FRB may lead to interest rate corrections that are too large or too small, which has .-onsequences for all of us. Every day decisionmakersface "how much" questionssimilar to thosefacing Chairman Bernanke.Some examplesmight include the following: A city council pondersthe question of how much violent crime will be reducedif an additional million dollars is spent putting uniformed police on the street. U.S. PresidentialcandidateClinton questionshow many additional California voters will supporther if she spendsan additional million dollars in advertisingin that state. The owner of a local PizzaHutfranchise must decide how much advertising spaceto purchasein the local newspaper,and thus must estimate the relationship between advertising and sales. Louisiana StateUniversity must estimatehow much enrollment will fall if tuition is raisedby $100 per semester,and thus whetherits revenuefrom tuition will rise or fall.
istype guide
The CEO of Proctor & Gamble must estimatehow much demandthere will be in 10 years for the detergentTide, and how much to invest in new plant and equipment.
v that certain iaddition,we how much a
A real estate developer must predict by how much population and income will increaseto the southofBaton Rouge,Louisiana, over the next few years,and ifit will be profitable to begin construction of a new strip mall.
ress,andthe l questions.
You must decide how much of your savingswill go into a stock fund and how much into the money market.This requiresyou to make predictionsof the level of economic activity, the rate of inflation, and interest rates over your planning horizon. A public transportationcouncil in Melbourne, Australia, must decidehow an increase in fares for public transportation(trams, trains, and buses)will affect the number of
I.4 HOW DO WE OBTAIN DATA?
regoing to
rovidedby heory and L on these rnderlying rles obtain oard has a & Gamble )u may get onometric lhe source here is an yield the
In every econometricmodel, whetherit is a demandequation,a supply equation,or a productionfunction,thereis a systematicportion and an unobservablerandomcomponent. The systematic portion is the part we obtain from economic theory, and it includes an assumptionabout the functional form. The random component representsa "noise" component,which obscuresour understandingof the relationshipamong variables,and which we representusing the random variablee.
7,4 How Do
'We
Obtain Data?
Where does data come from? Economistsand other social scientistswork in a complex world in which data on variables are "observed" and rarely obtained from a controlled experiment.This makes the task of leaming about economic parametersall the more difficult. Proceduresfor usingsuchdatato answerquestionsof economicimportancearethe subjectmatter of this book.
model and 1.4.7
r a general rreyou are tric model ; not claim describes g car sales c part and . Thus an
itted from ivity. iomething rample,in n of price. rart of the
variables. )mpatible
ExpsRrrurNrar Dere
Oneway to acquireinformation aboutthe unknown parametersof economic relationshipsis to conduct or observe the outcome of an experiment. In the physical sciences,and agriculture,it is easy to imagine controlledexperiments.Scientistsspecify the valuesof key control variablesand then observethe outcome.We might plant similar plots of land with a particular variety of wheat, and then vary the amounts of fertilizer and pesticide applied to each plot, observing at the end of the growing seasonthe bushelsof wheat producedon eachplot. Repeatingthe experimenton Nplots ofland createsa sampleofN observations.Such controlled experimentsare extremely rare in businessand the social sciences.The key aspectofexperimentaldatais that the valuesofthe explanatoryvariables can be fixed at specificvaluesin repeatedtrials of the experiment. One businessexamplecomesfrom marketingresearch.Supposewe areinterestedin the weekly salesof a particularitem at a supermarket.As an item is sold it is passedover a scanningunit to record the price and the amountthat will appearon your grocery bill. But, at the sametime a datarecord is created,and at every point in time the price of the item and the pricesof all its competitorsare known, as well as currentstoredisplaysand couponusage. The prices and shoppingenvironmentare controlledby storemanagement,and thus this "experiment" can be repeateda number of days or weeks using the samevaluesof the "control" variables.
1.4.2
NonnxprsrurNTAl
DATA
An exampleof nonexperimentaldata is survey data. The Public Policy ResearchLab at Louisiana State University (http://www.survey.lsu.edu/) conducts telephone and mail surveysfor clients. In a telephone survey numbers are selectedrandomly and called. Responsesto questionsare recordedand analyzed.In such an environmentdata on all variablesarecollectedsimultaneouslyandthe valuesareneitherfixed nor repeatable.These are nonexperimentaldata. Datathat arecollectedmay be of severalforms.As we analyzethe datatheform in which they arecollectedaffectswhich methodologicalissueswe may encounter.The datamay be collectedin a
AN INTRODUCTION
TO ECONOMETRICS
time-seriesform-data collected over discrete intervals of time-for example, the annual price ofwheat in the United Statesfrom 1880 to 2007, or the daily price of General Electric stock from 1980 to 2007. cross-sectionform-data collected over sample units in a particular time periodfor example,income by countiesin California durin920O6,or high schoolgraduation rates by state in 2006. panel data form-data that follow individual microunits over time. Such data are increasingly available. For example, the U.S. Department of Education has several ongoing surveys,in which the samestudentsaretrackedover time, from the time they are in the 8th gradeuntil their mid-twenties. Thesedatabasesrecord not only student characteristicsand performance,but also the socioeconomiccharacteristicsoftheir families, their schools,and so on. Suchdata arerich sourcesofdata for studiesrelated to labor economics, economics of the household, health economics, and of course education.
Ecc thr( cha
1.
Em thrt
These data may be collected at various levels of aggregation: micro-data collected on individual economic decision-making units such as individuals. households.or flrms. macro-data resulting from a pooling or aggregatingover individuals, households, or firms at the local, state, or national levels. The data collected may also representa flow or a stock: flow-outcome measuresover a period of time, such asthe consumptionof gasoline during the last quarter of 2006. stock----oulcomemeasuredat a particular point in time, such as the quantity of crude oilheldbyExxoninitsU.S. storagetanksonApril1,2O06,ortheassetvalueofthe Wells Fargo Bank on Jily I,2007. The data collected may be quantitative or qualitative: quantitative-outcomes such as prices or income that may be expressedasnumbers or some transformation of them, such as real prices or per capita income. qualitative-outcomes that areof an "either--or" situation.Forexample, aconsumer either did or did not make a purchaseof a particular good, or a personeither is or is not married. In Chapter 17 we list sourcesof economic datathat are availableboth in print, or electronic media, and on the Internet.
7. 5 Statistical Inference The phrasestatistical inference will appearoften in this book. By this we mean we want to "infer" or learn somethingabout the real world by analyzinga sampleof data.The ways in which statistical inference are carried out include .
Estimating economic parameters,such as elasticities, using econometric methods.
Th reI Gt
1.6 A RESEARCH FORMAT
mple, the / price of periodraduation r data are rs several time they y student ,sof their esrelated of course
h as indi useholds,
gasoline r of crude lue of the
.
Predicting economic outcomes,such as the enrollment in 2-year colleges in the United Statesfor the next l0 years.
e Testingeconomichypotheses, suchasthe questionof whethernewspaperadvertising is better than storedisplaysfor increasingsales. Econometrics includes all of these aspects of statistical inference, and as we proceed through this book you will learn how to properly estimate, predict, and test, given the characteristicsof the data at hand.
1.6 A Research Fortnat Empirical economicresearchfollows a pattern,and we will stressthis orderly procedure throughout the book. The steps are as follows: 1. It all startswith a problemor question.The idea may come afterlengthy studyof all that has been written on a particular topic. You will find that "inspiration is 99Vo perspiration." That means,after you dig at a topic long enough,a new and interesting questionwill occur to you. Alternatively, you may be led by your natural curiosity to an interestingquestion.ProfessorHalVarian("How to Build an EconomicModel in Your SpareTime," TheAmerican Economist, 4l (2), Fall 1997,pp. 3-10) suggests you look for ideasoutsideacademicjournals-in newspapers, magazines,andso on. He relatesa story about a researchproject that developedfrom his shoppingfor a new TV set. 2. Economic theory gives us a way of thinking about the problem: Which economic variablesareinvolved and what is the possibledirection ofthe relationship(s)?Every researchproject,given the initial question,beginsby building an economicmodel and listing the questions(hypotheses)of interest.More questionswill occur during the researchproject, but it is good to list those that motivate you at the beginning of the project. 3. The working economic model leads to an econometric model. We must choose a functional form and make some assumptionsabout the nature of the error term.
numbers )onsumer s or is not
,lectronic
e want to o ways in
nethods.
4. Sampledata are obtainedand a desirablemethod of statisticalanalysisis chosen, based on our initial assumotionsand our understandinsof how the data were collected. 5. Estimates of the unknown parametersare obtained with the help of a statistical software package,predictions are made, and hypothesis tests are performed. 6. Model diagnosticsareperformed to check the validity of assumptionswe havemade. For example,were all of the right-hand-sideexplanatoryvariablesrelevant?Was the correct functional form used? 7. The economic consequencesand the implications of the empirical results are analyzedand evaluated.What economic resourceallocation and distribution results are implied, and what are their policy-choiceimplications?What remainingquestions might be answeredwith further study or new and better data? This path is one we will follow in the following chapters.At the end of the work, a paperor report must be written, summarizing what you have found and the methods you used. Guidelinesfor writing economicspapersare -eivenin Chapter 17.
Chapter 2
Ke)
ASSUI:
The Simple Linear Regression Model
asym BLU biase degre depe devia for econ econ elast
Learning Objectives REMARK: Leaming Objectives andKeywords sectionswill appearat the beginning of eachchapter.We urgeyou to think about,and possiblywrite out answersto the questions, and make sureyou recognizeand can definethe keywords.If you are unsureabout the questionsor answersconsult your instructor.When examplesare requestedin lzarning Objectivessections,you shouldthink of examplesnot in the book.
Based on the material in this chapter you should be able to l.
Explain the difference between an estimator and an estimate, and why the least squaresestimatorsare random variables, and why least squaresestimatesare not.
2. Discuss the interpretation of the slope and intercept parameters of the simple regressionmodel, and sketch the graph of an estimated equation. 3. Explain the theoretical decompositionof an observablevariable y into its systematic and random components,and show this decomposition graphically. 4.
Discussand explain each of the assumptionsof the simple linear regressionmodel.
5. Explain how the least squaresprinciple is usedto fit a line through a scatterplot of data.Be able to define the least squaresresidual and the least squaresfitted value of the dependentvariable and show them on a graph. 6. Define the elasticity of y with respecttox and explain its computation in the simple linear regressionmodel when y and x are not transformedin any way, and when y and x are transformed by taking the natural logarithm. 7. Explain the meaning of the statement"If regressionmodel assumptionsSR1-SR5 hold, then the least squaresestimator b2 is unbiased." In particular, what exactly does "unbiased" mean?Why is b2biasedif an important variable hasbeenomitted from the model. 8. Explain the meaning of the phrase "sampling variability." 9. Explain how the factors o2, I(xi - x)2, andNaffect the precisionwith which we can estimate the unknown parameter B2. 10. State and explain the Gauss-Markov theorem.
Econ nomi supp prodr an ir funcl on th leve Ei chap ship price quan varie thes base tions appr
2.1
In o imp< betu rand cons supp this inte wee valu ans
2.1 AN ECONOMIC
MODEL
Keywords assumptions asymptotic BLUE biasedestimator de-qreesof freedom Jependentvariable Jeviation from the mean tbrm econometric model economic model elasticity
;r
Jor,I
t"l ,1
least e not. imple lematodel. 'lot of lue of imple hen y -SR5 lactly dtted
Gauss-Markov theorem heteroskedastic homoskedastic independentvariable least squaresestimates least squaresestimators least squaresprinciple least squaresresiduals linear estimator prediction random error term
regressionmodel regressionparameters repeatedsampling sampling precision sampling properties scatter diagram simple linear regressionfunction specification error unbiased estimator
Economic theory suggestsmany relationships between economic variables. In microeconomics you considereddemand and supply models in which the quantities demandedand supplied ofa good dependon its price. You considered "production functions" and "total productcurves" that explainedthe amountofa goodproducedasa function ofthe amountof an input, such as labor, that is used. In macroeconomics you specified "investment functions" to explain that the amount of aggregateinvestment in the economy depends on the interestrate and "consumption functions" that related aggregateconsumptionto the level of disposableincome. Each of these models involves a relationship between economic variables. In this chapter we consider how to use a sample of economic data to quantify such relationships. As economistswe are interestedin questionssuch as: If one variable (e.g., the price of a good) changes in a certain way, by how much will another variable (the quantity demandedor supplied) change?Also, given that we know the value of one variable,canweforecast or predict the correspondingvalue of another?We will answer thesequestionsby using a regression model. Like all models the regressionmodel is based on assumptions.In this chapter we hope to be very clear about these assumptions, as they are the conditions under which the analysis in subsequentchapters is appropriate.
2.7 An Econornic Model In order to develop the ideas of regression models we are going to use a simple, but important, economic example. Supposethat we are interestedin studying the relationship between household income and expenditure on food. Consider the "experiment" of randomly selecting households from a particular population. The population might consist of householdswithin a particular city, state,province, or country. For the present, supposethat we are interestedonly in householdswith an income of $1000 per week. In this experiment we randomly select a number of households from this population and interview them. We ask the question, "How much did you spendper person on food last week?" Weekly food expenditure, which we denote as y, is a random variable since the value is unknown to us until a household is selectedand the question is asked and answered.
10
THE SIMPLE LINEAR
REGRESSION MODEL
REMARK: In AppendicesB and C we distinguishedrandom variablesfrom their values by using uppercase(Y) letters for random variables and lowercase (y) letters for their values. We will not make this distinction any longer becauseit leads to complicated notation.We will use lowercaseletters,like "y," to denoterandom variablesas well as their values,and we will make the interpretationclear in the surroundingtext. The continuous random variable y has a probability density function (which we will abbreviate as pdJ) that describesthe probabilities of obtaining various food expenditure values.If you are rusty or uncertain aboutprobability conceptsseeAppendix B at the end of this bookfor a comprehensivereview.Clearly, the amount spenton food per personwill vary from one householdto anotherfor a variety of reasons:somehouseholdswill be devotedto gourmet food, somewill contain teenagers,somewill contain senior citizens, somewill be vegetarian,and somewill eat at restaurantsmore frequently. All of thesefactors and many others,including random, impulsive buying, will causeweekly expenditureson food to vary from one householdto another,evenif they all havethe sameincome.The pdf f (y) describes how expendituresare "distributed" over the population and might look like Figure 2.1. T\e pdf in Figure 2.lais actually a conditional probability density function since it is "conditional" upon householdincome.If .r: weekly householdincome : $1000,thenthe conditionalp df is f Qlx : $ 1000).The conditionalmean,or expectedvalue,of y is E(ylx : $1000) : Fy[ and is our population's mean weekly food expenditureper person. REMARIk The expectedvalue of a randomvariableis called its "mean" value,which is really a contraction of population mean, the center of the probability distribution of the randomvariable.This is not the sameas the samplemean,whichis the arithmeticaverage of numericalvalues.Keepthe distinctionbetweenthesetwo usagesof the term "mean" in mind. The conditionalvariance of y is var(ylx : $1000) : o2, which measuresthe dispersionof householdexpendituresy about their mean Fyp. The parametersFy[ and o2, if they were known, would give us somevaluable information about the population we are considering. If we knew these parameters,and if we knew that the conditional distribution fjlx : $1000) was normal, N(Fyp,o2;, then we could calculate probabilities that y falls in specific intervals using properties of the normal distribution. That is, we could compute the proportion ofthe householdpopulation that spendsbetween $50 and $75 per personon food, given $1000 per week income. fuF = 1000) fup = 2000)
fup= looo)
Fylx
[yll000
Fyl2000
\a)
prcunr 2 .7 (a) Probabilitydistributionf llx :1000) of foodexpenditure y givenincome y givenincomes.r: $1000and x: $1000.(b) Probabilitydistributions offood expenditure .r : $2000.
At varia and -r good "indt expe Ar impo week How hous valut
Fl
heal pred whe plan expe an ir Ir econ quar theo matl wec onf expc pdfl exp( inco l cont the s exp(
2. ] AN ECONOMIC
r their values ers for their complicated :s as well as (t.
hich we will I expenditure 9 at the end of rson will vary be devotedto somewill be ors and many n food to vary rfv) describes : Figure2.1. ion sinceit is 000, then the rfyisE(ylx: trson. ralue,which rution of the etic average I "mean" in
dispersionof , if they were considering. tion /(ylx: .at y falls in ,uld compute )er person on
000)
\ \
00)
)n mcome )00 and
MODEL
11
As economists we are usually more interested in studying relationships between variables, in this case the relationship between y : weekly food expenditure per person and r : weekly householdincome. Economic theory tells us that expenditureon economic goods dependson income. Consequently we call y the "dependent variable" and x the "independent" or "explanatory" variable. In econometrics,we recognize that real-world expendituresare random variables,and we want to use data to learn about the relationship. An econometric analysis of the expenditurerelationship can provide answersto some important questions,such as: If weekly income goes up by $100, how much will average weekly food expendituresrise? Or, could weekly food expendituresfall as income rises? How much would we predict the weekly per person expenditure on food to be for a household with an income of $2000 per week? The answers to such questions provide valuable information for decision makers. Using. . . per personfood spendinginformation. . . one can determinethe similarities of dffiring sirys, races,incomes, and disparitiesin the spendinghabitsof households geographic areas, and other socioeconomicand demographicfeatures. This informationis valuablefor assessingexistingmarketconditions,product distribution patterns, consumerbuying habits, and consumerliving conditions.Combinedwith demographicand incomeprojections,this information may be used to anticipate consumptiontrends. The informationmay also be used to developtypical market basketsoffood for specialpopulationgroups,suchas the elderly.Thesemarketbaskets pattemsof may,in turn, be usedto developprice indicestailoredto the consumption thesepopulation groups. lBlisard, Noel, Food Spendingin American Households, 1997-1998,ElectronicReportfrom the EconomicResearchService,U.S.Department of Agriculture,StatisticalBulletinNumber972,June20011 From a businessperspective,if we are managersof a supermarketchain (or restaurant,or health food store, etc.) we must consider long-range plans. If economic forecastersare predicting that local income will increase over the next few years, then we must decide whether to expand,and how much to expand,our facilities to serveour customers.Or, if we plan to open franchisesin high-income and low-income neighborhoods,then forecastsof expenditureson food per person,along with neighborhooddemographicinformation, give an indication of how large the storesin those areasshould be. In order to investigatethe relationshipbetweenexpenditureand income we must build an economic model and then a correspondingeconometric model that forms the basis for a quantitative or empirical economic analysis. In our food expenditure example, economic theory suggeststhat averageweekly perpersonhouseholdexpenditureon food, represented mathematicallyby the conditional mean E(ylr) : Fyl' dependson householdincome r. If we considerhouseholdswith different levels of income, we expect the averageexpenditure on food to change. In Figure 2.Lb we show the probability density functions of food expenditurefor two different levels of weekly income, $ 1000and $2000. Each conditional pdf f (ylx) showsthat expenditureswill be distributed about a meanvalue pr1",but the mean expenditureby householdswith higher income is larger than the mean expenditureby lower income households. In most economics textbooks "consumption" or "expenditure" functions relating consumptionto income are depicted as linear relationships,and we will begin by assuming the samething. The mathematicalrepresentationof our economic model of householdfood expenditure,depicted in Figure 2.2, is
- 9r * pzx E(vlx): P,,1,.
(2.r)
12
THE SIMPLE LINEAR
REGRESSION MODEL
Ertlx) g
.= o
E(1tlx)=fu+B2x
X a oo
dE0v)
o
dx
^(
Frl
--
L
FTGURE 2.2 The economicmodet:a [;:;":;ationship expenditureandincome.
between ur"rui"per personfood
The conditional mean E(ylx) in (2.1) is called a simple regression function. It is called simple regressionnot becauseit is easy,but becausethereis only one explanatoryvariableon the right-hand side of the equation.The unknown regression parameters B1and B2are the intercept and slope of the regression function, respectively. If you need a review of the geometry,interpretation, and algebra of linear functions seeAppendix A.3 at the end of the book. In our food expenditure example the intercept 91 representsthe mean per person weekly household expenditure on food by a household with no weekly income, x : $0. If income is measuredin dollars, then the slope B2representsthe changein EfJ,lx) given a $1 changein weekly income; it could be called the marginal propensity to spendon food. Algebraically,
^ YZ-
L,E(ylx) dn(ylx) DX
(2.2)
AX
where A denotes "change in" and dEQlx)ldx denotes the "derivative" of E(ylx) with respectto x. We will not use derivativesto any great extent in this book, and if you are not familiar with the concept, you can think of "d" as a "stylized" version of A and go on. The economic model (2.1) summarizes what theory tells us about the relationship betweenweekly householdincome (x) and expectedhouseholdexpendifureon food, E(ylr). The parametersof the model, B1 and 82, are quantities that help charucteize economic behavior in the population we are considering and are called population parameters. In order to use data we must now specify an econometric model that describeshow the data on householdincome and expenditure are obtained, and that guides the econometric analysis.
FIGUR
Thir expen assum E(ylx) econo Ino
RE rem the ass ofe
In Fig var(yl. at eac their n In Figr identic "sprea
The we ar!
2.2 An Econornetric Model
E(yl*
The model Ejlx) :9r * Fzx describeseconomic behavior, but it is an abstraction from reality. If we take a random sample of households with weekly income x : $1000, we know the actual expenditure values will be scatteredaround the mean value : 9r * Bz(1000),as shownin Figure 2.1. If we were to sample Sjlx : 1000) : pylx:1000 householdexpendituresat other levels of income, we would expect the samplevaluesto be scatteredaround their meanvalue E(ylx) : 9r * lzx.In Figure 2.3 we iurangebell-shaped figures like Figure 2.1, depicting the pdfs of food expenditure/(ylx), along the regression line for each level of income.
satisfy var(yl" We collecl ditures (rando value r Ma theore
2.2 AN ECONOMETRIC
MODEL
13
B1+p2x=E(y6.)
Jvr)
r
on food
r. It is called y variable on nd B2are the eview of the he end ofthe :l per person , m e ,X : $ 0 . (ylx) given a end on food.
(2.2) E(yl"r)with fyou arenot \ and go on. relationship food,E(ylx). zeeconomic parameters. bes how the econometric
abstraction ncome .tr : mean value re to sample valuesto be bell-shaped e regression
FTGURE2 ,3 T\e probabilitydensityfunctionsfor y at two levelsof income. This figure showsthat at each level ofincome the mean, or average,value ofhousehold expenditureis given by the regressionfunction Ejlxl :9r * pzx. It also showsthat we assumevaluesof householdexpenditureson food will be distributed around the meanvalue E{glx) : 9r -t lzx at eachlevel of income. This regressionfunction is the foundation of an econometric model for household food expenditure. In order to make the econometric model complete we have to make some assumptions.
REMARK: You will hear a great deal about assumptions in this chapter and in the remainderof the book. Assumptionsare the "if" part of an "if-then" type statement.If the assumptionswe make are true, then certain things follow. And, as importantly, if the assumptionsdo not hold, thenthe conclusionswe draw may not hold. Partof the challenge of econometric analysisis making realistic assumptionsand then checking that they hold.
In Figure 2.la we assumed that the dispersion of the values y about their mean is var(yl.r : $1000) : o2. We must make a similar assumptionabout the dispersionof values at each level of income. The basic assumption is that the dispersion of values y about their mean is the samefor all levels of income -r. That is, var(yl.r) : o2 for all values of x. In Figure 2.lb the pdfs for two different incomes have different means, but they have identical variances.This assumptionis also illustrated in Figure 2.3, as we havedepictedthe "spread" of each of the distributions, like Figure 2.1, to be the same. The constantvariance assumptionvar$lx) : o2 implies that at each level of income x we are equally uncertain about how far values of y might fall from their mean value, E(ll*) :9r * Fzr, and the uncertainty does not dependon income or anything else. Data satisfying this condition are said to be homoskedastic. If this assumptionis violated, so that var(ylr) f oz for all values of income x, the data are said to be heteroskedastic. We have described the sample as random. This description means that when data are collected they are statistically independent.Ify; and yj denote the per person food expenditures of two randomly selected households, then knowing the value of one of these (random) variablestells us nothing about the probability that the other will take a particular value or range of values. Mathematicians spend their lives (we exaggerate slightly) trying to prove the same theorem with weaker and weaker sets of assumptions. This mindset spills over to
14
THE SIMPLE LINEAR
REGRESSION MODEL
econometriciansto some degree. Consequently,econometric models often make an assumptionthat is weaker than statisticalindependence.If _yiand 1y are the expenditures of two randomly selectedhouseholds,then we will assumethat their covarianceis zero,or cov(y;,y;) : 0. This is a weaker assumptionthan statisticalindependence(sinceindependenceimplieszerocovariance,but zerocovariancedoesnot imply independence); it implies only thatthereis no systematiclinear association betweenli andyj.Referto AppendixB.4.3 for more discussionof this difference. In order to carry out a regression analysis we must make two assumptionsabout the valuesofthe variable;r.The ideaofregressionanalysisis to measurethe effectofchangesin one variable, x, on another,y. In order to do this r must take at least two values within the sample of data. If all the observationson x within the sample take the same value, say .r: $1000,then regressionanalysisfails. Secondly,we will assumethat the x-valuesare given,andnotrandom.All ourresultswill be conditionalon thegivenx-values.More will be said about this assumptionsoon. Finally, it is sometimesassumedthat the valuesof y arc normally distributed.The usual justificationfor this assumptionis that in naturethe "bell-shaped" curve describesmany phenomena,ranging from IQs to the length of corn stalks to the birth weights of Australian male children.It is reasonable, sometimes,to assumethat an economicvariableis normally distributedaboutits mean.We will saymoreaboutthis assumptionlater,but for now we will makeit an "optional" assumption,sincewe do not needto makeit in many cases,andit is a very strong assumptionwhen it is made. These ideas, taken together, define our econometric model. They are a collection of assumptionsthat describe the data.
ASSUMPTIONS .
OF THE SIMPLE LINEAR
REGRESSION MODEL-I
The meanvalue of y, for eachvalue of x, is given by the linear regressionfunction E(Ylx):FrfPzx
.
For each value of x, the values of y are distributed about their mean value, following probability distributions that all have the same variance, var(Ylx) : s2
.
The samplevalues ofy are all uncorrelatedand have zero covariance,implying that there is no linear associationamong them, c o v ( y ; , Y ; ): 0 This assumptioncan be made stronger by assumingthat the values of y are all statistically independent.
.
The variable x is not random and must take at least two different values. o (optional) The values of y are normally distributed about their mean for each value of x,
y - lgl(Fr-l Bzx),ozf
2.2.
Itisc of -y,t fbr st Th be de syste itisa lts co
If we
The < indel Ec
9r*
alrea (2.3
The t Si must ident the e \\
ASSUI
said desc outc the n the 1
2.2 AN ECONOMETRIC
)n make an )xpenditures t! is zero, or lce indepene); itimplies pendixB.4.3 ns about the rfchangesin :s within the .e value, say x-values are Morewillbe :d. The usual rcribesmany rf Australian : is normally 'nowwewill es,and it is a oollection of
2.2.t
lNrrooucrNc
MODEL
15
THE Ennon TBnu
It is convenientto describethe assumptionsof the simple linear regressionmodel in terms of y, which in generalis called the dependent variable in the regressionmodel. However, for statistical purposesit is useful to characterizethe assumptionsanother way. The essenceofregressionanalysisis that any observationon the dependentvariabley can be decomposedinto two parts: a systematic component and a random component. The systematiccomponentof y is its mean,E(ylx) : Fr * B2x,which itself is not random since it is a mathematicalexpectation.The random componentof y is the difference betweeny and its conditional mean value E(yl*). This is called a random error term, and it is defined as
e:j-E(ylx):J-Ft-lzx
(2.3)
If we rearrange(2.3) we obtainthe simple linear regressionmodel /:9r
l9zxle
(2.4)
The dependentvariable y is explained by a component that varies systematically with the independent variable x and by the random error term e. Equation (2.3) shows that y and the error term e differ only by the term E(yl;) : Br * Bzx,which is not random. Sincey is random, so is the error tern e. Given what we have already assumedabout y, the properties of the random error e can be derived directly from (2.3). The expectedvalue of the error term, given x, is E(elx) : E(Ylr) - B1 - B2x : 0
DEI.I onfunction
rean value,
:, implying
The mean value of the error term, given x, is zero. Since y and e differ only by a constant(i.e., a factor that is not random), their variances must be identical and equal to o2. Thus the probability density functions for y and e are identical except for their location, as shown in Figure 2.4. Note that the centerof thepdffor the error term, fle), is zero, which is its expectedvalue, E(elr) : 0. We can now discussa bit more the simplifying assumptionthat -r is not random. The assumptionthat x is not random meansthat its value is known. In statisticssuch,y-valuesare said to be "fixed in repeatedsamples." If we could perform controlled experiments, as describedin Chapter 1, the samesetofx-values could be usedover and over, so that only the outcomes) are random. As an example, supposethat we are interestedin how price affects the number of Big Macs sold weekly at the local McDonalds. The franchise owner can set the price (x) and then observe the number of Big Macs sold (y) during the week. The
ofy areall ilues. n for each
oF, FrcuRE 2 .4
Fr+ 9zr
Probability density functions for e and -r'.
16
THE SIMPLE LiNEAR
REGRESSION \1()I)EL
following week thepricecouldbe changed,andagainthedataon salescollected.In this case x : the price of a Big Mac is not random,but fixed. The numberof casesin which thex-valuesare fixed is small in the world of businessand economics. When we survey householdswe obtain the data on variables like food expenditureper personand householdincome at the same time. Thus y and x are both random in this case;their valuesare unknown until they are actually observed.However, making the assumptionthat x is given, and not random, does not changethe resultswe will discussin the following chapters.The additional benefit from the assumptionis notational simplicity. Sincer is treatedas a constantnonrandomterm, we no longer needthe conditioningnotation"l". So,insteadof E(elx): 0 you will seeE(e) :0. Therearesome importantsituationsin which treatingx asfixedis not acceptable, andthesewill be discussed in Chapter 10. It is customaryin econometricsto statethe assumptionsof the regressionmodelin terms of the random error e. For future referencethe assumptionsare named SRI-SR6, "SR" denoting "simple regression."Remember,since we are treatingx as fixed, and not random, henceforthwe will not use the "conditioning" notation ylx.
ASSUMPTIONS
OF THE SIMPLE LINEAR
REGRESSION MODEL-II
SRl. The valueofy, for eachvalueofx, is
):Br*Bzxle SR2. The expectedvalue of the randomerror e is
E(e) : g which is equivalentto assumingthat E(Y):Fr*92"t SR3. Thevarianceof therandomerrore is var(e):o2:var()) The random variablesy and e havethe samevariancebecausethey differ only by a constant. SR4. The covariancebetweenany pair ofrandom errorse; and e; is cov(e;,e1): cov(y;,y;) : 0 The strongerversionof this assumptionis that the randomenors e are statistically independent,in which case the values of the dependentvariable y are also statisticallyindependent. SR5. The variable"r is not randomand must take at leasttwo different values. SR6. (optional) The valuesof e arenormally distributedabouttheir mean e - N(0, o2) if the values of y are normally distributed, and vice versa.
2.2 AN ECONOMETRIC
MODEL
17
n this case sinessand like food c are both However, :esultswe mption is :r needthe ) arg some discussed
4
Egt)=fu+ p2x
e2
) el in terms R6, "SR" rt random, x1xZx3x4x FTGURE 2.5
The relationship among y, e, and the true regression line.
The random error e and the dependentvariable y are both random variables, and as we haveshown the propertiesof one can be determinedfrom the propertiesof the other. There is, howeveq one interesting difference between them, y is "observable" and e is "unobservable." If the regressionparametersB1 and B2 were known, then for any value of y we could calculatee : y - (Fr + B2x).This is illustrated in Figure 2.5. Knowing the regression functionEO):Ft*lzx,wecouldseparateyintoitsfixedandrandomparts.However,Bl and B2 are never known, and it is impossible to calculate e. What comprisesthe error term e? The random error e representsall factors affecting y other than x. These factors causeindividual observations), to differ from the mean value E(y) : Fr * 9zr. In the food expenditureexample, what factors can result in a difference betweenhousehold expenditure per person y and its mean, E(y)? l . We have included income as the only explanatory variable in this model. Any other economic factors that affect expenditureson food are "collected" in the error term. Naturally, in any economic model, we want to include all the important and relevant explanatory variables in the model, so the error term e is a "storage bin" for unobservableand/or unimportant factors affecting householdexpenditureson food. As such, it adds noise that masks the relationship between x and y.
2, The error term e captures any approximation error that arises becausethe linear functional form we have assumedmay be only an approximation to reality.
3 . The error term capturesany elementsofrandom behaviorthat may be presentin each individual. Knowing all the variablesthat influence a household'sfood expenditure might not be enoughto perfectly predict expenditure.Unpredictablehumanbehavior is also containedin e. lf we have omitted some important factor, or made any other serious specification error, then assumption SR2 E(e) : 0 will be violated, which will have serious consequences.
18
THE SIMPLE LINEAR
Table 2 .7
REGRESSION MODEL
Food Expenditure and Incorne Data
Observation (household)
Food expenditure ($)
I 2
115.22 135.98 : 25't.95 375.73
39 40
Weekly income($100)
3.69 4.39 29.40 33.40
Summary statistics Sample mean Median Maximum Minimum Std. Dev.
283.5735 264.4800 587.6600 109.7100 r12.7652
19.6048 20.0300 33.4000 3.6900 6.8478
2,3 Estfunating the Regression Parameters The economic and econometric models we developedin the previous section are the basis for using a sampleof data to estimatethe intercept and slopeparameters,B1 and B2.For illustration we examine typical data on household food expenditure and weekly income from a random sample of 40 households. Representative observations and summary statisticsare given in Table 2. 1. We control for householdsizeby consideringonly threeperson households. The values of y are weekly food expenditures for a three-person household,in dollars. Insteadof measuringincome in dollars, we measureit in units of $100,becausea $1 increasein income hasa numerically small effect on food expenditure. Consequently,for the first household, the reported income is $369 per week with weekly food expenditure of $115.22. For the 40th household weekly income is $3340 and weekly food expenditure is $375.73. The complete data set of observationsis in the file food.dat. REMARIL In the text, data files are referenced as *.dat; e.9.,food.dat Files in other formatswill havethe samename,but a differentextension,suchasfood.wfl,food.dta,and so on. The correspondingdatadefinition file will befood.def.Thesefiles arelocatedat the book Web site (www.wiley.com/college/trill). We assumethat the expenditure data in Table 2.1 satisfy the assumptions SRl-SR5. That is, we assume that the expected value of household food expenditure is a linear function of income. This assumption about the expected value of y is equivalent to assumingthat the random error has expectedvalue zero, implying that we have not omitted any important factors. The variance of y, which is the sameas the variance of the random error e, is assumedconstant,implying that we are equally uncertain about the relationship betweeny and x for all observations.The values of y for different householdsare assumed
un( Th, .'or ral .'hr
E\I ral par rnc the
Ot \\i it af ar thr ml
to Hr of in
E S T I M A T I N C ] T H E R E ( ; I { E S S I O N P A R A ] V lE T E R S
Weekly come($100)
19
a I a
:O
3.69 4.39
a a
a
EO
aa
29.40 33.40
a
a
'" t9.6048 20.0300 33.4000 3.6900 6.8478
re the basis rnd B2.For
; in other l.dta, and, ted at the
SR1_SR5. s a linear ivalent to ot omitted rerandom lationship : assumed
0 t TGL:RE 2 ,6
l0
20 r = weeklyincomein $100
Data for the food expenditure exampie.
uncorrelatedwith eachother,which follows if we obtainedthe data by random sampling. The valuesof x were actuallyobtainedby randomsampling,but we will make the analysis ;onditional on the x valuesin the sample,which allows us to treat them as nonrandom r aluesthat arefixed in repeatedsamples.At the end of the day,this simplificationdoesnot changethe analysis. Given this theoreticalmodel for explainingthe sampleobservationson householdfood erpenditure,the problem now is how to use the sampleinformation in Table 2.1, specific ralues of y; and Jr;,to estimate the unknown regressionparametersB1 and B2. These parameters representtheunknowninterceptand slopecoefficientsfor the food expenditureincomerelationship.If we representthe40 datapointsas(y;,x;), i : 1, .. . , N : 40, andplot ihem, we obtain the scatter diagram in Figure 2.6. REMARIC It will be our notationalconventionto use "i" subscriptsfor cross-sectional dataobservations,with the numberof sampleobservationsbeing N. For time-seriesdata observationswe usethe subscript"t" andlabel the total numberof observationsas Z In purely algebraicor genericsituations,we may use one or the other. Our problem is to estimatethe location of the mean expenditureline E(y) : Br * pzx. \\'e would expect this line to be somewherein the middle of all the data points since it representsmean, or average,behavior.To estimate B1 and B2 we could simply draw I freehandline throughthe middle of the dataandthen measurethe slopeand interceptwith ;rruler.The problemwith this methodis that differentpeoplewould draw differentlines,and rhelack of a formal criterionmakesit difficult to assessthe accuracyof the method.Another methodis to draw a line from the expenditureat the smallestincomelevel,observationi : 1, ro theexpenditureat largestincome level. r : -10.This approachdoesprovidea formal rule. informationon the exactposition However,it may not be a very good rule becauseit i-gnores It would be betterif we could devisearule that usesall the of the remaining38 observations. infbrmationfrom all the data points.
THE SIMPLE LINEAR
REGRESSiON- Nl()l)EI
Y= b t + b 2 x
v3 ?
r3
x
w) y4
!=bt+bzx
?' I
y*=ft+bit
)3
?
_r1
x2
x3
x4
x
(b\ (a) The relationshipamong ), 2, and the fitted regressionline. (&) The residuals FTGURE2,7 from another fitted line.
'l 2.3-l
TnE LEesr Squenrs PRrNcrprE
To estimateB1and B2we want a rule, or formula,that tells us how to makeuseof the sample observations.Many rules are possible,but the one that we will use is basedon the least squares principle. This principle assertsthat to fit a line to the data values we should make the sum of the squaresof the vertical distancesfrom each point to the line as small as possible.The distancesare squaredto preventlargepositivedistancesfrom being canceled by largenegativedistances.This rule is arbitrary,but very effective,andis simply oneway to describea line that runsthroughthe middle of the data.The interceptand slopeof this line, thelinethatbestfitsthedatausingtheleastsquaresprinciple, areblandb2,theleastsquares estimatesof B1 and 82. The fitted line itself is then
i
lt:brlbzxi
(2.s)
2.3 ESTIMATING
THE REGRESSION PARAMETERS
21
The vertical distancesfrom eachpoint to the fitted line arethe least squares residuals. They are given by -b2x; (2.6) 2r:yr-jtt:yi-b1 These residuals are depicted in Figure 2.7a. Now supposewe fit another line, any other line, to the data. Denote the new line as
yi : bi+b5xt nhere bf and bi are any other intercept and slope values. The residuals for this line, ii : yt - yi, are shown in Figure 2.7b. The least squaresestimates b1 and b2 have the property that the sum oftheir squaredresidualsis less than the sum of squaredresidualsfor rrn.r'other line. That is, if
: La? sse r- I
is the sum of squaredleast squaresresidualsfrom (2.6) and N^N^
SSE*:,\rui' : 2r0i - ii)' is the sum of squaredresidualsbasedon any other estimates,then
ssE<ssE. no matter how the other line might be drawn through the data. The least squaresprinciple saysthat the estimatesbl andb2 of B1and B2are the onesto use since the line using them as intercept and slope fits the data best. The problem is to find br andb2in a convenientway. Given the sampleobservationson y andx, we want to find valuesfor the unknown parametersB1and B2that minimize the "sum of squares"function N^
s(Fr' 9z): he residuals
,I,
(1,- 9r - 1zxi)'
This is a straightforwardcalculusproblem, the details of which aregiven inAppendix2A,at the end of this chapter.The formulas for the least squaresestimatesof B1and B2that give the minimum of the sum of squaredresiduals are THE LEAST SQUARES ESTIMATORS .
-
Z(xi - r)(yi - I)
Oc:-=*
of the sample I on the least should make : as small as :ing canceled ly oneway to e of this line, least squares
(2.s)
b1
2(x; -r)'
(2.7)
(2.8)
',rhere y : LyilN and 7 : Lx;lN are the sample means of the observationson y and x. The formulafor b2revealswhy we hadto assume[SR5]thatthevaluesofx; werenot the same r aluefor all observations.If x; : 5, for example,for all observations, thenb2is mathematically rndefined and doesnot exist since the numeratorand denominatorof (2.7) are zerol If we plug the samplevaluesyr and-rrinto (2.7) anO(2.8),thenwe obtainthe leastsquares . ilinwtes of the interceptand slopeparametersp 1and B2.It is interesting,however,and very
22
THE SIMPLE LINEAR
REGRESSION MODEL
important, that the formulas for D1and b2 are pertectly general and can be used no matter what the samplevaluesturn out to be. This should ring a bell. When the formulas for bl and b2aretaken to be rules that are used whateverthe sampledata tum out to be, thenbl andb2 arerandom variables.When actual samplevaluesare substitutedinto the formulas we obtain numbersthat arethe observedvaluesofrandom variables.To distinguish thesetwo caseswe call the rules or general formulas for b1 and b2 the least squares estimators. We call the numbers obtained when the formulas are used with a particular sample least squares estimates. o
Least squaresestimators are general formulas and are random variables.
a
Least squaresestimatesare numbersthat we obtain by applying the generalformulas to the observeddata.
The distinction betweenestimators andestimatesis a fundamentalconceptthat is essential to understandeverything in the rest of this book. 2.3.2
Esrrnerrs
FoR THE Fooo ExpsNDrruRE FtnrcrroN
Using the least squaresestimators(2.1) and(2.8), we can obtain the least squaresestimates for the intercept and slope parametersB1and B2in the food expenditureexample using the data in Table 2.1. From (2.7\ we have
u,:>$ffiC:ffi#:ro2oe6
i :\F :::e
I
rs I - b 2 7 : 2 8 3 . 5 7 3 ' 7- ( 1 0 . 2 0 9 6 ) ( 1 9 . 6 0 4:88)3 . 4 1 6 0
A convenient way to report the values for br and b2 is to write out the estimatedor fitted regressionline, with the estimatesrounded appropriately: li :83.42I
."S! :^rg aaol
l}.2lxi
This line is graphedin Fi gwe2.8. The line's slopeis 10.21andits intercept,whereit crosses the vertical axis, is 83.42.The least squaresfitted line passesthrough the middle of the data in a very precise way, since one of the characteristicsof the fitted line based on the least squaresparameterestimatesis that it passesthough the point definedby the samplemeans, (*,r) : (19.6048,283.5'135).This follows directly from rewritinC (2.8) as t - bt I bzi. Thus the "point of the means" is a useful reference value in regressionanalysis. 2.3.3
:h< lEr \
:::\
and from (2.8) h:y
,::a;
IwreRpRErrNG THE Esrrtlrerss
Once obtained, the least squaresestimatesare interpreted in the context of the economic model under consideration.The value bz : 10.21is an estimateof B2.Recall that x, weekly householdincome, is measuredin $100 units. The regressionslope B2 is the amount by which expected weekly expenditure on food per household increases when household weekly income increasesby $100. Thus, we estimatethat if weekly householdincome goes up by $100, expectedweekly expenditureon food will increaseby approximately$10.21. A supermarketexecutivewith information on likely changesin the income and the number
:.3. : - -, .:llu
_::le
htl
]..I ESTIMATING
usedno matter rulasfor b1and , then b1 and b2 lulas we obtain ie two caseswe rs. We call the least squares
THE REGRESSION PARAMETERS
a a la
] = 83.42+ 10.2rx ^,o
t a aa
Xo
7=28357
.
a
rl
../. a
:neralformulas
a
d rl
o
bles.
23
I
'lI
a
a a
ol' a
t= 19.60
thatis essential
20 r = weeklyincomein $100
Frc uRE 2 . 8 Thefittedresression. uaresestlmates rmple using the
) intatedorfitted
whereit crosses ddle of the data ;ed on the least samplemeans, s!:brIbzT. nalysis.
of householdsin an areacould estimatethat it will sell $ 10.21more per typical household perweek for every$100increasein income.This is a very valuablepieceof informationfor long-runplanning. S t r i c t l y s p e a k i n g , t h e i n t e r c e p t e s t i mbat :t 3e 3 . 4 2 i s a n e s t i m a t e o f t h e w e e k l y f o o d .\penditure for a householdwith zero income.In most economicmodelswe must be very carefulwhen interpretingthe estimatedintercept.The problemis that we usuallydo not have .inydatapointsnear-{ : 0, which is truefor the food expendituredatashownin Figure2.8. If lr e haveno observationsin the regionwhereincomeis zero,thenour estimatedrelationship rnaynot be a good approximationto reality in that region.So,althoughour estimatedmodel :uggeststhat a householdwith zeroincomeis expectedto spend$83.42per weekon food, it might be risky to takethis estimateliterally.This is an issuethat you shouldconsiderin each cconomicmodel that you estimate. 1.3.3a Elasticities Incomeelasticity is a useful way to characterizethe responsiveness of consumerexpenJitureto changesin income.SeeAppendixA.3 for a discussionof elasticitycalculationsin a linear relationship.The elasticity of a variabley with respectto anothervariablex is percentagechangein I' percentagechangein x
_Lvlv:Av.{ Lx y
Lrl*
In the lineareconomicmodel given by (2.1) we haveshownthat f the economic I thatx, weekly the amount by hen household ld incomegoes m a t e l y$ 1 0 . 2 1 . rnd the number
lEtr ) . P.: \ 'o the elasticity of mean expenditurewith respectto income is
L E ( y )l E , t ) Lrl*
-\E r
.\
En)
-
Y:
L\y )
(2.e)
24
THE SIMPLE LINEAR
REGRESSION MODEL
To estimatethis elasticity we replace 9zby bz: 10.21.We must also replace "x" and " EO" by something,sincein a linear model the elasticityis different on eachpoint upon the regressionline. Most commonly the elasticityis calculatedat the "point of the means" (x,y): (19.60,283.57)becauseit is a representativepoint on the regressionline. If we calculate the income elasticity at the point of the means we obtain
"- b, ro.2r , #9 :0.71 +: This estimated income elasticity takes its usual interpretation. We estimate that a lVo increasein weekly householdincomewill lead,on average,to a0.'717oincreasein weekly household expenditure on food, when .r and y take their sample mean values, ( x , y ) : ( 1 9 . 6 0 , 2 8 3 . 5 7 )S. i n c e t h e e s t i m a t e d i n c o m e e l a s t i c i t y i s l e s s tl ,hwaenw o u l d classify food as a "necessity" rather than a "luxury," which is consistentwith what we would expect for an averagehousehold. 2.3.3b Prediction The estimatedequationcanalsobe usedfor predictionor forecastingpurposes.Supposethat we wanted to predict weekly food expenditure for a household with a weekly income of $2000.This predictionis carriedout by substitutingx : 20 into our estimatedequationto obtain y i : 8 3 . 4 2 t l } . 2 l x i : 8 3 . 4 2 + 1 0 . 2 1 ( 2 0:)2 8 7 . 6 1 Wepredict thata householdwith a weekly incomeof $2000will spend$287.61per weekon food.
2.3.3c Cornputer Output Many different software packagescan compute least squaresestimates.Every software package'sregressionoutput looks different and usesdifferent terminology to describe the output. Despite thesedifferences,the various outputs provide the samebasic information, which you shouldbe able to locate and interpret.The matter is complicated somewhatby the fact that the packagesalso report various numberswhose meaning you may not know. For example, using the food expendituredata, the output from the software packageEViews is shown in Figure 2.9. In the EViews output the parameter estimates are in the "Coefficient" column, with names "C," for constant term (the estimateb), andINCOME (the estimateb2). Software programs typically name the estimateswith the name of the variable as assignedin the computer program (we named our variable INCOME) and an abbreviationfor "constant." The estimatesthat we report in the text are rounded to two significant digits. The other numbersthat you can recognizeat this time are .SSE: >Ai :304505.2, which is called "Sum squaredresid," and the samplemean of y, , :LyilN :283.5i35, which is called "Mean dependentvar." We leavediscussionof the restof the outputuntil later.By the end of this book, you will be able to interpret the remainder of the information.
2.3.4
OrHrn EcoNolrrc Moonrs
We haveusedthe householdexpenditureon food versusincomerelationshipasan example to introducethe ideasof simple regression.The simpleregressionmodel can be appliedto
2.3 ESTIMATING
, ",T" and ,ointupon e means" ine. If we
83.41600 10.20964
"co,nE i.-quared !.i'u.ted R-squared . E. of regression i-:n rquaredresid - . : likelihood ; ..irtistic - ,nrF-statistic)
pposethat ncome of luation to
25
-rerendent Variable: FOOD_EXP \{:thod: Least Squares .::lple: I 40 .r.-'iudedobservations:40 Coefficient
hat a lVo in weekly n values, we would r what we
THE REGRESSION PARAMETERS
,' px 2.9
0.385002 0.368818 89.s1700 304505.2 -235.5088 23.78884 0.000019
Std. Error
r-Statistic
Prob.
43.4t016 2.093264
1.92r5't8 4.87738r
0.0622 0.0000
Mean dependent var S.D. dependentvar Akaike info criterion Schwarz criterion Hannan-Quinn criter Durbin-Watson stat
283.5735 t12.6752 rr.87544 I 1.95988 I 1.90597 1.893880
outPut. EViewsregression
i.:imate the parametersof many relationshipsin economics,business,and the social i:cnces. The applicationsof regressionanalysisare fascinatingand useful. For example r week on
If the hourly wagerate of electriciansrisesby 5Vo,howmuch will new houseprices increase? Ifthe cigarettetax increasesby $1, how much additionalrevenuewill be generatedin the stateof Louisiana?
' software scribe the brmation, 'hat by the know. For EViews is rmn, with Software red in the )onstant." The other r is called r is called :, you will
r example rpplied to
If the centralbanking authorityraisesinterestratesby one-half a percentagepoint, how much will consumer borrowing fall within 6 months? How much will it fall within I year? What will happento the unemploymentrate in the months following the increase? If we increasefunding on preschooleducationprogramsin 2008, what will be the effect on high school graduationrates in 2020?What will be the effect on the crime rate by juveniles in2012 and subsequentyears? he rangeof applicationsspanseconomicsand finance,as well as most disciplinesin the - sial and physical sciences.Any time youaskhow mucft a changein one variablewill affect ::r)ther variable,regressionanalysisis a potentialtool. The simple linear regressionmodel is much more flexible than it looks at first glance .::rcethe variablesy and.r can be transformations,involving logarithms,squares,cubes,or ::;iprocals, of thebasiceconomicvariables.Thus,the simplelinearregressionmodelcanbe -.ed for nonlinear relationshipsbetweenvariables.The term linear in "linear regression" :;ruall! meansthat the parametersare not transformedin any way. In a linear regression -::olel the parameter^s must not be raised to powers or transformed, so expressionslike, . - F:), 9r 92, or Blr are not permitted. In Appendix A.4 we summarize a variety of nonlinear relationships between variables ::rr can be estimatedby using least squaresestimationand the simple linear regression :::.rdel.If y and x are transformed in some way. then the economic interpretations of the ::.gressionparameterschange, and the calculation of elasticity changesas well. For :rample, a populartransformationin economicsis the naturallogarithm.Economicmodels
26
THE SIMPLE LINEAR
REGRESSION MODEL
like ln(y) : Br * B2ln(x) are common. A nice feature of this model, if the assumptionsof the regressionmodel hold, is that the parameterB2ls the elasticity of y with respectto x for all points on the regressionline. Consequentlythe "log-log" model ln(y) : F r * B2ln(x) is also called the constant elasticity model. See Table A.2 in Appendix A.4 for alternative algebraicfunctional forms, with the expressionsfor their slopeandelasticity, andFigure A.2 for their geometric shapes.You will refer to Table A.2 and Figure A.2 often, so perhaps put a paper clip on the page for quick access.
2,4 Assessingthe Least Squares Estimators Using the food expendituredata we have estimatedthe parametersof the regressionmodel yi: Fr *9zxi *e;usingtheleastsquaresformulasin(2.7)and(2.8).Weobtainedtheleast squaresestimatesbt : 83.42 andb2: 10.21.It is natural,but aswe shallarguemisguided, to ask the question "How good are these estimates?" This question is not answerable. We will never know the true valuesof the population parametersPr or Fz, consequentlywe cannot say how close b1 : 83.42 and b2 : 10.21 areto the true values.The least squares estimatesare numbersthat may or may not be closeto the true parametervalues,and we will never know. Ratherthan asking aboutthe quality of the estimateswe will take a stepback and examine the quality of the least squaresestimation procedure.The motivation for this approachis this: If we were to collect anothersampleof data,by choosing anothersetof 40 households to survey, we would have obtained dffirent estimates b1 and b2, evefl if we had carefully selectedhouseholdswith the same incomes as in the initial sample. This sampling variation is unavoidable.Different sampleswill yield different estimatesbecausehousehold food expenditures, !i,i : l, . . . ,40, arerandomvariables.Their valuesarenot known until the sampleis collected. Consequently,when viewed as an estimationprocedure, br and bz arc alsorandom variables,becausetheir values dependon the random variable y. In this context we call b1 and b2 the least squaresestimators. We can investigate the properties of the estimators b1 and b2, whrch are called their sampling properties, and deal with the following important questions: l . If the least squaresestimators b1 and bz arc random variables, then what are their expected values, variances,covariances,and probability distributions? onewayof usingthedatatoobtainestimatesof 2 . Theleastsquaresprincipleisonly B1 md 92. How do the least squaresestimatorscomparewith other proceduresthat might be used,and how can we comparealternative estimators?For example, is there another estimatorthat has a higher probability of producing an estimatethat is close to B2? The answersto thesequestionswill dependcritically on whether the assumptionsSRI-SRS are satisfied.In later chapterswe will discusshow to check if the assumptionswe make hold in a specific application, and what we might do if one or more assumptionsare shown not to hold. REMARK: We will summarizethe propertiesof the leastsquaresestimatorsin the next severalsections."Proofs" of importantresultsappearin the appendicesto this chapter.In many ways it is good to seetheseconceptsin the context of a simpler problem before tacklingthemin theregressionmodel.AppendixC coversthetopicsin this chapter,andthe next,inthefamiliarandalgebraicallyeasierproblemofestimatingthemeanofapopulation.
2..I ASSESSING THE LEAST SQUARES ESTIMATORS
assumptionsof respectto.r for 3r f Fz ln(-r)is for alternative , andFigureA.2 ten, so perhaps
2.4.1
THE Esrnrneron b2
Formulas (2.7) and (2.8) are used to compute the least squaresestimatesb1 and b2. However,they arenot well suitedfor examining theoreticalpropertiesof the estimators. In this section we rewrite the formula for b2 to facilitate its analysis. In (2.7) b2 is given by
-x)(y-i) I(', 2(x; _r)z
bz:
'gressionmodel rtainedthe least gue misguided, rot answerable. onsequentlywe re least squares ues,andwe will .ckandexamine :his approach is f40 households f we had careThis sampling becausehouses are not known :ocedure,b1and riable y. In this are called their
r what are their rns? estimatesof B1 Juresthat might ,is there another is close to B2? tions SR1-SR5 s we make hold re shown not to
ors in the next his chapter.In roblem before rapter, and the fapopulation.
27
This is called the deviationfrom the mean form of the estimatorbecausethe data have their samplemeanssubtracted.Using assumptionSR1 and abit of algebra(Appendix2C), we can wite b2 as a linear estimator, bz:
\wtyi
(2.10)
i:l
where
'
Wi:-
xi-x
L(x; -t)"
(2.rr)
The term w; dependsonly on xi that are not random, so that w; is not random either. Any estimatorthat is a weighted averageof y;'s, asin (2.10), is called a linear estimator.This is an important classification that we will speak more of later. Then, with yet more algebra (Appendix 2D) we can expressb2 in a theoretically convenient way, bz : 9z +Lwiei
(2.r2)
where ei is the random error in the linear regressionmodel y;:9r * 9zxi+ e;. This formula is not useful for computations,becauseit dependson B2,which we do not know, and on e;, which are unobservable.However, for understandingthe sampling properties of the least squaresestimator, (2.12) is very useful.
2.4.2
THs Expscrro
Varuns on b1 x.iro b2
The estimator b2is a random variable since its value is unknown until a sampleis collected. What we will show is that if our model assumptionshold, then E(b) : B2; that is, the expected value of b2 is equal to the true parameter B2. When the expected vahte of any estimator of a parameterequals the true parametervalue, then that estimator is unbiased. SinceE(bz) : B2,the leastsquaresestimator b2is an unbiasedestimatorof B2.The intuitive meaningof unbiasednesscomesfrom the repeatedsampling interpretationof mathematical expectation. If many samples of size N are collected, and the formula fot b2 is used to estimateB2in eachof those samples,then, if our assumptionsarevalid, the averagevalue of the estimatesb2 obtained from all the samples will be B2. We will show that this result is true so that we can illustrate the part played by the assumptions of the linear regression model. In (2.12), what parts are random? The parameter B2 is not random. It is a population parameter we are trying to estimate. If assumptionSR5 holds, then x; is not random. Then w; is not random either, as it depends only on the values of .x;.The only random factors in (2.12) are the random error terms er.
28
THE SIMPLE LINEAR REGRESSION MOI)EL
We can find the expectedvalue of b2 using the fact that the expectedvalue of a sum is the sum of the expectedvalues: E(br) : E(92+Lwiei) : E(92-f wp1 I wye2+ ... + wueu) : E(92) -l E(wpv) -t E(w2e2)+ ' ' ' + E(wpe7s)
(2.r3)
: E(92)*LE(w;ei)
lnl
The rules of expectedvaluesarefully discussedinAppendix8.4 at the end of the book. In the last line of (2.13) we use two assumptions.First, E(wie;) : wiE(e) becausew; is not random, and constantscan be factored out ofexpected values. Second,we have relied on the assumptionthatE(e):0. If E(e)+0, then E(b)l B2,in which caseb2 is a biased estimator of B2. Recall that e; contains, among other things, factors affecting yi that are omitted from the economic model. If we have omitted anything that is important then we would expectthat E(e)t'O and E(b)*B2. Thus, having an economic model that is correctly specified,in the sensethat it includes all relevantexplanatoryvariables,is a must in order for the least squaresestimators to be unbiased. The unbiasednessof the estimatorD2is an important sampling property.When sampling repeatedlyfrom a population the least squaresestimatoris "correct," on average,and this is one desirableproperty of an estimator.This statisticalproperty by itself doesnot mean that b2is a good estimatorof B2,but it is part of the story. The unbiasednessproperty dependson havingmany samplesof datafrom the samepopulation. The fact that b2is unbiaseddoesnot imply anything about what might happen in just one sample. An individual estimate (a number) b2 ma! be near to, or far from, B2. Since B2 is never known we will never know, given one sample,whether our estimateis "close" to B2 or not. Thus the estimateb2 : 10.21 may be close to 92 or not. The leastsquaresestimatorD1of B1is also an unbiasedestimator,andE(b1): 9t if the model assumptionshold. REpEerEo Servrprrxc
To illustrate the concept of unbiasedestimation in a slightly different way, we present in Table 2.2least squaresestimatesof the food expenditure model from l0 random samples (table2-2.dat) of size N : 40 from the same population with the same incomes as the TabIe 2.2 Sample
I 2 J
4 5 6 '7 8 9 t0
th! sa es
: Pz * ZwiE(e;) : g,
2.4.3
ho tic diJ
r31.69 57.25 r03.91 46.50 84.23 26.63 64.21 79.66 97.30 95.96
Ta Un dll
pr! lm
dis kn
Itr po ,lif
.'h( rat rhe ast
tel ret \iu f\t
Estirnatesfrorn 10 Sarnples br
pa Iea
the b2
6.48 10.88 8.14 I 1.90 9.29 lJ.ff
10.93 9.76 8.05 7.7',7
2.+ ASSESSINGTHE LEAST SQUARES ESTIMATORS
sum is the
(2.r3)
book.Inthe e lrr is not re relied on is a biased I )j that are ant then we rdel that is , is a mustin :n sampling e, andthis is Jt mean that 'dependson seddoesnot estimate(a never know, ,timateb2: ):grifrhe
/e present in lom samples omes as the
29
householdsgiven in Table 2.1. Note the variability of the least squaresparameterestimates from sample to sample. This sampling variation is due to the fact that we obtained 40 dffirent householdsin each sample, and their weekly food expenditure varies randomly. averagevaltesof byandb2tf many samplesof Thepropertyofunbiasednessisaboutthe population. The averagevalue of b1 in these l0 the same size are drawn from the same samplesis bt:78.74.The averagevalue of bzisEz:9.68. If we took the averagesof estimatesfrom many samples,theseaverageswould approachthe true parametervalues B1 and B2.Unbiasednessdoesnot say that an estimatefrom any one sampleis close to the true parametervalue, and thus we cannot say that an estimateis unbiased.We can say that the least squaresestimation procedure (or the least squaresestimator) is unbiased. 2.4.4
Trre VenreNCES AND CovenreNcs or b1 nvo b2
Table 2.2 shows that the least squaresestimatesof B1 and Fz vary from sampleto sample. Understandingthis variability is a key to assessingthe reliability and sampling precision of an estimator.We now obtain the variancesand covarianceof the estimatorsb1andbz.Before presentingthe expressionsfor the variancesand covariance,let us consider why they are important to know. The variance of the random variable b2 is the averageof the squared distancesbetween the possible values of the random variable and its mean, which we now know is E(b) : Bz. The varianceof b2is definedas var(b2):Elbz-E(br))' Ir measuresthe spreadof the probability distributronofb2.In Figure 2.10 aregraphsof two possibleprobability distributions of 02, ft(b) and fz(b), that havethe samemeanvalue but different variances. The probability density function fz(b) has a smaller variance than ft(b). Given a choice, we are interestedin estimator precision and would prefer that bzhavethepdf f2(b2) ratherthan fi(b). With the distribution fz(b),the probability is more concentratedaround the true parameter value B2, giving, relative to ft(b), a higher probability of getting an estimate that is close to B2. Remember,getting an estimate close to B2 is our objective. The variance of an estimator measurestheprecision of the estimator in the sensethat it tells us how much the estimatescan vary from sample to sample.Consequently,we often refer to the sampling variance or sampling precision of an estimator. The smaller the variance of an estimator is, the greater the sampling precision of that estimator. One estimatoris more precisethan anotherestimator if its sampling varianceis lessthan that of the other estimator.
F: r rcuns
2. 7 0
Two possibleprobability densitv functionsfor &2.
30
THE SIMPLE LINEAR
REGRESSION MODET
2E we will now presentand discussthe variancesand covariance of b1 andb2.Appendix regression the If bz. estimator contains the derivation of the variance of the least squares then the model assumptions SR1-SR5 are correct (assumption sR6 is not required)' variancesand covarianceof bt afid b2arc
: var(b1)
I
s-z I - ::zl "'l:=i:L [NXxr r)"1
var(b-t\ : -.
o2
Z(xi-x)"
.f cov(b.bz):'2lr--
o 14\
(2.rs)
'r (2.16)
l
are At the beginning of this section we said that for unbiasedestimators,smaller variances and variances the affect better than larger variances. Let us consider the factors that covariancein (2.ID-Q.I6). 1. The variance of the random e11orterm, o2, appearsin each of the expressions. greater It reflectsthe dispersionof the valuesy about their expectedvalue E(y). The about uncertainty is the greater the varianceo',ih.greater is that dispersionand the the is larger, o2 When where the values oi y fall relative to their mean E(y). is variance the 2.3 In Figure information we have about B1 and B2 is less precise. variance the larger (ylr) .The reflecredin the spreadof the probability distributions f the terrn o2, the griater ls the uncertainty in the statistical model, and the larger variances and covariance of the least squaresestimators' - x)2, appears 2. The sum of squaresof the values of r about their samplemean, I(x; in each of the variances and in the covariance. This expression measureshow spreadout abouttheir mean arethe samplevaluesofthe independentor explanatory less variable x. The more they are spread out, the larger the sum of squares.The of sum this recognize may You of squares. they are spreadout, the smaller the sum C'4' Appendix See x-values. the of variance squaresas the numerator of the sample The larger the sum of squares,\(*, -t)2, the smaller the variances of the least squaresestimatorsand the moteprecisely we can estimatethe unknown parameters' (b) is a data The intuition behind this is demonsffatedin Figure 2.ll.In panel In panel (a) -r-axis. the along out spread widely scatterin which the values of ;r are ..bunched." Which data scatterwould you prefer given the task of fitting the data are a line by hand? Pretty clearly, the data in panel (b) do a better job of determining where the least squaresline must fall, becausethey are more Spreadout along the x-axis. 3. The larger the sample size N, the smaller the variancesand covarianceof the least squaresestimatorsfit is better to have more sampledatathan less.The samplesizeN uip"urc in eachofthe variancesand covariancebecauseeachofthe sumsconsistsof - t)2 N terms. Also, N appearsexplicitly in var(bl ). The sum of squarestetm}(xi gets larger as N increasesbecauseeach of the terms in the sum is positive or zero (b"ing ,".o if .r happens to equal its sample mean value for an observation). Consequently,as N gets larger, both var(b2) and cov(b1,b2) get smaller' since the sum of squaresappearsin their denominator' The sums in the numerator and denominatorof var(b1)both getlargerasNgets largerandoffsetoneanother,leaving
! .''. t :.iliT
,1 .\ ri a
2 . 5 T H E G A U S S _ M A R K O VT H E o R E M
.ppendix2E I regression l), then the
31
) i = b 1+ b 2 x ;
(2.r4)
(2.rs) (2.16)
(b\ b\ : : r; t'ne 2 . 7 7 The influence of variation in the explanatory variable r on precision of .'.rimation:(a) low.x variation,low precision:(b) high x variation,high precision.
'ariancesare Lriancesand
the Nin the denominatoras the dominant term, ensuring that var(b1) also gets smaller as N gets larger.
gxpressions. , The greater lainty about i larger, the : variance is the variance rc larger the
The term Ixf appearsin var(b1).The larger this term is, the largerthe varianceof the leastsquaresestimatorb1. Why is this so?Recall that the interceptparameterB 1is the expectedvalue ofy given that x : 0. The farther our data are from x : 0, the more difficult it is to interpret B1,asin the food expenditureexample,and the more difficult it is to accuratelyestimate B1. The terml xl measuresthe distanceof the data from the origin, x : 0. If the valuesof r are near zerothen Ixl will be small and this will reduce var(b). But if the values of x are large in magnitude, either positive or negative,the term Zx! wlll be large and var(b1) will be larger, other things being equal.
x)2, appears sasureshow 'explanatory :es.The less : this sum of ppendixC.4. of the least r parameters. (b) is a data In panel (a) :askof fitting determining rut along the : of the least amplesizeN rs consistsof n}(x; -t)z ;itive or zero rbservation). naller, since rmerator and rther,leaving
The samplemean of the r-values appearsin cov(b1, b). The absolutemagnitude of the covarianceincreaseswith an increasein magnitudeof the samplemean7, and the covariancehas a sign opposite to that of 7. The reasoning here can be seen from Figure 2. 11. In panel (b) the leastsquaresfitted line must passthrough the point of the means. Given a fitted line through the data, imagine the effect of increasing the estimatedslopeb2.Since the line must passthrough the point of the means,the effect must be to lower the point where the line hits the vertical axis, implying a reduced intercept estimate b1. Thus, when the sample mean is positive, as shown in Figure 2.ll,there is a negativecovariancebetweenthe leastsquaresestimatorsof the slope and intercept.
t.5
The Gauss-Markov Theorem
.\lrat can we say about the least squaresestimatorsb1 and b2 so far? The estimatorsareperfectly general.Formulas (2.7) and(2.8) can be usedto estimate the unknown parametersB1 and B2 in the simple linear regressionmodel no matter what the data turn out to be. Consequently,viewed in this way, the least squares estimators\ and bz arc random variables. The least squaresestimatorsarelinear estimators,as definedin (2.10). Both b1 andb2 can be written as weighted averagesof the -r';values.
32
THE SIMPLE LINEAR
REGRESSION \4OI)EL
If assumptionsSRl-SR5 hold then the least squaresestimatorsare unbiased.This meansthat E(b) : B1 and E(b) : p..
\ - j
We haveexpressionsfor the variancesof b1 andb2and their covariance.Furthermore, we have arguedthat for any unbiasedestimator,having a smallervarianceis better, as this implies we have a higher chanceof obtaining an estimateclose to the true parametervalue. Now we will state and discussthe famous Gauss-Markov theorem,which is proven in Appendix 2F.
GAUSS-MARI(OV THEOREM: Under the assumptionsSR1-SR5 of the linear regressionmodel, the estimatorsb1 and b2 have the smallestvarianceof all linear and unbiasedestimatorsof B1and 82. They are the best linear unbiasedestimators (BLUE) of B1and B2.
-':\ . !,\ -!
:r
Let us clarify what the Gauss-Markov theorem does, and does not, say.
l-{ I.r.r
t . The estimatorsb1 and bzare "best" when comparedto similar estimators,thosethat are linear and unbiased.The theorem doesnot say that b1 and b2 are the best of all possible estimators.
I .:3
2 . The estimatorsb1 and b2arebestwithin their classbecausethey havethe minimum variance.When comparingtwo linear and unbiasedestimators,we always want to usethe one with the smallervariance,sincethat estimationrule givesus the higher probability of obtaining an estimatethat is close to the true parametervalue. -1.
In orderfor the Gauss-Markovtheoremto hold, assumptionsSR1-SR5mustbe true. If anyof theseassumptions arenottrue,thenbl andb2arcnotthebestlinearunbiased estimatorsof By and 82.
A
The Gauss-Markov theorem does not depend on the assumption of normality (assumptionSR6).
5 . In the simple linear regressionmodel, if we want to use a linear and unbiased estimatoq then we have to do no more searching.The estimatorsb1 and b2 arc the onesto use.This explainswhy we are studyingtheseestimators(we would not have you study bad estimation rules, would we?) and why they are so widely used in research,not only in economicsbut in all social and physical sciencesas well.
.-E-..
^=a.-f :.s\: : rle -! :t( .--..: t \:.i ,
-t,
'---,:{
rfl
:':\r'
., .'|
6 . The Gauss-Markov theorem appliesto the least squaresestimators.Itdoes not apply to the least squaresestimatesfrom a single sample.
2.6 The Probability Distributions of the Least Squares Estimators The propertiesofthe leastsquaresestimatorsthat we havedevelopedso far do not dependin any way on the normality assumptionSR6.If we alsomakethis assumptionthat the random !ffors e; are normally distributed with mean 0 and variance o2, then the probability distributionsof the least squaresestimatorsare also normal. This conclusionis obtained
: ::te a\l
2.7 ESTIMATING
nbiased. This Furthermore, ance is better, rse to the true
r is proven in
THE VARIANCE
OF THE ERROR TERM
33
in two steps.First, basedon assumptionSR1, if e; is normal then so is y;. Second,the least squaresestimatorsare linear estimators,of the form bz:Zwiyi, and sums of normal random variables are normally distributed themselves. Consequently, f we make the normality assumption (assumption SR6 about the error term) then the least squares estimatorsare normally distributed.
b ,- N v' " ' r t-t)') i ^) " ( u , .wLe, \'"
(2.r7)
t/1\
bz- N(ur,*;-) of the linear ill linear and tors (BLIIE)
.tors,thosethat : the best of all e the minimum ilways want to :s us the higher Letervalue. 1.5must be true. linearunbiased n of normality r and unbiased 1 and b2 are the would not have widely used in rces as well.
(2.18) 'great
As you will see in Chapter 3, the normality of the least squaresestimators is of importance in many aspectsof statistical inference. What if the errors are not normally distributed? Can we say anything about the probability distribution of the least squaresestimators?The answer is, sometimes,yes. A CENTRAL LIMIT THEOREM: If assumptionsSRI-SR5 hold, andif the sample size N is sufficiently large, then the least squaresestimatorshave a distribution that approximatesthe normal distributionsshownin (2.17) and (2.18).
The million dollar questionis "How large is sufficiently large?" The answeris, there is no specific number. The reason for this vague and unsatisfying answer is that "how large" dependson many factors, such as what the distributions ofthe random errors look like (are they smooth?symmetric? skewed?)and what the xr valuesare like. In the simple regression modelsomewould saythatN: 30 is sufficientlylarge.Otherswould saythatN: 50 would be more a reasonablenumber. The bottom line is, however, that these are rules of thumb, and that the meaning of "sufficiently large" will change from problem to problem. \evertheless, for better or worse, this large sample, or asymptotic, result is frequently invoked in regressionanalysis. This important result is an application of a central limit theorem, like the one discussed in Appendix C.3.4. If you are not familiar with this important theorem, you may want to review it now.
2,7 Estfunating the Variance of the Error Terrn
t does not apply The varianceof the random error term, o2, is the one unknown parameterof the simple linear regressionmodel that remains to be estimated.The variance of the random error e .i s var(e;): 02 : Ele; - E(ei)l' : fk?)
lo not dependin that the random the probability iion is obtained
rf the assumptionE(e1): 0 is correct.Sincethe "expectation" is an averagevaluewe might .'onsider estimating o2 as the averageof the squarederrors,
^) >i
o- :
,1,
34
THE SIMPLE LINEAR
REGRESSION MOI)EL
This formula is unfortunately of no use since the random !rrorS e; are unobservablet However, while the random errors themselvesare unknown we do have an analog to them, namely, the least squaresresiduals. Recall that the random errors are ei:yi-9r-Fz;ri From (2.6) the least squaresresidualsare obtainedby replacing the unknown parametersby their least squaresestimates, 2i:yt-ji:yi-b1-b2x; It seems reasonableto replace the random eflors e; by their analogs, the least squares residuals,so that ^)
o-:
s;?
Ler
N
This estimator,though quite satisfactoryin large samples,is a biased estimatorof 02. There is a simple modification that produces an unbiasedestimator, and that is
.) (, - - L e i N-2
(2.re)
The "2" that is subtractedin the denominator is the number of regressionparameters (91, Bz)in the model, andthis subtractionmakesthe estimator62 unbiased,so that E(62) : 02. 2.7.1 EsrruerrNc rHr VaRTaNCESAND CovenraxcE oF THE Lpesr SquenEs EsrruanoRs Having an unbiasedestimator of the error variancemeanswe can estimatethe variancesof the least squaresestimators b1 and b2, and the covariance between them. Replace the unknown error varianceo2 in (2.14)-\2.16) by 62 to obtain
(2.20)
(2.21)
(2.22)
The squareroots of the estimatedvariancesare the "standarderrors" of b1 and b2.These quantitiesareusedin hypothesistestingandconfidenceintervals.They aredenotedasse(&1) and se(b2)
se(b1):
(2.23)
s e ( b 2 ):
(2.24)
2.7 ESTIMATING
Table 2,3
bservablel rg to them,
THE VARIANCE
OF THE ERROR TERM
L e a s t S q u a r e sR e s i d u a l s e:y-y
v t15.22 135.98 119.34 r14.96 187.05
3.69 4.39 4.',?5 6.03 12.47
'ametersby
35
-5.87
121.09 128.24 131.91 144.98 2t0.73
7;74 -12.51 -30.02 -23.68
ast squares
:.7.2
rator of o2. is
^-ctus make somecalculationsusing the food expendituredata.The least squaresestimates -: the parametersin the food expenditure model are shown in Figure 2.9. First we will -.rmpute the least squaresresiduals from (2.6) and use them to calculate the estimate of r:e error variance in (2.19). In Table 2.3 are the least squaresresiduals for the first five :..useholds in Table2.1. Recall that we have estimatedthat for the food expendituredata the fitted least squares ::Sressionline is ! : 83.42-t 10.21x.For eachobservationwe computethe least squares :e.idual ?i : yi - !;. Using the residualsfor all N: 40 observationswe estimatethe error .'rianceto be
(2.re) tmeters(81, E(62): 02.
variancesof Replace the
(2.20)
CRrcuranroNs Fon rrrp Foon ExprNnrrunE Dera
^) >a?
o-:
N -2
304505.2 :8013.29 38
l:e numerator,304505.2,is the sum of squaredleast squaresresiduals,reported as "Sum *.:raredresid" in Figure 2.9. The denominatoris the numberof sampleobservations,N : 40, --:nus the numberof estimatedregressionparameters,2; the quantityN - 2: 38 is often : ''led the "degreesof freedom" for reasonsthatwill be explainedin ChaEQr3.InFigure 2.9, --c ralue 62 is not reported. Instead, EViews software reports 6 : t/62 : t/8013ts : r"r -ilJ, labeled "S.E. of regression,"which standsfor "standarderror of the regression." It is typical for software not to report the estimated variances and covariance unless ':.:uested. However, all software packagesautomatically report the standarderrors. For .-r:mple, in the EViews output shown in Figure 2.9 the column labeled "Std. Error" . rtainsse(br):43.4landse(b2):2.O9.Theentrycalled"S.D.dependentvar"isthe
*:rple standard deviationof y, thatis r/I(yr -r)'lW
(2.2r) (2.22) tnd b2. These otedasse(01)
The full set ofestimated variancesand covariancesfor a regressionis usually obtained -. i simplecomputercommand,or option,dependingon the softwarebeing used.They are r::l ed in a rectangular array, or matrix, with varianceson the diagonal and covariances - :he "off-diagonal"positions.
l*r@,,br)ii6)
I
:.-r rhefood expendituredatathe estimatedcovariancematrix of the leastsquaresestimatorsis INCOME
(2.23) (2.24)
- 1) : 112.6752.
C INCOME
1884.1,12 -85.90316
-85.90316 4.381752
THE SIMPLE LINEAR
R E G R E S S I O N N IC ) I ) E L
where C standsfor the "constant term," which is the estimatedintercept parameterin the regression,or br; similarly, the softwarereports the variable nameINCOME for the column relating to the estimatedslope b2. Thus : 1884.442, i;6) "*(11-) The standarderrors are
: 4.381752,
t_---^_.
Se(rl):\/varlbrl:t/ v
l ?
*@,,
h- : -85.90316
IBUA42 : 43.410
/se(b2): 1f var(b2): t/4.381752:2.093 Thesevalueswill be usedextensively in Chapter3.
2.8 Exercises Answers to exercisesmarked "*" appearin Appendix D at the end of the book. 2.8.1
PnosrsNrs
2.1 Consider the following five observations.You are to do all the parts of this exercise using only a calculator.
I("i - t)(y' - y) :
(a) Complete the entries in the table. Put the sums in the last row. What are the samplemeansi and y? (b) Calculateb1 and D2using (2.7) and (2.8) and statetheir interpretation. (c) ComputeL?:p?,Ii=rxiyi.Using thesenumericalvaluesshow that I(", - x)2 :Lx!
- Nt'
and I(r, - l)(yi - y) -Lxtyr - Nxy
(d) Use the least squaresestimatesfrom part (b) to computethe fitted valuesofy, and completethe remainderof the table below. Put the sums in the last row. Xi J
2 -l
0 s*.-
I'
5 2 3 2 -2 s . . .-
Yi
ti,.-
s-
_
bl
xiei
s;2 -
s*;
-
2.8 EXERCISES
rameter in the for the column
37
(e) On graph paper, plot the data points and sketch the fitted regression line ji: h t b2xi. (0 On the sketchin part (e), locate the point of the means("r,y). Does your fitted line pass through that point? If not, go back to the drawing board, literally. (g) Show that for these numerical values ! : h -l bzZ. (h) Show that for thesenumericalvalues9 : y, wherei :2i,lt'l . (i) Compute62. () Compute var(b2).
1316
:.1
book.
A householdhas weekly income $1000. The mean weekly expenditure on food for : S125 and expendihouseholdswith this income is E(yl-r : $1000) : pylr:$1000 tures exhibit variancevar(ylx : $1000) : 49. fib:sr*o: (a) Assuming that weekly food expenditures are normally distributed, find the probabilitythat a householdwith this income spendsbetween$110 and $140 on food in a week. Include a sketch with your solution. (b) Find the probability in part (a) if the variance of weekly expenditures is var(ylx: $1000): olp:sr6s6:81.
-ix Graph the following observationsof x and y on graph paper. fthis exercise
(a) Using a ruler, draw a line that fits through the data. Measure the slope and intercept of the line you have drawn. (b) Use formulas (2.7) and (2.8) to compute, using only a hand calculator, the least squaresestimatesof the slope and the intercept. Plot this line on your graph. (c) Obtain the samplemeansof y :ZyilN and 7 : Lx;lN. Obtain the predicted value of y for -x : 7 and plot it on your graph. What do you observeabout this predicted value? (d) Using the leastsquaresestimatesfrom (b), computethe leastsquaresresiduals2;. Find their sum. (e) CalculateL-riOi.
What are the tation. 1l
1at
n
We havedefinedthe simple linear regressionmodel to be ] : Br * \zx * e. Suppose however that we knew, for a fact, that 9r : 0. (a) What does the linear regressionmodel look like, algebraically, if Bt : gr (b) What does the linear regressionmodel look like, graphically, if Br : gr (c) If Pt :0 the least squares "sum of squares" function becomes S(Fz) : ILr (y' - 9r*,)' . Using the data,
lo
ll
plot the value of the sum of squaresfunction for enoughvaluesof B2for you to locate the approximateminimum. What is the significanceof the value of B2 that minimizes S(B2)? (Hint: Your computations will be simplified if you
38
THE SIMPLE LINEAR
REGRESSION MODEL
algebraically expand S(92): L[r0'r _ 9zxi)2by squaring the term in parenthesesand carrying the summation operator through.) (d)OUsing calculus, show that the formula for the leastsquaresestimateof B2in this model is bz : 2 x iyi f Zxf . Use this result to compute b2 and comparethis value to the value you obtained geometrically. (e) Using the estimate obtained with the formula in (d), plot the fitted (estimated) regressionfunction. On the graph locate the point (7, y). What do you observe? (f) Using the estimatesobtained with the formula in (d), obtain the least squares residuals,2i : yt - bzxt.Find their sum. (g) Calculatelx;2;. 2.5 A small businesshires a consultant to predict the value of weekly sales of their product if their weekly advertising is increasedto $600 per week. The consultant takes a record of how much the firm spent on advertising per week and the corresponding weekly sales over the past 6 months. The consultant writes "Over the past 6 months the averageweekly expenditureon advertising has been $450 and averageweekly saleshave been $7500. Based on the results of a simple linear regression,I predict saleswill be $8500 if $600 per week is spenton advertising." (a) What is the estimated simple regression used by the consultant to make this prediction? (b) Sketcha graphof the estimatedregressionline. Locate the averageweekly values on the graph. 2.6* A sodavendorat LouisianaStateUniversityfootball gamesobservesthatmore sodas are sold the warmer the temperatureat gametime. Basedon 32 home gamescovering 5 years,the vendor estimatesthe relationship betweensodasalesand temperatureto be! : -240 + 6x, wherey: the numberof sodasshesellsandx: temperaturein degreesFahrenheit. (a) Interpret the estimatedslopeand intercept.Do the estimatesmake sense?Why or why not? (b) On a day when the temperatureat game time is forecastto be 80'F, predict how many sodasthe vendor will sell. (c) Below what temperatureare the predicted saleszero? (d) Sketch a graph of the estimatedregressionline. 2.7. You have the results of a simple linear regressionbasedon state-leveldata and the District of Columbia. a total of N : 51 observations. (a) The estimatederror variance62 : 2.04672.Whatis the sum of the squaredleast squaresresiduals? (b) The estimatedvarianceof bz is 0.00098.What is the standarderror of b2?What is the value of |,(xi - 7)'? (c) Supposethe dependentvariable )i : the state's mean income (in thousandsof dollars) of males who are I 8 yearsof age or older and x; the percentageof males 18 years or older who are high school graduates.If bz- 0.18, interpret this result. (d) Suppose7:69.139 and y:15.187, what is the estimateof the intercept parameter? (e) Given the resultsin (b) and (d), what is Ixf ? (0 For the Stateof Arkansasthe value of yr : 12.274and the value of xi : 58.3. Computethe leastsquaresresidualfor Arkansas.(Hint: Use the information in parts (c) and (d).).
2.8EXERCISES
e term in parrte of B2in this pare this value .ed (estimated) r you observe? e least squares
r sales of their The consultant week and the t writes "Over been $450 and r simple linear ,n advertising." rt to make this e weekly values that more sodas gamescovering I temperatureto ,temperaturein
39
2.8o ProfessorE.Z. Stuff has decidedthat the least squaresestimator is too much trouble. Noting that two points determinea line, Dr. Stuffchoosestwo points from a sampleof sizeN and drawsa line betweenthem, calling the slopeof this line the EZ estimatorof B2 in the simple regressionmodel. Algebraically, if the two points are (x1, y1) and (xz,yz),theEZ estimationrule is - Yt bu, :!z xz-xl Assuming that all the assumptionsof the simple regressionmodel hold: (a) Show that bs7 is a "linear" estimator. (b) Show that bB2is an unbiasedestimator. (c) Find the variance of b67. (d) Find the probability distribution of b6y. (e) Convince ProfessorStuffthattheBZ estimatoris not as good asthe least squares estimator. No proof is required here. 2.8.2 :.9*
Co^npursn ExEncrses An interesting and useful economic concept is the "learning curve." The idea is related to a phenomenon that occurs in assembly line production, such as in the automobile industry, or any time a task is performed repeatedly.Workers learn from experienceand become more efficient in performing their task. This meansit takes lesstime and labor coststo producethe final product. This idea forms the basisfor an economic model relating cost per unit at time I(UNITCOSZT) to the cumulative production of a good up to, but not including, trme t (CUMPROD).The relationship between the variables is often taken to be UNITCOSTT: UNITCOSTTx CUMPROD\
e sense?Why or )'F, predict how
vel data and the he squaredleast rr of b2?What is iin thousandsof )entageof males 8, interpret this rf the intercept
ue of x; : 58.3. s information in
where UNITCOSZ1equalsthe unit cost ofproduction for the first unit produced,and e equals the elasticity of unit costs with respectto cumulative production (which we expect to be negative). This nonlinear relationship between the variables is transformed to a linear one by taking logarithms of both sides: ln(uN ITC O ST') : ln(U NrcC OSTt) + eln(CU M PROD ) : Fr * $2ln(cuMPROD,) We have "renamed" Ln(UNITCOS4) and e so that the model looks more familiar. Ernst Berndt is the author ofan excellentbook, more advancedthan this one,entitled The Practice of Econometrics: Classic and Contemporary (Addison and Wesley, 1991). On page 85 of that book Berndt gives the example of learning in the production of a product called titanium dioxide, which is used as a thickener in paint. He provides dataon production and unit costsfrom the DuPont Corporationfor the years i955-1970. The data are given in leam.dat. (a) Use your computer software to plot a graph of UNITCOSTagainstCUMPROD, and In(UNITCOST) against ln(C UM PROD). (b) Obtain the least squaresestimatesbl andb2of B 1and B2and give their economic interpretation. Do these numbers make sense?Make a sketch of the fitted regressionline, by hand or using your software, in the plot from part (a). (c) Find the estimated variancesand covariance of the least squaresestimators. (d) Find 62.
40
THE SIMPLE LINEAR
REGRESSION MOI)EL
(e) Predict the unit cost of production when cumulative production is CUMPROD\: 2000. 2.10 The capitalassetpricing model (CAPM) is an importantmodelin the field of finance. It explains variations in the rate of return on a security as a function of the rate of return on a portfolio consisting of all publicly traded stocks, which is called the market portfolio. Generally the rate of return on any investmentis measuredrelative to its opportunity cost,which is the return on a risk free asset.The resulting difference is called the risk premium, since it is the reward or punishment for making a risky investment.The CAPM saysthat the risk premium on securityTis proportional to the risk premium on the market portfolio. That is rj - rf :\i(r^
-
ry)
where 1 and ryare the returns to securityj and the risk-free rate, respectively,r- is the return on the market portfolio, and B; is thejth security's "beta" value. A stock's betais important to investorssinceit revealsthe stock'svolatility. It measuresthe sensitivity of security j's return to variation in the whole stock market. As such, valuesof beta less than I indicate that the stock is "defensive" since its variation is less than the market's. A beta greater than I indicates an "aggressivestock." Investorsusuallywant an estimateof a stock'sbetabeforepurchasingit. The CAPM model shown above is the "economic model" in this case. The "econometric model" is obtainedby including an interceptin the model (eventhough theory saysit should be zero) and an error term, ri-11:a1-lB1Q^-ry)+e (a) Explain why the econometric model above is a simple regressionmodel like those discussedin this chapter. (b) In the datafile capm2.dat aredataon the monthly returnsof six firms (Microsoft, GE, GM, IBM, Disney, and Mobil-Exxon), the rate of return on the market portfolio (MKT), and the rate of return on the risk free asset(RKFREE). The 120 observationscoverJanuary 1995 to December2004. Estimate the CAPM model for each firm, and comment on their estimatedbetavalues. Which firm appears most aggressive?Which firm appearsmost defensive? (c) Finance theory says that the intercept parameter cr; should be zero. Does this seem correct given your estimates?For the Microsoft stock, plot the fitted regressionline along with the data scatter. (d) Estimate the model for each firm under the assumptionthat oi :0. Do the estimatesof the beta values change much? 2.lI
The file br2.dat containsdataon 1080housessold in BatonRouge,Louisianaduring mid-2005. The data include sale price, the housesize in squarefeet, its age, and whether it has a pool or fireplace or is on the waterfront. Also included is a variable nameds4,/eincluded in the realtor's description.Variable descriptionsare in the file br2.def. (a) Plot houseprice againsthousesizefor all housesin the sample.Constructanother plot for housesof traditional style. (b) Estimatethe regressionmodel PRICE : Fr f gzSQFT * e for all the housesin the sample. Interpret the estimates.Draw a sketch of the fitted line. (c) Estimatethe regressionmodelin (b) usingonly traditionalstylehouses.Interpret the estimates.How do the estimatesseemto compareto those in (bX
2.8 EXERCISES
41
duction is
(d) For eachof the regressionsin (b) and (c) computethe least squaresresidualsand plot them against SQFT. Do any of our assumptionsappear violated?
1offinance. : the rate of s called the rredrelative g difference king a risky tionaltothe
l. l2* The fiIe stockton2.datcontainsdataon 880 housessold in Stockton,CA, during mid2005. Variable descriptions are in the file stockton2.def. (a) Plot house price against house size for all housesin the sample. (b) Estimatethe regressionmodel PRICE : 9r * gzSQFT * e forall thehousesin the sample. Interpret the estimates.Draw a sketch of the fitted line. (c) Estimatethe regressionmodel in (b) using only housesthat arevacantat the time of sale.Repeatthe estimation for housesthat were occupied (not vacant)at time of sale. Interpret the estimates.How do the estimatesseem to compare to each other? (d) For each of the regressionsin (c) compute the least squaresresiduals and plot them against SQFT. Do any of our assumptionsappear violated? (e) Predict the price of a house with 2000 squarefeet of living area.
:tively, r. is re. A stock's neasuresthe et. As such, ; variation is sive stock." . TheCAPM econometric .heorysaysit
n model like s (Microsoft, r the market UE).Thel2O IAPMmodel firm appears ro. Does this lot the fitted :0.
Do the
isiana during , its age, and I is a variable are in the file itruct another the housesin 1e.
.ses.Interpret
(bx
:.13
One would suspectthat new home constructionand sales depend on mortgage interestrates.If interestrates are high, fewer people will be able to afford to borrow the funds necessaryto finance the purchaseof a new home. Builders are aware of this fact, and thus when mortgageinterestratesare high, they will be lessinclined to build new homes. While this is intuitively reasonable, let us ask the question "If mortgage interest rates go tp by l%o, how much does home construction fall?" Data on the 30-year fixed mortgage rate, housing starts (thousands),and houses sold (thousands)are contained in the file house_starts.dat.There are 184 monthly observationsfrom January 1990 to April 2005. (a) Plot each of the series against time. (b) Plot housing starts (STARfg against the 3O-yearfixed mortgage rate (FIXED_RATD. (c) Estimate the simple regression of STARZS on FIXED_MZE. Discuss the interpretation of the results as well as any comments you may have about how well the line fits the data. Plot the fitted regressionline along with the data scatter from (b). (d) Plot housessold (SOID) against FIXED_MTE. (e) Estimate the simple regression of SOLD on FIXED_RAZE. Discuss the interpretation of the results as well as any comments you may have about how well the line fits the data.Plot the fitted regressionline along with the data scatterfrom (d). (f) If the 30-year fixed rate mortgage rate is 6Vo,predict the number of monthly housing starts.
- , i " ProfessorRay C. Fair hasfor a number of yearsbuilt and updatedmodelsthat explain and predict the U.S. presidential elections. See his website at http://fairmodel. econ.yale.edu/vote2008/index2.htm,and see in particular his paper entitled "A Vote Equation for the 2004 Election." The basic premise of the model is that the incumbent party's shareof the two-party (Democratic and Republican)popular vote (incumbent means the party in power at the time of the election) is affected by a number of factors relating to the economy and variablesrelating to the politics, such as how long the incumbent party has been in power and whether the President is running for reelection.Fair's data,3 I observationsfor the election yearsfrom 1880to 2000,are inthefilefairdat.The dependentvariableisVOTE: percentageshareof the popular vote won by the incumbent party. Consider the explanatory variable
42
THE SIMPLE LINEAR
REGRESSION MODEL
growth rate in real per capita GDP in the first three quarters of the GROWH: election year (annual rate). One would think that if the economy is doing well, and growth is high, the party in power would havea betterchanceof winning the election. (a) Plot a scatter diagram of VOTE against GROWH. Does there appear to be positive association? (b) Estimatethe regressionvoTE : fu -l 7zGROWH + e by leastsquaresusing all the data from 1880 to 2000. Report and discussthe estimation result. Sketch, by hand, the fitted line on the data scatter from (a)' (c) Economy wide inflation may spell doom for the incumbent party in an election. The variable INFLATION is the growth in prices over the first 15 quartersof an administration. Plot VOTE againstINFIATION. Report and discuss the estimation results. 2.15 How much does education affect wage rates?The data fi7ecps-small.dat contains 1000observationson hourly wagerates,education,andother variablesfrom the 1997 Current Population Survey (CPS). (a) Obtain the summary statistics and histograms for the variables WAGE and EDUC. Discuss the data characteristics. (b) Estimate the linear regressionWAGE: 9t * \2EDUC * e and discussthe results. (c) Calculate the least squaresresiduals and plot them against EDUC. Are any patternsevident?If assumptionsSRl-SR5 hold, should any patternsbe evident in the least squaresresiduals? (d) Estimate separateregressionsfor males, females,blacks, and whites. Compare the results.
Appendix 2A Derivation of the Least Squares Estitnates Given the sample observations on y and .r, we want to find values for the unknown parametersB1 and 92 that minimize the "sum of squares" function
s(Br,9z): ILr(v, - Br - gzxi)2
(2A.1)
Since the points (y;,x;) have been observed,the sum of squaresfunction S dependsonly on the unknown parameters B1 and 82. This function, which is a quadratic in terms of the unknown parametersB1 and B2, is a "bowl-shaped surface" like the one depicted in Figure 2A.1. our task is to find, out of all the possiblevalues B1and B2,the point (b v b) at which the sum of squaresfunction S is a minimum. This minimization problem is a common one in calculus, and the minimizing point is at the "bottom of the bowl." Those of you familiar with calculus and "partial differentiation" can verify that the partial derivatives of S with respect to B1 and B2 are dJ
E9t
as 092
2N9, - 2Zyi -r 2(}xi)Bz (21'.2) z(L x?)Pz- 2L xiyi + 2(I -rr)9r
APPENDIX 28 DEVIATION
FROM THE MEAN FORM
OF b"
uarters of the ring well, and g the election. appear to be squaresusing result. Sketch, in an election. quartersof an scussthe esti.ll.dat contains rfromthe 1997 es WAGE and nd discussthe
rtcvnp 2A.7
functionandthe minimizingvaluesb1 andb2. The sumof squares
Thesederivativesare equationsofthe slope ofthe bowl-like surfacein the directions ofthe a.res.Intuitively, the "bottom of the bowl" occurs where the slope of the bowl, in the directionof each axis, 0S/dB1 and 0Sl0B2, is zero. Algebraically, to obtain the point (h, b) we set equations(2A.2) to zero and replace B1 and B2 by b1 and 12, respectively,to obtain
zf\ri - Nh - (bi)bz]: o
DUC. Are any erns be evident hites. Compare
r the unknown
2[Lxiyi- (Ixr)br- (Zt)uz]: o Simplifying these gives equationsusually known as the normal equations,
&2)at which the common one in Lverify that the
(24.3)
( 2 x ; ) b 1 +( 2 f ) b z : Z x i y i
(2A.4)
Thesetwo equationshave two unknowns bt andbz. We can find the least squaresestimates rr solving thesetwo linear equationsfor b1 andb2.To solve for b2 multiply (2{.3)by Zxi, rrultiply (2A.Oby N,then subtractthe first equationfrom the second,and then isolate D2on Jre left-hand side.
(24.r) S dependsonly ltic in terms of one depicted in
Nh -r (Zxi)bz:}yi
, D2:
Nlxiyi -Lxilyi
(2A.s)
N>f,, - (>)rif
This formula for b2is in terms of datasums,cross-products,and squares.The deviation from :he mean form of the estimator is derived in Appendix 28. To solve for 01, given bz, divide both sides of (2A.3) by N and returange.
Appendix 2B Deviation frotn the Mean Fortn of b2 Tlrefirst stepin the conversionof the formula for b2into (2.7) is to use sometricks involving 'ummation signs. The first useful fact is that
(2A.2)
I(xi - i)z :Lt
- 272xi+ Nl = :d
:2*?-2NT2+Nx2:I-i
- o('*r",) -
N'.2
+ Nrz
(2B.1)
44
THE SIMPLE LINEAR
REGRESSION MODEL
- l)2,usingtheshortcutformulaL(ri-7)' :2*? ShouldyoueverhavetocalculateI(x; lf l is much easier.Then
I (r,- i )' :2r 7 - Nf :2*? - xlxi :Z*? -
#
-
( 28.2)
To obtain this result we have usedthe fact that 7:lx;f N, so lx; :7tg;. The seconduseful fact is similar to the first, and it is
L(*, -7)(y,- y) :ZxJi - Nry :1xiyi-
+
QB.3)
This result is proven in a similar manner. If the numerator and denominator of b2 in equation (2A.5) are divided by N, then using (28.1)-(28.3) we can rewrite b2in deviationfrom the meanform as
-;)0r -Y-)
o, r : =L(*, Q,_;a
Iftis formula for b2is one that you shouldremember,as we will useit time and time again in the next few chapters.
Appendix 2C b2Is a Linear Estirnator In order to derive (2.10) we make a further simplification using another property of sums. The sum of any variable about its averageis zero, that is, L(xi-t):0 Then, the formula for b2 becomes
ur:\ffiA | ( x i - r ) y i: - f
t6=;f
(*,-t)l : FLwiYi 'l-x" Yi t,1
where w; is the constantgiven in (2.11).
Appendix 2D Derivation of Theoretical Expression for b2 To obtain (2.12) replaceyi in (2.10) by y, :9r bz : Lwiyi:
* B2x;I ei and simplify:
Iwi(9r * gzxi * et)
: prlwi llzLwixi
lLwiet
: 9z l}w;e; We used two more summationtricks to simplify this. First, Lwi :0, this eliminatesthe term B1)w;. Secondly,Ltt';x;: l, so B2lw;"v,:9:, and (2.10)simplifiesto(2.12).
APPENDIX
,\2 _ - LS^ i - 2 _
The term Lwi :0
THE VARIANCE
OF D2
45
because 1
I
(28.2)
2E DERIVING
: ='f---r(', :: - " l!:iLl ' - r) : o Lwi -;)tl ,)'-' I('' fl,t'' wherein the last stepwe usedthe fact that I("t - t) : 0' for L(r; - ;)2 is ToshowthatI wixi :1 weagainuse)('r; - ;) : 0' Anotherexpression I(", - x)2 :\(x, - x)(x;- r) : I(xi - *)ri -72(xr -t) : !(x; - -).r; Consequently
V, then using
L(x; - r)x; : , , IG;p,
L(xi - x)xi
Lwixi:
>;_*:
Appendix 2E Deriving
the Variance of b2 The least squaresestimatoris a random
The starting point is (2.I2), bz:Fz*Zwiei. rariable whose variance is defined to be
var(b2): Elbz- E(bz)l'
rertyof sums.
Substitutingin(2.12)andusingtheunbiasednessoftheleastsquaresestimator,E(b2):82, *e have var(b2): E(92 +7wiei - 9)2
: E(Zr,r,), : E(2,7"? *222wp1e;e1)
term) (squareof bracketed
: 2w?E (e?)-f 2LL wiw1E (e;e) i+i : o'Lwi
(because w; not random)
:-
oZ
2(x;-r)'
The next to last line is obtained by using two assumptions:First, o2 : Second,cov(e;,e) : usesthe fact that
var(e) : Elei - n(",))' : E(et - 02 : n('?) El(ei- E(e))(e1- E(ej))): E(eie):0'
Then' the very last step
f
I ("'-t"..rl L*?:rl, -7)" : j9-+:t= Z(xi-t)z _
fiI(",
j I
tir.,
il'\
Alternatively, we can employ the rule for finding the variance of a sum. If X and I are random variables, and a and b are constants,then vat(aX + bY) : azvat(X) + b2var(Y) r 2ab cov(X'Y)
THE SIMPLE LINEAR
REGRESSION MODEL
Appendix B.4 reviews all the basic propertiesof random variables.In the secondline below we use this rule extendedto more than two random variables. Then, : var(}wiei)
var(b2): var(92+lwiei)
: \w! va{e ;) +
iwi cov(e;, e)
fiw
(sinceB2is a constant)
(generalizing the variancerule)
:\w?var(ei)
(using cov(e;,e.,): 0)
: o22r?
(using var(e;) : o')
o2
Z(xi -t)' Carefully note that the derivation ofthe varianceexpressionfor D2dependson assumptions SR3 and SR4. If cov(e;. e1)lO then we cannot drop out all those terms in the double summation. If var(e;) f oz for all observations then o2 cannot be factored out of the summation.If either of theseassumptionsfails to hold then var(b) is somethingelse andis not given by (2.15).The sameis true for the varianceof b1 and the covariance.
Appendix 2F Proof of the Gauss-Markov Theorern We will prove the Gauss-Markov theorem for the least squaresestimatorb2 of B2.Our goal is to show that in the classof linear and unbiasedestimatorsthe estimatorb2hasthe smallest variance. Let bi: fk;y; (where k; are constants)be any other linear estimator of B2. To make comparisonto the leastsquaresestimator b2easier,supposethatk; : wi I ci, where c; is anotherconstantand w; is given in (2.11). While this is tricky, it is legal, since for any t; that someonemight choosewe can find c;. Into this new estimator substitute/; and simplify, using the properties of w; in Appendix 2D b) : lkiyi :|(wi
-l ci)yi: I(r;
+ cr)(81't gzxi -l ei)
: I(wi + cr)81* I(w; + ci)gzxi* I(wi I c;)e; : BrIw; * grlci l gz}wixi t FzLcixi+ I(w; I c;)e1 : BrIcr * Fz + 9zLcixi + I(wi -f c;)e;
(2F.1)
sincefw; : 0 andZw;x;:1. Takethe mathematicalexpectationof the last line in (2F.1),using the propertiesof expectationandthe assumptionthatE(ei) : Qi E(b;) :9rIci
+ Fz* gzLcixi+ I(w, 'r ci)E(ei)
: Brlci f 9z * lz}riri In orderfor the linearestimatorb):
(2F.2)
lkiyi to be unbiased,it mustbe true that
Ici : 0 and Lcixi :0
(2F.3)
These conditions must hold in order for b):Zk* to be in the class of linear and unbiased estimators.So we will assumethat conditions (2F.3) hold and use them to simplify expression(2F.1):
bi : Lkli:
9z * 2(*, + ,t)r'
(2F.4)
'A: i '. :l
l
APPENDIX
ond line below
2F PROOF OF THE GAUSS-MARKOV
THEOREM
47
\\'e can now find the variance of the linear unbiased estimator bj following the steps in ,\ppendix 2E and using the additional fact that
nt) nce rule)
-t- L:&a4l Lc;w; - r)'l [I(rr
,o) ,)
Z(xi - x)'
-=i zcixi
- r., : o
2(x; -x)"-
L-sethe properties of variance to obtain varl9z * I(wi * ci)el : L(wi + c;)zvar(ei)
var(b): : on assumptions s in the double lred out of the :hing else andrs iance.
re1rr zof 92.Our goal hasthe smallest mator of B2. To tlr * ci,wherec; , since for any k; )); and simplify,
(2F.1)
he properties of
(2F.2)
re that (2F.3) ;s of linear and nd use them to
(2F.4)
o22(wi + cr)z: o22w! * o22c!
: var(bz) -t o22c! > var(b2) The last line follows since)cf ) 0 and establishesthat for the family of linear and unbiased stimators bi, eachof the alternativeestimatorshas variancethat is greaterthan or equal to 5at of the least squaresestimator b2.The only time that var(bj) : var(bz) is when all the -- : 0, in which caseb\: Dz.Thus there is no other linear and unbiasedestimator of S2 :iat is better than b2, which proves the Gauss-Markov theorem.
Ir p
Chapter
a
d b n \[
Interval Estimation and Hypothesis Testing
a S
b 4
Learning Objectives
o q
Based on the material in this chapter, you should be able to l.
Discuss how "repeated sampling theory" relates to interval estimation and hypothesistesting.
2. Explain why it is important for statisticalinferencethat the least squaresestimators b1 and b2 are normally distributed random variables.
l -
S
n
3. Explain the "level of confidence" of an interval estimator, and exactly what it means in a repeatedsampling context, and give an example. 4. Explain the difference between an interval estimator and an interval estimate. Explain how to interpret an interval estimate. 5. Explain the terms null hypothesis, alternative hypothesis, and rejection region, giving an example and a sketch of the rejection region. 6. Explain the logic of a statisticaltest,including why it is important that a test statistic have a known probability distribution if the null hypothesis is true. 7 . Explain the term p-value and how to use a p-value to determine the outcome of a hypothesis test; provide a sketch showing a p-value. 8. Explain the difference betweenone-tail and two-tail tests.Explain, intuitively, how to choosethe rejection region for a one-tail test. 9. Explain Type I error and illustrate it in a sketch.Define the level of significanceof a test. 10. Explain the difference between economic and statistical significance. 11. Explain how to choosewhat goes in the null hypothesis,and what goes in the alternative hypothesis.
Kevwords alternative hypothesis confidence intervals critical value degreesof freedom hypotheses hypothesistesting inference 48
interval estimation level of significance null hypothesis one-tail tests point estimates probability value p-value
rejection region test of significance test statistic two-tail tests Type I error Type II error
S
h
t
I S p \
lr
e t
3.1 INTERVAL ESTIMATION
estimation and luares estimators I exactly what it nterval estimate. rejection region, .hata test statistic ue. the outcome of a , intuitively, how 'significanceofa ance. vhat goes in the
In Chapter 2 we used the least squaresestimators to develop point estimates for the parametersin the simple linear regressionmodel. These estimatesrepresentan inference aboutthe regressionfunction E(y) : 9 t * pzx describinga relationshipbetweeneconomic tariabTes.Infermeans"to concludeby reasoningfrom somethingknown or assumed."This dictionary definition describesstatisticalinference as well. We have assumeda relationship betweeneconomicvariablesandmadevariousassumptions(SR1-SR5) aboutthe regression model.Basedon theseassumptions,andgiven empirical estimatesof regressionparameters, * e want to make inferencesabout the population from which the data were obtained. In this chapterwe introduce additional tools of statisticalinference: interval estimation and hypothesis testing. Interval estimation is a procedure for creating ranges of values, sometimescalled confidence intervals, in which the unknown parametersare likely to be located. Hypothesis tests are proceduresfor comparing conjecturesthat we might have aboutthe regressionparametersto the parameterestimateswe have obtainedfrom a sample of data.Hypothesistestsallow us to say that the data arecompatible, or are not compatible, sith a particularconjectureor hypothesis. The proceduresfor hypothesis testing and interval estimation depend very heavily on usumption SR6 of the simple linear regressionmodel and the resulting normality of the ieast squaresestimators.If assumption SR6 does not hold, then the sample size must be 'ufficiently large so that the distributions of the least squaresestimators areapproximately normal. In this case the procedureswe develop in this chapter can be used but are also approximate.In developingthe proceduresin this chapterwe will be using the "Student's" r-distribution. You may want to refresh your memory about this distribution by reviewing AppendixB.5.3.Also, it is sometimeshelpful to seethe conceptswe areabouttodiscussin a .impler setting. In Appendix C we examine statistical inference, interval estimation, and h1'pothesistesting in the context of estimating the mean of a normal population. You may $ ant to review this material now, or read it along with this chapter as we proceed.
3.7 Intewal Estirnation In Chapter 2 we estimatedthat householdfood expenditurewould rise by $10.21 given a S100increasein weekly income.The estimateb2:10.21isapoint estimateof the unknown population parameterB2 in the regressionmodel. Interval estimation proposesa range of r alues in which the true parameter B2 is likely to fall. Providing a range of values gives a 'enseof what the parametervalue might be, and the precision with which we haveestimated rr. Such intervals are often called confidence intervals. We prefer to call them interval esfimatesbecausethe term "confidence" is widely misunderstoodand misused.As we will i!e, our confidenceis in the procedurewe use to obtain the intervals, not in the intervals rhemselves.This is consistent with how we assessedthe properties of the least squares estimatorsin Chapter2. 3.1.1
:ction region . of significance . statistic ptail tests re I error re II error
49
TnE /-DrsrRrBUTroN
Let us assumethat assumptionsSR1-SR6 hold for the simple linear regression model. In this casewe know that the least squaresestimatorsbl and byhavenormal distributions, as discussedin Section 2.6. For example, the normal distribution of b2, the least squares e s t i m a t oor f B z ' i s (
or
., ^ \ bz-N l B : . _ ; ; - x) z(-\', \ |
INTERVAL ESTIMATION
AND HYPOTHESIS TESTING
A standardizednormal random variable is obtained from b.t bv subtractins its mean and dividing by its standarddeviation:
Z_
bz-92
-N(0,1)
(3.1)
The standardizedrandom variable Z is normally distributed with mean 0 and variance 1. Using a table of normal probabilities (Table 1 at the end of the book) we know that
P ( - r . 9 6 < Z < 1 . 9 6:)0 . 9 s Substituting(3.1) into this expressionwe obtain
I
P t - l . e 6- < - L = ,.*) oz x)2 ,f 121x, \
:0.95
Rearranging gives us
r(u,-,sa
o2l\(x;
< 9z < bz -l1.96 ozlL(x;
r')
:0.95
This defines an interval that has probability 0.95 of containing the parameter 82. The two -\ | provide an interval estimator.In repeatedsamendpoints (ur+rca1f o2l\(xi -4') pling 95Voof the intervals constructedthis way will contain the true value of the parameter B2.This easy derivation of an interval estimator is basedon both assumption SR6 andthat we know the varianceof the error term o2. While we do not know the value of o2 we can estimateit. The least squaresresidualsare ?i:li-br-bzxtandourestimatorofo2is62: > A ? l ( N - 2 ) . R e p l a c i nog' b y 6 2 i n ( 3 . 1 ) createsa random variable we can work with, but this substitution changesthe probability distribution from standardnormal to a r-distribution with N - 2 degreesof freedom, bz-92
o21L1x,
:-
bz-Fz bz-92 - tlyv-z) se(br) l-. V var\D2)
-ts-z) ror k:1,2
- 't -cl
(3.2) -'\l
Theratiot:(bz-82)lse(b2)hasar-distributionwithN-2degreesoffreedom,which we denoteas t - tw-z\. A similar result holds for bl, so in generalwe can say,if assumptions SRI-SR6 hold in the simple linear regressionmodel, then
,:uFP
: '1
r3 ?)
This equation will be the basis for interval estimation and hypothesistesting in the simple linear regressionmodel. The statisticalargumentof how we go from (3.1) to (3.2) is in Appendix 3A, at the end of this chapter. When working with the r-distribution rememberthat it is a bell-shapedcurve centeredat zero. It looks like the standardnormal distribution, exceptit is more spreadout, with a larger varianceand thicker tails. The shapeofthe r-distribution is controlled by a single parameter called the degreesof freedom, often abbreviatedas dl We use the notation /1.; to specify a r-distribution with m degreesof freedom. In Table 2 at the end of the book (and inside the
'Ai
:\t
:i .{l
3.1 INTERVAL ESTIMATION
51
its mean and
(3.1)
d variance 1. row that -tc i tccns
3,7
0
Critical values from a r-distribution.
iront cover) are percentilevaluesof the t-distribution for various degreesof freedom.For nt Jegreesoffreedom the 95th percentileofthe r-distribution is denoted/1o.es, n; . This value has :he propertythat 0.95 of the probability falls to its left, so P[11.y< tegs,d]:0.95. For erample,if the degreesof freedomarem:20, then,from Table2, tegs,zo): 1.725.Should r ou encountera problem requiring percentilesthat we do not give, you can interpolatefor an epproximateanswer,or use your computer software to obtain an exact value. = 0.95
3.1 .2
: B2.The two :peated sam.heparameter SR6 and that residualsare by 62in 1:.t; e probability lreedom, (3.2)
edom, which assumptions
.
(3.3)
in the simple to (3.2) is in e centeredat with a larger ;leparameter ; to specify a nd inside the
OsrArNrNc IxrEnver Esrrnanrs
From Table 2 we car.find a "critical value" /" from a l-distribution such that P(r ) t.) : P t 1 - t,) : uf2,where a is a probability often taken to be cr : 0.01 or a : 0.05. The .-ritical value /. for degreesoffreedom rn is the percentile value tg-.o12,*1. The valuesl. and -t.. are depictedin Figure 3.1. Eachshaded"tail" areacontainsef Zofthe probability, so that I - crof the probability is .'r)ntainedin the center portion. Consequently,we can make the probability statement P(-t,
(3.4)
For a95%oconfidenceinterval the critical valuesdefine a central region of the r-distribution iontaining probability I - cr :0.95. This leaves probability a :0.05 divided equally ttween the two tails, so that el2:0.025. Then the critical value /.:t11-o.ozs,-): | ,gt5.n).In the simple regressionmodel the degreesof freedom are m:N-2, so erpression(3.4) becomes : 0.95 P [- tlo.rzs,rvz1I t < tp.ots,w-21) \\'e find the percentile values /1o.szs,w-zy in Table 2. Now, let us seehow we can put all thesebits together to create a procedurefor interval .stimation. Substituter from (3.3) into (3.4) to obtain
pl -,.- -
se(rr)
Rearrangethis expressionto obtain
Plh - t,se(bp)< 9r < b* -t t,se(br)]: 1 - a
(3.s)
The interval endpoints h - tcse(br) andb1 -t t.se(b1) are random becausethey vary from -rmple to sample. These endpoints define an interval estimator of Be. The probability
52
INTERVAL ESTIMATION
AND HYPOTHESIS TESTING
statementin (3.5) saysthat the interval bp i t"se(ba) hasprobability I - ct of containing the true but unknown parameter 81. When bpand se(Da)in (3.5) are estimatedvalues(numbers),basedon a given sampleof data, then b2 i t,se(b) is called a 100(l - a)Vo interval estimate of B7..Equivalently it is calleda100(l - a)Toconfidenceinterval. Usuallya : 0.01oro :0.05, sothatweobtain a997o confidenceinterval or a95Vo confidenceinterval. The interpretationof confidenceintervals requiresa greatdealof care.The propertiesof the interval estimation procedureare basedon the notion of repeatedsampling. If we were to select many random samples of size N, compute the least squaresestimate bp and its standarderror se(bp)for each sample,and then constructthe interval estimatebp I t,se(bp) for each sample, then 100(l - u)Vo of all the intervals constructedwould contain the true parameter Bs Any one interval estimate,basedon one sampleof data,may or may not contain the true parameterB1,and becauseB1is unknown, we will never know if it does or doesnot. When "confidence intervals" are discussedrememberthat our confidenceis in thepro cedureused to construct the interval estimate; it is not in any one interval estimate calculated from a sample of data.
1-rrtt . elu :hatt
3.1.3
hS
AN IrrusrnerroN
\iloo
\\ rodr
:l\ eI
3\ttn .rea.
--rrr '\\'t
.rm
:.tin .nc
-Por 1.1
.ilO!
For the food expendituredata,N : 40 andthe degreesoffreedom areN - 2 : 38.For a95%o : /1o.lzs,:a) : 2.024 is the confidenceintervalct: 0.05. The critical value /, : tg-e1z,w-21 97.5 percentile from the r-distribution with 38 degreesof freedom. For B2 the probability statementin (3.5) becomes
\ITli
I:r T i
-Jlll
This vJTl
:0.95 < 9z < bz-t 2.024se(bz)] P[b, - 2.024se(b2) To constructan interval estimatefor B2 we usethe least squaresestimatebz : standarderror
(36) 10.21andits
:]
S relu ::re
t-
: t/4.38:2.09 se(b) : 1fvar(b2)
"la i\P
Substitutingthesevaluesinto (3.6) we obtaina"95Vo confidenceintervalestimate"for B2:
:
b2 t t,se(b2): 10.21+2.024(2.09) : [5.97,14.45] That is, we estimate"with957o confidence"that from an additional$100of weekly income householdswill spendbetween$5.97 and $14.45 on food. Is B2 actually in the interval [5.97, 14.45]? We do not know, and we will never know. What we do know is that when the procedurewe usedis applied to many random samplesof datafrom the samepopulation, then95%oof all the interval estimatesconstructedusing this procedurewill contain the true parameter.The interval estimation procedure"works" 95Vo of the time. What we can say about the interval estimate basedon our one sample is that, given the reliability of the procedure,we would be "surprised" if B2 is not in the interval
T.t i:II
14.4sl. 1s.97,
What is the usefulnessof an interval estimateof B2?When reporting regressionresultswe alwaysgive a point estimate,suchasbz:10.21. However,thepoint estimatealonegivesno senseof its reliability. Thus, we might also report an interval estimate.Interval estimates incorporateboth the point estimateand the standarderror of the estimate,which is a measure of the variability of the least squaresestimator.The interval estimateincludes an allowance
I
3.1 INTERVAL ESTIMATION
containing the ;iven sampleof luivalently it is ) that we obtain te propertiesof ing. If we were nate bp and its e bp * t"se(bp) ontain the true :ontain the true loes not. When ryocedureused culated from a
for the sample size as well, becausefor lower degreesof freedom the r-distribution critical value /. is larger.If an interval estimateis wide (implying a large standarderror), it suggests that there is not much information in the sampleabout B2.If an interval estimateis narrow,it suggeststhat we have learned more about B2. What is "wide" and what is "narrow" dependon the problem at hand.For example,in our model b2 : 10.21is an estimateof how much weekly householdfood expenditurewill rise givena $100increaseinweeklyhouseholdincome. ACEO of a supermarketchain canusethis estimateto plan future store capacity requirements,given forecastsof income growth in an area.However, no decision will be basedon this one number alone. The prudent CEO will cany out a sensitivity analysisby considering values of B2 around 10.21. The question is "Which values?" Oneanswerisprovidedbythe interval estimate15.91,14.4sl.While B2may or may not be in this interval, the CEO knows that the procedureusedto obtain the interval estimate"works" 95Voof thetime. If varying B2within the interval hasdrasticconsequences on company salesand profits, then the CEO may concludethat there is insufficient evidence upon which to make a decision and order a new and larger sampleof data. 3.1.4
=38.Fora95Vo : 2.024is the the probability
(3.6) = 10.21andits
imate" for B2:
THr RrprerED SAMpLTNG CoNTExr
In Section 2.4.3 we illustrated the sampling properties of the least squaresestimatorsby showing what would happenif we collected 10 additional samplesof size N : 40 from the same population that gaveus the food expendituredata.The dataare in the file table2-2.dat. In Table 3.1 we presentthe least squaresestimates,the estimatesof o2, and the coefficient standarderrorsfrom eachsample.Note the samplingvariation illustratedby theseestimates. This variation is due to the simple fact that we obtained 40 dffirent householdsin each sample.The 957oconfidenceinterval estimatesfor the parametersB 1and B2aregiven in Table 3.2 for the same samples. Sampling variability causesthe centerof eachof the interval estimatesto changewith the valuesof the least squaresestimates,and it causesthe widths of the intervals to changewith the standarderrors. If we ask the question "How many of these intervals contain the true parameters,and which onesare they?" we must answerthat we do not know. But since 957o of all interval estimatesconstructedthis way contain the true parametervalues, we would expect perhaps 9 or 10 of these intervals to contain the true but unknown parameters. Note the difference betweenpoint estimation and interval estimation. We have used the least squaresestimatorsto obtain point estimatesof unknown parameters.The estimated
veeklyincome
Table 3.7
ll never know. cm samplesof ;ted usingthis "works" 95Vo ample is that, in the interval
Sample
;ionresultswe llone gives no rval estimates h is a measure an allowance
53
I 2 J 4
5 6 7 8 9 10
Least SquaresEstirnatesfrorn 10 Random Samples br 131.69 57.25 103.91 46.50 84.23 26.63 64.21 79.66 97.30 95.96
se(br) 40.58 JJ.IJ
3'7.22 33.33 41.15 45.78 32.03 29.8'7 29.14 37.18
bz
6.48 10.88 8.14 I 1.90 9.29 IJ.f)
10.93 9.76 8.05 7.77
se(b2)
r.96 1.60 r.79 1.61 1.98 2.21 1.54 1.44 t.4r r.79
62
7002.85 4668.63 589r.75 4722.58 7200.16 8 9 11 . 4 3 4362.12 3793.83 3610.20 5878.71
INTERVAL ESTIMATION
54
Table 3.2
AND HYPOTHESIS TESTING
3.2.
IntervalEstirnates frorn 10 Randorn Samples
Sample
bt - t,se(bt)
I 2 3 4 5 6 'l
49.54 -9.83 28.56 -20.96 0.93 -66.04 -0.63 t9.t9 38.32 20.69
8 9 10
br I t,se(br)
213.8s 124.32 r79.26 1t3.97 t6'7.53 119.30 129.05 140.13 156.29 171.23
b2 - tase(b2)
2.52 '7.65 4.51 8.65 5.27 9.08 7.81 6.85 5.2r 4.14
bz -t t,se(bz)
r0.44 14.12 11.77 1 5 .51 13.30 18.02 14.06 t2.68 10.89 11.40
The t parar as Hr regre the s
3.2.
Paire if th iom(
alter
variance;;6),for ft : 1 or 2, andits squareroot ,m: se(br) provideinformation about the sampling variability of the least squares'estimatorfrom one sample to another. Interval estimatorsare a convenientway to report regressionresultsbecausethey combine point estimationwith a measureof samplingvariability to provide a rangeof valuesin which the unknown parametersmight fall. When the sampling variability of the least squares estimator is relatively small, then the interval estimateswill be relatively narrow, implying that the least squaresestimatesare "reliable." If the least squaresestimators suffer from large sampling variability, then the interval estimateswill be wide, implying that the least squaresestimatesare "unreliable."
a
3,2 Hypothesis Tests Many businessand economic decision problems require a judgment as to whether or not a parameteris a specific value. In the food expenditureexample,it may make a good deal of difference for decision purposes whether B2 is greater than 10, indicating that a $100 increasein income will increaseexpenditureon food by more than $10. Also, basedon economic theory, we believe that$2 should be positive. One check of our data and model is whether this theoretical proposition is supportedby the data. Hypothesis testing procedurescompare a conjecturewe have about a population to the information contained in a sample of data. Given an economic and statistical model, hypothesesare formed abouteconomic behavior.Thesehypothesesare then representedas statementsabout model parameters.Hypothesistestsusethe information about a parameter that is containedin a sampleof data,its leastsquarespoint estimate,and its standarderror, to draw a conclusion about the hypothesis. In each and every hypothesis test five ingredients must be present: COMPONENTS
OF FTYPOTHESIS TESTS
17
The .l3t
-'on
rul I :1)'
li r
1. A null hypothesist1s 2. An alternative hypothesis 1{ 3. A test statistic 4. A rejection region 5. A conclusion
_i.2
Thr :as :eg
3.2 HYPOTHESIS TESTS
-i.2.1
bz-t t"se(bz) 10.44 r4.12 11.77 15.15 13.30 18.02 14.06 12.68 10.89 11.40
55
Trc Nurr HYporHrsrs
The null hypothesis,which is denotedby Hs(H-naugftr), specifiesa value for a regression :rrameter, which for generality we denoteas B1,for ft : I or 2. The null hypothesisis stated :r Hs: Br : c, where c is a constant,and is an important value in the context of a specific :sgressionmodel. A null hypothesisis the belief we will maintain until we are convincedby :re sample evidence that it is not true, in which casewe reject the null hypothesis. 3.2.2
TrrE ArrrnNATrvE FIyporHESrs
Pairedwith everynull hypothesisis a logical alternativehypothesisHlthat we will accept :: the null hypothesis is rejected. The alternative hypothesis is flexible and dependsto eme extent on economic theory. For the null hypothesis fls: 9t : c the three possible iltemative hypothesesare
: information Leto another. hey combine lues in which least squares rw, implying ; suffer from that the least
H y : $p ) c. Rejecting the null hypothesisthat Br : c leadsus to acceptthe conclusion that Bp > c. Inequality alternativehypothesesare widely usedin economicsbecause economic theory frequently provides information about the signs of relationships betweenvariables.For example,in the food expenditureexample we might well test 0 againstI1r : Fz ) 0 becauseeconomictheory strongly the null hypothesisHo,lz: suggeststhat necessitieslike food are normal goods, and that food expenditurewill rise if income increases. H1:$p (c. Rejectingthe null hypothesisthat 9r: c in this caseleadsus to acceptthe conclusionthat B7.( c. h:9r * c. Rejectingthe null hypothesisthat Br : c in this caseleadsus to acceptthe conclusion that B1 takes a value either larger or smaller than c.
:ther or not a good deal of rhat a $100 io, basedon md model is lation to the tical model, presentedas a parameter lard error, to
3.2.3
Trm Tesr Srerrsrrc
The sampleinformation about the null hypothesisis embodiedin the samplevalue of a test iratistic.Basedon the valueof a teststatisticwe decideeitherto rejectthe null hypothesisor ror to reject it. A test statistic has a special characteristic: its probability distribution is jompletely known when the null hypothesisis true, and it has someother distribution if the :rull hypothesisis not true. It all starts with the key result in (3.3), t : (b* - p2)lse(bp) - tg-z). If the null hvpothesisHo:9r: c is true, then we can substitutec for Bp and it follows that t:
bt-c ^
-trv-z)
(3.7)
Se(47..f
Ii the null hypothesisis not tue, then the t-statistic in (3.7) doesnot have a r-distribution rr ith N - 2 degreesof freedom.This point is elaboratedin Appendix 38. 3.2.4
Tur RrJrcrroN REGToN
The rejection region dependson the form of the alternative.It is the range of values of the reststatisticthatleadstorejectionof thenull hypothesis.Itispossibleto constructa rejection region only if we have
INTERVAL ESTIMATION
56
AND HYPOTHESIS TESTING
o a test statistic whose distribution is known when the null hypothesis is true .
an alternative hypothesis
.
a level of significance
The rejection region consists of values that are unlikely and have low probability of occurring when the null hypothesisis true. The chain of logic is "If a value of the test statistic is obtainedthat falls in a region of low probability, then it is unlikely that the test statistichas the assumeddistribution, and thus it is unlikely that the null hypothesis is true." If the alternativehypothesisis true, thenvaluesof the test statisticwill tendto be unusually large or unusually small. The terms "large" and "small" are determinedby choosing a probability cr, called the level of significance of the test, which provides a meaning for "an unlikely event." The level of significanceof the test a is usually chosento be 0.01, 0.05 or 0.10. If we reject the null hypothesiswhen it is true, then we commit what is called a lYpe I error. The level of significanceof a test is the probability of committing a Type I error, so P(Type I enor) : cr.Any time we reject a null hypothesisit is possible that we have made such an esel-1[e1g is no avoiding it. The good news is that we can specify the amount of Type I error we will tolerate by setting the level of significancecr. If such an error is costly, then we make cr small. If we do not reject a null hypothesis that is false, then we have committed a Tlpe II error. In a real-world situation we cannot control or calculate the probability of this type of error becauseit dependson the unknown true parameterFr. For more about Type I and Type II errors see Appendix C.6.9.
::i is .- |
a.u
lw
I re
lle t .
3.2.5
r
_ i
^
. . t: :.-ul
A CoNcrusroN
::c c When you have completedtestinga hypothesisyou shouldstateyour conclusion.Do you reject the null hypothesis, or do you not reject the null hypothesis? As we will argue below you should avoid saying that you "accept" the null hypothesis,which can be very misleading.Also, we urge you to make it standardpracticeto saywhat the conclusionmeans in the economic contextof the problem you areworking on and the economic significanceof the finding. Statistical proceduresare not ends in themselves.They are carried out for a reason and have meaning, which you should be able to explain.
-:!)\l
,:. th' -:.In '-:c n !-r id - tu:
If 1.\ \l
3.J Rejection Regions for Specific Alternatives _ r- J .
In this sectionwe hope to be very clear aboutthe nature of the rejection rules for eachof the three possible alternativesto the null hypothesisHs:Bp: c. As noted in the previous section, to have a rejection region for a null hypothesis we need a test statistic, which we have; it is given in (3.7). Second,we needa specificalternative,9r> c, 9r 1c, or $pl c. Third, we need to specify the level of significanceof the test. The level of significanceof a test, o., is the probability that we reject the null hypothesiswhen it is actually true, which is calledaTypelerror. 3.3.1
ONE-TnrrTssrs
wrrH ArrEnNATrvE "GR-EnrEnTnaN"
(>)
-: rh \ac ..-JLl
3l\'
\ .\tt
,, I
When testing the null hypothesisHs:$k : c, if the alternative hypothesisFI1:Bp ) c is true, then the value of the r-statistic (3.7) tends to become larger than usual for the t-distribution. We will reject the null hypothesisif the test statistic is larger than the critical value for the level of significancea. The critical value that leavesprobability a in the right
I
3.3 REJECTION REGIONS FOR SPECIFIC ALTERNATIVES
rs true
,robability of e test statistic ;t statistichas true." If the ually large or a probability "an unlikely 0.05or 0.10. lled a Tlpe I 'pe I error, so ,e have made 5e amount of nor is costly, hen we have calculate the neter B,r,.For
;ion. Do you e will argue can be very usionmeans gnificanceof .ed out for a
0 prcune
tr= t6 -o., u-2\
Rejection region for a one-tail test of Hs:9t :
c a g a i n s t F 1 r : B 1) c .
as shown in Figure 3.2. For example,if a : 0.05 tail is the (l- ct)-percentile/11-o.1y-21, and N - 2:30, then from Table 2 the critical value is the 95th percentile value : 1.691' /1o.ls.:o; The rejection rule is Wfrentestingthe null hypothesisf/e: B6 : c againstthe alternativehypothesisH1:Bp) c, I I reject the null hypothesisand acceptthe altemativehypothesisif t 2tg-o,1,1-21. The testis called a "one-tail" test becauseunlikely valuesofthe /-statisticfall only in one tail of the probability distribution. If the null hypothesisis true, then the test statistic(3.7) has a r-distribution, and its value would tend to fall in the center of the distribution, to the left of the critical value,wheremost of the probability is contained.The level of significancea is chosenso that ifthe null hypothesisis true, then the probability that the /-statistic value falls in the extreme right tail of the distribution is small; an event that is unlikely to occur by chance.Ifwe obtain a teststatisticvalue in the rejection region, we take it asevi denceagainst the null hypothesis,leading us to conclude that the null hypothesisis unlikely to be true. Evidence against the null hypothesis is evidence in support of the alternative hypothesis. Thus if we reject the null hypothesis then we conclude that the alternative is true. If the null hypothesis.Fl6| 9r : c is true, thenthe test statistic(3.7) hasa r-distributionand its valuesfall in the nonrejectionregion with probability I - a. If t l tg a.u-z), then thereis no statisticallysignificant evidenceagainstthe null hypothesis,and we do not reject it. 3.3.2
r eachofthe he previous ;, which we :,or Bplc. ficance of a ue,which is
3 .2
ONE-TelTssrs
wrrH ALTERNATTvT"LEss THaN" (()
If the alternativehypothesisF11: B1 ( c is true,thenthe valueof the /-statistic(3.7) tendsto become smaller than usual for the t-distribution. We reject the null hypothesis if the test statisticis smallerthan the critical value forthe level of significancea. The critical value that leavesprobability cr in the left tail is the a-percentile to.,l-z\, as shown in Figure 3.3. When using Table 2 to locate critical values,recall that the r-distribution is symmetric is the negative of the (l-a)-percentile about zero, so that the cr-percentile/1o.,,,i-21 if ct : 0.05andN - 2 :20,then from Table2the 95thpercentile I, r-o,,v-2).Forexample, . 7) 2 5 . zo o f t h e l - d i s t r i b u t i o n i s t p g s , z o ) : I . 7 2 5a n d t h e 5 t h p e r c e n t i l e v a l u e i s / ( o . o s: -, 1 The rejection rule is:
l:Bp>cis ual for the the critical in the right
: Be : c againstthe alternativehypothesisH1 :Bp 1c, ' I Wtrentestingthe null hypothesis^FIe the alternativehypothesistf t 1 t6.*-r1. null hypothesis and accept reject the I I
58
INTERVAL ESTIMATION
AND HYPOTHESIS TESTING
laik .rbt rrol ',ritl ran
Reject //6: B1 = c
-c\t
tr= t(a,t't_z) FIGURE3,3
:he
0
(c. Therejection regionfor a one-tailtestof H6:9r,: c against 11r:B1
The nonrejectionregion consistsof /-statisticvaluesgreaterthan /(a,N-2;.When the null hypothesisis true,the probability of obtainingsucha /-valueis I - a, which is chosento be large.Thus if t >
[email protected])then do not rejectHs:$k : g. Rememberingwheretherejectionregionis locatedmay be facilitatedby thefollowing trick: MEMORY TRICK: The rejection region for a one-tail test is in the direction of the arrow in the altemative.If the alternativeis ">", then reject in the right tail. If the altemativeis "(", rejectin the left tail.
i \\'e
J, \\'e l'\O
'is 3.3.3
Two-Terr
'(Nor Equer To" Trsrs wrrH ALTERNATTvE
(l)
leg
.ltU
When testing the null hypothesisIIs :9r : c, if the alternativehypothesisF1l: Bpf c istrue, then the value of the /-statistic(3.7) tendsto becomeeitherlargeror smallerthan usualfor the r-distribution. To have a test with level of significanceo we define the critical values so that the probability ofthe t-statistic falling in either tail is u 12. The lefrtail critical value is the percentile t61z,7i-21and the right-tail critical value is the percentile tg-o1z.N-21. We rejectthe null hypothesisthatI/s : 9r : c in favor of the alternativethatHy :81,1 c if the test statistict 1 t61z,w-z)or t ) tg-o1z.N-2),as shownin Figure 3.4. For example,if a : 0.05 andN - 2 : 3},then af 2 : 0.025 andthe lefrtail critical valueis the 2.5-percentilevalue : -2.042; the right-tail critical valueis the 97.5-percentile/1o.rzs.:o; :2.042.The /1o.ozs,:o; right-tail critical value is found in Table 2, and the left-tail critical value is found using the
3.{
svmmetrv of the t-distribution.
-r.-l Us .pe
Reject Hs: B1= c Accept 111: p1 * c
Do not rejecl Hs:Pp=s
Reject 116:B1= c
'\t
Accept H1 : B1+ r
J)
.qr rtbt
,iet iol -
_ l
J5t
sig rc = Ira:. r-z)
prcunx
3 .4
'( -'l]Jl.N
2)
Rejectionregion for a test of .F16: B1-: c a g a i n sHt 1 : B p l c .
rci .al
3.4 EXAMPLES OF HYPOTHESIS TESTS
59
Sincethe rejection region is composedof portions of the t-distribution in the left and right tails, this test is called a two-tail test. When the null hypothesisis true, the probability of obtaininga value of the test statisticthat fallsin eithertail areais "small." The sumof the tail probabilitiesis cr.Samplevaluesof the test statisticthat are in the tail areasareincompatible *'ith the null hypothesisand areevidenceagainstthe null hypothesisbeing true. On the other hand,if the null hypothesis,Fls: 9r : c is true, then the probability of obtaining a value of the test statistic r in the central nonrejection region is high. Samplevaluesof the test statistic in the central nonrejection area are compatible with the null hypothesis and are not taken as evidenceagainst the null hypothesis being true. Thus the rejection rule is 3p{c.
When the null
When testingthe null hypothesisHs: B1 : c againstthe alternativehypothesisH1:$p t' c, reject the null hypothesisand accept the altemative hypothesisif t I tp1z,u-zy or if t ) t6_o1z,u_z).
is chosento be followingtrick: .rectionof the ;ht tail. If the
:) : B7.f c is true, r thanusualfor itical valuesso :ritical valueis
\\-e do not reject the null hypothesisif t61z,N-21' 1t ( tg_t1z,w_z).
3.4 Exarnples of Hypothesis Tests \\'e illustrate the mechanicsof hypothesistesting using the food expendituremodel. We give eramplesof right-tail,left-tail, andtwo-tailtests.Ineachcasewe will follow aprescribedset nisteps,closely following the list of requiredcomponentsfor all hypothesistestslisted at the beginning of Section 3.2. A standardprocedure for all hypothesis testing problems and .ituationsis STEP-BY-STEP PROCEDURE
FOR TESTING HYPOTHESES
Determine the null and alternative hypotheses. Specify the test statistic and its distribution if the null hypothesis is true.
r-d/z,N-z).We i1 I c if the test rle,if cr : 0.05
4. Calculate the sample value of the test statistic.
,ercentilevalue
5 . State vour conclusion.
l:2.042.The ound using the
k=c
t
Select a and determine the rejection region.
J.4.1 Rrcnr-Ten Tpsrs i.{.1a One-Tail Test of Signficance L-suallyour first concernis whether there is a relationshipbetweenthe variables,as we have .precified in our model. If 92 :0 then there is no linear relationship between food :rpenditure and income. Economic theory suggeststhat food is a normal good, and that rr income increasesfood expenditure will also increase,and thus that B2 > 0. The least yluaresestimateof B2is bz : 10.21,which is certainly greaterthan zero. However, simply ..bservingthat the estimatehasthe right sign doesnot constitutescientific proof. We want to ietermine if there is convincing, or significant, statistical evidence that would lead us to :onclude that B2 ) 0. When testing the null hypothesis that a parameter is zero, we are rsking ifthe estimate b2 is significantly different from zero, and the test is called a test of significance. A statistical test procedurecannot prove the truth of a null hypothesis.When we fail to :eject a null hypothesis, all the hypothesis test can establish is that the information in a -rmple of data is compatible with the null hypothesis.Conversely,a statistical test can lead
60
INTERVAL ESTIMATION
AND HYPOTHESIS TESTING
us to reject the null hypothesis,with only a small probability cr of rejecting the null hypothesiswhen it is actuallytrue.Thus rejectinga null hypothesisis a strongerconclusion thanfailing to rejectit. For this reasonthe null hypothesisis usuallystatedin sucha way that if our theory is correct,then we will reject the null hypothesis.In our example,economic theory implies that there should be a positive relationshipbetween income and food expenditure. We would like to establish that there is statistical evidence to support this theoryusing a hypothesistest.With this goal we set up the null hypothesisthat thereis no relation between the variables, Ho:92:0. In the alternative hypothesis we put the conjecturethat we would like to establish,Hr:92 > 0. If we then rejectthe null hypothesis we can make a direct statement, concluding that B2 is positive, with only a small (ct) probability that we are in error. The stepsof this hypothesistest are as follows: The null hypothesisis F1s:B2 : 0. The alternativehypothesisis I11:B2 > 0. 2. The test statisticis (3.7). In this casec : 0, so t : bzlse(bz) - t(u-z) if the null hypothesisis true. 1
3. Let us selecta :0.05. The critical value for the right-tail rejection region is the 95th percentile of the r-distribution with N - 2 :38 degrees of freedom, /ro.ss.:s):1.686.Thus we will reject the null hypothesisif the calculatedvalue of t > 1.686.If t < 1.686,we will not reject the null hypothesis. 4. Using the food expendituredata, we found that b2: se(b2):2.99. The value of the test statisticis
r:
b't
10.21 with standarderror
10.21
: -- 4'uu ,.(br) 2.og
5. Sincel:4.88 > 1.686,we rejectthe null hypothesisthat B2 : 0 and acceptthe alternativethat B2 > 0. That is, we reject the hypothesisthat there is no relationship between income and food expenditure, and conclude that there is a statistically significant positiverelationshipbetweenhouseholdincome and food expenditure. The last part of the conclusionis important. When you report your resultsto an audience,you will want to describe the outcome of the test in the context of the problem you are investigating,not just in terms of Greek letters and symbols. What if we had not beenable to reject the null hypothesisin this example?Would we have concludedthat economictheory is wrong and that thereis no relationshipbetweenincome and food expenditure?No. Rememberthat failing to reject a null hypothesisdoesnot mean that the null hypothesisis true. 3.4.1b One-Tail Test of an Economic Hypothesis Suppose that the economic profitability of a new supermarket depends on households spendingat least $5.50 out of each additional $100 weekly income on food and that constructionwill not proceedunlessthereis strongevidenceto this effect. In this casethe conjecturewe want to establish,the one that will go in the alternativehypothesis,is that 9z > 5.S.If Fr < 5.5, then the supermarketwill be unprofitableand the ownerswould not wanttobuildit. The leastsquares estimateofS2tsb2=10.21,whichis greaterthan 5.5.What we want to determineis if there is convincing statisticalevidencethat would lead us to conclude,basedon the availabledata,that 9z > 5.S.This judgment is basednot only on the estimateb2,blutalso on its precisionas measuredby se(b:).
3.4 EXAMPLES OF HYPOTHESIS TESTS
)cting the null ger conclusion iuch a way that rple,economic )me and food .o support this hatthere is no Ls we put the iull hypothesis y a small (cr)
61
What will the null hypothesisbe? We have been statingnull hypothesesas equalities, '-:.-has Fz : 5.5. This null hypothesisis too limited becauseit is theoreticallypossiblethat r: < 5.5. It tums out that the hypothesistestingprocedurefor testingthe null hypothesis ::et Hs:B2 < 5.5 againstthe alternativehypothesisHt:92>5.5 is exactly the same as ':.ting Ilo : 9z : 5.5 againstthe alternativehypothesisHt:92 > 5.5. The test statisticand :-:rcctionregionareexactlythe same.For a right-tail testyou canform the null hypothesisin :.:her of theseways dependingupon the problem at hand. The steps of this hypothesis test are as follows:
l . The null hypothesisis I1s: B2 < 5.5. The alternativehypothesisis 111: B2> 5.5. t . The test statistict : (bz - 5.5) lse(b2)- \u_zt if the null hypothesisis true. Let us select cr: .01. The critical value for the righrtail rejection region is the 99th percentile of the l-distribution with N - 2 :38 degrees of freedom, :2.429. We will reject the null hypothesisif the calculated value of /1o.er,:s;
: P z> 0 ' -21 if the null
t> 2.429.If t < 2.429 we will not reject the null hypothesis. r region is the of freedom, lculated value
Using the food expendituredata, bz : 10.21 with standardenor se(b2):2.09. The valueof the test statisticis ,a^-\\ "Z
'-^-
,
;tandard error i.
nd accept the o relationship a statistically I expenditure. audience,you rlem you are /ould we have ween income oes not mean
n households ood and that Lthis casethe thesis,is that :rs would not ran5.5.What :ld lead us to rt only on the
-
se\b2)
10.2r- 5.5- ' ,
1<
2.09
Since r : 2.25 <2.429 we do not reject the null hypothesisrhat B2 ( 5.5. We are not ableto conclude that the new supernarket will be profitable and will not besin construction.
-- this examplewe have poseda situationwhere the choice of the level of significancecr -r.omes of great importance.A construction project worth millions of dollars dependson --ting convincing evidencethat householdswill spendat least $5.50 out of eachadditional ! - tJOincomeon food. While the "usual" choiceis cr : 0.05,we havechosena conservative . ofct : 0.01becausewe seeka testthathasa low chanceofrejectingthenull hypothesis ",ue o hen it is actually true. Recall that the level of significanceof a testdefineswhat we meanby :-: unlikely value of the test statistic. In this example, if the null hypothesisis true, then r.:ilding the supermarket will be unprofitable. We want the probability of building an -::profitablemarket to be very small, and thereforewe want the probability of rejecting the - :ll hypothesiswhen it is true to be very small. In eachreal-world situation,the choice of cr --ustbe madeon an assessment of nsftand the consequences of making an incorrectdecision. A CEO unwilling to make a decisionbasedon the aboveevidencemay well ordera new i:d largersampleof datato be analyzed.Recallthat as the samplesizeincreases,the least \:uares estimator becomesmore precise (as measuredby estimator variance) and conse:.:ently hypothesistestsbecomemore powerful tools for statisticalinference. \.1.2
Lsnr-TArL TEsrs
l ,r completenesswe will illustrate a test with the rejection region in the left tail. Consider ::e null hypothesisthat 9z ) 15 and the alternativehypothesis9z < 15.Recall our memory '::ck for determiningthelocationofthe rejectionregionfor a /-test.The rejectionregionis in ':e direction of the arrow " ( " in the alternativehypothesis.That tells us that the rejection ::gion is in the left tail of the r-distribution.The stepsof this hypothesistest are as follows: 1. The null hypothesisis F1e:B2 > 15. The alternativehypothesisis 11r: B2 < 15.
INTERVAL ESTIMATION
AND HYPOIHESIS
TESTING
2. The test statistict : (bz - 15)lse(b2)-/1,r, r) if the null hypothesisis true. 3. Let us selecta : 0.05. The critical value for the left-tail rejectionregion is the 5th percentile of the t-distribution with N - 2 :38 degreesof freedom, l1oos,:s): -1.686. We witl reject the null hypothesisif the calculatedvalue of t < -1.686. If t> - 1.686we will not reject the null hypothesis.A left-tail rejectionregion is illustratedin Figure 3.3. 4. Using the food expendituredata,bz : 10.21with standarderror se(b2): 2.09.The value of the test statisticis I:
b z- l 5 t4b):
1 0 . 2 1- 1 5 :-:'tv Log
5. Sincet : -2.29 < - l.686werejectthenullhypothesisthat B2 > 15andacceptthe alternativethat 9z < 15.We concludethathouseholdsspendlessthan$ l5 from each additional$100 income on food. 3.4.3
Tvo-Ten
Tpsrs
3.4.3a Two-Tail Test of an Econornic Hypothesis A consultantvoicesthe opinion that basedon other similar neighborhoodsthe households near the proposedmarket will spendan additional $7.50 per additional $100 income. In termsof our economicmodel we can statethis conjectureasthe null hypothesis9z :7 .5.|f we wantto testwhetherthis is trueor not,thenthealternativeis that B2 l1 .5.This alternative makesno claim aboutwhetherB2is greaterthan7.5 or lessthan7.5, simply that it is not 7.5. In such caseswe use a two-tail test, as follows: The null hypothesisis Hs : B2 : 7.5. The alternativehypothesisis H1 :82 17.5. 2. The test statistict : (bz - 7 .5) lse(b2)- t6-z) if the null hypothesisis true. 3. Let us selectcr : 0.05.The critical valuesfor this two-tail testarethe 2.5-percentile :2'024. Thus we will reject : -2.024 and the 97.5-percentile /1o.ers,:s) /ie.ozs,:a) the null hypothesis if the calculated value of t>2.024 or if r < -2.024. lf -2.024
value of the test statisticis h-_-7\ -, : " t
""
se(bz)
lnrl_7.5 2.09
5. Since-2.204 < t : 1.29< 2.204we do not rejectthe null hypothesisthat 9z : I .5. The sample data are consistentwith the conjecture householdswill spend an additional$7.50 per additional$100 income on food. We must avoid reading into this conclusion more than it means.We do not conclude from this test that B2 : 7.5, only that the data arenot incompatiblewith this parametervalue. The data are also compatiblewith the null hypotheses116:B2 : 8.5 (/ : 0.82), Hs:$2 testcannot be usedto prove 6.5 (t : 1.77),andHo:92 : 12.5(r : - 1.09).A hypothesis that a null hypothesisis true. There is a trick relatingtwo-tail testsand confidenceintervalsthat is sometimesuseful. Let c be a valuewithin a 100(1 - a)% confidenceinterval,so that if t, : t,r-o,r.7,r-2;,then bp - t,se(bp)I c I br + t.se(b1)
3.4 EXAMPLES OF HYPOTHESIS TESTS
;is is true. 'egion is the 5th : lom, /1o.os,:a) ofr<-1.686. ection region is bz) :2.09.The
5 and acceptthe n $15 from each
63
li we testthe null hypothesisHs:Bk : c againstI11:9r I c, when c is insidethe confidence :nterval,then we wlll not reject the null hypothesisat the level of significancect. If c is outsidethe confidenceinterval, then the two-tail test will reject the null hypothesis.We do rot advocateusing confidenceintervals to test hypotheses,they serve a different purpose, rut if you are given a confidence interval this trick is handy. i.{.3b Two-Tail Test of Significance \\-hile we are confident that a relationship exists between food expenditure and income, nodels areoften proposedthat aremore speculative,and the purposeofhypothesis testing is :o ascertainwhether a relationship between variables exists or not. In this case the null :li pothesisis B2: 0; that is, no linear relationship existsbetweenx and y. The alternativeis j: + 0, which would meanthat a relationshipexists,but that there may be either a positive or :r'sative association between the variables. This is the most common form of a test of rignificance. The test steps are as follows: l . The null hypothesisis I/s : B2 : 0. The alternativehypothesisis $:8210. t . The test statistict : bzI se(bz)- t(N-z\ if the null hypothesisis true.
the households i100income.In rsisB2 : 7.5.11 This alternative that it is not 7.5.
fu:5217.5.
Let us selecta : 0.05.The critical valuesfor this two-tail testarethe 2.5-percentile :2.024. We will reject the : -2.024 and the 97.5-percentile/1erzs,:s) /1o.ozs,:a) null hypothesis if the calculated value of t>2.024 or if t < -2.024. If -2.024 < t < 2.024 we will not reject the null hypothesis.
+.
Using the food expendituredata, bz : 10.21 with standarderror se(b2):2.09. The value of the test statisticts t : bzlse(bz) : 10.2112.09: 4.88. Sincel:4.88 >2.024 we reject the null hypothesisthat F2 = 0 and concludethat thereis a statistically significant relationshipbetweenincome and food expenditure.
,sisis true. e 2.5-percentile s we will reject t < -2.024. If 5z) : 2.09. The
s that 92 : 7.5. will spend an
I conclude from arametervalue. J . 8 2 ) ,H s : 8 2 : re used to prove metimesuseful. I a/z,N-21,then
:'.\o points should be made about this result. First, the value of the t-statistic we - .mputed in this two-tail test is the same as the value computed in the one-tail test of .: :nificancein Section3.4.1a.The differencebetweenthe two testsis the rejectionregion ::J the critical values. Second, the two-tail test of significance is something that .:.ould be done each time a regressionmodel is estimated,and consequentlycomputer .. it*are automatically calculates the /-values for null hypothesesthat the regression ::rameters are zero. Refer back to Figure 2.9. Consider the portion that reports the -'.:imates:
',rable
"( oME
Coefficient
83.41600 1.0.20964
Std.Error 43.41016 2.093264
f-Statistic
Prob
t.921578 4.877381
0.0622 0.0000
'.'re that there is a column labeled r-Statistic.This is the r-statisticvalue for the null : ^rothesisthat the correspondingparameteris zero. It is calculatedas t :brlse(br). -''',iding the least squaresestimates(Coefficient) by their standarderrors (Std. Error) ::: r'sthe t-statisticvalues(r-Statistic)for testingthe hypothesisthat the parameteris zero. l:r'r-statistic value forthe variableINCOME is 1.817381,which is relevantfortesting the :0. We have roundedthis valueto 4.88 in our discussions. :,-i hypothesis,Fl6:B2
INTERVAL ESTIMATION
AND HYPOTHESIS TESTING
: The r-valuefor testingthe hypothesisthat the interceptis zeroequalsl-92.The ct 0'05 -2.024 : and /1o.ezs.:t) 2'024 whether critical valuesfor thesetwo-tail testsare/1o.ozs.:s ): we are testing a hypothesis about the slope or intercept, so we fail to reject the null hypothesisthatH():9r : 0 giventhe alternativeH1:$110' The final column, labeled "Prob." is the subjectof the next section'
REMARK: "statistically significant" does not necessarilyimply "economically significant." For example,supposethe CEO of a supermarketchain plansa certaincourseof actionif B2l 0. Furthermore,supposea large sampleis collectedfrom which we obtain the estimatebz:0.0001 with se(b2):0.00001, yielding the t-statistict : 10'0' We would reject the null hypothesisthat B2 : 0 and acceptthe altemativethat B2 f0'Herc bz : 0.0001is statisticallydifferent from zero.However,0.0001may not be "economically" different from 0, and the CEO may decide not to proceedwith the plans. The messagehereis that one mustthink carefully aboutthe importanceof a statisticalanalysis beforereportingor using the results.
3 .5 The p-Value When reporting the outcome of statisticalhypothesistests,it hasbecome standardpractice to report the p-value (an abbreviation for probability value) of the test. If we have the p-value of a test,p, we candetermine the outcome of the test by comparing the p-value to ihe chosenlevel of significance,a, without looking up or calculatingthe critical values. The rule is p-VALUE RULE: Rejectthe null hypothesiswhen thep-value is lessthan, or equal , i (f a t h e n r e j e c t . F l 6 I. f p > c I t h e n d o n o t t o , t h e l e v e l o fs i g n i f i c a n c e c r . T h a t i s p reject Hs. If you havechosenthe level of significanceto be e : 0.01, 0.05, 0.10, or any other value, you can compareit to thep-value of a test and then reject, or not reject, without checking the critical value.In written works reporting thep-value of a test allows the readerto apply his or her own judgment about the appropriate level of significance. How thep-value is computeddependson the alternative.If r is the calculatedvalue of the /-statistic, then o a a
if ft:Bp) c, p: ifIl1:Bp 1c, p:
probabilityto the right of / probabilityto the left of/
-ltl if I/r : Br l r, p : sum of probabilitiesto the right of ltl and to the left of
TRICK: The direction of the alternativeindicatesthe tail(s) of the disI MEMORY thep-valuefalls. in which tribution I
'.:1
.
S
-:a:t
::'l
:
_t.
3.5 THE p-VALUE
) 2 . T h ec t : 0 . 0 5 = 2.024 whether r reject the null
3.5.1 p-Verun FoR A Rrcnr-Ten
Trsr
In Section 3.4.1b we tested the null hypothesis I1s:B2 < 5.5 against the one-sided alternativeHr:92 >5.5. The calculatedvalue of the r-statisticwas
bo- 5.5 .-^-
t0.2r- 5.5- ' ) ) \
se\D2)
rnomically sig:rtain courseof 'hich we obtain t : 10.0. We at 8210. Here be "economicthe plans. The listical analysis
65
2.09
In this case, since the alternative is "greater than" (>), the p-value of this test is the probability that a r-random variable with N - 2 :38 degreesof freedom is greater than
1.25, or U: Pltps)>2.251:0.0152. This probability value cannot be found in the usual r-table of critical values,but it is easily found using the computer. Statistical software packages,and spreadsheetssuch e. Excel, have simple commands to evaluate the cumulative distribution function cdl) (see Appendix B.2) for a variety of probability distributions. If Fa(r) is the cdf ior a random variable X, then for any value x: c the cumulative probability is PIX < cl : FxG).Given such a function for the r-distribution, we compute the desired r-\.alue
u - P l t p z ) > 2 . 2 5 1 t: - P h : s l < 2 . 2 5 1 :I - 0 . 9 8 4 8 : 0 . 0 1 5 2 tandardpractice If we have the g the p-value to , critical values.
than, or equal octhen do not
Following the p-value rule we conclude that at cl : 0.01 we do not reject the null hypo:hesis.If we had chosenct :0.05 we would reject the null hypothesisin favor of the rlternative. The logic of thep-value rule is showninFigure 3.5. The probability of obtaining a r-value :teater than 2.25 is 0.0152,O: Plfuat>2.25J :0.0152. The 99th percentile/1o.or,:a;, * hich is the critical value for a right-tail test with level of significancaof cr : 0.01, must fall :o rheright of 2.25.This meansthat / : 2.25 doesnot fall in therejectionregionif ct : 0.01 :-'rdwe will not reject the null hypothesisat this level of significance.This is consistentwith :nep-value rule:Whenthep-value (0.0152)is greaterthan the chosenlevel of significance ,t.01)we do not reject the null hypothesis.
any other value, rut checking the rr to apply his or rted value of the
--nnr
lefrof -lll
'/
2
l',t
= 2'429 I , o q s . r s=r 1 ' 6 8 6 tlo.ss,tty :.-{; L'RE 3 . 5
The p-value for a right-tail test.
66
INTERVAL ESTIMATION
AND HYPOI'HESIS TESTING
-rl-,
0
I
2
3
t
\-1
FTGURE 3 ,6
-2.429 t10.0s,3s; = -l'686 tro.or.tu= Thep-valuefor a left-tailtest.
On the otherhand,the 95th percentile/(o.qs,:s),which is the critical value for a right-tail testwith cr.: 0.05, mustbe to the left of 2.25.This meansthat t : 2.25 falls in the rejection region and we reject the null hypothesisat the level of significancecr :0.05. This is consistentwith the p-value rule'.When thep-value (0.0152)is less than or equal to the chosenlevel of significance(0.05) we will reject the null hypothesis. 3.5.2
,l
.-U
p-Varua FoR A LErr-Tan TEsr '.f,
In Section 3.4.2 we carried out a test with the rejection region in the left tail of the ) 15 and the alternativehypothesiswas t-distribution.The null hypothesiswas -F1s:B2 Ht:92 < 15.The calculatedvalue of the /-statisticwas / : -2.29. To computethep-value for this left-tail test we calculatethe probability of obtaining a t-statistic to th eleft of -2.29. Using your computersoftwareyou will find this value to Ue P [(:s; < -2.29]: 0.0139. Following thep-valuerule we concludethat at cr : 0.01we do not rejectthe null hypothesis. If we choose cr : 0.05 we will reject the null hypothesis in favor of the alternative. See Figure3.6 to seethis graphically.Locatethe I st and5th percentiles.Thesewill be thecritical valuesfor left-tail testswith a : 0.01anda : 0.05 levelsof significance.When thep-value (0.0139)is greaterthanthe level of significance(a : 0.01) thenthe /-value-2.29 is notin the test rejection region. When the p-value (0.0139)is less than or equal to the level of significance(a : 0.05) then the t-value -2.29 is in the test rejection region. 3.5.3
p-Verur
FoR A Two-Tnrr
, rtl
:.:!I
Tnsr
For a two-tail test,the rejectionregionis in the two tails of the r-distribution,andthep-value is similarly calculatedin the two tails of the distribution.In Section3.4.3awetestedthe null hypothesisthatgz :7.5 againstthealternativehypothesis9z l7 .5.The calculatedvalueof the /-statisticwas / : | .29. For this two-tail testthep-value is the combinedprobability to the right of 1.29 and to the left of -1.29:
P[(:a;< -r.29]:0.2033 , : Pltpt)>r.291+
"Cr
) : \'i, -_-i(
This calculationis depictedin Figure3.7. Oncethep-valueis obtainedits useis unchanged. If wechoosect:0.05.o :0.10,orevenct:0.20wewillfailtorejectthenullhypothesis becausep > cr.
' .itl
:n
3.5 THE p-VALUE
-t/-,
-1
0
1
,\
3
67
t
= -2'024 Iro.o:'s.:s,
FIGuRE 3 .7
2.024 \ogts.t)t= The p-valuefor a two-tail testof significance.
or a right-tail r the rejection 0.05. This is : equal to the
At the beginning of this section we statedthe following rule for computingp-values for two-tailtests:ifl11:9rlc, p:sumofprobabilitiestotherightof ltlandtotheleftof-lrl. The reasonfor the use of absolutevalues in this rule is that it will apply equally well if the value of the /-statistic turns out to be nositive or nesative.
3.5.4 ft tail of the pothesiswas le thep-value leftof -2.29.
9] : o'01:9. ll hypothesis. ernative. See be the critical :n thep-value 2.29 isnotin r the level of
Ldthep-value estedthe null latedvalue of rrobability to
s unchanged. ll hypothesis
p-Venn
FoR A Two-Tnn
Trsr or SrcNrrrcRNcE
All statistical software computes the p-value for the two-tail test of significance for each ;oefficient when a regressionanalysisis performed. In Section 3.4.3b we discussedtesting the null hypothesisHo:92:0 against the alternative hypothesisHt:9zl 0. For the --alculatedvalue of the /-statistic / : 4.88 the p-value is
+Pl(:sl < -4.88]: 0.0000 o : Pltpt)>4.88] \bur software will automatically compute and report this p-value for a two-tail test of ,.ignificance. Refer back to Figure 2.9 ar.d consider just the portion reporting the estimates:
\ ariable
Coefficient
Std. Error
f-Statistic
Prob.
{ :\.COME
83.41600 10.20964
43.41016 2.093264
1.92rs78 4.8'77381
0.0622 0.0000
Next to each /-statistic value is the two-tail p-value, which is labeled "Prob" by the EViews software. Other software packageswill use similar names.When inspecting com-3uteroutput we canimmediately decideif an estimateis statistically significant (statistically Jifferent from zero using a two-tail test) by comparing the p-value to whatever level of lignificance we care to use. The estimated intercept has p-value 0.0622 so it is not 'tatistically different from zero at the level of si,snificancea : 0.05, but it is statistically .rgnificantifa:0.10.
68
INTERVAL ESTIMATION
AND HYPOTI]ESIS TESTING
The estimatedcoefficient for income has a p-value that is zero to four places.Thus p I a :0.01 or even ct : 0.0001,and thus we rejectthe null hypothesisthat income has no effecton food expenditureat theselevelsofsignificance.Thep-valueforthis two-tail test of significance is not actually zero. lf more places are used then p:0.00001946. Regressionsoftwareusually doesnot print out more than four placesbecausein practice levels of significanceless than cr :0.001 are rare.
3,6 Exercises Answers to exercisesmarked * appear in Appendix D at the end of the book.
3.6.1
Pnonrmrs
3.1 Using the regressionoutput for the food expendituremodel shown in Figure 2.9: (a) Constructa957o interval estimatefor B1 and interpret. (b) Testthe null hypothesisthat Br is zero,againstthe alternativethat it is not, at the 5Vo level of significancewithout using the reportedp-value. What is your conclusion? (c) Draw a sketch showing the p-value 0.0622 shown in Figure 2.9, the critical value from the r-distribution used in (b) and how the p-value could have been usedto answer(b). (d) Testthe null hypothesisthat B I is zero,againstthe alternativethatit is positive.at the 5Volevel of significance.Draw a sketchof the rejection region and compute the p-value. What is your conclusion? (e) Explain the differencesand similaritiesbetweenthe "level of significance"and the "level of confidence." (f) The resultsin (d) showthatwe are957oconfidentthat B 1is positive.Trueor false? If false,explain. 3.2 The generalmanagerof an engineeringfirm wants to know if a technical artist's experience influences the quality of their work. A random sample of 24 artists is selectedand their years of work experienceand quality rating (as assessedby' their supervisors)recorded.Work experience(EXPER) is measuredin years and quality rating (RAZNG) takes a value of 1 through 7, with 7 : excellentandI poor. The simple regressionmodel RAZNG: 9r f \2EXPER f e is proposed. The least squaresestimatesof the model, and the standarderrors of the estimates. are RATING : 3.204 + 0.0'76EXPER ('") (0.70e) (0.044) (a) Sketch the estimated regressionfunction. Interpret the coefficient of EXPER. (b) Constructa95VoconfidenceintervalforB2,theslopeoftherelationshipbetween quality rating and experience.In what are you 95Vo confident'! (c) Testthe null hypothesisthat B2is zeroagainstthe alternativethat it is not usinga two-tail test and the e : 0.05 level of significance.What do you conclude? (d) Testthe null hypothesisthat B2 is zero againstthe one-tail alternativethat it is positive at the ct : 0.05 level of significance.What do you conclude?
3.6 EXERCISES
69
(e) For the test in part (c), the p-value is 0.0982. If we choosethe probability of a Type I error to be q : 0.05, do we reject the null hypothesisor not, just basedon an inspection of the p-value? Show,in a diagram, how this p-value is computed.
places.Thus rt income has s two-tail test
0.00001946. se in practice
In an estimated simple regressionmodel, based on 24 observations,the estimated slope parameteris 0.310 and the estimated standarderror is 0.082. (a) Testthe hypothesisthat the slopeis zero,againstthe alternativethat it is not, at the lVo level of significance. (b) Testthe hypothesisthat the slope is zero, againstthe alternativethat it is positive at the l%olevel of significance. (c) Test the hypothesisthat the slope is zero againstthe alternativethat it is negative atthe 5Volevel of significance. Draw a sketch showing the rejection region. (d) Testthe hypothesisthat the estimatedslope is 0.5, againstthe alternativethat it is not, at the 570 level of significance. (e) Obtain a 99Vaintewal estimate of the slope.
k.
r Figure 2.9: :-l
t is not, at the Mhat is your ), the critical Lldhave been
Consider a simple regressionin which the dependentvariable MIM : mean income of males who are 18 years of age or older, in thousandsof dollars. The explanatory variable PMHS - percent of males 18 or older who are high school graduates.The data consist of 51 observationson the 50 statesplus the District of Columbia. Thus MIM and,PMHSare "state averages."The estimatedregression,along with standard errors and t-statistics,is
(a) +O.[80PMHS MIM: ('") (2.174) (b)
is positive,at and compute
(r)
(r.2s7)(s.7s4)
ificance" and (a) What is the estimated equation intercept? Show your calculation. Sketch the estimated regressionfunction. (b) What is the standarderror of the estimated slope? Show your calculation. (c) What is the p-value for the two-tail test of the hypothesis that the equation intercept is zero? Draw a sketch to illustrate. (d) State the economic interpretation of the estimated slope. Is the sign of the coefficient what you would expect from economic theory? (e) Construct a 99Voconfidence interval estimate of the slope of this relationship. (0 Test the hypothesisthat the slopeofthe relationship is 0.2, againstthe alternative that it is not. State in words the meaning of the null hypothesisin the context of this problem.
Trueor false? mical artist's of 24 artists ; assessedby in years and :llentand 1 : is proposed. he estimates,
3.6.2
. of EXPER. ship between is not using a conclude? Ltivethat it is ude?
ConpurrR
ExrRcrsEs
.:.5 A life insurancecompany wishes to examine the relationshipbetweenthe amount of life insurance held by a family and family income. From a random sample of 20 households,the company collected the data in the file insundat. The data are in units of thousandsof dollars. (a) Estimate the linear regressionwith dependentvariable INSURANCE and independentvariableINCOME. Write down the fitted model and draw a sketchof the fitted function. Identify the estimated slope and intercept on the sketch.Locate the point of the means on the plot.
70
INTERVAL ESTIMATION
AND HYPOTHESIS TESTING
(b) Discuss the relationship you estimated in (a). In particular (i) What is your estimate of the resulting change in the amount of life insurancewhen income increasesby $1000? (ii) What is the standarderror of the estimate in (i) and how do you use this standarderror for interval estimation and hypothesis testing? (c) One member of the managementboard claims that, for every $1000 increasein income, the amount of life insurance held will go up by $5000. Choose an alternativehypothesisand explain your choice.Does your estimatedrelationship support this claim? Use a 5Vosignificance level. (d) Test the hypothesis that as income increases the amount of life insurance increasesby the same amount. That is, test the hypothesis that the slope of the relationship is 1. (e) Write a short report (200-250 words) summarizing your findings about the relationship between income and the amount of life insuranceheld. 3.6* Consider the learning curve example introduced in Exercise 2.9. (a) Construct a 95Vointerval estimate for B2 and interpret. (b) Test at the 5Vo level of significance whether there is no learning against the alternative that there is learning. Formulate the null and alternativehypotheses and discussyour reasoning.Explain your conclusion. 3.7 Considerthe capital assetpricing model (CAPM) in Exercise2.10. (a) Testatthe5%levelof significancethehypothesisthateachstock's "beta" value is I against the alternative that it is not equal to 1. What is the economic interpretation of a beta equal to 1? (b) Test at the 5Vo level of significance the null hypothesis that Mobil-Exxon's " beta" value is greaterthan or equal to 1 againstthe alternativethat it is lessthan 1. What is the economicinterpretationof a betaTessthan 1? (c) Test at the 57o level of significance the null hypothesis that Microsoft's "beta" value is less than or equal to I against the alternative that it is greater than 1. What is the economic interpretation of a beta more than 1? (d) Construct a 95Vointerval estimateof Microsoft's "beta." Assume that you are a stockbroker.Explain this result to an investor who has come to you for advice. (e) Test (at a 5Vo significance level) the hypothesis that the intercept term in the CAPM model for eachstock is zero, againstthe alternativethat it is not. What do you conclude? 3.8 Consider the housing starts data (house_starts.dat)introdtced in Exercise 2.13. (a) Estimate the simple regressionof housing starts(STARIS) on the 30-year fixed rate (FIXED_RAIE). Using a 5Vosigniflcancelevel, test the null hypothesisthat there is no linear relationshipbetweenthesevariablesagainstthe alternativethat there is an inverse relationship. (b) It is conjecturedthat if the 3O-yearfixed interestrate increasesby 17othen house starts will fall by 150,000.Test this hypothesis at the 5Volevel of significance using a two-tail test. (c) Construct a 95Eointerval estimateof the slope from the regressionestimated in part (a). State the meaning of this interval estimate.In part (b) you tested that the slope of the relationship was B2 : - 150.Is the value - 150 inside the 95Vointewal estimate?How does this findine relate to the hypothesis test in (b)?
3.6 EXERCISES
mount of life ) you usethis 0'l
00 increasein 0. Choose an d relationship life insurance I the slope of rgs about the td.
-j.9* Reconsiderthe presidential voting data (faindat) introduced in Exercise 2.14. (a) Using the regression model VOTE : 9r -t \zGROWTH + e, test (at a 57o significance level) the null hypothesis that economic growth has no effect on the percentage vote earned by the incumbent party. Select an alternative hypothesisand a rejectionregion. Explain your choice. (b) Using the regressionmodel in part (a), construct a95%ointewal estimate for 82, and interpret. (c) Using the regressionmodel VOTE:fu *\zINFLATION + e, test the null hypothesis that inflation has no effect on the percentagevote earned by the incumbent party. Select an alternative hypothesis, a rejection region, and a significancelevel. Explain your choice. (d) Using the regressionmodel in part (c), construct a95%ointewal estimate for 82, and interpret. i l0
g against the re hypotheses
'beta"
value he economic obil-Exxon's : it is lessthan Microsoft's t it is greater
t? me that you te to you for I term in the not.Whatdo
:cise2.73. iO-yearfixed pothesisthat ernativethat 6 then house significance n estimated ) you tested 0 inside the othesis test
71
The file brdat containsdata on 1080 housessold in Baton Rouge, Louisiana during mid-2005. (a) Estimatethe regressionmodel PRICE : Br * \1SQFT * e for (i) all the houses in the sample,(ii) town houses,and (iii) French style homes.Constructa 957o interval estimate for $2 in each case and discussthe differences. (b) Test the hypothesisthat an additional squarefoot of living areaincreaseshouse price by $80 for eachof the casesin part (a). Use a two-tail testusing the a : 0.05 level of significance.
: I I + The file stockton2.datcontainsdata on 880 housessold in Stockton, CA during mid2005. This data was consideredin Exercise2.12. (a) Estimatethe regressionmodel PRICE : Br * PISQFT * e for all the housesin the sample. Test the hypothesis that an additional square foot of living area increaseshouseprice by $80. Use a two-tail test using the ct :0.05 level of significance. (b) Repeatpart (a) using housesthat were vacant at the time of sale. (c) Repeatpart (a) using housesthat were occupied (not vacant) at time of sale. (d) Using the housesthat were occupied at the time of sale,test the null hypothesis that the value ofan additional squarefoot ofliving area is less than or equal to $80, against the alternative that the value of an additional squarefoot is worth more than $80. (e) Using the housesthat were vacant at the time of sale, test the null hypothesis that the value of an additional squarefoot of living area is more than or equal to $80, against the altemative that the value of an additional square foot is worth less than $80. (fl Construct a 95Vo intewal estimate for B2 using (i) the full sample, (ii) houses vacant at the time of sale, and (iii) occupied at the time of sale. ll
How much does experienceaffect wage rates?The data file cps_small.dat contains 1000 observationson hourly wage rates, experience,and other variables from the 1997 Current PopulationSurvey (CPS). (a) Estimate the linear regression WAGE: Br * P2EXPER* e and discuss the results. Using your software plot a scatterdiagram with WAGE on the vertical axis and EXPER on the horizontal axis. Sketch in by hand, or using your software, the fitted regressionline. (b) Test the statistical significance of the estimated slope of the relationship at the 5Volevel. Use a one-tail test.
72
INTERVAL ESTIMATION
AND HYPOTHESIS TESTING
(c) Repeatpart (a) for the subsamplesconsisringof (i) females, (ii) males, (iii) blacks,and (iv) white males.What differences,if any, do you notice? (d) For each of the estimatedregressionmodels in (a) and (c), calculatethe least squaresresiduals and plot them against EX?ER. Are any patterns evident? 3.13i) Qspenl Exercise 3.12 using the data file cps.dar,which contains 4?33 observations. This exercise may not work with "student" versions of software.
Appendix 3A Derivation of the /-Distribution Interval estimationand hypothesistestingproceduresin this chapterinvolve the r-distribution. Here we develop the key result. The first result that is neededis the normal distribution of the least squaresestimator. Consider,for example,the normal distribution of b2theleastsquaresestimatorof p2, which we denote as /, l'\ o" b't-Nl 8,. =l
>(*,- x)')
A standardizednormal random variable is obtained from b2 by subtracting its mean and dividing by its standarddeviation:
-N(0,l)
(3A.1)
That is, the standardized random variable Z is normally distributed with mean 0 and variance 1. The secondpieceofthe puzzleinvolvesa chi-squarerandomvariable.IfassumptionSR6 holds,thentherandomerrorterme;hasanormaldistribution, e;-N(0,o2).Again,wecan standardizetherandomvariablebydividingbyitsstandarddeviationsothateilo-N(0,1). The square of a standard normal random variable is a chi-square random variable (see Appendix B.5.2) with one degreeof freedom,so (e;f o)2 - xf,,. trm the randomeffors are independentthen
,(?)':(7)'*O'** (3)'-*'r,
(3A.2)
Since the true randomerrorsareunobservablewe replacethem by their samplecounterparts, the least squaresresiduals Ai : yi - b1 - bzxi to obtain
>a? (N - 4oz ,r v:a
(3A.3)
The randomvariableVin (3A.3) doesnot havea X2r, distributionbecausethe leastsquares residuals arenot independentrandom variables.All N residuars?; - y; - b1 - b2xi depend on the leastsquaresestimatorsby andb2.Itcanbe shownthat only N - 2 of the leastsquares residualsare independentin the simplelinear regressionmodel.Consequently,the random variablein (3A.3) hasa chi-squaredistributionwith N - 2 degreesoffreedom.That is, when
APPENDIX
s, (ii) males, (iii) ru notice? ;alculate the least .tternsevident?
38 DISTRIBUTION
OF THE I-STATISTIC UNDER
I11
73
multiplied by the constant(N - 2) I oz ,the random variable 62 hasa chi-squaredistribution t'.ith N - 2 degrees of freedom,
,:t
(N - 2\62
-Xir-zr
(3A.4)
'733 observations.
: the r-distribution. iquaresestimator. mtorof B2,which
ting its mean and
\\-e have nal established the fact that the chi-square random variable V is statistically :ndependent of theleastsquaresestimatorsb1andb2,but it is. The proof is beyondthe scope oi this book. Consequently, V and the standardnormal random variable Z in (3A.1) are :ndependent. From the two random variables V and Z we can form a /-random variable. A r-random rariable is formed by dividing a standardnormal random variable, Z-N(0,1), by the lquare root of an independent cli-square random variable, V -X?^t, that has been divided rr its degreesof freedom, la. That is,
t:
Z
tr*-tt^t
Thet-distribution's shapeis completely determinedby the degreesof freedomparameter,m, ind the distributionis symbolizedby 4^t. SeeAppendix B.5.3.Using ZandVfrom (3A.1) :nd (3A.4), respectively,we have
Z
\/vl(N -2)
(3A.1)
(bz
vith mean0 and f assumption SR6 2).Again,wecan *e;fo - N(O,l). om variable(see :andomerrorsare
(34.2) tplecounterparts,
(3A.3)
-u,I ttrtxr-X nN-aqa Y
(3A.s)
N1
bz-Fz
bz-92
bz-92
l1/ var\oz)
se(br)
-.t
' \ t- -\ - ._ 1
The last line is the key result that we statein (3.2), with its generalizationin (3.3).
Appendix 38 Distribution
of the /-Statistic Under F11
frr examinethe distribution of the /-statisticin (3.7) when the null hypothesisis not true, .rpposethat the true B2 : l Following the stepsin (3A.5) in Appendix 3,{we would find ::l3t
the least squares b1- b2x;depend 'the least squares :ntly, the random rm.Thatis, when
bz-I t:r.ib;
-ttN-21
:: P' : 1 and c I I thenthe test statisticin (3.7) doesnot havea r-distributionsince,in its : rrmation, the numeratorof (3A.5) is ,?otstandardnormal. It is not standardnormal because ':e incorrect value B2 : c is subtractedfrom b2.
74
INTERVAL ESTIMATION
AND HYPOTHESIS TESTING
: c,thenthenumeratorin(3A.5)thatis If Bz : l andwe incorrectlyhypothesizethatFz (3.7) has the distribution used in forming bz-c
(3B.1t
Jiat@, where var(b.,)
o2 S/.. L\^I
-;\z
Since its meanis not zero, the distribution of the variable in (3B.1) is not standardnormal, as reouired in the formation of a /-random variable.
(3A.5)thatis
(38.1)
Chapter
Prediction, Goodness-of:Fit, and Modeling Issues
'd normal. as
Learning Objectives Basedon the material in this chapter, you should be able to 1. Explain how to usethe simple linear regressionmodel to predict the value of y for a given valueofx. 2. Explain, intuitively and technically, why predictions forr valuesfurther fromx are less reliable. 3. Explain the meaning of SS?i SSR,and SSE, and how they are related to R2. 4. Define and explain the meaning of the coefficient of determination. 5. Explain the relationship between correlation analysis and R2. 6. Report the results of a fitted regressionequation in such a way that confldence intervals and hypothesis tests for the unknown coefficients can be constructed quickly and easily. 7 . Describehow estimatedcoefficientsandother quantitiesfrom a regressionequation will change when the variables are scaled. Why would you want to scale the variables? 8. Appreciate the wide range of nonlinear functions that can be estimated using a model that is linear in the parameters. 9. Write down the equations for the log-log, logJinear, and linear-log functional forms. 10. Explain the differencebetweenthe slopeof a functional form and the elasticity from a functional form. 11. Explain how you would go about choosing a functional form and deciding that a functional form is adequate? 12. Explain how to test whether the equation "errors" are normally distributed? 13. Explain how to compute a prediction, a prediction interval, and a goodness-of-fit measurein a log-linear model.
76
PREDICTI ON, COODNESS_OF_FIT. A\D
MODELING
ISSUES
Keywords coefficient of determination correlation data scale forecast error forecast standarderror functional form goodness-of-fit growth model
Jarque-Beratest kurtosis least squarespredictor linear model linear relationship linear-logmodel log-linear model log-log model
log-normal distribution prediction prediction interval R2 residual skewness
In Chapter 3 we focused on making statistical inferences, constructing confidence intervals, and testing hypotheses about regression parameters.Another purpose of the regressionmodel, and the one we focus on first in this chapter,is prediction. A prediction is a forecast of an unknown value of the dependentvariable y given a particular value ofx. A prediction interval, much like a confidence interval, is a range of values in which the unknown value of y is likely to be located. Examining the correlation between sample values of y and their predicted values provides a goodness-of-fit measure called R2 that describes how well our model fits the data. For each observation in the sample the difference between the predicted value ofy and the actual value is a residual. Diagnostic measuresconstructed from the residuals allow us to check the adequacy of the functional form used in the regressionanalysis and give us some indication of the validity of the regressionassumptions.We will examineeachof theseideasand concepts in turn.
4.7 Least Squares Prediction In Section2.3.3bwe briefly introducedthe idea that the leastsquaresestimatesof the linear regressionmodel provide a way to predict the value of y for any value of x. The ability to predict is important to businesseconomistsand financial analystswho attempt to forecast the salesand revenuesof specific firms; it is important to governmentpolicy makers who attempt to predict the rates of growth in national income, inflation, investment, saving, social insurance program expenditures, and tax revenues; and it is important to local businesses to havepredictionsof growth in neighborhoodpopulationsand income so that they may expand,or contract,their provision of services.Accuratepredictionsprovide a basisfor better decision making in every type of planning context.In this sectionwe explore the use of linear regressionas a tool for prediction. Given the simplelinear regressionmodel and assumptionsSR1-SR6,let.{0be a valueof the explanatoryvariable.We want to predictthe conespondingvalueof H which we call ys. In orderto useregressionanalysisasa basisfor prediction,we mustassumethatys andxs are related to one another by the same regressionmodel that describesour sample of data, so that, in particulaq SR1 holds for theseobservations )o : 9r I }zxo I eo
(4.1)
where es is a random error. We assumethat E0i0): 9t * B2.tsand E(eg):0. We also assumethat es has the same varianceas the regressionerrors, var(es): o2. and e6 is
4.1 LEAST SQUARES PREDICTION
b2 xi
listribution terval
.:r;t'nt 4.7
A pointprediction.
-ncorrelatedwith the random errors that are part ofthe sampledata, so that cov(es,e) :0 = | ,2, . . . , N. The least squares predictor of ye comes from the fitted regressionline confidence ose of the ediction is valueofx. ; in which r between I measure ion in the . residual. equacy of ion of the I concepts
j,o:h*b2xs
(4.2)
That is, the predicted value !e is given by the point on the least squaresfitted line where : = -r0,as shown in Figure 4.1. How good is this prediction procedure?The least squares .itimators bl andb2 are random variables-their valuesvary from one sampleto another.It :'-rllowsthat the least squarespredictor io: br * b2xsmust also be random. To evaluate :ou well this predictor performs we define the forecast error, which is analogousto the -
f : vo -.lo : (9r * 9zxoa no)- (h * bzxo)
(4.3)
1\'ewould like the forecasterror to be small, implying that our forecastis close to the value ae are predicting. Taking the expectedvalue of/we find
the linear ability to r forecast kers who :, saving, to local re so that lrovide a : explore value of e call ye. nd-16are data,so
(4.1) We also rd e6 is
E(f) : Fr * Fzxo+ E(es) - ln(bt) + E(b2)xgl :9r*1zxo+0-[9rfFzro] -0 r hich means,on average,the forecasterror is zero, and !6 is an unbiased predictor ofys. However,unbiasednessdoesnot necessarilyimply that a particular forecastwill be close to jre actualvalue. The probability ofa small forecasterror also dependson the varianceofthe :orecasterror. While we will not prove it, !s is the best linear unbiased predictor (BLIIP) rf re if assumptionsSR1-SR5 hold. This result is reasonablegiven that the least squares stimators b1 and b2 are best linear unbiasedestimators. Using (4.3) and what we know about the variancesand covarianceof the least squares 3:timators,we can show (seeAppendix 44, at the end of this chapter)that the varianceof the i.'r199351 errOr iS
:"[,***#=rA var(/)
(4.4)
\otice that someof the elementsof this expressionappearin the formulas for the variances .rfthe leastsquaresestimatorsand affect the precisionof prediction in the sameway that they rffect the precision of estimation.We would prefer that the varianceof the forecasterror be 'mall, which would increasethe probability that the prediction j's is close to the value ys we r,retrying to predict. Note that the variance of the forecast error is smaller when
PREDICTION,
78
i.
GOODNESS-OF_FIT. .\\I)
\4ODELING ISSUES
the overalluncertaintyin the model is smaller,asmeasuredby the varianceof the random errors ol;
ii.
the samplesize N is larger;
iii.
the variation in the explanatoryvariableis larger; and the valueof (xs - ;)2 is small.
iv.
.:lllp '.:dt
,t
,-: S
The new additionis the term (ro - t)'.which measureshow farxe is from the centerof the x-values.The more distant.reis from the centerof the sampledatathe larger the forecast variancewill become.Intuitively, this meansthat we are able to do a betterjob predicting in the region where we have more sample information, and we will have less accurate predictionswhen we try to predict outside the limits of our data. In practicewe replaceo2 in 14.+7by its estimator62 to obtain I
:
i
u--
-f | r
IL
/
|
..t1.
-
-\Z
(x0-xl
-]. ----------------;
N
I
I I
- T)' ,-t I -).(x, ,_-, J
The squareroot of this estimated variance is the standard error of the forecast
se("f):
(4.s)
var(f)
Definingthecritical valuet. to be the 100(1- af 2)-percentilefrom the r-distribution,we can obtain a 100(1- ct)7oprediction interval as
j'et r.se(/)
(4.6)
See Appendix 4A for some details related to the development of this result. Following our discussionof var(/) in (4.41.the farther.xeis from the samplemeanx, the larger the varianceof the prediction error and the less reliable the prediction is likely to be. In other words, our predictions for valuesof xs close to the samplemeanx are more reliable than our predictions for values of xe far from the samplemean t. This fact shows up in the size of our prediction intervals. The relationship betweenpoint and interval predictions for different valuesofxe is illustrated in Figure 4.2. A point prediction is given by the fitted least squaresline !s : bt * bzxo.The prediction interval takesthe form of two bandsaroundthe fitted least squaresline. Becausethe forecast variance increasesthe farther xo is from the
!o + r"se(/)
) s = b 1 +b 2 x s fo - t" se("f)
prcunp 4,2
Point and intervalorediction.
':!'n
:th
4.1 LEAST SQUARES PREDICTION
riance of the
'ample mean of .r, the confidencebands are their narrowest when xs : f, and increase1n ridth as lxo - tl increases.
f.1.1
center of the t the forecast rb predicting less accurate
79
PnrnrcrroN rN THE Fooo ExpENDrrrrRE MooEr
Ln Section 2.3.3b we predicted that a household with xs : $2000 weekly income would gend $287.61on food using the calculation io : br * b2xs: 83.4160+ 10.2096(20): 287.6089 \ow we are able to attacha "confidence interval" to this prediction. The estimatedvariance :f the forecast error is
@1:a,[r+*.#+,; )cast
(4.s)
- o .^-r+. 6_ '+ N
1+
\xo - 7)'var(bz)
rtion,we can Lr the last line we haverecognizedthe estimatedvarianceof b2from(2.21).In Section2.7.2 (4.6)
re obtainedthe values62 :8013.2941ardi *@r-) :4.3818. For the food expenditure :rta N : 40 and the samplemean of the explanatory variable is t : 19.6048.Using these 'alues we obtain the standard error of the forecast se(/) :
: meanx, the likely to be. rore reliable ws up in the :dictions for e fitted leasr s around the is from the
rl 6328. If we select I - cr : 0.95, then t, : t(o.sts.zz\ :2.0244 and the 957o prediction ''rterval for ys is !e * r.se(/) :287.6069 +2.0244(90.6328) : [104.1323,471.08s4) thrr prediction interval suggeststhat a household with $2000 weekly income will spend r-rmewherebetween$104.13and $471.09on food. Sucha wide interval meansthat ourpoint lediction $287.61 is not very reliable. We have obtained this wide prediction interval for re valueof xs : 20 that is closeto the samplemean7 : 19.60. For valuesof t that aremore 3\reme the prediction interval would be even wider. The unreliable predictions may be i.rghtly improved if we collect a larger sample of data, which will improve the precision eith which we estimatethe model parameters.However, in this example the magnitude of :re estimatederror variance62 is very close to the estimatedvariance of the forecast error .fr|;, suggestingthat the primary uncertainty in the forecastcomesfrom large uncertainty r the model. This should not be a surprise, since we are predicting household behavior, rhich is a complicatedphenomenon,on the basis of a single householdcharacteristic, -:L-ome.While income is a key factor in explaining food expenditure,we can imagine many -rher household demographic characteristics that may play a role. To more accurately redict food expenditurewe may needto include theseadditional factors into the regression rodel. Extending the simple regression model to include other factors will begin in Chapter5.
80
PREDICTION,
GOODNESS_OF_FIT.,1.\T) MODELING
ISSUES
4. 2 Measuring Goodness-of-Fit
:we :rea
Two major reasonsfor analyzing the model Yi:9r
(4.7)
l9zxilei
are to explain how the dependentvariable (v;) changes as the independent variable (x;.1 changes,and to predict ),0given an r0. Thesetwo objectivescome under the broad headings of estimation and prediction. Closely allied with the prediction problem discussedin the previous section is the desire to use x; to explain as much ofthe variation in the dependent variable y; as possible.In the regressionmodel (4.7) we call r; the "explanatory" variable becausewe hope that its variation will "explain" the variation in y;. To develop a measureof the variation in y; that is explained by the model, we begin b1, separatingy; into its explainable and unexplainable components.We have assumedthat yi:
Ihe :rd 1 .ld I .C!
l-qu r:fd
(4.8)
E(yi) * ei
I
where E(y;) : 9r f B2x;is the explainable,"systematic" componentof y;, and e; is the random, unsystematicand unexplainable component of y;. While both of these parts are unobservableto us, we can estimatethe unknown parametersB1 and B2 and, analogousto (4.8), decomposethe value of y; into
l
(4.9)
li:yi-t?i
where j,; : br * b2x; and ?i: yi - yi. In Figure 4.3 the "point of the means" (f, y) is shown, with the least squaresfitted line passing through it. This is a characteristic of the least squaresfitted line whenever the regressionmodel includes an interceptterm. Subtractthe samplemean y from both sidesof the equation to obtain (4 lo\
yi_!:(j,i_y)+Zi
As showninFigure4.3 the differencebetweeny;andits meanvalueyconsistsof apart thatis "explained" by the regressionmodel yi - y and a part that is unexplained2;. The breakdownin (4. 10) leadsto a decompositionof the total samplevariability in y into explainedand unexplainedparts.Recall from your statisticscourses(seeAppendix C.4) that
- \ll
.nl
det
= unexplained component \xi, Ji)
i=bt+bzx
(x, ii) Cx,y)
xxi
prcunn
4.3
Explained and unexplainedcomponentsof ,r.;
At .\i
4.2 MEASURING
GOODNESS_OF-FIT
81
:: *e havea sampleof observations)r,)2,. . .,y7v,two descriptivemeasuresarethe sample :rean t and the sample variance
_ r)' ur.:Z(Y, ,
(4.7)
N_l
lent variable (x;) e broad headings discussedin the in the dependent .natory" variable
Thenumeratorof this quantity, the sum of squareddifferencesbetweenthe samplevaluesy; r:rd the samplemeany, is a measureof the total variation in the samplevalues.If we square r-rd sum both sidesof (4.10), and using the fact that the cross-productterm l(!; - y)Oi : 0 .ee Appendix 4B), we obtain
del, we begin by re assumedthat
F4uation(4.11)givesus a decompositionof the "total samplevariation" iny into explained nd unexplained components.Specifically, these "sums of squares" are
(4.8) y;, and e; is the ,f these parts are nd, analogousto
(4.e) truaresfittedline re wheneverthe 'ombothsides of (4.10) ts ofa partthatis :d 2;. riabilityiny into rpendixC.4)that
romponent
r(y' - y)' :2(i, --y)2 +2a?
(4.11)
I . I(yr - r)' : total sum of squares: SSZ:a measureof total variationin y about the sample mean. l. I(lr - y)2 : ,u* of squaresdue to the regression: SSR:that part of total variation in y, about the sample mean, that is explained by, or due to, the regression.Also known as the "explained sum of squares." 1. Za! : sum of squaresdue to error : SSE:that part of total variation in y about its mean that is not explained by the regression.Also known as the unexplainedsum of squares,the residual sum of squares,or the sum of squarederrors. '-'iing
theseabbreviations(4.11) becomes SS?": SSR+ SSE
This decompositionof the total variation in y into a part that is explained by the regression :,odel and a part that is unexplainedallows us to define a measure,called the coefficient of &termination, or R2, that is the proportion of variation in y explained by x within the =gressionmodel.
(4.t2)
R':ssR:l-SSE
s^sr
ssr
The closer R2 is to 1, the closer the sample values y; are to the fitted regressionequation . = br I b2x;.If Rz : 1, then all the sampledatafall exactly on the fitted least squaresline, *. SSE : 0, and the model fits the data "perfectly." If the sample data for y and -r are :ncorrelatedand show no linear association,then the leastsquaresfitted line is "horizontal," r-ndidentical to t, so that SSR: 0 and R2 : 0. When 0 < Rz < 1, it is interpreted as "the :roportion of the variation in y about its mean that is explained by the regressionmodel." r.2.1
Conngerro\T
ANALvsrs
Lr Appendix B.4.3 we discuss the covariance and correlation between two random iariables x and y. The correlation coefficient p- between .r and y is defined in (B.20) as Ptv :
cov(x,y)
liardlinj
ory
oxoy
(4.r3)
\\'e did not discussestimating the correlation coefficient. We will do so now to develop a :'eful relationship between the sample correlation coefficient and R2.
82
PREDICTION,
GOODNESS_OF_FIT. AND MODELING
ISSUES
Given a sampleof data pairs (.r;,y;), i : 1, .. . , N, the samplecorrelationcoefficientis obtained by replacing the covariance and standard deviations in (4.13) by their sample analoss: 'A'e o-.
(4.14t
ord,
:ls
where
-')'lw-
J:OSS(/.1\t
r(y, - fl'l@ The sample correlation coefficient r, has a value between - I and 1, and it measuresthe strength of the linear associationbetween observedvalues ofr and y. 4.2.2
a
:3por . iriati - "lnst
6,y:Z(xi -7)(y' - t)/(N - 1) 0x:
co :rplai
CoRnpranroN ANALYSTsAND R2
There are two interesting relationshipsbetween R2 and r- in the simple linear regression model. 1 . The first is that Po: R2. That is, the square of the sample correlation coefficient between the sample data values x; and !; is algebraically equal to R2 in a simple regressionmodel. Intuitively this relationship makessense:f, falls between0 and I and measuresthe strength of the linear associationbetweenx andy. This interpretation is not far from that of R2: the proportion of variation in y about its mean explained by x in the linear regressionmodel.
2. The second, and more important, relation is that R2 can also be computed as the squareof the sample correlation coefficient betweenyi andyi : h -l b2x;.Thatis. Rz : *yi. As suchit measuresthe linear association,or goodness-of-fit,betweenthe sample data and their predicted values. Consequently R' is sometimes called a measure of "goodness-of-fit." This result is valid not only in simple regression models but also in multiple regression models that we introduce in Chapter 5. Furthermore,asyou will seein Section4.4,the conceptof obtaininga goodness-offit measureby predicting y as well as possible and finding the squaredcorrelation coefficient between this prediction and the sample values of y can be extendedto situations in which the usual R2 does not strictly apply.
:agre '..set
:',alu :,-rns
-a sir
:recis spen .ill b Cc -.-.ns ':lue
- : l ec :-aon ::'..r,r -,:. A In ::Il
C
1. 1 .
-: an ':t4
:.lln
.rbrr . tel
4.2.3
TnE Fooo ExpENonurc
Exervrprr
Look at the food expenditureexamplein Section 2.3.2, andin particularthe data scatterand fitted regressionline in Figure 2.8, and the computer output Fi gure2.9. Go ahead.I will wait until you get back. The questionwe would like to answeris "How well doesour model fit the data?" To compute the R2 we can use the sums of squares
SSZ: I(y; - y)' :495132.160 sSE: I(y, - y,)' :2a? :304505.116
. ! l a
4.2 MEASURING GOODNESS-OF-FIT
:fficient is :ir sample
(4.r4)
(4.1s)
asuresthe
regression
Then R2:1
ssE ssr
"-I _ -304505.176 :O.3lt5 495132.160
We conclude that 38.57oof the variation in food expenditure (about its sample mean) is explainedby our regressionmodel, which usesonly income as an explanatory variable. Is this a good R2? We would argue that such a question is not useful. While finding and reportingR2 provides information about the relative magnitudesof the different sourcesof rariation, debates about whether a particular R2 is "large enough" are not particularly .'onstructive.Microeconomic household behavior is very difficult to explain fully. With cross-sectionaldata R2-valuesfrom 0.10 to 0.40 are very common even with much larger regressionmodels. Macroeconomic analyses using time-series data, which often trend togethersmoothly over time, routinely report R2-valuesof 0.90 and higher. You should not evaluatethe quality of the model basedonly on how well it predicts the sampledatausedto .onstruct the estimates.To evaluatethe model it is as important to consider factors such as the signs and magnitudesof the estimates,their statistical and economic significance,the precision of their estimation, and the ability of the fitted model to predict values of the dependentvariable that were not in the estimation sample. Other model diagnostic issues *ill be discussedin the next section. Correlation analysisleads to the sameconclusionsand numbers,but it is worthwhile to considerthis approachin more detail. The sample correlation between the y and x sample valuesis
418.75
on
:oefficient r a simple en0andI interpret; its mean ted as the ;. That is, )tweenthe s called a :egression )hapter 5. rdness-oforrelation .tendedto
catterand iwillwait rdelfitthe
83
-
Oxoy
:0,62
(6.848)(tr2.67s)
The correlation is positive, indicating a positive associationbetweenfood expenditureand income. The sample correlation measuresthe strength of the linear association, with a maximum value of 1. The value r,y :0.62 indicates a non-negligible but less than perfect t i t .A s e x p e c t e d fr:0.622 :0.385 : R2. In Figure 4.4 we plot the samplevaluesof y; and their correspondingfitted values!;. You s-ancorfiffi that the correlation between these values is riu: 9.62. 1.2.4
RrponrrNc
rHE RssuLTs
In any paper where you write the results of a simple regression,with only one explanatory variable,theseresults can be presentedquite simply. The key ingredientsare the coefficient estimates,the standarderrors (or r-values),an indication of statistical significance,and R2. Also, when communicating regression results avoid using symbols like x and y. Use abbreviationsfor the variablesthat arereadily interpreted,defining the variablespreciselyin a separatesection of the report. For the food expenditure example, we might have the variable definitions: FOOD_EXP - weekly food expenditure by a household of size 3, in dollars INCOME:
weekly householdincome, in $100 units
Then the estimated equation results are
FOOD-EXP :83.42 + 10.21INCOME (r") (43.4r). (2.09)--.
R 2: 0 . 3 8 5
A-PDF Split DEMO : Purchase from www.A-PDF.com to remove the watermark
84
PREDICTION,
GOODNESS-OF-FIT. A\I)
MODELING
ISSUES
i3
.-.r'fe
: ,llar - rlin Cc Jol a loo
o
a
Yo
a:
.- a
-
o
a
:.JOn
aao a
a aa
Oa
ot
N
ata _clc .-,L
' .rJ t00 FTGURE 4.4
y = weekly food expenditure in $ Plotof predictedH j',againsty.
600
-:ih t
v
-.,'lOr T}
'::nd Report the standarderrors below the estimated coefficients. The reason for showing the standardelrors is that an approximate 957o interval estimate (if the degreesof freedom N - 2 are greater than 30) is b7,f 2 x se. The reader may then divide the estimateby the standarderror to obtain the value of the t-statistic if desired. Furthermore, testing other hypothesesis facilitated by having the standarderror present.To test the null hypothesis Ho:92: 8.0, we can quickly constructthe r-statistic7 : l(10.21 - 8)12.09)land proceed with the steps of the test procedure. Asterisks are often used to show the reader the statistically significant (that is, significantly different from zero using a two-tail test) coefficients, with explanations in a table footnote: * indicatessignificantat the 107olevel x* indicatessignificantat the 5Volevel *** indicatessignificant atthe l%olevel The asterisks are assigned by checking the p-values from the computer output, as in Fisure 2.9.
4.3 t[odeling Issues 4.3.1
TnE Eprncrs or ScarrNc rHr Dana
Data we obtain are not always in a convenient form for presentationin a table or use in a regressionanalysis.When the scaleof the data is not convenient,it can be alteredwithout changingany ofthe real underlyingrelationshipsbetweenvariables.For example,the real personalconsumptionin the United States,asof the lst quarterof 2006,was$8023.6billion annually.That is, written out, $8,023,600,000,000, or 8 trillion 23 billion 600 million dollars.While we cor.r/duse thelong form of thenumberin a tableor in aregressionanalysis,
Sr . . - : lt c-
--... tc\ -. ir
4.3 MODELING
ISSUES
85
there is no advantageto doing so. By choosing the units of measurementto be "billions of dollars," we have taken a long number and made it comprehensible.What are the effects of scaling the variables in a regressionmodel? Consider the food expenditure model. In Table 2.1 we report weekly expenditures in dollars but we report income in $ 100 units, so a weekly income of $2000 is reported as x : 20. Why did we scale the data in this way? If we had estimated the regressionusing income in dollars. the results would have been
TNCOME($)R',:0.385 FOOD_EX?:83.42 + 0.1021 (43.41)(0.0209)--(r")
howing the of freedom nate by the ,sting other hypothesis nd proceed rt (that is, anations in
tput, as in
There are two changes. First, the estimated coefficient of income is now 0.1021. The interpretationis "If weekly householdincome increasesby $1 then we estimatethat weekly food expenditurewill increaseby about 10 cents." There is nothing mathematically wrong with this, but it leads to a discussionof changesthat are so small as to seemirrelevant. An increasein income of $100 leadsto an estimatedincreasein food expenditureof $10.21, as before, but these magnitudes are more easily discussed. The other changethat occursin the regressionresultswhen income is in dollars is that the standarderror becomessmaller,by a factor of 100.Since the estimatedcoefficient is smaller by a factor of 100 also, this leaves the t-statistic and all other results unchanged. Such a changein the units of measurementis called scaling the data. The choice of the scale is made by the researcherto make interpretation meaningful and convenient. The choice of the scale does not affect the measurementof the underlying relationship, but it doesaffect the interpretation of the coefficient estimatesand some summary measures.Let us list the possibilities: l.
Changing the scale ofx: In the linear regressionmodel y : Br * B2x * e, suppose we changethe units of measurementof the explanatory variable r by dividing it by a constant c. In order to keep intact the equality of the left- and righthand sides, the coefficientof r must be multiplied by c. That is, y- Fr *Fzxf e: 9r* (cB2)(xlc) * + F)x- * e,where9): c9zand.r*: x/c.Forexample,ifxis ":9t measuredin dollars, and c : 100,thenr* is measuredin hundredsof dollars. Then Bj measuresthe expectedchange in y given a $100 increasein x, and Bi is 100 times larger than B2. When the scale of x is altered the only other change occurs in the standarderror of the regressioncoefficient, but it changesby the samemultiplicative factor as the coefficient, so that their ratio, the r-statistic, is unaffected. All other regressionstatistics are unchanged.
2. Changing the scale ofy: If we changethe units of measurementof H but not .r, then all the coefficients must change in order for the equation to remain valid. That is,
ylc : (fu lc) + (9zlc)x+ (elc) ory* - Bi + Bix* e*.In thisrescaled modelBi
rr use in a :d without .e,the real 3.6billion l0 million r analysis,
measuresthe changewe expect in y* given a l-unit changein -r. Becausethe error term is scaled in this processthe least squaresresiduals will also be scaled.This will affect the standarderrors of the resression coefficients. but it will not affect /-statistics or R2. If the scaleofy and the scaleofx arechangedby the samefactor, then therewill be no change in the reported regression results for b2, but the estimated intercept and residuals will change; /-statistics and R2 are unaffected.The interpretation of the parametersis made relative to the new units of measurement.
PREDICTION,
86
4.3.2
GOODNESS-OF_FIT. A\I)
- \ 1O D E L I N G
ISSUES
CHoosrNc e FuNcrroNer Fonnr
In our ongoing example,we have assumedthat the mean householdfood expenditureis a linear function of householdincome. That is, we assumedthe underlying economic relationship to be E(y) : Pr * B2x, which implies that there is a linear, straightJine relationship between E$t) andx. Why did we do that? While the world is not "linear," a straight line is a good approximation to many nonlinear or curved relationshipsover narrow ranges.AIso, inyourprinciples of economicsclassesyou may havebegunwith straightlines for supply, demand and consumption functions, and we wanted to easeyou into the more "artistic" aspectsof econometrics. The starting point in all econometricanalysesis economic theory. What doeseconomics really say about the relation between food expenditure and income, holding all else constant?We expect there to be a positive relationship between these variables because food is a normal good. But nothing saysthe relationshipmust be a straightline. In fact, we do not expect that as household income rises that food expenditures will continue to rise indefinitelyat the sameconstantrate.Instead,as incomeriseswe expectfood expenditures to rise, but we expect suchexpendituresto increaseat a decreasingrate. This is a phrasethat is usedmany times in economicsclasses.What it meansgraphicallyis that there is not a straighrline relationship between the two variables, and that it might look something like Figure 4.5. The simple linear regressionmodel is much more flexible than it appearsat first glance. By transfurmfugthe variablesy andr we can representmany curved,nonlinearrelationships and still usethe linear regressionmodel.If you havenot doneso, readAppendix A.2.4 and Appendix A.4. Mark Table A.2 and Figure A.3 with paperclips so that you can find them easily. Choosing an algebraic form for the relationship meanschoosing transformatio,rzs of the original variables.This is not an easyprocessand it requiresgood analyticgeometryskills and some experience.Itmay not come to you easily. The variable transformationsthat we besin with are
l . Power: if ;r is a variable then / meansraising the variable to the powerp; examples are quadratic (x2) and cubic (x3) transformations.
2 . The natural logarithm: if r is a variable then its natural logarithm is ln(x). 3 . The reciprocal:ifr is a variablethen its reciprocalis llx.
FIGURE 4. 5
A nonlinearrelationshipbetweenfood exoenditureand income.
. + . 3M O D E L I N G I S S U E S
nditure is lconomic dght-line linear," a er nalTow .ightlines the more ;onomics ; all else ; because rct,we do te to rise enditures rrasethat eisnota hing like ;t glance. tionships \.2.4 and ind them ,nsof the try skills s that we
:xamples
87
L singjust thesethreealgebraictransformationsthereareamazingvarietiesof "shapes" that rre can represent,as shown in Figure A.3. A difficulty introduced when transforming variables is that regressionresult interpretationschange.For each different functional form, shown in Table A.2, the expressionsfor both the slope and elasticity change from the linear relationship case.This is so because the variablesare related nonlinearly. What this means for the practicing economist is that great attention must be given to result interpretation whenever variables are transformed. Becauseyou may be less familiar with logarithmic transformations,let us summarize the rnterpretationin threepossibleconfigurations.You should readAppendix A.4.4- AA.6for a ;omplete discussion of the algebra. 1. In the log-log model both the dependentand independentvariablesare transformed by the "natural" logarithm. The model is ln(y) : 9t * B2ln(x). In orderro usethis model both y and x must be greaterthan zero, becausethe logarithm is defined only for positive numbers. The parameter B2 is the elasticity of y with respect to r. Referring to Figure A.3, you can seewhy economistsusethe constantelasticity, loglog model specificationso frequently. In panel (c), if 9z > 1 the relation could depict a supply curye, or if 0 < 92 < 1 a production relation. In panel (d), if 92 < 0 it could represent a demand curve. In each case interpretation is convenient becausethe elasticity is constant. 2. In the log-linear model ln(y) : 9r * Fz.r only the dependentvariable is transformed by the logarithm. The dependentvariable must be greater than zero to use this form. A convenient interpretation for the log-linear model is developed in Appendix A.4.5. A one-unit increase in x leads to (approximately) a I0O x B2% changein y. The log-linear form is common; we will treat it fully in Section4.4. For now, note its possible shapesin Figure A.3e. If Bz > 0 the function increasesat an increasing rate; its slope is larger for larger values of y. If 9z < 0, the function decreases,but at a decreasingrate. 3. In the linear-log model ):9r *B2ln(x) the variable x is transformedby the naturallogarithm.SeeFigureA.3f.Theslopeofthisfunctionis LylL*:B2f xandit changesat every point. We can interpret B2 by rewriting the slope expressionas
AY : 9z rooG.{A) loo The term 100(Ax lx) is thepercentagechangein x. Thus in the linear-log model we can say that a lVo increasein.r leadsto a B2/ 100 unit changein y. For example,supposethat we are considering a production relationship where y is output, and x is an input, such as labor. In this casethe slope of the function is the "marginal product" of labor, which we expectto bepositive but diminishing, asimplied by the linearJog model. Lety be daily manufacturing output and x the number of hours of labor input per day. If the estimated relationship is OUTFjT: 100 * 7001I(LABOR),thenweestimatethatalZoincreaseinlaborinputleads to a7-unit increasein y, or al0%o increasein labor will increasedaily output by 70 units.
4.3.3
TnE Fooo ExpsNDrruR-EMoon
Supposethat, in the food expendituremodel, we wish to choose a functional form that is consistentwith Figure 4.5. From the array of shapesin Figure A.3 two possiblechoicesthat are similar in some aspectsto Figure 4.5 arethe reciprocal model and the linear-log model.
88
PREDICTION.
GOODNESS_OF_FIT. A\I)
\ 1O D E L I N G
ISSUES
Thesetwo functional forms are different ways of modeling the data and lead to estimatesof the unknown parametersthat have different economic interpretations. The reciprocal model in this case is
FOOD-EXP: BI- bffi+
C
For the food expendituremodel we might assumethat B1> 0 and 9z < 0. If this is the case, thenasincomeincreases,householdconsumptionoffood increasesat a decreasingrateand reachesan upperbound 81.This model is linear in theparametersbut it is nonlinearin the variables. Even so, if the error term e satisfiesour usual assumptions,then the unknown parameterscan be estimatedby least squares,and inferencescan be made in the usual way. Another property of the reciprocal model, ignoring the error term, is that when /NCOME the model predicts expenditure on food to be negative. This property is <-9zl9r unrealistic; it implies that this functional form is inappropriatefor small values of income. When choosing a functional form, one practical guideline is to consider how the dependent variable changes with the independent variable. In the reciprocal model the slopeof the food expenditurerelationshipis -$21 INCOME2.If the parameterB2 ( 0, then there is a positive relationship between food expenditure and income, and, as income increases,this "marginal propensityto spendon food" diminishes,as economictheory predicts.Indeed,in the reciprocal model, asx becomesvery large the slopeapproacheszero, which says that as income increasesthe household expenditure on food stops increasing after some point. An alternative to the reciprocal model is the linearJog model FOOD_EXp:
gr * B2Ln(INCOME)_r e
For B2 > 0 this function is increasing, but at a decreasingrate. As INCOME increases the slope \yIINCOME decreases.As noted above, in this context the slope is the marginal propensity to spend on food from additional income. Similarly, the elasticity. B2l FOOD-EXP, becomessmaller for larger levels of food expenditure.Theseresults are consistentwith the idea that at high incomes, and large food expenditures,the effect of an increasein income on food expenditure is small.
REMARK: Given this array of models, that involve different transformationsof the dependentand independentvariables,and some of which have similar shapes,what are someguidelinesfor choosinga functional form? 1. Choosea shapethat is consistentwith what economic theory tells us about the relationship. 2. Choose a shapethat is sufficiently flexible to "fit" the data. 3. Choose a shape so that assumptionsSR1-SR6 are satisfied,ensuring that the least squaresestimatorshave the desirable properties describedin Chapters2 and 3.
While theseobjectivesareeasilystated,the reality of modelbuilding is muchmore difficult. You must recognize that we never know the "true" functional relationship between economic variables and the functional form that we select, no matter how elegant, is only an approximation. Our job is to choosea functional form that satisfactorily meetsthe three obiectivesstatedabove.
1.
Re :h
-in
)r li: :to ,lt I ,l
.e
:to
4.3 MODELING
estimatesof
isis the case, sing rate and ,.linearin the he unknown Leusual way. enINCOME property is ,sof income. Jer how the rl model the Bz ( 0, then [, as income .omic theory 'oacheszero, rs increasing
fE increases slope is the he elasticity. re results are : effect of an
Recallthat hypothesistestsand interval estimatesfor the coefficientsrely on the assumption :hat the errors, and hencethe dependentvariabley, are normally distributed.While our tests rnd confidenceintervals arevalid in large sampleswhether the dataarenormally distributed .rr not, it is neverthelessdesirableto havea model in which the regressionerrorsarenormally listributed, so that we do not have to rely on large sampleapproximations.If the errors are not normally distributed we might be able to improve our model by considering an elternativefunctional form or transforming the dependentvariable. As noted in the last "Remark," when choosing a functional form, one of the criteria we might examine is rhether a model specificationsatisfiesregressionassumptions,and in particular whether it -eadsto errors that are normally distributed (SR6). How do we check out the assumptionof rormally distributed errors? We cannotobservethe true random errors,sowe must baseour analysisof their normality ,rn the least squaresresiduals,2i : yi - j,;. Most computer software will createa histogram of theresidualsfor this purposeand may also give statisticsthat canbe usedto formally testa rull hypothesisthat the residuals(and thus the true errors) come from a normal distribution. The relevantEViews output for the food expenditureexample,using the linear relationship n ith no transformation of the variables, appearsin Figure 4.6. What does this histogram :elf us?First, notice that it is centeredatzero. This is not surprising becausethe mean of the .east squaresresiduals is always zero if the model contains an intercept, as shown in Appendix 48. Second, it seemssymmetrical, but there are some large gaps, and it does eot really appearbell-shaped.However, just checking the shapeofthe histogram is not a rutistical "test." There are many testsfor normality. The Jarque-Bera test for normality is basedon two :neasures,skewnessand kurtosis. In the presentcontext, skewnessrefers to how symmetric :he residualsare around zero. Perfectly symmetric residualswill have a skewnessof zero. The skewness value for the food expenditure residuals is -0.097. Kurtosis refers to .tre "peakedness" of the distribution. For a normal distribution the kurtosis value is 3. For more on skewnessand kurtosis seeAppendicesB.4.2 and C.4.2.From Figure 4.6, we see rhat the food expenditure residuals have a kurtosis of 2.99. The skewnessand kurtosis r aluesare close to the valuesfor the normal distribution. So, the question we have to ask is
Series: Flesiduals Sample1 40 Observations40
I
I I
Mean Median Maximum Minimum Std.Dev. Skewness Kurtosis
I ne that the I
z cianters I ore difficult. rip between r elegant, is ly meetsthe
89
+.3.4 Anr rnn RpcnrssroN Ennons NorutlRrlv Drsrnrsurpo?
il "*l :s,whatare s aboutthe
ISSUES
6.93e-15 -$.324473 212.0440 -223.0255 88.36190 -0.097319 2.989034
Jarque-Bera 0.063340 Probability 0.968826
-200
-100
100 200 0 r tcunr- 4. 6 EViewsoutput:residualshistogramandsummarystatistics for food exoenditure :xample.
90
PREDICTION,
GOODNESS_OF-FIT..{\I)
\,IODELING ISSUES
whether 2.99 is sufficiently different from 3, and -0.097 sufficiently different from zero, to concludethe residualsare not normally distributed.The Jarque-Berastatisticis given br
tt:z
(".+)
where N is the sample size, S is skewness,and K is kurtosis. Thus, large values of the skewness,and/orvaluesofkurtosis quite different from 3, will lead to a largevalue ofthe Jarque-Berastatistic.When the residualsare normally distributed,the Jarque-Berastatistic has a chi-squareddistribution with two degreesof freedom. We reject the hypothesis of normally distributed errors if a calculated value of the statistic exceeds a critical value selectedfrom the chi-squareddistribution with 2 degreesof freedom. The 5Vocritical value fromaX2-distributionwithtwodegreesoffreedomis5.gg,andthe IVocnticalvalueisg.2l. Applying these ideas to the food expenditure example, we have
,u:T(-o.onr' .*;-)
:0.063
Because0.063 < 5.99 thereis insufficientevidencefrom the residualsto concludethat the normal distribution assumptionis unreasonableat the 5Volevel of significance.The same conclusion could have been reachedby examining the p-value. The p-value appearsin the EViews output describedas "Probability." Thus, we also fail to reject the null hypothesison the groundsthat 0.9688 > 0.05. 4.3.5
ICt
.rh
.ia I I rt!
ANoruEn Eruprnrcar Exauprr
How doesone choosethe besttransformationfor y andx, and hencethe bestfunctional form for describing the relationship between y and x. Unfortunately there are no hard and fast rules that will work for all situations.The best we can do is give examplesand describeall the issuesthat are worth considering. So far, we have focused on theoretical issues,with questionslike: Does a selectedfunction possessthe theoretical propertiesconsidered desirablein the economic relationship being examined?We also mentionedthat we should choosea function suchthat the propertiesof the simple regressionmodel hold. More will be said on thesepoints as we move through the text. In this sectionwe turn to some empirical issuesrelated to the choice of functional form. Figure 4.7 describesa plot of averagewheat yield (in tonnes per hectare-a hectare i: about 2.5 acres,and a tonne is a metric ton that is 1000 kg or 2205 lb-we are speaking Australian here!) for the Greenough Shire in Western Australia, against time. The observations are for the period 1950_1997, and time is measured using the values l. 2, . . . ,48. Thesedatacanbe found in the filewa-wheat.dat.Noticein Figure4.7 that wheat yield fluctuatesquite a bit, but, overall, it tendsto increaseover time, and the increaseis at an increasing rate, particularly toward the end of the time period. An increase in yield is expectedbecauseof technologicalimprovements,suchas the developmentof varietiesof wheat that are higher yielding and more resistantto pestsand diseases.Supposethat we are interestedin measuringthe effect of technologicalimprovementon yield. Direct data on changesin technologyarenot available,but we can examinehow wheatyield haschanged over time as a consequenceof changing technology.The equationof interestrelates YIELD to TIME, where TIME - 1, . . . , 48. One problem with the linear equation
YIELU: 9r * B2TIMEI- e1
.te ,rD __ r
. -':tt '-'r
]S
+.3 MODELING
ISSUES
91
)nt from zeto,to istic is given by
e values of the rge value of the e-Bera statistic a hypothesisof a critical value 7ocritical value al valueis 9.21.
Time
rncludethat the rnce.The same : appearsin the I hypothesison
nt I . 7
Scatter plot of wheat yield over time.
::.rt it implies that yield increasesat the sameconstantrcte 82, when, from Figure 4.7, . .\pect this rate to be increasing.The least squaresfitted line (standarderrors in :intheses)is
Rz:0.64s
fiEID,:0.638+o.o21orIMEt (s") (0.064)(0.0022) unctionalform r hard and fast nd describeall al issues,with ies considered that we should J.More will be ome empirical
- -' titted valuesfrom this regression
lftEIO,T,along with the actual valuesfor YIELD, are .rl.rvedin the upperpart of Figure4.8. The lower part of the graphdisplaysthe residuals, '::eredaroundzero.The valueson the x-axis representthe years 1950-1997.Notice that
-.:!'is a concentrationof positiveresidualsat eachend of the sampleanda concentrationof 2'5 yierd - - ---
:-a hectareis : are speaking rst time. The the values 1, 4.7 thatwheat rcreaseis at an rsein yield is of varietiesof )sethat we are )irect data on d haschanged relates YIELD
Residual Actual Fitted
rr\ i I v:.
i\f'
,:i::/Vi:-i-"',V-'tr
J
2.0 1.5 i.0 0.5 0.0
-1.0 75 80 85 90 95 70 55 60 65 50 , t - p t 4 . 8 Predicted,actual,and residualvalue. tiom linear equation.
92
PREDICTION,
GOODNESS-OF_FIT. A\D
MODELING
ISSUES
0.6 0.4
0.0
-0.2
-0.4
-o.6
FTGURE 4.9
50 55 60 65 70 7s 80 Bar chartof residuals from linearequation.
85
90
9s
negativeresidualsin the middle. The bar chart in Figure 4.9 makestheseconcentrationseven more apparent.They are causedby the inability ofa straightline to capturethe fact that yield is increasingat an increasingrate. What alternativecan we try? Two possibilities arex2 and x3. It tums out that x3 provides the better fit, and so we consider insteadthe functional form
YIEL\: gr * gzTIME? + e, The slopeof the expectedyield function is 392TIMEZ. So,providing the estimateof B2turns out to be positive, the function will be increasing. Furthermore, the slope is increasing as well. Thus the function itself is "increasing at an increasing rate." Before estimating the cubic equation note that the values of TIME3 can get very large. This variable is a good candidate for scaling. If we defirc TIMECUBET : TIME? 11.000.000the esrimated equationis
VIEID,:0.8i4 + 9.68TIMECUBET R z: 0 . 7 5 1 (s") (0.036)(0.082) The fitted, actual, and residual valuesfrom this equationappearin Figure 4.10. Notice hou the predicted(fitted) valuesarenow increasingat an increasingrate. Also, the predominance of positive residuals at the ends and negative residuals in the middle no longer exists. Furthermore, the R2-valuehas increasedfrom 0.649 to 0.751, indicating that the equation withTIMECUBE fits the databetterthanthe onewith just TIME.Boththese equationshave the samedependentvariable and the samenumber of explanatoryvariables(only 1). In these circumstancesthe R2 can be used legitimately to compare goodnessof fit. What lesson: have we learnedfrom this example?First, a plot of the original dependentvariable seriesl against the explanatory variable "r is a useful starting point for deciding on a functional form in a simple regressionmodel. Secondly,examininga plot of the residualsis a useful deviceforuncovering inadequacies in any chosenfunctionalform. Runs ofpositive and/or negativeresidualscan suggestan alternative.In this example,with time-seriesdata,plotting theresidualsagainsttime wasinformative.With cross-sectional datausingplotsof residuals againstboth independentand dependentvariablesis recommended.Ideally we will seeno
t1
+.+ LOG_LINEAR
MODELS
93
2.5 Yield
---'*::* ----
i>:
Fitted I
,. /\^-4 \t)L!t+to-{-t i - t .;7/L_ /)i+ ;i! +:Lv*Jer^-
l.\
2.O 1.5
'l-
1.0 0.5 0.0
0.0 4.2
-0.4 -0.6 50 , t p.a 4.7 0
ronseven thatyield rex2 and >nalform
55 60 65 70 75 80 85 90 95 Fitted, actual, and residual values from equation with cubic term.
-:ir!rlS, andthe residualhistogramandJarque-Beratestwill not rule out the assumptionof ' :mality.As we travelthroughthe book, you will discoverthat patternsin theresidualscan - ..r mean many other specificationinadequacies,such as omitted variables,heteroske:..ticity, and autocorrelation.Thus, as you becomemore knowledgeableand experienced, : should be careful to consider other options. For example, wheat yield in Western '-.-.tralia is heavily influencedby rainfall. Inclusion of a rainfall variable might be an option . ,rrh considering.Also, it makes senseto include TIME and TIMEz in addition to TIME-red. A further possibility is the constantgrowth rate model that we considerin the next
,f B2turns ncreasing stimating iable is a estimated
1.1 Log-Linear Models
otice how ominance 1erexists. : equation tionshave ). In these at lessons le series-r' functional is a useful ive and/or a, plotting fresiduals vill seeno
-,rnometric models that employ natural logarithms are very common. In this section we .;ussthis topic furtherandgive someexamples.You shouldreviewAppendixA.2.4 where ..rc propertiesof logarithmsareintroduced,andAppendicesA.4.4-A.4.6 and Section4.3 ^r'reregressionmodelsinvolving logarithmictransformationsof variablesare discussed. garithmic transformationsare often used for variablesthat are monetary values, such as tr-{es,salaries,income, prices,sales,and expenditures,and in generalfor variablesthat :'rsure the "size" of something.These variableshave the characteristicthat they are -, .rtive and often havedistributionsthat arepositivelyskewed,with a long tail to the right. ' :ure B.2 in Appendix B is representative of the income distributionin the United States. - iact the probability density function /(r) shown is called the "log-normal," because - r ) hasa normal distribution.Becausethe transformationln(x) hasthe effect of making .:-cervalues of .r less extreme, ln(x) will often be closer to a normal distribution for .rables of this kind. The log-normal distributionis discussedin Appendix 4C. The basicinterpretationof B2 in the log-linearmodel ln(y) : Fr f B2,ris discussedin .rpendix A.4, with the slopeandelasticitygiven in TableA.2. The slopefor this functional rn is B2y. Since y > 0, the slope is increasin-eas -r' increasesif B2 ) 0, so that the .'..rtionshipbetweeny and x is said to be "increasing at an increasingrate." To make a - .!'ussionrelevantin a specificcontextthe slopecan be evaluatedat the samplemeany, or
.i- tlOn.
94
PREDICTION.
GOODNESS-OF-FIT..{NDMODELINGISSUES
anotherinterestingvalue.The elasticityof I'with respectto x is B2x,and for specificitywe might computethis value at the mean of the x-values,-, or some other value. An easier interpretationcan be obtained by using the propertiesof logarithms. See equation (A. l3) and the discussionsurroundingit. In the log-linear model, a one-unit increasein r leads,approximately,to a 100B2%changein y. 4.4.1
A Gnowru Moonr
Earlier in this chapter,in Section4.3.5, we consideredan empirical examplein which the production of wheat was tracked over time, with improvements in technology leading to wheat production increasing at an increasing rate. Another way to representsuch a relationshipis using a log-linear model. To see how, supposethat due to advancesin technologythe yield of wheat produced(tonnesper hectare)is growing at approximatell a constant rate per year. Specifically, supposethat the yield in year t is YIELDT: 0 + )YIELD,-1, with g being the fixed growth rate in I year.By substitutingrepeatedll we obtain YIELDT: YIELDo(I * g)'. Here YIELDois the yield in year "0," the yearbefore the samplebegins,so it is probably unknown. Taking logarithmswe obtain
: rn(YIELDo) Ln(YIELD,) + [tn1t+ s)]xr :Br*92/ This is simply a log-linear model with dependentvariable In(YIELD,) and explanatory variable/, or time. We expectgrowth to be positive,so that 9z > O,in which casethe plot of YIELD againsttime looks like the upward slopingcurve in Appendix A, FigureA.3, which closely resemblesthe scatterdiagramin Figure 4.7. Estimating the log-linear model for yield we obtain
II(YIELD'): -0.3434 * 0.01781 (o.os84)(o.oo2l) (se.) T h e e s t i m a t e d c o e f f i c i e n t b 2l n: ( l + S ) : 0 . 0 l T 8 . U s i n g t h e p r o p e r t y t h a t l n (*1x ) ! x i f x is small [seeAppendix A, equation(A.l l) and the discussionfollowing it], we estimate that the growth rate in wheatyield is approximatelyg : 0.0178,or about l.78Vaper year. over the period of the data. 4.4.2
AVacp
EquerroN
The relationshipbetweenwagesandeducationis a key relationshipin laboreconomics(and no doubt in your mind). Supposethat the rate of return to an extra year of education is a constantr. That is, in the first yearafter an additionalyearof educationyour wagerate rises : (l + r)WAGEs.Foranextra2yearsof fromaninitialvalueWAGEstoW-AGE1 education this becomesWAGEy: (1 + r)"WAGEs, and so on. Takinglogarithmswe havea relationship betweenln(WAGE) and yearsof education(EDUC)
tn(wAGE): tn(WAGEo) + lln(I + r)lxEDUC : gr * \TEDUC l00\z% increase in wages. An additionalyearof education leadsto an approximate
+.] LOG_LINEARMODELS
UES
I for specificity we r value. lf logarithms. See model, a one-unit
95
-hta on hourly wages,yearsof educationand other variablesarein the file cps_small.dat. -':.e dataconsistof 1000observationsfrom the 1997CurrentPopulationSurvey(CPS). -'^: CPSis a monthly surveyof about50,000householdsconductedin the United Statesby :-,:Bureauof the Censusfor the Bureauof Labor Statistics.The surveyhasbeenconducted - :rore than 50 years.Using this data the estimatedlog-linear model is
: 0.7884 x EDUC ln(WAGE) + 0.1038 ('") (0.084e) (0.0063) Lmplein which the rnology leading to represent such a ue to advancesin ; at approximatelr ar t is YIELDT: tituting repeatedlr 0," the yearbefore btain
) and explanatorl ich casethe plot of FigureA.3, which
-'.timatethat an additionalyear of educationincreasesthe wage rate by approximately : ; . A 957ointerval estimatefor the value of an additional year of education is 9. 17oto
' : -j
rN THE Loc-LrNran
PRrnrcrroN
Moonr
- :nav have noticed that when reporting regressionresults in this section we did not -. -Je an R2-value.In a log-linear regressionthe R2-valueautomaticallyreported by - .iical softwareis the percentof the variationin ln(y) explainedby the model.However, rriectiveis to explain the variations in y, not ln(y). Furthermore, the fitted regression ' : f redictsln(y) : bt * bzx, whereaswe want to predict y. The problemsof obtaining . --.:iul measureofgoodness-of-fit and prediction are connected,as we discussedin ' . ' " ' . ' . n1 . 2 . 2 . :i ,rr shall we obtain the predicted value of y? A first inclination might be to take the 'r of ffi : bt l b2x.Forthenaturallogarithmthe antilogis the exponentialfunction, . . :Jt a naturalchoice for prediction is y, : exp(ln(y)): exp(br * b2x) ^.:lo,g-linearmodelthis is not necessarilythebestwe cando. Using propertiesof the log--::rl distributionit can be shown (seeAppendix 4C) that an alternativepredictor is
ratln(l*x)ryxii rg itl, we estimate fi l.78Voper year.
rr economics(and of educationis a 'ur wageraterise: rearsofeducation 'e havea relation-
y, : E6:
exp(br I bzx -t
"'
12): jns"'lz
- i .ample sizeis large,the "corrected" predictor!. is, on average,closerto the actualvalue .rd shouldbe used.In small samples(lessthan30) the "natural" predictormay actuallybe . ^r-'iir'rchoice.The reasonfor this incongruousresult is that the estimatedvalue of the error ---'.rce 62 adds a certain amount of "noise" when using j'., leading it to have increased -.-.r.ility relative to j,, that can outweigh the benefit of the correctionin small samples. . re effect of the correction can be illustrated using the wage equation. What would we -":::;t the wage to be for a worker with 12 years of education?The predictedvalue of . .i {GE) is I"(WAG\:
0 . 7 8 8 4+ 0 . 1 0 3 8x E D U C - 0 . 7 8 8 4+ 0 . 1 0 3 8x 1 2 : 2 . 0 3 3 5
-"::
the value of the natural predictor is j;, - exp(inS): exp(2.0335): 7.6408.The : . -r' of the correctedpredictor,using o' 0.2402 from the regressionoutput, is
Lsein wages.
fc
-
E6 :
j t , g 6 ' 1 :2 7 . 6 4 0 8x 1. 1 2 7 6 : 8 . 6 1 6 1
GOODNESS_OF_FIT, AND MODELING
PREDICTION,
ISSUES
We predict that the wage for a worker with 12 yearsof educationwill be $7.64per hour if we usethe natural predictor, and $8.62 ifwe usethe correctedpredictor. In this casethe sample is large (N: 1000) so we would usethe correctedpredictor. Among the 1000workers there are 379 with 12 yearsofeducation. Their averagewage is $8.30,so the correctedpredictor is consistentwith the sample of data. How doesthe correction affect our prediction? Recall that d" must be greater than zero ar.rde0 :1. Therefore the effect of the correction is always to increase the value of the prediction because,&/2 i" always greaterthan 1. The natural predictor tends to systematically under-predict the value of y in a log-linear model, and the correction offsets the downward bias in large samples. 4.4.4
A GElueRAr-rzrioR2 MEasrrr,r
It is a generalrule that the squaredsimple correlationbetweeny and its fitted value!, wherei is the "best" prediction one can obtain, is a valid measureof goodness-of-fitthat we can use as an R2 in many contexts.As we have seen,what we may considerthe "best" predictor can changedependingupon the model under consideration.That is, a general goodness-of-fit measure,or general R2 is
y)]' : f, Rl : [con(y, In the wage equationR! : kon(y, y.) 12: 0.47392 : 0.2246, as comparedto the reported R" : 0.2146 from the regressionof ln(WAGE) on EDUC. (In this casesince the corrected and natural predictors differ only by a constantfactor, the correlation is the samefor both.) These R2-valuesare small, but we repeat our earlier message:R2-valuestend to be small the variations in individual behavior are with microeconomic, cross-sectionaldata,because difficult to fully explain. 4.4.5
INrunvALs rN THE L,oc-LrNnen MooEr
PnEorcrrou
We have a correctedpredictor j'" for y in the log-linear model. It is the "point" predictor, or point forecast,that is relevant if we seekthe single number that is our best prediction of ,r'. If we prefer a prediction or forecastinterval for y, then we must rely on the natural predictor j,,.1 Specifically we follow the procedure outlined in Section 4.1, and then take antilogs. That is, compute ln(y) : h I bzx and then ln(y)+ t"se(f), where the critical value /. is the 100(1-cf2)-percentrle from the r-distribution and se(/) is given in (4.5). Then a 100(l-a)Vo prediction interval for y is f
/+
lexp(tn(y)-
.\
/+
.\l
t,se(f)), exp(ln(y) + t.se(/),)]
For the wage data, a 95Vo prediction interval for the wage of a worker with 12 years of education is [exp(2.0335 1.96 x 0.4905),exp(2.0335+ 1.96 x 0.a905)]: 12.9184,20.00461
' SeeAppendix 4A. The correctedpredictor includes the estimatederror variance,making the r-distribution no longer relevant in (4A.1).
Ibc .lEs tlir
rfii
1.i
.Las
{-5.
ES
64 per hour if we s casethe sample 00 workers there octedpredictor is greaterthan zero the value of the tends to system:ction offsets the
4.5 EXERCISES
The interval predictionis $2.92-$20.00,which is so wide that ir is basicallyuseless.Whar doesthis tell us?Nothing we did not alreadyknow. Our model is not an accuratepredictor of individualbehaviorinthis case.In laterchapterswe will seeif we canimprovethismodelby adding additional explanatory variables, such as experience,that should be relevant.
1.5 Exercises Answer to exercisesmarked * appearin Appendix D at the end of the book. {.5.1
I valuej,, wherej' it that we canuse st" predictor can I goodness-of-fit
:d to the reported rce the corrected e samefor both.) tend to be small iual behavior are
int" predictor, or rt prediction of y. natural predictor en take antilogs. ritical value /. is in (4.5). Then a
with 12 yearsof
184,20.0046]
3 the r-distribution no
97
PRonrnns
J.1* (a) Suppose that a simple regressionhas quanrities |'(yr -t)r:631.63 and 2a! :182'85' find R2' (b) Suppose that a simple regression has quantities N : 20, Zy? : 5930.94, , :16.035, and SSR:666.72, find R2. (c) supposethatasimpleregressionhasquantitiesR2:0.79ll,ssz:552.36,and N:20, find6'. -1.2* consider the following estimated regression equation (standarderrors in parentheses): y : 5 . 8 3* 0 . 8 6 9 r R z : 0 3 5 6 ( s e )( 1 . 2 3 ) ( 0 . 1 1 7 ) Rewrite the estimated equation that would result if (a) All values of x were divided by 10 before estimation. (b) Al1 values of y were divided by 10 before esrimation. (c) All values of y and r were divided by l0 before estimation. 1.3 Using the data in Exercise 2.r and only a calculator (show your work) compute (a) the predicted value of y for xe : 5. (b) the se(/) correspondingto parr (a). (c) a95Vo prediction interval for y given ro : 5. (d) a 99Voprediction interval for y given ro : 5. (e) a95%oprediction interval fory givenx : 7. Comparethe width of this interval to the one computed in part (c). -1.-l Giventhesimplelinearmodely : B, l gzx I e,andtheleastsquaresestimators,we can estimate E(y) for any value of x : xs asE(ys) : bt I bzxo. (a) Describe the difference between predicting )s and estimating Z(ye). (b) Find the expectedvalue and variance otffi; : h I bzxo. (c) When discussing the unbiasednessof the least squarespredictor we showed that E(f): E(yo -lo) :0, where/is rhe forecasterror. Why did we define unbiasednessin this strangeway? what is wrong with saying,aswe havein other unbiasednessdemonstrations,that E(yg) : ys? 1.5 Supposeyou are estimatinga simple linear regressionmodel. (a) If you multiply all the x values by 10, but not they values, what happensto the parametervaluesB1 and B2?what happensto the leastsquaresestimatesb1 and b2? What happensto the variance of the error term? (b) Supposeyou are estimating a simple linear regressionmodel. If you multiply all they valuesby 10,but not the.r values.what happensto the parametervalues B1
98
PREDICTION,
GOODNESS-OF_FIT, AND MODELING
ISSUES
and B2?What happensto the least squaresestimatesbl andb2?What happensto the variance of the error term?
4.6 The fitted least squaresline is !; : bt * bzxi. (a) Algebraically, show that the fitted line passesthrough the point of the means.
(x,r).
(b) Algebraically show_thatthe averagevalue of j; equals the sampleaverageof -r'. That is, show that i : t, wherey:}yilN. 4.7 In a simple linear regressionmodel supposewe know that the intercept parameteris zero, so the model is y; : B2x;+ er.The least squaresestimatorof P2is developedin Exercise 2.4. (a) What is the least squarespredictor of y in this case?^ (b) When an intercept is not present in a model, R' is often defined to be R?,: | - SSE12y7, where SSEis the usual sum of squaredresiduals.Compute Rl for the data in Exercise 2.4. (c) Compare the value of Rl in part (b) to the generalizedRz : fi., where ! is the predictor based on the restricted model in part (a).^ (d) Compute SSZ : I(y' - y)2 and SSR: I(y, - y)2, where ! is the predictor basedon the restrictedmodel in part (a). Does the sum of squaresdecomposition SSI: SSR+ SSEhold in this case? 4.5.2 4.8
CovrpursR. ExnncrsEs The first three columns in the filewa-wheat.dat containobservationson wheat yield in the Western Australian shires Northampton, Chapman Valley, and Mullewa. respectively. There are 48 annual observationsfor the years 1950-1997. For the Chapman Valley shire, consider the three equations }r:Bol9'/let )r:cto+c-lr'(t)+et )r:^Yo+^yiz+e, (a) Estimate each of the three equations. (b) Taking into consideration (i) plots of the fitted equations, (ii) plots of the residuals, (iii) enor normality tests, and (iv) values for R2, which equation do you think is preferable? Explain.
4.9* For each of the three functions in Exercise 4.8 (a) Find the predicted value for yield when t : 49. (b) Find estimatesof the slopesdy,f dt at the point t : 49. (c) Find estimatesof the elasticities (dy,ldt)(tly,) at the point t : 49. (d) Comment on the estimates you obtained in parts (b) and (c). What is their importance? 4.I0
The file london.datrs a crosssectionof 1519householdsdrawnfrom the 1980-1981 British Family Expenditure Surveys. Data have been selected to include onlr households with one or two children living in Greater London. Self-employed and retired households have been excluded. Variable definitions are in the file london.def. The budget shareof a commodity, say food, is defined as
WFOOD:
expenditure on food total expenditure
4.5 EXERCISES
happensto
99
A functional form that has been popular for estimating expenditure functions for commodities is WFooD:
the means,
Fr * B2rn(TOTEXp)_t e
(a) Estimate this function for householdswith one child and householdswith two children. Report and comment on the results. (You may find it more convenient to use the files lonl.dat and lon2.dat that contain the data for the one and two children households,with 594 and 925 observations,respectively.) (b) It can be shown that the expenditure elasticity for food is given by
rerageofy. arameteris lveloped in
Br+9zfh(rorExP)+Il Fr * Fz1n(ZOTEXP)
Lned to be r. Compute
Find estimatesof this elasticity for one and two children households,evaluatedat average total expenditure in each case. Do these estimates suggest food is a luxury or a necessity?(Hint: Are the elasticities greater than I or less than 1?) (c) Analyze the residuals from each estimatedfunction. Does the functional form seemappropriate?Is it reasonableto assumethe errors are normally distributed?
ere ! is the a predictor omposition
vheat yield I Mullewa, )7. For the
lots of the :quation do
hat is their 1980-1982 clude only [-employed in the file
-1.I 1x Reconsiderthe presidentialvoting data(faindat)introducedin Exercises2. I 4 and3.9. (a) Using all the data from 1880 to 2000, estimate the regression model VOTE : fu t 9zGROWH + e. Basedon theseestimates,what is the predicted value of VOTE in 2000? What is the least squaresresidual for the 2000 election observation? (b) Estimate the regressionin (a) using only data up to 1996 (observations1-30). Predict the value of VOTE in 2000 using the actual value of GROWTH fot 2000, which was 1.603Vo.What is the prediction error in this forecast?Is it larger or smaller than the error computed in part (b). (c) Using the regressionresults from (b), construct a95Vo interyal estimate for the 2000 value of VOTE using the actual value of GROWTH : 1.603Vo. (d) Using the estimationresults in (b), what value of GROI4TII would have led to a prediction that the nonincumbentparty (Democrats) would have won 50.17oof the vote? :.12
Considerthe housing startsdata (house_starts.dat)introdtced in Exercises2.13 and 3.8. The data extendto April 2005. (a) The fixed interest rates for May and June 2005 were 6.00Vo and 5.82Vo, respectively.Predict the number of housing starts in May and June 2005. (b) The actual number of housing startsin May and June 2005 were 2041 and 2065 (thousands),respectively. (Source: Economagic.com.) How large were your prediction errors? (c) Construct 95Vo prediction intervals for the number of housing starts in May and June 2005, based on the sample data. Did your intervals contain the true values?
:. l3* The file stockton2.dalcontainsdataon 880housessold in Stockton,CA, duringmid2005. Variable descriptionsare in the file stockton2.def.Thesedata were considered in Exercises2.12 and3.ll . (a) Estimate the log-linear model Ln(PRICE): Fr * ?zSQFT * e. Interpret the estimated model parameters.Calculate the slope and elasticity at the sample means,if necessary.
PREDICTION,
GOODNESS-OF-FIT, AND MODELING
ISSUES
(b) Estimate the log-log model ln(PRICE) : Br + Fzln(SQFf) + e. Interpret the estimatedparameters.Calculate the slope and elasticity at the samplemeans,if necessary. (c) Comparethe R2-valuefrom the linear modelPRICE : Fr * gzSQFT * e to the "generalized" R2 measurefor the models in (b) and (c). (d) Construct histograms of the least squaresresidualsfrom each of the models in (a), (b), and (c) and obtain the Jarque-Bera statistics. Based on your observations, do you consider the distributions of the residualsto be compatible with an assumptionof normality? (e) For eachof the models (a)-(c) plot the least squaresresidualsagainstSQFT.Do you observe any patterns? (f) For each model in (a)-(c), predict the value of a house with 2700 squarefeet. (g) For eachmodel in (a)-(c), construct a95%optediction interval for the value of a house with 2700 squarefeet. (h) Based on your work in this problem, discuss the choice of functional form. Which functional form would you use? Explain. 4.14 How much does education affect wage rates?This question will explore the issue further. The datafile cps_small.dafcontains 1000observationson hourly wage rates. education, and other variables from the 1997 Current Population Survey (CPS). (a) Constructhistogramsof thewAcBvariable and its logaritbn ln(WAGE). Which appearsmore normally distributed? arrd log-linear (b) Estimate the linear regression WAGE:9r * !2EDUCle regressionLI(WAGE):9r * P4EDUC *e. What is the estimatedreturn to education in each model? That is, for an additional year of education, what percentageincreasein wages can the averageworker expect? (c) Construct histograms of the residuals from the linear and log-linear models in (b), and the Jarque-Beratestfor normality. Does one setof residualsappearmore compatible with normality than the other? (d) Compare the Rz of the linear model to the "generalized" R" for the logJinear model. Which model fits the data better? (e) Plot the least squaresresidualsfrom eachmodel againstEDUC. Do you observe any patterns? (f) Using each model, predict the wage of a worker with 16 years of education. Comparethesepredictionsto the actual averagewage of all workers in the sample with 16 years of education. (g) Based on the results in parts (a)-(0, which functional form would you use? Explain. 4.15 Does the return to education differ by race and gender?For this exerciseuse the file cps.dat. (This is alatge file with 4733 observations.If your software is a student version, you can use the smaller file cps-small.dat.)In this exerciseyou will extract subsamplesof observationsconsistingof (i) all males,(ii) all females,(iii) all whites. (iv) all blacks, (v) white males, (vi) white females,(vii) black males, and (viii) black females. (a) For each sample partition, obtain the summary statistics of WAGE. (b) Avariable's cofficient of variation is 100 times the ratio of its sample standard deviation to its sample mean. For a variable y it is (T..
C V : 1 0 0 x;1 t
Ap tL flu rl;a
APPENDIX
4A DEVELOPMENT
OF A PREDICTION
INTERVAL
101
It is a measureof variation that takesinto accountthe size of the variable.What is the coefficient of variation for WAGE within each sample partition? (c) For each sample partition estimate the log-linear model
rterpret the e means,if f*etothe
ln(WAGE): : models in /our obserratible with
What is the approximatepercentagereturn to anotheryear of educationfor each group? (d) Does the model fit the data equally well for each sample partition? (e) For each sample partition, test the null hypothesis that the rate of return to education is lOVoagainstthe alternativethat it is not, using a two-tail test at the 5Volevel of significance.
tSQFT.Do quare feet. e value of a ional form. re the issue wage rates, :y (CPS). ?E). Which . log-linear ,d return to ation, what r models in ppear more : log-linear ,ou observe education. the sample d you use? use the file s a student will extract )all whites, (viii) black
le standard
Br * B2EDUC * e
- .6
The November 2000 U.S. Presidential Election was won by George Bush over Al Gore. In Florida, especially Palm Beach county, voting irregularities were claimed. lnthe fileflorida.dat arecountyvoting datafor candidatesBush, Gore, and two minor candidates,Pat Buchanan and Ralph Nader. (a) Estimate a simple regression using as a dependent variable the votes for Buchanan and as the explanatory variable the votes for Bush. Use the data on the 66 countiesexcluding Palm Beach.Interpret the model R2.How well doesthe model explain the Buchananvote? (b) Predict the Buchanan vote for Palm Beach county and construct a 99.97o prediction interval. Is the actual vote for Buchanan within the prediction interval? (c) Plot a scatter diagram of the actual Buchanan vote (vertical axis) against the predicted Buchanan vote (horizontal axis) using data on all 67 observations. (d) Repeatparts (a)-(c) basedon a regressionusing as dependentvariable the votes for Buchanan and as explanatory variable the votes for Gore. (e) To control for total county population, createvariablesthat are the sharesofvotes by county. That is, Buchanan'sshareof the vote is Buchanan'scounty vote divided by the total number of votes cast in the county. Estimate the simple regressionof Buchanan's share on Bush's share using data on the 66 counties excluding Palm Beach. Construct a 99.9Voprediction interval for Buchanan's shareof the vote in Palm Beach county.Multiply the endpointsof the prediction interval by the total votes cast in Palm Beach county to determine a prediction interval for Buchanan'svote. Where doesBuchanan'sactual vote in Palm Beach county fall relative to this interval?
-{ppendix 4A Development
of a Prediction Interval
-yo: l : e f o r e c a s t e r r o r /i s: ) 0 (Fr l9zxoaro) - (br*bzxo).Toobtainitsvariance : c: us first obtain the varianceofj's br * bzxo.The variancesand covarianceofthe least \:jares estimatorsare given in Section2.4.4.Using them, we obtain var(yo):var(br * b2xs): var(br) + fivar(b2) *2xscov(b1,b2)
ozb?)ol^)-, "l(x;-r)Nl(xi-x)' T4rr--4rrv
-
I(xi-t)'
PREDICTION,
GOODNESS-OF-FIT. AND MODELING
ISSUES
Now we usea trick. Add the termo2Nf f NX"r - r)i afterthe first term (insidebracesbelow) and subtractthe sameterm at the end. Then combine the terms in brackets,as shownbelow:
:h* var(j,s)
{d5}l
o'*i
o2q_2xox1 f "2N-'2 I I -
T rI- T - - r - _ r l , LGi -7)' [I(x; r)'
[NXxi
7)' t t
,l >l - xt' xf - 2xst+x2] ZGi-7)' I LN:(xi t)' 'l , I I(x, - r)2 ("0- r)t lNl(x; x)' I(x; x)'l
-vr-T-l
-vr-T-l
-" c l r ( x 0 - j r J I LN-tG-;Jzl Taking into account that xs and the unknown parametersB 1 and B2 are not random, you shouldbe able to show that var(/) : var(yo) * var(ee) : var(io) + o2. A little factorin-s gives the result in (4.4). We can construct a standardnormal random variable as f
ffi-*{o''1, If theforecasterrorvariancein (4.4)is estimated by replacingo2 by its estimator62, (t x" 'n - x ) 2 . l -lt "'=' i l d / 1: a 2 l t + - - ; -;;21 L " I(xr then
(4A.1r
where the squareroot ofthe estimatedvarianceis the standarderror ofthe forecastgiven in
(4.s)
Using theseresults we can construct an interval prediction procedurefor y6 just as we constructedconfidenceintervals for the parametersB1.If r. is a critical value from the /1,v-: distribution such that P(t> tc) : al2, then P(-t,
(44.2t
Substitutethe /-randomvariablefrom (4A.1) in to (44.2) to obtain
Pl-t,
L
--
_:
l
I -o
UES
APPENDIX 4C THE LOG_NORMAL
ide bracesbelow) as shownbelow:
DISTRIBUTION
Simplify this expressionto obtain Pl)o -r.se(/)
( yo ( lo + tse("f)l : 1 - cr
A 100(1-cr)7oconfidenceinterval, or predictioninterval, for ye is given by (4.6).
Appendix 48 The Surn of Squares Decomposition To obtain the sum of squaresdecompositionin (4.I 1), we squareboth sidesof (4.10)
(v,-r)':
[(vr- i +a,]': (yi- fl'+a? +2(it -y)ai
Then sum
I(y, - y)2:2(j', - y)' +Za?+22(yt--iAt Expanding the last term we obtain tot random, you \ little factoring
I(vr - y)Oi :2yi2i - tl,at :L(bt + b2x)?; - yZOi : br\?i I bzZxi/i - y22i Consider first the term22i Z e i : 2 1 r . - b 1 - b 2 x i ): Z y i
estimator62,
(4A.1)
brecastgiven in or )0 just as we from the /11,i_21-
(44.2)
- Nh - bzbi :0
This last expressionis zero becauseof the first normal equation, (2A.3). The first normal equation is valid only if the model contains an intercept. The sum of the least squares residualsis always zero if the model contains an intercept. It follows then that the sample neanofthe leastsquaresresidualsis also zero (sinceit is the sum ofthe residualsdivided by the sample size) if the model contains an intercept. That is, 2 : \A,lN : 0. The next termZxi|i: 0 because Zxi?;:Zr,(y,
- b 1 - b 2 x ; ): } x i y i
- btZxt - bz\*? : O
This result follows from the secondnormal equation, (2A.4). This result always holds for the least squaresestimator and does not depend on the model having an intercept. See Appendix 2A for discussionof the normal equations.substituting 22i : o and\x;Oi : e back into the original equation,we obtain I(i, - y)Oi :0. Thus if the model contains an intercept it is guaranteedthat SSZ : SSR + SSE. Il however,the model does not contain an intercept, then IZi l0 and SS?"+ SSR + SSE.
Appendix 4C The Log-Normal Distribution Supposethat the variabley has a normal distribution, with mean p and varianceo2. By now vou are familiar with this bell-shapeddistribution. If we consider w: ev, then y: ln(w) *N(p, o2) and w is said to have a log-normal distribution.The questionthen is, rvhatare the mean and varianceofw? Recall that the "expected value ofa sum is the sum of
104
PREDICTION,
GOODNESS-OF-FIT,
AND MODELING
ISSUES
the expectedvalues."But, unfortunately,the exponentialfunctionis nonlinear,and the expectedvalueof nonlinearfunctionof y is not just the samefunctionof E(y). That is, if SO) is some tunction of 1 then in generalEtg$)l * SIEO)1.So the expectation E(w): E(et)fsE\t). Happily the expectedvalue and varianceof w havebeenworked out and are E(') :
"*+a"/2
Ch
T
N
and var(w) :
(""' - t) "2tr+o2
Theseresultsrelate to the log-linear regressionmodel in severalways. First, given the loglinear model ln(y): Br * $zx * e,if we assumethat e-N(0,o2), then E(yr):
-- E(etu+Fzne"i): ePr+\zr'E("",): s1rtlzx;"a2/2 - elt+P2xi+*12 E(e1r+92*t+a1
kar ibed l.
Consequently,if we want to predict E(y) we should use E6
: ebrtbzxi+6212
:.
where b1, bz, and 62 are from the log-linear regression. The second implication comes from the growth and wage equations discussed in Section 4.4. For example, in the wage equation we estimated 9z : ln(l * r). Solving for r we obtain r - e9, - 1. If assumptionSR6 holds, then the least squaresestimator is normally distributedb2-N(9z,var(b) : o2l\@i -;)2). Then E[eb') :
"92+vt(b)/2
Therefore, an estimator of the rate of return r is
?: 'b'+i&6p - | -t)2. whereifrD : 62121p"i
Icyr aL e cr:da bsr crtcal vi aE tFvi fEdne
]S
linear, and the 'E(y). That is. le expectation : been worked
, given the log-
=g\r'+l2'i+&12
Chapter
The Multiple Regression Model
karning
Objectives
*r.ed on the material in this chapter, you should be able to . . Recognizea multiple regressionmodel andbe ableto interpret the coefficientsin that model.
in s discussed * r). Solving es estimatoris
:
Understandand explain the meaningsof the assumptionsfor the multiple regression model.
:
Use your computer to find least squaresestimatesof the coefficients in a multiple regressionmodel. Interpret those estimates.
:
Explain the meaning of the Gauss-Markov theorem.
:.
Use your computerto obtain varianceand covarianceestimates,and standarderrors, for the estimated coefficients in a multiple regressionmodel.
6. Explain the circumstancesunder which coefficient variances(and standarderrors) are likely to be relatively high, and relatively low. Find interval estimatesfor single coefficients. Interpret the interval estimates. \.
Test hypotheses about single coefficients in a multiple regression model. In particular (a) What is the difference between a one-tail and a two-tail test? (b) How do you compute the p-value for a one-tail test, and for a two-tail test? (c) What is meant by "testing the significance of a coefficient"? (d) What is the meaning of the /-values and p-values that appearin your computer output?
9. Compute and explain the meaning of R2 in a multiple regressionmodel.
Keywords BLU estimator :r.rariance matrix of ieast squnresestimator --ntical value :ror variance estimate crror variance estimator r.,rrdness-of-fit
interval estimate least squaresestimates least squaresestimation least squaresestimators multiple regressionmodel one-tail test p-value
regressioncoefficients standarderrors sum of squarederrors sum of squaresof regression testing significance total sum of squares two-tail test
106
THE MULTIPLE REGRESSION MODEL
The model in Chapters 2-4 is called a simple regression model becausethe dependent variabley is relatedto only one explatatory variablex. While this model is useful for a range of situations, in most economic models there are two or more explanatory variables that influence the dependent variable y. For example, in a demand equation the quantity demandedof a commodity dependson the price of that commodity, the prices of substitute and complementarygoods,andincome. Output in a production function will be a function of more than one input. Aggregatemoney demandwill be a function of aggregateincome and the interest rate. Investment will depend on the interest rate and changesin income. When we turn an economic model with more than one explanatory variable into its correspondingeconometric model, we refer to it as a multiple regression model. Most of the results we developedfor the simple regressionmodel in Chapters24 can be extended naturally to this general case. There are slight changes in the interpretation of the B parameters,the degreesof freedom for the r-distribution will change,and we will need to modify the assumptionconcerningthe characteristicsof the explanatory(x) variables.These and other consequencesof extending the simple regressionmodel to a multiple regression model are describedin this chapter. As an examplefor introducing and analyzing the multiple regressionmodel, we consider a model usedto explain salesrevenuefor a fast-food hamburgerchain with outlets in small U.S. cities.
5,1 Introduction 5.1.1
9: : thecb unit (l AS AP(A
TsE EcoNourc Mooer
We will set up an economic model for a hamburgerchain that we call Big Andy's Burger Barn.' Importantdecisionsmadeby the managementof Big Andy's areits pricing policy for different products and how much to spendon advertising.To assessthe effect ofdifferent price structuresand different levels of advertisingexpenditure,Big Andy's BurgerBarn sets different prices and spendsvarying amountson advertising in different cities. Of particular interestto managementis how salesrevenuechangesasthe level of advertisingexpenditure changes.Does an increasein advertisingexpenditurelead to an increasein sales?If so,is the increasein salessufficient to justify the increasedadvertisingexpenditure?Managementis also interestedin pricing strategy.Will reducing prices lead to an increaseor decreasein salesrevenue?Ifa reductionin price leadsonly to a small increasein the quantitysold,sales revenuewill fall (demandis price inelastic); a price reduction that leadsto a large increasein quantity sold will produce an increasein revenue(demandis price elastic). This economic information is essentialfor effective management. The first step is to set up an economic model in which salesrevenuedependson one or more explanatoryvariables.We initially hypothesizethat salesrevenueis linearly relatedto price and advertising expenditure. The economic model is
S:9r-tFzp+FzA
'.: sr mbol " * .., havesee - 'ne variabl
,---r:ionally u The signof - :> revenue :r ersely,a 1 ' ,i hich case I : ..ricity of de ; .:n price ch The param -.crtising ex
9r : thech oneur AS LAel
( s.1)
where S representsmonthly salesrevenuein a given city, P representsprice in that city, andA is monthly advertising expenditure in that city. Both S and A are measuredin terms of thousandsof dollars. Because sales in bigger cities will tend to be greater than sales in smaller cities, we focus on smaller cities with comparablepopulations. '
Sincea he :.;h product :-:.:ation(5.1 - how this : . :\truCted a :--;es in eacl The remair '.=Jependen :,::,rnfetefB1 ::.:ianatoryv :, :lear econ -:,:reP:A -',i niodel,ev( -' .: tits the di The other I -:..iblegiven : : erample,
The data we use reflect a real fast-food franchise whose identity we disguise under the name Big Andy's
,\. axpectthe 't;'enditure,u :, : the increas i:ed costof p *. 00 in adve : : p - r) 1 , i t ., i\ very imp
5.l INTRODUCTION
dependent . for a range riables that re quantity rf substitute function of income and ICOme.
ble into its lel. Most of re extended ,n of the I ryill need to rbles.These : regression weconsider Letsin small
107
Since a hamburger outlet sells a number of products: burgers, fries, and shakes,and -'achproduct has its own price, it is not immediately clear what price should be used in .-quation(5.1). What we needis somekind of averageprice for all productsand information r how this average price changes from city to city. For this purpose managementhas -..nstructed a single price index P, measuredin dollars and cents, that describesoverall :rces in each city. The remaining symbols in (5.1) are the unknown parametersB1,B2,and B3that describe r:e dependenceof sales(S) on price (P) and advertising (A). Mathematically, the intercept :irameter B1 is the value of the dependent variable when each of the independent, .'rplanatory variables takes the value zero. However, in many cases this parameter has -', clear economic interpretation.In this particular caseit is not realistic to have a situation ; hereP : A : 0. Except in very special circumstances,we always include an intercept in ::.e model, even if it has no direct economic interpretation. Omitting it can lead to a model '--.atfits the data poorly and does not predict well. The other parametersin the model measurethe change in the value of the dependent , iriable given a unit changein an explanatory variable, all other variables held constant. ! rr example,in (5.1) 9: : thechangein monthly salesS ($1000)whenthepriceindexP is increasedby one unit ($1) andadvertisingexpenditureAis heldconstant
dy's Burger rg policy for of different 1erBarn sets )f particular expenditure ? If so,is the nagementis decreasein y sold,sales ; increasein is economic ls on one or ly related to
as
as
AP(eheldconstani
0P
he symbol "d" standsfor "partial differentiation." Those of you familiar with calculus -:..l haveseenthis operation.It meansthat, as the definition states,we calculatethe change :. one variable, S, when the variable P changes,all other factors,A, held constant.We will ccasionallyuse this symbol, but you will not be askedto evaluatesuch derivatives. The sign of B2could be positive or negative.If an increasein price leadsto an increasein ..les revenue,then B2 ) 0, and the demand for the chain's products is price inelastic. .-.)nversely, a price elasticdemandexistsifan increasein price leadsto a declinein revenue, : s hich caseB2 ( 0. Thus, knowledge of the sign of B2provides information on the price :..isticity of demand.The magnitude of B2measuresthe amount of changein revenuefor a ::ren price change. The parameter B3 describesthe responseof sales revenue to a change in the level of .jrertising expenditure.That is, B: : thechangein monthlysales.l($ 1000)whenadvertisingexpenditureAis increasedby oneunit ($1000)andthepriceindexP is heldconstant
as
as
M(Ph"ld"onttunt\
0A
(5.1) rtcity, andA in terms of han salesin
re Big Andy's.
.\'e expectthe sign of B3 to be positive.That is, we expectthat an increasein advertising :rpenditure,unlessthe ad is offensive,will leadto an increasein salesrevenue.Whetheror :ot the increasein revenueis sufficient tojustify the addedadvertisingexpenditure,and the .Jded costof producingmorehamburgers,is anotherquestion.With 93 < l, an increaseof SI 000 in advertisingexpenditurewill yield an increasein revenuethat is lessthan $ 1000. l.rr B3 ) l, it will be greater.Thus,in termsof the chain'sadvertisingpolicy, knowledgeof j, is very important.
THE MULTIPLE
REGRESSION MODEL
The next stepalong the road to leaming about Br, 92, and B3,is to convert the economic model into an econometric model. 5.1.2
Trn EcoNoMETRrc Moorr
lx)
The economic model (5.1) describesthe expectedor averagebehavior of many individual franchisesthat make up the complete chain run by Big Andy's Burger Barn. As such we should write it as E(^f) : Fr * 9zP -t 9zA, where E(S) is the "expected value" of sales revenue. Data for salesrevenue, price, and advertising for different cities will not follow an exactlinearrelationship.Equation(5.1)describes,not a line asin ChaptersZ4,butaplane. As illustrated in Figure 5.1, the plane intersectsthe vertical axis at B1.The parametersB2and B3measurethe slope of the plane in the directions of the "price axis" and the "advertising axis," respectively.Representativeobservationsfor salesrevenue,price, and advertisingfor somecities are displayed in Table 5.1. The complete set of observationscan be found in the file andy.dat andis representedby the dots in Figure 5.1. Thesedata do not fall exactly on a plane, but instead resemble a "cloud." To allow for a difference between observablesalesrevenue and the expectedvalue of salesrevenue,we add a random error term, e : S - E(S). This random error representsall factors,other than price and advertisingrevenue,which causesalesrevenueto differ from its expectedvalue. Thesefactors might include the weather,the behaviorof competitors,a ne\r SurgeonGeneral'sreport on the deadly effectsoffat intake, and so on, as well as differences in burger-buyingbehavioracrosscities. Denoting the observationfor city i by the subscripti. we have S; : E(S,) I ei : 9t * $zPi * }zAi I ei
= Fr+p2P+fuA E(,9) Fz = slopein P direction
9: = slopein A direction
: t
'nple
m *rdian llrr.imum lGaimum trl- Dev.
(5.2)
The economic model in (5.1) describesthe average,systematicrelationship between the variablesS, P, andA. The expectedvalue E(,f) is the nonrandom,systematiccomponent,to which we add the random efior e to determine S. Thus S is a random variable. We do not know what the value of salesrevenue will be until we observe it.
prcunp 5 .7
TtbIe ! Andy's I
The in *econo I oore rei tur devel
t.l.2a 'l L rs usefu I' the gen n a numt rrinen ar
fbe coefl raiables .riable.r trivative
fbe parar - 'is alr plane. The multipleregression
lEameter
109
5.1 INTRODUCTION
ert the economic
Ta b I e 5 , 7 Observations on Monthly Sales, Price, and Advertising in Big Andy's Burger Barn Sales (.9; $1000 units
Price (P) $l units
Advertising (A) $1000 units
5
73.2 71.8 62.4 67.4 89.3
5.69 6.49 5.63 6.22 5.02
1.3 2.9 0.8 0.7 1.5
73 74 75
75.4 81.3 75.0
5.71 5.45 6.05
O,.l 2.0 2.2
City
many individual arn. As such we value" of sales rill not follow an 24,butaplane. arameters92 and the "advertising d advertisingfor n be found in the fall exactly on a rpectedvalue of or representsall todifferfromits mpetitors,a new lll as differences y the subscripti,
I 2 3 ,|
Summarystatistics Samplemean \ledian \taximum \linimum Std. Dev.
77.37 76.50 91.20 62.40 6.4885
5.69 5.69 6.49 4.83 0.5184
1.84 1.80 3.10 0.50 0.8317
(s.2) rip between the 3 Component,to able. We do not
The introduction of the error term and assumptionsaboutits probability distribution turn '}reeconomic model into the econometric model in (5.2). The econometricmodel provides a morerealistic descriptionof the relationshipbetweenthe variablesas well as a framework tor developing and assessingestimators of the unknown parameters. 3-1.2a The General Model It is useful to digressfor a moment and summarizehow the conceptsdevelopedso far relate ro the generalcase.In a generalmultiple regressionmodel, a dependentvariabley is related :o a number of explanatory variables xz, x3,. . . ,,r5 through a linear equation that can be *ritten as
yi : 9r I \zxizI 9zxl+ "' + Fxxix* ei
(5.3)
---/e The coefficients B2,9s, . . . , B6 are unknown coefficients correspondingto the explanatory r ariablesx2,x3, . . . , xK.A single parameter,call it Pr, measuresthe effect of a changein the r ariable.q,upon the expectedvalue of y, all other variablesheld constant.In terms of partial Jerivatives,
^
YK -
Ar(y)l Lx1, loth.rt', heldconsrmt
ae(y) 0x*
The parameterB1is the intercept term. We can think of it as being attachedto a variable x1 :hat is always equal to 1. That is,.r;1 : l. We use Kto denotethe number of unknown rarametersin (5.3).
110
THE MULTIPLE REGRESSION MODEL
5
The equation for sales revenue can be viewed as a special case of (5.3) where :3, K )i : Si, xil : I, xiz : Pr,andxs:Ai. Thus we rewrite(5.2) as Yi : 9r l9zxiz * 9zxit -l ei
(5 4\
In this chapter we will introduce point and interval estimation in terms of this model with K : 3. The resultsgenerally will hold for modelswith more explanatoryvariables(rK > 3). 5.1.2b The Assurnptions of the Model To make the econometric model in (5.4) complete, assumptions about the probability distribution of the random errorserneedto be made.The assumptionsthat we introducefor e; are similar to those introduced for the simple regressionmodel in Chapter 2. They arc 1 . E(e;) :0. Each randomerror has a probability distributionwith zero mean.Some errors will be positive, some will be negative; over a large number of observations, they will averageout to zero.
2 . var(.ei): o2. Each random error hasa probability distribution with varianceo2. The varianceo2 is an unknown parameterand it measuresthe uncertaintyin the statistical model. It is the samefor eachobservation,so that for no observationswill the model uncertaintybe more, or less,nor is it directly relatedto any economicvariable. Errors with this property are said to be homoskedastic. 3. cov(ei,e) : 0. The covariancebetweenthe two random errorscoffespondingto any two different observationsis zero. The size of an error for one observationhas no bearing on the likely size of an error for anotherobservation.Thus, any pair of errors is uncorrelated.
r ariable.This i .rndadvertisin :asesin which ,f the explana :. taken up fur The second : unctionofthe -\swe will see .quaresprocec To summar ::rodelin (5.3) ASSUMPl MRl. Ii: MR2. E(y
MR3. var(
MR4. cov
MR5. The exp MR6. Ji-
4. We will sometimesfurther assumethat the random errors er havenormal probability d i s t r i b u t i o n sT.h a t i s , e i - N ( 0 , o 2 ) .
-'-.2 Estin Regressi
Becauseeachobservationon the dependentvariabley; dependson the random error term er, each y; is also a random variable. The statistical properties ofy; follow from those of e;. These properties are
.- this section .rknown para - ,ntextof the
1. E0):9t I $2xi2* F:xr. The expected(average)value of y; dependson the values of the explanatoryvariables and the unknown parameters.It is equivalentto E(e;):0. This assumptionsays that the averagevalue of yr changesfor each observationand is given by the regression function E(l) : 9t i \zxiz * p:xi:.
. ris modelis s ,-'neralcasew
2. vtrCyi) :var(e): o2. The varianceof the probability distribution of y; does not change with each observation. Some observationson ); are not more likely to be further from the regressionfunction than others. 3. cov(y;,y;) : cov(ei,e) :0. Any two observationson the dependentvariable are uncorrelated.For example,if one observationis aboveE(y;), a subsequentobservation is not more or less likely to be aboveE(y;). 4. We sometimeswill assumethat the values of y; are normally distributed about their mean.That is, yr - N[(Br * gzxiz* B:xi:). o21.which is equivalentto assumingthat ei - N(0, oz). In addition to the above assumptionsabout the error term (and hence about the dependent variable), we make two assumptionsabout the explanatory variables. The first is that the explanatoryvariablesare not random variables.Thus we are assumingthat the valuesofthe explanatory variables are known to us prior to observing the values of the dependent
i.1.1
LEes
. find an esti ':,,cedurethat -'.r\tSqUareS p -,iuesof y; an :.:ze the sum ':rameters,gi\
5.2 PARAMETERS OF THE MULTIPLE
of (5.3) where
(s.4) this model with dables(K > 3).
the probability : introduce foret ,r 2. They are :ro mean. Some rf observations, ,arianceo2. The in the statistical s will the model variable.Errors spondingto any ervation has no ny pair of errors
REGRESSION MODEL
111
variable.This assumptionis realistic for our hamburgerchain where a decision aboutprices and advertisingis made for eachcity and valuesfor thesevariablesare set accordingly.For .-asesin which this assumptionis untenable,our analysiswill be conditional upon the values of the explanatoryvariablesin our sample,or further assumptionsmust be made.This issue is taken up further in Chapter 10. The secondassumptionis that any one of the explanatoryvariablesis not an exact linear tunctionof the others.This assumptionisequivalenttoassumingthatno variableis redundant. .{s we will see,if this assumptionis violated, a condition called exact collinearity, the least quares procedurefails. To summarize,we construct a list of the assumptionsfor the generalmultiple regression nodel in (5.3), much aswe havedonein the earlier chapters,to which we canrefer asneeded: OF THE MTILIIPLE
ASSUMPTIONS
R-EGR-ESSION MODEL
MR1. yi : Fr llzxiz+... + \xxix I ei, MR2. E(y) : 9t * $2x;2+...+ BxxiaeE(e;): Q var(e;) :62
MR3.
var(yi) :
MR4.
cov(y;,y;) :
cov(ei,ej)
MR5. The valuesof eachx* arenot randomand arenot exactlinear functions of the other explanatory variables -N[(Fr -l gzxiz+ . . . + grxix),ozle e;-N(0, o2) yr MR6.
mal probability
5.2 Estirnating the Pararneters of the Multiple RegressionModel
tmerfof tetmei, 'om those of e;.
In this section we consider the problem of using the least squaresprinciple to estimatethe :.rnknownparametersof the multiple regressionmodel. We will discussestimation in the iontext of the model in (5.4), which we repeat here for convenience. yi : Br l9zxiz l9zxiz * ei
(s.4)
lepends on the is equivalent to anges for each '\zxiz * 9zxis.
This model is simpler than the full model and yet all the resultswe presentcaffy over to the Eeneralcasewith only minor modifications.
r of y; does not ore likely to be
5.2.1
rnt variable are :quent observauted about their .oassumingthat
t the dependent , first is that the he valuesofthe 'the dependent
Lnesr SquanEs Esrrmerrox PnocsottRE
To find an estimator for estimating the unknown parameterswe follow the least squares procedurethat was first introduced in Chapter 2 for the simple regressionmodel. With the leastsquaresprinciple we minimize the sum of squareddifferencesbetween the observed r.aluesof y; and its expectedvalue E(y;) : I r * xiz\z * x6 B3. Mathematically we minimize the sum of squaresfunction S(Ft,pz,F:), which is a function of the unknown parameters,given the data
S(9r,92,9r) : .I,lrr- r(y,)]' (5.5)
:,1(r, - 9r - gzxiz- gzxiz)z
112
THE MULTIPLE REGRESSION MODEL
Given the sample observationsy;, minimizing the sum of squaresfunction is a straightforward exercisein calculus.Details of this exercisearegiven in Appendix 5A at the end of this chapter.The solutionsgive us formulas for the leastsquaresestimatorsfor the B coefficients in a multiple regressionmodel with two explanatoryvariables.They are extensionsof those given in equations (2.7) and (2.8) for the simple regressionmodel with one explanatory variable. There are three reasonsfor relegating these formulas to Appendix 5A insteadof inflicting them on you here.First, they arecomplicatedformulas that we do not expectyou to memorize. Second, we never use these formulas explicitly; computer software uses the formulas to calculate least squaresestimates.Third, we frequently have models with more than two explanatoryvariablesin which casethe formulas becomeeven more complicated. If you proceedwith more advancedstudy in econometrics,you will discoverthat thereis one relatively simple matrix algebraexpressionfor the least squaresestimator that can be used for all models, irrespective of the number of explanatory variables. Although we always get the computer to do the work for us, it is important to understand the least squaresprinciple and the difference between least squaresestimators and least squares estimates. Looked at as a general way to use sample data, formulas for b1,b2, and b3, obtainedby minimizinC (5.5), are estimationprocedures,which are called the least squares estimators of the unknown parameters.In general,since their valuesare not known until the data are observed and the estimates calculated, the least squares estimators are random variables. Computer software applies the formulas to a specific sampleof data producing least squares estimates, which are numeric values.To avoid too much notation,we use b1,b2, and D3to denoteboth the estimatorsand the estimates. 5.2.2
LEesr SquanEs Esrrlnerss usrNc HeununcEn
CnerN Dnra
Table 5.2 contains the least squaresresults for the sales equation for Big Andy's Burger Barn. For the moment, we are concernedonly with the least squaresestimates,which are br : 118.91 bz : -7.908
bz : 1.863
The various other entries on the output will be discussedas we move through the remainder of this chapter. To interpret the coefficient estimates, note that the regression function that we are estimatingis E(y,) :9r
ln terms of
Ii1'ou were
Basedon th
1 . The r with revel incre the o to co price devie rever f
The < an in rever addit will i
The r zeto zero to re( have direc
h addition .hange,the, :r predictinl Includinger
* \zxiz * 9:x;:
and the fitted regressionline is Yi:hlb2x;2 lfux;3 : 118.91- 7.908x;2* 1.863x;3
The predictt
Ta b I e 5 . 2 Least Squares Estimates for Sales Equation for Big Andy's Burger Barn Variable
c P A R2:0.4483
Coefficient
118.9136 -7.9079 r.8626 SSE: 1718.943
Std.Error
t-Statistic
Prob.
6.3516 1.0960 0.6832
18.7217 -7.2152
0.0000 0.0000 0.0080
d :4.8861
2.7263 6' : 6'48854.
REMAF negativer Iftaken li not keep regressio similar tc generally explanat the surro
5.2 PARAMETERS
r is a straightforat the end ofthis he B coefficients tensionsof those one explanatory ix 5,A.instead of notexpectyou to rftware uses the odels with more rre complicated. r that thereis one that can be used mt to understand nators and least a, formulas for which are called : their values are he least squares las to a specific res.To avoid too he estimates.
3 Andy'sBurger nates,which are
3hthe remainder ion that we are
REGRESSION MODEL
113
In terms of the original economic variables,
S i : 1 1 8 . 9 1- 7 . 9 O B P+i 1 . 8 6 3 A t
(s.6)
If youwerewritinga paperyoumightusevariablenamesthataremoreinformative,suchas Sffi
: I 18.91- 7.g08PRICE+ 1.863ADvERT
Basedon theseresults,whatcan we say? I . The negativecoefficient on P, suggeststhat demandis price elastic; we estimatethat, with advertisingheld constant,an increasein price of $l will lead to a fall in monthly revenueof $7908. Or, expresseddifferently, a reduction in price of $ 1 will lead to an increasein revenueof$7908. Ifsuch is the case,a strategyofprice reduction through the offering of specialswould be successfulin increasingsalesrevenue.We do need to consider carefully the magnitude of the price change,however. A $l change in price is a relatively large change.The samplemean of price is 5.69 and its standard deviation is 0.52. A l0 cent changeis more realistic, in which casewe estimatethe revenuechangeto be $791.
2 . The coefficient on advertisingis positive; we estimatethat, with price held constant, an increase in advertising expenditure of $1000 will lead to an increase in sales revenueof $ 1863.We can usethis information, along with the costsof producing the additional hamburgers,to determinewhether an increasein advertisingexpenditures will increaseprofit. J.
TA
OF THE MULTIPLE
The estimatedintercept implies that if both price and advertising expenditurewere zero the salesrevenuewould be $118,914.Clearly,this outcomeis not possible;a zero price implies zero salesrevenue.In this model, as in many others,it is important to recognizethat the model is an approximation to reality in the region for which we have data. Including an intercept improves this approximation even when it is not directly interpretable.
In addition to providing information about how sales change when price or advertising change,the estimatedequation can be used for prediction. SupposeBig Andy is interested in predicting salesrevenue for a price of $5.50 and an advertising expenditureof $1200. Including extra decimal places to get an accuratehand calculation, this prediction is S : t tS.St -'7.908PRICE+ 1.863ADVERT = 1 1 8 . 9 1 4- 7 . 9 0 7 9 x 5 . 5 * 1 . 8 6 2 6 x 1 . 2 :77 .656 The predictedvalue of salesrevenuefor P : 5.5 and A : 1.2 is $77,656.
,ndy's
Prob.
0.0000 0.0000 0.0080
REMARK: A word of caution is in order about interpreting regressionresults. The negative sign attachedto price implies that reducing the price will increasesalesrevenue. If takenliterally, why shouldwe not keepreducingthe priceto zero?Obviouslythat would not keep increasing total revenue.This makes the following important point: Estimated regressionmodels describethe relationshipbetweenthe economic variablesfor values similar to those found in the sample data. Extrapolating the results to extreme values is generallynot a good idea.Predictingthe value of the dependentvariablefor valuesof the explanatory variables far from the samplevalues invites disaster.Refer to Figure 4.2 and the surroundingdiscussion.
114
5.2.3
THE MULTIPLE
REGRESSION MODEL
EsrruerroN oF THE EnnonVanreNcs
J.3 SAMPL
5. J Sampli
o2
There is one remaining parameter to estimate, the variance of the error term. For this parameterwe follow the same steps that were outlined in Section 2.7. We know that o2 :var(ei):
n(r?)
Thus, we can think of o2 as the expectation or population mean of the squarederrors ef. A natural estimatorof this population mean is the sample mean62 : 2e?I tl . However, the squarederrorsel are unobservableand so we developan estimatorfor o2 that is basedon the squaresof the least squaresresiduals.For the model in (5.4) these residuals are 0r : Yi - ji : yi - (h * b2xi2* fuxp) An estimator for o2 that usesthe information from ?! and has good statisticalpropertiesis
^) *:'a? ' o;': N-K
(s.1)
where K is the number of B parametersbeing estimatedin the multiple regressionmodel. We can think of 62 as an averageof 2l with the denominatorin the averagingprocessbeing N - K insteadof N. It can be shownthat replacing e! by 2f requiresthe useof N - K instead of N for 62 to be unbiased.Note that in Chapter 2, equation(2.1g), where there was one explanatory variable and two coefficients, we had K :2. To appreciate further why 0i provide information about o2, recall that o2 measures the variation in e; or, equivalently, the variation in y; around the mean function Fr -t \zxiz * F:xis. Since 2; are estimatesof ei,big valuesof 2; suggesto2 is large while small 2; suggesto2 is small. When we refer to "big" valuesof 2;, we meanbig positive ones or big negativeones.Using the squaresofthe residuals2l meansthat positive valuesdo not cancel with negative ones; thus, 2l provide information about the parametero2. In the hamburgerchain examplewe have K : 3. The estimatefor our sampleof data in Table 5.1 is
nB.s4i 6^1 z : ->17'a? i-:17i:23.814 Go back and have a look at Table 5.2.'[here are two quantitiesin this table that relate to the above calculation. The first is the sum of squarederrors s^?
SSE: L e 7 :
'-
.i generalconte :r- different valu :,--iectedand the e-. us how the es ::ability of the =:iased and tha - model assun 'r:ression model
Itr#'
-: lre rre able tc r,:t'mally distribu r.rt'ability distril : rributed, thent 'i:lples. What c :"r-h application :oortance for th ::remeters of tht
i-1.1
Tns Ve
llc variancesan ':-:ability of the r
:r smaller their ':ear" the true p - en algebraic f
-rmator. For ex
1718.943
The secondis the squareroot of 62, given by
rtrere r23is the si B l0) in Appen<
t : '/zysza:4.8861 Both thesequantitiestypically appearin the output from your computer software.Different software refer to it in different ways. Sometimes6 is referred to as the standarderror of the regression.Sometimesit is called the root znse(short for mean squarederror). A major reasonfor estimating the error varianceis to enable us to get an estimateof the unknown variancesand covariancesfor the least squaresestimators.We now considerthose variances and covariances in the context of the overall properties of the least squares estimator.
i'-.r the other varii :.: understandthe l.
Largererr is to be ex1 If o2 is lar
115
5.3 SAMPLING PROPERTIES OF THE LEAST SQUARES ESTIMATOR
5.-7 Sampling Properties of the Least Squares Estimator )rm. For this now that
red errors ef. However, the ; basedon the . are
properties is
(s.t) :ssionmodel. rrocessbeing / - K instead here was one o2 measures ean function is large while positive ones valuesdo not r o2. Lpleof data in
Ltrelate to the
In a generalcontext,the leastsquaresestimators(h,bz,b) arerandomvariables;they take on different values in different samples and their values are unknown until a sample is collected and their values computed. The sampling properties of a least squaresestimator tell us how the estimatesvary from sampleto sample.They provide a basisfor assessingthe reliability of the estimates.In Chapter 2 we found that the least squaresestimator was unbiasedand that there is no other linear unbiasedestimator that has a smaller variance,if the model assumptions are correct. This result remains true for the general multiple regressionmodel that we are considering in this chapter. For the multiple regression model, if THE GAUSS-MARKOV THEOREM: assumptionsMRl-MR5 listed at the beginningof the chapterhold, then the leastsquares estimatorsare the bestlinear unbiasedestimators(BLUE) of the parameters. If we are able to assumethat the erors are normally distributed, then y will also be a normally distributed random variable. The least squaresestimatorswill also have normal probability distributions, since they are linear functions of y. If the errors are not normally distributed,then the leastsquaresestimatorsare approximatelynormally distributed in large iamples. What constitutes large is tricky. It dependson a number of factors specific to each application. Frequently, N - K: 50 will be large enough. These facts are of great importancefor the constructionof interval estimatesand the testing of hypothesesabout the parametersof the regressionmodel.
5.3.1
Tns VARrnNcEs AND CovenreNcEs oF THE LEesr Squenrs Esrrrvr,qrons
The variancesand covariancesofthe least squaresestimatorsgive us information about the reliability of the estimatorsb1,b2, and fu. Sincethe leastsquaresestimatorsare unbiased, the smaller their variancesthe higher is the probability that they will produce estimates "near" the true parametervalues.For K : 3 we can expressthe variancesand covariances in an algebraic form that provides useful insights into the behavior of the least squares estimator.For example, we can show that var(b2) :
02
0 - 4)2!:'(*,, -ir)'
(s.8)
rvherer23is the samplecorrelation coefficient betweenthe valuesofx2 and xz; seeequation r8.20) in Appendix B.4.3 and Section4.2.I.Its formula is given by rz3: ue. Different d ercor of the r). timate of the rnsider those east squares
(5.e)
For the other variancesand covariancesthere are formulas of a similar nature.It is important to understandthe factors affecting the variance of b2: I . Larger error varianceso^2lead to larger variancesof the leastsquaresestimators.This is to^beexpectedsinceo'measurestheoveralluncertaintyin themodel specification. If o' is large,then datavaluesmay be rridely spreadabout the regressionfunction
116
THE MULTIPLE REGRESSION MODEL
5.J
E(y):FrlB2xi2*Frxi: and there is less information in the data about the parameter values. If o' is small then data values are compactly spread about the regressionfunction E(y;) : 9t * \zxiz * B3r;3and there is more information about what the parametervalues might be. 2. Larger sample sizes N imply smaller variancesof the least squaresestimators. A larger value of N meansa larger value of the summationL (xiz - 7)2 . Since this term appearsin the denominatorof (5.8), when it is larye,var(b) is small. This outcomeis also an intuitive one; more observationsyield more precise parameterestimation. 3. More variation in an explanatory variable around its mean,measuredin this caseby Z(xiz - x2)2,leadsto a smallervarianceof the least squaresestimator.To estimateB2 precisely,we prefer a large amount of variation in x;2.The intuition here is that, if the variation or changein 12 is small, it is difficult to measurethe effect of that change. This difficulty will be reflected in a large variance for 02. 4.
A larger correlation between x2 and x3 leads to a larger variance of b2. Note that 1 - 4, appearsin the denominatorof (5.8).A valueof lryl closeto I means1 - fzt will be small, which in turn meansv ar(b) willbe large.The reasonfor this fact is that variation in x;2 about its mean addsmost to the precision of estimation when it is not connectedto variation in the other explanatoryvariables.When the variation in one explanatory variable is connectedto variation in another explanatory variable, it is difficult to disentangletheir separateeffects.In Chapter6 we discuss"collinearity," which is the situation when the explanatoryvariablesarecorrelatedwith one another. Collinearity leads to increasedvariancesof the least squaresestimators.
Although our discussionhasbeen in terms of a model where K : 3, thesefactors affect the variancesof the least squaresestimators in the same way in larger models. It is customary to arrangethe estimatedvariancesand covariancesof the least squares estimators in a square array, which is called a matrix. This matrix has variances on its diagonal and covariancesin the off-diagonal positions. It is called a variance-*ovariance matrix or, more simply, a covariance matrix. When K:3, the arrangement of the variances and covariancesin the covariance matrix is
c o v ( b y ,b z , b s ) :
fu)l I var(b1) cov(b1,b2) cov(b1, I cov(b1,b2) var(b2) cov(b2,fu) | var(b3) .l f cov(b1,b3) cov(b2,fu)
Using the estimate 62:23.874 and our computer software package, the estimated variancesand covariancesfor br,bz, and D3in the Big Andy's Burger Barn exampleare
cou(br, hE
+o.t+t -6.ies -0.i4841 | -6.7es : 1.20t -0.0re7| | -0.0197 0.46681 l-0.7484
(s.10)
)
Table 5 -
C P a
Table 5.3 .oftware. Of part \qUaferO(
.\gain, it i rtandard e These t astlmates erample, quares es thenbase leastsqua .ri themee rrould lie
:ts CoITeS[
.stimates betweenZ "large" c .stimates estimate .-onstruc
r.3.2 1 \onmer
\\'e have i rt the end the paran
Thus. we have
:40.343 *(bt, D : -6.795 "-(ail : -0.1484 ;;@r-) : r.2or "*@td -o.ot97 ;ful-) :0.4668 ]rr.----_): "*(br,
If we add then the r
5.3 SAMPLING PROPERTIES OF THE LEAST SQUARES ESTIMATOR
117
data about the rread about the brmation about
Table 5 .3
s estimators.A Since this term fhis outcomeis :ter estimation.
P
t in this caseby ToestimateB2 :re is that, if the of that change.
Table 5.3 shows how this information is typically reported in the output from computer software. Of particular relevanceare the standarderrors of h, bz, and bz; they are given by the \quare roots of the correspondingestimated variances.That is,
C L
f b2. Note that meansl - fu 'this fact is that r when it is not ariation in one I variable, it is "collinearity," th one another. tors. :tors affect the : least squares .rianceson its e-covariance ;ement of the
;he estimated example are
(s.10)
Covariance Matrix for Coefficient Estimates
-6.7951
40.3433 -6.7951 -0.7484
1.2012 -0.0197
s e ( 0 1 ):
var(b1 : \/40,3433:6.3516
se(b2) :
var(b2)
se(b3) :
var(fu): t/oA6G8:0.6832
-0.7484 -0.0197 0.4668
: JL2otz: t.o96o
Again, it is time to go back and look at Table 5.2. Notice that these values appear in the .tandard error column. Thesestandarderrors can be used to say something about the range of the least squares .stimates if we were to obtain more samplesof 75 burger barns from different cities. For erample, the standarderror of b2 is approximatelyse(b2):1.1. We know that the least {uares estimatoris unbiased,so its mean value is E(b2) : B2.If bz is normally distributed, thenbasedon statisticaltheory we expect95Voof the estimatesb2,obtainedby applying the ;eastsquaresestimatorto other samples,to be within approximately two standarddeviations olthe meanB2.Givenoursample,2xse(b) :2.2,soweestimate that 95Voof theb2values * ould lie within the interval9z + 2.2.It is in this sensethat the estimatedvarianceof 82, or :ts correspondingstandarderror, tells us somethingabout the reliability ofthe least squares :stimates.If the differencebetweenb2 and 92 can be large,b2is notreliable; if the difference betweenD2and B2is likely to be small, then b2 is reliable. Whether a particular difference is "large" or "small" will depend on the context of the problem and the use to which the estimatesare to be put. This issue is consideredagain in later sections when we use the estimated variances and covariances to test hypotheses about the parameters and to ronstruct interval estimates. 5.3.2 TnE PnopsRTrES oF rHE Lnesr Squenrs EsrrmeroRs AssulrrNc \onrlrerry DrsrnrsurED ERRoRS \\'e have assertedthat, under the multiple regressionmodel assumptionsMRl-MR5, listed atthe end of Section5. 1,the leastsquaresestimatorb6is the bestlinear unbiasedestimatorof the parameter B7.in the model lt : 9t * \zxiz* 9:-rrs+ "' + \rxx
* et
If we add assumptionMR6 that the random errors !; have normal probability distributions, :hen the dependentvariable y; is normally distributed, y i - N [ ( 9 r I g z x i z + " ' + 9 r . t ; r ) .o 2 ] e e i - l r ( 0 , o 2 )
118
THE MULTIPLE REGRESSION MODEL
Since the least squaresestimatorsare linear functions ofdependent variablesit follows that the least squaresestimators are also normally distributed,
Rearrangin
br - N[9t,,var(be)] That is, eachbphas a normal distribution with mean B1and variance var(b1).By subtracting its meanand dividing by the squareroot of its variance,we can transform the normal random variable bp into the standard normal variable Z,
t :m-N(o,r),
fort: r,2,...,K
( s.11)
that hasmean zero and a varianceof l. The vari anceof b1,dependson the unknown variance of the error term, o2, as illustrated in (5.8) for the K : 3 case.when we replace o2 by its estimator 62, from (5.7), we obtain the estimated var(br) which we denote ut il@). Replacing var(b) by6@D in (5.11) changesthe N(0,1) random variablero a /-random variable. That is.
h - F r -r1rir-K) se(bt
This intervt .omewhere :rore realis S1009.Bas \ndy can c Followir t 957o inte
(
\\'e estimatr :elenue of
5,4 Interval Estimation
.\tra adve
Supposewe are interestedin finding a 95Vointewal estimate for B2, the responseof sales revenueto a changein price at Big Andy's Burger Barn. Following the proceduresdescribed in Section 3. 1, and noting that we haveN - K - 75 - 3 : j2 degreesoffreedom, the first step is to find a value from the /1721-distribution, call it r., such that
(5.l3)
Using the notationintroducedin Section3.1, t, : tp.sts,w_r1isthe 97.5-percentileof the /1rv-ry-distribution(the areaor probability to the left of t, is 0.975), and-t, : /1o.ozs, rv_r; is the 2.5-percentile of the /1r,'-r;-distribution (the area or probability to the left of -/. is 0'025). Consulting the /-table inside the front cover of the book, we discoverthereis no entry for 72 degreesoffreedom, but, from the entriesfor 70 and 80 degreesoffreedom, it is clear that, correct to two decimal places,t, : l.99.If greateraccuracyis required,your computer softwarecan be used to find /. : 1.993.Using this value, and the result in (5.12) for the secondcoefficient(k :2), we can rewrite (5.131as
r(-r
Jefinea 951 :he populat :actbeforer -'ollectedw :ts perform A957oi :eplacingb :ntervalest
(s.r2)
One differencebetweenthis result and that in Chapter3, seeequation(3.2), is the degreesof freedom of the r-random variable. In Chapter 3, where there were two coefficients to be estimated,the number of degreesof freedom wasN - 2. In this chapterthere areKunknown coefficients in the general model and the number of degreesof freedomfor t-statisticsis N_K. We now examine how the result in (5.12) can be used for interval estimation and hypothesistesting. The proceduresare identical to those describedin Chapter 3, only the degreesof freedom change.
P(-t, < t1tz1.--t,) : 0.95
The interva
(s.14)
S1000),ori \nother wi . erv reliabl In gener :nmediate ..timated s -.f an inten rarrower ir :he varianc rore indep :engeofpn :le rvouldnr :enerally r \lternativ! The quest lrocess is
r For this
5.4 INTERVAL ESTIMATION
iablesit follows that
119
(5.14)we obtain Reananging P[b, - l.993xse(b2)< 9, < bz 11.993x se(b2)]: 0.95
'(be).By subtracting t the normal random
(s.11) g unknown variance ve replace o2 by its ----'-:> denote as var(Dp). iable to a t-random
(s.r2) 2), is the degreesof r coefficients to be rere areKunknown m for t-statistics is val estimation and Jhapter 3, only the
e responseof sales rceduresdescribed ,f freedom, the first
(5.13) 5-percentile ofthe tc : tp.oz5,p_61is the left of -/. is 'er thereis no entry [reedom,it is clear ed, yourcomputer It in (5.12) for the
(s.14)
Theintervalendpoints [b2 1.993x se(b2),bz * 1.993x se(b2)]
(s.1s)
definea95%ointerval estimatorof B2.If this interval estimatoris usedin many samplesfrom the population, then 95Voof them will contain the true parameter92. We can establishthis factbefore any dataare collected,basedon the model assumptionsalone.Before the dataare collectedwe have confidencein the interval estimation procedure (estimator) becauseof its performance when used repeatedly. A95Vo interval estimatefor B2basedon ourparticular sampleis obtainedfrom (5.15) by replacingb2 and se(bz)by their valuesbz: -7.908 and se(bz): 1.096.Thus, our 957o interval estimate for B2 is given by (- 10.092,-5.724) This interval estimatesuggeststhat decreasingprice by $1 will lead to an increasein revenue somewherebetween$5724 and$10,092.Or, in terms of aprice changewhosemagnitudeis more realistic, a 10 cent price reduction will lead to a revenueincreasebetween $572 and 51009. Based on this information, and the cost of making and selling more burgers, Big Andy can decide whether to proceed with a price reduction. Following a similarprocedureforB3, the responseof salesrevenueto advertising,we find a 95Vointerval estimate is given by' (L8626 - 1.993x0.6832, 1.8626+ 1.993x0.6832) : (0.501,3.224) We estimatethat an increasein advertisingexpenditureof $1000 leadsto an increasein sales revenueof between $501 and $3224. This interval is a relatively wide one; it implies that extra advertising expenditure could be unprofitable (the revenue increase is less than S1000),oritmay lead to arevenueincreasemore than threetimes the cost of the advertising. Another way of describing this situation is to say that the point estimatefu : 1.9626is not very reliable, as its standarderror (which measuressampling variability) is relatively large. In general,if an interval estimateis uninformative becauseit is too wide, there is nothing immediate that can be done. A wide interval for the parameter B3 arises because the estimatedsampling variability of the least squaresestimator b3 is large. In the computation of an interval estimate,a large sampling variability is reflectedby a large standarderror. A niurower interval can only be obtainedby reducing the varianceof the estimator.Basedon rhe variance expressionin (5.8), one solution is to obtain more and better data, exhibiting more independentvariation. Big Andy could collect data from other cities and set a wider rangeof price and advertisingcombinations.It might be expensiveto do so,however,and so hewould needto assesswhetherthe extra information is worth the extra cost.This solution is -eenerallynot open to economists,who rarely use controlled experiments to obtain data. Altematively we might introduce somekind of nonsampleinformation on the coefficients. The question of how to use both sample and nonsample information in the estimation processis taken up in Chapter 6. 2 For this calculation we used more digits so that it would match the more accuratecomputer output. You may see us do this occasionally.
120
THE MULTIPLE REGRESSION MOI)EI
We cannot say, in general, what constitutes an interval that is too wide, or too uninformative. It dependson the context of the problem being investigated and how the information is to be used. To give a general expression for an interval estimate we need to recognize that the critical value t will dependon the degreeofconfidence specifiedfor the interval estimate and the number of degreesof freedom. We denote the degree of confidenceby 1 - cr; in the caseof a95Vointervalestimatect:0.05 and I - cr:0.95. The numberof degrees of freedom is N - K; in Big Andy's Burger Barn example this value was 75 - 3 : "12. The value t, is the percentile value /1t-o72.N-n, which has the property that Pltq-r1 1 tg-a1z.r,r-rl: t - a1Z. In the case of a 95Vo confidence interval,
rind whether tl h1'pothesis
rgainst the alt
To carry out tl
| - a 12 : 0.975;we usethis valuebecausewe require0.025in eachtail of the distribution. Thus, we write the general expressionfor a 100(1 - a)Vo confidence interval as lb1, tg-o1z,p-ryxse(b1), b1,I tg-o1z,y-r;xse(bt)l
5,5 Hypothesis Testing for a Single Coefficient As well as being useful for interval estimation,the r-distributionresult in equation(5.12) provides the foundation for testing hypotheses about individual coefficients. As you discoveredin Chapter 3, hypothesesof the form IIs ilz : c versusI11:92* c, where c is a specifiedconstant,are called two-tail tests.Hypotheseswith inequalitiessuch as.F1s: 9z < c versusI11:52> c are called one-tail tests.In this sectionwe considerexamplesof each type of hypothesis. For a two-tail test, we consider testing the significance of an individual coefficient; for one-tail tests some hypothesesof economic interest are considered. We will follow the step-by-step procedure for testing hypotheses that was introduced in Section 3.4. To refresh your memory, here again are the steps: STEP.BY-STEP PROCEDTIRJ
FOR TE STING HYPOTHESE S
1. Determine the null and alternative hypotheses. 2. Specify the test statistic and its distribution if the null hypothesis is true.
For the alterni .r.3.3,and reje :rom the right .ctl side of tl Jnd -t^ :
l'^
:'-r'alue,we rt In the Big .i hethersales l.
The nt
l.
The te
i.
Using freedo distrib hypotl t0
-1. The c
3. Select a and determine the rejection region. 4. Calculate the sample value of the test statistic and, if desired, the p-value. 5. Stateyour conclusion. From At the time these steps were introduced in Chapter 3 you had not discoveredp-values. Knowing aboutp-values(seeSection3.5) meansthat steps3-5 can be framed in terms of the test statistic and its value and/or the p-value. We will use both.
P Corn
5.5.1
TEsrrNc tHr SrcNrrrcANCEor e SrNcre CosrrrcrENr
When we setup a multiple regressionmodel, we do so becausewe believethe explanatory variables influence the dependentvariable y. If we are to confirm this belief, we need to examine whether or not it is supportedby the data.That is, we needto ask whether the data provide any evidenceto suggestthat y is related to each of the explanatory variables. If a given explanatoryvariable, sa! .{1,has no bearing on r', then 9r :0. Testing this null hypothesisis sometimescalleda testof significancefor the explanatoryvariablex7..Thus,to
5. Sinct from perf(
For testir
t. FIo: 2. The
5.5 HYPOTHESIS TESTING FOR A SINGLE COEFFICIENT
121
)o wide, or too rted and how the
find whether the data contain any evidence suggestingy is related to.rr, we test the null hypothesis
,cognize that the interval estimate nce by I - cr; in Lmberof degrees rus75-3:'12. e property that idence interval,
F1s:Bp : Q againstthe alternative hypothesis
fu:Br*o To carry out the test we use the test statistic (5.12), which, if the null hypothesisis true, is
f the disrriburion. erval as
E r equation(5.12) ficients. As you : 9 z l c , w h e r ec ities such as .Ils : ider examplesof gnificance of an interest are conrtheses that was leps:
t:
For the alternative hypothesis "not equal to" we use a two-tail test, introduced in Section 3.3.3, and reject F1sif the computed /-value is greaterthan or equal to t, (the critical value from the right side of the distribution) or lessthan or equal to -t, (the critical value from the left side of the distribution). For a test with level of significanced, tc: t6-o1z,u-rc) and-t, : t@/z,N-n. Alternatively, if we statethe acceptance-rejectionrule in terms of the p-value, we reject Hs if p ( ct and do not reject Hg 1f p > a. In the Big Andy's Burger Barn example we test, following our standardtesting format, whether salesrevenue is related to price: 1. The null and alternativehypothesesareHs:92 : 0 andH1:92*0. 2. The test statistic,if the null hypothesisis true, is t : bzlse(bz)- tru-n. 3. Using a 5% significancelevel (ct :0.05), and noting that there are 72 degreesof freedom, the critical values that lead to a probability of 0.025 in each tail of the : -l '993' Thus we reject the null distribution arc tq.s7s,lz): l '993 and /1o.ozs,rz) hypothesis if the calculated value of t from step 2 is such that t) 1.993 ot we do not reject F16.Stating the acceptancet < -L993.If -1993 0.05. 4.
The computed value of the r-statistic is ' :
rveredp-values. :d in termsof the
the explanatory lief, we need to vhetherthe data y variables. If a bsting this null ablexr.Thus,to
br -r17v-r) '"(b)
-7.908 Lo%
: -l '215
From your computer software the p-value in this case can be found as < - 7 .215) : 2 x (2.2 x l0-r0) : 0.000 P (t 1t4> 7'215) + P (t 1tz1 Correct to three decimal places the result is p-value : 0.000' 5. Since -7.215 < - 1.993,we rejectHo:92 : 0 and concludethat thereis evidence from the data to suggest sales revenue depends on price. Using the p-value to perform the test, we reject .F1sbecause0.000 < 0.05. For testing whether salesrevenue is related to advertising expenditure, we have 1. Ilo:9: :0
a n dF I 1: 9 2 * 0 .
2. The test statistic,if the null hypothesisis true. is t : bzI se(b) - torn.
122
3. Using a 5Vo significance level, we reject the null hypothesis if t> I.993 or t < -1.993.In terms of thep-value, we reject Hsif p < 0.05. Otherwise,we do not reject I1e. 4. The value of the test statistic is
r.8626 ': oss32:2'726 The p-value is given by P (t 1tz:> 23 26) + P (t 1tz1< - 2 j7 26) : 2 x 0.004 : 0.008 5. Because2.726 > 1.993,we rcject Hs; the data supportthe conjecturethat revenueis related to advertising expenditure.The sametest outcome can be obtainedusing the p-value. In this casewe reject I1s because0.008 < 0.05. Note that the /-values -7.215 and2.726 and their correspondingp-values0.000 and 0.008 were reported in Table 5.2 at the same time as we reported the original least squares estimatesand their standarderrors.Hypothesistestsofthis kind are carried out routinely by computer software, and their outcomescan be read immediately from the computer output that will be similar to Table 5.2. Significanceof a coefficient estimateis desirable.It confirms an initial prior belief that a particular explanatory variable is a relevant variable to include in the model. However, as mentioned in Section 3.4.3, statistical significance should not be confused with economic importance. If the estimatedresponseof salesrevenue to advertising had been bz :0.01 with a standarderror of se(b:) : 0.005, then we would have concluded b3 is significantly different from zero; but, since the estimate implies increasing advertising by $1000 increasesrevenueby only $10, we would not conclude advertising is important. We should also be cautious about concluding that statistical significance implies precise estimation. The advertising coefficient fu : 1.9626 was found to be significantly different from zero, but we also concludedthe corresponding95vo intewal estimate(0.501,3.224) was too wide to be very informative. In other words, we were not able to get a preciseestimateof 83. 5.5.2
5.5
THE MULTIPLE REGRESSION MODEL
ONE-Tarr HyporHesrs TssrrNc FoR A SrNcrs CoErrrcrsxr
In Section 5.1 we noted that two important considerationsfor the managementof Big Andy's Burger Barn were whether demand was price elastic or inelastic and whether the additional salesrevenuefrom additional advertising expenditurewould cover the costs of the advertising.We now are in a position to statethesequestionsas testablehypothesesand to ask whether the hypothesesare compatible with the data. 5.5.2a Testing for Elastic Demand With respect to demand elasticity, we wish to know if Fz > 0: a decreasein price leads to a decreasein sales revenue (demand is price inelastic),or 9z ( 0: a decreasein price leads to an increase in sales revenue (demand is price elastic).
If we are not Prepa the datato supportt as our null hypotht alternativehyPothe L
Flo:Bz>0 H 1 : $ 2 < O(
2. To createa t so is valid b, assuming1/
3. The rejectic occur if tht significance r Q . O 5 . 7 2_) -
,1. The value c
The corresl
5. Since -7.1 (demandis the proposi Since 0.00(
\otethe similariti( ryrformed in Secti Jifferent. Not onll --ritical values, onc .-ritical value, fron i\ually half that c :rample becauseI
5.5.2b Testing . The other hypothe .rnincreasein sale ruch an increaser
1. I/s: 9r < 1
2. Treatingth distributiot
3. Choosing t(o.gs,72)-
5.5 HYPOTHESIS TESTING FOR A SINGLE COEFFICIENT
is if l> 1.993or Otherwise, we do
Ifwe are not preparedto acceptthat demandis elastic unlessthere is strong evidencefrom the datato supportthis claim, it is appropriateto take the assumptionof an inelastic demand as our null hypothesis. Following our standardtesting format, we first state the null and alternativehypotheses: l.
008 :urethat revenueis obtainedusing the
:s 0.000 and 0.008 ;inal least squares :d out routinely by e computer output I prior belief that a odel. However, as ed with economic d b e e nb z : 0 . 0 I &3is significantly :rtising by $1000 |ortant.We should ,reciseestimation. fferent from zero, 1,3.224)was too ise estimateof B3.
]NT
nagementof Big : and whether the ;over the costsof Lehypothesesand
Ho:92> 0 (demandis unit elasticor inelastic). Ht:92 < 0 (demandis elastic).
2. To createa test statistic we act as if the null hypothesisis the equality 9z : 0. Doing so is valid becauseif we reject lLlsfor B2 : 0, we also reject it for any B2 > 0. Then, assumingHs:$2 : 0 is true, from (5. 12) the test statisticis t : bz lse(bz)- tW-xt. 3. The rejection region consists of values from the r-distribution that are unlikely to occur if the null hypothesis is true. If we define "unlikely" in terms of a 5Vo significance level, then unlikely values of t are those less than the critical value te.os;2): -1.666. Thus, we reject lls if t < -1.666 or if the p-value < 0.05. -1. The value of the test statistic is ':
b" rqb):
-7 90t _ : - i _' Z. 7t )1 t < l"o%
The correspondingp-value is P(t1tzy< -'7 .215) : 0.000. 5 . S i n c e - 7 . 2 1 5 < - 1 . 6 6 6 , w e r e j e c t H o t ! z ) 0 a n d c o n c l u d et h a t H 1 : 9 2 < 0 (demandis elastic) is more compatible with the data.The sampleevidencesupports the proposition that a reduction in price will bring aboutan increasein salesrevenue. Since 0.000 < 0.05, the sameconclusionis reachedusing the p-value. \ote the similarities and the differencesbetweenthis test andthe two-tail test of significance performedin Section5.5.1. The calculated/-valuesare the same,but the critical t-valuesare different. Not only are the values themselvesdifferent, with a two-tail test there are two ;ritical values,one from each side of the distribution. With a one-tail test there is only one ;ritical value, from one side of the distribution. Also, the p-value from the one-tail test is usually half that of the two-tail test, although this fact is harder to appreciatefrom this erample becauseboth p-values are essentially zero. 5.5.2b Testing Advertising Effectiveness The other hypothesisofinterest is whether an increasein advertisingexpenditurewill bring en increasein salesrevenuethat is sufficient to cover the increasedcost of advertisins. Since .uch an increasewill be achievedif F, > 1, we set up the hypotheses: 1 . I 1 o : 9 : < | a n dH 1 : 9 s > l . 2. Treating the null hypothesisasthe equality I10: 93 : l, the test statisticthat hasthe rdistributionwhen -F1s is true is, from (5.12),
ldemand is price idemand is price
3. Choosing a:0.05 as our level of significance,the relevant critical value is t@sslz) : 1.666.We reject Ho if t > 1.666or if the p-value < 0.05.
124
4.
THE MULTIPLE REGRESSION MODEL
and so
The value of the test statistic is '-
bt - 9z 1.8626- | r"(bt 05832
T h e p - v a l u eo f t h e t e s ti s P ( t r 7 2 , >1 . 2 $ ) :
For Big An SSE: 1718
0.105.
5. Since 1.263<1.666, we do not reject I1e. There is insufficient evidencein our sample to conclude that advertising will be cost effective. Using the p-value to performthe test,we againconcludethatIls cannotbe rejectedbecause0.105 > 0.05. Another way of thinking about the test outcome is as follows: Becausethe estimate bz:1.8626 is greaterthan 1, this estimateby itself suggestsadvertisingwill be effective. However,when we take into accountthe precisionof estimation,measured by the standarderror, we find that b2 : | .8626 is not significantly greaterthan I . In the context of our hypothesis testing framework, we cannot conclude with a sufficient degree of certainty that B3 > 1.
5.6 Measuring Goodness-of:Fit For the simple regressionmodel studiedin Chapter4, we introducedR2 as a measureof the proportion of variation in the dependent variable that is explained by variation in the explanatoryvariable.In the multiple regressionmodel the samemeasureis relevant,and the same formulas are valid, but now we talk of the proportion of variation in the dependent variable explained by all the explanatory variables included in the linear model. The coefficient of determination is
R SR->L'ii"-Y)1 " r:S ssr I[,0r - y)' : ' -,s sssE z : ,t - r 1 -2!:,a? _Ur,-f
(s.16)
whereSSRis the variation iny "explained" by the model (sum of squaresof regression),SSZ is the total variation in y about its mean (sum of squarestotal), and SSEis the sum of squared least squaresresiduals(enors) and is the portion of the variation in y that is not explainedby the model. The notation !; refers to the predicted value of y for each of the sample values of the explanatory variables. That is, yt : h I b2x;2* b3xs l'
" I byx;y
The samplemeany is both the mean of y; andthe mean ofj;, providing the model includes an intercept(B1 in this case). The value for SSEwill be reportedby almost all computer software,but sometimesSSZis not reported. Recall, however, that the sample standarddeviation for y, which is readily computed by most software, is given by
o., -
I
$,.,
N - | i?r"'
.tsz N-1
The interpre the variation rhat, in our s r ariation in t 3rror term. As menti neasureof tl row well tht iample corr! :he linear as essociationt :hemodel is r :he valuesol Jata well. One diffic .Jding more r,lgebraica :rd thusR2 I : a model j An altern ,?:. is usuall
ior the dati
. ? -: 0 . 4 3 3 . :e degrees :.-reases,sl rhich SSEl -rfortunatel ..rriationex
5.6 MEASURING
GOODNESS-OF.FIT
125
and so
ssr: (N- l),i; For Big Andy's Burger Barn we find that SSZ:74x6.488542:3115.485 SSE : 1718.943.Using these sums of squares,we have dence in our ]e p-value to ) . 1 0 5> 0 . 0 5 . : the estimate :ising will be on, measured Lterthan1.In :lude with a
easureofthe iation in the :vant,and the re dependent model. The
(s.16)
:ession),.9S2 m ofsquared rxplainedby ,aluesof the
I includes an ,timesSSZis :h is readily
R2:l -
and
1118'943 : u'446 ,,}L'u? ^:, 3l 15485 L|:rfi y)"
The interpretationof R2 is that 44.8Voof the variation in salesrevenueis explainedby the variation in price and by the variation in the level of advertisingexpenditure.It means that, in our sample, 55.2Voof the variation in revenue is left unexplained and is due to variation in the error term or to variation in other variables that implicitly form part of the error term. As mentioned in Section 4.2.2, the coefficient of determinationis also viewed as a measureof the predictive ability of the model over the sampleperiod or as a measureof how well the estimatedregressionfits the data. The value of R2 is equal to the squared samplecorrelationcoefficient betweenj'; and y;. Since the samplecorrelationmeasures the linear associationbetweentwo variables,if R2 is high, it meansthat there is a close associationbetweenthe valuesofy; and the valuespredictedby the model,!;. In this case the model is saidto "fit" the datawell. If R2is low, thereis not a closeassociationbetween the valuesof y; and the valuespredictedby the model, j';, and the model doesnot fit the data well. One difficulty with R2 using as a goodness-of-fitmeasureis that it can be made large by addingmore and more variables,evenif the variablesaddedhaveno economicjustification. .{lgebraically it is a fact that asvariablesare addedthe sum of squarederrorsSSEgoesdown and thus R2 goesup. If the model containsN - I variables,then R2 : 1. The manipulation of a model just to obtain a high R2 is not wise. An alternativemeasureof goodness-of-fitcalled the adjustedR2,and often symbolized as Rl, is usually reported by regressionprograms; it is computed as /(N-1() ^n 2: r - ,s ss7s E/ w -D For the data from Big Andy's Burger Barn the value of this descriptive measure is R- :0.433. This measuredoes not always go up when a variable is added,becauseof rhe degrees of freedom term N - K in the numerator. As the number of variables K increases,SSEgoesdown, and so doesN - K. The effect on R' dependson the amount by *hich SSE falls. While solving one problem, this corrected measureof goodnessof fit unfortunatelyintroducesanotherone.It losesits interpretation;R' is no longer the percentof r ariation explained. This modified F is sometimes used and misused as a device for selectingthe appropriateset of explanatory variables.This practice should be avoided.We preferto concentrateon the unadjustedR2and think of it asa descriptivedevice for telling us about the "fit" of the model; it describes the proportion of variation in the dependent r ariableexplained by the explanatory variablesand the predictive ability of the model over the sample period. One final note is in order. The intercept parameter By is the y-intercept of the regression"plane," as shown in Figure 5.1. If, for theoretical reasons,you are certain :hat the regressionplane passesthrough the origin. then B1 : 0 and can be omitted from
126
THE MULTIPLE
REGRESSION MODEL
the model. While this is not a common practice, it does occur, and regressionsoftware includes an option that removes the intercept from the model. If the model does not contain an intercept parameter, then the measure R2 given in 15.16) is no longer appropriate.The reason it is no longer appropriate is that, without an intercept term in the model, N
N
N^
^ - t)'* Lr(n -r)"+,\rai ,I_(yr
or SSTI SSR+ SSE.To understandwhy, go back and check the proofin Appendix 48 of Chapter 4. In the sum of squaresdecomposition the cross-productt".- LLr ( yi - y)Oi no longer disappears. Under these circumstances it does not make sense to talk of the proportion of total variation that is explained by the regression.Thus, when your model does not contain a constant, it is better not to report R2, even if your computer displays on9.
5.6.1
). /
.\nswe
REpoRrrNc rrrp REcnpssroN REsurrs
5 . 7. 1 The discussion and calculation of R2 completes our description of all the entries in Table 5.2. We are now in a position to report the results along the lines described in Section 4.2.4.rVhen writing a report or paper,the information from estimating a multiple regression equation is summarized by writing down (i) the estimated equation using informative variable names, (ii) standard errors of coefficients below the estimated coefficients (or the /-values for testing zero null hypothesis),and (iii) the R2-value. For Big Andy's Burger Barn we have
sfB: (s")
1r8.9- i.g\SpRrcE+ r.8626ADVERT R2: 0.448 (6.4)(1.0e6) (0.6832)
_<.1:f (
\
I (
(5.17)
From this summary we can read off the estimated effects of changesin the explanatory variableson the dependentvariable and we can predict valuesofthe dependentvariable for given values of the explanatory variables. For the construction of an interval estimate we needthe least squaresestimate,its standarderror, and a critical value from the r-distribution. For a95Vointerval estimateand at least moderatedegreesof freedom the critical /-value is approximately 2. Thus, from the information in (5.17), an approximate 957o interval estimatecan be obtainedby mentally calculating the points two standarderrorseither side of the least squaresestimate. Similarly, the /-value usedto test a null hypothesisof the form Ho:$r: 0 is given by the ratio of the least squares estimate to its standard error that appears underneath. It too can be calculatedmentally by inspectionof (5.17). If the ratio of an estimateto its standard error is absolutely greater than 2 (approximately), you know that a null hypothesisof the form Hs:$k:0 would be rejectedat the cr: 0.05 level of significance.Furthermore,as discussedin Section 3.4.3, from an interval estimatewe can tell whether any hypothesisof the form I1s t}r: c would be rejectedor not in a two-tail is rejected. Remembering test. If c is within the interval, FIs is not rejected, otherwise ,F1s of these quick "inspection techniques" will help you make "on the spot" assessments reported results.
i
1i:
5.7EXERCISES
Table 5.4
ressionsoftware model does not 6) is no longer n intercept term
D a t a f o r E x e r c i s e5 . 1
0 1 2
I z 5
-1 0 Appendix 48 of ,^ ,i:1(}t - J)ei no l to talk of the 'hen your model rmputerdisplays
I -2 I 0 -1
a
1 -2
-l -N
127
z
0
I 2
1
-l I l
0
I
5.7 Exercises Answers to exercisesmarked * appear in Appendix D at the end of book. 5.7.1 ll the entries in res described in Latinga multiple equation using r the estimated : R2-value.
I
Pnosrprvrs
-5.1* Considerthe multiple regressionmodel li : xn9r I xiz9z -f xiz9z* ei with the nine observationson yi,xit,xi2 urd x3 given in Table 5.4. Use a hand calculator to answer the following questions: (a) Calculate the observationsin terms of deviations from their means.That is, find '-t z
(s.r7)
: xiz - 72,
x;7:x3-x3,
*-
ii:ii-j
(b) Calculate Zyi*iz,l;rt, }yi xh,bizris, andbi? . the explanatory ient variable for val estimate we re r-distribution. ritical l-value is .e 95Vo interval rrs either side of : 0 is given by lrs underneath. 1 estimate to its rw that a null 5 level of sigstimate we can rot in a two-tail . Remembering assessments of
(c) UsetheexpressionsinAppendix5Atofindleastsquaresestimatesbl, b2,andfu. (d) Find the leastsquaresresiduals21,22,.. .,2e. (e) Find the varianceestimateo2. (f; Use equation (5.9) to find the sample correlation between x2 and x3. (g) Find the standard enor for b2. (h) Find SSE,SSI SSR,and R2.
: . t * Use your answersto Exercise5.1 to (a) Compute a95Vo rntewal estimate for 82. (b) Test the hypothesis Hs: $2 : 1 against the alternative that H1 :92 * l.
ri
Consider the following model that relates the proportion of a household's budget spent on alcohol WALC to total expenditure TOTEXP, age of the household head AGE, and the number of children in the householdNK. fiALC : gr * B2tn(TOTEXp) + %AGE -t g+NK -r e The datain thefile london.daf were usedto estimatethis model. SeeExercise4.10 for more details about the data. Note that only householdswith one or two children are being considered.Thus, NKtakes only the values I or 2. Output from estimating this equation appearsin Table 5.5.
128
THE MULTIPLE
Table 5.5
REGRESSION MOI)EL
O u t p u t f o r E x e r c i s e5 . 3
Itble5.7
Dependent Yariable: WALC Included observations:1519 Variable
-)ependentVariab -:;luded observa
Std. Error
Coefficient
C tn(TOTEXP) AGE N/<
0.0091 0.0276
/-Statistic
0.0002 0.0033
Prob.
6.6086 -6.9624 -4.0750
0.6347 0.0000 0.0000 0.0000
Mean dependentvar var S.D. dependent
0.0606 0.0633
0.0191
-0.0133
R-squared S.E. of regression Sum squaredresid
(
5.752896
..:riable
.RI.VE "ITOX
ar)OMS tt-;E r/5f
1,(-cEss {.{ : IRITIO
(a) Fill in the following blank spacesthat appear in this table. (i) The /-statisticfor b1. (ii) The standard enor for b2. (iii) The estimatebr. (iv) R2. (v) 6. (b) Interpret each of the estimatesb2, fu, and ba. (c) Compute a957o intewal estimate for B3.What does this interval tell you? (d) Testthe hypothesisthat the budgetproportion for alcohol doesnot dependon the numberof childrenin thehousehold.Canyou suggesta reasonfor thetestoutcome? 5.4*
(e) Predic one-a sampl
This ques usesthe d Demand f 8 1 - 1 0 2 .T
VALUE: CRIME: NITOX: ROOMS = AGE : PT DIST:W ACCESS . TAX : fu PTRATIO
The datasetusedin Exercise5.3 is usedagain.This time it is usedto estimatehow the proportion ofthe householdbudget spenton transportationITIRANS dependson the log of total expenditureIn(TOTEXP),AGE, and number of children NrK.The output is reported in Table 5.6. (a) Write out the estimatedequation in the standardreporting format with standard errors below the coefficient estimates. (b) Interpret the estimatesb2, b3, andba.Do you think the resultsmake sensefrom an economicor logical point of view? (c) Are there any variables that you might exclude from the equation? Why? (d) What proportion of variation in the budget proportion allocated to transport is explained by this equation.
Table 5,6
O u t p u t f o r E x e r c i s e5 . 4
Dependent Variable: WRANS lncluded observations:1519 Variable
C Ln(TOTEXP) AGE NK R-squared
(a) Repor (b) Find ! (c.) Test t value (d) Test a will ir . . 1. 2
Coefficient
Std. Error
/-Statistic
-0.0315
0.0322 0.0071 0.0004 0.0055
-0.9776
0.04t4 -0.0001 -0.0130 0.0247
5.8561 -0. l 650 -2.3542 Mean dependentvar S.D. dependentvar
Compu
Prob.
0.3284 0.0000 0.8690 0.0187 0.1323 0.1053
Use a con a-
(a) The fi file lo equat
for al the de
5.7 EXERCISES
:tble 5.7
129
O u t p u t f o r E x e r c i s e5 . 5
- e;'endent Variable: UALUE --r.uded observations:506
ar r
Prob.
. i--.lable
0.6347 0.0000 0.0000 0.0000
-?t rtE ', TOX
0.0606 0.0633
i ,().US
Coefficient
Std.Error
t-Statistic
Prob.
28.4067 -0.1834 -22.8109
5.3659 0.0365 4.1607 0.3924 0.0141 0.2001 0.0723 0.0038 0.1394
5.2939 -5.0275 -5.4824
0.0000 0.0000 0.0000 0.0000 0.0008 0.0000 0.0002 0.0009 0.0000
6.3'115 -0.0478 - 1.3353
t,E
-.'tr
0.2'723 -0.0126 - l.1768
r.-CfSS
::-{ ; . R|TIO
16.23't8 -3.3861 -6.6'714 3.7673 -3.3399 -8.4409
(e) Predict the proportion of a budget that will be spent on transportation,for both one- and two-children households,when total expenditureand ageare setat their sample means,which are 98.7 and 36, respectively.
I tell you? depend on the testoutcome?
; -i This question is concernedwith the value of housesin towns surroundingBoston. It usesthe data of Harrison,D. and D.L. Rubinfeld (1978), "Hedonic Pricesand the Demand for Clean A:r," Joumal of Environmental Economicsand Management,5, 8l-102. The output appearsin Table 5.7. The variables are defined as follows: VALUE : median value of owner-occupiedhomes in thousandsof dollars, CRIME: per capita crime rate, NITOX: nitric oxide concentration (parts per million), ROOMS : ?v!rogenumber of rooms per dwelling, AGE : proportion of owner-occupied units built prior to 1940, DIST : weighted distancesto five Boston employment centers, ACCESS : index of accessibility to radial highways, TAX: full-value property-taxrate per $10,000,and pupil-teacherratio by town. PTMTIO:
imate how the ependson the . The output is with standard sensefrom an n? Why? o transport is
(a) Report briefly on how each of the variables influences the value of a home. ft) Find 95Vointewal estimatesfor the coefficients of CRIME and ACCESS. (c) Test the hypothesis that increasing the number of rooms by one, increasesthe value of a house by $7000. (d) Test as an alternativehypothesisF11that reducing the pupil-teacher ratio by 10 will increasethe value of a house by more than $10,000. :.i.2
Prob.
0.3284 0.0000 0.8690 0.0187 0.1323 0.1053
Conpurrn
ExsncrsEs
< 6 Use a computerto verify your answersto Exercise5.1, parts(c), (e), (0, (g), and (h). : - (a) The file lond_small.dot contains a subsetof 500 observationsfrom the bigger file london.dar. Use the data in the file lond small.daf to estimatebudeet share equations of the form w : gt -r g2tn(ToTEXp) + %AGE -t F+NK * e for all budget shares(food, fuel, clothing, alcohol, transportation,and other) in the dataset.Reportanddiscussyour results.In your discussioncommenton how
t a
$
130
THE MULTIPLE REGRESSION MODEL
total expenditure, age, and number of children influence the various budget proportions.Also comment on the significanceof your coefficientestimates. (b) Commoditiesare regardedas luxuries if 9, > 0 and necessitiesif B2 < 0. For eachcommoditygrouptest^F/e:Fz< 0 againstH1:$2) 0 and commenton the outcomes.
(c) (d)
. l 0 The
mid (a)
5.8 The file cocaine.datcontains56 observationson variablesrelatedto salesof cocaine powderin northeasternCalifornia over the period 1984-1991.The dataare a subset of thoseusedin the study Caulkins, J.P.and R. Padman(1993), "Quantity Discounts and Quality Premia for Illicit Drugs," Journal of the American Statistical Association. 88. 7 48-757. The variables are PRICE - price per gram in dollars for a cocaine sale, QUANT : number of grams of cocaine in a given sale, QUAL : quality of the cocaine expressedas percentagepurity, and TREND: a time variablewith 1984: 1 up to 1991: 8.
(b)
, . I t Rec
(a)
Consider the regressionmodel PRICE:
Fr * \2QUANT + fuQUALI
9qTREND-r e
(a) What signs would you expect on the coefficientsB2,B3, and Ba? (b) Use your computer software to estimate the equation. Report the results and interpret the coefficient estimates.Have the signs turned out as you expected? (c) What proportion of variation in cocaineprice is explainedjointly by variation in quantity, quality, and time? (d) It is claimed that the greater the number of sales,the higher the risk of getting caught.Thus, sellersare willing to accepta lower price if they can make salesin larger quantities. Set up I1s and H1 that would be appropriate to test this hypothesis. Carry out the hypothesis test. (e) Testthe hypothesisthat the quality ofcocaine hasno influenceon price against the alternative that a premium is paid for better quality cocaine. (f) What is the averageannual changein the cocaine price? Can you suggestwhy price might be changingin this direction? 5 .9*
Data on per capita consumptionof beef, the price of beef, the price of lamb, the price of pork, andpercapitadisposableincomeforAustralia,forthe period 1949-1965,are given in the file meat.dat.All prices and income have beendeflatedwith 1953as the baseyear. Considerthe log-log demandcurve
(b) (c)
I
Eac Can driv, lighr Mur worl DEI (a)
II(QB1): fu * $2ln(PB,) * B3ln(Pr,) * Baln(PP,)* Bsln(INr)* e, Becausethe data are observationsover time, we use a / subscript instead of an I subscript. The variables are defined as QB1is per capitaconsumptionof beef in year / (pounds), PBr is the price of beef in year / (penceper pound), Pt is the price of lamb in year t (pence per pound), PPr is the price of pork in year t (pence per pound), and IN1 is per capita disposableincome in year t (Australiancurrencypounds). (a) What signs do you expecton each of the coefficients? (b) EstimateFz, Bl, and Ba using leastsquares.Interpretthe results.Do they seem reasonable?
(b) (c)
(d) (e)
5.7 EXERCISES
the various budget :fficient estimates. sitiesif Bz ( 0. For Lndcomment on the
(c) Compute and interpret the estimated covariance matrix for the least squares estimator and the standarderrors. (d) Compute 95Vointerval estimatesfor each of the parameters.
: 1 0 The file brdat containsdata on 1080 housessold in Baton Rouge, Louisiana, during mid-2005. (a) Estimatethe regressionmodel PRICE : 9r * !ySQFT -t \yAGE * e for (i) all the housesin the sample,(ii) town houses,and (iii) French style homes.Construct a95Vo interval estimate for $3 in each case and discussthe differences. (b) For each of the casesin part (a), test the hypothesisthat having an older house reducesthe price by $1000per year for eachyear ofits age.Use a two-tail test with an cr : 0.05 level of significance.
J to salesof cocaine 'he data are a subset Quantity Discounts Statistical Associa-
. and
131
. I l * Reconsiderthe presidential voting data (fairdat) introduced in Exercise 2.14. (a) Estimate the regressionmodel voTE : h -l \zGROWTH + $3INFLATION * e Report the results in standard format. Are the estimates for B2 and 9: significantly different from zero at a lOVosignificance level. Did you use one-tail testsor two-tail tests?Why? (b) Predict the percentagevote for the incumbent party when the inflationrate is 4Vo and the growth rate is -4To. (c) Supposethattheinflationrateis4%oandthegrowthrateis-4Yo.Alsoassumethat 9r : h and B3 : bs. For what values of B2 will the incumbent party get the majority of the vote. Using this range of values as the null hypothesis,test the hypothesisthat the incumbent party will get the majority of the vote againstthe alternative that it will not.
le td Ba? port the results and rt as you expected? intly by variation in r the risk of getting :y can make salesin opriate to test this { il nceon price against :aine. rn you suggestwhy :e of lamb, the price i,od1949-1965,arre ted with 1953asthe
Each morning between 6:30 AM and 8:00 AM Bill leavesthe Melbourne suburb of Carnegie to drive to work at the University of Melbourne. The time it takes Bill to drive to work (TIME) dependson the departuretime (DEPART), the number of red lights that he encounters(REDS),and the number of trains that he hasto wait for at the Mumrmbeena level crossing (IRAINS). Observationson thesevariablesfor the 231 working days in 2006 appearin the fi le commute.dat. TIM E is measuredin minutes. DEPART is the number of minutes after 6:30 AM that Bill departs. (a) Estimate the equation TrME : h -t FzDEpART + q3REDS* p+?RAINS * e
(rN,)+ e, :ript instead of an i
rcy pounds). sults.Do they seem
Report the results and interpret each of the coefficient estimates,including the interceptB1. (b) Find 95Vo interval estimates for each of the coefficients. Have you obtained precise estimatesof each of the coefficients? (c) Using a 5Vo significance level, test the hypothesis that each red light delays Bill by 2 minutes or more against the alternative that the delay is less than 2 minutes. (d) Using a lUVosignificancelevel, test the hypothesisthat eachtrain delays Bill by 3 minutes. (e) Using a 5Vo significancelevel, test the null hypothesisthat leaving at 7:30 AM insteadof 7:00 AM will make the trip at least 10 minuteslonger (other things equal).
132
THE MULTIPLE REGRESSION MODEL
(f) Using a 57o significancelevel test the hypothesisthat the minimum time it takes Bill is less than or equal to 20 minutes againstthe alternativethat it is more than 20 minutes.What assumptionsabout the true valuesof 92, F:, and Ba did you have to make to perform this test? 5.13* The file rice.dat contains352 observationson 44rice farmers in the Tarlac region of the Philippines for the 8 years 1990 to 1997. Variables in the data set are tonnes of freshly threshed rice (PROD), hectaresplanted (AREA), person-daysof hired and family labor (I,ABOR), and kilograms of fertilizer (FERT). Treating the data set as one sample with N : 352, proceed with the following questions: (a) Estimate the production function
(b) (c)
Appen
. . .\ppend -.nple reg - .iltiplerr : rlel witl : : , . b l e mi s
-t gln(tABOR) + p+h(rE'nT) + e In(PROD): Br * B2Ln(AREA) Report the results, interpret the estimates, and comment on the statistical significance of the estimates. (b) Using a l%o level of significance,test the hypothesisthat the elasticity of productionwith respectto land is equal to 0.5. (c) Find a 95Vointerval estimate for the elasticity of production with respect to fertilizer. Has this elasticity been precisely measured? (d) Using a 57o level of significance, test the hypothesis that the elasticity of production with respectto labor is lessthan or equal to 0.3 againstthe alternative that it is greater than 0.3. What happensif you reversethe null and alternative hypotheses? 5.14a Reconsiderthe productionfunction for rice estimatedin Exercise5.13. (a) Predict rice production from t hectare of land, 50 person-daysof labor, and 100 kg of fertilizer. (b) Your economic principles suggest that a farmer should continue to apply fertllizer as long as the marginal product of fertilizer APRODIAFERT is greater than the price of fertilizer divided by the price of output. Supposethat this price ratio is 0.004. For FERT : 100 and the predicted value for PROD found in part (a), show that the farmer should continue to apply fertilizer as long as Ba>0.1242. (c) Using Ht:9c>0.1242 as the alternativehypothesis,test whether the farmer should apply more fertilizer. Use a 57o significance level. Why did we choose 94> 0.1242 as the alternativerather than the null hypothesis? (d) Find a 957o interval estimate for rice production from t hectare of land, 50 person-daysof labor and I kg offertilizer.
- :.e first stt .jer partir
:::ting the
-.s leastsc -:tltaneo
. ..rtionit ir
--.r'n the le
5.15 Considerthe following aggregateproductionfunction for the U.S. manufacturing sector
y, : .*t('r'nlo ttf'exp{e,} wheree, - N(0, o27,Y, is grossoutput in time r, K, is capital, t, is labor, E, is energy. andMl denotesother intermediatematerials.The dataunderlying thesevariablesare given in index form in the file manuf.dat. (a) Show that taking logarithms of the production function puts it in a form suitable for least squaresestimation.
t : models ' .trix alge :nputatio : -::mates.
APPENDIX
mum time it takes hat it is more than 3, and Ba did you
LeTarlac region of r set are tonnes of days of hired and ing the data set as
Appendix 5A Derivation of Least SquaresEstimators In Appendix 2Awe derived expressionsfor the least squaresestimatorsb1 and b2 inthe simple regression model. In this appendix we proceed with a similar exercise for the multiple regressionmodel; we describe how to obtain expressionsfor fu,b2, and b3 in a model with two explanatory variables. Given sample observationson ltx2t and x:, the problem is to find values for 9r, Fz, and B3 that minimize
s(Pr,92,Pr) : i (y,- Fr - gzxiz- gzxiz)z i:l
"FERT).t on the statistical
,n with respect to the elasticity of Lnstthe alternative ill and alternative
The first stepis to partially differentiate S with respectto B1,92, and B3 and to set the firstorder partial derivatives to zero. This yields
0s
: zrBt *292L'ciz*29zLriz-22Yi aBu
0s fr:'Br>.iz
0s : e 5.13. Laysof labor, and :ontinue to apply t)FERT is greater rosethat this price r PROD found in tilizer as long as rether the farmer hy did we choose :? ,ctare of land, 50
133
OF LEAST SQUARES ESTIMATORS
(b) Estimate the unknown parameters of the production function and find the corresponding standarderrors. (c) Discuss the economic and statistical implications of these results.
e
the elasticity of
5A DERIVATION
#
-t 2B2ls2^ - ZLxizvi * 2ls>-xizxtz
'BtOiz *2gzLxizxiz -12$3\4?'- 2lsizti
Settingthesepartialderivativesequalto zero,dividing by 2, andreanangingyields Nh *Lxizbz*bnb3:ly, l*r izh * *rU, * Zxizxizbz: b ;2yi bizb t + \xaxizbz + b%h : L\3y ;
(5A.1)
The least squaresestimatorsfor fu,b2, and b3 aregiven by the solution of this set of three simultaneousequations, known as the normal equations. To write expressionsfor this 'olution it is convenientto expressthe variablesas deviationsfrom their means.That is, let li : Ii - l,
xi2: xi2 - xZ)
xiz : xB - x3
Then the least squaresestimatesb1,b2,a\d b3 are bt:!-bz7z-bzlz
S. manufacturing
b2:
(}yi,D (bi?) - (Iyix,lXbir*ir)
- (bi,,i)' (>r*)(>ri?)
(Irixir)Qrin - (rvlx,tz)(Ixhxb) " _ (>i;)(Ixt') - (L'b*f)" rbor,E, is energy, resevariablesare .n a form suitable
For modelswith more thanthreeparametersthe solutionsbecomequite messywithout using matrix algebra; we will not show them. Computer software used for multiple regression computationssolves normal equationslike those in (5A.1) to obtain the least squares estimates.
Chapter
Further Inference in the Multiple Regression Model
Learning Objectives Based on the material in this chapter, you should be able to 1. Explain the conceptsofrestricted and unrestrictedsumsof squarederrors and how they are used to test hypotheses. 2. Use the F-test to test single null hypothesesor joint null hypotheses. 3. Use your computer to perform an F-test. 4. Testthe overall significanceof a regressionmodel, and identify the componentsof this test from your computer output. 5. Explain how to test a single null hypothesis involving two or more coefficients when (a).F/1is a "not equal to" hypothesisand (b) I11is a "greater than" or "less than" hypothesis. 6. Fromoutputofyourcomputersoftware,locate(a)thesumofsquarederrors,(b)the F-value for the overall significance of a regression model, (c) the estimated covariance matrix for the least squaresestimates,and (d) the correlation matrix for the explanatory variables. 1 . Obtain restrictedleast squaresestimatesthat include nonsampleinformation in the estimation procedure. 8. Explain the properties of the restricted least squaresestimator.In particular, how does its bias and variance compare with those of the unrestricted least squares estimator?
Keywords
auxiliary regres collinearity F-test irrelevant varial nonsampleinfo omitted variabl
Economistsdev ceduresare usec consistingof a r This analysisca a single restrict restinga null hy lor thesetestsar ro test a null hy The theories canbe usedalo regressionmod restricted least .i condition call procedr rQu&rOS lhesetopics we rnd the constn iunctional forn: :ssuesrelated t irappensif we it .lre not sufficie The assump In particular, w, :he r- and F-tes arrorsarenot n( :hey hold apprc
5.1 The I
9. Explain the issuesthat need to be consideredwhen choosing a regressionmodel. 10. Explain what is meant by (a) an omitted variable and (b) an irrelevant variable. Explain the consequencesof omitted and irrelevant variablesfor the properties of the least squaresestimator. 11. Explain what is meant by collinearity and the consequencesfor least squa.res estimation. 12. Explain how the RESET test can pick up model misspecification.
134
In Chapter 5 wr rultiple regres :han one pararr .i explanatory ...cioeconomi ;.ed to explain rrices of subst :estsinvolving -i variables,ar :.ndincomego -:nchanged?
6.1 THE F-TEST
135
Keywords
ilel
luared errors and hou, rpotheses. ify the componentsof or more coefficients lreaterthan" or "less squarederrors,(b) the :1, (c) the estimated he correlation matrix ple information in the or. In particular, how stricted least squares
auxiliary regressions collinearity F-test irrelevant variable nonsampleinformation omitted variable
omitted variable bias overall significance RESET restrictedleast squares restricted model
restricted SSE single and joint null hypotheses testing many parameters unrestricted model unrestricted SSE
Economistsdevelopand evaluatetheoriesabouteconomicbehavior.Hypothesistesting pro;eduresare usedto test thesetheories.In Chapter5 we developed /-testsfor null hypotheses ;onsisting of a single restriction on one parameterBpfrom the multiple regressionmodel. This analysiscan be extendedin two ways.We may encountera null hypothesisconsistingof .i single restriction that involves more than one parameter,or we may be concernedwith iestinga null hypothesiswith two or more restrictionson two or more parameters.The tools :or thesetestsareconsideredin this chapter.An important new developmentis the F-test used :rr test a null hypothesiswith two or more restrictions on the parameters. The theories that economists develop sometimesprovide nonsample information that .'an be used along with the information in a sample of data to estimate the parametersof a :e-eression model. A procedurethat combines these two types of information is called restricted least squares.It can be a useful techniquewhen the dataarenot information-rich, r condition called collinearity, and the theoretical information is good. The restricted least \quaresprocedurealsoplays a usefulpracticalrole when testinghypotheses.In additionto :hesetopics we discussmodel specificationfor the multiple regressionmodel, prediction, rnd the constructionof prediction intervals. Model specificationinvolves choosing a functional form and choosing a set of explanatory variables. In this chapter we focus on rssuesrelated to variable choice. What happensif we omit a relevant variable?What happensif we include an irrelevant one?We also discussthe problems that arise if our data r.renot sufficiently rich becausethe variables are collinear or lack adequatevariation. The assumptionsMR1-MR6 listed in Section5.1 are adoptedthroughoutthis chapter. In particular, we assumethe errors are normally distributed. This assumptionis neededfor the t- and F-test statisticsto have their required distributions in samplesof all sizes.If the .rrors arenot normal, then the resultspresentedin this chapterare still valid in the sensethat they hold approximatelyif the samplesize is large.
6.1 The F-Test 3 a regressionmodel. n irrelevant variable. i for the properties of :es for least squares ation.
In Chapter 5 we learned how to use /-tests to test hypothesesabout single parametersin a multiple regressionmodel. There are, however,many instanceswhere testsinvolving more than one parameterare appropriate.For example, we might want to test whether a group rrf explanatory variables should be included in a particular model. Should variables on .ocioeconomicbackground,along with variablesdescribingeducationand experience,be usedto explain a person'swage?Does the quantity demandedof a productdependon the pricesof substitutegoodsor only on its own price?Other questionsthat lead to hypothesis testsinvolving more thanone parameter,but do not involvetestingthe relevanceof a group ofvariables,are:Does a productionfunction exhibit constantreturnsto scale?Ifall prices andincomego up by the sameproportion,will quantitydemandedfor a commodityremain unchansed?
136
FURTHER
INFERENCE
IN THE MULTIPLE REGRESSION MODEL
We distinguish between a single null hypothesis that is a null hypothesiswith a single restriction on one or more parametersand a joint null hypothesis that containstwo or more restrictions on two or more parameters.What we discover in this chapter is that a two-tail test for a single null hypothesiscan be conductedvia a t-test or an F-test. The two testsare equivalent.A one-tailtestfor a singlenull hypothesismustbe testedvia a t-test.The F-test must be used to test a joint null hypothesis. The F-test forajoint null hypothesisis basedon acomparisonofthe sum ofsquarederrors from the original, unrestrictedmultiple regressionmodel with the sum of squarederrorsfrom a regressionmodel in which the null hypothesisis assumedto be true. To illustrate what is meantby an unrestrictedmultiple regressionmodel and a model that is restrictedby the null hypothesis,considerthe Big Andy's Burger Barn examplewhere sales(,S)are a function of a price index of all products sold (P) and expenditureon advertising (A) si : Fr -t gzPi -t gtAi -t ei
( 6 . 1)
Supposethat we wish to test the hypothesisthat changesin price have no effect on sales revenue against the alternative that changes in price do have an effect. The null and alternativehypothesesarcH0: 9z : 0 and.F11: 9z *0. The restrictedmodel,which assumes the null hypothesisis true. is Si : Br t }tAi -t ei
16.2t
SettingFz : 0 in the unrestricted model in (6.1) meansthat the price variableP; doesnot appearin the restricted model in (6.2). When estimating a model where a null hypothesisis assumedto be true, we place conditions, or constraints,on the valuesthat the parameterscan take. Insteadof finding least squaresestimatesthat minimize the sum of squarederrors, we find estimatesthat minimize the sum of squarederrors subjectto parameter constralnrs.The imposition of the parameter constraints means the sum of squared errors will increase; a constrained minimum is larger than an unconstrainedminimum. Thus, the sum of squarederrors from (6.2) will be larger than that from (6.1). The idea of the F-test is that if these sums of squarederrors are substantiallydifferent, then the assumptionthat the null hypothesisis true has significantll reduced the ability of the model to fit the data, and thus the data do not support the null hypothesis.Conversely,if the null hypothesisis true, we expectthat the data are compatible with the conditions placed on the parameters.Thus, we expect little changein the sum of squarederrors when the null hypothesisis true. The sum of squarederrors in a model that assumesa null hypothesisis true is called the restricted sum of squared errors or SSEa;the subscript R indicates that the parameters have been restrictedor constrained.To make a clear distinction betweenthe restricted sum of squarederrors and the sum of squarederrors from the original, unrestrictedmodel, the sum of squarederrors from the original model is called the unrestricted sum ofsquared errors or SSE6'.It is aiu'a_r's true that SSEa- SSEy > 0. The null hypothesiscan be a single null hypothesisor a joint null hypothesis.Let "/ be the number of restrictionsin the null hypothesis.For example,for testingthe null hypothesisthat led to the restrictedmodel in (6.2),J: 1. The generalF-statisticis given by F_
- ssBu)ll (s.sER
ssEul(N - K)
(6.3)
If the null hypothesisis true, then the statisticF has what is called an .F-distributionwith J numerator degrees of freedom and N - K denominator desrees of freedom. Some
6.1 THE F.TEST
a single or more two-tail testsare reF-test :d errors rrs from what is the null tion of a
( 6 . 1, rn sales rull and
Itails about this distribution are given in Appendix B.5.4, with its typical shapeillustrated r Figure8.8; the reasonwhy the expressionin (6.3) has an F-distributionis given in an rrrrendix to this chapter,Appendix 6A. If the null hypothesisis not true, then the difference =t*een SSE6andSSEybecomes large,implying thattheconstraintsplaced onthe modelby = null hypothesishave a large effect on the ability of the model to fit the data.A large value r.r SSE4- SSEymeansthe valueof Ftends tobe large.Thus,we rejectthenull hypothesis r '.ite value of the F-test statistic becomes too large. What is too large is decided by :..mparingthe valueof Fto a critical valueF,, which leavesa probabilitya in the uppertail :t :he F-distribution with ./ and N - K degreesof freedom. Tables of critical values for r , 0.01 and ct : 0.05 are provided in Tables4 and 5 at the end of the book. The rejection r:::on where F ) F, is illustrated in Figure B.8. L-sing the hypothesis testing steps introduced in Chapter 3, the F-test procedure for r-::ns whether price should be excluded from the sales equation is as follows: .
Specifythe null and alternativehypotheses: Hs:$2:0
i
Specify the test statisticand its distribution if the null hypothesisis true: Having one restrictionin lle means./: l. Also, recall that N: 75, so the distributionof the F-test statistic when I1e is true is
ISSUInC\
(6.1, F_
loesnor e place ng least inimize rameter llum
rn with Some
Fg.tz1
Calculate the sample value of the test statistic and, if desired, the p-value: For the unrestricted and restricted models in equations (6.1) and (6.2), respectively, we find
l:
(6.3r
- ssEu) (,ssER 11ssBu lQs - 3)
and H1:$210.
Seta and determinethe rejection region: Using cr : 0.05, the critical value from the :3.97. Thus, Ilo is rejected lf F > 3.9'1. Flr,zzy-distribution is F, : F1o.es,t,tz) Alternatively, FIs is rejected if p-value < 0.05.
will be rors are ficantlr Lhenull tpatible sum of lled the tmeters ed sum lel, the quared r single he null odel in
137
: 2961.827 SSEu: 1718.943, S,SER Imposing the null hypothesisHo:92:0 on the model has increasedthe sum of squarederrors from 1718.943to 2961.827. The value of the F-test statistic is
F:
- ssEu)lJ (2e6r.827 - r7r8.e43) (ssER I 1:52.06 ssEu/(N K) r7r8.e43 3) /(7s
Thep-value for the test is p : f lFlr,tz) > 52.06] : 0.0000, correct to four decimal places. '
State your conclusion: Since F:52.06)F,, we reject the null hypothesisand concludethat price doeshave a significant effect on salesrevenue.Alternatively, we reject I1e becausep : 0.0000 < 0.05.
: 3.97 camefrom. The F critical valuesin : .- might ask wherethe valueF, : F1g.9s,r,tz) liries 4 and 5 at the end of the book are reported for only a limited number of degreesof ::rdom. However,exact critical valueslike the one for this problem can be obtainedfor any :,-nber of degreesof freedom using your statistical software.
138
6.1.1
FURTHER
INFEIIENCE IN THE MULTIPLE REGRESSION MODEL
THs RErATroNSHrpBsrvGsN /- AND F-Tpsrs
Do you rememberthat we useda /-testto testI/s : 9z : 0 againstHr:92l0 in Chapter: Why are we now using an F-test? What is the relationshipbetweenthe t- and F-te:t. When testinga single "equality" null hypothesis(a singlerestriction)againsta "not equto" alternativehypothesis,eithera /-testor an F-testcan be usedand the testoutcomes\\ be identical.The reasonfor this correspondence is an exactrelationshipbetweenthe /- a: F-distributions.The square of a /-random variable with m degreesof freedom is F-randomvariablewith one degreeoffreedom in the numeratorand mdegreesoffreed' ' . in the denominator.It has distribution Fg,^1.When using a /-test for .F1s : Fz : 0 a_sai: -1.215 Hr :92 10, we found that / : and t, : 1.993.The F-value that we calculatec F :52.06: t2 : (-7.215D2and the correspondingcritical value is Fc:3,97 : r(1.9%)2. Becauseof this exactrelationship,thep-valuesfor the two testsare identi.meaningthat we will always reachthe sameconclusionwhicheverapproachwe take There is no equivalencewhen using a one-tail t-test sincethe F-test is not appropr:. when the alternativeis an inequalitysuchas ">" or "<". The equivalencebetweent-rr.and F-testsalso doesnot carry over when a null hypothesisconsistsofmore than a sin: restriction.Underthesecircumstances, where./ > 2, the /-testcannotbe used,but an F-te.: available.Examplesof F-testswhere"/ ) 2 aregiven in the next two sectionsof this chap:. We can summarizethe elementsof an F-test as follows: L
The null hypothesisF10consistsof one or more equality restrictions.The nun:-, of restrictionsis denotedby -/. When "/: 1, the null hypothesisis called a sir.null hypothesis.When J > 2, lt is called a joint null hypothesis.The null hr:' thesismay not include any "greater than or equal to" or "less than or equal : hypotheses.
2. The alternative hypothesis states that one or more of the equalities in the :
a J. A t.
hypothesisis not true.The alternativehypothesismay not includeany "greaterth-' or "less than" options. The test statisticis the F-statisticin (6.3). If the null hypothesisis true, F has the F-distributionwith "I numeratordegrec. freedomandN - K denominatordegreesof freedom.The null hypothesisis r2.1, if F > F,, whereF. : F1t-o,.r,ru-r;is the critical value that leavescrpercent(r: probability in the upper tail of the F-distribution.
5 . When testinga singleequalitynull hypothesisit is perfectlycorrectto useeithe: ' t- or F-testprocedure. They areequivalent. In practice.it is customaryto test.:: restrictionsusing a 1-test.The F-test is usually reservedforjoint hypotheses.
6.2 Testing the Significance of the Model An importantapplicationof the F-testis for what is calledtestingthe overallsignifican.. amodel.In Section5.5.l, we testedwhetherthedependent variableyis relatedto aparti.explanatoryvariable-rr usinga /-test.In this sectionwe extendthis idea to a joint test(): relevanceof all the included explanatoryvariables.Consideragain the generalmul:.: regressionmodel with K - 1 explanatoryvariablesand K unknown coefficients )i : 9r * xiz9z*xi:B: + "' + xix9x-f ei
)EL
6.2 TESTING THE SIGNIFICANCE
OF THE MODEL
139
lr examinewhether we have a viable explanatorymodel, we set up the following null and r:ernative hypotheses Chapter 5 i nd F-testsl "not equa:comesu,il. n the t- anc :dom is ar of freedorr= 0 agains: rlculatedi' O ? -- ' l - L c
. t ,
-
e identical. we take. rppropriat< veen/-tesl! .ana single an F-testi. hischapter
he numbe: ed a single null hypc: equal to" in the nul. 3aterthan"
Ho:92:0,9: - 0,...,9r :0 H 1: At leastoneof theB1is nonzerofor k : 2, 3, . . ., K
l:e null hypothesisis ajoint onebecauseit hasK - 1 components.It statesas a conjecture -t eachand every one of the parameters81, other than the intercept parameterB1,is zero. -: ihis null hypothesis is true, none of the explanatory variables influence y, and thus our -.rlel is of little or no value. If the alternativehypothesis^F1q is true, then at least one of the :u-ametersis notzero, and thus one or more of the explanatoryvariablesshouldbe included :--:he model. The alternativehypothesisdoes not indicate, however,which variablesthose ::Eht be. Since we are testing whether or not we have a viable explanatory model, the test ''r t6.5) is sometimesreferred to as a test of the overall significance of the regression ndel. Given the l-distribution can only be usedto test a single null hypothesis,we use the ' :i'st for testingthe joint null hypothesisin (6.5). The unrestrictedmodel is that given in (6.4). The restrictedmodel, obtainedassumingthe null hypothesisis true, is "-;ration
ificanceof Lparticular testof the Llmultiple
(6.6)
\:Bt*e;
l:e leastsquaresestimatorof B1in this restrictedmodel is bi : }Lry t I N : y, which is the .:-:rple mean of the observationson the dependentvariable. The restricted sum of squared :-..rs from the hypothesis (6.5) is N
: 21r' - D ,r*)r 2 SSE4 i:l
N^
:
Z,(ti -l)'
: SSZ
' :i'tisone case, in which we are testing the null hypothesisthat all the model parameters ir. zeroexceptthe intercept,the restricted sum ofsquared errors is the total sum ofsquares 'r-ft from the original unconsffainedmodel. The unrestricted sum of squarederrors is, i. :'efore, the sum of squarederrors from the unconstrainedmodel. That is, SSEy : 559. l:c numberof restrictions is .I : K - |. Thus, to test the overall significanceof a model, ---r-F-test statistic can be modified and written as
degreeso: is rejectec cent of the : eitherthe test single heses.
F:
(ssz- ssE) /( I( -l) ssE/(N - 1()
(6.7)
--:c calculated value of this test statistic is compared to a critical value from the ' r r..v-,<)-distribution. It is used to test the overall significanceof a regressionmodel. ::-: outcome of the test is of fundamental importance when carrying out a regression r---lr sis,and it is usually automaticallyreportedby computersoftwareas the "F-value". To illustrate, we test the overall significance of the regression used to explain Big r--J)'s salesrevenue.We want to test whether the coefficients of price and of advertising :r:enditure are both zero, againstthe alternativethat at leastone ofthese coefficients is not -::o. Recallingthat the model is Si : 9r -t 9zPi * B:Ai + ei,the hypothesistesting steps r: as follows: We want to test Hs:$2: Q, Fl : 0
,S
against the alternative
(6.4,
(6.5)
Hr:9zl 0orB3f 0, or botharenonzero
140
FURTHER
INFERENCE
2 . If F1ois true F :
IN THE MULTIPLE REGRESSION MODEL
( s s r - s s E ) / ( 3 - -1 ) ssE/(7s- 3)
F,r rrr.
U sing a 5 Vosignificancelevel, we fi nd the critical value for the F-statistic with (2,72) degreesof freedom is ft : 3.12. Thus, we reject H0 if F > 3.12.
4 . From Section5.6, we have S,SZ: 3115.485and SSE :1718.943, which give an F-value of F-
(ssr-ssE)/(I(r) (31rs.48s- r718.943) 12: ssE/(N - rK)
-3) r7r8.e43lQs
29.2s
Also, p-value : PIF > 29.251: 0.0000,correct to four decimal places. 5. Since29.25>3.I2,wereject 116andconcludethattheestimatedrelationshipisa significant one. A similar conclusion is reachedusing the p-value. We conclude that price or advertising expenditure or both have an influence on sales.Note that this conclusion is consistentwith conclusionsreachedusing separate/-testsfor testing the significanceof price and the significanceof advertisingexpenditurein S e c t i o n5 . 1 . 1 . Go back and check the output from your computer software.Can you find the F-value29.25 and the correspondingp-value of 0.0000 that form part of the routine output?
6,3 nn Extended Model
: IC URE
\dding .rf expec rising.S ,-hangei
So far in Chapters5 and 6 we havehypothesizedthat Big Andy's salesrevenueis explained by product price and advertising expenditures,
s:gr *gzP_lgzA_le
(6.8)
One aspectof this model that is worth questioningis whetherthelinearrelationshipbetween revenue,price, and advertising expenditure is a good approximation to reality. Having a linear model implies that increasingadvertisingexpenditurewill continue to increasesales revenueat the samerate irrespectiveofthe existing levels ofsales revenueand advertising expenditure.That is, the coefficient B3,which measuresthe responseofE(S) to a changein A, is constant; it does not depend on the level ofA. In reality, as the level of advertising expenditureincreases,we would expectdiminishing returnsto setin. An extra $100 spenton advertising is likely to have a bigger marginal impact on saleswhen the level of advertising is, say,$600, than when it is $1600. To illustrate what is meantby diminishing returnsconsiderthe relationshipbetweensales and advertising (assuminga fixed price) graphedin Figure 6. I . The figure showsthe effect on sales of an increase of $200 in advertising expenditure when the original level of advertisingis (a) $600 and (b) $ 1600. Note that the units in the grapharethousandsofdollars and so thesepointsappearas 0.6 and 1.6.At the smallerlevel of advertising,salesincrease from $72,400 to $74,000,whereasat the higher level of advertising the increaseis a much smallerone,from $78,500to $79,000.The linear model in (6.8) with the constantslopeB3 will not capture thesekinds of diminishing returns. What is required is a model where the slopechangesasthe level ofA increases.One suchmodel with this characteristicis obtained by including the squaredvalue of advertising42 as anotherexplanatoryvariable, making the new model
s:gr
IgzP*F:A*gqAz+e
(6.e)
To find t a chang diminis Havi estimat r ariable
where
The let
What c negati coeffic dimini in terrr
)EL
6.3 AN EXTENDED
MODEL
141
80
wirh(2,72) 78
ch give an
7'7 SALES 76
74
;. rnship is a : conclude Note that t-tests for :nditurein alue29.25
72 7l
0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 2.O 2.2 rrcunn 6.7
ADVERT A modelwheresalesexhibitsdiminishingreturnsto advertisingexpenditure.
Adding the term BaA2 to our original specification yields a model in which the response of expectedrevenueto a changein advertising expendituredependson the level of advertising. Specifically, using a little calculus, it can be shown that the responseof E(S) to a chansein A is
AA
(6.8) p between Having a easesales lvertising changein lvertising ) spenton lvertising reensales the effect I level of ofdollars ; increase Lsa much ;slopeB3 vhere the obtained aking the
(6.e)
_ aE(s): 9 2
aE(s)
explained
(Ptt"ta constant)
AA
-t29qA
(6.10)
To find the expectedsignsfor B3and B4,note that we expectthe responseof salesrevenueto a changein advertisingto be positivewhenA :0. That is, we expectF: > 0. Also, to achieve diminishing returns the responsemust decline as A increases.That is, we expect 9+ < 0. Having proposed an extendedmodel that might be more realistic, our next step is to estimate it. For estimation purposes the squared value of advertising is 'Just another rariable." That is, we can write (6.9) as Yi : 9t * 1zxiz l9zxit * Fqxi+I et rr'here /i : Si,
xiz:
Pr,
xrz: At, xic : A?
The least squaresestimates,using the data in the file andy.dat, are
- 7.640P Si : 109.72 i + l2.l51Ai- 2.768A7 ('")
(6.80)(1.046)
(3.s56) (0.e41)
(6.1l)
What can we say about the addition of A" to the equation?Its coefficient has the expected negativesign andis significantly different from zero at a 5Vosignificancelevel. Moreover.the coefficient of A has retained its positive sign and continues to be significant. Allowing for diminishing returnsto advertisingexpenditurehasimproved our model both statisticallyand in terms of meeting our expectationsabout how saleswill respondto changesin advertising.
142
FURTHER
INFERENCE
IN THE MULTIPLE
REGRESSION MODEL
From(6 ..rlesre'
6.4 Testing Some Economic Hypotheses As well as being more realistic, the extended model of the previous section provides a vehicle for testing some interesting economic hypotheses,illustrating the use of l- and F-tests in economic analysis.The first hypothesisrelatesto the significanceof advertising.
6.4.I
The ma :reparir ,'osts,a( : r o m$ 1
THs SrcNrprcANcEor ApvrnrrsrNc
ln the context of our extendedmodel.
si : Br -t lzPi * p:Ai * \qAi -t ei
(6.r2) * ith Ao
how do we test whether advertising has an effect upon salesrevenue?If either B3 or Bais not zero, then advertisinghas an effect on revenue.Basedon two separateone-tail t-testswe can conclude, in turn, that B3 is not zero and Ba is not zero, and in both cases they are of the correct sign. But the question we are now asking involves testing both B3 and Ba at the sametime. A joint test is appropriate.The joint test usesthe F-statistic in (6.3) to test Ho:9s : 0, Fo : 0. We will comparethe unrestrictedmodel in (6.12) to the restricted model, which assumesthe null hypothesis is true. The restricted model is
(6.13)
Si:9r*lzPt*ei
:nd Ba
Solving !1014. Big, :his am .ufficie :lterna
The steps for carrying out the test are as follows:
The joint null hypothesisis 11e:9:: 0, 9+ : 0. The alternativehypothesisis h:92 l0 or Bal0 or botharenonzero. 2 . The test statisticandits distributionif the null hypothesisis true are l.
F:
- ssEu) (ssER 12ssEu /(7s- 4)
Fpll
\fter c
This h,
3 . Selectinga5Vo significancelevel, we find that the critical value is F, : Fp.ss,z,tt1: 3.126, giving a rejectionregion of F > 3.126. 4 . The sum of squarederrors from (6.12) is S^tEu: 1532.084; the sum of squared errorsfrom (6.13)is SSEa: 1896.391,leadingto a valuefor the teststatisticof F : 8.44.The corresponding p-valueis P [F1z.zr; >8.44]: 0.0005. :8.44> F, 5 . Since F 3.126, we reject the null hypothesisthat both 9: : 0 and 9+:0 and concludethat at leastone ofthem is not zero.Advertising doeshave a significanteffect upon salesrevenue.The sameconclusionis reachedby noting that p-value:0.0005<0.05.
6.4.2
Tnr
Oprrruar Lpvsr or AovsRTrsrNc
Economic theory tells us that we should undertakeall those actionsfor which the marginal benefit is greater than the marginal cost. This optimizing principle applies to Big Andy's Burger Barn as it attempts to choose the optimal level of advertising expenditure.
taS a 1 lhe de
\ppen
The e calcu
6.4 TESTING SOME ECONOMIC
HYPOTHESES
143
From (6.10)the marginalbenefitfrom anotherunit ofadvertisingis the increasein expected salesrevenue: )vides /- and tising.
: Fz-t zfr+A
Ag9 AA
1en.taconstant)
The marginalcostof anotherunit of advertisingis the costof the advertisingplus the costof preparing the additional products sold due to effective advertising. If we ignore the latter costs,advertisingexpendituresshouldbe increasedto the point wherethe marginalbenefit tiom $1 of advertisingfalls to $1, or where I 9z*29qAo:1
(6.r2) 9: or re-tail
* ith A, denoting the optimal level of advertising.Using the least squaresestimatesfor B3 and Ba in (6.11)we can estimateAofrom 12.1512+ 2x (-2.76796)A, : I
CASCS
1both ttistic .6.12) ricted
.6.13)
Solving,we obtainAo :2.Il4,implying
the optimal monthly advertisingexpenditureis
s1014. Big Andy hasbeenspending$1900per month on advertising.He wantsto know whether :his amountcould be optimal. Does the information from the estimatedequation.provide .ufficient evidenceto reject a hypothesisthat $1900 per month is optimal? The null and rlternative hypothesesfor this test are
F / s : B *3 2 x $ a x 1 . 9 : l , STS TS
1 / r : 9 :- | 2 x B a x l . 9 t ' l
canbe written as \fter carryingout the multiplication,thesehypotheses F 1 3 : B* 3 . 8 8 4: 1 , H t : 9 21 3 . 8 9 4 + l Thishypothesis canbe testedusingeithera /-testor an F-test.The/-teststatistic t:
rared :F-
I and lve a ;that
(bz 1- 3.8bq)- | ,;2u +3'8ba)
if the null hypothesisis true.The tricky part of this testis chlculating ras a /1711-distribution :he denominatorof the /-statistic.Using the property of variance given in (B.21) of \ppendix B.4.3,
+2x3.8x;ov@;74) ,;@, +3.8b^): uil@J r 3.82x "*@;
- 2x3.8x3.288746 : 12.6463 f 3.8',x0.884774 :0.421967
The estimatedvariancesand covarianceare providedby your statisticalsoftware.The is :alculated valueof ther-statistic linal dy's :ure.
|:::
- 1 1.6330
0.633 " :0.9676 0.65419 ,/0.+ZlS0l
144
FURTHER
INFERENCE
IN THE MULTIPLE REGRESSION MODEL
The critical value for this two-tailed test comes from the /1711-distribution. At the a : 0.05 levelof significance t,: 1.994.Thus,becaus e -1.994 <0.9676 < l.994, we cannotreject the null hypothesisthat the optimal level of advertisingis $1900 per month. There is insufficient evidence to suggestAndy should change his advertising strategy. Alternatively, using an F-test, the test statistic is F_
- ssEu)lJ (ssER ssEu/(N- K)
where"/: l,N:75,andK:4.Thesumof squarederrorsfromthefullunrestrictedmodel in(6.12)isSSEu: l532.084.Somealgebraisrequiredtoobtaintherestrictedmodelthatis neededtoobtainavalueforSSEa.Whenlloistrue,B3:1-3.89+andtherestrictedmodel becomes -r ei Si : 9r 'r FzPi+ (l - 3'884)Ai* 9a,,4? Collecting terms and rearrangingthis equationto put it in a form convenientfor estimation yields
(Si- Ai) : 9r * gzPi-r 9067 - 3.8Ai)+ ei Estimating this model by least squareswith dependentvariable )i : Si - A; and explanatory variableSriz : P; and xiz: A? - 3.8Ai yields the restrictedsum of squarederrors SSER: 1552.286.The calculated value of the F-statistic is
bt ul
ul
Ir fc H rf
r! lr 6 $
u
cl h
- rs32.084) (rss2.286 / | :0.9362 Frs32.084171 The valueF : 0.9362is equalto t2 : (0.9676)2,obeyingthe relationshipbetween/- and^Frandom variables that we mentioned previously. The critical value F, comes from the F1r,zr;-distribution. For ct : 0.05 the critical valueis F,:3.976 - 4 :0.99q2. Given that F : 0.9362 .--F, :3.976, usingthe F-testyields the sameresultas usingthe t-test.If the test is done on the basisof its p-value, we find p - v a l u e: r l F s , t g > 0 . % 6 2 ] :
Pltltj>0.9616) + Plt1tg< -0.96761:0.3365
The result 0.3365> 0.05 leads us to conclude that Andy's advertisingstrategyis compatible with rhe data. You may have noticed that our descriptionof this test hasdeviatedslightly from the stepby-stephypothesistestingformat introducedin Chapter3 and usedso far in the book. The same ingredients were there but their arrangementvaried. From now on we will be less formal aboutfollowing thesesteps.By being lessformal we can exposeyou to the type of discussionyou will find in researchreports.Pleaseremember,however,that the stepswere introducedfor a purpose:To teachyou goodhabits.Following the stepsensuresyou include a descriptionofall the relevantcomponentsofthe test and that you think aboutthe stepsin the correct order. It is not correct, for example,to decide on the hypothesesor the rejection region after you observethe value of the statistic. 6.4.2a A One-Tail Test with More than One Parameter Supposethat, insteadof wanting to testwhether the datasupportsthe conjecturethat A : 1.9 is optimal,Big Andy wantsto testwhetherthe optimal valueofA is greaterthan 1.9.If he has
h
t(
i
(
n
n a
a i
DEL :hecr :0.05 )annotreject Lth.There is
6.4 TESTING SOME ECONOMIC
145
beenspending$ 1900per monthon advertising,andhe doesnot want to increasethis amount unlessthereis convincing evidencethat the optimal amountis greaterthan $ 1900,he will set up the hypotheses F1o:9: * 3.8Ba< 1,
ricted model model thatis rictedmodel
HYPOTHESES
I/r : 9: * 3.894> 1
In this casewe obtain the samecalculated/-value of t :0.9676, but the rejection region for a57o significancelevel is different. For the two-tail test the a : 0.05 rejectionregion was: Reject Hg if t > 1.994or t < -l .994.In this case,for the one-tail test, it is: Reject Hs if t>1.661 . Because0.9676<1.667, we do not reject 11s.There is not enough evidencein the data to suggestthe optimal level of advertising expenditureis greater than $1900. Becausethe F-distribution is not well suitedfor performing one-tail testsof this type, we restrict ourselves to the /-distribution when considering alternative hypothesesthat have inequalitysigns like "<" or ">".
)f esllmatron
6.4.3
and explanaluared errors
reent- and Fres from the 99q2. t as usingthe
:0.3365 .egy is com'om the steprebook. The will be less r the type of e stepswere you include . the stepsin he rejection
thatA:1.9 .9.If he has
LIsrNc Conpurpn
Soprrvanp
While it is possible and instructive to compute an F-value by using the restricted and unrestricted sums of squares,it is often more convenient to use the power of modern statisticalsoftware. Most statistical software packageshave commandsthat will automatically compute t- and F-values and their correspondingp-values when provided with a null hypothesis.You should check your software. Can you work out how to get it to test null hypotheseslike those we constructed?These tests belong to a class of tests called "Wald tests." Your softwaremight refer to them in this way. Can you reproducethe answerswe got for Big Andy's tests? We concludethis sectionwith a joint testof two of Big Andy's conjectures.In addition to proposing that the optimal level of monthly advertising expenditureis $1900, Big Andy is planning staffing and purchasing of inputs on the assumptionthat a price of P : $6 and advertisingexpenditureofA : 1.9will, on average,yield salesrevenueof $80,000.That is, in the context of our model
E(S): 9r * FzP -l}tA + 9+Az :Br *69z*1.99:+ 1.9'90 :80 compatiblewith Isthisconjectureaboutsales,andtheconjectureaboutoptimaladvertising, joint null We formulate the hypothesis in sample of data? theevidencecontained the F / s : B 3 * 3 ' 8 9 4: I
and 9r *6Br+
1 . 9 8 3 * 3 . 6 1 B :4 3 6
The alternative is that at least one of these restrictions is not true. Because there are "/: 2 restrictionsto testjointly, we usean F-test. Constructing the restrictedmodel requires substitutingboth of theserestrictions into our extendedmodel, which is left as an exercise. Using instead computer output obtained by supplying the two hypothesesdirectly to the software,we obtain a computedvalue for the F-statistic of 5.74 and a correspondingp-value of 0.0049.At a sEosignificancelevel, the joint null hypothesisis rejected.As an exercise, usethe least squaresestimatesto predict salesrevenuefor P : 6 andA: I .9. Has Andy been too optimistic or too pessimistic about the level of sales?
FURTHER
INFERENCE
IN THE MULTIPLE REGRESSION MODEL
6,5 T}ne tlse of Nonsample Inforrnation
lable 6.
In many estimation problems we have information over and above the information contained in the sample observations.This nonsample information may come from many places,suchaseconomicprinciplesor experience.When it is availableit seemsintuitivethat we should find a way to useit. If the nonsampleinformation is correct, and if we combine it with the sample information, the precision with which we can estimate the parametersis improved. To illustrate how we might go about combining sample and nonsample information, consider a model designedto explain the demand for beer. From the theory of consumer choice in microeconomics,we know that the demandfor a good will dependon the price of that good, on the prices of other goods, particularly substitutesand complements,and on income. In the caseof beer,it is reasonableto relatethe quantity demanded(Q) to the price of beer (PB), the price of liquor (PL), the price of all other remaining goodsand services(PR), and income (I). To estimatethis demandrelationshipwe needa further assumptionaboutthe functional form. Using "ln" to denote the natural logarithm, we assume,for this case,that the log-log functional form is appropriate
rn(Q): Br * Bzln(PB)+ p3ln(pz) + Baln(pR)+ Fsln(/)
(6.1s)
Comparing(6.14)with (6.15)showsthat multiplying eachprice andincomeby ). will give a changein ln(O) equalto (92 + B3* F+ f 9s) ln(f). Thus,for thereto be no changein ln(O) when all prices and income go up by the sameproportion,it must be true that
Bz*P3*9++F::0
(6.16)
Thus, we can say somethingabout how quantity demandedshould not changewhen prices and income changeby the sameproportion, and this information can be written in terms of a specific restriction on the parametersof the demand model. We call such a restriction nonsample information. If we believe that this nonsampleinformation makes sense,and hencethat the parameterrestrictionin (6.16) holds, then it seemsdesirableto be able to obtain estimatesthat obey this restriction. To obtain estimatesthat obey (6.16), we begin with the multiple regressionmodel
ln(Q,): Fr * B,ln(pB,)* B3ln(pl,)+ B4ln(pR,) * B5ln(f)* e,
\tedian \taximum \linimum i:d. Dev.
jnd a samp ::om a ranc :.ita aretirr :, denoteti To intro i.fBs: -'rplainedI
(6.r4)
This model is a convenientone becauseit precludesinfeasiblenegativeprices,quantities,and income, and becausethe coefficients92, 9:, 9+, and B5 are elasticities.SeeAppendix A.4.4. A relevant piece of nonsampleinformation can be derived by noting that, if all prices and income go up by the same proportion, we would expect there to be no change in quantity demanded.For example, a doubling of all prices and income should not change the quantity of beer consumed.This assumptionis that economic agentsdo not suffer from "money illusion." Let us impose this assumptionon our demand model and see what happens.Having all prices and income change by the same proportion is equivalent to multiplying each price and income by a constant. Denoting this constant by \, and multiplying each of the variablesin (6.14) by \ yields
I"(Q): Fr * pzln(\PB) + g:rn(r,pr)+ B4rn(Xrn) + B5rnlXr; : 9r * B2ln(PB)+ + B51n(1) B3ln(Pr)+B4ln(PR) +(92+B:+Bo+9s)ln(I)
..imple mea
(6.r7)
Substitutin In(Q)
=
\\'e have u -i:ing the .n\PLtf PR .quareses estimates 6 . 1 8 )T .h
t"@, (r")
Let the re .in estima
By using constrai estimate
6.5 THE USE OF NONSAMPLE
Ta h I e 6 , 7 Ition conom many uitivethat ombine it rmetersis
Surnmary Statistics for Data Used to Estirnate Beer Demand
Sample mean Vedian Maximum \{inimum
56.11 54.90 81.70 M.30 7.8574
3.08 3.11 4.07 1.78 0.6422
Fq:
(6.ls) vill give a : in ln(Q)
1.25 1.18 |.73 0.67 0.2983
8.37 8.39 9.52 6.95 0.7696
32.602 32.457 -11,593 25,088 4 \4)
and a sampleof dataconsistingof 30 yearsof annualdata on beer consumptioncollected trom a randomly selectedhousehold.Thesedata are storedin the file beerdat. Becausethe dataaretime-seriesrather than cross-sectionaldata,a subscriptt (insteadofl) hasbeenused to denote the rth observation.Summary statisticsfor the data appear in Table 6.1. To introduce the nonsampleinformation, we solve the parameterrestriction 9u + F: * 9.r -1_Bs : 0 for one of the B1's.Which one is not important mathematically,but for reasons explained below we solve for Ba
(6.14) rtities,and dix A.4.4. all prices change in ot change uffer from see what ivalent to ry \ and
147
PL
Std. Dev.
ormation, consumer le price of ts, and on re price of ices (PR), Laboutthe case,that
INFORMATION
-Fz - F: - 9s
Substitutingthis expressioninto the original model in (6.17) gives ln(Q,) : Fr * Bzln(PB,)* B3ln(PZ') + (-9, - F: - B5)ln(PR')* Fs ln(Ir) + er
: gr * Bz[m(rr,; - ln(pR,)]+ Br[n(Pt,) - ln(PR1)] +gr [m(r,)- ln(PR,)]+ e1 :Fr
+e,h(#) * u,'"(#) *s,,"G+)
let
(6.18)
We have used the parameterrestriction to eliminate the parameterBa and in so doing, and using the propertiesof logarithms,we have constructedthe new variablesln(PB,lPR,), ln(PL,lPR), andln(I,lPR ). The last line in (6.18) is our restrictedmodel. To get least iquares estimates that satisfy the parameter restriction, called restricted least squares estimates,we apply the leastsquaresestimationproceduredirectly to the restrictedmodel in r6.l8). The estimatedequationis
t.2es4'"(#)+0.r868'"(#)+0.e4s8'"(ul)(6.re) fi,D : _-4.7s8_ (6.16) en prices terms of :striction :nse,and : able to rodel (6.17)
(0.166)
(s")
(0.284)
(0.427)
Let the restrictedleastsquaresestimatesin (6.l9) be denotedby b\, bi, b\, andb!. To obtain an estimate for pa we use the restriction bi:
-b\-
b\ - bt:
- e l . 2 9 9 4 ) - 0 . 1 8 6 8- 0 . 9 4 5 8: 0 . 1 6 6 8
By using the restriction within the model we have ensured that the estimates obey the constraint,so that b; + b; + b; + b! : O. While it is always possibleto obtain restricted estimatesby substituting the constraintsinto the model, it may becomemessy if there are a
148
FURTHER
INFERENCE
IN THE MULTIPLE REGRESSION MODEL
number of restrictions or if the restrictions involve several parameters.Some software packageshave commandsthat automatically compute the restrictedleast squaresestimates when provided with the constraints.You should check out the commandsavailable in your software. What are the properties of this restricted least squaresestimation procedure?First, the restrictedleast squaresestimatoris biased,E(bD + $p, unlessthe constraintswe impose are exactly true. This result makes an important point about econometrics.A good economist will obtain more reliable parameterestimatesthan a poor one, becausea good economist will introduce better nonsampleinformation. This is true at the time of model specification and later, when constraintsmight be applied to the model. Nonsample information is not restricted to constraints on the parameters;it is also used for model specification. Good economic theory is a very important ingredient in empirical research. The secondproperty of the restrictedleast squaresestimatoris that its varianceis smaller than the varianceof the least squaresestimator,whether the constroints imposedare true or not. By combining nonsample information with the sample information we reduce the variation in the estimation procedure caused by random sampling. This reduction in variance obtained by imposing restrictions on the parameters is not at odds with the Gauss-Markov theorem. The Gauss-Markov result that the least squaresestimator is the best linear unbiasedestimator appliesto linear and unbiasedestimatorsthat use data alone, and no constraintson the parameters.Including additional information with the data gives the addedreward of a reducedvariance.If the additional nonsampleinformation is correct. we are unambiguouslybetter off; the restricted least squaresestimator is unbiasedand has lower variance.If the additional nonsampleinformation is incorrect, the reduced variance comes at the cost of biasedness.This bias can be a big price to pay if it leads to estimates substantiallydifferent from their correspondingtrue parametervalues.Evidenceon whether or not a restriction is true can be obtained by testing the restriction along the lines of the previous section.In the caseof this particular demandexample,the test is left as an exercise.
choice r conside function choosin 6.6.t
h is pot onncipk rrhen it -'onside rsedby r ariabler Jepende rusband cducati Table 6. :espect
\\'e estir rv $31 Wha :'itimat
6.6 Model Specification In what has been covered so far, we have generally taken the role of the model as given. Questionshave beenof the following type: Given a particular regressionmodel, what is the best way to estimateits parameters?Given a particular model, how do we test hypotheses about the parameters of that model? How do we construct interval estimates for the parametersof a model? What are the propertiesof estimatorsin a given model? Given that all these questions require knowledge of the model, it is natural to ask where the model comesfrom. In any econometricinvestigation,choice of the model is one of the first steps.In this section we focus on the following questions:What are the important considerations when choosing a model? What are the consequencesof choosing the wrong model? Are there ways of assessingwhethera model is adequate? Three essentialfeaturesof model choice are ( 1) choice of functional form, (2) choice of explanatoryvariables(regressors)to be included in the model, and (3) whether the multiple regression model assumptionsMRI-MR6, listed in Chapter 5, hold. Later chapters on heteroskedasticity,autocorrelation, and random regressors deal with violations of the assumptions.For choice of functional form and regressors,economic principles and logical reasoning play a prominent and vital role. We need to ask: What variables are likely to influencethe dependentvariabley? How is y likely to respondwhen thesevariableschange? At a constantrate?At a decreasingrate?Is it reasonableto assumeconstantelasticitiesover the whole range of the data? The answersto these questionshave a bearing on regressor
table
i.rmple \ledian
\larimu \lrnimu
ild. De ,'onela
i LUINI
TEDU .,1'EDU ,\|6
{. {^
,L
software )stimates e in your First, the rposeare :onomist )onomist iflcation )n is not n. Good rsmaller I true or luce the ction in vith the rr is the a alone, ta gives colTect, and has ariance timates vhether r of the <ercise.
glven. Ltis the rtheses or the en that model eps.In 'ations l? Are rice of rltiple lrs on rf the cgical ely to ange? S OVEf
:essor
6.6 MODEL
SPECIFICATION
149
choice and choice of a suitable functional form. Alternative functional forms were consideredin Appendix A and Sections2.3.4, 4.3, and 4.4; fiirther issuesin relation to tunctional form are addressedin Chapter 7. We turn now to consider the consequencesof choosing the wrong set of regressorsand some questions about regressorchoice. 6.6.1
OurrrEr
Venranrns
It is possible that a chosen model may have important variables omitted. Our economic principlesmay haveoverlookeda variable,or lack of datamay lead us to drop a variableeven rvhenit is prescribedby economic theory. To introduce the omitted-variable problem, we considera sampleof married coupleswhereboth husbandsand wives work. This samplewas usedby labor economistTom Mroz in a classicpaperon female labor force participation.The r ariablesfrom this samplethat we usein our illustration are storedin the file edu_inc.dat.The dependentvariable is annual family income FAMINC defined as the combined income of husbandand wife. We are interestedin the impact of level of education,both the husband's educationandthewife's education,on family income.Summarystatisticsfor the dataappearin fable 6.2 with HEDU and WEDU denoting years of educationfor the husbandand wife, respectively.We definethe othervariablesasthey areintroduced.The estimatedrelationshipis
FAfriNC,: -5534+ 3r3zHEDUi* 4523wED,i ('") (11230) (803) (1066) (p-value) (0.622) (0.000) (0.000)
(6.20)
\\'e estimatethat an additional year of educationfor the husbandwill increaseannualincome by $3132, and an additional year of educationfor the wife will increaseincome by $4523. What happensif we now incorrectly omit wife's educationfrom the equation?Then, the estimatedequation becomes
FAMINCi: -26191+ 5155HEDUi ('") (8s41) (6s8) (p-value) (0.002)(0.000) TabIe 6.2
(6.2r)
Sumrnary Statistics for Data Used for Farnily Incorne Exarnple FAMINC
Sample mean \ledian \laximum \linimum Std. Dev.
9r2r3 83013 344t46 9072 44147
HEDU
WEDU
12.61 I2 l7 4 3.035
t2.65 t2 t7 5 2.285
0.14 0 2 0 0.392
12.57 12.60 20.82 2.26 3.427
25.13 24.91 37.68 9.37 5.052
1.000 0.594 0.105 0.836 0.821
1.000 0.129 0.518 0.-/99
1.000 0.149 0.160
1.000 0.900
1.000
KL6
X5
X6
Correlation matrix
F.4MINC HEDU IT'EDU KL6 .{5 \6
1.000 0.355 0.362 -0.072 0.290 0.351
ff0
FURTHER
INFERENCE
IN THE MULTIPLE REGRESSION MODEL
Relative to (6.20), omitting WEDU leads us to overstate the effect of an extra year of educationfor the husbandby about $2000. This changein the magnitudeof a coefficient is typical of the effect of incorrectly omitting a relevant variable. Omission of a relevant variable (defined as one whose coefficient is nonzero) leads to an estimator that is biased. Naturally enough,this bias is known as omitted-variable bias. To give a generalexpression for this bias for the casewhere one explanatory variable is omitted from a model with two explanatory variables, we write the underlying model for (6.20) as yi : 9t l9zxiz * Fzxiz* e,
(6.22)
.ecauseof ^e add tw -.)nstructe rfluencef
fIMiNC,
lse,, ,n-value
wherey; : FAMINCi, xiz: HEDUi, and x;3 : WEDUi. Omitting x3 from the equationis equivalentto imposing the restriction 9s : 0. It can be viewed asan exampleof imposing an incorrect constrainton the parameters.As discussedin the previous section,the implications of an incorrectconstraintarebiasedcoefficientestimates,but a reducedvariance.Let b)be the leastsquaresestimatorfor B2whenx3 is omitted from the equation.In an appendixto this chapter, Appendix 6B, we show that
..\'hat can r p-vah :.i\e .:.rndarde --r aluesir ::ecisiono -ccause,b' ':.e minim
b i a s ( b :i ) E ( b ; ) - 9 z : 9 r $ r 3 ) varlx2)
r.6.3 C
(6.23)
Knowing the sign of B3and the sign of the covariancebetweenx2 andx3 tells us the direction of the bias.Also, while omitting a variablefrom the regressionusually biasesthe leastsquares estimator, if the sample covariance (or sample correlation) between x2 and the omitted variable-r3is zero, then the leastsquaresestimatorin the misspecifiedmodel is still unbiased. Analyzing (6.23)in the contextof our example,first note that B3 ) 0 becausehusband's educationhas a positive effect on family income. Also, from Table 6.2, *r<E,"J>O becausehusband'sand wife's levels of educationare positively correlated.Thus, the bias exhibitedin (6.21)is positive.Thereare,ofcourse,othervariablesthat could be includedin (6.20) as explanators of family income. In the following equation we include KL6, the number of children lessthan 6 yearsold. The larger the number of young children, the fewer the number of hours likely to be worked and hence a lower family income would be expected. FAMINCT : -7755 + 32Il HEDUi + 4777 WEDUi - l43ll KL6i ('") (11163) (7ei) (1061) (s004) (p-value) (0.488) (0.000) (0.000) (0.004)
(6.24)
We estimatethat a child under6 reducesfamily incomeby $14,31l. Notice that,compared to (6.20), the coefficient estimatesfor H ED U andWED U have not changeda greatdeal.This outcome occurs becauseKL6 is not highly correlated with the educationvariables.From a generalmodeling perspective,it means that useful results can still be obtained when a relevant variable is omitted if that variable is uncorrelatedwith the included variables,and our interest is on the coefficients of the included variables. Such instancescan arise. for example, if data are not available for the relevant omitted variable. 6.6.2
IRnrnvRNr
l. e possi :',r'anthat -,:tely,doi .rplied to --.:oretica - trh expe
::'JSOnabl!
Ch un(
Ifr ma an
-1. On inc su rej Hc oc ab an (b)
+. Th RT
VeRrRnr-rs
The consequencesof omitting relevantvariablesmay lead you to think that a good strategyis to include as many variables as possible in your model. However, doing so will not only complicate your model unnecessarily,it may also inflate the variancesof your estimates
r.6.3a ir'sting fi .:nproveI .-elevan
iL
6.6 MODEL SPECIFICATION
a year of fficient is Lrelevant is biased. rpression with two
becauseof the presenceof irrelevant variables. To see the effect of irrelevant variables, ire add two artificially generated variables Xs and X6to (6.24). These variables were ;onstructed so that they are correlated with HEDU and WEDU, but are not expected to :nfluencefamily income. The resultingestimatedequationis FAMiNa, : ('")
(6.22) luation is rosingan rlications Letb)be lix to this
(6.23) direction it squares : omitted rnbiased. usband's
itrl > o , the bias ;luded in KL6, the :hefewer vould be
p-varue)
rategyis rot only stimates
-7159 +334OHEDUi -t 5869wEDui - l42O0KL6i i 889X;5- t067Xio (llles) (1250) (2278) (s044) (2242) (1e82) (0.s00) (0.008)
(0.010)
(0.00s)
(0.692)
(O.sel)
\\'hat can we observefrom theseestimates?First, as expected,the coefficients of X5 and X6 :ravep-valuesgreaterthan0.05.They do indeedappearto be irrelevantvariables.Also, the .tandard errors of the coefficients estimated for all other variables have increased,with ,"-r'aluesincreasing correspondingly.The inclusion ofirrelevant variableshas reduced the :recision of the estimatedcoefficientsfor other variablesin the equation.This result follows Tcause, by the Gauss-Markov theorem,the least squaresestimator of the correct model is :he minimum variance linear unbiasedestimator. 6.6.3
CHoosrNc rHE MoDEL
The possibilities of omitted-variable bias or inflated variances from irrelevant variables :rean that it is important to specify an appropriate set of explanatory variables. Unfortur..itely,doing so is often not an easytask. There is no one set of mechanicalrules that can be .pplied to come up with the best model. What is neededis an intelligent application of both :reoretical knowledge and the outcomes of various statistical tests. Better choices come ,i ith experience.What is important is to recognize ways of assessingwhether a model is ::asonableor not. Some points worth keeping in mind are as follows: l.
-)
Choosevariables and a functional form on the basis of vour theoretical and seneral understandingof the relationship. If an estimated equation has coefficients with unexpected signs, or unrealistic magnitudes,they could be causedby a misspecificationsuch as the omission of an important variable. One method for assessingwhether a variable or a group of variables should be includedinanequationistoperformsigniflcancetests.Thatis, l-testsforhypotheses suchas Hs:$3:0 or F-testsfor hypotheses suchas lls:B3 - F+ :0. Failureto reject hypothesessuchasthesecan be an indication that the variable(s)areirrelevant. However, it is important to rememberthat failure to reject a null hypothesiscan also occur if the dataare not sufficiently rich to disprovethe hypothesis.More will be said aboutpoor datain the next section.For the moment we note that, when a variable has an insignificant coefficient, it can either be (a) discardedas an irrelevant variable or (b) retained becausethe theoretical reason for its inclusion is a strong one.
(6.24)
rmpared eal.This . From a when a lles, and Lrise,for
151
+.
The adequacyof a model can be testedusing a general specification test known as RESET. We conclude this section with a description of this test.
r.6.3a The RESET Test festing for model misspecificationis a way of asking is our model adequate,or can \\'e .nprove on it? It could be misspecifiedif we have omitted important variables,included -elevant ones, chosen a wrong functional form, or have a model that violates the
152
FURTHER
INFERENCE
IN THE MULTIPLE REGRESSION MODEL
assumptionsof the multiple regressionmodel. The RESET test (RegressionSpecification Error Test)is designedto detectomitted variablesand incorrect functional form. It proceeds as follows. Supposethat we have specified and estimated the regressionmodel Yi : 9r 1-9zxiz * 9zxiz* ei Let (h, b2, fu) be the least squaresestimatesand let ji:
bt * b2x;2* b3xg
(6.2s)
be the predicted values of y;. Consider the following two artificial models
Y; : 9t l gzxiz* 9:x;: -t ItY? + "t yi : 9t l gzxizI Fzxiz+ YY?* VY? + ei
(6.26) (6.27)
In (6.26) a test for misspecification is a test of Hs:11 :0 against the altemative H t : ^ Y r * 0 . [ n ( 6 . 2 7 ) , t e s t i n gH s : " t 1 : " l z : O a g a i n sftt : 1 1 1 0 a n d / o rV * 0 i s a t e s t for misspecification.In the first casea /- or an F-test can be used.An F-test is required for the second equation. Rejection of F16implies the original model is inadequate and can be improved. A failure to reject I1s sals the test has not been able to detect any misspecification. To understand the idea behind the test, note that y! and !f will be polynomial functions of x;2 and x6. If you square and cube both sides of (6.25), you will get terms like f,.r, x3s,xi2x3, x;2fp, and so on. Since polynomials can approximate many different kinds of functional forms, if the original functional form is not correct the polynomial approximation that includes !f and y? *uy significantly improve the fit of the model. If it does, this fact will be detected through nonzero values of "),rand 12. Furthermore, if we have omitted variables, and these variables are correlated with x;2 and rp, then they are also likely to be correlated with terms like *,, and x!3, and so some of their effect may be picked up by including the terms !/ and/or !f. overall, the general philosophy of the test is: If we can significantly improve the model by artificially including powersof the predictionsof the model, then the original model must havebeen inadequate. Applying the two RESET tests in (6.26) and (6.27) to the family income equation in (6.24) yields the following results F 1 6 : ^ y :1 Q
F :5.984
p-value: 0.015
H s : 1 1: " l z : O
F :3.123
p - v a l u e: 0 . 0 4 5
In both casesthe null hypothesisofno misspecificationis rejectedat a 57osignificancelevel. So, although this equation was a useful one for illustrating the effect of omitted-variable bias, it could be improved upon as a model for explaining family income. Perhapsage and experiencecould be included in the model, along with whether the householdis in a city or the country. Perhapsthe linear functional form is inappropriate. Although the RESET test is often useful for picking up poorly specifiedmodels, it needs to be kept in mind that it will not always discriminate between alternative models. For example,if two different functional forms arebeing consideredfor a particular relationship, it is possible for RESET to reject neither of them.
6.7 Po
\lost econ mental. In purposes specified I sariablesi can be ider erperimen r ariablesa ls no guar rsolatethe Asane Burger Ba to advertis .-oupon a( advertisin are flyers r measurin ofa salesr positivere of the two revenue,b to sort oul As a se tunctiono productio As produc proportior i)'stemati or separa difficult. It is no rsolate th erplanato difficult tc rariable. l 6.7.1 T
The conse ing an ext estimatio the model
and wrote
6.7 POOR DATA, COLLINEARITY,
JL
:cification ; proceeds
(6.2s)
(6.26) (6.27) rlternative 0 is a test s required quate and letect any olynomial get terms y different olynomial he model. rthermore, C x6, then re of their re general utificially havebeen quation in
ls, it needs odels. For lationship,
153
6.7 Poor Data, Collinearity, and Insignificance Most economic data that are used for estimating economic relationships are nonexperimental. Indeed, in most casesthey are simply "collected" for administrativeor other purposes.They are not the result of a plannedexperimentin which an experimentaldesignis specified for the explanatory variables. In controlled experiments the right-hand-side variables in the model can be assignedvalues in such a way that their individual effects can be identified and estimatedwith precision. When data are the result of an uncontrolled experimentmany of the economicvariablesmay move togetherin systematicways. Such variablesare said to be collinear, and the problem is labeled collinearity. In this casethere is no guaranteethat the data will be "rich in information," nor that it will be possible to isolate the economic relationship or parametersof interest. As an example,considerthe problem facedby the marketingexecutivesat Big Andl's Burger Barn when they are trying to estimate the increase in sales revenue attributable to advertisingthat appearsin newspapersandthe increasein salesrevenueattributableto u'oupor advertising. Supposeit has been common practice to coordinate these tu'o advertising devices, so that at the same time advertising appearsin the newspapersthere are flyers distributed containing coupons for price reductions on hamburgers.If variables measuringthe expenditureson thesetwo forms of advertising appearon the right-hand side of a salesrevenueequationlike (5.2),thenthe dataon thesevariableswill showa systematic, positiverelationship; intuitively, it will be difficult for suchdatato revealthe separateeffects !rf the two types of ads.Although it is clear that total advertisingexpenditureincreasessales revenue,becausethe two types of advertisingexpendituremove together,it may be difficult io sort out their separateeffects on salesrevenue. As a secondexample,considera productionrelationshipexplainingoutputovertime asa tunction of the amountsof variousquantitiesof inputs employed.There arecertain factors of production (inputs), such as labor and capital, that are used in relativelyfixed proportions. .\s production increases,the changing amounts of two or more such inputs reflect equiproportionateincreases.Proportional relationships between variables are the very sort of .\'stematicrelationshipsthat epitomize "collinearity." Any effort to measurethe individual t.rrsep&rflt!effects (marginal products) of various mixes of inputs from such data will be Jifficult. It is not just relationshipsbetweenvariablesin a sampleof datathat make it difficult to rsolate the separateeffects of individual explanatory variables. If the values of an :rplanatory variabledo not vary or changemuch within a sampleof data,then it is clearly Jifficult to use that data to estimate a coefficient that describesthe effect of changein that rariable. It is hard to estimatethe effect of chanseif there has been no chanse. 6.7.1
ancelevel. d-variable ps ageand in a city or
AND INSIGNIFICANCE
Tnr Cor.rsEeuENcESoF CoLLINEARITv
The consequences ofcollinearity and/orlack of variation dependon whether we areexamin:ng an extreme case where estimation breaks down or a bad but not extreme casewhere -:stimationcan still proceedbut our estimateslack precision.In Section5.3.1,we considered :he model Yi : 9t l9zxiz I 9zxiz* ei end wrote the varianceof the least squaresestimatorfor Br as var(b2) :
oZ
1r- 'i)IL, @iz- xz)z
(6.28)
154
FURTHER
INFERENCE
IN THE MULTIPLE REGRESSION MODEL
where ry is the conelation betweenx2 and4. Exact or extreme collinearity exists when 12 andx3 areperfectly correlated,in which casar23- 1 and var(b2)goesto infinity. Similarly, if .{2 exhibits no variation Z(xiz - -2)2 equals zero and var(b) again goes to infinity. In this c&sex2is collinear with the constantterm. In general,wheneverthere are one or more exact linear relationships among the explonatory variables, then the condition of exact collinearity exists.In this case the least squares estimator is not defined. We cannot obtain estimatesof 87.'susing the least squaresprinciple. One of our least squaresassumptions MR5, which says the values of x;1 are not exact linear functions of the other explanatory variables,is violated. The more usual caseis where correlationsbetweenexplanatoryvariablesmight be high, but not exactly one, variation in explanatory variables may be low but not zero, or linear dependenciesbetween more than two explanatory variables could be high but not exact. These circumstancesdo not constitute a violation of least squaresassumptions.By the Gauss-Markov theorem, the least squares estimator is still the best linear unbiased estimator. There may be a problem, however, if the best we can do is not very good becauseofthe poor characteristicsofour data.From (6.28) we can seethat when r23is close to I or l(xa - -2)2 is close to zero,the varianceof b2will be large. A large variancemeansa large standarderror, which meansthe estimatemay not be significantly different from zero and an interval estimatewill be wide. The sampledata have provided relatively imprecise information about the unknown parameters.The effectsof this impreciseinformation can be summarized as follows. l.
When estimator standarderrors are large, it is likely that the usual /-testswill lead to the conclusion that parameter estimates are not significantly different from zero. This outcome occurs despite possibly high R2- or F-values indicating significant explanatorypower of the model asa whole. The problem is that collinear variablesdo not provide enough information to estimate their separateeffects, even though economic theory may indicate their importance in the relationship.
2. Estimatorsmay be very sensitiveto the addition or deletion of a few observations,or the deletion of an apparently insignificant variable.
3 . Despite the difficulties in isolating the effects of individual variables from such a sample, accurate forecasts may still be possible if the nature of the collinear relationship remains the samewithin the out-of-sample observations.For example, in an aggregateproduction function where the inputs labor and capital are nearly collinear, accurateforecastsof output may be possiblefor a particular ratio of inputs but not for various mixes of inputs.
6.7.2
AN Exelnprs
The file cars.dat contains observationson the followins variables for 392 cars: MPG : miles per gallon CYL : number of cylinders ENG : engine displacementin cubic inches WGT : vehicle weight in pounds Supposewe are interestedin estimating the effect of CYL, ENG, and WGTon MPG. All the explanatory variables are related to the power and size of the car. Although there are
i\ceptions, or --arsand smal rhose separa r ields
\\'e estimatetl :allon, and tl .bservewhat
frl
(s
(P-'
fhe estimated :umber of cyl ':reir coefflcier --rpothesesIft :.refficient of -onelationbel :ilects of eac .iternativel1r :'-ralueof 0.0 Cl'L and ENC - .refficient is fi :ust be nonze :=ducethe var r' highly corre 6.7.3
IoEN
3ecausenone :rake senseto i}timated an (
:eve the expe :c* observati :ritigate collir :roblem. Hov :rpectations, One simple rtween pairs --f linear asso iran two of th :airwise corn :dentifying th ln theseleast
IEL
6.7 POOR DATA, COLLINEARITY,
AND INSIGNIFICANCE
155
;ts when -r2 imilarly, if Lity.In this nore exsct xact collinot obtain sumptions rplanatory
exceptions,overall we would expectthe valuesfor CYL, ENG, and IVGTto be large for large cars and small for small cars. They are variablesthat are likely to be highly correlated and whose separateeffect on MPG may be difficult to estimate.A regressionof MPG on CYL f ields
ht be high, r, or linear not exact. ns. By the unbiased very good r23is close le meansa from zero imprecise :ion can be
We estimatethat an additionalcylinder reducesthe gasolineconsumptionby 3.6 miles per eallon, and the significanceof its coefficient suggestsit is an important variable. Nou. observewhat happenswhen ENG and WGT areincluded.The estimatedmodel becomes
vill lead to iom zero. ;ignificant riables do 3n though yations,or >m such a collinear 'example, ue nearly r ofinputs
3. All the there are
frFd,:
42.s-3'ssScYLi
('")
(0.83) (0.146)
(p-value)
(0.000) (0.000)
MPGi : 44.4 - 0.268CYL: - 0.0l27ENGi - 0.00571WGTi ('") (1.s) (0.413) (0.0083) (0.00071) (p-value) (0.000) (0.s17) (0.12s) (0.000) The estimatedcoefficient on CYL has changed dramatically, and although we know that number of cylinders and engine size are important variables, when consideredseparately, their coefficientsare not significantly different from zero at a 5Vosignificancelevel. The null h1'potheses Ho:92: 0 and Hs:$3 :0 are not rejectedby separatet-tests,where B2is the ;oefficient of CYL and B3 is the coefficient of ENG. What is happening is that the high ;onelation betweenCYL andENG (r: 0.95) is making it difficult to accuratelyestimatethe : 0 againstthe effectsof each variable.When we test the null hypothesisHo:92:9: elternativeHr:9210 anilor 9z*0, we obtain an F-value of 4.30 with corresponding -"-r'alueof 0.014. The null hypothesisis firmly rejected.The data are telling us that together CYL and ENG influence MPG, but it is difficult to sort out the influence of each. If one .'oefficientis free to takeany value,the dataarenot good enoughto prove the other coefficient nust be nonzero.Should you drop one of the insignificant variables, sayCYL? Doing so will :educethe variancesof the remaining estimates,but, given CYLis an important variablethat :. highly correlatedwith ENG andWGT, it is also likely to introduce omitted-variablebias. 6.7.3
IoENrrrvwc
aNo MrrrcerrNc
CoLLrNEARrry
Becausenonexactcollinearity is not a violation of least squaresassumptions,it doesnot :rake senseto go looking for a problem ifthere is no evidencethat one exists.Ifyou have jstimated an equation where the coefficients are precisely estimated and significant, they rave the expectedsignsand magnitudes,and they are not sensitiveto addingor deletinga :erv observations,or an insignificant variable, then there is no reasonto try and identify or nitigate collinearity. If there are highly correlated variables, they are not causing you a :roblem. However, if you have a poorly estimated equation, which does not live up to .'\pectations,it is useful to establishwhy the estimatesare poor. One simple way to detectcollinear relationshipsis to use samplecorrelation coefficients retweenpairs of explanatoryvariables.Thesesamplecorrelationsare descriptivemeasures 'i linear association.However, in some caseswhere collinear relationships involve more :han two of the explanatory variables, the collinearity may not be detectedby examining :airwise correlations. In such instances, a second simple and effective procedure for :dentifying the presenceof collinearity is to estimate the so-called auxiliary regressions. In these least squaresregressions,the left-hand-side variable is one of the explanatory
156
FURTHER
INFERENCE
IN THE MULTIPLE
REGRESSION MODEL
variablesand the right-hand-sidevariablesare all the remaining explanatory variables.For example, a general auxiliary regressionfor x2 is
eITOTSel ?nd
linear unbias
xiz : atxit + a3xB+ "' + aKxiK+ error If R2 from ttris artificial model is high, above0.80 say,the implication is that a large ponion of the variation in x2 is explained by variation in the other explanatory variables. In Section 5.3.I we made the point that it is variation in a variable that is not associatedwith any other explanatory variable that is valuable for improving the precision of the least squaresestimator Dz.If R" from the auxiliary regressionis not high, then the variation in -r2is not explained by the other explanatory variables, and the estimator b2's precision is not affected by this problem. The collinearity problem is that the data do not contain enough "information" about the individual effects of explanatoryvariablesto permit us to estimateall the parametersof the statistical model precisely. Consequently,one solution is to obtain more information and include it in the analysis.One form the new information can take is more, and better,sample data. Unfortunately, in economics, this is not always possible. Cross-sectionaldata are expensive to obtain, and, with time-series data, one must wait for the data to appear. Alternatively, if new data are obtainedvia the samenonexperimentalprocessasthe original sampleof data, then the new observationsmay suffer the samecollinear relationships and provide little in the way of new, independentinformation. Under these circumstancesthe new data will help little to improve the precision of the least squaresestimates. A second way of adding new information is to introduce, as we did in Section 6.5, nonsample information in the form of restrictions on the parameters.This nonsample information may then be combined with the sampleinformation to provide restricted least squaresestimates.The good news is that using nonsampleinformation in the form of linear constraintson the parametervaluesreducesestimator sampling variability. The bad newsis that the resulting restricted estimatoris biased unlessthe restrictions are exactly trae.T\us, it is important to usegood nonsampleinformation, so that the reducedsampling variability is not bought at a price of large estimator biases.
6.8 Prediction The prediction problem for a linear model with one explanatory variable was covered in depth in Section 4. 1. The results in this section extend naturally to the more generalmodel that has more than one explanatory variable. To describethe extensions,consider a model with an intercept term and two explanatory variables .r2 and .r3.That is Yi:Bt *xiz9z*xBFs*ei
(6.2e)
where the !; &reulcolr!lated random variableswith mean 0 and varianceot. Given a set of valuesfor the explanatoryvariables,say(1, xs2, xs3), the prediction problem is to predict the value of the dependentvariable ye, which is given by yo:9r
lxoz9z-t'xoz9z*eo
If the data are time-series data, (l,xs2,x$) will be future values for the explanatory variables; for cross-sectiondata they represent values for an individual or some other economicunitthat was not sampled.We areassumingthattheparametervaluesdetermining ye are the sameas thosein the model (6.29) describing how the original sampleof data was generated.Also, we assumethe random error e0to be uncorrelatedwith each of the sample
where bp's ar, averagevalue E ("f) :0. Th, linear and un The variar occursbecauf a consequenc val
To obtain va explanatoryv rhusis uncorre .'ovarianceso varianceof a Each of th
r arianceofthr
of the forecas distributed, or
Following the io t t"se(/),
Thus, the r of the results similar way.
6.9 Exe
.{nswersto e
6.9.L Pnor
6.1 When
you ot (a) R2 (b) Th rej
6.8PREDICTION
)EL
riables. For
!rrorSe; and to have the samemean 0 and variance o2. Under these assumptions,the best linear unbiased predictor of ys is given by yo:h*xs2b2
e portion of ection5.3.1 r any other )s estimator dnedby the is problem. " about the etersof the mation and .ter,sample al data are to appear. .heoriginal rnshipsand stancesthe S.
ection 6.5, nonsample ricted least m oflinear radnewsis true. Thus, riability is
157
Ixrr.b3
where Dp'sare the least squaresestimators.This predictor is unbiasedin the sensethat the averagevalue of the forecasterror is zero. That is, if f : 1yo- yo) is the forecasterror, then E (f) :0. The predictor is bestin the sensethat the varianceof the forecasterror for all other linear and unbiasedpredictors of y6 is not less than var(ye - ts). The variance of forecast error var(ye - j,e) contains two components.One component occursbecausebt, bz,and D3areestimatesof the true parameters,andthe other componentis a consequenceof the unknown random error ee.The expressionfor var(yo - yo) is given by - (br * bzxoz-l t:xr)l var(/) : varl(Fr * \zxoz * F:xos1 "o) : vat(eo b1 b2xs2 b3x$) : var(eo)* var(b1) -l fi2var(b) + fi3var(fu) *Zxozcov(b1,bz) +2xozcov(b1, bz) *2xozxozcov(b2,fu) To obtain var("f) we recognized that the unknown parameters and the values of the explanatoryvariables are constants,and that es is uncorrelatedwith the sample data, and thusis uncorrelatedwith the leastsquaresestimators(h, bz,b3).The remaining variancesand covariancesof the least squaresestimatorsare obtained using the rule for calculating the varianceof a weighted sum in Appendix B.4.3. Each of the terms in the expression for var("f) involves o'. To obtain the estimated lariance of the forecasterror frffi,
we replaceo2 with its estimator62. The standarderror
of the forecast is given by se(/) : \/ v&T). If the random errors e; and es are normally distributed, or if the sample is large, then
f_ se(,f)
---- Y o - Y o
- t1N_K)
I
r/var(v" - i,n) vV
coveredin xal model rplanatory
(6.2e)
Following the steps we have used many times, a 100(l-ot)Vo interval predictor for -r'eis i1 t t se(/), where /. is a critical value from the /11v-r;-distribution. Thus, the methodsfor prediction in the model with K: 3 are straightforwardextensions ..rfthe results from the simple linear regressionmodel. For K > 3, the methodsextend in a .imilar wav.
6.9 Exercises
'en a set of
{nswers to exercisesmarked x appear in Appendix D at the end of the book.
predictthe
6.9.7
PnonrErnls
6.1 When using N : 40 observationsto estimatethe model rplanatory rme other termining f datawas he sample
Yi:9r
l9zxil9tzi*ei
you obtain SSE : 979.830 and 6, : 13.45222.Find (a) R'. (b) The valueof the F-statisticfor testingHo:92 : F: : 0. Do you reject or fail to reject Hs?
FURTHER
6.2
IN THE MULTIPLE REGRESSION MODEL
INFERENCE
Consideragain the model in Exercise6.1. After augmentingthis model with the squaresand cubesof predictionsyl andyl,we obtainSSE: 696.5357.Use RESET to testfor misspecification.
6.3x Considerthe model yi:9r
I u,f fo.eosszl I o.ztstz o.oleles-0.050301I -0.03t2n : : url o.oser+ . diDl o.oreres 0.048526 I I | I -o.oso:or I -0.031223 (a) (b) (c) (d) (e)
6.4
0.037120 )
Ll.7i6e ) | : 2 . 5 1 9 3 o' R z: 0 . 9 4 6 6
Find the total variation, unexplainedvariation, andexplainedvariation for this model. Find 957o intewal estimatesfor B2 and B3. H1:$2 11 . Use a t-testto testthe hypothesisHo:92> I againstthe alternative Use your answersin part (a) to test the joint hypothesisHo:92: 0, 9: : 0. Testthe hypothesisF1s:2B2: Br.
Supposethat, from a sampleof 63 observations,the leastsquaresestimatesand the correspondingestimatedcovariance matrix are given by
[ el
1t,,1
|
T 3
-2
lbrl:
| 31. covlbl:l-2
4
I ut ttttl
|
lb,l
-
|
|
L I
L-r.l
t
Dependent Varia Includedobserva Variable
lxiz9z* xiz\tlei
and supposethat application.ofleast squaresto 20 observationson thesevariables yields the following results (.* [D] denotesthe estimated covariance matrix) :
lu, )
Table 6,4
1l
ol
C l n ( f) ln(K) ln(PL) l n ( P F) In(PM) In(STAGE) R-squared \djusted R-squa
S.E.of regressio Sum squaredresi
^.1* In thepap Permane 645-657 contain o
o 3l
ln(VC) =
Test each of the following hypothesesand state the conclusion: ( a ) B z: 0 . (b) Fr + 292:5.
where VC price of f (a) Inte4 (b) Do tt (c) Whic (d) Does (e) Cons (0 Ifalli the sa (g) The tt you v
(c) 9r-92*$3:4'
6.s
The RESET test suggestsaugmenting an existing model with the squaresof the predictions !f, or their squaresand cubes O?,i?1. What would happen if you augmentedthe model with the predictions themselves!;?
6.6
Table 6.3 contains output for the two models Yi : Ft l9zxi I B3wiI e; yi:9i*9zxi*ei obtainedusingN: 35 observations. RESET testsappliedto the secondmodel yield F-valuesof17.98(forl?) and8.72(forliandli).Thecorrelationbetween,randwis rm :0.975. Discussthe following questions: (a) Should wi be included in the model? (b) What can you say about omitted-variablebias? (c) What can you say about the existenceof collinearity and its possible effect?
Table 6.3
O u t p u t f o r E x e r c i s e6 . 6
Variable
Coefficient
Std. Error
/-value
Coefficient
Std. Error
C X
3.6356 -0.99845
1.316 -0.8085
- 5.83 82 4.t072
2.000 0.3383
w
0.49785
2.763 1.235 0.1174
4.240
t-value
-2.919 12.14
TabIe 6,5
t
c '
0.33 :r f ) 0.00 -0.00 rtK) r PL) 0.01 rrPF) 0.01 -0.05 r PM) :rSTAGE)-0.00
)DEL
159
6,9 EXERCISES
rdel with the Use RESET
Table 6.4
Least SquaresOutput forExercise 6.7
DependentVariable: ln(VC) Includedobservations:268 \ariable
Coefficient
Std. Error
0s0301 I
C r n {I ) lntK) tnlPL) intPF) 'tnt PM) ''ntSTAGE)
7.528901 0.679157 0.350305 0.275366 0.321864 -0.068318 -0.194390
0.582172 0.053399 0.052879 0.043807 0.036098 0.100338 0.028s77
orthis model.
R-squared
t H 1 : 8 21 7 .
{djusted R-squared S.E. of regression Sum squaredresid
esevariables : matrix): 031223 | 037120)
0 ,F : : 0 . ratesand the
0.989528 0.989287 0.tt75t2 3.604139
p-value
t-value
12.93244 12.71856 6.624638 6.28s921 8.916433 -0.680879 -6.802349
0.0000 0.0000 0.0000 0.0000 0.0000 0.4966 0.0000
Mean dependentvar S.D. dependentvar F-statistic Prob(F-statistic)
6.243818 1.135334 41 10.3l0 0.000000
1.7* In thepaperBaltagi,B.H., J.M. Griffen,andS.R.Vadali(1998),"ExcessCapacity:A PermanentCharacteristic of U.S. Airlines," Journal of Applied Econometics, 13, 645-657, the authors consider estimation of airline cost functions. Tables 6.4-6.6 contain output obtained using their data to estimate a function of the form
ln(vc) : Br * B2h(r) + B3ln(iK)+ Baln(Pr) + 95ln(pr) + g6rn(pM) * e * BTIn(SZAGE) where VC : variable cost; f: output; K : capital stock; PZ - price of labor; PF : price of fuel; PM - price of materials; and STAGE : avera5!flight length. (a) Interpret the coefficients of ln(If, ln(r!, and ln(PD. (b) Do the estimated coefficients have the anticipated signs? (c) Which coefficients are not significantly different from zero? (d) Does the RESET test suggestthe model is misspecified? (e) Constantreturnsto scaleexist if 9z f B3 : 1. Test this hypothesis. (0 If all input prices increaseby the sameproportion, variable cost will increaseby the sameproportion if Fo + Fs * B6 : 1. Test this hypothesis. (g) The testsin parts(e) and (0 could alsobe carriedout using /-statistics.Explain how you would use the information in Thbles6.4 and 6.5 to computetheset-statistics.
uares of the ppen if you
model yield enx andw is
Ta b I e 6 . 5
Covariance Matrix for Least SquaresEstimates: Exercise 6.7 C
ble effect?
t-value
-2.919 12.14
ln(Ii)
ln(K)
ln(PL)
In(PF)
ln(Plt4)
In(SIAGEI
0.338924 0.007059-0.005419 0.0rr217 0.0t7152-0.056298 -0.004939 -0.000364-0.001097 0.007059 0.002851-0.002753 -9.898(05 0.000179 -0.005419-0.002753 0.002796 2.838-05-0.000110 0.000294 0.000887 -8.60E-05-0.002159 3.64E-05 o.otr2r7 -9.89E-05 2.83E-05 0.001919 -8.60E-05 -0.000110 0.001303-0.002929 -0.000102 0.0t7152 0.000179 -0.000364 -0.002159 -0.002929 -0.056298 0.010068 0.000104 0.000294 tr PM) A(STAGE)-0.004939-0.001097 0.000887 3.64E-0s-0.000102 0.000104 0.000817
C .rtf) .rtK) .u PL) . : uP F )
160
FURTHER
Table 6,6
INFERENCE
IN THE MULTIPLE REGRESSION MODET
il
T e s t R e s u l t sf o r E x e r c i s e 6 . 7
RamseyRESET Test: F-statistic(l term) F-statistic(2 terms)
3.380323 1.860108
Wald Test: Null Hypothesis: F-statistic
9z*B3:1 6.104834
p-value
0.014121
Wald Test: Null Hypothesis: F-statistic
9+*Bt*90:l 75.43246
p-value
0.00000(r
Con and Cob
0.06'/120 0.157729
p-value p-value
(a)
(b)
l';' Usir
I
6.8 In Section6.4.3 we testedthe joint null hypothesis Hs:B3i3.8F4: I in the model
and 9r *692-ll.9F:*3.619+
whi :80
sr : gr -l gzpi* g:Ai + gqA?+ ei
By substitutingthe restrictionsinto the model and rearrangingvariables,show horr the modelcanbe written in a form whereleastsquaresestimationwill yield restricted least squaresestimates. 6.9.2
Cortlpurrn ExrncrsEs
6.9 In Exercise5.15 we exoressed the model '
Y,:
The tow Thor
Y=
RG= RD= RF=
The r resp(
,uYfz rT'nf't+'tl'exP {e,}
in termsof logarithmsand estimatedit using datain the file manuf.dat.Use the data and resultsfrom Exercise5.15 to test the following hypotheses: ( a ) H o : 9 z: 0 a g a i n sHt 1 : B 2 l 0 . ( b ) 1 1 0 : 9 2: 0 , 9 : : 0 a g a i n sH t r:92*0and/orFzl0. ( c ) F 1 oB: z : 0 , B +: 0 a g a i n sH t 1:$2l0and/orBal0. ( d ) H 0 : 9 2 : 0 , F : : 0 , F + : 0 a g a i n sHt t : 9 z l 0 a n d / o r B 3 l 0 a n d / o r F c l 0 . ( e ) H o : 9 2* 9 r * F +* 9 s : I a g a i n sHt t : 9 2 - l - F r + 9 q + 9 s * 1 . (0 Analyze the impact of collinearity on this model. 6. l0'' Use the sampledata for beer consumptionin the file beer.datto (a) Estimate the coefficients of the demand relation (6.14) using only sample information. Compare and contrastthese results to the restrictedcoefficient resultsgiven in (6.19). (b) Does collinearity appearto be a problem? (c) Testthe validity of the restrictionthat implies demandwill not changeif prices and income go up in the sameproportion. (d) Use model (6.19) to construct a 95Va prediction interval for Q when PB : 3.00.PL : 10,PR : 2.00,and1 : 50,000.(Hint: Constuct the interval for ln(p) and then take antilogs.) (e) Repeatpart (d) using the unconstrainedmodel from part (a). Comment.
(a) 1 ( ' (b)
(c) I
f
Follt subs are c HW HE HA CIT
(a) I
I
(b) I
6 . 9E X E R C I S E S 1 6 1
)DEL
-. I 1 Considerproductionfunctionsof theform Q:f&, K-;,whereQ is the outputmeasure andL andK arelabor and capital inputs,respectively.A popular functional form is the Cobb-Douglasequation 0.067120 0.15'7729
0.014121
rn(Q): Fr * B2ln(r)+ B3ln(K)+ e (a) Use the data in the file cobb.dat to estimate the Cobb-Douglas production function. Is there evidence of collinearity? (b) Re-estimatethe model with the restriction of constant returns to scale,that is, Fz * B3 : 1, and comment on the results. ' ilr' Using data in the file beer.dat,perform RESET tests on the two alternative models
0.000000 rn(Q,) :9r
* 9z ln(PB,) + B3ln(Pt) * Baln(PR,) * B5ln(f ) * e1 Qt:9t
* B2PB.-f $3PL7* lqPRt * !sI, * e,
Which model seemsto better reflect the demand for beer? . .l Thefiletoodyay.dat contains48 annualobservationson anumberof variablesrelated to wheat yield in the Toodyay Shire of WesternAustralia, for the period 1950-1997. Those variables are )s, show ho\ ' eld restricted
Y: / : RG : RD : RF :
wheat yield in tonnes per hectare, trend term to allow for technological change, rainfall at germination (May-June), rainfall at development stage (July-August), and rainfall at flowering (September-October).
The unit of measurementfor rainfall is centimeters.A model that allows for the yield responseto rainfall to be different for the three different periods is Yt : 9t * \zt * B3RG,+ p4RD, + S5RF,I e1 Use the data
x$alo.
(a) Estimate this model. Report the results and comment on the signs and significance of the estimated coefficients. (b) Testthe hypothesisthat the responseof yield to rainfall is the sameirrespectiveof whether the rain falls during germination, development,or flowering. (c) Estimatethe model underthe restriction that the threeresponsesto rainfall arethe same.Comment on the results. -i Following on from the example in Section 6.6.1, the file hwage.datcontains another subsetof the data usedby labor economistTom Mroz. The variableswith which we are concernedare
only sample d coefficient
nge if prices for Q when t the interval nment.
HW : husband'swage in 2006 dollars, I1E : husband'seducation attainment in years, IIA : husband'sage, and CIT : a variable equal to 1 if living in a large city, otherwise 0. (a) Estimate the model HW:h*gzHE*gzHA-te What effect do changesin the level of education and age have on wages? (b) Does a RESET test suggestthe model in part (a) is adequate?
162
FURTHER
INFERENCE
IN THE MULTIPLE REGRESSION MODEL
(c) Add the variables HE andHA2 to the original equation and reestimateit. Describe the effect that education and age have on wages in this newly estimated model. (d) Does a RESET test suggestthe model in part (c) is adequate? (e) Reestimatethe model in part (c) with the variable ClI included. What can you say about the level of wages in large cities relative to outside those cities? (0 Do you think CIf should be included in the equation? (e)For both the model estimated in part (c) and the model estimated in part (et evaluate the followins four derivatives: .. 1HW ^ (i) ,-- for HE: " OHE
.., atw ^
(11) ------ for HA:
6 and HE : 15 35 and HA : 50
Does the omission of CITlead to omitted-variablebias? Can you suggestwhy? 6.15 In Exercise5.10 you used data in the file bxdatto estimatethe regressionmodel PRICE : Fr * \zSQFT + %AGE * e for (i) all the housesin the sample,(ii) torln houses,and (iii) French style homes. (a) For eachof thesecategoriestest that, on average,the price of a 4O-year-oldhouse of size 3600 squarefeet is more than double that of a 5-year-oldhousewith 180(t square feet. Set up this conjecture as the alternative hypothesis and use a 5Q significance level. (b) Using all the houses in the sample, and a 5Vo significance level, test thr hypothesis that, on average,the following houses are all equally priced: (i) a new houseof 2000 squarefeet, (ii) a2}-year-old houseof 2200 squarefeet, and (iii) a 40-year-old house of 2400 squarefeet. (Cany out the test in terms of the expected pnce E(P RICE).) (c) Does a RESET test suggestthis model is a reasonableone?If not, can you make suggestionsfor improving it? 6.16* Reconsider the presidential voting data (faindat) introduced in Exercise 2.14 and used asain in Exercise5.11 to estimatethe resressionmodel voTE : fu t \zGROWTH + S3INFI.ATION* e In order to get reelected,PresidentWillie B. Great believes it is worth sacrificing some inflation as long as more growth is achieved. (a) Test the hypothesisthat a lVo increasein both GROWTH and INFLATION wrll leave VOTE unchangedagainst the alternative that VOTE will increase. (b) Test the hypothesisthat Willie will not get reelectedagainstthe alternativethat he will get reelected when the growth rate is 4Vo and the inflation rate is 5%. (Cany out the test in terms of the expected vote E(VOTE) and use a 5-r significance level.) 6.11 Reconsiderthe commutinstime model estimatedin Exercise5.12 usinethe datafile commute.dat
(b) Us
isi tht (c) Wt lea tes thr (d) Su ler hir thr (e) Su on
^ . 18 * Recon
the file ln(l
Us wi (b) Us fur (c.) Us (b) (d) Us eq alt
(a)
Reest and (ii estima RESE'
.{.ppendi
This appendi
:ras an F1,r,l
:ntroducea X 'ingle or joir elready have --onespondin The startit
TIME : fu * \zDEPART + 93REDS* BaZRAINS* e (a) Using a57o significancelevel, test the hypothesisthat the delay from a train is equal to three times the delay from a red light.
ln words, V1| r:ed to test 1
J MODEL
,timateit. Describe :stimatedmodel. :? ed. What can you e thosecities? imated in part (e
r suggestwhy? regressionmode. : sample,(ii) tou r 40-year-oldhous.' ldhousewithlStX esis and use a 5'i rce level, test thc lually priced: (i) .' C0squarefeet,anu testin terms of thc not, can you make Exercise2.14 anc
APPENDIX 6A. CHI_SQUARE AND F_TESTS: MORE
DETAILS
163
(b) Using a 5Vosignificancelevel, test the null hypothesisthat the delay from a train is at leastthreetimes greaterthan the delay from a red light againstthe alternative that it is less than three times greater. (c) Wonied that he may miss an important meeting if there are three trains, Bill leavesfor work at 7: l0 AM insteadof 7:15 AM. Using a 57osignificancelevel, testthe null hypothesisthat leaving5 minutesearlieris enoughtime to allow for three trains against the alternative that it is not enough time. (d) Supposethat Bill encountersno red lights andno trains.Using a 5% significance level,testthe hypothesisthat leavingCarnegieatl:15 AM is early enoughto get him to the University before 8:00 AM againstthe alternativeit is not. (Carry out the test in terms of the expectedtime E(TIMD.) (e) Supposethat Bill leavesCarnegieat 7:00 AM and encounterssix red lights and one train. Find a957o interval estimatefor the time he arrives at the University. ' 18't Reconsiderthe productionfunction for rice estimatedin Exercise5. 13 using datain the file rice.dat rn(PROD):
9r * B2Ln(AREA)+ lzln(tABOR) + F+tn(FERT) -t e
(a) Using a57olevelofsignificance,testthe hypothesisthat the elasticityofproduction with respectto land is equal to the elasticity of productionwith respectto labor. (b) Using a l}Vo level of significance,test the hypothesisthat the production function exhibits constantreturnsto scale.That is, Ho:92 * 9: + 9+ : 1. (c) Using a5Tolevel ofsignificance,jointly test the two hypothesesin parts(a) and (b). That is, Ils: Fz : 9: andBz * F: * Ba : 1. (d) Using a 5Volevel of significance,testthe hypothesisthat the meanof log output equals1.5whenAREA : 2, IABOR : 100, andF ERT : 175.Statethe null and alternativehypothesesin terms of Fr, Fz, 83, and Ba. .9'i' Reestimatethe model in Exercise6.18 with (i) FERZomitted, (ii) LABOR omitted, and (iii) AREA omitted.In eachcasediscussthe effect of omitting a variableon the estimatesof the remaining two elasticities.Also, in each casecheck to see if the RESET test has picked up the omitted variable.
Appendix 6A Chi-Square and F-tests: More Details s worth sacrificing I INFI,ATION wil:. ill increase. the alternativethat flation rate is 5% E) and use a 5ci lusing the datafile
e ,lay from a train i:
.his appendixhas two objectives.The first is to explain why the statistic F_
(ssER- ssEu)lJ ssEu/(N- 1()
(6A.1)
-..rran F1;,,v-r1-distributionwhen a specified null hypothesisis true. The other is to :rroducea 1t 1chi-square) statisticthat is also usedfor testingnull hypothesescontaining .:rsle or joint hypothesesabout the coefficientsin a regressionrelationship.You may .,ready have noticed and wondered about computer output that gives a 1'-value and - 'rrespondingp-value in addition to the F-value and its p-value. The startingpoint is the result that, when the null hypothesisbeing testedis true, Vt:
- s.sEu) (s.sER o-
x?n
(64.2)
.:: ivords,V1has a 12-distributionwith ,I degreesof freedom.If o2 was known, V1could be -.ed to test the null hypothesis.There are t\e'oways of overcoming the problem of an
FURTHER
INFERENCE
IN THE MULTIPLE REGRESSION MODEL
unknown ot. One leadsto the F-statistic in (6A.1); the other yields the 12-statisticyou may have beenwondering about. Consideringthe secondone first, one way to obtain a workable test statistic is to replace o' in 161..2;with its estimate o2 from the unrestrictedmodel. If sample size is sufficiently large, it will be approximately true that
(ssER- ssEu) .i v':7#-x?vt
(6A.3)
For testing
The F- and X2
This statistic can be used to test hypothesesabout the unknown regressioncoefficients. AtasqosignificancelevelwerejectH1ifitisgreaterthanthecriticalvaluel2a.nr,r,,orifthe p-value PIX?,> Vrl is lessthan 0.05.
Appendix
To describe the secondway of eliminating the unknown o2 we introduce the result
Consider the n
vz:w -r)62-x?r-*t
(6A.4)
This result is the multiple regressionextensionof the simple regressionresult given in equation (3A.4) of the appeldix to Chapter 3. We are now in a position to use the result that the ratio of trvo independent1'random variables,eachdivided by their respectivedegreesoffreedom, is an F random variable.That is, from (B.35) in Appendix B at the end of the book,
Supposethat v
*here v; : B3
F:y#r-F(m1,m2) In the context of our problem
The secondequ 68.1) yields
(6A..5)
(ssER- ssEu)/J -
: ----F-
r17'w-K)
Hence,the mei
The two o''s in Vl andV2cancel.Also, although we have not done so, it is possibleto prove that V1 andV2 are independent. Noting thatGL : SSE7I(N - K), we can seethat (64.5) and (64.1) are identical. The F-statistic in (6,{.5) is the one we have usedthroughout this chapterfor testing hypotheses. What is the relationship between V 1 andF given in (6A.3) and (6A.5), respectively?A moment's thought reveals that -Vl
,:T
The F-value is equal to the 12-value divided by the number of restrictions in the null hypothesis.We can confirm this relationship by reexamining some examples. When testingHs:$3 : B+ : 0 in the equation Si : Br *gzPi*
F : A i + g q A ?+ e i
we obtain F :8.44
p-value: 0.0005
X2 : 16.88 p-value:0.0002 Becausethere are two restrictions (J :2),the different becausethe tests are different.
F-value is half the 12-value.Thep-values are
Thus, the restr .-ovariancebetv tiom the regres :he simple corn estimatorin the that omitting an $at sucha viole sntement. Omi equationyields nonzero mean remaining varic
DEL
APPENDIX 68
,ticyou may r a workable ld model. If
(6A.3) coefficients. s , ; ; ,o r i f t h e the result
OMITTED-VARIABLE
BIAS: A PROOF
165
For testingt1e: B3 * 3.8 94 : I (seeSection6.4.2), we obtain F :0.936
p-value: 0.3365
x2 :0.936
p - v a l u e: 0 . 3 3 3 3
lhe F- and 12-valuesare equal becauseJ : l, but again the p-valuesare different.
.\ppendix 68 Omitted-Variable
Bias: A Proof
'onsider the model )i : 9t l9zxiz * 9zxrzI ei
(6A.4r n in equation rt the ratio of f freedom,is ok,
\ jppose that we incorrectly omit x3 from the model and estimateinstead )i:Br*9zxizlvi ^rere vi :9$n
+ er. Then,the estimatorusedfor B2 is b;:
L(*,, -tz)(y, - y) : 9 2 *Lwtvt L(xn - 7z)z
^ rere
-\
/
'
(68.1)
\xi2 - x2) L1xi2 - i212
l:e secondequalityin (68.1) follows from Appendix2D in Chapter2. Substitutingfor v; in -B.l ) yields
(6A.5r
bi:
9 z I F z Z w i x i zl L w i e i
E(bil:92 llz}wixiz ible to prove Ientical.The hypotheses. rectively?A
:92
^ I(to
-iz)xiz
+D?----.-...--..........-.".".........'' S/ \Z
L\x;2 - x21
-72)(4 -tu) : B z * 9 : L(r,, -
Z(xiz - 7z)z
cov(x2.x3) : ^Pz -r, ^I'3 rlt2 @ r in the null
p-values are
-.rs. the restricted estimator is biased. Knowing the sign of B3 and the sign of the . '.ariancebetweenx2andx3tells us thedirectionof thebias.Also, while omitting a variable - ,n the regressionusually biasesthe leastsquaresestimator,if the samplecovariance(or -: simplecorrelation)betweenx2 and the omittedvariablex3 is zero,thenthe leastsquares : -:rmatorin the misspecifiedmodel is still unbiased.Way back in Section2.2,we suggested -,it omitting an importantfactor will lead to violation of the assumptionSR2 E(e) : Q3n6 ::t sucha violation canhaveseriousconsequences. We cannow be morepreciseaboutthat '-:terrrert.Omitting an importantvariablethat is correlatedwith variablesincluded in the :";:-ration yields an error that we havecalledv1in the abovediscussion.This error will havea ' rzero mean and the consequencesare biased estimatesfor the coefficients of the -:lainins variablesin the model.
Chapter
Nonlinear Relationshipt
Learning Objectives
:iscoveredthat by .,.e couldaccomm \.3 and A.4, for f We will consid .:tuationsin whicl ,.rnsiderusefulne :re use of dummS ,rually 0 and 1. T -raracteristicsofi :e neral,we usedu i'inally, we make t ' .i o or more expla , eriables,or intera ..e will discussea
Based on the material in this chapter you should be able to explain L
The difference between qualitative and quantitative economic variables.
2. How to include a 0-1 dummy variable on the right-hand side of a regression,hou this affects model interpretation, and give an example. 3. How to interpretthe coefficienton a dummy variable in a log-linearequation. 1. How to include a slope dummy variable in a regression,how this affects model interpretation, and give an example. -5. How to include a productof two dummy variablesin a regression,how this affect: model interpretation, and give an example. 6. How to model qualitative factors with more than two categories(like region of the country), how to interpret the resulting model, and give an example. 7. The consequencesof ignoring a structural change in parametersduring part of the sample. 8. How to test the equivalenceof two regressionequationsusing dummy variables. 9. How to interpret the coefficient of an interaction betweentwo continuousvariables.
7.1 Polyno
?..lvnomialsarea r -:: curved,with o -.1.1
Cosr eN
-:. microeconomi - -ral cost and totr :ubic" shapessh :rr;rges, averageaI .:orrn in Figure ' 'r'lresentedby reg -:e easily repres --'.etionship,in Fi
Keywords annualdummy variables binary variable Chow test collinearity dichotomousvariable dummy variable
dummy variable trap exact collinearity hedonic model interaction variable intercept dummy variable log-linear models
nonlinear relationship polynomial reference group regional dummy variable seasonaldummy variables slope dummy variable
In this chapter we consider methods for adding more flexibility to the regressionmodel. The tools we introduce allow us to deal with the fact that most economic relationships are not linear.By that,we meanthatrelationshipsbetweeneconomicvariablescannotalwaysbe represented by straightlines.The essenceof a linear relationshipis that its slopeis constant anddoesnot dependon the valuesofthe variables.Ifthe slopeofthe relationshipchangesfor any reason,then the relationshipis said to be nonlinear. This problem was introducedin Sections 4.3 and 4.4, where we examined the choice of model functional form and
166
Ila r
, ; c n e7 . 7 ( a )
T.l POLYNOMIALS
167
discoveredthat by creativelyusing logarithms,reciprocals,squares,and cubesofvariables s e could accommodaterelationshipswith manydifferentshapes.SeeAppendixA, Sections .\.3 and A.4, for further discussionof linearity versusnonlinearity. We will consider several procedures for extending the multiple regression model to .ituationsin which the slope of the relationshipchangesin one way or another.First, we ,-onsiderusefulnessof polynomials to capturecurvilinear relationships.Second,we explain lhe use of dummy variables, which are explanatory variables that take only two values, ::sually 0 and l. These simple variables are very powerful tools for capturing qualitative .-haracteristicsofindividuals, such as gender,race, and geographicregion ofresidence. In :eneral,we usedummy variablesto describeany eventthat hasonly two possibleoutcomes. Finally, we make use of interaction variables. These are variablesformed by multiplying :$ o or more explanatory variables together.When using either polynomial terms, dummy '' ariables,or interaction variables, some changesin model interpretation are required, and ;ie will discusseach of thesecases.
iables.
7.1 Polynornials
regression,how 3ar equation. is affects model
Polynomialsarea rich classof functionsthatcanparsimoniouslydescriberelationshipsthat .re curved, with one or more peaks and valleys. Consider the following examples. -.1.1
Cosr eNo Pnooucr Cunvps
how this affects ike region of the rle. uring part of the mmy variables. nuousvariables.
In microeconomicsyou studied "cost" curves and "product" curves that describe a firm. Total cost and total product curves are mirror images of each other, taking the standard "cubic" shapesshown in Figure 7.1. Averageand marginal cost curves,and their mirror .:nages,averageandmarginalproductcurves,takequadraticshapes,usuallyrepresentedas -hown in Figure 7.2. The slopes of these relationships are not constant and cannot be :r'presentedby regressionmodels that are "linear in the variables." However, theseshapes .re easily representedby polynomials. For example, if we consider the averagecost :r'lationship,in Figure 7 .2a, a suitableregressionmodel is
AC:9t+?zQ-rFzQ'*e
(7.1)
:lationship roup mmy variable Lmmyvariables ny variable
gressionmodel. elationships are annotalwaysbe ilopeis constant shipchangesfor n introducedin ional form and
Cost
a (a)
, r c u np 7 . 7 (a) Total costcurve and (b) total productcurve.
(b)
168
NONLINEAR
RELATIONSHIPS
Cost
Table 7.7 \ ariable
C EDUC T.XPER LXPER2 !t- :0.2709 Input FrcuRE 7.2
(b\ \a) Averageand marginal (a) cost curves and (D) product curves.
This quadratic function can take the "U" shape we associate with average cost functions. For the total cost curve in Figure 7.la a cubic polynomial is in order,
TC : ar -t aze I azez+ aae3+ e Thesefunctionalforms, which representnonlinearshapes,can still be estimatedusing the least squaresmethodswe have studied. The variables Q2 and Q3 are explanatoryvariibles that are treated no differently from any others. An interesting aspect of models of nonlinear relationships is the interpretation of the parameters-they are no longer slopes. Indeed, the slope of the average cost curve ( 7 . 1 )i s
dEAC) :92-r 2$tq ,O
(7.3)
That is, the slope of the averagecost curve changesfor every value of p and dependson the parametersB2 and B3.For this U-shapedcurye we expect gz < 0 and B: > 0. The slope of the total cost curve (7.2), which is the marginal cost, is
dE(TC\ t:
cr2* 2uzQ* 3a+e2
(7.4)
The slope is a quadratic function of Q, involving the parameterscr.2)d.3,ord cr4.For a U-shapedmarginal cost curve we expect the parametersigns to be cr2) 0, a3 ( 0, and a.r ) 0. Another example of a polynomial nonlinear relationship is the quadratic nonlinear relationshipbetweensalesand advertisingexpenditureintroducedin Section6.3. It was similar to (7.1) with a slope that dependedon advertisingexpenditure. Using polynomial terms is an easy and flexible way to capture nonlinear relationships betweenvariables.As we have shown care must be taken when interpreting the parametersof models containing polynomial terms. Their inclusion does not comp_ licate least squaresestimation, with one exception. It is sometimestrue that having a variable and its square or cube in the same model causescollinearity problems. See Section6.7.
'.1.2
A Wa
ln Section4.4 :-ducation.Nov ;rorker's years :raverelatively .rill begin to de J) cle pattern of ,i wages
Ttr obtain the i :'\perienceon r
fhe peakofthe :t which the slr To illustrate -ontainedin tht :trefficients on :nd are statisti For these 19 :nateachadditi< -.6), the estim . S yearsis
' This importan . .;ob Mincer ( 195 :.. otrcmy,66(4),28 .idrtion the relatior -:etly summarized \tA: Addison-Wes
T.l POLYNOMIALS
Table 7 .7
AP
\r,
\ ariable
Coefficient
C EDUC EXPER EXPERz
-9.8t77
R::0.2709 -
Input
: cost functions
(7.2) matedusing the natory variables nterpretation of rragecost curve
(7.3) I dependson the >0.
(7.4) ], and ct4.For a t 0, cr3( 0, and Jratic nonlinear :tion 6.3. It was rlinear relationinterpreting the loes not compe that having a problems. See
Wage Equation with Quadratic Experience Std. Error
1.0550 0.0702 0.0514 0.0012
t.2r0r 0.3409 -0.0051
/-Statistic
Prob.
-9.3062
0.0000 0.0000 0.0000 0.0000
17.2282 6.6292 -4.2515
SSE: 28420.08
t'.1.2 AWecs EquerIoN ln(wages)asa functionof In Section4.4.2we introduceda wageequation,andwe expressed education.Now we add another important determinant of wagesto the model, namely the u orker's years of experience.What we expect is that young, inexperienced workers will rave relatively low wages; with additional experiencetheir wages will rise, but the wages + ill begin to decline after middle age,as the worker nearsretirement.' To capture this life.'1cle pattern of wageswe introduce experienceand experiencesquaredto explain the level .ri wages
WAGE: 9r * P2EDUC* 1tEXPER* 9+EXPER'-l e
(7.s)
To obtain the inverted-U shape, we expect B3 > 0 and F+ < 0. The marginal effect of :rperience on wage, holding education and other factors constant,is
aE(wAGE): Br * 2B-'EXIER AEXPER
(1.6)
The peakof the wage--experiencecurve occurswhere EXPER : -92129q. This is the point :t which the slope in (7.6) is zero. To illustrate we use data on 1000 workers from the 1997 Current PopulationSurvey :onrainedin the file cps_small.dar.The estimationresultsare in Table 7.1. The estimated --.)gfficients on education,experience,and experiencesquaredhave the anticipatedsigns are statistically significant. "nd For these 1997 data,the dependentvariable is earningsper hour ($). Thus we estimate :hateachadditionalyearofeducationleadsto an increasein hourly earningsof$1.21. Using -.6), the estimateof the marginal effect of experienceat the samplemedian experienceof . S yearsis
YWJI 1EXPER
:0.1s76 : o.34oe + 2(-o.oo5l)18 lurrr*:,,
I This important economic relationship, still widely studied by labor economists,was discussedby economist .::ob Mincer (1958) "Investment in Human Capital and Personal Income Distribution," Journal of Political ,..()nomy,66(4),28I-302.Wagesare often expressedin logarithmicform, as we do in Section7'5, and with this ..iJirion the relation in equation (7.5) is called the "Mincer equation." Both the theory and empirical issuesare :carly summarized in Emst Bemdt (1991) The Practice of Econometrics: Classic and Contemporary, Reading. \1.\: Addison-Wesley,Chapter5.
170
NONLINEAR
RELATIONSHIPS
We estimatefor a worker with 18 yearsof experience,that an additional year of experience increaseshourly wage by 15.76 cents. The turning point in the relationship, after which wages are expected to diminish with additional experience, is estimated to occur at -0340912(- 0.0051):33.47 years. EXPER : -Fsl29+:
Dummy variable \\'e consider thes
7.2 Dsrnrny Variables
The most comm( rarameter. Addir :.]rameter 6, we ,
Dummy variables allow us to construct models in which some or all regression model parameters,including the intercept, changefor some observationsin the sample.To make matters specific, let us consider an example from real estateeconomics.Buyers and sellers of homes, tax assessors,real estate appraisers,and mortgage bankers are interested in predicting the current market value of a house. A common way to predict the value of a houseis to usea hedonic model, in which the price of the houseis explainedas a function of its characteristics,suchas its size,location, number of bedrooms,age,and so on. The idea is to break down a good into its component pieces, and then estimate the value of each characteristic.2 For the present,let us assumethat the size of the house,measuredin squarefeet,SQFT,is the only relevantvariable in determining houseprice, PRICE. Specify the regressionmodel AS
PRICE: gr * B2SQFT* e
( 1 . 1r
In this model B2 is the value of an additional squarefoot of living area,and $1 is the value of the land alone. In real estatethe three most important words are "location, location, and location." Hos can we take into accountthe effect of a property being in a desirableneighborhood,such a. one near a university, or near a golf course?Thought of this way, location is a "qualitative " characteristicof a house. Dummy variablesareusedto accountfor qualitative factorsin econometricmodels.Ther areoften called binary or dichotomous variablesasthey takejust two values,usually 1 or 0. to indicate the presenceor absenceof a characteristic.This choice of valuesis arbitrary but very convenient as we will see.Generally we define a dummy variable D as | n _ - | " i0
if characteristicis present if characteristicis not present
(7.8r
7.2.1
lNrsacr
The effect of the i :ramining the re1 -orrectly specifie
.r thedesirablenr -r other areasthe ij:eure7.3, assu Adding the dt :.'lationshipby th :rrameter 6 is th . tated in the des ::'gressionmodel -rlled an interce :.:gherin a desir The least squ :rplanatoryvaria ..iriable.We can :'astsquaresesti :ouseprice is "st :.*ighborhoodin
Thus, for the houseprice model, we can define a dummy variable, to accountfor a desirable neighborhood,as
^ _ - lt " l0
if propertyis in the desirableneighborhood if propertyis not in the desirableneighborhood Fr+6
2
Such models have been used for many types of goods, including personal computers,automobiles and wine. This famous idea was introduced by Sherwin Rosen (1978) "Hedonic Prices and Implicit Markets," Joumal o.i Political Economy,82,357-369. The ideas are summarized and applied to asparagusand personal computers in Ernst Bemdt (1991) The Practice of Econometrics: Classic and Contemporary, Reading, MA: Addison-Wesle1. Chapter 4.
Fr
: t c u n n 7 . 3 Ar
7.2 DUMMY
r of experience lp, after which :d to occur at
VARIABLES
171
Dummy variablescan be usedto capturechangesin the model intercept,or slopes,or both. \\'e considerthesepossibilitiesin turn.
7.2.1 INrrRcspr Duuuv Venreerss
ressionmodel nple. To make 'ersand sellers : interestedin the value of a n a function of on. The ideais value of each :feet,SQFT,is ;ressionmodel
(1.'7) is the value of rcation."Hou rhood,suchas "qualitative" models.Thel usually I or0. s arbitrary but IS
(7 8)
The most common use of dummy variables is to modify the regression model intercept rrrameter. Adding the dummy variable D to the regression model, along with a new rrrameter 6, we obtain
PRICE: gr * 6D+ gzSeFT+ e
(7.e)
The effect of the inclusion of a dummy variable D into the regressionmodel is best seenby :ramining the regressionfunction, E(PRICE), in the two locations.If the model in (7.9) is -orrectly specified,then E(e): 0 and
[ ( B-r+ 6 ) + B z s O r Z w h e n D : I E(PRICE\: i " whenD:0 l9r + BzSQFZ
(7.r0)
. r thedesirableneighborhoodD : 1,andthe interceptofthe regressionfunctionis (B r * 6). .r otherareasthe regressionfunction interceptis simply B1.This differenceis depictedin fisure 7.3, assumingthat 6 > 0. Adding the dummy variable D to the regression model causesa parallel shift in the ::'lationshipby the amount 6. In the context of the houseprice model the interpretationof the :rrameter E is that it is a "location premium," the difference in house price due to being 'catedin the desirableneighborhood.A dummy variable like D that is incorporatedinto a :r'gressionmodel to capturea shift in the intercept as the result of some qualitative factor is -illed an intercept dummy variable. In the houseprice examplewe expectthe price to be -.:gherin a desirablelocation, and thus we anticipatethat 6 will be positive. The least squaresestimator's properties are not affected by the fact that one of the -'rplanatoryvariablesconsistsonly of zerosandones-D is treatedasany otherexplanatory . rriable. We can constructan interval estimatefor E, or we can test the significanceof its 3dstsquaresestimate.Sucha testis a statisticaltestofwhether the neighborhoodeffect on :r)useprice is "statisticallysignificant."If 6 : 0, then thereis no locationpremium for the :.eighborhoodin question.
or a desirable
E(PRICE)=(Br +6) +pzSQFr = Bt + p2SQFT E (PRICE)
0r*6 Fr
tobilesand wine. <ets,"Joumal of :lal computers in \ddison-Wesley.
: rcunp
7.3
An interceptdummv variable,
NONLINEAR
RELATIONSHIPS
7.2.1a Choosing the Reference Group The convenienceof the valuesD : 0 and D : I is seenin (7. 10). The value D : 0 defines the reference group, or basegroup, ofhousesthat arenot in the desirableneighborhood. The expectedprice of thesehousesis simply E(PRICE): 9r * PISQFT. Using (7.9) we are comparingthe houseprices in the desirableneighborhoodto thosein the basegroup. A researchercan choose whichever neighborhood is most convenient for expositorl purposes,to be the referencegroup. For example,we can define the dummy variable LD to denote the less desirable neishborhood 1 if property is not in the desirable neighborhood ,n _ - I "" \O if property is in the desirable neighborhood ThisdummyvariableisdefinedjusttheoppositefromD,andLD:I - D.IfweincludeID in the model specification PRICE:
BI * }\LD + \ZSQFT * C
,-hangein the slopt tor housesin the r reighborhoods.D, .rtherexplanatory :rvodifferent locat :nto the economic E ( P R I C E ): 9
In the desirable nt 'rher locations.\\ lesirable neighbo Anotherway to :artial derivative ,rhich gives the s
then we make the reference grorp, LD : 0, the housesin the desirable neighborhood. You may be tempted to include both D and LD in the regressionmodel to capture the effect of each neighborhood on house prices. That is, you might consider the model
ii the assumption :ave their usualgc
PRICE: Br * ED + \tD l \2SQFT _l e In this model the variablesD andLD aresuchthat D I LD : 1. Since the intercept variable xr : 1,we havecreateda model with exact collinearity, and asexplainedin Section6.7, the least squaresestimator is not defined in such cases.This error is sometimesdescribed as falling into the dummy variable trap. By including only one of the dummy variables,either D or LD, the omitted variable defines the reference group, and we avoid the problem.'
PRICE
I L
Pr> 7.2.2
SropE Duumy VRnresrss
Instead of assumingthat the effect of location on house price causesa change in the interceptof the hedonicregressionequation(7.7),let us assumethat the changeis in the slope of the relationship. We can allow for a changein a slope by including in the model an additional explanatory variable that is equal to the product of a dummy variable and a continuous variable. In our model the slope of the relationship is the value of an additional square foot of living area. If we assume this is one value for homes in the desirable neighborhood,and anothervalue for homes in other neighborhoods,we can specify
PRrCE: Br * \2SQFT+ t6QFTxD) -l e
PRICE
( 7. r 1 t
The new variable (SQFT xD) is the product of house size and the dummy variable, and is called an interaction variable, as it capturesthe interaction effect oflocation and size on house price. Alternatively, it is called a slope dummy variable, becauseit allows for a
3 Another way to avoid the dummy variable trap is to omit the intercept from the model. This choice is less desirable than omitting one dummy variable, since omitting the intercept alters several of the key numerical properties of the least squaresestimator.SeeAppendix 48 and Section 5.6. Omitting the intercept causesthe basic definition of model R2 to fail. which is inconvenient.
I
,rr
Fr
tu?
L
rcunn 7,4
(a
7.2 DUMMY
) :0 defines ighborhood. ring(7.9) we basegroup. 'r expository riable LD to
173
changein the slopeof therelationship.The slopedummy variabletakesa valueequalto size lor housesin the desirableneighborhood,when D : l, and it is zero for homes in other neighborhoods. Despiteits unusualnature,a slopedummy variableis treatedjust like an1 otherexplanatoryvariable in a regressionmodel. Examining the regressionfunction for the trvodifferent locationsbestillustratesthe effect of the inclusion of the slopedummy variable into the economicmodel,
E(pRrcE):gr * gzseFr+t6eFrxD) :
s include lD
VARIABLES
whenD : f Br * (Br + "y)SOFZ -
t;; *Or'sqi7
I
whenD:0
In the desirableneighborhood,the price per squarefoot of a home is (Fz + 1); it is p1 in other locations. We would anticipate "y> 0 if price per squarefoot is higher in the more desirableneighborhood.This situationis depictedin Figure 7.4a. Anotherway to seethe effectof includinga slopedummy variableis to usecalculus.The partial derivative of expectedhouse price with respect to size (measuredin square feet). shich gives the slope of the relation,is
aEeRrcE)
rborhood. , capture the : model
ASQFT
[92+t lBz
whenD: I whenD:0
If the assumptionsof the regressionmodel hold for (7. I 1), then the least squaresestimators havetheirusualgoodproperties,asdiscussedin Section5.3.A testofthe hypothesisthat the :eptvariable :tion 6.7,the lescribed as ables,either problem.3
PRICE
Slope=B2+y
E (PRICE= Ft + (Fz+9 SQFT E (PRICD = Bt + p2SQFT
Slope= P,
ange in the rge is in the [e model an iable and a n additional Ledesirable specify
= (Br + 6) + (gz+i SQFT E (PRICE)
(7.r1) able, and is and size on Lllowsfor a
choice is less <ey numerical rusesthe basic
E ( P R I C E=) ( p r + 6 ) + p 2 S Q F T E (PRICE=Bt+ B2SQFT
Fr*6 Fr
SQFT (b)
r r cune 7 . 4 (a) A slopedummy variable.(b) A slopeand interceptdummy variable.
174
NONLINEARRELATIONSHIPS
Table 7 .2
Table7.3
RepresentativeReal Estate Data Values
PRICE
SQFT
AGE
UTOWN
POOL
FPLACE
205.452 185.328 248.422 287.339 255.325 301.037
23.46 20.03 27.77 23.67 21.30 29.8'l
6 5 6 28 0 6
0 0 0 1 I I
0 0 0 I I 0
1 I
0 0 I I
value of a squarefoot of living areais the samein the two locations is carried out by testing the null hypothesisHs:1 :0 againstthe alternativeHt:^y lO.ln this case,we might test Hs:1 :0 againstH1:\) 0, sincewe expectthe effectto be positive. If we assumethat house location affects both the intercept and the slope, then both effects can be incorporated into a single model. The resulting regressionmodel is PRrCE: Br *6D+\2SQFT
+t6QFTxD)
*e
(7.121
In this casethe regressionfunctions for the house prices in the two locations are
E(PRICE): ' {f'
.iliil'
f Br + 92SQFT
F
Variable L
UTOWN SQFT SQFTXUTOWN .1GE POOL FPI.4CE
R2: 0.8706
R 2: 0 . 8 7 0 6 i n c SQFTxUTOWN null hypothesesI exceptfor the coe r-tests,we conclu rheir price per sq The estimate
PRICE: (24
: 1 + t)sQFr whenD
+.
whenD :0
51
In Figure 7 .4b we depict the house price relations assuming that D > 0 and ^y> 0.
For housesin ot 7.2.3
AN ExauprE:
FFidE:2
TrrE UNrvrRsrry EFFEcT oN FIousE Pnrcss
A real estateeconomist collects information on 1000 house price salesfrom two similar neighborhoods,one called "University Town" bordering alarge state university, and one that is a neighborhood about 3 miles from the university. A few of the observationsare shown in Table 7.2. The complete data file is utown.dat. House prices are given in $ 1000; size (SQFT) is the number of hundredsof squarefeet of living area.For example,the first housesold for $205,452andhas2346squarefeetofliving area. : lforhomesneartheuniversity, AlsorecordedarethehouseAGE(inyears),locanon(UTOWN 0 otherwise),whetherthe househas a pool (POOL: 1 if a pool is present,0 otherwise)and whetherthe househasa fireplace(FPLACE : I if a flreplaceis present,0 otherwise). The economistspecifiesthe regressionequationas
Basedon these a
a
the locatir
the price p in other a
a
that hous
I
that a po(
a
that a fire
7,3 Appr PRICE: Br * \:UTOWN+ \ISQFT + TGQFTxUTOWN) + %AGE+\IPOOL-I\zFPLACE + e
(7.r3)
We anticipatethat all the coefficientsin this model will be positiveexcept83, which is an estimateof the effect of age,or depreciation,on houseprice. Note that POOL andFPLACE are intercept dummy variables.By introducing thesevariableswe are asking if, and by how much, thesefeatureschangehouseprice. Becausethesevariables stand a7one,and arc not interacted with SQFT, we are assuming that they do not affect the price per squarefoot. The estimatedregressionresults are shown in Table l.3.The goodness-of-fitstatisticis
Dummy variabl ive consider sot 7.3.I
INrrn-t
We have seen regressionmodr effect of each q
7.3 APPLYING DUMMY
Table 7 .3 !'ariable
Coefficient
Std. Error
I I 0 0 I 1
C LITOWN
24.5000 27.4530 7.6122 r.2994 -0.1901
6.19t7 8.4226 0.2452 0.3320 0.0512 1.1967 0.9720
,, then both del is
(7.r2) are
175
flouse Price Equation Estimates
FPLACE
lt by testing 3 might tesr
VARIABLES
sQFT SQFTxUTOWN 4GE POOL FPI-A,CE
4.3772 1.6492
Rr : 0.8706
t-Statistic
3.9569 3.2594 31.0478 3.9133 -3.7 t23 3.6577 1.6968
Prob.
0.0001 0.00r 2 0.0000 0.0001 0.0002 0.0003 0.0901
SSE:230184.4
R' :0.8706 indicating that the model fits the data well. The slope dummy variable is SQFTxUTOWN.Basedonone-tailr-testsofsignificance,atthee:0.05 levelwerejectzero null hypothesesfor eachof the parametersand acceptthe alternativesthat they are positive, erceptfor the coefficientonAGE, which we acceptto be negative.In particular,basedon these r-tests,we concludethat housesnearthe university have a significantly higher baseprice and their price per squarefoot is significantly higher than the comparisonneighborhood. The estimated regressionfunction for the housesnear the university is
- o:slrAGE FRrcE: (24.s+ 27.4s3) + (7.6122 + r.zss4)seFT + 4.3772POOL + l.6492FPL4,CE : 51.953+ 8.9I I 6SOFT- 0.190IAGE+ 4.3772POOL+ t.6492FP tAC E
>0.
For housesin otherareas,the estimatedregressionfunctionis F RidE : 24.5+ 7.6r22SQFT- 0.90rAGE + 4.3772PooL + r.6492FPLACE .wo similar ty, and one vations are uarefeet of living area. :university. :rwise) and ;e).
Basedon theseregression results,we estimate a
the location premium for lots near the university is $27,453
a
thepriceper squarefoot is $89.12for housesnearthe universityand$76.12for houses in other areas
a
that housesdepreciate$190.10per year
a
that a pool increasesthe value of a home by $4377.20
a
that a fireplace increasesthe value of a home by $1649.20
7,3 Applying Dummy Variables (1.r3) zhichis an I FPLACE nd by how nd are not uare foot. ltatistic is
Dummy variablescan be usedto ask, and answer,a rich variety of questions.In this section ire considersome common applications. 7.3.1
INrERecrroNs BETvEEN Quaurarrvn
FecroRs
We have seen how dummy variables can be used to represent qualitative factors in a regressionmodel. Intercept dummy variablesfor qualitative factors ne additive. That is, the effect of each qualitative factor is added to the regressionintercept, and the effect of any
176
NONLINEAR
RELATIONSHIPS
dummy variable is independentof any other qualitative factor. Sometimes,however, we might question whether qualitative factors' effects are independent. For example,supposewe are estimating awage equation,in which an individual's wages areexplainedasa function oftheir experience,skill, and other factorsrelatedto productivity. It is customaryto include dummy variablesfor raceand genderin suchequations.If we have modeledproductivity attributeswell, and if wage determination is not discriminatory, then the coefficientsofthe race and genderdummy variablesshould not be significant. Including just race and gender dummies, however, will not capture interactions between these qualitative factors. Is there a differential in wages for black women? Separatedummy variablesfor being "black" and "female" will not capture this extra interaction effect. To allow for sucha possibility considerthe following specification,where for simplicity we use only education (EDUa as a productivity measure,
FEMALEist statisticallyr Suppose do we test tt rhejoint null ofthe indicat of the regres To test tht reststatistic
WAGE: Fr * \2EDUC + 64I_A,CK * \zFEMALE-f "y(BLACKxFEMALE)* e
uhere SSEa * hich the nu the original, numberof de lhe test statis Jenominato
('7.r4)
whereBI,ACK atd FEMALE aredummy variables,and thus so is their interaction.Theseare intercept dummy variablesbecausethey are not interactedwith any continuousexplanatory variable. They have the effect ofcausing a parallel shift in the regression,as in Figure 7.3. When multiple dummy variables are present, and especially when there are interactions between dummies, it is important for proper interpretation to write out the regression function, E(WAGD, for each dummy variable combination.
+ g|EDUC
WHITE-MALE BLA,CK-MALE
E(^AGE, {ili
f. isthecritic To testthe "/ = .um of squa .um of squar leadingto th
+ 6r)+ !2EDUC WHITE-FEMALE + 62) + \\EDUC ^y) * 6r * 6z -| * !zEDUC BI^A,CK-FEMALE
In this specification white males are the referencegroup, becausethis is the group defined when all dummy variablestake the value zero, in this caseBIA,CK : 0 and FEMALE : 0. The parameter bl measuresthe effect of being black, relative to the reference group; the parameter62 measuresthe effect of being female, and the parameter^ymeasuresthe effect of being black and female. Using the CPS data (cps-small.dat) we obtain the resultsin Table 7.4. Holding the effect of education constant, we estimate that black males earn $1.83 per hour less than white males, white females earn $2.55 less than white males, and black females earn $3.80 less than white males. The coefficients of EDUC, BI-{CK, and FEMALE are all significantly different from zero usine individual /-tests. The interaction effect between BLACK and
Table 7.4
W a g e E q u a t i o nw i t h R a c e a n d G e n d e r
Variable
C EDUC BIACK FEMALE BI,ACKxFEMALE R2:0.2482
Coefficient
Std. Error
t-Statistic
Prob.
-3.2303
0.9675 0.0697 0.8957 0.3597 t.2170
-3.3388
0.0009 0.0000 0.04r2 0.0000 0.6291
1 .I 1 6 8 - 1.8312 -2.552r 0.5879 SSE:2930'7.71
16.0200 -2.0444 -'t.0953 0.4831
* hich hasSS '-rnrestrictedt
The lvo cnt .'oncludetha '.3.2
Que
\tany qualit .-fthecountr -ourregion
b,strmann :rimate of 1.2 .naller standan i.rmplex effects :s\ults can be r(
7.3 APPLYING DUMMY
lowever, we lual's wages rroductivity. s.If we have lnatory,then rt. Including tween these :ate dummy rn effect. To licity we use
(7.r4) n. Theseare explanatory Figure7.3. interactions : regression
VARIABLES
177
FEMALEisnot estimatedvery preciselyusingthis sampleof 1000observations. andit is not jtatistically significant.a Supposewe are askedto test the joint significanceof all the qualitative factors.Horr' do we test the hypothesisthat neitherrace nor genderaffectswages?We do it br testing ihejoint null hypothesisI1s:61 : 0, 6z : 0, ^y: 0 againstthe alternativethat at leastone of theindicatedparametersis not zero.If the null hypothesisis true,raceand gendertall out of the regression,and thus they have no effect on wages. To test this hypothesiswe usethe F-test procedurethat is describedin Section6.1. The teststatisticfor ajoint hypothesisis F:
- ssau)lJ (ssER ssEul(N- K)
*here SSE6is the sum of squaredleast squaresresidualsfrom the "restricted" model in * hich the null hypothesisis assumedto be true,SSEyis the sum of squaredresidualsfrom the original, "unrestricted,"model, J is the numberof joint hypotheses.and N - K is the numberof degreesof freedomin the unrestrictedmodel. If the null hypothesisis true,then the test statistic F has an F-distribution with "I numerator degreesof freedom and N - K denominatordegreesof freedom,F1t, N-x). We reject the null hypothesisif F > F., where F. is thecritical value,illustratedin Figure8.8 of AppendixB, for the levelof significancea. To testthe,/ : 3 joint null hypothesesF1e: 61 : g, 6z : 0, ^y: 0, we obtainthe unrestricted .um of squarederrors SSEy : 29308 from the model reported in Table 7 .4. The restricted ium of squaresis obtainedby estimating the model that assumesthe null hypothesisis true, leadingto the fitted model
fredE:-+.s122 + r.r3ssEDUC ('") (0.e668)(0.0716)
U
s rup defined VIALE:0. , group; the .heeffect of
rvhichhasSSEa: 3 1093.The degreesof freedomN - K : 1000- 5 : 995 comefrom the unrestrictedmodel. The value of the F-statisticis
'p
g the effect than white r $3.80less ignificantly TLACK and
Prob. 0.0009 0.0000 0.0412 0.0000 0.6291
- ssEu)lJ (31093 -2e308)13 (ssER 2930819es ssEul(N ,K)
-
-
, I l
, l l
The l% critical value (i.e., the 99th percentilevalue) is [email protected],z,sss):3.80. Thus ue gender affect the wage equation. that race and/or "-onclude 7.3.2
-
-
QuernerrvE
FACToRSwrrrr SEvrneL CATEGoRTES
\lany qualitative factors have more than two categories.An example is the variable re_eion of thecountryin ourwageequation.The CPSdatarecordworkerresidencewithin oneof the tbur regions: Northeast,Midwest, South, and West. Again, using just the simple u aee
* yieldsa coefficient Estimatingthis model usingthe largerdatasetqrr.dat, which contains4733 observations, !'stimateof 1.2685with a t-value of 2.37. Recall from Sections2.4.4 and 5.3.1 that larger samplesizeslead to rmaller standard errors and thus more precise estimation. Labor economists tend to use large data sets so that complex effects and interactions can be estimated precisely. We use the smaller data set as a text example so that results can be replicated with student versions of software.
NONLINEAR
RELATIONSHIPS
specification for illustration, we can incorporate dummy variables into the wage equation as WAGE : 9r * 92EDUC * 6$OUTH -t \zMIDWEST -t itWEST * e
(7.15)
Notice that we have not included the dummy variablesfor all regions.Doing so would have created a model in which exact collinearity exists. Since the regional categories are exhaustive, the sum of the region dummy variables is NORTHEAST + SOUTH+ MIDWEST + WEST : 1. Thus the "intercept variable" \ : I is an exact linear combination of the region dummies.Recall, from Section6.7, that the leastsquaresestimatoris not defined in such cases.Failure to omit one dummy variable will lead to your computer software returning a messagesaying that least squaresestimation fails. This error is the '7.2.1a. dummy variable trap that we mentioned in Section The usual solution to this problem is to omit one dummy variable, which defines a reference group, as we shall see by examining the regressionfunction,
E(WAGE):
* 9zEDUC
NORTHEAST
The omitted dummy variable,NORTHEAS4 identifies the referencegroup for the equation. to which workers in other regions are compared. It is the group that remains when the regional dummy variablesWEST,MIDWEST, andSOUTH are setto zero. Mathematically it does not matter which dummy variable is omitted and the choice can be made that is most convenient for interpretation. The intercept parameter 91 representsthe base wage for a worker with no education who lives in the northeast. The parameter 61 measuresthe expectedwage differential between southernworkers relative to those in the northeast;62 measuresthe expected wage differential between midwestern workers and those in the northeast. Using the CPS data cps_small.daf,let us take the specificationin Table 7.4 and add the regional dummies SOUTH, MIDWEST, andWEST.The results are in Table 7.5. Based on thoseresultswe can saythat workers in the southearnsignificantly lessper hour thanworkers in the northeast,holding constantthe factors education,experience,race, and gender.We estimatethat workers in the south eam $1.24 less per hour than workers in the northeast. Ta b I e 7 . 5
Wage Equation with Regional Durnrny Variables
Variable
C EDUC BIACK FEMALE BI,ACKxFEMALE SOUTH MIDWEST WEST R2:0.2535
Coefficient
Std.Enor
-2.4557
1.0510 0.0700 0.9034 0.3600 r.2152 0.4794 0.5056 0.5154
1.1025 -t.6077 -2.5009 0.6465 -1.2443 -0.4996 -0.5462 SSE:29101.3
r-Statistic -2.3365 15.7526 -1.7795 -6.9475 0.5320 -2.5953 -0.9880 -1.0597
F 0 . g 5 , 3 , 9 g :22)
l07o level of s: significant regi, 7.3.3
Tpsrn
In the Section hedonic equatir
The regression
+ E:)+ !2EDUC WEST + Ez)+gzEDUCMIDWEST + 61)* pzEDUCSOUTH
I[[l lfi'
Howwould' ioint test of the the context of t null hypothesis This yields an i
Prob 0.0197 0.0000 0.0755 0.0000 0.5949 0.0096 0.3234 0.2895
u h e r e o . 1: P 1 and slope dum neighborhoods estimatingsep rhis idea, whicl Chow test is ar By includinl rariable in an Consider again
WAGE:9
' We might ask rest of the cou regions can be intercept.How dummy for ev dummy variab WAGE:9t +
+
In (7.16)we h addedfive nev SOUTH and tl
5
Using the la
7.3 APPLYING DUMMY
Lto the wage
e
(7.r5)
io would have :ategories are
,+
SOUTH+
linear combiitimatoris not rur computer is error is the ich defines a
179
VARIABLES
How would we test the hypothesisthat there areno regional differences?This would be a joint test of the null hypothesisthat the coefflcients of the regional dummies are all zero. In the context of the CPS data, SSEu : 29101 for the wage equation in Table 7.5. Under the null hypothesis the model in Table 7.5 reducesto that in Table 1 .4 where SSEa: 29308. This yields an F-statisticvalue of 2.3452.The cr : 0.05 critical value (95th percentile)is :2.6138 and for ct : 0.10 (90th percentile)it is Fp.oo3,eszl: 2.0893.At the Fe.ss,z,ssz) l07o level of significance we reject the null hypothesis and conclude that that there are significant regional differences.We cannot make this statementat the 57o level.5 7.3.3
TesrrNc rnr EqunzarENCEoF Two REcnrssroNs
In the Section 7.2.2 we introduced both an intercept and slope dummy variable into the hedonic equation for house price. The result was given in equation (7.12)
pRrCE: Fr * 6D+ g2SeFT+ tGeFTxD) * e Theregressionfunctionsfor the housepricesin the two locationsare (er+ozsQFT D:l
: E(PRICE) tB, +g,zseFTD :o the equation, ins when the rematicallyit : that is most e wage for a neasuresthe northeast;62 those in the .and add the .5. Basedon lhanworkers Lgender.We re northeast.
Prob. 0.0197 0.0000 0.07s5 0.0000 0.s949 0.0096 0.3234 0.2895
wherec.1 : 9r * 6 and a2 : Fz * 1. Figure 7.4b showsthat by introducing both intercept and slope dummy variables we have essentially assumedthat the regressionsin the two neighborhoods are completely different. We could obtain the estimates for (7.12) by estimatingseparateregressionsfor eachofthe neighborhoods.In this sectionwe generalize this idea, which leads to the Chow test, named after econometrician Gregory Chow. The Chow test is an F-test for the equivalenceof two regressions. By including an interceptdummy variable and an interaction variable for each additional variable in an equation we allow all coefficients to differ based on a qualitative factor. Consideragain the wage equationrn (7.14) WAGE: 9r * !2EDUC *6$LACK
-t\zFEMALE +1@LACKxFEMALE) + e
We might ask "Are there differencesbetweenthe wage regressionsfor the south and for the rest of the country?" If there are no differences, then the data from the south and other regions can be pooled into one sample, with no allowance made for differing slope or intercept.How can we test this? We can carry out the test by creating an intercept and slope dummy for every variable in the model, and then jointly testing the significance of the dummy variable coefficients using an F-test. That is, we specify the model -T\zFEMALE *1(BI,ACKxFEMALE) WAGE: Fr * gzEDUC -f\fitACK -r \:SOUTH -r }2(EDUCxS)UTH) + U(BLA,CKXSOUTH) -l 0+(FEMALE x SOUTH) + 05(BLACK x FEMALE x SOUTH) * e
(7.16)
In (7.16) we have twice the number of parametersand variables than in (7 .14). We have addedfive new variables, the SOUTH intercept dummy variable and interactions between SOUTH and the other four variables,and correspondingparameters.Estimating(7.16) is
5
Using the larger CPS data file cps.dat, F : 8.7909, which is significant at the l%olevel.
180
NONLINEARRELATIONSHIPS
Ta h I e 7 . 6
Comparison of Fully Interacted to Separate Models (l) (2) (3) Full sample Non-south South Coefficient Std. Error Coefficient Std. Error CoefficientStd.Enor
Variable
-3.577 c 5 EDUC 1.1658 -0.4312 BLACK -2.7540 FEMALE BLACKxFEMALE 0.06'73 souTH 13023 -0.1917 EDUCxSOUTH -1.7444 BLACKxSOUTH FEMALExSOUTH 0.9119 BLACKxFEMALExSOUTH 0.5428
ssE
1 . 1 5 1 3 -3.5775 0.0824 1.1658 1.3482 -0.4312 0.4257 -2.7540 1.9063 0.0673 2.1147 0.1542 1.8267 0.7960 2.5112
29012.7 1000
N
1.2106 -2.2752 0.0866 0.9741 l.4t'16 -2.1756 0.M76 -1.8421 2.0044 0.6101
22031.3 685
1.5550 0.114_1 t.08Gr 0.5896 1.4329
:egion relative 3stimatesin tt The test ing \)r the sum o 19307.7from
The denominat The 107ocritic 'Jre same in tl .isnificance.6I
REMARK of the lineat : regressions all obse for of the data, . data.In suc Section8.4. be aware th,
6981.4 Jl)
equivalentto estimating (7.14) twice-once for the southernworkers and again for workers in the rest of the country. To see this examine the reqressionfunctions
E(AAGE):{
fu * \zEDUC +6$L-A,CK-I\zFEMALE )- 1(BLACKxFEMALE)
SOUTH:0
(9r + 01)+ (Bz+ 0)EDUC+ (6r+ U)Br"\CK S0UTH: I (r + 05)(BLACKxFEMALE) + (Ez+ L4)FEMALE+
Note that each variable has a separatecoefficient for southernand non-southernworkers. In column (1) of Table 7.6we report the estimates and standard errors for the fullr interactedmodel (7.16),using the full sample.The basemodel (7.i4) is estimaredoncefor workers outside the south [column (2)] and again for southernworkers [column (3)]. Nore that the coefficient estimateson the non-southdatain (2) are identical to thoseusing the full sample in (l). The standarderrors differ becausethe estimatesof the error variance, o:. differ. The coefficient estimatesusing only southern workers are obtained from the full model by adding the dummy variable interaction coefficients 0; to the correspondingnonsouth coefficients. For example, the coefficient estimate for BLACK in column (3) is obtained as (6r + 0r): -O.4f t2- 1.7444: -2.1756. Similarly the coefficienr on FEMALE in (3) is (Sz+0+):-2.1540+0.9119:_1.8421. F u r r h e r m o r en.o t e r h a r the sum of squared residuals for the full model in column (l) is the sum of the .lsE from the two separateregressions SSEyru : SSEnor-,oulp * SSE,orly: 22031.3 + 6981.4 : 29012.7 Using this dummy variable approach we can test for a southern regional difference. We estimate(7.16) and test the joint null hypothesis l l s : 0 1: 0 2 : 0 :
- 0 +: 0 s : 0
against the alternative that at least one 0; I 0. This is the Chow test. If we reject this null hypothesiswe conclude that there is some difference in the wage equation in the southern
1.3.4 CoNr
The earlierext '.rsedin regres '
.3.4a Seas Summermea of charcoalbr Jependentvar .i supermarke :ompetitive b charcoal ligl ;oupons).W} .easonaleffec months than i erampleAUC .\merica, SUi In regression. In theUnited I Jt. and Labot
6 Using the I t The.loint te :nteraction varia :undamentally d
7.3 APPLYING DUMMY
(3) South ficient Std. Error
2752 1.5550 9741 0.1143 17s6 1.0804 8421 0.5896 5101 t.4329
6981.4 JIJ
ain for workers
S O U T H: 0
S O U T H: 1
thern workers. s for the fulll nated once for mn (3)1.Note 3 usingthe full : variance,o2. from the full lponding non:olumn (3) is :oefficient on )re, note that n of the SSE
7 ifference. We
ject this null the southern
VARIABLES
181
region relative to the rest of the country. The test can also be thought of as comparing the estimatesin the non-southand south in columns (2) and (3) in Table 7.6. The testingredientsarethe unrestrictedSSEu:29012.7 from the full model in Table7.6 tor the sum of the SSEs from the two separateregressions),and the restricted SSER: 29307.7 from Table 7.4. The test statistic for the ,/ : 5 hypothesesis F-
(ssER- ssEu)lJ ssEu/(N- 1()
(29307.7 - 29012.7) /s : 2 . 0 1 3 2 29012.11990
Thedenominatordegreesoffreedomcomefromtheunrestrictedmodel,N-K:1000-10. Thel0TocriticalvalueisF.: l.85,andthuswerejectthehypothesisthatthewageequationis the same in the southem region and the remainder of the country - at the 10Volevel of .ignificance.6 The p-valueof-this test is p : g.g144.7
REMARK: The usualF-test of a joint hypothesisrelieson the assumptionsMRI-MR6 of the linear regressionmodel. Of particularrelevancefor testingthe equivalenceof two regressions is assumptionMR3 that the varianceof the errorterm, var(e;): o2, is the same all observations. Ifwe areconsideringpossiblydifferentslopesand interceptsfor parts for of the data,it might also be true that the error variancesare different in the two parts of the data.In sucha casethe usualF-test is not valid. Testingfor equalvariancesis coveredin Section8.4.2, andthe questionof pooling in this caseis coveredin Section8.3.3.For now, be awarethat we are assuminsconstanterror variancesin the calculationsabove.
7.3.4
CoNrnortrNc
FoR TrME
The earlier exampleswe havegiven apply to cross-sectionaldata.Dummy variablesare also usedin regressionsusing time-series data, as the following examples illustrate. 7.3.4a SeasonalDurnmies Summermeansoutdoor cooking on barbequegrills. What effect might this have on the sales of charcoal briquettes,a popular fuel for grilling? To investigatelet us define a model with dependentvariabley, - the number of 20 pound bagsofRoyal Oak charcoalsold in week r at a supermarket.Explanatory variables would include the price of Royal Oak, the price of competitive brands (Kingsford and the store brand), the prices of complementary goods rcharcoal lighter fluid, pork ribs and sausages),and advertising (newspaperads and coupons).While these standarddemand factors are all relevant, we may also find strong :eaSon&leffects. All other things being equal, more charcoal is sold in the warm summer monthsthan in other seasons.Thus we may want to include either monthly dummies(for exampleAUG : 1 if month is August,AUG : 0 otherwise)or seasonaldummies(in North America,SUMMER:1 if month: June,July,orAugust; SUMMER:0 otherwise)into the regression. In additionto theseseasonaleffects,holidaysarespecialoccasionsfor cookouts. Day (July In theUnited StatestheseareMemorial Day (lastMonday in May), Independence l), and Labor Day (first Monday in September).Additional salescan be expectedin the o Using the larger data file cps.dat F :3.975, which is significant at the 17olevel. t The ioint test is significant at the l\Vo level despite the fact none of the individual coefficients on the south interaction variables are significant at even the 207o level, reminding us that joint and individual lests are fundamentally different.
182
NONLINEAR
RELATIONSHIPS
week before theseholidays, meaningthat dummy variablesfor eachshouldbe included into the regression. 7.3.4b Annual Durnrnies In the same spirit as seasonaldummies, annual dummy variables are used to capture year effects not otherwise measuredin a model. The real estatemodel discussedearlier in this chapter provides an example. Real estate data are available continuously, every month. every year. Supposewe have data on house prices for a certain community covering a l0-year period.In addition to housecharacteristics, such as thoseemployedin (7.13),the overall price level is affected by demand factors in the local economy, such as population change,interestrates,unemploymentrate,and incomegrowth.Economistscreating"costof-living" or "house price" indexes for cities must include a component for housing that takes the pure price effect into account.Understandingthe price index is important for tax assessors,who must reassessthe market value of homes in order to compute the annual property tax. It is also important to mortgage bankers and other home lenders, who must reevaluate the value of their portfolio of loans with changing local conditions, and to homeownerstrying to sell their housesas well as potential buyers, as they attempt to agree upon a selling price. The simplest method for capturing theseprice effects is to include annual dummies (for example,D99 : I if year: 1999;D99: 0 otherwise)into the hedonicregressionmodel.An examplecan be found in Exercise7.4. 7.3.4c Regime Effects An economic regime is a setof structuraleconomicconditions that exist for a certain period. The idea is that economic relations may behaveone way during one regime, but they mar behave differently during another. Economic regimes may be associatedwith political regimes (conservativesin poweq liberals in power), unusual economic conditions (oil embargo,recession,hyperinflation), or changesin the legal environment(tax law changes.) An investment tax creditE was enacted in 1962 in an effort to stimulate additional investment.The law was suspendedin 1966, reinstatedin 1970, and eliminated in the Tax Reform Act of 1986. Thus we might create a dummy variable rr| I IIL:<^
Iu
1962- 1965,tglo-
1986
.ife-cyclemodel ',rish to study tht rurposewe take :'\penditure on .bservationsare As an initial
The implication t.
aE(PIZZ pizza ch exPect hr consum the effe< expendit
2. aEeIZ
of $1 in normal 5 called th
Ertimatesof (7
otherw$e
A macroeconomic investment equation might be
INV: : Fr + 61TC'* lzGNPt* $3GNP,-1ae, If thetax creditwassuccessful then6 > 0.
7.4 Interactions Between Continuous Variables When the product of two continuousvariablesis included in a regressionmodel the effect is to alter the relationshipbetweeneachof them and the dependentvariable.We will considera
The signs of tht .ignificant coe Thesearethr .,'qardless of th :n pizzaexPen :hatasa Person : persona$es' *hich the effe< .,ariableis mod :nteractionvar l\COME aret :egressionmo(
o
Intriligator,BodkinandHsiao,EconometricModels,TechniquesandApplications,2ndedition,UpperSaddle River. NJ: Prentice-Hall.1996. o. 53.
PZZ
7.4 INTERACTIONS
be includedinto
TabIe 7 .7 PIZZA
to capture year d earlier in this t, every month. rity covering a d i n ( 7 . 1 3 ) t, h e r as population creating"costrr housing that lportant for tax rutethe annual lers,who must litions, and to :temptto agree dummies(for rionmodel.An
109 0 0 108 220
V A R I . { I ]L E S
183
PizzaExpenditure Data INCOME
15000 30000 I 2000 20000 15000
AGE
;. 20 28 25
life-cyclemodel to illustratethis idea.Supposewe areeconomistsfor GutbustersPizza and rvishto study the effect of income and age on an individual's expenditure onpizza. For that purposewe take arandom sampleof 40 individuals, age 18 and older, andrecord their annual erpenditure on przza (PIZZA), their income (INCOME), and age (AGD.The first five observationsare shown in Table 1.1. The full data set is contained in the file pizza.dat. As an initial model let us consider
PIZZA: gr _FgzAGE_l \zINCOME* e
(7.r7)
Theimplicationsof this specificationareas follows: I
:ertain period. but they may with political cnditions (oil law changes). Lte additional Linatedin the
BETWEEN CONTINUOUS
AEIITZ,A) IAAGE : }z:For a given level of income, the expectedexpenditureon pizza changesby the amount B2 with an additional year of age. What would you expect here? Based on our casual observation of college students,who appear to consumemassivequantities of pizza, we expect the sign of B2 to be negative.With the effects of income removed, we expect that as a person ages their pizza expenditure will fall. AEIIZZA) IAINCOME : B3:For individuals of a given age, an increasein income of $1 increasesexpectedexpenditureson pizza by B3. Since pizza is probably a normal good, we expect the sign of B3 to be positive. The parameter B3 might be called the marginal propensity to spend on pizza.
are Estimatesof (7.17), with r-statisticsin parentheses,
PIZZA: (r)
the effect is Il considera
, UpperSaddle
-7.58AGE+ 0.0024[NCOME 342.88 (3.es) (-3.27)
The signs of the estimatedparametersare as we anticipated.BothAGE andINCOMEhave significant coefficients, basedon their r-satistics. Thesearethe implicationsof the modelin('7.17).However,is itreasonableto expectthat, regardlessofthe ageofthe individual,an increasein incomeby $ I shouldleadto an increase in pizzaexpenditureby B3dollars?Probablynot. It would seemmore reasonableto assume thatasa persongrowsolder,their marginalpropensityto spendon pizzadeclines.That is. as a personages,less ofeach extra dollar is expectedto be spentonpizza. This is a casein which the effect of income dependson the age of the individual. That is, the effect of one variableis modified by another.One way of accountingfor suchinteractionsis to include an interaction variable that is the product of the two variables involved. Since AGE and INCOME are the variables that interact we will add the variable AGExINCOME) to the resressionmodel. The result is
PIZZA: Fr * gzAGE* \zINCOME1- Sa(AGExINCOME) * e
(7.18)
A-PDF Split DEMO : Purchase from www.A-PDF.com to remove the watermark
184
NONLINEAR
RELATIONSHIPS
Just as in the caseswhen we interacteda continuousvariablewith a dummy variable,anc when we interacteda continuousvariablewith itself, when the productof two continuou. variables is included in a model the interpretation of the parametersrequires care. Th.' effects of INCOME andAGE are as follows: | . IE(PIZZA) rcAGE : Fz * \aINCOME : The effect ofAGE now dependson income As a personagestheir pizza expenditureis expectedto fall, and,becauseBais expectec to be negative,the greaterthe incomethe greaterwill be the fall attributableto a chanst in age.
-.5.1
Dun
--etus consi(
I'hat is the , iriable, cre lhat is
2. AEe ITZA) I aINCOME : 9: * I qAGE:The effectof a changein incomeon expectec pizzaexpendirure,which is the marginalpropensityto spendon pizza,now dependso:: AGE. If our logic concerningthe effect of aging is conect then Ba shouldbe negative Then, as AGE increases,the value of the partial derivativedeclines. The estimatedmodel (7.18) including the product (AGExINCOME) is FIn
(r)
:
161.47- 2.}8AGE + 0.\\\\NCOME
(-0.8e)
(2.47)
- 0.00016(AGExINCOME)
(- t.8s)
The estimatedcoefficient of the interaction term is negativeand significant at the cr : 0.0j level usinga one-tailtest.The signsof othercoefficientsremainthe same,butAGE, by itself. no longer appearsto be a significant explanatoryfactor. This suggeststhatA GE affectspizz' expenditurethroughits interactionwith income,that is, on the marginalpropensityto spenc on pizza. Using theseestimateslet us estimatethe marginaleffect of age upon pizza expenditurc. for two individuals;one with $25,000income and the other with $90.000income.
AE(PIZZA\ ' -dAwz : bz I b4 INCOME ;: -2.98 - 0.00016INCOME fot INCOME: $25,000 : {-0.e4 -17.40 for INCOME: $90,000 f Thatis, we expectthatan individualwith $25,000incomewill reducepizzaexpenditures by $6.98per year,while theindividualwith $90,000incomewill reducepizzaexpendituresby $17.40per year,all otherfactorsheldconstant.
7.5 Log-Linear Models In Section4.4 and Appendix 4C we examinedthe log-linear model in some detail. You might havenoticedthat the examplein Section4.4.2wasof a wageequationwith dependenr variable ln(wAGE), but then in (7.5) we changedthe example to have wage level as the dependentvariable.The reasonfor the changewas to simplify expositionof models with polynomial terms,dummy variablesand interactionterms.In this sectionwe explore the interpretationof dummy variables and interaction terms in logJinear models. Some additionaldetail is provided in Appendix 7A.
3ut what abo' lhe answerir -.5.1a
A R i :rst. take th(
.tccall from -rproximatel 0E%. Usin
. rus we esti -:ick andsim ' : r s l a r g ea d
-.i.1b An. .'\.e can ove ::iferenceis
Ir
-.ing the Pro .rd the anti-
:ubtract I fr
wA W
7.5 LOG-LINEAR
nmy variable, and rf two continuous equires care. The
:pendsonincome. useBais expected utableto a change comeon expected L now dependson Louldbe negative.
7.5.1
N,IOT)ELS
185
Durvriuy VenrelrEs
Let us considerthe los-linear model
ln(WAGE): 9r f \2EDUC + \FEMALE
(1.19)
\\'hat is the interpretation of the parameter 6? FEMALE is an intercept dummy variable, creating a parallel shift of the log-linear relationship when FEMALE : l. That is
rur,, y:i::,r, tn(wAG E): { lJ,*,? o* But what about the fact that the dependentvariable isln(WAGE)? Does that have an effect? The answer is yes and there are two solutions.
TNCOME)
atthea:0.05 rtAGE,byitself. CE affectspizza pensityto spend zzaexpenditure income.
:.5.1a A Rough Calculation First, take the difference between ln(WAGE) of females and males - ln(WAGE) ln(WAGE) p61at1s5 rotut : A ln(lIzAGE) : 6 Recall from Appendix A.4.5, equation (A.12), that 100 times the log difference is rpproximately the percentage difference, so that I00L,Ln(WAGE)=%LWAGE: l00E%. Using the data file cps ,small.dar,the estimatedlogJinear model (7.19) is lt (WAGE) : 0.9290 + 0.1026EDUC - 0.2526FEMALE ('") (0.0837) (0.0061) (0.0300) Thuswe estimatethat there is a25.26Vodifferential betweenmale and female wages.This is quick and simple, but as shownin Table A.2 thereis close to al}Vo approximationerror with this large a difference.
a expenditures tizza expendi-
7.5.1b An Exact Calculation W'e can overcome the approximation error by doing a little algebra. The wage difference is
w-,!,?!^ ^* t) : a rn(WAGE)vp7a 6,u, - tn(WAGE), orr-s : n ( lu' \ wAGEya15; 1
using the property of logarithms that ln(x) - ln(y) : ln(xli. and the anti-log is the exponential function, re detail. You ith dependent e level as the 'models with e explore the Lodels.Some
These are natural logarithms,
WAGEptuercs _ -6 -' WAGEyaL65 Subtract I from each side (in a tricky way) to obtain WAGErnuatos_WAGEuercs _WAGEreueus - WAGEuercs: p6 . _ WAGEyaTBs WAGElatlBs WAGElaayBg
185
NONLINEAR
RELATIONSHIPS
The percentagedifferencebetweenwagesof femalesandmalesis 100(eE- I)%.From this. we estimate the wage differential between males and females to be 1 0 0 ( e 6- r ) % :
t 0 o ( e - 0 2 5 2- u D % :
dollars, G. econometl which coI regression
-22.32%
sl (s
The approximatestandarderror for this estimateis 2.337o,which is a calculation that may be provided by your software, making this exact calculation more than one standard eno: different from the approximate value of -25.267o. 7.5-2
INrEnecrroN
AND Queonanrc Tpnus
In the simple log-linearmodel lnlv) : Fr + B2x,wegive an interpretationto B2by saying that given a l-unit changein x the approximatepercentagechangein y is 100B2%.Se.' Appendix A, (A.13) and the surroundingdiscussion.How do we extendthis resultto higher order models? Consider the wage equation ln(wAGE) : 9r * \2EDUC -t \zEXPER -r ^y(EDUCxEXPER)
(7.20
What is the effect of another year of experience,holding education constant?Roughll. ALn(WAGE\l
LiT*^
lrrurn*o:
B: * IEDUC
In AppendixA, equation(A.12),we showthat 100timesthe log differenceis approximatell the percentagedifference.Using this result, the approximatepercentagechangein wa-gc given a l-year increase in experience is 100(9: + IEDUC)%. Using the data filc cps_small.dal we estimate (7.20) to obtain fiWAG,: ('")
(a) Interpr (b) How t ries th otherv (c) How v same I
0.1528+ o]34tEDUC + 0.o249EXpER- 0.000962( ED|CxEXpER (0.r'722) (0.0127) (0.0071) (0.000s4)
t'
In Septem data for thr affecting t emergenc were229c moons (a t Memorial . will adjus change the Using tl following ti HOLIDAY 0 otherwis there is a fr.
Emergen For a person with 16 years of education,we estimatethat an additional year of experience leadstoanincreaseinwagesof approximately 100(0.0249- O.000962x16)%:0.95189,. with standard error 0.216. What if there is a quadratic term on the right-hand side, as in ln(wAGE):
^y(EDUCxEX?ER) 9r * 7\EDIJC -t gzEXpER-r la,EXpER2+
Then, using a little calculus, we find that a l-year increasein experienceleads to an approximate percentagewage change of %LWAGE - 100(93+ 294EXPER+ IEDUC)%
7.6 Exercises Answers to exercisesmarked * appear in Appendix D at the end of the book. 7.6.7 1.1
Prom"Eus An Economics department at a large state university keeps track of its majors' startingsalaries.Does taking econometricseffect starting salary?Let SAI: salaryin
Variable C I
HOLIDAY FRIDAY SATURDA FULLMO( NEWMOO
Rz : 0.1':
( a ) Interp
calls? (b) The r MOO. ( c ) Test tl alterni
7.6 EXERCISES
dollars, GPA : grade point averageon a 4.0 scale,METRICS : I if student took otherwise. Using the data file metrics.dat, econometrics, and METRICS :0 which contains information on 50 recent graduates, we obtain the estimated regression
)%. Fromthis.
sn : ('")
onthatmaybe standard error
.')*
(7.20)
rt?Roughly,
pproximately ange in wage the data file
CxEXPER)
24200+ r643GPA+\033METRrCS R2:0.74 (1078) (3s2) (4s6)
(a) Interpret the estimated equation. (b) How would you modify the equation to see if women had lower starting salaries than men? (Hint: Define a dummy variable FEMALE: 1, if female; 0 otherwise.) (c) How would you modify the equationto seeif the value of econometricswas the same for men and women?
r B2by saying 10092%. See esultto higher !)
187
In September1998, a local TV station contactedan econometricianto analyze some data for them. They were going to do a Halloween story on the legend of full moons affecting behavior in strange ways. They collected data from a local hospital on emergencyroom casesfor the period from January 1, 1998 until mid-August. There were 229 observations.During this time there were eight full moons and sevennew moons (a related myth concernsnew moons) and three holidays (New Year's Day, Memorial Day, andEaster).If there is a full-moon effect, thenhospital administrators will adjust numbers of emergency room doctors and nurses,and local police may change the number of officers on duty. Using the data in the file fullmaon.dat we obtain the regression results in the following table: Zis a time trend (f : | ,2,3, . . . ,229) andtherestaredummy variables. FNDAY:1if thedayis aFriday; HOUDAY:1if thedayis aholiday;0otherwise. 0 otherwise.SATURDAY:1 if the day is a Saturday;0 otherwise.FUI,LAIOON : I if there is a full moon; 0 otherwise.NEWMOON : 1 if there is a new moon; 0 otherwise. Emergency Room Gases Regression-Model 1
rf experience :0.9518%.
ixPER) leadsto an
Variable
C T HOLIDAY FRIDAY SATURDAY FULLMOON NEWMOON R 2: 0 . 1 7 3 6
its majors' L: salaryin
Coefficient
93.6958 0.0338 13.8629 6.9098 10.5894 2.4545 6.4059
Std. Error
r.5592 0.0111 6.44s2 2.1tr3 2.1184 3.9809 4.2569
t-Statistic
60.0938 3.0580 2.1509 3.2727 4.998'l 0.6166 l.5048
hob.
0.0000 0.0025 0.0326 0.0012 0.0000 0.5382 0.1338
SSE:27108.82
(a) Interpret these regressionresults. When should emergencyrooms expect more calls? (b) The model was reestimated omitting the variables FULLMOON and NEWMOON, as shown below. Comment on any changesyou observe. (c) Test the joint significanceof FULLMOON and NEWMOON. State the null and alternativehypothesesandindicatethe teststatisticyou use.What do you conclude?
188
NONLINEAR
RELATIONSHIPS
Emergency Room Cases Regression-Model 2 Variable
Coefficient
C
HOLIDAY FRIDAY SATURDAY
94.0215 0.0338 13.6168 6.849r 10.3421
R2 : 0.1640
SSE: 27424.19
t
7.3
Std. Error
1.5458 0.0111 6.4511 2.1r37 2.1r53
-l
t-Statistic
60.8219 3.0568 2 . 11 0 8 3.2404 4.8891
Prob 0.0000.00'.< 0.035! 0.001: 0.0frr:,
Henry Saffer and Frank chaloupka ("The Demand for Illicit Drugs,,, Economit Inquiry,37(3), 1999,40r4rr) estimatedemandequationsfor alcohol,marijuana cocaine, and heroin using a sample of size N : 44,9g9.The estimatedequati,on fu alcohol use after omitting a few control variables is Demand for lllicit Drugs Variable
Coefficient
C ALCOHOL PRICE INCOME GENDER MANTAL STATUS AGE t2-20 AGE 21-30 BLACK HISPANIC
4.099 -0.045
17.98 5.93
0.000057 r.637 -0.807 -1.531
1'7 A<
0.03s -0.580 -0.564
lr-statisticl
29.23 12.13 17.97 0.51 8.84 6.03
The variable definitions (sample means in parentheses)are as follows: The dependentvariable is the number of days alcohol was used in the past 3l dar.s (3.4e) PRICE-pice of a liter of pure alcohol in r9g3 dollars (24.7g) lLCoHoL INCOME-rotal personal income in l9g3 dollars (12,425) GENDER-a binary variable : I if mate (0.479) MARITAL STATUS-a binary variable : 1 if married (0.569) AGE l2-20-a binary variable : l if individual is 12-20 years of age (0.155) AGE 2r-30-a binary variable : I if individual is 21-30 yearsof aie (0.r9i) BLACK-a binary variable : I if individual is black (0.116) HISPANIC-a binary variable : 1 if individual is Hispanic (0.07g) (a) Interpret the coefficient of alcohol price. (b) Compute the price elasticity at the means of the variables. (c) compute the price elasticity at the means of alcohol price and income, for a married black male, age 2l-30. (d) Interpret the coefficient of income. If we measuredincome in $1000 units, whar would the estimated coefficient be? (e) Interpret the coefficients of the dummy variables, and their significance.
In the file prices, sq Stockton construct we estim the size r variables
PRICE.
The resul Stockto Variable
C SQFT AGE D92 D93 D94 D95 D96
(a) Discu ation, (b) Discu (c) What
An agricr relationsh acre) and NITRO: PHOS: A total N, quadratic
YIE
(a) Find a
of yie (b) Find a of yie
7.6 EXERCISES
I Prob 0.000t-t 0.002: 0.0359 0.001l 0.000(,
189
In the file stockton.datwe havedatafrom January1991to December1996on house prices, squarefootage,and other characteristicsof 4682 housesthat were sold in Stockton,CA. One of thekey problemsregardinghousingpricesin a regionconcerns constructionof "house price indexes,"as discussedin Section1.3.4b.To illustrate. we estimatea regressionmodel for houseprice, including as explanatoryvariables the size of the house (SQFT), the age of the house (AGE), and annual dummr' variables, omitting the dummy variable for the year 1991. ?RICE : 9r * \|SQFT * \tAGE -t 6rD92 -r 62D93 * 6:D94 * 6+D95 -l6sD96 * e The results are as follows:
," Economic l, marijuana. equation for
Stockton House Price Index Model Variable
C SQFT AGE D92 D93 D94 D95 D96
Coefficient
21456.2000 72.7878 -r79.4623 -4392.8460 -10435.4700 - l 3 1 7 3 .150 0 - 19040.8300 -23663.5100
Std. Error
t-Statistic
1839.0400 1.0001 t7.0112 r270.9300 1231.8000 1211.4770 1232.8080 1194.9280
r1.66',71 72.7-7'73 - 10.5496 -3.4564 -8.4',7 t7 - 10.8739 -15.445r - 19.8033
Prob
0.0000 0.0000 0.0000 0.0006 0.0000 0.0000 0.0000 0.0000
(a) Discussthe estimatedcoefficients on SQFZ and AGE, including their interpretation, signs,and statisticalsignificance. (b) Discussthe estimatedcoefficientson the dummy variables. (c) What would have happenedif we had included a dummy variablefor 1991?
rast31 days +.78)
(0.rss) (0.197)
An agricultural economist carries out an experiment to study the production relationshipbetweenthe dependentvariable YIELD: peanut yield (poundsper acre) and the productioninputs NITRO : amount of nitrogen applied (hundredsof pounds per acre) PHOS : amount of phosphorusfertilizer (hundredsof poundsper acre) A total N :27 observationswere obtainedusing differenttest fields.The estimated quadratic model, with an interaction term, is
fEB
- r.I44NITRO2 :1.385 + 8.01lNlrRo+ 4.800PHOS (0.e41) (r.264)(0.e41) (0.220) - O.S'IU|TRO - 0.T78PHO9 x PHOS (0.220) (0.r5s)
ome, for a units,what tnce.
(a) Find and commenton the estimatedfunctionsdescribingthe marginalresponse of yield to nitrogen, when PIIOS : 1. PHOS : 2, and PHOS : 3. (b) Find and comment on the estimatedfunctions describing the marginal response of yield to phosphorus,when NITRO : I, NITRO :2, and NITRO : 3.
190
NONLINEAR
RELATIONSHIPS
(c) Test the hypothesisthat the marginal responseof yield to nitrogen is zero, when (1) PHOS: I and NITRO : I (11)PHOS: 1 and NITRO :2 (iii) PHOS: I and NITRO :3 Note: The following information may be useful:
(c) Are t withr 7.5.| (d) Test, (i) ,
(iD , var(bz* Zbq-f ba) : 0.233 var(bz -l 4ba,* be) : 0.040 var(bz-t 6b+* ba) :0.233 (d)O(This partrequiresthe useof calculus.)Forthe function estimated,what levelsof nitrogen and phosphorus give maximum yield? Are these levels the optimai fertilizer applications for the peanut producer?
7.6.2
CoMpursn Exsncrsns
1.6 In (7.13) we specifieda hedonic model for houseprice. The dependentvariable u a-. the price of the house,in dollars. Real estateeconomistshave found that for manl data sets a more appropriate model has the dependentvariable In(PRICE). (a) Using the data in the file utown.dat, estimatethe model (7.13) using Ln(PRICE, as the dependentvariable. (b) Discuss the estimatedcoefficients on SQFT andAGE. Refer to Section 4.4 for help with interpreting the coefficients in this log-linear functional form. (c) Computethe percentagechangein price due to the presenceof a pool. Use both the rough approximationin Section7.5.la and the exactcalculationin Section7.5.1b. (d) Compute the percentagechangein price due to the presenceof a fireplace. Use both the rough approximation in Section 7.5.1a and the exact calculation in Section7.5.1b. (e) Compute the percentagechange in price of a 2500 squarefoot home near the university relative to the samehousein anotherlocation using the methodolo-er in Section7.5.1b.
7.7 Data on the weekly salesof a major brand of cannedtuna by a supennarketchain in a largemidwesternU.S.city during a mid-1990'scalendaryeararecontainedin the file tuna.dat. There are 52 observationson the variables SAL| : unit sales ofbrand no. I canned tuna APRI : price per can ofbrand no. 1 canned tuna APR2, APR3 - price per can ofbrands nos. 2 and 3 of cannedtuna DISP : a dummy variable that takes the value I if there is a store display for brand no. 1 during the week but no newspaperad; 0 otherwise DISPAD - a dummy variable that takes the value I if there is a store display and a newspaperad during the week; 0 otherwise (a) Estimate, by least squares,the log-linear model ln(SAL1) : Fr * P1APRI + %APR2 + P4APR3* gsDISPa $6DISPAD-t e (b) Discussand interpretthe estimatesof 92, 9:, and Ba.
(iii) j (iv) ,
(e) Disct exect
F.G. Mixr in the Mu investigat to be pric ferencesi lessthan 1 TV-progr or tape,ar this they <
PRICE: AGE: a OLD: a NET:a sites).
(a) Estin mate (b) Estin mate
In Sectio age of th (a) Use t expe rem{t to cft (b) Estin paral (c) Estin paral incor (d) In (7 test t (e) Cons to sP estin (0 Modi incor soby
7.6 EXERCISES
191
(c) Are the signs and relative magnitudesof the estimatesof B5 and B6 consistent with economiclogic? Interprettheseestimatesusingthe approaches in Sections 7 . 5 . 1 aa n d 7 . 5 . l b . (d) Test, at the cr : 0.05 level of significance,each of the following hypotheses:
en is zero,when
(i) 110:Fs : (ii) illo:p0: (iii) I1o:9s: < (iv) F1s:90
0, 0, 0, Fo - 0 ; 9:,
ft:g5lo H1:g6lo I11:B5orFol-0 11r:9e> B:
(e) Discuss the relevanceof the hypothesistests in (d) for the supermarketchain's executives. d, what levelsol els the optimai
:nt variable wa: d that for manr )RICD.
7.8 F.G.Mixon and R.W. Ressler(2000)in "A Note on Elasticity and PriceDispersions in the Music RecordingIndustry" (Reviewof Industrial Organization,17,465470) investigatethe pricing of compact disks.They note that it is common for new releases to be priced lower than older CDs. Their explanationof this pricing schemeis differencesin the price elasticity of demand.The number of substitutesfor old CDs is lessthanfor new.For example,new musiccanbe heardon VHl, MTY in movie and TV-program soundtracks,on radio, and live. That old favorite you had on vinyl record or tape,andnow want on CD, hasmuch morelimited competition.To empiricallytest this they obtaindataon 118CDs. The dataarein the file music.dat The variablesare PRICE: retail price of the CD (US $) AGE : age of the recording (l999-copyright date) OLD : a dummy variable : I if the recording is not a new release NET : a dummy variable : 1 for Internet prices (Tower Records and Amazon web sites).
sing ln(PRlCEr Section4.4 for nal form. ,ol.Useboth the Section7.5.1b. t fireplace. Use calculation in home near the e methodolog\ arket chain in a ainedin the file
I
iplay for brand 'display and a
SPAD -l e
-.9*
(a.) Estimate the model PRICE:
Fr * gzAGE + ENEr * e. Inrerpret the esri-
mated model. (b) Estimate the model PRICE: mated model.
Fr * 7zOLD +6NEZ*e.
Interpret the esti-
In Section7 .4 the effect of income onpizzaexpenditure was permitted to vary by the age of the individual. (a) Use the data in the file pizza.dat to estimatethe regressionmodel in which pizza expendituredependsonly on income.Before estimatingthis model, andfor the remainderofthis exercise,useincomemeasuredin $1000s.(It might be simplest to create a new variable INC = INCOMEllO00.) (b) Estimate the model in ('7.17). Comment on the signs and significance of the parameterestimatesand on the effect of scaling the income variable. (c) Estimate the model in (7.18). Comment on the signs and significanceof the parameter estimates.Is there a significant interaction effect between age and income?What is the effect of scaling income? (d) In (7.18) test the hypothesisthat age doesnot affectpizza expenditure.That is, test thejoint null hypothesisHo:92 :0, 9+ : 0. What do you conclude? (e) Construct point estimatesand 957ointewal estimatesof the marginal propensity to spendon pizza for individualsof age 20, 30,40, and 50. Commenton these estimates. (f) Modify (7.18) to permit a "life-cycle" effect in which the marginal effect of income onpizza expenditureincreaseswith age,up to a point, and thenfalls. Do soby addingtheterm(AGE2xlNC) to themodel.What signdo you anticipateon
192
NONLINEAR
RELATiONSHIPS
(iii) Whr (iv) Atv year (v) Wh,
this term? Estimate the model and test the significanceof the coefficient for thi. variable. (g) Checkthemodelusedin part (0 for collinearity(SeeSection6.7.3).Add theterrr. (AGE3xlNC) to the model in (f) and check the resulting model for collinearitl 7 .10 The file pizza.darincludes additional information aboutthe 40 individuals usedin the pizza expenditureexample in Section 7 .4. The dummy variable FEMALE : 1 for females;0 otherwise.The variables HS,COLLEGE, andGRAD aredummy variable' indicating level of educational attainment. FIS : I for individuals whose highe:t degreeis a high schooldiploma. COLLEGE : 1 for individuals whosehighestdegree is a college diploma. GRAD : 1 if individuals have a graduate degree. If llS. COLLEGE, and GRAD are all 0, the individual did not complete high school. (a) Begin with the model in (7.17).Include gender(FEMALD as an explanatorl variable and estimate the resulting model. What is the effect of including thi. dummy variable? Is gender a relevant explanatory variable? (b) Begin with the model in (7.11). Include the dummy variables HS, COLLE GE, and GRAD asexplanatoryvariablesand estimatethe resulting model. What is the effect of including these dummy variables? Is level of educationar attainment a significant explanatory variable? (c) Consider(7 .17). Testthe hypothesisthat separateregressionequationsfor male. and females are identical, against the alternative that they are not. Use the 5c? level of significanceand discussthe consequences of your findings. 7.11 Use the data in pizza.datto do the following: (a) Estimatethe model (7.18) and compareyour resultsto thosein Section7.4. (b) Calculate the marginal effect dE(PIZZA)IAINCOME for an individual of averageage and income and test the statistical significance of the estimate. (c) Calculate a 95%ointewal estimate for the marginal effect in (b). (d) CalculatethemarginaleffectdE(PIZZA)ldAGEtoranindividualofaverageage and income and test the statisticalsignificanceof the estimate. (e) Calculate a 95Vointerval estimate for the marginal effect in (d). (f) (This part requires the completion of Exercise 7.10.) Write a repon (250 + words) to the president of Gutbusterssummarizing your findings from Exercises7.10 and7 .lL 7.12r' Lion Foresthasbeena very successfulgolf professional.However,at age45 his game is not quite what it usedto be. He startedthe pro-tour when he was only 20 and he ha: beenlooking back examining how his scoreshavechangedas he got older. In the file golf.dat, the first column contains his final score (Actual score - par) for 150 tournaments.The secondcolumn contains his age (in units of 10 years). There are scoresfor six major tournamentsin eachyear for the last 25 years.Denoting his score by SCORE andhis age by AGE, estimatethe following model and obtain the withinsamplepredictions. scoRE:
gr * 7\AGE * gzAGE2* 9+AGE3+ e
(a) Test the null hypothesis that a quadratic function is adequateagainst the cubic function as an alternative.What are the characteristicsof the cubic equationthat might make it appropriate? (b) Use the within-samplepredictionsto answerthe following questions: (i) At what age was Lion at the peak of his career? (ii) When was Lion's game improving at an increasingrate?
(c) When he
13 Use the data tn(.WAGE): 9r -
+l
(a) Discusst its sign z (b)o(large d do you c
I 1o (large data s (a) Reestim Estimate basedor (b) Estimatt Test the significa (c) Estimate to those (d) Estimatt betwee (e) Estimatt wagesb -
15* ProfessorR models that at http:llfait paper entitlt model is tt Republican election)is relating to tl whether the electionyea VOTE: pe
The exP
PARW : I there is PERSON:
DURATIO incumb partyha and so
7.6 EXERCISES
efficient for this
(iii) When was Lion's game improving at a decreasingrate? (iv) At what age did Lion start to play worse than he had played when he was 20 years old? (v) When could he no longer score less than par (on average)?
3). Add theterm for collinearity. luals usedin the TMALE: I for ummy variables r whose highesr r highestdegree degree. If F1S. igh school. an explanatorJ f including this s HS, COLLErg model.What of educational ttionsfor males ot. Use the 57r ings. LSection7.4. individual of :he estimate. ofaverageage
r.it" u ,"pon 'findings from
ge 45 his game y 20 and he has rlder.In the file - par) for 150 ars). There are Lotinghis score ain the within-
193
(c) When he is aged70, will he be able to break 100?Assume par is72. l3
Use the data in the file cps2.dat to estimate the following wage equation:
.nt,WAGE) : 9r * PyEDUC -t !1EXPER -t F+FEMALE * \sBLACK -t \aMARRIED -T\UNION -r \ySOUTH t \gFULLTIME a $rcMETRO * e (a) Discussthe resultsof the estimation.Interpret each coefficient, and comment on its sign and significance. Are things as you would expect? (b)i(large data set) Use the data cps.dat to reestimatethe equation. What changes do you observe? , Jr (large data set) Use the data file cps.dat for the following. (a) Reestimatethe model reportedin Table 7.1. What changesdo you observe? Estimatethe marginal effect of experienceon wage and comparethe result to that basedon Table 7.1. (b) Estimatethe model given in (7.l4). Comparethe resultsto those inTable L4. Test the hypothesis that the interaction between BLACK and FEMALE is significant. (c) Estimate the model reported in Table 7.5. Discussthe results and comparethem to thosereportedin Table 7.5. Testthe hypothesisthat there is no regional effect. (d) Estimate the model in (7.16) and test the hypothesisthat there is no difference between the wage equations for southern and non-southernworkers. (e) Estimate the log-linear model in (7.19). Estimate the percentagedifference in wages between males and females. l5'r ProfessorRay C. Fair's voting model was introducedin Exercise2. 14. He builds models that explain and predict the U.S. presidentialelections.See his website at http.'I lfairmodel.econ.yale.edu/vote2008/index2.htm,and see in particular his paper entitled "A Vote Equation for the 2004 Election." The basic premise of the model is that the incumbent party's share of the two-party (Democratic and Republican) popular vote (incumbent means the party in power at the time of the election) is affected by a number of factors relating to the economy, and variables relating to the politics, such as how long the incumbent party has been in power, and whether the Presidentis running for reelection. Fair's data, 31 observationsfor the electionyearsfrom 1880to 2000, are inthe filefaixdat.The dependentvariableis VOTE : percentageshare of the popular vote won by the incumbent party. The explanatory variables include
dnst the cubic o equationthat tions:
PARTY : 1 if there is a Democratic incumbent at the time of the election and - 1 if there is a Republican incumbent. PERSON: 1 if the incumbentis running for election and 0 otherwise. DURATION : 0 if the incumbent party has been in power for one term, I if the incumbentparty hasbeenin powerfor two consecutiveterms,1.25if the incumbent party hasbeenin powerfor threeconsecutiveterms,1.50for four consecutiveterrns, and so on.
NONLINEAR
APPENDIX 7
RELATIONSHIPS
WAR : 1 for the elections of 1920, 1944, and 1948 and 0 otherwise. GROWTH : growth rate of real per capita GDP in the first three quarters of the election year (annual rate). INFIATION : absolutevalue of the growth rate of the GDP deflator in the first 15 quarters of the administration (annual rate) except for 1920,1944, and 1948, where the values are zero. GOODNEWS : number of quartersin the first 15 quartersof the administration in which the growthrate of real percapita GDPis greaterthan3.2Voat an annualrate except for 1920,1944, and 1948, where the values are zero. (a) Consider the regressionmodel VOTE : h * 9zGROWTH + 93INFL^\TION -f FTGOODNEWS *Q1WAR* * psPERSON* FaDURATION*\PARW
e
Discuss the anticipated effects of the dummy variables PERSON and WAR. (b) The binary variable PARII is somewhatdifferent from the dummy variableswe have considered.Write outtheregressionfunction E(VOZE) forthe two valuesof PARTY.Discuss the effects of this specification. (c) Use the data for the period 1916-2000to estimate the proposedmodel. Discuss the estimation results.Are the signs as expected?Are the estimatesstatistically significant? How well does the model fit the data? (d) Predict the outcome of the 2004 election using PARZI : - l, DU RATION : 0. WAR :0, and PERSON : 1. For the other variables specify GOODNEWS : 1. GROWTH :2.0, and INFLATION : I.7. (e) Construct a 95Voprediction interval for the outcome of the 2004 election. (f) Using datavaluesof your choice (you must explain them), predict the outcomeof the 2008 election. 7.16 The data file br2.dat containsdata on 1080housesalesin Baton Rouge,Louisiana. during July and August 2005. The variables arePRICE ($), SQFZ(total squarefeet). BEDROOMS (number),BAZFIS(number),AGE (years),OWNER(:1 if occupiedby owner; 0 if vacant or rented), POOL (:1 if present), TRADITIONAL (:1 if traditional style; 0 if other style), FIREPLACE (:I if present), and WATERFRONT (:1 if on waterfront). (a) Compute the data summary statistics and comment. In particular construct a histogram of PRICE. What do you observe? (b) Estimate a regressionmodel explaining ln(PR1CE/1000) as a function of the remaining variables. Divide the variable SQFT by 100 prior to estimation. Comment on how well the model fits the data. Discuss the signs and statistical significanceof the estimatedcoefficients.Are the signswhat you expect?Give an exact interpretation of the coefficient of WATERFRONT. (c) Createa variable that is the product of WATERFRONTandTRADITIONAZ. Add this variable to the model and reestimate. What is the effect of adding this variable?Interpret the coefficient ofthis interaction variable, and discussits sign and statistical significance. (d) It is arguable that the traditional style homes may have a different regression function from the diversesetofnontraditional styles.Carry out a Chow test ofthe equivalenceofthe regressionmodels for traditional versusnontraditional styles. What do you conclude?
(e) Usin hous atthe the u -
17* Recentd stockton2 BEDS(nr dummy l (a) Exan varia (b) Estin SQF the s (c) How (d) Omil that i (not (e) Carq
Appendix Interpreta
\bu may haven, rodel we omitt To do so we mal r e noted that fc .:len the expect
E
Srarting from t :rleraction teflI LetDbea 3:+92,t*6D
I: ne let E(y) a --omputetheir 1
%^n67
APPENDIX 7A
'wise. e quarters of the lator in the first 15 t, 1944, and 1948. : administration in Voatanannualrale
IEWS :WAR+ e SON and,WAR. mmy variableswe )rthe two valuesof :d model. Discuss matesstatisticallr DURATION:0. 1OODNEWS:1. 1004election. ict the outcomeof louge, Louisiana. .totalsquarefeet). :1 if occupiedb) ITIONAL (:l if dWATERFRONT .cular constructa a function of the or to estimation. ;ns and statistical u expect?Give an IDITIONAL. Add ct of adding this rd discussits sign Terentregression a Chow test of the traditionalstyles.
DETAILS OF LOG_LINEAR
MODEL
195
INTERPRETATION
(e) Using the equationestimatedin part (d), predict the value of a traditionalst1le housewith 2500 squarefeet of area,that is 20 yearsold, which is orvneroccupied at thetime of sale,with a fireplace,3 bedroomsand2 baths,but no pool andnot on the waterfront. l7* Recentdataon 880 housesalesfrom Stockton.Califomia. arecontainedin the datafile stockton2.dat The variables arePRICE ($), SOFZ (total squarefeet of living area), BEDS (numberbedrooms),BAIFIS (number),AGE (years),STOR/ES(number),and a dummy variable VACANTthat equalsone if the housewas vacantat the time of sale. (a) Examine the histogram of the variable PRICE. What do you observe?Createthe variable Ln(PRICE) and examine its histogram. Comment on the difference. (b) Estimate a regressionof ln(PRICEl1000) on the remaining variables. Divide SQFT by 100 prior to estimation.Discussthe estimationresults.Commenton the signs and significance of all variables except VACANT, (c) How does vacancy at the time of sale affect house price? (d) Omitting UACANT,estimate the regressionmodel separatelyfor those houses that are vacant at the time of sale and those that are occupied at the time of sale (not vacant). Compare the estimation results. (e) Cany out a Chow test of the equivalenceof the regressionmodels in (d).
Appendix 7A Details of Log-Linear Model lnterpretation \bu may havenoticed that in Section7.5 while discussingthe interpretationof the log-linear rodel we omitted the error term, and we did not discussthe regressionftnction E(WAGE). To do so we make useof the propertiesof the log-normal distribution in Appendix 4C. There ; e noted that for the log-linear model ln(y) : 9r * $2x I e, if the error term e - N(0,o2) :ren the expectedvalue ofy is : exp(9r + Fzx) xexplo2f2) "'12) Srarting from this equation we can explore the interpretation of dummy variables and rrteractionterms. Let D be a dummy variable. Adding this to our logJinear model we have ln(y) : E(y) : exp(Pr * Fzx *
3: *Fzr*6D+eand ety) : exP(9r * B2x-F 6o)xexp(o2 /2) Ii rvelet E(y1) and E(y6)denotethe caseswhen D : I andD: :ompute their percentagedifference as
0, respectively,then we can
' w - - \(i r t : -1ss " * LfE(-l:10' l 7' %a'E " E(vo) I
-
exp(9r+ Fz-r+ E)xexp(o2/2) exp(Pr+ 9zx)xexp(o2/2)l : l00l " e * p ( g '+ W)xe*p(o14
:t*[
- exp(9r1- 9zx exp(9r* B2"r)exp(6) ,]
exp(pr-| 9zr)
)"
- 1l% %: 100lexp(6)
196
NONLINEAR
RELATIONSHIPS
The interpretationof dummyvariablesin log-linearmodelscarriesoverto the regression function.The percentage differencein the expectedvalueof y is 100[exp(6)- 1]%. Insteadof a dummyvariable,let usintroduceanothervariablez andits interactionwith,r. ln(y) : 9r * 9z,r* 9zz+ 1@z)* e. Then E(y) : exp(Pr* Fzx* 9zz+ 1@z))xexp(ozlz)
Chaptt
Heterr
The derivativeof this expression, with respectto z, holdingx constant,is 0E(v) exp(Fr* lzx r F:z + 1(xz))xexp(o2lz)x(B:+ 1x)) 3:: Dividing both sidesby E0) andmultiplyingby 100gives
Learning Ol
Basedon the mate
: roo(F: ts6foE(YX!!l^j +y)% Lozj
Interpreting "0" as "A", then the left-hand-sidenumeratoris the percentagechangein the expectedvalue of y resulting from a 1-unit changein z. The interpretationswe developedin Section 7.5 hold in general; the only refinement is that here we have calculated the percentagechange in the expectedvalue of y.
l . Explain the exhibit hete f
-1.
+.
Describe at squaresesti
Compute ht
Compute gt the variance function of partitioned
-). Describe h< 6 . Explain hor 1 . Specify a t
Breusch-Pe
8 . Test for ht subsample is hypothes
Keywords
3reusch-Pagante :eneralizedleast s Goldfeld-Quandt I :eteroskedasticpa :eteroskedasticit
8.1 The Na
.r Chapter2 the re .nd household inc
)r to theregression )xp(6)- 1l%. iinteraction with"r.
Chapter
Heteroskedasticity ,is - ^yx))
Learning Objectives Basedon the material in this chapteryou should be able to
rtagechangein the rs we developedin Lvecalculated the
-)
Explain the meaningof heteroskedasticity and give examplesof data setslikely' to exhibit heteroskedasticity. Describe and compare the propertiesof the least squaresand generalizedleast squaresestimatorswhen heteroskedasticity exists.
-3. Computeheteroskedasticity-consistent standarderrors for least squares.
-+. Computegeneralizedleastsquaresestimatesfor heteroskedasticmodelswhere (a) the varianceis known exceptfor the proportionalityconstanto'. (b) the varianceis a function of explanatory variables and unknown parameters,and (c) the sample is partitionedinto two subsampleswith different variances.
) . Describe how to transform a model to eliminate heteroskedasticity. 6 . Explain how and why plots of leastsquaresresidualscan revealheteroskedasticity. l . Specify a variance function and use it to test for heteroskedasticity with (a) a Breusch-Pagantest, and (b) a White test. Test for heteroskedasticity using a Goldfeldt-Quandt test applied to (a) two subsampleswith potentiallydifferentvariancesand (b) a model wherethe vanance is hypothesizedto dependon an explanatoryvariable.
Keywords Breusch-Pagantest ;eneralizedleast squares Goldfeld-Quandttest reteroskedasticpartition reteroskedasticity
heteroskedasticity-consistent standarderrors homoskedasticity Lagrangemultipliertest mean function
residual plot transformed model variance function weightedleast squares White test
8.1 The Nature of Heteroskedasticity In Chapter2 the relationshipbetweenaverageor meanhouseholdexpenditureon tbod Elr r rnd householdincome x was describedby the linear function
E(v):Fr*Pzx
( 8 . 1)
197
198
HETEROSKEDASTICITY
The unknown parametersB1 and B2 conve! information about this expenditure function. The responseparameterB2describeshow meanhouseholdfood expenditurechangeswhen householdincome increasesby one unit. The intercept parameterB1 measuresexpenditure on food for a zero income level. Knowledge of theseparametersaidsplanning by institutions suchas governmentagenciesor food retail chains.To estimateBq and 92 we considereda sampleof N : 40 householdsindexedby i: l, 2,. . . , 40, with the pair (y;, ,r;) denoting expenditure on food and income for the ith household. To recognize that not all householdswith a particular income will have the same food expenditure,and in line with our generalspecificationof the regressionmodel, we let e; be the difference betweenexpenditureon food by the ith householdyrand meanexpenditureon food for all householdswith income x;. That is. ei : yi - E(yi) : li - 9r - pzxi
Probability density function
f(r)
(8.2r
:ccne 8,7 He Thus, the model used to describe expenditure on food for the ith household is written a: Yi: Fr *9zxi*ei
(8.3,
We canview E(y,) : 9r * B2x;asthat part of food expenditureexplainedby income.r;and e; as that part of food expenditure explained by other factors. We begin this chapterby asking whether the mean function E(y) : 9r -t \zx is better ar explaining expenditure on food for low-income householdsthan it is for high-income households.If you were to guessfood expenditurefor a low-income household and food expenditurefor a high-incomehousehold,which guessdo you think would be easier?Lorr income householdsdo not have the option of extravagantfood tastes.Comparatively,ther havefew choicesand are almost forced to spenda particular portion of their income on food. High-income householdson the other handcould havesimple food tastesor extravagantfood tastes.They might dine on caviar or spaghetti,while their low-income counterpartshave to take the spaghetti.Thus, income is relatively less important as an explanatoryvariable for food expenditureof high-income households.It is harder to guesstheir food expenditure Another way of describingwhat we havejust said is to saythe probability of getting large positive or negativevaluesfor e;is higher for high incomesthan it is for low incomes.Factor: other than income can have a larger impact on food expenditurewhen householdincome is high. How can we model this phenomenon?A randomvariable,in this casee;,hasa higher probability of taking on large valuesif its varianceis high. Thus, we can capturethe effect we aredescribingby having var(e;)dependdirectly on incomex;.An equivalentstatementis to say var(y;) increasesas rr increases.Food expenditureyr can deviate further from its mean E(yi) : 9r * pzxi when x; is large.In sucha case,when the variancesfor all observation-s are not the same,we say that heteroskedasticity exists. Alternatively, we say the random variabley;andthe randomerrorerareheteroskedastic.Conversely,ifall observationscome from probability density functions with the samevariance,we say that homoskedasticitl exists,and yi and ei are homoskedastic. The heteroskedastic assumptionis illustratedin Figure 8.1. At 11,the probability densitr function f (yrlxr) is suchthatyl will be closeto E(yr) with high probability.When we move to 12,the probability densityfunction f (yzl*z) is more spreadout; we arelesscertainabout where y2 might fall, and larger values are possible.When homoskedasticityexists, the probability density function for the errors doesnot changeasx changes,as we illustrated in Figure 2.3. Note that the existenceof heteroskedasticity is a violation of one of our least squares assumptionsthat were listed in Section5.1. When we previouslyconsideredthe model in
r.-i;, we assum ,nstantvariance
l:re assumptionu -,r l'i) - var(ei) :,,n of the form
.' hereh(.r;) is a I This chapter i '.\.hat are the con -'.rimationtechn We can furth :.monstrate an .-:uaresestimati --juaresresidual ' ,d.dat is
\ vraphofthis e . . ,r-;),appearst :.)ints that devia f :.rints Scattered 'jirture is to say
, increasein ab Sincethe obs . ).givenby e; =
8.1 THE NATURE
3nditurefunction. ure changeswhen suresexpenditure ing by institutions z we considereda r (yi, xi) denoting
OF HETEROSKEDASTICITY
199
Probability density function
fo)
ve the same food todel, we let e; be ln expenditureon 9r + Frr ( 8 . 2r rold is written a: (8.3,
r rcune 8. 7 Heteroskedastic enors.
8.3), we assumedthat the !;lvgte uncorrelatedrandom error terms with mean zero and ronstantvarianceo2. That is.
by income x1and E ( e 1 ): 0 l- $zx is betterar for high-income rseholdand food be easier?Lou nparatively,ther incomeon food. extravagantfood nterpartshave to tory variable for rod expenditure. r ofgetting large ncomes.Factor: ehold income is ) ei,hasa higher rre the effect we t statementis to ,r from its mean all observations say the random iervationscome noskedasticitl babilitydensit\ Whenwe move is certainabout city exists, the re illustratedin r least squares d the model in
v a r ( e i ): 6 '
c o v ( e ; ,e i ) : 0
The assumptionwe arequestioningnow is the constantvarianceassumptionMR3 that states r ar(y;) : var(e;) : o2. Our discussionsuggeststhat it shouldbe replacedwith an assump:ion of the form var(yr): var(er): h(xi)
(8.4)
.rhere ft(x;) is a function of x; that increasesas -rr increases. This chapter is concernedwith the consequerrcesof a variance assumption like (8.4). \\-hat are the consequencesfor the properties of least squaresestimators?Is there a better 3\timationtechnique?How do we detectthe existenceof heteroskedasticity? We can further illustrate the nature of heteroskedasticity,and at the same time lemonstrate an informal way of detecting heteroskedasticity,by reexamining least .quaresestimation of the mean function E(yi): Fr * pz.ri and the correspondingleast .quaresresiduals.The least squaresestimatedequationfrom the observationsin the file ',tod.datis li :83.42 -l l0.2l xi \ _eraphof this estimatedfunction, along with all the observedexpenditure-incomepoints i,. "{r),appearsin Figure 8.2. Notice that, as income (x;) grows, the prevalenceof data rcints that deviate further from the estimated mean function increases.There are more foints scatteredfurther away from the line asrr gets larger. Another way of describing this :eatureis to say that there is a tendencyfor the least squaresresiduals,definedby ?i:yi-83.42-l}.2lxi :') increasein absolutevalue as income grows. Sincethe observableleastsquaresresiduals(2i) areestimatesof the unobservableerrors errorstend c,), givenby ei : yi - 9r - !zxi, Figure8.2 alsosuggeststhat the unobservable
HETEROSKEDASTICITY
a
a
lrv=83'+z+ tl.
EV
I . The least best.The:
aa a
!; c
!
7a aa
o
a
a t
a
a
at
a
1f
r,8
05101520253035 ,r = weeklyincomein $ 100 FIGURE
8,2
:ince the existen 'l is violated, w -'.timatorand wl
a
a
a
8.2 Using
l.
The stand Confident misleadin
,\'e considerthe For the simpl
Least squares estimated expenditure function and observed data points.
e showedin Cl to increasein absolutevalueasincomeincreases. That is, thevariationoffood expenditure aroundmean food expenditureE(y;) increasesas income -rl increases.This observation consistentwith the hypothesisthat we posedearlier,namely,that the meanfood expenditL::. function is betterat explainingfood expenditurefor low-income(spaghetti-eating) hou.: holds than it is for high-incomehouseholdswho might be spaghettieatersor caviareatr':. We can capturethe increasingvariation of y; around its mean by the heteroskedastici:,. assumptiongiven in (8.4). Heteroskedasticityis often encounteredwhen using cross-sectionaldata. The ter:: cross-sectionaldata refersto having data on a number of economicunits suchas firms , households, at a givenpoint in time.The householddataon incomeandfood expenditu:, fall into this category.Otherpossibleexamplesincludedataon costs,outputs,and inpu:. for a number of firms, and data on quantitiespurchasedand prices for somecommodir. or commodities, in a number of retail establishments.Cross-sectionaldata invariat'. involve observationson economic units of varying sizes.For example, data on hourr holds will involve householdswith varying numbersof householdmembersand differe: levels of householdincome. With data on a number of firms, we might measurethe sizof the firm by the quantity of output it produces.Frequently,the larger the firm, or tllarger the household,the more difficult it is to explain the variation in some outconrvariable1; by the variation in a set of explanatoryvariables.Larger firms and househoie. are likely to be more diverse and flexible with respectto the way in which values for are determined.What this meansfor the linear regressionmodel is that, as the size of th. economicunit becomeslarger,thereis more uncertaintyassociated with the outcomes-. This greateruncertaintyis modeledby specifyingan error variancethat is larger,th. l a r g e rt h e : i z e o f t h e e c o n o m i cu n i t . Heteroskedasticity is not a propertythat is necessarilyrestrictedto cross-sectional dat., \\'ith time-seriesdata,wherewe havedataover time on one economicunit, suchas a firm. . household,or evena whole economy,it is possiblethat the error variancewill change.Th:. w'ouldbe true if therewasan externalshockor chansein circumstances that createdmore r : less uncertaintyabout,]'. The plotting of leastsquaresresidualsis an informal way of detectingheteroskedasticirr More formal tests are consideredlater in the chapter.First, however,we examine thl consequences of heteroskedasticity for least squaresestimation.
\ori.' supposethe ::i. differenceb
is shownin ,A .iirrator for B2
'here lu; : (;r; :.timatorand its ..6) to comput This problem ..lch an estima .:rors (the stan :her coefficien heteroskedast robust,standar ,r both hetero To obtain thr ..tuaresresidua .,.iuares residua
f J . 2U S I N G T H E L E A S T S Q U A R E S E S T I M A T O R
201
8.2 Using the Least Squares Estirnator meansthat the leastsquaresassumptionvar(e;) : Sincethe existenceof heteroskedasticity o2 is violated, we need to ask what consequences this violation has for our least squares estimator and what we can do about it. There are two implications: l.
The leastsquaresestimatoris still a linear and unbiasedestimator,but it is no longer best. There is another estimator with a smaller variance.
2. The standarderrors usually computed for the least squaresestimator are incorrect. Confidence intervals and hypothesis tests that use these standard errors may be misleading. We consider the secondimplication first. What happensto the standarderrors? For the simple linear regressionmodel without heteroskedasticity yi : 9r * gzxi I ei
var(e;) : 62
(8.5 )
ri e showed in Chapter 2 that the variance of the least squaresestimator for b2 is )xpenditurer rbservationi. d expenditure ating) housecaviar eaters. oskedasticitr .ta. The term ;h as firms or I expenditure ts,and input: : commoditr. ta invariablr ta on houseand differenr .surethe size : firm, or the me outcome I households valuesfor 1 re size of the outcomes_'\'r. s larger, the :ctionaldata. hasafirm,a change.This atedmore or skedasticitl. :xamine the
o2
v?rlD,tl:-
(8.6)
Lli-t?i -r)'
\ow supposethe error variancesfor each observationare different, and that we recognize this differenceby putting a subscriptI on o2. so that we have yi : Fr * gzxi * ei
var(ei) : s!
(8.7)
It is shownin Appendix 8A at the end ofthis chapterthat the varianceofthe leastsquares .'stimatorfor B2 under the heteroskedastic specificationin (8.7) is N
var\b2):
Lwioi
;-t
I[' [('' -t)'"?] fIilr (x'
-x)')'
(8.8)
,,rherew;: (xi - t)12(xt -x)2. Consequently, if we proceedto use the least squares .'stimatorand its usual standarderrorswhen var(e;) : oi, we will be using an estimateof 3.6) to computethe standarderror of D2when we should be using an estimateof (8.8). This problemcanbe overcomeby using an estimatorfor the varianceof bz givenin (8.8). Such an estimatorwas suggestedby econometricianHal White. The resulting standard r-rrors(the standarderror for b2and the standarderrors for the least squaresestimator of ..ther coefficients in the multiple regressionmodel) have become known as White's heteroskedasticity-consistent standard errors, or heteroskedasticity robust, or simply robust, standard errors. The term "robust" is usedbecausethey arevalid in large samples lor both heteroskedasticand homoskedasticerrors. To obtain the White standarderror for b2 corresponding to (8.8), we obtain the least luares residuals2i : li - b1 - b2xi andreplaceof in t8.8) with the squaresof the least .duaresresiduals. The White variance estimator is siven bv
;i6):
L w7e7:
_7)']' [I[, (",
(8.9)
202
HETEROSKEDASTICITY
and the White standard error is given by the square root of this quantity. In multiplc regressionmodels the formulas are more complex, but the principle is the same.Replacing of with the squared residuals ?! leads to a variance estimator with good properties ir large samples because large variances tend to lead to large values of the squarec residuals. Most regressionpackagesinclude an option forcalculating standarderrorsusing White'i estimator. If we do so for the food expenditure example, we obtain yi:
83.42f- l0.2lxi (27.46) (1.81) (White se) (43.41) (2.09) (incorrect se)
In this case,ignoring heteroskedasticityand using incorrect standarderrors, basedon tlr usual formula in (8.6), tends to understate the precision of estimation; we tend to -ee: confidence intervals that are wider than they should be. Specifically, following the resul: in (3.6) in Chapter 3, we can construct two corresponding 95Vo confidence inten'al' for p2. White : Incorrect :
b2 L t"se(b2): 10.21L2.024x 1.81: 16.55,13.87] b2 t t,se(b2) : l}.2l I2.024x2.09 : [5.97, 14.45]
If we ignore heteroskedasticity,we estimatethat B2 lies between 5.97 and 14.45.When u e recognize the existence of heteroskedasticity,our information is more precise, and ue estimatethat 9z lies between6.55 and 13.87. White's estimator for the standarderrors helps us avoid computing incorrect inten'ai estimates or incorrect values for test statistics in the presence of heteroskedasticitl However, it does not addressthe first implication of heteroskedasticitythat we mentioned at the beginning of this section,that the least squaresestimator is no longer best.In the nerr sectionwe describean alternativeestimatorthat hasa smallervariancethan the leastsquare! estimator.
.ubsections.D :ecome clear i.3.1
Tnar.
'lur earlier in; .rggestedthat re variance o
Thatis, we ass irrnstant parar :roportional tc .:.Iat,for low le' : rnction E(Y;) :rplained by tl rore from the I a\ specific tast ,.rriation in fo, The least sq reteroskedast -ipproachthis P :rrors. Leavin .kedasticerror :erried out, aP 3!timator. To demons s.l0) by .fl
8.-l The GeneralizedLeast Squares Estirnator Consider again the food expenditure example and the heteroskedasticity assumption employed in the last section, namely Pr*9zxi*ei var(e;): ol, cov(ei,e) :0
/i: E(e;) : g,
\ow, define tl
(8.10r
Although it is possibleto obtain the White heteroskedasticity-consistent varianceestimates by simply assumingthe error variancesof can be different for eachobservation,to develop an estimator that is better than the least squaresestimator we need to make a further assumption about how the variances ol change with each observation. This further assumptionbecomesnecessarybecausethe best linear unbiasedestimator in the presence of heteroskedasticity,an estimator known as the generalized least squares estimator. depends on the unknown ol. tt is not practical to estimate N unknown variances ol,ol,. ..,ozN with only N observationswithout making a restrictiveassumptionabout how the of change.Thus, to make the generalizedleastsquaresestimatoroperational,some structure is imposed on of . Three alternative structuresare consideredin the next three
.o that (8.12)
The beauty < homoskedas
.\lso, the tra and zero co
8.3 THE GENERALIZED
In multiple e. Replacin-e )roperties in the squared
LEAST SQUARES ESTIMATOR
203
subsections. Details ofthe generalizedleastsquaresestimatorand the issuesinvolvedrvill becomeclea.ras we work our way through these sections. 8.3.1
TneNsron-nnrNcTHE Moon
singWhite's Our earlier inspection of the least squaresresiduals for the food expenditure example suggestedthat the error varianceincreasesasincome increases.Onepossibleassumptionfor rhe variance o! that has this characteristicis
( 8 . 11 )
vn(e;):o?:ozxi asedon the tend to ger g the resulr :e intervalr
i. When we le, and u'e :ct interval ledasticitr. mentioned In the next rst squares
That is, we assumethat the varianceof the lth error term of is given by a positive unknown !'onstant parameter o2 multiplied by the positive income variable x;, so that var(e) is proportional to income. As explained earlier, in economic terms this assumptionimplies rhat,for low levels of income (x;), food expenditure(y;) will be clusteredcloser to the mean runctionE(y;):Fr*B2r;.Expenditureonfoodforlow-incomehouseholdswillbelargely :xplained by the level of income. At high levels of income, food expenditurescan deviate morefrom the meanfunction. This meansthat there arelikely to be many other factors,such as specific tastesand preferences,that reside in the error term, and that lead to a greater rariation in food expenditure for high-income households. The least squaresestimator is not the best linear unbiasedestimator when the elrors are heteroskedastic.What is the best linear unbiasedestimator under thesecircumstances?We approachthis problem by changing or transforming the model into one with homoskedastic errors. Leaving the basic structure of the model intact, it is possible to turn the heteroskedasticerror model into a homoskedasticenor model. Once this transformationhasbeen ;arried out, application ofleast squaresto the transformedmodel givesabest linearunbiased estimator. To demonstratethese facts, we begin by dividing both sides of the original model in r8.10)by ./7J
ft:,,(#."(h).-* lSumption
(8.12)
\ow, define the following transformed variables
1 --Il
-J
*xi
'-12
t;.'
\/
i\
* c;
Ll --
Jxi
(8.10)
( 8 .I 3 )
!nt
so that (8.12) can be rewritten as estimates o develop a further s further presence rstimator. rariances on about ral, some ext three
yi : Fph * g2xi2-rei
(8.1-l)
The beauty of this transformed model is that the new transformed error term ei is homoskedastic.The proof of this result i s : var(ei)
-
rrer r q
/ e,\
| ----:-
I
-
I |
-
-
l11 -var\ei)
:
-o-xi:
o'
)
( 8 .l 5 )
\r/xi/
Also, the transformed error term will retain the properties of zero mean, E(ei) :0. and zero correlation between different observations, cov(ei, eri):0 for ifj. As a
204
H E T E R O S K ED A S T I C I T Y
consequence,we can apply least squaresto the transformedvariables,yi,xir, andxit to obtain the best linear unbiased estimator for B1 and B2. Note that the transformed variablesyi , xir, andxi, are all observable;it is a straightforwardmatter to compute "the observations"on thesevariables.An important difference,however,is that the model nc longer containsa constantterm. The old x;1 is implicitly equal to 1 for all observation: The new transformedvariable xh : I I t/n is no longer constant.You will have to be careful to exclude a constantif your software automaticallyinsertsone,but you can stiI . proceed.The transformedmodel is linear in the unknownparametersB1and B2.Theseare the original parametersthat we are interestedin estimating.They havenot been affectetj by the transformation.In short,the transformedmodel is a linear model to which we car: apply least squaresestimation. The transformed model satisfiesthe conditions of the Gauss-Markov theorem, and the least squares estimators defined in terms of the transformedvariables are BLUE. To summarize, to obtain the best linear unbiased estimator for a model with heteroskedasticityofthe type specifiedin equation(8.11) l.
Calculatethe transformedvariablesgiven in (8.13).
rnterpretationsfor lhe untransformed .levicefor convert ts somethingthat The standarder :han their least sql .e(b1):27.46 an procedurethan lea .ower. This staten .tandard errors ar nagnitudes of va r'stimates.Thus, lr :eductionin variar :he varianceshav The smaller sta ;onfidence interv :onfidenceinterv
2. Use least squaresto estimatethe transformedmodel given in (8.14).
t
The estimator obtained in this way is called a generalizedleast squaresestimator. One way of viewing the generalized least squaresestimator is as a weighted least squares estimator. Recall that the least squaresestimator yields values of B1 and B2 that minimize the sum of squarederrors. In this case, we are minimizing the sum of squared transformed errors that is given by
The least squares :_1.871.
\.3.2 N
N.2
Z e i 2 :it- l1X:1
i-l
N
Esrrlrer
,,.
tG,'/'",)'
ln the previous r ariancefunctior
i:l
The errors areweightedby *,' /',the reciprocalof Jn.When .ft is small,the datacontain more information about the regressionfunction and the observationsare weighted heavih. When afi is large, the data contain less information and the observationsare weighted lightly. In this way we take advantageof the heteroskedasticityto improve parameter estimation. Most software has a weighted least squaresor generalizedleast squaresoption. If your software falls into this category,you do not have to worry about transforming the variables before estimation,nor do you have to worry about omitting the constant.The computer will do both the transforming and the estimating.If you do the transforming yourself, that is, you create yi, xi, and xi, and apply least squares,be careful not to include a constant in the regression.As noted before,there is no constantbecausexh*1. Applying the generalized(weighted)least squaresprocedureto our householdexpenditure data yields the following estimates: /i:78.68*10.45x;
(se)(23.7e)(1.3e)
(8.l6)
That is, we estimatethe interceptterm as B1 : 78.68andthe slopecoefficientthat showsthe responseof food expenditureto a change in income as B2 : 10.45. These estimates are somewhatdifferent from the least squaresestimatesbt : 83.42 and b2: 10.21that did not allow for the existenceof heteroskedasticitv.It is important to recosnize that the
lar(er): ozfiat :ncreases.Why :ncludesall thes
'.rhere ^yis an ur How do we Jiscussionsugg ,n each variablt ',arianceo2. Do :quation with 1 Because^Yis .ition. To do so :notivate this fri
Then, taking th
8.3 THE GENERALIZED
i ,xir, andxi, to re transformed r compute "the at the model no I observations. ,vill have to be ,utyou can still d 82. Theseare 't beenaffected >which we can nditions of the terms of the el with hetero-
LEAST SQUARES ESTI]\{ATOR
205
interpretations for B1 and B2 are the samein the transformedmodel in (8.14) as the) are in the untransformedmodel in (8. l0). Transformationof the variables should be re-qardedas a error ntodel.rtol error model into a hontoskedastic devicefor convertinga heteroskedastic as something that changes the meaning of the cofficients. The standarderrors in (8.16),namelyse(B1): 23.79andse(811 : 1.39.arebothlower rhantheir least squarescounterpartsthat were calculatedfrom White's estimator,namely :e(b1):27.46 and se(b):1.81. Since generalizedleast squaresis a betterestimation procedurethan leastsquares,we do expectthe generalizedleastsquaresstandarderrorsto be lower. This statementneeds to be qualified in two ways, however. First, remember that standarderrors are square roots of estimated variances; in a single sample the relative magnitudes of variances may not always be reflected by their corresponding variance estimates.Thus, lower standarderrors do not always mean better estimation. Second,the reductionin variancehascome at the cost of making an additional assumption,namely, that the varianceshave the structuregiven in (8.11). The smaller standarderrorshave the advantageof producing narrower more informative .-onfidenceintervals. For example, using the generalized least squares results, a 95Vo .'onfidenceinterval for B2 is given by
).
9 2 + r " s e i B 2 ;: 1 0 . 4 5 1* 2 . 0 2 4 x 1 . 3 8 6 : 1 7 . 6 5 , 1 3 . 2 6 )
timator. veighted leasr Fr and B2 that um of squared
The least squaresconfidence interval computed using White's standarderrors was [6.55, t3.871.
8.3.2
le datacontain ghtedheavill. are weighted )ve parameter
EsrruerrNc rHE VARTANcpFuNcrtoN
In the previous section we assumed that heteroskedasticitycould be described by the r ariancefunction var(ei) : o2xi. There are other alternatives,however.For example,both rar(e;) : ozx! andvar(r,): o2*',1'havethepropertythattheerrorvarianceincreasesasx; increases.Why not choose one of these functions? A more general specification that includesall thesespecificationsas specialcasesis
v a r ( e ;:)o ? : o 2 x 7 rption.If your ; the variables oomputerwill )lf,that is, you rnstant in the ehold expen-
( 8 .t 6 )
( 8 .l 7 )
*.here 1 is an unknown parameter. How do we proceed with estimation with an assumptionlike (8.17)? Our earlier .liscussionsuggeststhat we should transformour model by dividing the ith observation t'rnesch variable Ay i'!/'. Doing so will lead to a transformed error term with constant rarianceo2. Do you understandwhy? Go back to (8.15) and redo the little proof in this equationwith 1 included. Because^yis unknown, we must estimate it before we can proceed with the transformrlion. To do so it is convenientto consider a framework more general than (8.17). To motivatethis framework we take logs of (8.17) to yield h(ol):tn(o2)+1ln(x1)
hatshowsthe se estimates : 10.21that nize that the
Then, taking the exponential of both sides,
of : expfln1o2) + azzi) + lln(x;)] : exp(.,1
(8.l 8)
206
HETEROSKEDASTICITY
where a1 : ln(o2), d.2: \;aod 4; : ln(x;). Writing the variancefunction in this form is convenientbecauseit showshow the variance can be related to any explanatory variable : that may or may not be one of the variablesin the mean function E(y;) : 9r * B2r;. Also, if we believe the variance is likely to dependon more than one explanatory variable, say :i:. (8.18)can be extendedto the function zi3,...,zis,
o? : exp(ar* o.zziz*"' * asz;s)
( 8 .l 9 ,
The exponential function is convenient becauseit ensuresthat we will get positive values for the variancesof for all possiblevaluesof the parameterscr1,cr2,...,os. Returningto (8.18), we rewrite it as h(ol) :o.t+s2zi
(8.20,
and now addressthe question of how to estimate ct1 and cr2.Recall how we get the least squaresestimator for the mean function E(yr) : 9r * pzri. We expressedthe observations )t as yi: E(yi)* e;: 9r * lzxi * ei and then applied least squares.We can follow a similar strategyfor estimating the variance That is, ue function using the squaresofthe leastsquaresresiduals2f asour observations. write nG?) :h(o/)
+ vi : crl I qzzi r vi
rnd then dividt :eneralizedlea 3 . 1 0 )b y o ; Y i
(8.2r
and apply leastsquares.Regressingln(Zf) on a constantandziyieldsleastsquaresestimates for ctl and ct2. Whether or not this procedureis a legitimate one dependson the properties of the neu error term v; that we introduced in (8.21). Does it have a zero mean?Is it uncorrelatedanC homoskedastic?The answerto thesequestionsis no; E(v;) I 0 and the vr areboth correlated and heteroskedastic.However, it can be shown that the least squaresestimator for ct2(anC any other slope parametersthat might be present) is unbiasedin large samples.The lea-st squaresestimator for the intercept al is asymptotically biased downward by the amount 1.2104,and thus the obvious "fix'o is to use the intercept estimatordr : dr +1.21oJ Interestingly,this correction hasno effect on the generalizedleast squaresestimatesofthe p coefficients becauseol cancels out during the calculations.' In the food expenditureexample,with z; defined as zi: ln(x), the least squaresestimate o f ( 8 . 2 1 )i s t n ( 6 -)f : 0 . 9 3 7 8 * 2 . 3 2 9 2 i Notice that the estimatedz : i : 2.329 is more than twice the value of ^y: 1 that was an implicit assumptionof thevariancespecificationusedin Section8.3.1.It suggeststhe earlier assumptioncould be too restrictive. The next stepis to transform the observationsin such a way that the transformedmodel hasa constanterror variance.As suggestedearlier, we could do so by dividing both sidesoi the equationyi : 9r * gzxi * ,,by il'. However, in line with the more general specification in (8.19), we can obtain varianceestimatesfrom
6?:exp(d.r+drz;) t Further discussion of this advanced point can be found in Introduction to the Theory and Practice r,.' Econometrics,2nd Edition (Wiley, 1988) by Judge, Hill, Griffiths, Liitkepohl and Lee, pp. 365-369.
The variance c
Thus,to obtain :iace of the ur
::rd apply leas
To summar
ihe stepsfor r l.
Estima residua
Estima d2Z;2 1
Compt
Compt K>2
Applv
iieps4and5 .rttware aut( : -nunate,you Following :rpenditure e
8.3 THE GENERALIZED
n this form is rry variable z; $z-ri.Also, if iable, say zi:,
andthen divide both sidesof the equationby 6;. Both strategiesultimately lead to the same generalizedleastsquaresestimatesfor B1and B2.Why doesthe secondonework? Dividing t8.10)by o; yields /v,\ ^ /l\ ^ /r,\ /r,\ (;/ : B,(a,l* B,(;,). (;/
(8.19) rsitivevalues
LEAST SQUARES ESTIMATOR
The varianceof the transformederror is constant(homoskedastic)because
: (1)":: ' "-(;): (#)"-,",r
(8.22)
(8.20) get the least observations
Thus,to obtaina generalizedleastsquaresestimatorfor B 1 and B2,usingthe estimates6f in place of the unknown o! , we define the transformed variables
,': (#)"o: (;) "r: (#) the variance . That is, we
(8.23)
and apply least squaresto the equation
yi : gtxh -t gzxiz+ ei (8.21) 'esestimates
(8.24)
To summarize these steps for the general case, supposewe are estimating the model yi : Ft * 9zxiz+ "' + \rxir * ei
s of the new rrelated and h correlated for ct2(and s. The least the amount t -l1.2704. rtesof the B 'esestimate
.hat was an ; the earlier ned model rthsidesof specifica-
Practice of
(8.25)
rl here
var(ei): o! : exp(ct1I azziz* ...*
aszis)
(8.26)
The stepsfor obtaining a generalizedleast squaresestimatorfor B1, Fz, . . . , Br are 1. Estimate (8.25) by least squaresand compute the squaresof the least squares residuals?f. 2. Estimateo4,oz,...,os by applyingleastsquaresto the equationln(Z/) :ar* a 2 Z i I2, ' ' + c t s z r s + v i . . 1 . C o m p u t ev a r i a n c e s t i m a t e6s7 : e x p ( , i r - a z z i z + . . . + & s z i s ) . l.
Compute the transformedobservationsdefinedby (8.23), including xiz,. .. ,xi* if K>2.
5. Apply least squaresto (8.24), or to an extendedversionof (8.24), if K > 2. Steps4 and5 can be replacedby weightedleastsquareswith weightsdefinedby o, t if your software automatically computes weighted least squares estimates.If you are very tortunate,you will havesoftwarethat performsall five stepswith one click of 1'ourmouse. Following these steps to obtain generalizedleast squaresestimates for the food expenditureexample yields
ti :76.05 * 10.63x (s") (e.tI) (0.e7)
( 8 . 1)7
208
t
HETEROSKEDASTICITY
Comparedto the generalizedleast squilresresults for the variance specificationo? : o2xi. the estimatesfor B1 and B2 have not changeda great deal, but there has beena considerable drop in the standarderrorsthat, underthe previousspecification,were se(B1):23.79 and se(82): 1.39. As mentioned earlier, because standard errors are themselves estimates, we cannot conclude with certainty that allowing for a more general variance specification has improvedthe precisionwith which we haveestimatedB1and B2.However,in this particular caseit is distinctlypossiblethat our improvedresultsareattributableto bettermodelingand better estimation. 8.3.3
A HErmosKEDASTrc PeRrrrroN
To introduce a third form of heteroskedasticityand the generalizedleast squaresestimator corresponding to it, we return to a data set that was used in Chapter 7 to estimate wage equations and to illustrate the use of dummy variables. In that chapter several equations were estimated using variables such as level of education, race, gender, and region of residenceto explain differencesin the mean level of wages.Although the equationswere simplifications designed to aid the interpretation of dummy variables, the explanatory variables that were used are typical of those included in labor market applications.In this sectionour illustrative exampleis anothersimplified wage equationwhereearningsper hour (WAGE) depends on years of education (EDUC), years of experience (EXPER), and a dummy variableMETRO that is equal to I for workers who live in a metropolitan areaand 0 for workers who live outside a metropolitan area. Using data in the file cps2.dat the least squaresestimated equation for this model is
WAGE: -9.914 + |.234EDUC+ O.|33EXPER + |.524METRO ('") (1.08) (0.070) (0.01s) (0.431)
(8.28)
The resultssuggesteducationand experiencehave a positive effect on the level of wagesand that, given a particular level ofeducation and experience,the averagemetropolitan wage is $1.52 per hour higher than the averagewage in a rural area. The question we now ask is: How does the variance of wages in a metropolitan area comparewith the varianceof wagesin a rural area?Are the varianceslikely to be the sameor different? One might suspect that the greater range of different types of jobs in a metropolitan area will lead to city wages having a higher variance. To introduce a framework for investigating this question, we partition our sample into two parts, one for the metropolitan observations(for which we usethe subscript " M") andthe other for the rural observations(for which we use the subscript"R") -t \zEDUCm * S 3 E X P E R y*i e y i i : 1 , 2 , . . . , N u W A G E R i : B n rf B 2 E D U C p* ; g t E X P E R n*i e n i i : 1 , 2 , . . . , N n
W A G E 6 ;: 9 m
fhe new assumPtio :.oteQual.For the le ' x(ey1) : vatleni) ::Stl[Ie that the er ihat is,
ilaving two subset rto what we have t ln the file cPs2. bservations.Usin6 .sparatelyYieldsvt
lhe estimatederror : 'r the rural wage e( . sufficientlYlarge ..\'e defer such a tes One setof estim -'estsquaresestim
i{orvever,a Proble -'.timatesfor B3, "r .i agesis the same (o "ctter estimates irta and aPPlYin ::cognition given The strategYfo rrevious section' .:andarddeviatio rertition, that me 'bservationsare t (wAGEMi\ \""
)
(8.29a7 (8.29b)
/wAGEni\ Implicit in the above specification is the assumptionthat the coefficients for EDUC and EXPER (9: and B3) are the same in both metropolitan and rural areas,but the intercepts differ. This assumptionis in line with the estimatedequationin (8.28)wherethe estimatefor Ba1is Dp1: -9.914 and the estimatefor py1 is
\on/
bm : -9.914+ 1.524: -8.39
The variances rre both equal to
8.3 THE GENERALIZED
cation03 : o2xi. en a considerable
i0') : 23.79and :lates,we cannol specificationhas , in this particular ;termodelingand
;quaresestimator :o estimatewage ieveralequations :r, and region of : equationswere the explanatory rlications.In this :arningsperhour (EXPER), and a rolitanareaand 0 ps2.datthe least
o -
(8.28r
vel ofwagesand rpolitan wage is etropolitan area o be the sameor s of jobs in a To introduce a two parts, one theother for the
(8.29a)
?
(8.29b)
tor EDUC and . the intercepts he estimatefor
209
The new assumption in (8.29) is that the variancesof the two error terms e74;lnd ep; are not equal.For the least squaresestimatesin (8.28) to be best linear unbiased,we require tar(ey1): var(en;) : o2; the error variance is constant for all observations.We nou' assume that the error variances in the metropolitan and rural regions are different. That is, var(e1ai): 62r,
(8.30)
var(ea;) : srfl
Having two subsetsof observations,each with a different variance, partitions the sample rnto what we have called a heteroskedastic partition. In the file cps2.dat there are N,u: 808 metropolitan observationsand Np : 192 rural ,rbservations. Using theseobservationsand least squaresto estimate(8.29a)and (8.29bt \eparately yields variance estimates 6 2 u: 3 1 . 8 2 + ,
oA:
15.243
The estimatederror variancefor the metropolitanwage equationis approximatelydouble that Ior the rural wage equation.Whetherthis differencecould be attributableto samplingerror.or r\ sufnciently large to conclude that o2r l do, it somethingto decideusing a hypothesistest. \\'e defer such a test until the next sectionand, for the moment, assumethat oh + 62R. One set of estimatesthat recognizesthat the error variancesare different are the separate .eastsquaresestimatesof (8.29a)and (8.29b) that turn out to be but : -9.052, bm : -6.166,
buz : 1.282, buz :0.1346 bnz:0.956, b n z: 0 ] 2 6 0
However, a problem with these estimatesis that we have two estimates for B2 and two :'stimatesfor B3,when in (8.29)we tue assumingthe effect of educationand experienceon .i agesis the samefor both metropolitan and rural areas.Given this assumptionis correct, "etter estimates(oneswith lower variances)can be obtainedby combining both subsetsof jata and applying a generalizedleast squaresestimator to the complete set of data, with ::cognition given to the existenceof heteroskedasticity. The strategyfor obtaininggeneralizedleastsquaresestimatesis the sameasit was in the rrevious section. The variables are transformedby dividing each observationb1' the .tandarddeviation of the correspondingerror term. In our example of a heteroskedastic rartition, that means that all metropolitan observations are divided by o.y and all rural 'bservationsare divided by oa. Equations(8.29a)and (8.29b) become r \ (EDUCpli\, ^ (EXpERazi\ /rr,\ | - l : oir.r F u t t - | + F z l -/ I + F :l o oM ay
(wAGEMi\ _, \
IM
LEAST SQUARES ESTIMATOR
/
\oul
\
/
\
/
\ou/
i:1,2,...,NM (wAGERi\ ^ (t\,^(EDUC*,\ l:l5RIl-l-uzl-l.rl5:l \onl\onl\oRl\oR/\on/
^(EXqERR\
(8.3Ia)
/.n,\ lrl-l
i:1,2,...,NR
t8.3lbr
The variancesof the transformederror terms (e1a;f o y) and (, o,I o ^) are the same.Thel :re bothequalto 1.Is this fact obviousto you?No? Go back andcheckout (8.15)and (8.221
210
HETEROSKEDASTIC ITY
When you are comfortable, it will be clear to you that the combined set of error terms is homoskedastic. Thus, application of least squares to the complete set of transformed observationsyields best linear unbiasedestimators. There aretwo complications,however.The first is that oaaand oa areunknown. We solve this problem by transforming the observationswith their estimates 67,aand 6p. Doing so yields a generalizedleast squaresestimator that has good propertiesin large samples.The second complication relates to the fact that the metropolitan and rural intercepts are different. This complication will not necessarilybe present in all models with a heteroskedasticpartition, but it arisesin this casebecauseboth the mean and variance of wage depend on the dummy variable METRO. The different interceptsareaccommodatedby includingMETRO aswedid in the original equation (8.28), but this time it is transformedin the same way as the other variables. Collectingall thesefactstogether,we can combineequations(8.31a)and (8.3Ib) and summarize the method for obtaining generalizedleast squaresestimatesin the following wa). l.
Obtain estimates6,y and 6a by applying least squaresseparatelyto the metropolitan and rural observations.
whenMETROi: I I o, 2. Letoi : { ^'whenMETROi:0 o^ I 3. Apply least squaresto the transformed model
/1\ rtouc;\,.(EXPERi\ (wlcn.. ^ , _ fp 3 r _ _ _ _ _ _ : _ , t _ _oi = _ _ r:)B * , [ . ) + B r [ : \ / \oi/ \ oi / \ c; ,
,r.rr,
+{uU!o,) . (:) o; \
)
\",)
where B,yr : Bnr * 6. Following these stepsusing the data in the file cps2.dat yields the estimated equation
WAGE: -9.398 + l.|96EDUC+ O.I32EXPER + l.539METRO ('") (1.02) (0.06e) (0.01s) (0.346)
(8.33 )
Thesecoefficient estimatesaresirnilar in magnitudeto thosein (8.28), an outcomethat is not surprising given that both least squaresand generalizedleast squaresare unbiasedin the presenceof heteroskedasticity.We would hope, however, that the greaterprecision of the generalizedleastsquaresestimatoris reflectedin smallerstandarderrors.The standardenors in (8.28) are not a good basis for comparison becausethey are incorrect under heteroskedasticity.However,we comparethosein (8.33) with thoseobtainedby applying least : squaresseparatelyto the metropolitan and rural observations.ForEDUC they are se(bya2) : 0.080and se (bp2): 0.133, andfor EXPERthey arese (b6): 0.018and se (ba3) 0.025. Using the larger combined set of observationshas led to a reduction in the standarderrors. REMARK: The generalizedleast squaresestimatorsdescribedin the last three subsectionsrequire an assumptionabout the form of the heteroskedasticity,Using least squareswith White standard errors avoids the need to make an assumption about the form of heteroskedasticity,but does not realize the potential efficiency gains from generalizedleast squares.
8.4 Detec
in our discussi< rroblem and dt r-lowever, in tt .incertaintY abr .,.k:How do Ik ..f data? Is the :eneralized lei .luestions.The .'lassesof stati S.4.1 Rrsm
OnewaYof inv .eastsquaresa .hould be no Pi :end to exhibit srpenditure da --hapter(Secti :eported them :ndeedtendto .'an be followt When we h l5 not so easil residualsagai .l stematic we 8.1.2 Tnr
Our second different vari ;onsideredm that the error ^ ' ,
lls Oi4 :
- l
Jr'(
I
conclude tha The back degreesof I
subsamPle samPlesof that
whereNy UsuallY,K,r ratio of one
8.4 DETECTING
)f error terms i: of transformec nown. We solr e d 6p. Doing sc ;e samples.Thc I intercepts arc. r with a heteroriance of wage d in the origina. rther variable: i . 3 l b )a n ds u m following war re metropolitan
tx
{
d equation (8.:.r cmethatis noi nbiasedin thc ecision of the tandardenor, under heterotpplying leasr arese(by2) = bns):0.025 tndard errors. t three subUsing least n about the gains from
HETEROSKEDASTIC ITY
211
8. 4 Detecting Fleteroskedasticity In our discussionof the food expenditureequation,we usedthe nature of the economic problemand data to arguewhy heteroskedasticityof a particular form might be present. However, in this and in other equations that use other types of data. there *ill be :-rncertainty about whether a heteroskedasticassumptionis warranted. It is natural to rsk: How do I know if heteroskedasticityis likely to be a problem for my model andmy set .rf data? Is there a way of detecting heteroskedasticityso that I know whether to use reneralized least squarestechniques?We consider three ways of investigating these questions.The first is the informal use of residualplots. The other two are more formal ,'lassesof statisticaltests. S.1.1 Rrsrouer Prors Oneway of investigatingthe existenceof heteroskedasticity is to estimateyour modelusing .eastsquaresand to plot the leastsquaresresiduals.Ifthe errorsare homoskedastic,there .hould be no patternsof any sort in the residuals.If the errors are heteroskedastic,they may :end to exhibit greatervariation in some systematic way. For example, for the household r'\pendituredata,we suspectthat the varianceincreasesasincomesincreases.Earlier in this -'hapter(Section8.1) we plotted the estimatedleastsquaresfunction and the residualsand :eported them in Figure 8.2. We discovered that the absolute values of the residuals do :ndeedtendto increaseasincomeincreases. This methodof investigatingheteroskedasticity -'an be followed for any simple regression. When we have more than one explanatoryvariable, the estimatedleast squaresfunction .. not so easily depicted on a diagram. However, what we can do is plot the least squares :esidualsagainsteachexplanatoryvariable,or against!;, to seeifthose residualsvary in a .r stematic way relative to the specified variable. r..{.2
THr Gororrm-QuaNor
TEsr
Our second test for heteroskedasticityis designedfor two subsampleswith possibly .lifferent variances.In Section 8.3.3, under the title "A HeteroskedasticPartition," we ;onsideredmetropolitanand rural subsamplesfor estimatinga wage equation.Suspecting :hatthe error variancescould be differentfor thesetwo groups,we estimatedtheir variances ..' 62, :31.824 and&2p: 15.243.We now ask: Are thesevaluessufficientlydifferent to jr)ncludetnat o2r 7 olt The backgroundfor this test appearsin Appendix C.7.3.The only differenceis that the Jegreesof freedom change becausewe are consideringthe enor variancestiom t*rr .ubsamplesof a regressionequation rather than the variancesfrom two independent .amples of data. In the context of the example we are considering,it can be shorrn :hat
, :#ffi
- F11s,-ry,wp-Kn)
t 8 . - l - 1r
rihereN,y- KyandNp- Kparcthe degreesof freedomforthe two subsamplere-sres\ion\. Usually,Ku: Kn.In words, (8.34) says:The F statisticthat has a numeratorequal to the :atio of one varianceestimateto its true populationvalue,and a denominatorequal to the
212
HETEROSKEDASTICITY
ratio of the other variance estimate to its population value, has an F distribution with (Nu - Ku, Nn - Kn) degreesof freedom. Supposewe want to test
H():ozM:62Ragainst4:o2, lo2* Then, when F16is true, (8.34) reducesto F:&rlAk, equationis 6,M
F-
estimationin Sect letween the meal rppears in the reg
(8.35)
and its value for the wage
t#:2.0e
The variance func :size that the vari :'r)rm for the varia
oF
Given that (8.35)is a two-tail test,andrecallingthatN1a:808, NR: 192,andKy : IQ: 3. the relevant lower and upper critical values for a 5Vo significance level are F1, : : 0.81 and Fgr: F(o.qzs,sos,resl: F(o.ozs,aos,rsq) 1.26.We reject Holf F I Fy, or F ) Fy.. Since 2.09 > I.26, in this casewe reject /1s and concludethe wage variancesfor the rural and metropolitan regions are not equal. When following the aboveprocedure,it doesnot matterwhetheryou put the largervariance estimatein the numeratoror the denominatorof the F-statistic.However,if you alwaysput the larger estimate in the numerator, then you reject lls at a 5Vo level of significance if F ) Fu, : [email protected],Ny-Ky,Np-Kp). For a one-tail test the critical value changes. For : I.22. ft:o27a > ofr, we reject F1sat a 5Volevel of significanceif F > F,: F(o.qs,sos,rss) Sincewe originally hypothesizedthat greaterjobvariety in the metropolitanareamight leadto a greatervariance,one could arguethat a one-tail test is appropriate. Although the Goldfeld-Quandt test is specifically designed for instances where the sample divides naturally into two subsamples,it can also be used where, under fly, the variance is a function of a single explanatory variable, say 2,. To perform the test under thesecircumstances,we order the observationsaccordingto zi so that, ifheteroskedasticitl exists,the first half of the samplewill correspondto observationswith lower variancesand the last half of the samplewill correspondto observationswith higher variances.Then, we split the sample into approximately two equal halves, carry out two separateleast squares regressionsthat yield variance estimates, say 6l and 61, and proceed with the test as describedpreviously. Following these stepsfor the food expenditureexample, with the observationsordered according to income x;, and the ymple split into two equal subsamplesof 20 observations each, yields 6i : 3574.8 and 6i : 12921.9,from which we obtain ^1
o;
D_ t -:i-
oJ
h(o
.nd its specialca
\nother examPle
\otice what haP
The term h(or) .,ords, when ctz ,onstant. In terr r-onsequently,th ,n the variance I
r292r.9
:3.61 3574.8
Believingthat the variancescould increase,but not decreasewith income,we usea one-tail test wirh 5Vocritical value F1o.es,rs,tz;:2.22. Since 3.61 > 2.22, a null hypothesisof homoskedasticity is rejected in favor of the alternative that the variance increaseswith lncome.
8.4.3
Thisis a generalf iesirablefeatures Eramplesare the
TEsrrNc rnr VaRraucn FuNcrroN
In this section we consider a test for heteroskedasticitybasedon the variancefunction. The form of the variance function used for testing is slightly different from that used for
Thenull and alte .:teststatistic.Tc .re substituteinl
CtV;:e?-E !.-10),we can r
8.4 DETECTING
distribution with
HETEROSKEDASTICITY
estimation in Section 8.3, but the idea is the same.As before, it is important to distinguish between the mean function E(y) and the variance function vtr(y;). The mean function appearsin the regressionmodel
(8.35) y i : E ( y i ) * !i :
9r
-l
B 2 x 2+ " '
+ 9xxix t ei
(8.36)
re for the wage The variance function is relevant when heteroskedasticityis a possibility and we hypothesizethat the variance dependson a set of explanatory variableszi2, zi3,. . . ,zis.A general form for the variance function is
var(y,)- o7 : n@I : h(ot + d2zi2+. " * asz,s) ndKy: I(^- 3, evel are Fu : '{FuorF> rariancesfor the relarger variance lu alwaysput the : significance if : changes. For : 1.22. s,8os,r8e) reamight lead to
(8.37)
This is a general formbecausewe have not been specific aboutthe function ft(.). One of the desirablefeaturesof the testthat we developis that it is valid for all reasonablefunctions ft('). Examples are the function specified in Section 8.3.2, namely a s z ; s :) e x p ( c r+l a - 2 z i *z " ' * c t s z ; s )
h(ot +azZiz*..'+
and its special casethat appearsin (8.18) T
-.
I
ft(or + rtzzi): exP + lln(o') 1ln(x;).1 .{notherexampleis the linearfunction
nces where the :, under H1, the r the test under teroskedasticity ariancesandthe r.Then, we split e least squares vith the test as irationsordered l0 observations
e use a one-tail hypothesis of increaseswith
h(ot + d2zi2 + "'
* ctszis): ar * qzZiz*'
" * aseis
(8.38)
\otice what happensto the function ft(') when dz : 6y3: ' 'i ' : as : 0. It collapsesto h(ot + dzZiz* "' + crers) : ft(crr) The term h(ctr) is a constant; it does not depend on any explanatory variables. In other *ords, when olz: ry3: .. - : crs:0, heteroskedasticityis not present;the varianceis ;onstant. In terms of notation that you are familiar with, we can write o2:h(er). Consequently,the null and alternative hypothesesfor a test for heteroskedasticitybased on the variance function are HO:
ct2:Ct3:!r':695:0
Hr:
not all the cr"in /1s are zero
(8.3e)
The null and alternativehypothesesarethe first componentsof a test.The next componentis a teststatistic.To obtain a test statistic we considerthe linear variancefunction in (8.38) that t|e substituteinto (8.37) to obtain
var()r)- o? : E(e!) : o-r+ azzi2* "' * aszis
(8.40)
Let v; : e! - n1e!) be the difference between a squarederror and its mean. Then, from ,8.40).we can write function. The that used for
* v; e? : E(e?) -l v; : a1 | a2Z;2+ . .. + cr5Z;5
(8.41)
214
HETEROSKEDASTICITY
Notice that the addition of v; to the variancefunction servesa similar purposeto addition of e to the mean function in (8.36). There is an important difference, however. In (8.36) the dependentvariabley; is observable.If we try to estimate(8.41), we find that the "dependent variable" ef is not observab^le. We overcomethis problem by replacing e! with the squaresof the leastsquaresresiduals?f , obtainedfrom estimating(8.36).Thus,we write an operational version of (8.41) as O?: ot * azziz+ "' + crsZis * vr
(8.42t
Strictly speaking,replacing e! Ay a! also changesthe definition of v;, but we will retain the same notation to avoid unnecessarycomplication. The variance function test for heteroskedasticityuses quantities obtained from least squaresestimation of (8.42). We are interested in discovering whether the variables :i:. Zit. . . . ,z;5help explainthe variationin 2f . Sincethe R2goodness-of-fitstatisticfrom (8.41r
r.4.3a The Vhi ,)ne problem with :-.iveknowledge o :r pothesisof heter .t:.. . .,Zs.In reality :re relevant varial :efining the z's as t :requently,the var tlso, by usinga qu :.rnctions.SuPPos
lhe White test wit
measuresthe proportion of variation in 2f explainedby the z's, it is a natural candidatefor a test statistic.It can be shownthat, when I1sis true, the samplesizemultiplied by R2hasa chisquare (12) OistrlUutionwith S - 1 degreesof freedom. That is,
x2 : NxR2-x'tr-,1
(8.43r
It is likely that, so far, your exposure to the X2-distribution has been limited. It was introducedin Appendix 8.5.2,it was usedfor testingfor normality in Section4.3.4,and its relationship with the F-test was explored in an appendixto Chapter 6, Appendix 6,4,.It is a distribution that is used for testing many different kinds of hypotheses.Like an F random variable, a X2random variable takesonly positive values.Becausea large R2value provides evidence against the null hypothesis (it suggeststhe z variables explain changes in the variance),the rejectionregionfor the statisticin (8.43)is in the right tail ofthe distribution. Thus, for a5Vo significancelevel, we reject FIs and conclude that heteroskedasticityexists when 12 ) X2ro.ss.s_ri. There are severalimportant features of this test: l.
-rcluding interacti -..radraticterms (e. The White test :re 12 : NxR2 te -'-'onometric soft\ :iouse.
..1.3b Testing t :o test for hetero r.)tentially a func I the variance fur t *02x;*v;bY
It is a large sampletest.The result in (8.43) holds approximatelyin large samples.
2. You will often seethe test referred to as a Lagrange multiplier test or a BreuschPagan test for heteroskedasticity.Breusch and Paganused the Lagrange multiplier principle (seeAppendix C.8.4) to derive an earlier version of the test that was later modifiedby otherresearchers to theform in (8.43).The testvaluesfortheseandother slightly different versions of the test, one of which is an F-test, are automaticalll calculated by a number of software packages.The one provided by your software may or may not be exactlythe sameasthe NxR'version in (8.43).The relationships between the different versions of the test are described in Appendix 88. As you proceed through the book and study more econometrics, you will find that man.v Lagrange multiplier testscan be written in the form NxR2 where R2 comes from a convenient auxiliary regressionrelated to the hypothesis being tested. 3. We motivated the test in terms of an alternative hypothesis with the very general variancefunction o? : h(q + o2zi2^+.'.* crszis),yet we proceededto carry out the test using the linear function 0i : s..r* azziz+ ... +ctsz;sf v;. One of the amazingfeaturesof the Breusch-Pagantest is that the value of the statisticcomputed from the linear function is valid for testing an alternative hypothesis of heteroskedasticity where the variance function can be of any form given by (8.37).
:1d
>rncethere is onl ::eedom.The 5Vor -oncludethat the For the White' .nd testHg"ot2: ' .nd p-values,we .
x2
The 57ocritical vu .i ith the variance
8.4 DETECTING
r to addition ofe; er. In (8.36) the the "dependenr th the squaresof e an operational
(8.42) e will retain the Lnedfrom least te variables z;2. stic from (8.42)
imited. It was 'n 4.3.4,and its ndix 6,4.It is a e an F random value provides :hanges in the re distribution. lasticity exists
215
8.4.3a The White Test One problem with the variance function test describedso far is that it presupposesthat \\'e have knowledge of what variables will appear in the variance function if the alternative hypothesisof heteroskedasticity is true.In otherwords,it assumeswe areableto specifl'::, without preciseknowledgeof may wish to testfor heteroskedasticity . .,2s. In reality we :1,. rhe relevant variables.With this point in mind, econometricianHal White suggested definingthe z's as equalto the x's, the squaresof the x's, and possiblytheir crossproducts. Frequently,the variablesthat affect the varianceare the sameas those in the meanfunction. .\lso, by using a quadraticfunction we can approximatea number of other possiblevariance tunctions. Supposethe mean function has two explanatory variables E(vi):9rl$2xi2*9:x,: The White test without cross-productterms (interactions) specifies
candidatefor a byR2hasachi-
(8.43t
HETEROSKEDASTICITY
z2: x2 z3: x3 z+: xZ zs: x? Including interactions adds one further variable, z6: xzx3.If the mean function contains quadratic terms (e.g., 4 : xl), then some of the z's are redundant and are deleted. The White test is performed as an F-test (see Appendix 88 for details) or using rhe 12 - NxR2 test defined in (8.43). Its test values are routinely calculatedby many cconometric software packages and can often be found by the simple click of a mouse. 8.4.3b Testing the Food Expenditure Exarnple To test for heteroskedasticity in the food expenditure example where the variance is potentially a function of income, we test Hs;a2:0 againstthe alternativeH1:a2f 0 in the variance function o? : h(q + oLzxi).We begin by estimating the function 2f : crI * ct2x;* v; by least squares,from which we obtain
169 SS7: 4,610,749,441SSE: 3,159,556, large samples. or a Breuschnge multiplier that was later heseand other automatically your software : relationships x 88. As you nd that many comesfrom a
t. very general d to carry out ,. One of the itic computed ris of hetero-
r (8.37).
Rz: r-
o'1846 ss"ry4:
end
X 2:
N x R z : 4 0 x 0 . 1 8 4 6: 1 . 3 8
Sincethere is only one parameterin the null hypothesis,the 12-testhas one degreeof treedom.The 5Vocritical value is 3.84.Because7.38 is greaterthan 3.84,we rejectlls and ;onclude that the variance dependson income. For the White version of the test we estimate the equation 2? : ot I ctzxi+ %t + vi end test Hs:a2 : o: : 0 againstH1:u2 f 0 or ct310. In this case,including both the test .rndp-values,we have X 2:
NxRz:40x0.18888 :7.555
p - v a l u e: 0 . 0 2 3
The5Vocritical value is X?o.nr,rl: 5.99.Again, we concludethat heteroskedasticinexists riith the variance dependenton income.
216
HETEROSKEDASTICITY
8,5 Exercises 8.5.1
PnoersN{s
8.1 Show that the varianceof the leastsquaresestimatorgiven in (8.8) simplifiesto that given in (8.6) when o? : o2. That is
I
[('' - r)2"i] :
o'
- *)'= ('' rL' (', r)'1' f>L,
8.2 Considerthe model yi : 9r I lzxr * ei with heteroskedastic variancevar(e;) : 6ri and its transformed homoskedastic version yi : Broit + gzxi 4- ei where yi : niryi, xl : oitxi, ande! : o,tei. The normal equationswhose solution yields the generalizedleast squaresestimatorsB1 and B2 are
(Ioi') 0' + (Io;txi) Bz : 2oi 1vi (Io;rxi)B' + (Ixi'z)8,:Z*ivi (a) Show that B1 and B2 can be written as 2yixi Loi -I"--
9,:
/Iorty'\
/Io;2x;\
\E-l\x;'7
- (Lo;2xi\2 >"7 \ >;a,
Zo;zxl
o,:Y2-(#)u
(b) Show that 91 and B2 are equal to the least squaresestimatorsby and b2 when o? : o2 for all i. That is, the error variancesare constant. (c) Does a comparison of the formulas for p1 and B2 with those for b1 and b2 suggestan interpretationfor 0r anO Bzf
tctnp 8.3 Res
8.3 Considerthe simple regressionmodel Yi:9r *7zxilet where the ei are independenterrors with E(ei) : 0 and var(ei) : ozx?. Supposethat you have the following five observations ! : (4,3,1,0,2) x:
(1,2,1,3,4)
Use a hand calculatorto find generalizedleast squaresestimatesof B1 and 82. 8.-+ A sample of 200 Chicago householdswas taken to investigate how far American householdstend to travel when they take vacation.Measuringdistancein miles per year, the following model was estimated MILES:
(a) Theequ age and (b) Orderin aPPlYin observa
9r * 9IINCOME -l\zAGE + 9qKIDS -r e
The variablesare self-explanatoryexceptperhapsfor AGEthatis the averageage of the adult members of the household. The data are in the file vacation.dat.
Use the fication (c) Table 8 least sq squares (i) Hc kic (ii) Hc en
8,5 EXERCISES
217
2000
1000 mplifies to that
Eo -1000 evar(ei):6)
where
i+
"i ,hose solution
-2000
0
20
40
60
80
100
r20
Income
2000
1000
!
l)0, -1000
and b2when 'or
-2000
20
b1 and b2
30
40
s0
60
Age
i,rcunn 8.3
Supposethar
plotsfor Exercise8.4:vacationdata. Residual
(a) The equationwas estimatedby least squaresand the residualsare ploned against age and income in Figure 8.3. What do thesegraphssuggestto you? (b) Ordering the observations according to descending values of INCOME, and applying least squaresto the first 100 observations,and again to the second 100 observations,yields the sums of squarederrors SSEr: 2.9471x107 SSE2:
1 and B2. a.rAmerican in miles per
erageage of dat.
1.0479x107
Use the Goldfeld-Quandt test to test for heteroskedasticerrors. Include specification of the null and alternative hypotheses. (c) Table 8.1 contains three sets of estimates,those from least squares,those from least squareswith White's standard errors, and those from generalized least squaresunder the assumption o? : o2 xINCOMEZ. (i) How do vacation miles traveled dependon income, age,and the number of kids in the household? (ii) How do White's standarderrors compare with the least squaresstandard errors? Do they change your assessmentof the precision of estimation?
218
HETEROSKEDASTICITY
TabI e 8. 7 Variable
t . t b I e8 . 3 -
Output forExercise 8.4 Coefficient
Std. Enor
r-Value
p-Value
-391.55 14.20 15.74 -81.83
169.78 1.80 3.76 27.13
-2.3r 7.89 4.19 -3.02
,:nable
0.022 0.000 0.000 0.003
Least squaresestimates with White standard errors
-391.55
C INCOME AGE
t4.20 15.74 -81.83
KIDS
142.65 1.94 3.97 29.r5
-425.00 13.95 t6.72 -76.81
;,)OMS
;')oMS^z RI.VE 'Rl.vE^2
.'.tr -2.74 7.32 3.97 -2.81
0.007 0.000 0.000 0.006
;' : 0.08467 (b) Do yt (c) What . 6
Generalized least squaresestimates
C INCOME AGE KIDS
, -:PendentVariab
r_,,!a"9:9f-g!
Least squaresestimates
C INCOME AGE KIDS
121.44 1.48 3.02 2t.85
-3.50 9.42
5.s3 -3.52
0.00r 0.000 0.000 0.001
Continuin following
??:qf
where ?; a 8.5. (a) Discu variar (b) Test f
(iii) Is there evidence to suggest the generalized least squaresestimates are better estimates? 8.5 In Exercise 5.5 an equation used for the valuation of homes in towns surroundin-e Boston was estimated. Reestimating that equation with White's standard errors yields the output in Table 8.2. (a) For the coefficients of CRIME, ROOMS, AGE, and TAX, compare95Voconfrdenceintervals obtainedusing the standarderrors from Exercise 5.5 with those from Table 8.2. Table 8.2
Consider
v
You have
1.1 -0.5
EstimatedMean Function forExercise 8.5
Dependent Variable: VALUE Observations: 506 Heteroskedasticity-ConsistentStandard Errors Variable
I
Coefficient
C CRIME NITOX ROOMS AGE DIST ACCESS TAX PTRATIO
28.407 - 0 . 18 3 -22.811
R 2: 0 . 6 5 7
SSE:14,652.22
6.372 -0.048 - 1.335 0.2'72 -0.013 -1.1't7
J.J
Std. Error 7.380 0.035 4.360 0.665 0.011 0.190 0.075 0.003 0.r24 S S T: 4 2 , 7 1 6 . 2 9
r-Value 3.849 -5.283 -5.232 9.574 -4.433 -7.019 3.644 -4.430 -9.522
Use a hat (a) Find (b) Find (c) Estin (d) Find (e) Find Exerr
p-Value
0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000
..3.2
CovrP
r.g The file CAinl Exercise
8.5EXERCISES
Table p-Value
0.022 0.000 0.000 0.003
0.007 0.000 0.000 0.006
0.001 0.000 0.000 0.00r
8.3
Estimated Variance Function forExercise
219
8.6
DependentVariable: EHAT_SQ Included observations:506 \hriable
Coefficient
C ROOMS
ro1't.037 - 3 0 5 . 3 1l 23.822 2.285 -0.039 -4.419
ROOMS^2 CRIME CRIME^2 DIST
Rr : 0.08467
Std.Error 204.522 63.088 4.844
p-Value
4.92 -4.84
0.000 0.000 0.000 0.067 0.0-11 0.07-l
4.92 1.84 -2.04 -t.79
1 )4)
0.019 2.466
S S d: 5 , 0 3 8 , 4 5 8
t-Value
SST: 5, 504,525
(b) Do you think heteroskedasticityis likely to be a problem? (c) What misleading inferencesare likely if the inconect standarderrors are used? $ . 6 Continuing with the example in Exercise 8.5, Table 8.3 contains output for the following leastsquaresregression ?7 : o, + azROOMSi * qROOM$
+ (\ CRIMEi + asCntUn? + q6DISTi + vi
where 2; are the least squaresresidualsfrom the meanfunction estimatedin Exercise 8.5. (a) Discuss how each of the variables ROOMS, CRIME, and DISZ influences the variance of house values. (b) Test for heteroskedasticity.
3s estimates are vns surrounding standard errors
Consider the model parc 95qo confie 5.5 with those
yi : 9r * gzxi * ei E(e) :0
var(e;):
"!
: exp(azr)
You have the following eight observationson yi, xi, and zi.
1.1 -0.5 3.3
0.3
18.9 J.Z
7.0
-0.9 - 1.8 4.7
6.4 3.4 1.9
1.8 -3.5 6.8
A< 1A !.
-1
-0.: o.+
Use a hand calculatorto: (a) Find least squaresestimatesof B1 and B2. (b) Find the least squaresresiduals. (c) Estimatea. (d) Find varianceestimatescif . (e) Find generalizedleastsquaresestimatesof B1 and 82. (Hint: Use the resultsin Exercise8.2)
p-Value 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000
-0.5 -3
{.5.2 r.8
Conrpursn ExEncrsEs The file stockton96.dalcontains 940 observationson home sales in Stockton. CA in 1996. Thev are a subset of the data in the file stockton.datused tor Exercise7.4.
220
HETEROSKEDASTICITY
(a) Use least squaresto estimatea linear equationthat relateshouseprice PRICE to the size of the housein squarefeet SQFT andthe age of the housein yearsAGE. Comment on the estimates. (b) Supposethat you own two houses.One has 1400squarefeet; the otherhas 1800 squarefeet. Both are 20 years old. What price do you estimate you will get for eachhouse. (c) Use the White test (with cross-productterm included) to test for heteroskedasticity. (d) Estimate cr.rand cr2in the variance function o7 : exp(ctr -t a2SQFT). (e) Using the variance assumption from part (d), find generalized least squares estimatesfor the parametersof the equationestimatedby leastsquaresin part (a1. Comment on the results. (0 Use the results from part (e) to estimate the prices you will get for your two houses. 8.9 (a) Using the estimatesobtainedin part (a) of Exercise8.8 as the parametervalues. and assumingnormally distributed errors, find the probability that (i) your 1400 squarefeet housesells for more than $115,000and (ii) your 1800 squarefeet housesells for less than $110,000. (b) After making the correctioncrr : crr + 1.2'704,use the estimatesobtainedin parts(d) and (e) of Exercise8.8 asthe parametervalues,and assumingnormallr distributed errors, find the probability that (i) your 1400 squarefeet house sell: for more than $l 15,000and (ii) your 1800squarefeet housesellsfor lessthan $ I 10,000. (c) Comment on and compare the answersyou obtained in parts (a) and (b). 8. l0'r' (a) The purposeof this exerciseis to test whether the variancespecificationol : o2x; introduced in Section 8.3.1 has been adequateto eliminate heteroskedasticity in the food expenditureexample in the text. Compute the squaresof the residuals from the transformed model used to obtain the estimates in (8.16). Regressthe squaresof the residualson x; and test for heteroskedasticity. (b) We now ask whether the variancespecificationo? : ozi'! introduced in Section 8.3.2 eliminatesheteroskedasticity. Computethe squaresof the residualsfrom the transformedmodel usedto obtain the estimatesin(8.27). Regressthe squares of the residuals on x; and test for heteroskedasticity. 8. I I
Reconsiderthe householdexpendituremodel that appearsin the text, and the datafor which are in the file food.dat. That is. we have the model yi:Br
*9zxi*ei
wherey; is food expenditurefor the ith householdand x; is income. Find generalized least sq^uares estimatesfor Br-and $r-under the assumqtions 1(a) var\ei) : o't/xi (b) var(ei) : o'4 (c) var(e1): o2ln(xt) Comment on the sensitivity of the estimatesand their standarderrors to the heteroskedasticspecification.For eachcase,use the White NxR2 statisticand the residuals from the transformed model to test to see whether heteroskedasticityhas been eliminated.
l 2 In the fil
domesti hypothe GDP,TI
where
It is sus (a) Wh1 (b) Estn resi< (c) Test (d) Use stan stan conl ignc (e) Ree resu relat
1 - t t ' Conside
Assume time-se
(a) Finc (b) Tes (c) Wht in (l (d) Unc esti
-l':' In the fil (B) for functior
where I indeper (a) Est Do
8.5 EXERCISES
seprice PRICE to rusein yearsAGE. the otherhas 1800 te you will get for
))1
12 ln the file pubexp.dat there are data on public expenditureon education (EE). -eross domesticproduct (GDP), and population(P) for 34 countriesin the year 1980.It is hypothesizedthat per capita expenditureon educationis linearly related to per capita GDP. That is. );:Fr
l9zxi*ei
for heteroskedaswhere a2SQFT). zed least squarei squaresin part (at.
.,:(T) ,,:(T) and
It is suspectedthat"ei may be heteroskedasticwith a variance related to .r,. (a) Why might the suspicionabout heteroskedasticity be reasonable? (b) Estimatethe equationusing leastsquares,plot the leastsquaresfunction and the residuals.Is there any evidenceof heteroskedasticity? (c) Test for the existenceof heteroskedasticity using a White test. (d) UseWhite's formula for leastsquaresvarianceestimatesto find somealternative standarderrors for the least squaresestimatesobtained in part (b). Use these standarderrors and those obtained in part (b) to construct two alternative 95Vo confidenceintervals for B2.What can you say about the confidenceinterval that ignores the heteroskedasticity? (e) Reestimatethe equation under the assumption that var(e;) : o2xi. Report the results. Construct a 95%oconfidence interval for B2. Comment on its width relative to that of the confidence intervals found in pan (d).
get for your t\r'o parametervalues. that (i) your 140[) 1800 squarefeer nates obtained in lsumingnormallr 'e feet housesell: sellsfor less than (a) and (b). recificationof : inate heteroskete the squaresof the estimates in rr heteroskedas-
l 3 ' l Consider the following cost function where C denotescost and Q denotesoutput.
Assumethat var(ey,): o2Qtt. We use a subscriptr becausethe observations are time-series data. They are stored in the file cloth.dat. C r , : B r - t B z Q r ,+ \ z Q i , - t \ q Q l , + e t , (a) Find generalizedleast squaresestimatesof 9r, Bz,83, and Ba. (b) Test the hypothesisBr : F+ : 0. (c) What can you say aboutthe natureof the averagecost function if the hypothesis in (b) is true? (d) Under what assumption about the error term would it be more appropriate to estimate the averagecost function than the total cost function?
,ducedin Section re residuals from gressthe squares :, and the datafor J'r
Find generalized
In the file cloth.dat there ne 28 time-seriesobservationson total cost (C) and output (Q) for two clothing manufacturing firms. It is hypothesizedthat both firms' cost functions are cubic and can be written as:
firm 1: Cu : 9t -t 7zQv+ ?tQi, * \qQi, * eu firm 2: Cz,: 6r * 6zQz,+ 6tQ3,-t E+Q1, + ez, rrsto the heteromd the residuals iticity has been
whereE(e1) : E(ezt): 0, var(err): o?, and var(e2,): 03. AIso. eh arTdezt ate independentof each other and over time. (a) Estimate each function using least squares.Report and comment on the results. Do the estimatedcoefficientshave the expectedsisns?
?,?
APPENDIX 8
HETEROSKEDASTICITY
(b) Using a 107osignificancelevel, testthe hypothesisthat -F1s :o! : 63 againstthr alternativethatH1:oll ol. (c) Estimateboth equations jointly assumingthat Bl :6r, Fz :62, 9: :61, an; 9+ : E+.Report and comment on the results. (d) Test the hypothesis I I s : B 1- 6 t ' F z : 6 2 , 9 : : 6 3 a n d B 4 :
\()te that we areasl ..e wrote the least
' nere
$o
Comment on the test outcome. 8.15* (a) Reconsiderthe wage equationthat was estimatedin Section8.3.3. Insteadt: estimatingthe variancesfrom two separatesubsamples,one for metropolitan an; the other for rural, estimate the two variancesusing the model
. :llsexpresslonrs a ' :reroskedasticity
" rs derived unde - .'reroskedasticit '\ e summarize the
o3:exp(ctr *a2METRoi) and one single combined sample. Are your variance estimatesdifferent fror thoseobtainedusing two separatesub-samples? Why? (b) Find a new set of generalizedleast squaresestimatesfor the mean function an: comparethem with thosein (8.33). (c) Find White standarderrors for the least squaresestimatesof the mean functio: How do they comparewith the generalizedleast squaresstandarderrorsobtaineJ in part (b)? 8.16 Consider the following model used to explain gasoline consumptionper car i:. Germanyand Austria for the period 1960-1978:
. re nextresultis th -:.biased, it is no lo . .: by considering -.rt were best und The final conse -nulas for the lei '-.m (8A.1),
ln(GAS) : 9r * B2ln(INC) * B3ln(pRICE) + B4In(CARS)+ e where 1NC is per capita real income, PRICE is the real gasoline price, and CARSr. the per capita stock of cars. Data on these variables appear in the file gasga.dar. (a) Using separate least squares estimations, estimate the error variance f,.: Germany o2", and the error variance for Austria ol. (b) Test the hypothesisHs:ozc: oi againstthe alternativeH1:o261&o at a S, significancelevel. (c) Find generalizedleast squaresestimatesof the coefficientsB1, Fz, 9:, 9+. (d) Use the resultsin (c) to test the null hypothesisthat demandis price inelasri. (9: > - l) againstthe alternativethat demandis elastic Br < - t.
rhevariancesare nplification is no rhat derived in I
Appendix 8A Properties of the Least Squares Estirnator We are concemed with the properties of the least squaresestimator for B2 in the model Yi:Br l9zxi*et where E ( e ; ): 0
v a r ( e i ): c f i
c o v ( e ; , e ): 0
(i+ i)
, rus, if we usethe l . present,we will t =,'twe shouldbe t rlerval estimatesa . 'tiware for least s ..\.3), unlesstold
APPENDIX 8A
: o? againstth.-
PROPERTIES OF THE LEAST SQUARES ESTIMATOR
223
\ote thatwe areassumingthe existenceof heteroskedasticity. In Appendix2D of Chapter2. ive wrote the least squaresestimator for B2 as
62, Fs : E3,anr:
( 8 A . )1
bz : Bz lZwiei ',\ here
'
8.3.3.Insteado: metropolitananc :l
xi--r
r1t, - 11'
This expressionis a useful one for exploring the propertiesof leastsquaresestimationunder reteroskedasticity.The first property that we establishis that ofunbiasedness.This property '.ras derived under homoskedasticity in equation (2.13). The same proof holds under reteroskedasticitybecausethe only error te(n assumptionthat was used is E(e;) : g. \\-e summarize the results here for comDleteness: E(bz): E(92)+ E(\w;e;) : 9z *ZwiE(e,) : g,
:s different frorr. ean function anc 3 meanfunction d errorsobtainec ption per car ir.
Thenext result is that the least squaresestimatoris no longer best.That is, although it is still ;nbiased,it is no longer the besrlinear unbiasedestimator.We showedthis result in Section ..-3by considering alternative variance specifications,and deriving alternative estimators :hat were best under thesespecifications. The final consequenceof using least squaresunder heteroskedasticityis that the usual :ormulas for the least squaresstandarderors are incorrect. To prove this result we write, : r o m( 8 A . 1 ) , var(b2):
-fe ce, and CARSi. file gasga.dat. cr variance for
L+"i ata 5r, , 9z'9:, 9+. s price inelastie - l.
lstimator 2 in the model
var(Iltie,)
: 2w?var(ei) I V,
i, jcov(e;, e)
:Lwlo!
(8A.2)
r
- ^t2 ll(xr r)'] Ii the variancesare all the same(o? : o'), then the next to last line becomeso?lr'1. this .rmplificationis not possibleunderheteroskedasticity, and sothe resultin (8A.2) is different :o that derivedin Appendix 2E. Specifically,it follows from (8A.2) that ct2 YLfl0'tl#
-
I(xi - l)'
(8A.3)
Thus,if we usethe leastsquaresestimationprocedureand ignore heteroskedasticitywhen it :. present,we will be usingan estimateof (8A.3) to obtainthe standarderror for Dzwhen in :lct we should be using an estimate of (8A.2). Using incorrect standarderrors meansthat :.terval estimatesand hypothesistestswill no longer be valid. Note that standardcomputer ..rtiware for least squaresregressionwill compute the estimatedvariance for b2 basedon iA.3), unlesstold specificallyto computeWhite standarderrors.
224
\PPENDIX 8B
HETEROSKEDASTICITY
This test statisti '.ersionsoccur I
Appendix 88 Variance Function Tests for Heteroskedasticity More insights into variance function tests can be developedby relating them to the F-te-: introducedin equation(6.7) for testing the significanceof a meanfunction. To put that testi: the context of a variance function consider eouation (8.42)
^)
ei :
"'
ctl * dzZiz l
f
(8B.r
ct5.4;5* v;
and assumeour objective is to testHs :cr2 : cr3: . . . : as : 0 againstthe alternativeth.:: at leastone crr,for s : 2,. . ., S, is nonzero.In Section8.4.3we considereda more genera vrriar,ce{ur,ctionthantha-.ir'(8B.1),butwe alsopoirned outthat using thelinear tunction : (8B.1) is valid for testing more general alternative hypotheses. Adapting the F-value reportedin equation(6.7) to test the overall significanceof (88 . we have F_
(ssr-ssE)/(s-r) ssE/(N- s)
( 8 B:
Nr
12
-F I >la? i:ll J
\ote that oj : r ,nd SSE, its est -nexpected.Hi . 'N(0, o3),tl . =nableis 2. Tl
. r n g( 8 B . 6 ) w , :-:.n a critical v
For the seco :LrJassary. Beca :::usch-Pagan -<J as an estin
where SST:
If it is assum .tatistic for the
N
and SSE: L0? t:l
arethe total sum of squaresand sum of squarederrorsfrom estimating(8B . 1). Note that i: :r the mean of the dependentvariable in (8B.1), or, equivalently,the averageof the sQUsr!:r the leastsquaresresidualsfrom the meanfunction.Ata5Vo significancelevel, a valid te.: ,' to reject 110if the F-value is greater than a critical value given by F1o.ls,s-r,n-s;.
-::.
quantity is r::lten as the t( *--:ple size. Su
Two further tests, the original Breusch-Pagantest and the NxR2 version of it can '*: obtainedby modifying (8B.2). Pleasebe patient aswe work through thesemodifications. begin by rewriting (8B.2) as
x2: (s- l)xF: ffi
-xrtr_,r
(8Bl
in larg The chi-squarestatisticX2: (S- 1)xF has an approximateX2{s_t)-distribution samples. That is, multiplying an F-statistic by its numerator degreesof freedom gircr another statistic that follows a chi-squaredistribution. The degreesof freedom of the ci: squaredistribution are S - 1, the sameas that for the numerator of the F-distribution. Tbr background for this result is given in Appendix 6,{. Next. note that
var(e!):;('r)-:
.ssE N-S
( 8 8:
That is, the varianceof the dependentvariable is the sameasthe varianceof the error, which crr be estimatedfrom the sum of squarederrorsin (88.1). Substituting(8B.4) into (88.3) i'ieir A
, - ssr-ssE ---;-, var\ei )
(8B:
. tre R2is the .:rificance lel :r.:sedsthe cril
Softwareoft ; . e l u eo r a y z . endtheir squ r . : t t h ez ' s c h t
\PPENDIX ItB VARIANCE FUNCTION
TESTS FOR HETEROSKEDASTICITY
225
This test statisticrepresentsthe basicform ofthe Breusch-Paganstatistic.Its two different
g them to the F-te. rn. To put that testr:
, ersionsoccur becauseof the alternativeestimatorsusedto replacevarle!). If it is assumedthat e;is norrnallydistributed,it canbe shownthat var(e!) : Zo!, andthe -:.ltisticfor the first version of the Breusch-Pasantest is
ssr - ssE
(88.6)
^^4
/o;
(8B. t the alternativeth. ereda more gener* he linearfunction :' pificanceof(8B l
(88.:
B. 1).Notethatge ofthe squares ' level, a valid tesr . 9 5S , - 1 ,N - S ) .
rersionof it can i' :modifications.\\
\ote that o! : (o2")2is the squareof the error variancefrom the meanfunction;unlike SSZ .rd SSE,its estimatecomes from estimating(8.36). The result var(e!): 2oj might be -rexpected. Here is a little proof so that you know where it comes from. When -1/(0, ol), then (r,lo") -N(0, 1), and (r?1"?) -X7,r.The varianceof a 1f,, random .,riableis 2. Thus,
rl\
/ v a r [ ; I :2 \oa /
l^
=>lvare!)
2oj
.ing (8B.6), we reject a null hypothesisof homoskedasticitywhen the 1r-value is greater .rna critical value from the 12u 1, distribution. For the secondversion of (88.5) the assumptionof normally distributederrors is not -',-essary. Becausethis assumptionis not used,it is often called the robustversionof the ,:eusch-Pagantest.The samplevarianceofthe squaredleastsquaresresiduals.the ef, is .:d as an estimatorfor var(e!). Specifically,we set
/ )\ vat\ei ) :
-''l':Y *,i'f'r
(8B.7)
:rs quantity is an estimatorfor var(ef) under the assumptionIls is true. It can also be rtten as the total sum of squaresfrom estimating the variance function divided by the :iip1esize. Substituting(8B.7) into (8B.5) yields X2:
(88 '
- 2 -> varleit
ssz- ssE ssz/N
:N^(t-ssE)
\
(88.8)
ssr/
: NxR/ istribution in lar-i; of freedom gir.' ieedom of the ch. F-distribution.Th-
(8s: the error,which c.,r ) into (88.3)yield (88.-<
rere R2is the R2 goodness-of-fitstatisticfrom estimatingthe variancefunction. At a 5Eo :nificance level, a null hypothesisof homoskedasticityis rejected when 12 :NxRz , e e d st h e c r i t i c a lv a l u ex 2 * n rr - , , . Softwareoften reportsthe outcomeof the White test describedin Section8.4.3aas an ,,rlue or a 12-value.The F-value is from the statisticin (88.4). with the z's chosenas the . lnd theirsquares andpossiblycrossproducts.The 1l-r'alueis tiom thestatisticin (8B.8), :rh the z's chosenas the x's and their squaresand possibly'crossproducts.
Chapte, 9
9.7 lntrod
Dynarnic Models, Autocorrelation, and Forecasting
Learning Objectives Based on the material in this chapter, you should be able to 1. Explain why lags are important in models that use time-seriesdata and the q ar . in which lags can be included in dynamic econometric models. 2. Describethe propertiesof an AR(l) enor. 3. Compute nonlinear least squaresestimatesfor a model with an AR(l) enor. 4. Use a conelogram of residuals to test for autocorrelation. 5. Use a Lagrange multiplier test to test for autocorrelation. 6. Estimate an autoregressivemodel and use it to obtain forecasts. 7. Specify, estimate, and interpret the estimates from a finite distributed la: model.
'A'hen modeling :.rllectedhas an -: particular, it ir -'.'onomicunits r 'r oneparticula oservations are -:.rss-sectionda! i. a particularva -,.usehold,for er . res'househo 'rner than anoth With time-se iJered accordin -:il be correlate . :,ir is likely to b ,-..iddition, relat *. erplanatoryvi . ,'ccuned.The .iables can las : .posableincon : .uppliers,whi, -:,: input supplie . :ese effectsdo :.-':'rods. As shora :,:re effectson t Giventhatthe rodel the dyn, : Joing so. One :4.1 valuesof ar
8. Compute and explain delay and interim multipliers and the total multiplier. 9. Specify and estimate autoregressivedistributed lag models. 10. Find distributed lag weights from an estimated ARDL model.
;: aresaying,fr . -::ent interestrl -: und slightly, -:tmployment r -.nse to work
Keywords autocorrelation autoregressivedistributed lag models autoregressiveerror autoregressivemodel correlogram delay multiplier distributed lag weight
226
dynamic models finite distributed lag forecast error forecasting HAC standarderrors impact multiplier infinite distributed lag interim multiplier
lag length lagged dependentvariable LM test" nonlinear least squares sample autocorrelation functio: standarderror of forecast error total multiplier 7xR2 form of LM test
['l'
ffi i'RE
9.1 T
9.1 INTRODUCTION
227
9.1 Introduction
data and the ur t.
n AR(l) error.
S.
te distributed l.
. ren modeling relationshipsbetween variables,the nature of the data that have been lectedhas an important bearing on the appropriatechoice of an econometricrrrodel. data (dataon a numberoi :.rrticular,it is importantto distinguishbetweencross-section , ,nomicunits at a parlicularpoint in time) and time-seriesdata(datacollectedover time ' ,)neparticulareconomicunit). Examplesof economicunits from which we often obtain -..'rvationsareindividuals,households, firms,geographical regions,andcountries.Becaus.- .\-sectiondataareoftengenerated by way of a randomsample,cross-section obsenation. .rparticularvariablearetypicallyunconelated.The levelof incomeobsen'edin the Smith'. -.ehold,for example,doesnot impacton, or is not affectedby, the level of incomein the ' ;'.' household.Thereis no particularorderingof the observations that is more naturalor .,::L'rthan another. \\-ith time-seriesdataon the other hand,thereis a naturalordering:the observation:are '-:red accordingto time.Moreover,with time-series datait is likell' thatthe obserration' . be correlatedovertime.The levelof incomeobservedin the Smith'shouseholdin one :.r is likely to be relatedto the level of incomein the Smith'shouseholdin the 1'earbefbre. ,,ddition,relationshipsbetweenvariablescan becomedynamic.A changein the level of :rplanatory variablemay havebehavioralimplicationsbeyondthe time periodin which ;curred. The consequences of economicdecisionsthat result in changesin economic ::.rblescan last a long time. When the income tax rate is increased,consumershaveless -r\)sableincome,reducingtheir expenditureson goodsandservices,which reducesprofits .uppliers,which reducesthe demandfor productiveinputs,which reducesthe profits of - rnputsuppliers,and so on. The effect of the tax increaseripples throughthe economy. :'\e effectsdo not occur instantaneously but are spread,or distributed,over future time , ::r)ds.As shownin Figure 9. l, economicactionsor decisionstakenat one point in time I .c effectson the economy at time /, but also at times t + l,t * 2, and so on. Giventhattheeffectsofchangesin variablesarenot all instantaneous, we needto askhow ':rodelthe dynamicnatureof relationships.We beginby recognizingthreedifferentways Joing so. One way is to specify that a dependentvariableyr is a function of currentand -l valuesof an explanatoryvariable.That is,
.al multiplier. j,:
( 9 . 1)
f(xt,x, t,-tr:....)
-' .rresaying,for example,that the currentrate of unemploymentdependsnot just on the :ent interestrate,but alsoon theratesin previoustime periods.Turnin_e thi. interpretation :rnd slightly,it meansthat a changein the interestrate now will hare ln irrrpacton -'nrploymentnow and in future periods;it takes time for the effect of un interest rate .,n9!to work its way through the economy.
Economic action at time I
+
ent variable squares relation functior of forecasterror LM test
Effect at time r
, L'ne 9 .1
Effect at time r+l
The disrributedlas effect.
Effect at time r+2
228
DYNAMIC
MODELS, AUTOCORRELATION,
AND
FORECASTING
A secondway of capturing the dynamic characteristicsof time-seriesdata is to specify a model with a lagged dependentvariable as one of the explanatory variables.For example.
Y, : f (Y,-r,xt)
(9.1r
In this casewe are saying that the unemployment rate in one period will depend (among other things) on what it was in the previousperiod. Assuming a positive relationship,period< of high unemploymentwill tend to follow periodsof high unemploymentandperiods of lou unemployment will tend to follow periods of low unemployment. A third way of modeling the continuing impact of changeover severalperiods is via tk error term. For examole. we can write yv: f(x,)re1
et:
f(et-t)
(9.3'
where the function et : f (et-t) is used to denote the dependenceof the error on its value in the previous period. Because(9.3) implies et+t:f(et), the dynamic nature of thii relationship is suchthat the impact ofany unpredictableshock that feedsinto the error terrn will be felt notjust in period r, but also in future periods.The current error e, affects notjus the current value of the dependentvariabley,, but also its future values!t+rt!t+2,. . . . As an example,supposethat a terrorist act createsfear of an oil shortage,which then drives up tlr price ofoil. The terrorist act is an unpredictableshockthat forms part ofthe error term er.It is likely to affect the price of oil in the future as well as during the current period. In this chapter we consider these three ways in which dynamics can enter a regression relationship-lagged values of the explanatory variable, lagged values of the dependent variable, and lagged values of the error term. What we discover is that these three war s are not as distinct as one might at first think. Including a lagged dependentvariable can capturesimilar effects to thoseobtainedby including a lagged error, or a long history of pas values of an explanatory variable. Thus, we not only consider the three kinds of dynamic relationships,but also explore the relationships between them. Relatedto the idea of modeling dynamic relationshipsbetweentime-seriesvariablesis tlr important conceptof forecasting.We arenot only interestedintracingthe impact of a change in an explanatoryvariable or an error shock through time. Forecastingfuture valuesofeco nomic time series,such as the inflation rate, unemployment,or the exchangerate, is something that attractsthe attentionofbusiness, governments,and the generalpublic. Describing how dynamic models can be usedfor forecastingis anotherobjective of this chapter. An assumption that we maintain throughout the chapter is that the variables in our equationsarestationary.This assumptionwill take on more meaningin Chapter 12 when it is relaxed. For the moment we simply illustrate the nature of stationary and nonstationan variableswith somegraphs.Figure 9.2 containsgraphsof the observationson three differenr variables, plotted against the observation number. Becausethe observationsare ordered according to time, the graphsshow how the value ofeach variable changesover time. Plot. of this kind are routinely consideredwhen examining time-seriesvariables.The variabler that appearsin Figure 9.2a is consideredstationary becauseit tends to fluctuate around a constantmean without wandering or trending. On the other hand, !2 ard y3 that appearin Figure 9.2b and 9.2c possesscharacteristicsof nonstationary variables. In Figure 9.2b r; tendsto wanderor is "slow turning," while y3 in Figure 9.2c is trending.Theseconceptswill be defined more precisely in Chapter 12. At the present time the important thing to remember is that this chapter is concerned with modeling and estimating dynami.^ relationships between stationary variables whose time serieshave similar characteristic: to those of y1. That is, they neither "wander" nor "trend."
. : r ; t ' r - x9 . 2 ( a . eriablethat is "s '-t "trends".
CASTING
9.l INTRODUCTION
data is to specifl ' bles. For example.
-. tl
lR
I6
(e: ill depend(amoni ,lationship,period. andperiodsofloro
14 I2 10 8
Llperiodsis via th.'
6
/
(e.-:
z
150
) error on its valuc nic nature of thi. into the error tern: r e, affectsnotju:: tltlt+zt'...Asa:" then drives up thr' e errorterm er.It i. nt period. enter a regresslo: of the dependen: . thesethree war. Ldentvariable ca-' rng historyofpa.: kinds of dynami.
t \a)
Y2 20 15 l0 5 0
_10
ies variablesis th.. mpact of a chan-si ure valuesofeconge rate,is somerublic. Describing [is chapter. , variablesin our apter12whenit i. Ind nonstationar-\ on threedifferenr tions are orderec s over time. Plot. s. The variabler luctuate around a y3 that appearin ln Figure 9.2b v1 Lese conceptswili rportant thing to mating dynamic ar characteristic:
s0
100
150
2m
250
300
t (b)
,,^ 15n 300 250 200 150 100 50 0
r00
150 (c)
icuns 9,2 (a) Time seriescf a stationary variable; (b) time seriesof a nonstatlonar\' rariable that is "slow turning" or "wandering"; (c) time seriesof a nonstationary variable :aat "trends".
229
230
DYNAMIC
MODELS, AUTOCORRELATION,
AND FORECASTING
9.2 Lags in the Error Terrn: Autocorrelation
re related to perce \ model that capt
The first dynamic model that we consideris one with a lag in the error term. This model is a convenient one to introduce first becauseit leads naturally into other models that involve lags in the dependentvariable y and lags in one or more explanatory variables, say x. We introduce a direct way of modeling impacts of the error term that last beyond the current period. We ask the question: Is there a way of including lags in the error term to recognize that any unexpectedshockfelt through the error term will take time to work its way through the economy or the industry describedby the equation?Given that a shockdoestake several periods to work through the system,if we consider an error term in any one period, it will contain not only the effect of a current shock,but also the caffyover from previous shocks. This carryover will be related to, or correlated with, the earlier shocks. When circumstancessuch as these lead to error terms that are correlated, we say that autocorrelation exists. To make theseconceptsmore concrete,we introduce them in the context of an area responsemodel for sugarcane.
9.2.L
Anra RrspoNsn Moopr ron SucRnceNs
One way of modeling supply responsefor an agricultural crop is to specify a model in whicl: areaplanted (acres)dependson price. When the price of the crop's output is high, farmer. plant more of that crop than when its price is low. LettingA denoteareaplanted,andP denore output price, and assuminga log-log (constantelasticity) functional form, an arearesponse model of this type can be written as ln(A) : Fr * Fz ln(P) We usethis model to explain the areaof sugarcaneplantedin aregion of the SoutheastAsiar country of Bangladesh. Information on the area elasticity 9z is useftrl for governmen: planning. It is important to know whether existing sugar processingmills are likely to b< able to handle predicted output, whether there is likely to be excessmilling capacity, anc whether a pricing policy linking production, processing,and consumption is desirable. Data comprising 34 annualobservationson areaandprice aregiven in the file bangla.da: To convert our economic model into an econometricone, we use the subscriptr to describe area and price in year t, and add a random error term e, to give ln(A,) : gr f B2rn(P1)* e1
i.2.2
Fnsr-On
.i: beginby summ 'r: are considerin
i:iv oneexplanato -.- liameworkof th ::i_ehtforward.Th -.metimes called r
(e.r --..'
In line with our earlier notation, we can write this equation as
/r: Fr*9zx,le,
.i here p (rho) is a :rndom error term. :eriod r is compos :reviousperiod,du - determining the r .'--onomicvariable :'riernment policir :snouncementOf i :-.r)delassertsthat :rriod. The param - re larger the mag --:.rreslowly the sh Given a particuli . -toregressivemoc :.
(qi
where y, : ln(A,) andx, : ln(P1). Supposenow that we wish to investigatehow to include lags in the error term where c d e p e n d s o n o n e o r m o r e oi tf s p a s t v a l u e s e , - 1!t, - 2 , . . . . I t i s l i k e l y t h a t f a r m e r s ' d e c i s i o n . about the areaof sugarcaneplantedwill dependon their perceptionsaboutfuture prices, anc about government policies on prices and the establishment of processing mills. Since variablesfor theseperceptionsare not explicitly included in the model, their effect on area plantedwill be felt through the error term e1.Also, if perceptionschangeslowly over time. o: at least not in a completely random manner,the current perceptionsthat form part of e, wii-
random errors .':ough 7. That is, I :-ranceol (seeas
E
* -' alsoassume thz
--:. assumpt'on'rT ' n period to peri
STING
9.2 LAGS IN THE ERROR TERM:
AUTOCORRELATION
231
:e relatedto perceptionsheld in the previous period that form part of the lagged !rror !1-r. \ model that capturesthese effects is This model is : els that involre ,bles,say l. \tr'e ond the curreni m to recognrze its way through oestake severa. e period,it uil. reviousshocks. When circumutocorrelation ntextof an are;
modelin whicl: s high, farmer. d, andPdenot.'
et:pet-t+vt
t9.6r
;here p (rho) is a parameterthat describesthe dependenceof !7 otr !;1 and v, is a neu' :"ndomerrorterm.The rationalefor (9.6)is a simpleone.The randomcomponente, in time :criod / is composedof two parts: (i) pel-1 is a cruryoverfrom the random error in the r:eviousperiod,dueto the inertiain economicsystems,with the magnitudeof the parameter - determining the extent of the carryover, and (ii) vr is a "new" shock to the level of the .'lonomic variable. In our example, the "carryover" might be farmers' perceptions of : rvernmentpolicies on pricing and the establishmentof mills. A new shockcould be the ::roouflcerrrorrtof a new policy or information on sugarcaneshortagesor excesses.This :-.odel assertsthat shocks to an economic variable do not work themselves out in one -riod. The parameterp in (9.6) determineshow quickly the effect of a shock dissipates. lne larger the magnitude of p the greaterthe carryover from one period to anotherand the :'.ore slowly the shock dissipates. Given a particular setof assumptionsaboutyr, the model in (9.6) is known as a first-order :jtoregressive model or, more simply, an AR(1) model. It is called "first-order" because -:ereis a one-periodlag on the right side of the equation.The existenceof AR(1) errorshas rplications for estimationof the unknown parametersin our regressionequation,B1and B2. 3etore tuming to those implications, we needto investigatethe propertiesof AR(1) errors.
1&fe0 f9SpOnsr'
9.2.2
outheastAsiar: f,r govemmen: rre likely to be ; capacity,anc is desirable. llebangla.dat pt / to describe
Frnsr-OnosR AuroREGRESsrvE Enrons
'A'e begin by summarizingthe model and statingthe new assumptionsaboutthe AR( 1) enor. ',\'e are considerins the regressionmodel )r : Fr *9zxt I et
)nly one explanatoryvariable is included to keepthe discussionrelatively simple and to use ':e framework of the sugarcaneexample.Extensionto more than one explanatoryvariable is .:aightforward.The erroret is assumedto follow a first-orderautoregressive AR(1) model .ometimescalled an AR(l) process)
(9.-1
(95r
!7 : p !7 - 1 * 1 1
( 9 . 8)
lhe random effors v, are assumedto have the properties assumedabout e, in Chapters 2 ':rough 7.That is, the v, are uncorrelatedrandomvariableswith mean zero and a constani :nance ol lsee assumptionsMR2, MR3, and MR4 statedin Section5.1):
E(v,): g, term where e. rers' decisions ure prices, and g mills. Since effect on area y overtime,or Lpart of e, will
(e.7)
var(v,): oi,
cov(v,,u") : 0 fort f s
t 9 . 9r
'.\'e value.Thatis, alsoassume that p is lessthanI in absolute -1
t9.l0t
^his assumptionimplies that the e, are stationary and have properties that do not chanse --:omperiod to period.
232
DYNAMIC
AND FORECASTING
MODELS, AUTOCORRELATION,
What arethe implicationsof theAR(l) error modeldefinedby (9.8),(9.9),and (9.10)fo: the properties (mean, variance, and correlations) of the enors e1?This question is ar: importantone because,as we saw in Chapters2,5 and8, the propertiesof the leastsquare. estimator for B1 and B2 dependon the properties of the er. We need to assesswhether the leastsquaresestimatoris still the best,linear unbiasedestimator.It can be shownthat tht mean of er is E(e,) : g
( 9 . 1I
That is, when the equationerrorsfollow an AR( I ) model, they continue to have a zero mear: For the variance of the e,. it can be shown that
/\
o?
,
var\et)-oi:
(9.1:
_ t-p'
This equation describesthe relationship between the variance oj of the original eQuatio: erroret and the varianceol of the uncorrelatedhomoskedasticerror v,. Becauseol doesno: change over time, the error e, is also homoskedastic.Where the properties of the er difte: from those assumedin Chapters 2 through 7, is in the covariance between the errorcorrespondingto different observations.Since we are using time-seriesdata, when we sa. "the covariancebetweenerrorscorrespondingto different observations,"we arereferring t, the covariancebetween errors for different time periods. This covariancewill be nonzer, becauseof the existence of a lagged relationship between the enors from different tirnc periods. Specifically, it can be shown that the covariancebetween any two errors that are . periods apart is ( 9 .t :
c o v ( e , , e , - r :) o ? p k k > O
In Chapters2 through 7, we assumedthis covariancewas zero. So, we will needto examinr the implicationsof (9.13)for estimation.However,beforedoing so,it is usefulto get a bette: understandingof (9. l3) and what it meansfor the sugarcaneexample.Note that the symb. . ft is used to represent the time between errors. The expression in (9.13) says that th< covariancebetweentwo errorsthat areft periodsapartdependson the varianceol. and on thc parameter p raised to the power k. To describethe correlation implied by the covariancein (9.13),we use the correlatio: formula (B.20) from Appendix 8.4.3. Writing this formula in terms of our quantitiest': interestyields
corr(er, !t-k) :
enor modelsthat k :o (9.14)is oneof t :'eenthe most com .:sedto describea From (9.14),wt lecome further aP rpart, and so on, v
/\)L
cov\e|,et-k)
COV\et) eFk) :--
,|
vat\er)
JiiGJii@i
1
oa
. :reresultsindicat .-..rtthe elasticity :.iuationappearir From both the ':.idualsto follow ! 'r example,thet ' .lowedby a long i. somemore of -..itive correlati
Ir
.tble9.7
oAp' :
iince-l
-
y
k
(9.r-:
::........'......: :te
et
-0.303 An interpretationor definition of the unknown parameterp can be obtainedby settingk : Specifically, C O I T ( e / ,e r t )
:
P
rq t<
Thus, p representsthe correlation between two errors that are one period apart; it :. sometimescalled the autocorrelation coefficient. Also. when the errors in different timc periods are correlated,we say that autocorrelation exists. There are a variety ofdynamr.
0.254 0.18 0.503 0.275 -0.115 -0.43'.7 -0.423 -0.36'l
]ASTING
233
9.2 LAGS IN THE ERROR TERM: AUTOCORRELATION
).9),and(9.10)fi.: ris question is a: rf the leastSQUarc-. rssesswhetherrh: be shown that thr
-'normodelsthat leadto the existenceof autocorrelation.The AR(l) error model that leads : ' (9.14)is oneof thosemodels.However.because the AR(1) errormodelhas,historicallr. "centhe most commonly usedone,the more generalterm "autocorrelation" is sometimes ..ed to describea regressionmodel with AR(l) enors. From (9.14),we can also considerthe sequenceofcorrelations betweenerrors as the\ -
( 9 . 1; p.p2.p3....
havea zeromea:
(e.1: original equati(' ,causeol doesn, ' ies of the e, difl'r: )tween the ent :. lata,when we :;. we arereferrin-e: e will be nonze:rm different tin.: o errors that are .
i:nce -l < p < l, the values in this sequenceare declining. The greatestcorrelation -ctweenerrorsis forthose that areone period apart;as the errorsbecomefurther apart.the - ,nelation betweenthem becomessmaller and smaller,and eventually ne,eligible.This -raracteristicof an AR(l) error model is one that seemsreasonablefor manl econonric ^:renomena. Let us investigatethe errors in the sugarcanemodel to see if they might exhibit -tocorrelation. Becausethe e1arenot observable.we cannotinvestigatedirectly whether ':.e\ arecorrelated.However,we can use(9.7) to find leastsquaresestimatesb1 andb2 and - b1 - b2x,. Becausethe ?, will have .1'ncompute the least squaresresiduals2t:lt -raracteristicssimilar to those of the errors et, autocoffelationcan be investigatedby - *;rminingthe characteristicsof the 2,. Application of least squaresto the arearesponse -rrrflslir (9.4) yields the following estimatedequation
j,t :3.893 10.776x, (se) (0.061)(0.277)
(e.16)
( 9 . 1. I needto examineful to get a betr:: te that the symb, 13) saysthat rhnceof ,andon thr se the correlatit,: our quantities, :
. :reresultsindicate that both coefficients are significantly different from zero, and suggest :.it the elasticity of arearesponseto price is approximately0.8. The residualsfrom this iiuation appearin Table 9.1 and are plotted againsttime in Figure 9.3. From both the table and the figure we can see that there is a tendency for negative --'.idualsto follow negativeresidualsand for positiveresidualsto follow positiveresiduals. : ,r example,there is a run of positive residualsfrom observation2 to observation5, ' llowedby a long run of negativeresidualsfrom observation6 to observationI 1. Can you ..i. somemore of these"runs" ? This kind of behavioris consistentwith an assumptionof -,,.itive correlationbetweensuccessiveresiduals.With uncorrelatederrors,we would not
. thle 9.7 = pt
Least SquaresResidualsfot the SugarcaneExample
( 9 .t ;
Time
:te
-0.303 bysettingk: .
(9.1-' :riod apart; it :. in different tinr. .rietyof dynamr.
0.254 0 . 18 2 0.503 0.275 -0.r15 -0.437 -0.423 -0.367
10 1l t2 l3 l4 l5 16 I7 18
-0.254 -0.145 0.091 0.304 0.6s6 0.134 -0.059 0.435 -0.106
Time
at
19 20 2l 22
-0.036
L) 1A
25 26
0 . 3 6r -0. I38 0.017 0.336 -0.I75 -0.517 -0.t37
Time
27 28 2.9 30 31 3l 33 -t+
el
-0.651 - 0 . 2 18 0.137 0.121 0.0J0 0.0-18 0.lEl 0.t81
234
DYNAMIC
MODELS, AUTOCORRELATION,
AND FORECASTING
0.8
9.J Estimr
0.6
':r en thatan AF
0.4
,,delthe dynan ::reeprocedure .,tion,and leas
o.2 !
0.0 ,o.2
,.i.1
Lnasr I
*0.4
,,:en the errors lR-l,cov(e,,e.) :i u ay, without .-r'ntiallythe si
-0.6 -0.8 51015202530 Observation residuals plottedagainsttime. FTGURE 9. 3 Leastsquares
expectto seeany particularpattern.If the errorsare negativelyautocorrelated,we wouiexpectthe residualsto show a tendencyto oscillatein sign. The magnitudeof the correlationbetweenresidualsthat areone periodapartis given t . the sample correlation between 2, and 2r-1. Recall from Section 4.2 that the sampi. correlationbetweentwo variablesx, andy, is given by
,]]
cov\hlyt )
LT:r @,*r ) ( y,- y) t. l Ls rr - .t t\ "- r r- ^r r)l sLrr - t. /\ ,l ,t - -l l' 2 Y
(9.1-
To obtainan expressionfor thecorrelationbetween2t and2t-r, we canreplace xyby?1and . by ?, t.Before doing so, it is convenientto note two simplificationsthat can be usec In (9.17),x andy areestimatesof thepopulationmeansE(x) andE(y). We areassumingth": all errorshavezero means.That is, E(e,) : E(r,-r) : 0. Thus,when replacing x,by e, ani yt by et-r we can replacex and y by zero. The secondsimplification comes from th. assumptionthat the varianceof e, is constantover time, and hence var(e1): var(e, 1 This resultmeansthat only one term is requiredin the denominator.The samplecorrelatic.: between?1and?1-1is then givenby
. -' -
*,Gi;J ,*l
b,-t Ll]:zo, L!-ral-'
I . The least It is poss variancer to its tru( probabili l.
The formt are no lor thesestar
. :iroughthe us :rpute corTec -elated.These ;r.istent)stand . :Jasticitycons ,.:rc consisten ,::ification of t .::Jnce. To illustrate I :.ect and inc( .:n1ple.The t\
(9.1E
We haveusedthe subscript"1" on rbecausewe are consideringerrorsthat are one perioc apart.The summationsstart from t : 2 to accommodatethe laggedresidual2,-1. In some books and some software you will find slight variationsof (9.18) that girr siightly differentnumericalvalues.For example,sometimesthe denominatoris written r.
:. that the co ,rcorrelation Itidenceinter rind the follc
Zl.1 ,i to includeall observations. Applying (9.18) to the residuals from the sugarcaneresponseequation yields r correlationof 11: 0.404, suggestingthe residualsare moderatelycorrelated;although:, formal hypothesistest is neededto assesswhether this value is sufficiently large to bc significantlydifferentfrom zero.We considersuchtestslaterin this chapter.Our next taski. to suggesthow to estimatea model with autocorrelatederrors.
.\e are unawa . between0.2
9.3 ESTIMATING
STING
AN AR(1) ERROR
MODEL
235
9.3 Estirnating an AR(1) Error Model t.iiren that an AR(l) enor model is being usedto captureautocorrelationin the errors and to :odel the dynamic effects of the error term, we need to ask how to estimate such a model. lfuee proceduresare considered-least squaresestimation, nonlinear least squaresesti:ration, and least squaresestimation of a more general model.
9.3.1
Lresr Squenrs EsrruerroN
-A-hen the errors follow an AR(l) model e, : pet_r I vt, the least squaresassumption \1R4,cov(er,e,):}for t f s, is violated.Supposewe proceedwith leastsquaresestimation x\way, without recognizing the AR(l) error. What are the consequences?They are "-.sentiallythe same as ignoring heteroskedasticityshould it exist.
ted, we would rartis given br at the sample
(9.11'
>xlby 21andt. . can be used. assumingthat ngxlby e, and mes from the ) : v a r ( e rr ' . rle correlation
( 9 . 1 8t
l . The least squaresestimatoris still a linear unbiasedestimator,but it is no longer best. It is possible to find an alternative estimator with a lower variance.Having a lower variancemeansthere is a higher probability of obtaining a coefficient estimateclose to its true value. It also means hypothesis tests have greater power and a lower probability of a Type II error. The formulas for the standarderrorsusually computedfor the least squaresestimator are no longer correct, and hence confidenceintervals and hypothesis tests that use these standarderrors mav be misleadine. \lthough the usual least squares standard errors are not the correct ones, it is possible to :"rmpute correct standard errors for the least squares estimator when the errors are auto-.r.related. These standarderrors are known as IIAC (heteroskedasticityand autocorrelation :.rnsistent) standardelrors, or Newey-West standarderrors, and are analogousto the hetero-iedasticiry consistent standarderrors introduced in Chapter 8. They have the advantageof ting consistent for autocorrelatederrors that are not necessarily AR(l) and do not require .recification of the dynamic elror model that is needed to get an estimator with a lower r:rriance. To illustrate how confidence intervals can be misleading, it is instructive to compute :.rrrect and incorrect standard errors for the least squares estimator of the sugarcane .-rample.The two sets of standarderrors, along with the estimated equation are h :3.893 -t 0.776x, (0.061) (0.277) (0.062) (0.378)
incorrectse's correctse's
re one period tl ?;1. .18) that give r is written as
\ote that the correct standarderrors are larger than the incorrect ones. If we ignored the rutocorrelation,we would tend to overstatethe reliability of the leastsquaresestimates.The ,-onfidence intervalswould be narrowerthanthey shouldbe.For example,using/. : 2.037, ae find the following 95Vo confidenceintervals for B2:
fion yields a rd; although a y large to be turnext task is
(0.211,1.340) (inconect) (0.006,1.546) (correct) Ii rve are unaware of the autocorrelation, we estimate that the elasticity of area response .resbetween0.211 and 1.340.In reality, the reliability of least squaresestimationis such
236
DYNAMIC
MODELS, AUTOCORRELATION,
AND FORECASTING
that the interval estimate should be from 0.006 to 1.546.Although autocorrelationcar: lead to either overstatementor understatementof the reliability of the least square. estimates,overstatementof reliability, as illustratedin this example,is the more commor occurTence.
9.3.2
NoNrrNraR LEasr Squerus Esurr.ranroN
While it is possibleto computecorrectstandarderrorsfor the leastsquaresestimatorwhe: the errors follow an AR(l) error model, it is preferableto employ a better estimatio,procedurethat gives us narrower, more informative confidence intervals than the corre.r onesfrom least squares. To developsuch an estimator,we return to the model in (9.7) )r:9r
(9.|e
*?zxtlet
and the AR(l) enor representation !7 : p !1 - 1
Jsativeprodu -.ir.although .-rctionof thr ^tainedby us : r , d e r nc o m p ::thodsusea .ril thosetha :3 not compu -:'\'are Calle : usualdesir :trvare. The second .rfendent var , rlanatoryva Returningtr . _ .a n d p y i e l
/q 1{
*v1
Substituting(9.20)into (9.19)yields )r : 9r *Fzx,
t
pe1-1 t
v1
/q 1t
Now note that (9.19) holds for every single observation. In Darticular. in terms of th. previousperiod we can write !t - l
:
lt-t
-
9r - 9zxr-t
/O a'
Multiplying (9.22) by p yields P!t-r : P!t-t - PFr - PFz-r,-r
r'q ai
Substituting(9.23)into (9.21)yields * v1 )r : Pr (1 - P) I }zx, f P),-r - PB2x1-1
lq 1j
What have we done? We have transformedthe original model in (9.19) with th. autocorrelatederror term et into a new model given by (9.29 that has an error ternr', that is uncorrelatedover time. The advantageof doing so is that we can now proceed. to find estimatesfor (81, B2,p) that minimize the sum of squaresof uncorrelatederror. S,:Ll:zvT.Minimizing the sum of squaresof the correlated errors ,S":LT:rr' yields the least squaresestimator that is not the best, and whose standard errors &rr not correct.However,minimizing the sum of squaresof uncorrelatederrorsS, yield. an estimator that is best and whose standarderrors are correct (in large samples).Not.. that this result is in line with earlier practicein the book. The least squaresestimaroused in Chapters2 through 7 minimizes a sum of squaresof uncorrelatederrors. There are,however,two importantdistinctivefeaturesabout the transformedmodel ir. (9.24).To appreciatethe first, note that the coefficientof x1_1is equal to -ppz that is rht
.eseestimat .-:eesta sligt -.-: not exact - -':rieen least li9 is smal : - 8 1 ,s u g g e -:ter, somet ::h a lower ' : m s e l v e se s -sT:34n '-':rerestima
.i.la Gene - Chapter8 v . . : n ga n e s t i m r\ to obtaing . rew uncolT ::nsformed rr ..r>ts. Indeed using an it : ,cedure.De -1.3
Esrtv
: resultsfor -rrr model w
ECASTING
9.3 ESTIMATING
autocorrelation car f the least squarei s the more commor
res estimator wher a better estimatioc als than the correcr
(9.re
ln(A,) : 3.899+ 0.888In(P1) et :0.422et-t * vt
('")
ar, in terms of the
(9'2..
(9.23
(9.21
n (9.19) with the has an error term ) can now proceed ncorrelated erron
rrorsS, :2T=r", :andard errors are :d errors Suyields ge samples).Nore squaresestimator 'elated errors. rsformedmodel in o -ppz that is the
MODEL
rgative product of p (the coefficient of y'-1) ild Fz (the coefficient of x,). This fact means tat. although(9.24) is a linear function of the variabl!s.r1,)1_1,and4_1, it is not a linear functionof the parameters(B1, B2, p). The usual linear least squaresformulas cannot be ..buinedby using calculusto find the valuesof (Fr, B2,p) that minimize S, Nevertheless. arodern computer software can be used to find the estimates numerically. Numerical athods use a systematicprocedure for trying a sequenceof alternative parametervalues rntil those that minimize the sum of squaresfunction are found. Becausetheseestimates rre not computed from a linear formula, but they still minimize a sum of squaresfunction. tbey are called nonlinear least squares estimates. Estimates obtained in this way have rie usualdesirableproperties in large samplesand are computed routinely by econometric oftware. The seconddistinguishing feature about the model in (9.24) is that it containsthe lagged dcpendent variable )r-r, aS well as x, and x1-1, the current and lagged values of the arplanatory variable. For this reason the summation Su : ZT:zr7 begins at t : 2. Returningto the sugarcaneexample and applying nonlinear least squaresto estimatePl, 3:. and p yields the following estimated equation
(9.20
(9.2r
AN AR(1) ERROR
(0.0e2)(0.2se)
(0.166)
(e.2s)
Theseestimatesare similar to the leastsquaresestimates(bt:3.893, bz:0.776),but ir.r-sgest a slightly higher arearesponseelasticity.The estimate0:0.422 is also similar 'cut not exactly the same as the estimate rt :0.404 obtained from the correlation between least squaresresiduals that are one period apart. The standard error se(p2) : ,.t.159is smaller than the corresponding correct least squares standard enor [se(b2) : ,l 3781,suggestinga more reliable estimate,but the standarderror for p1 is unexpectedly .arger, something that we do not expect since we have used an estimation procedure rith a lower variance. It has to be kept in mind, however, that standard errors are dremselvesestimates of true underlying standard deviations. Also, a sample size larger ihan I: 34 might be necessaryto realize the potential lower variance benefits from the lener estimator. 9.3.2a Generalized Least Squares Estirnation h Chapter 8 we discoveredthat the problem of heteroskedasticitycould be overcome by ising an estimation procedure known as generalizedleast squares,and that a convenient r ay to obtain generalizedleastsquaresestimatesis to first transform the model, so that it has r new unconelated homoskedastic error term, and then apply least squares to the transformed model. This same kind of approach can be pursued when autocorrelation erists. Indeed,it can be shown that nonlinear least squaresestimation of (9.24) is equivalent o using an iterative generalized least squares estimator called the Cochrane-Orcutt procedure.Details are provided in Appendix 9A. 9.3.3
EsrruanrNc e Monr
GENEner MooEr
The resultsfor the sugarcaneexample presentedin (9.25) came from estimating the AR( I ) error model written as the transformed model
)r : 9r (l - p) * Fzx,- p9zxt_t* py,_tI v,
(9.26)
238
DYNAMIC
MODELS, AUTOCORRELATION,
AND
FORECASTING
Applying the r ields
Suppose,now, that we consider the model )r :
6 * 6sr1 * 61x1-1* 9jt-t
* vt
(9.2'
How do (9.26) and (9.27) differ? What characteristicsdo they have in common? The firsr thing to notice is that they contain the samevariables; in both cases1l,dependsotrxs, x1and y,- 1. There is a difference in the number of parameters,however. In (9 .26) there ar threeunknown parameters,9t,9z, and p. In (9.27) thereare four unknown parameters.b. 66,61,and 01.Also, the notationin (9.27) is new; we haveusedthe symbolsD (delta)andt (theta). The intercept is denoted by E, the coeff,cients of x and its lag are denoted b1 subscripted Es, and the coefficient of the lagged dependent variable )r-t is given b1' a subscripted 0. This new notation will prove to be convenient in Section 9.7 where uc discussa generalclassof autoregressive distributed lag (ARDL) models.Equation (9.1is a member of this class. To establishthe relationship between (9.26) and (9.27), note that (9.27) rs the samear (9.26) rt we set 6:9r(1-p)
Eo:92
6r:-ppz
0r:p
Then, it can be seen that (9.26) is a restricted version of (9.27) with the restrictior6r: -0160. If this restriction is imposed on(9.27), the number of parametersreduce> from four to three and (9.21) is equivalent to the AR(l) error model. These observationsraise a number of questions.Instead of estimating the AR(l) errs model, would it be better to estimate the more general model in (9.27)? What techniqr should be used for estimatingQ.zT)? Is it possible to estimate (9.27) and then test tlr validity of the AR(l) error model by testing a null hypothesisI1s:61 : -Q1[o? Considering estimation first, we note that (9.27) can be estimated by least squarel providing the v1satisfy the usual assumptionsrequired for least squaresestimation,namell" they have zero mean, constantvariance, and are uncorrelated.The presenceofthe laggec dependentvari abley;l meansthat a large sampleis required for the desirablepropertiescv. the least squaresestimator to hold, but the least squaresprocedureis still valid. We can nc longer make this statementif the vr are correlated,however.Suchcorrelation will lead to thc least squaresestimator being biased, even in large samples. In the introduction to this chapterwe observedthat dynamic characteristicsof time-serier' relationships can occur through lags in the dependent variable, lags in the explanatorl variables,or lags in the error term. We went on to examinehow a lag in the error term can be modeled through an AR( I ) process,and showedsuch a model is equivalentto (9.26), whicl in turn is a special case of (9.2'7).Notice that (9.26) and (9.27) do not have lagged errcr terms,but they do havea laggeddependentvariable and a laggedexplanatoryvariable.Thus. the dynamic featuresof a model implied by an AR( 1) error can be capturedby using instead a model with a lagged y and a lagged x. This observation raises issues about a generaj modeling strategyfor dynamic economic relationships.Insteadof explicitly modeling lagr through an autocorrelatederror, we can capturethe samedynamic effects by simply addins lagged variables)1-1 and.rr-1 to the original linear equation. Is it possibleto test I1e: 61 : -6160 and hencedecide whether the AR(l) model is a reasonablerestricted version of (9.21), or whether the more generalmodel in (9.27) would be preferable?The answeris "yes"; the test is similar to, but more complicated, than those consideredin Chapter 6. Complications occur becausethe hypothesisinvolves an equation that is nonlinear in the parameters,but, nevertheless,it can be performed using modem software.
Becausethe lagg than 34 observat {uares estimate estimate0 : 0.4 rnterceptestimat ;oefficients are s 'quare (Wald) tel
e corresponding error model. Mo( eppearsto be a r
9.4 Testin
\\'e have discove modelingof dynt AR(1) errors me laggedexplanat .'rnwhetheror no :s important, the We considert :u o proceduresa Lagrangemulti SeeAppendix C The third tes \bu will almost \ ourcomputer ol end smaller valu iong-standingffr :heLagrangemu rssumptions are
9..{.1 REsrou
Our objectiveis I .-onelated.Inthe nndom errors w :ort I s.Itfollo :n the context ol 3utocorrelationi
\\hen looking ; retween the lear
9.4 TESTING FOR AUTOCORRELATION
ECASTING
(9.21 n common?The firsi 1 dependsotr\, x1'r. In (9.26) there are .nownparameters.6. mbols 6 (delta) and t' ; lag are denoted b1 le yr 1 is given by " ',ction 9.7 where ir.' dels.Equation(9.2, (9.27) is the samea.
239
Applying the least squaresestimator to (9.27) using the data for the sugarcaneexample i ields i : 2 . 3 6 6 - t 0 . 7 1 7 x 1 -0 . 6 1 t x , - t I 0 . 4 0 4 y 1 - 1 tQ rRr (0.167) (se) (0.6s6) (0.280) (0.2e7) Becausethelaggedvariablesyr-landrr lappearontherightsideoftheequation,33rather :han34 observationsare usedfor estimation.The estimate9z : 0.777 is similar to the least .quaresestimate from the original equation relating 1og of area to log of price, while the tsrimate0 : 0.404is the sameasthe correlationobtainedusingleastsquaresresiduals.The :nterceptestimateis lower than the valuesobtainedin our earlier estimations.All estimated ;oefficients are significantly different from zero at a 0.05 level of significance.Usin-ea chi* 1 u a r e ( W a l d ) t e s t t o t e s t t h e h y p o t h e s i s F:l s :-6911 6 o y i e l d s a v a l u e o f X i r , : 1 . l 1 5 u i t h . correspondingp-value: 0.29.Thus,we do not rejectthe restrictionimplied by the AR( I t via an AR( I ) enor model crrormodel.Modeling the dynamicsof arearesponseof sugarcane strategy. rppears to be a reasonable
9.4 Testing for Autocorrelation with the restrictior. parametersreduce. Ltingthe AR(1) eno: Z7)?What techniqu.' t7) and then test thr' r : -0rDo? ted by least square. s estimation,namell. 'esenceof the laggec esirablepropertieso: ltill valid. We can nc lation will lead to the aristicsof time-serie. s in the explanatory the error term can be rlent to (9.26),which rot have lagged error ratoryvariable.Thus. uredby using insteac rues about a general rlicitly modelinglag. :ctsby simply adding he AR(1) model is r rodelin (9.27)wouli nplicated, than those involves an equation :rmed using modern
\\'e have discoveredthat the existenceof autocorrelationhas an important bearing on the nodeling of dynamic relationshipsand the choice of estimationtechnique.The existenceof \R(1) errors means we should transform our model into one with lagged dependentand -aggedexplanatoryvariablesand estimateit with nonlinear or linear leastsquaresdepending ,.n whetheror not we imposethe coefficient restriction implied by the AR(l) error model. It :s important, therefore, to be able to test for the existenceof correlated errors. We considertwo testing proceduresin this sectionand a third in Appendix 98. The main :u o proceduresare ( 1) examinationof a device known as a residual correlogram, and (2) a Lagrange multiplier test, so called becauseit is basedon a testingprinciple with that name. SeeAppendix C.8.4c for some general information about Lagrange multiplier tests. The third testing procedure, consideredin Appendix 98, is the Durbin-Watson test. \bu will almost certainly seea value for a Durbin-Watson statistic d routinely reported on r ourcomputeroutput.Becaused x 2(l - 11), valuesof d near2 suggestno autocorrelation end smallervalues(lessthan about 1.4) suggestpositive autocorrelation.Although it is a iong-standingtraditional test,the Durbin-Watson is lessconvenientthan the correlogramor :heLagrangemultiplier test.Theselatter two testsareeasierto implement and,becausetheir issumptions are less restrictive, they are applicable to a wider variety of situations. 9..1.1 Rlsrruer
Connsrocnam
Ourobjectiveistotestwhethertheerrorselinthercgressionmodelyr: 9r I Bzxt + etare ;onelated.In theAR(l) errormodelet : pet-r * vr, we assumethat the vr areuncorrelated randomerrorswith zero mean and constantvariance.That is, v, - (0, ol) and E(v,v,) : g : o r t I s .I t f o l l o w s t h a t t h e e y w i l l b e u n c o r r e l a t e d w h e n p : 0 , b e c a u s e t h e n ev,,:. T h u s , :n the context of the AR(l) enor model, the null and alternative hypothesesfor a test for lutocorrelation are Hs:P:0
h:Pl0
\Vhen looking at the sugarcaneexample in Section 9.2.2, we computedthe correlation tretweenthe least squaresresiduals 2r and those residuals lagged by one period ?' 1 and
240
DYNAMIC
AND FORECASTING
MODELS, AUTOCORRELATION,
found it to be r1 - 0.404.We recognizedthat this valueis an estimateof the autocorrelation coefficientp. Thus,one way to testfor autocorrelationis to examinewhetherrr : 0.404 i: significantlydifferent from zero. To obtain a test statisticfor testing,Flsagainstthe alternativeI11, we use a relativell is true, namely simple result that holds in large sampleswhen -F16
z:,/Tn -N(0,1)
(9.29'
()
The product of the squareroot of the samplesize and the correlation rl has an approximate significancelevel,werejectlls:p: (l standardnormaldistribution.Consequently,ata5Vo whenfr1 > l.96orJTr, < 1.96. In the sugarcaneexamplewhere I : 34 and h : 0.404, we have
Z : t/34x0.404:2.36> 1.96
(9.30 r;t'np,9.4
Hence, we reject the null hypothesis of no autocorrelation and conclude the errors are correlated. Most econometric software automatically performs the test in (9.30) in a slightlr different and more general form. To describe this form, note that rejecting .F1ewhen t/Tn> 1.96ort/Tn < -1.96 is equivalentto rejectingIls when
r.96 r .96 or nan. g vT We can view | .961fi and -1 .961t/T as boundson rr outsideof which 11is significantlr different from zero. This result can be generalizedto all lags. Suppose4 is the correlation between?t and 2,-d in otherwords,it is the correlationbetweenresidualsthat aretperiodr apart. Then, rp is significantly different from zero at a 5Vosignificance level if
1.96
" - - '/T
ft)
Or
h,1
--
r.96
(9.31
'/T
The sequenceof correlations4, k : l, 2, ... , is called the sample autocorrelation function or the correlogram ofthe residuals.It showsthe correlationbetweenresidualsthat are one period apart,two periodsapart,threeperiods apart,and so on. Thesecorrelationsare estimatesof the populationcorrelationfunction given by pb pz, %, . . . , where
Cone
Thebarsin Figure 9 .rgnificancebounds :rom zero.The signi -'r-rormodel. In Sect aetweenerrors ft pe :lised to the power t :hey are hard to er J!)rrelationor it mi1 Your software wi :right have the con .nsteadofbars to de .,ndits significance :.olate and focus o -:isturbedif the outl ..reslightly differen . 74 -N(0, 1). Figure9.4 illustr :he model
(9.32,
To provide an easyway of checkingthe inequalitiesin (9.31),most econometricsoftware providesa graphof the correlogramwith thebounds+1 .96I t/T superimposed on that graph sothat the econometriciancan seeat a glancewhich conelationsaresignificant.In the sugar examplethe correlationsup to six lags are \ : 0.404 rz :0.122
rs : 0.084 r+ :
and the bounds are *1 96lt/T: Figure 9.4.
-0353
rs :
-0.420
ra :
-0.161
t0.336. These results appear in the correlogram in
Foliowingdetectio \R(l) error model
Ii theassumptiontl :esidualsfrom estit .rsingthe correlogr :orrelation at lag :emain.
\STING
9.4 TESTING FOR AUTOCORRELATION
autocorrelation ef r1 : 0.404i.
241
0.6 0.4
use a relativel\
I
nt '!
6
(9.29,
0.0
ll
F
(.) an approximate iectHs:p : 0
(9.30,
) in a slightlr :ting ^F1e when
s significantlr .hecorrelation rt are fr period: tif (9.31t :ocorrelation residualsthar rrelations are rere (9.32t
tric software )n that graph . In the sugar
=- 0 . 1 6 1 :elogram in
I
Lag rcuns
the errors are
r
9,4
Correlogram for least squares residuals from sugar cane example.
The bars in Figure 9.4 representthe correlations and the horizontal lines at +0.336 are the 'ignificance bounds. They suggestthe correlatioDs11,t4, and 15are significantly different liom zero. The significant negativecorrelationsat lags 4 and 5 are not capturedby an AR(1) errormodel. In Section9.2.2we found that,for the AR(l) model, pk : pk.The correlation hetweenerrors k periods apart is equal to the correlation between effors one period apart, raisedto the power of t. The correlations 14 and 15do not conform to this expectation,and lhey are hard to explain in terms of sugarcanearea response.It might be a spurious ;orrelation or it might suggestsome other kind of model misspecificatron. Your software will not produce a correlogram that is exactly the same as Figure 9.4. It might have the correlations on the x-axis and the lags on the y-axis. It could use spikes insteadof barsto denotethe correlations,it might provide a host of additionalinformation, .indits significanceboundsmight be slightly differentthan ours.So, be prepared!Learn to rsolate and focus on the information corresponding to that in Figure 9.4 and do not be ,listurbedifthe output is slightly but not substantiallydifferent.Ifthe significancebounds ereslightly different,it is becausethey usea refinementof the large sampleapproximation r i"4 -1919,1;. Figure 9.4 illustratesthe correlogramfor the residuals ?, fromleast squaresestimationof the model it :9t
* 9 z x ,* e ,
Followingdetectionof a significantcorrelationat lag l, a reasonablestrategyis to assumean {R(1) error model e, : pet_r * vr and then estimatethe derivedmodel (9.24) /r : Fr (l - P) I !zx, * PY'-r - P1zx-_t* vt If theassumptionthat the enors v/ areuncorrelatedis a reasonableone,we would expectthe residualsfrom estimatingthis model, the i,'s. to be uncorrelated.This fact can be checked usingthe correlogramfortheil's that appearsin Figure9.5.From this figurewe notetharthe correlation at lag I has been eliminated, although the large correlations at lags 4 and 5 remain.
242
DYNAMIC
MODELS, AUTOCORRELATION,
AND FORECASTING
Rearrangingthis r
0.6
I
0.2 ii
I
0.0
I
U 4.2 4.4
TI
*here11 : Fr .ignificanceofthe :.st result-the s! :he intercePt and :'stimating(81 *hich the ZxR2 b - b) are cer rower, that Powe IfHs:P:Qi5 Rl are the samP :'stimationof (9.
I
-0.6 Lag Frc uRE 9 , 5
9.4.2
Conelogram
for nonlinear least squares residuals from sugarcane example.
A LecncNcE MULTIPLIEnTPsr
A secondtest that we considerfor testing for autocorrelationis derived from a generalsetot hypothesistestingprinciples that produceLagrangemultiplier (LM; tests.In more advanced coursesyou will learn the origin of the term Lagrange multiplier. The general principle is describedin Appendix C.8.4c.To introducethis test, return to equation(9.21) that war written as ]r:Fr
* 9 z x t * P e 1 - 1l v ,
1. When es
(r:1)rr
(e.33
If e1-1 was observable,an obvious way to test the null hypothesisI1e: p : 0 is to regress-'ro n x l a n d e y - 1a f l d t o u s e a / - o r F - t e s t t o t e s t t h e s i g n i f i c a n c e o ft h e c o e f f i c i e n t o fe r - 1 . Becauseey-1is not observable,we replaceit by the laggedleastsquaresresiduals26-1,and then perform the test in the usual way. Proceeding in this way for the sugarcaneexample yields t :2.439
The 5Vocritical v ..f autocorrelati LV:5.4'74,wh Further variat :ollowing:
are often, our calcu omitted i 2. We have AttOf !1-1
order lag additione of their i degrees< test is th
F : 5.949 p-value: 0.021
The p-value is less than 0.05 and thus the LM test rejects the null hypothesis of no autocorrelationat a 57o significancelevel. As we discoveredin Chapter8, LMtests are suchthat they can frequently be written asthe simple expression TxR2, where Z is the number of sample observations and R' is the goodness-of-fit statistic from an auxiliary regression. To derive the relevant auxilia4 regression for the autocorrelation LIUI test, we begin by writing the test equation from ( 9 . 3 3 )a s )r:Br
*\zxtlP?t-tlit
Noting that y, : bt I bzxt * 2t, we can rewrite (9.34) as br I bzx,+ at : 9t * S2x,I P2,: I i'
(9.34t
9.4.3
RrcaP
ln the introduc introducinglag trf the dependen so far has been model.Howeve ;overed have r crpressedasa n Section9.7 we models. These
]ASTING
9 . . +A N I N T R O D U C T I O N
TO FORECASTING
243
Rearrangingthis equation yields
?, : (h - br) + (92- bz)x,-r p\,_t I i, : lr * ^lzxt I p?,_1 I i1
le example.
(e.35)
n'here "y1: Pr - b1 and ^yz: 9z - b2. When testing for autocorrelation by testing the significanceofthecoefficientof 2t-t,onecanestimate(9.34)or(9.35).Bothyieldthesame test result-the samecoefficient estimate for 21-1and the same /-value. The estimatesfor the intercept and the coefficient of x, will be different, however,becausein (9.35) we are estimating(Pi - br) and (B2 - b2) insteadof B1 and 92. The auxiliary regressionfrom *'hich the TxR' version of the LM test is obtained is (9.35). Because(Br - bt) and r9z - b) are centeredaround zero, if (9.35) is a regressionwith significant explanatory power, that power will come from 2r-1. If H(): p : 0 is true, thenLM : TxR2 hasan approximate1f,,-distributionwhereZand R? are the sample size and goodness-of-fitstatistic, respectively,from least squares estimation of (9.35). For the sugarcaneexample L M : T x R " : 3 4 x 0 . 1 6 1 0 1: 5 . 4 7 4
n a generalsetoi n more advanced Ieneral principle t (9.21) that war
(9.3-3 0 is to regress,irfficient of e1 i iiduals 21-1,and
rpothesis of no bewritten asthe s and R2 is the evant auxilian equation from
(9.34t
fhe 5Vocritical value from a 1f,, -distribution is 3.84, leading us to reject the null hypothesis of autocorrelation. Alternatively, we can reject Hs by examining the p-value for LvI :5.474. which is 0.019. Further variations and generalizationsof the Ltr[ test are available. You should note the tollowing: 1. When estimating the regressionsin (9.34) or (9.35), using the first observation (r : 1) requiresknowledge of 2o. Two ways of overcoming this lack of knowledge are often employed.One is to set 2o : 0. The other is to omit the first observation.In our calculationswe set2n : 0. The resultschangevery little if the first observationis omitted instead. 2. We have beenconcernedwith testing for autocorrelationinvolving only one lagged ertot !7-1.To test for more complicated autocorrelationstructures,involving higher order lags such as !t-2, !t-3, and so on, the LM test can be used by including the additional laggederrorsin (9 .34) ot (9.35). An F-test can be usedto testthe relevance of their inclusion, or, a 12-test can be used for the TxR2 version of the test. The degreesoffreedom for the X2-testand the numerator degreesoffreedom for the Ftest is the number of lagged residuals that are included. 9.4.3
REcnpprNc eNl LooxrNG FoRvARD
In the introduction to this chapter we indicated that we would consider three ways of introducing lags into a regressionequation-through the error term, through lagged values ofthe dependentvariable,and through laggedvaluesofthe explanatoryvariables.Our focus >ofar has been on a particular model for a lagged error term, the first-order autoregressive model. However, it is important that you appreciatethat some of the things that we have covered have wider applicability. The simple regression model with AR(l) error was expressedasa model with a lagged explanatoryvariable and a laggeddependentvariable. In Section 9.7 we study a class of models known as autoregressivedistributed lag (ARDL) models. These are models containing both lagged explanatory variables and lagged
244
DYNAMIC
MODELS, AUTOCORRELATION,
AND
FORECASTING
dependentvariables.Thus, the AR(l) enor model can be viewed as a specialcaseARDL model. In general, ARDL models can capture very flexible dynamic relationships. The AR(l) error model was not only a convenientmodel for leading into a discussionof more general ARDL models, it was a convenientone for introducing testsfor autocorrelation in the errors. What needsto be emphasizedis that the correlogram and the Lagrange multiplier test, designed for picking up correlated enors, are valid testing proceduresin more general models. Before we formally introduce ARDL models in Section 9.7, we discussAR models (that contain laggedvaluesofthe dependentvariable) in Section9.5 and finite distributed lag models (that contain lagged values of the explanatory variable) in Section 9.6. In all these types of models (AR, DL, and ARDL), it is important to test for autocorrelation in the errors. The correlogram and the Lagrange multiplier test are appropriate proceduresfor doing so. Finally, as you read through the next three sectionskeep in mind the two generalreasons for estimating dynamic relationships:(1) To forecastfuture values of a dependentvariable and (2) to trace out the time path of the effect of a changein an explanatoryvariable on a dependentvariable. The AR model discussedin Section9.5 is useful for forecastingand so the discussionin that sectionconcentrateson the important ingredientsof forecasting-hori to forecast,and find forecaststandarderrors and forecastintervals.The finite distributedlag model discussedin Section 9.6 is useful for estimating the impact of a change in an explanatory variable; here we are concernedwith delay and interim multipliers. The more general ARDL model is used for forecasting and estimating multiplier effects.
9.5 An Introduction to Forecasting: Autoregressive Models The forecasting of values of economic variables is a major activity for many institutions including firms, banks, governments,and individuals. Accurate forecastsare important for decision making on government economic policy, investment strategies,the supply of goods to retailers, and a multitude of other things that affect our everyday lives. Because of its importance, you will find that there are whole books and coursesthat are devoted to the various aspects of forecasting-methods and models for forecasting, ways of evaluating forecasts and their reliability, and practical examples. In this section we give a brief introduction to forecastingusing one classof models known as autoregressive(AR) models. In the earlier sectionsof this chapter we were concemedwith an AR(l) error model !t : p!t-r * vr and its implications for estimating B1 and B2 in the regressionmodel )r : Fr I 9zx, * e1.However,the AR classof modelshas wider applicability than its use for modelingdynamic error terms.It is alsousedfor modelingobservedvaluesof a time series y,. In this context,an autoregressive model of orderp, denotedas AR(p), is given by /r :
E * 0rlr-r * 0z!;z *
"'
I Qryl-o *v1
(9.36)
In this model the current value of a variabley, dependson its valuesin the lastp periodsand a random error that is assumedto have a zero mean, a constantvariance and be uncorrelated over time. The order of the model p is equal to the largest lag of y on the right side of the equation. Notice that there are no explanatory variables in (9.36). The value ofy, depends only on a history ofits pastvaluesandno x's. Thus,when (9.36)is usedfor forecasting,we are using the current and past values of a variable to forecast its future value. The model relies on correlations between values of the variable over time to help produce a forecast.
Supposethat it is :rte for each of the :ronthly observatiot Then,you comPute
-, _ |
It-L
Recall from APPen - hangein the logarit :r'rmsof percentag :ie changein the lol 169.The data are s To use an AR m' -r r9.36).What tec :.umberof lags P? :i\cussion in Secti isged valuesof th :roblem. Also, the .,ritablep. Recall . eriablecan elimin :rough laggedY's \ simple model w :re may not wanl ,Jdition, estimate ::fferent from zer The estimated
riF|t'r,:0.iS ('.)
(0.02
ileving threelagsI bservationsused -nderstandhowot ::at observation ,hservationsare n ::om/:p+l.l In (9.37) the e :'r el; that for lag :utocorrelationsu .re not significant .re significant bu r.'luding distant -omplicatethe m To seehow the :or 3 months into -rse where P :
CASTING
pecial caseARDL ,lationships. nto a discussionoi ts for autocorrelaand the Lagrange ing proceduresin r Section 9.7, u'e in Section9.5 and atory variable) in portant to test for rultiplier test are 'o generalreasons ependentvariable :ory variable on a orecastingand so brecasting-hos ite distributed lag 'a changein an ipliers. The more fects.
rany institutions re important for s, the supply of y lives. Because lhat are devoted asting, ways of sectionwe give regressive(AR r (1) error model :gressionmodel ility than its use l of a time series given by (9.36t .p periodsand a re uncorrelated ight side of the e ofy, depends Forecasting,we ue. The model iuce a forecast.
9.5 AN INTRODUCTION
TO FORECASTING
245
Supposethat it is May 2006, and that you are interestedin forecasting the U.S. inflation rate for each of the next 3 months. To find past data on the inflation rate you collect 270 monthly observationson the consumerprice index (CP1)from December 1983 to May 2006. Then, you compute values of the inflation rate y, as follows CPIt - CPII_ r \ y, : lln(CPI1)- ln(CPI,) Jx f OO= ( x 100 cPIt_l )
Recall from Appendix A.4.5 that the rate of changein a variable can be approximatedby a .-hangein the logarithm ofthat variable.Also, it is conventionalto expressratesofchange in terms of percentages;to do so we multiply the changein logarithms by 100. Defining y1 as the changein the log of the CP1meansthe number of observationson y1is one less,namely 169. The data are stored in the file inflation.dat. To use an AR model to forecast the inflation rate, the next step is to estimatethe model in (9.36). What technique should be usedto estimatethis model? How do we decide on the number of lags p? Providing the errors vr are uncorrelated, then, following the line of discussionin Section9.3.3, equation(9.36) can be estimatedusing least squares.Having lagged values ofthe dependentvariable as our explanatory variable does not present any problem. Also, the requirement that v, be uncorrelated provides one way of selecting a suitable p. Recall from Section 9.3.3 that including lagged values of the dependent r ariable can eliminate autocorrelationin the errors. Thus, one way to proceedis to include enough lagged y's to eliminate autocorrelation. There are also other issues, however. A simple model with fewer lags is preferred to a complicated one with many lags. Thus, one may not want to be concerned about eliminating autocorrelations at long lags. In addition, estimates of the coefficients can be checked to see if they are significantly different from zero. The estimated AR model for the inflation rate, with p :3, is given by
IfFr,: ('")
- 0.2179INFLNT-z -t o.rol3INFLNy4 0.r883+ 0.3733INFrN1-1 o37\ (0.02s3)(0.061s) (0.064s) (0.0613)
Having three lags meansthat the equationis estimatedfor t : 4,5, . . . , 269; the number of observationsused in estimation is reduced by three from 269 to 266. To ensure you understandhowobservationsare "lost," try substitutingt : I into (9.37).You will discover that observations are needed for lNFtNo, INFLN-b and INFLN 2. These presample observationsare not available.For all observationsto be available for estimation, we begin t-romr: p + I. Most software automatically makes this adjustment. In (9.37) the estimatedcoefficientsat lags I and2 are significantat a l7c significance level; that for lag 3 is significant at a lOVosignificance level. The residual correlogram for autocorrelationsup to lag24 appearsin Figure 9.6. Most of the estimatedautocorrelations arenot significantly different from zero,with thosefor the early lags being negligible. There are significant but not large autocorrelationsat lags 6, Il, 13, and 14. It was decided that including distant lags to eliminate theserelatively small correlations would unnecessarill' complicate the model. To seehow the estimatedAR(3) model in (9.37) can be usedto forecastthe inflation rate tor 3 months into the future, we begin by rewriting the more generalmodel in (9.36) for the casewherep:3 Yr : E + 0]tq
-l 9z!t-z *9zYt-z * vt
(e.38)
246
DYNAMIC
MODELS, AUTOCORRELATION,
AND FORECASTING
0.4
Table 9 .2
Fort
0.3 \lonth
0.2 .
0.1
'i:
F
U.U
I
rll-trl
J u n0 6 ( i : 1 ; J u l0 6 ( 7 : / ; \ u g 0 6( i : 3 )
'lrll-rrlt-- r
a -0.1 I 4.2
rheunpredictablera \\ e were forecasting to ignore the enor fr' rs large relative to torecasterrorsfrom rheerrorfromestimi I month ahead as
-0.3 -0.4 t0 12 14 16 r8 20 22 24 Lag
FrcuRE 9.6
for leastsquares Conelogram residuals from AR(3)modelfor inflation.
Now supposewe haveobservedthe last sampleobservationy1 and we wish to forecasty1- I . y711by changingthe !7a2, and!7a3. Using (9.38),we canobtainthe equationthat generates time subscriptsappropriately. This equation is
For 2 months ahead not only v112,but al .ide of (9.39).Thus
uz: (
Yr+t: D* OrYrI 0zYr-rI 9zYr-z* vr+r
For three Periods al
andthe forecastof this valuegivenby the estimatedequationis
u3: 9 r + r : 6 + O r y r* } z y r - t I } z y r - z : 0 . 1 8 8 3* 0 . 3 7 3 3 x 0 . 4 4 6 -8 0 . 2 1 7 9 x 0 . 5 9 8 + 8 0.1013x0.3510 :0.2602
Becausethe v,'s a ionvenient for deri tnd (9.42), we hav
Moving to the forecast for two months ahead,we have + 0r!r*r -t Tzyr * 0:yr-r : 0 . 1 8 8 3* 0 . 3 7 3 3 x 0 . 2 6 0 -2 0 . 2 1 7 9 x 0 . 4 4 6 *8 0 . 1 0 1 3 x 0 . 5 9 8 8 :0.2487
jr+z :6
(9.39t
There is an important difference in the way the forecasts!111 and j7a2 areobtained. It is possibleto calculatej7a1 usingonly pastobservationson y. However,because!7a2depends on )r+ r , which is unobservedat time ?l on the right side of the equationfor j,1..2,we replace yr+t by its forecast!.*1. For forecastingy713, the forecastsfor both J712 and yr+l are neededon the right side ofthe equation,and for forecastsfurther into the future, all values for the lagged y's are forecastsobtained for earlier periods. The forecastsobtained for 3 months into the future are given in Table 9.2. Also appearingin Table 9.2 are standarderrors for the forecast errors and 95Voforecast intervals. How are they obtained? The forecast error at time Z * I is ut :
yr-t - i,rt
: (6 - S) + (0r - 0,)y. + (02 - Or)yr-t + (0: - }t)yr-, -t v7-1
The difference betweenthe forecastj,7*1 and the correspondingrealized value yz+l can be attributedto the differencebetweenthe actualcoefficientsand the estimatedcoefficientsand
Then, along the linr chapters,in large s
The standarderrot 6 . 0 r , 0 2 0, 3 )a n dt are comPuted fror negativeaswell as rhe forecaststand into the future, rel The AR model rariable can be ur popular model, th
ECASTI NG
9 5 A\
I\TI{OI)UC-TION
TO FORECASTING
247
Forecasts and Forecast Intervals for Inflation Rate
TabIe 9.2
Forecast \lonth
ir+i
l u n0 6 ( 1 : l ) iut 06 (j : /1 \ug 06(7: 3)
0.2602 0.248'/ 0.2697
Standarderror of tbrecastenor (o7)
0r*;
Forecast interval - 1.96961,1r+1+ 1.969Gj)
0.1972 0 . 21 0 5 0 . 2 1l I
(-0.1282,0.6485) ( - 0 . 16 5 8 ,0 . 6 6 3 3 ) (-0.1460,0.6854)
:heunpredictablerandomelror v711.A similar situationarosein Sections4.1 and 6.8 when .\e wereforecastingusingthe regressionmodel.What we aregoing to do differentlynow is :o ignorethe errorfrom estimatingthe coefficients.Usually the varianceof therandomerror :\ large relative to the variancesof the estimatedcoefficientsand so, when examining :orecasterrorsfrom AR models,andcomputingtheir standarderrors,it is commonto ignore :heenor from estimatingthe coefficients.Doing so meanswe canwrite the forecasterrorfor . month aheadas
rr inflation.
ul :
ish to forecast_yr_ bychangingtl^7"11
vT+l
(9..+0)
For 2 monthsaheadthe forecasterror getsmore complicatedbecausewe haveto allow fbr rot only v712,butalsofor the error that occursfrom using!7a1 insteadof )r+ r on the right .ide of (9.39).Thus, the forecasterror for two periods aheadis u z : 0 t ( ) r * ' - y r + r ) * v r + z : 0 1 4 1 * v r + 2: 0 t v r + t I v r + z
(9.41)
For three neriods aheadthe error can be shown to be uz : 0fl2 I 02u1I vr+z: (0i + 0z)rr*r I 01v7a2* vr+z
Becausethe v1's are uncorrelated,expressingthe forecast errors in terms of them is :onvenientfor deriving expressionsfor the forecastenor variances.From (9.40), (9.41), nd (9.42), we have
tx0.3510
).5988
\9.42)
(9.:q
are obtained.It i. ause!712depend. ,r jr+2, we replacr' t7'r2 and )i"11 arc
: future,all value. sts obtained for ,: and95Voforeca:l
ir)vr-, * vr+r r'aluelr+l canbe J coefficientsand
ol : var(ut): "3 ol : var(uz): + o?) "3(t o! : var(4): 10r)'+ 0?+ t] "?[(o? Then,along the lines of confidenceintervals that were constructedfor parametersin earlier ,-hapters,in large samplesa 95Vo confidenceinterval for a forecast of yr+j is given by (!r '1 - tro 1'
!r. 1-t t,o 1)
(e.43)
The standarderrors in Table9.2 &re01, 02, and 03, evaluatedat the coefficientestimates 6. 01,02,e3) anAthe error standarddeviationestimate6, : 0.1972.The forecastintervals rre computedfrom (9.43) with t. : to.gts.zaz):1.969. The wide intervals that include negativeaswell aspositivevaluessuggestthatthe forecastsarenot very reliable.Notice how rhe forecast standarderrors and the widths of the intervals increaseas we forecast further into the future, reflectingthe additionaluncertaintyfrom doing so. The AR model is just one of a largenumberof forecastingmodelswherethe history of a r ariable can be used to forecastits future values.In Appendix 9D we examine another popularmodel, the exponentialsmoothingmodel.
248
DYNAMIC
MODELS, AUTOCORRELATION,
AND FORECASTING
9.6 Finite Distributed Lags
fable 9,3
In the previoussectionwe usedthe framework of an autoregressivemodel to investigatehou the past history ofthe inflation rate can be used to forecast future inflation rates.Now, ue turn to a different question.What impact do wage changeshaveon the inflation rate?On the supply side we expect wage increasesto increasethe costs of production and to drive up prices. On the demand side wage increasesmean greaterdisposableincome, and a greater demand for goods and services, that also pushes up prices. Irrespective of the line of reasoning,the relationship between wage changesand inflation is unlikely to be a totalll instantaneousone; it takes time for wage changes to impact on inflation. One way of modeling the impact of wage changeson inflation is through a finite distributed lag model To introduce the conceptof a finite distributed lag, lety, be the inflation rate and let-r, be the rate of changein the wage rate in period r. If we assumea linear relationshipbetweenthe inflation rate and the rate of changein wages,and also that wage changesup to q monthsinto the pasthave an influenceon the current rate of inflation, then the econometricmodel for thi: relationship can be written as yt :d.*90x,*Fr.rr-r l!zx,-z+
... +Qqx,-qlv,,
t : q*1,...,7
(9.U,
where we assumethat E(v.): 0, var(vr) : o2, andcov(v,, v,) : 0. If we have Zobservations on the pairs (yr, .r,), then only Z - q complete observations are available for estimationsince 4 observationsare "lost" in the creationof xr-1, xt-2, ..., &-4, an issue we also encounteredfor the AR model of the previous section. Equation (9.44) is called a distributed lag model becausethe effect of changesin.r, is distributed over time. It is called a finite distributed lag model because it is assumed that after a finite number of periods q, changes in r no longer have an impact on r. The parameterct is the intercept and B, is called a distributed-lag weight or an s-period delay multiplier. It measuresthe effect of past changesin the growth of wages Axl-, on expected current inflation, when the growth of wages is held constant in other periods. That is. oE(_y,) :- ttR Otr-"
To appreciatefurther the interpretationof the lag weights B, supposethat x and y have been constantfor at leastthe last q periods and that x, is increasedby 1 unit (l%oin our example). and then returned to its original level. Then, ignoring the error term, the immediate effect will be an increaseinyr by Beunits.Oneperiodlatery,*1will increaseby Bl units,theny,-; will increaseby 92 units,andso on, up to periodr * 4 whenyr*n will increaseby Buunits.In period r * q -l I the value of y will return to its original level. Thus, B, is the s-period delayedeffect of a changein x. If the rate of growth of wagesincreasesby 170in period r and then returns to its original level, there will be a BrVoincreasein the inflation rate in period r * s. Setting s : 0 yields what is called the impact multiplier Bs. It is also relevantto ask what happensifxr is increasedby I unit and then maintainedat its new level in subsequentperiods (r + 1), (t + 2),. . . . In this casethe immediateimpact will againbe Fo;the total effect in period t * I will be 9o + B1,in period t + z,it will be Bo * B1 f Fz, and so on. We add together the effects from the changesin all proceeding periods. These quantities are called interim multipliers. For example, the two-period interim multiplier ir (Bo + 9r + Br). The total multiplier is the final effect on y of the sustainedincrease after q or more periods have elapsed;it is given by I3:0F,.
Leas
Variable Constant
i.
l
Forestimationof the. ire uncorTelatedand ,:nbiasedestimateso. Constructionof the d ' timilar way, we de
xs: ln(WA,
'thereWAGEris the i 're for the U.S. ecor Allowing for a 3-r Yluaresestimatesrel .:,nimmediate increa l-month laggedincr ::npactsarefelt immt .ignificantly differet qultipliers are repor The interim multiPli :tier 3 months,lead Finite distributed :stimated equationtr :nknown, it maY be :orecasty, with a con rclicy variable like .r'ed to forecast the
T
0 1 2 J
ASTING
9.6 FINITE DISTRIBUTED LAGS
Ta b I e 9 , 3
o investigatehou In rates.Now, ue ltion rate?On the n and to drive up me, and a greater re of the line of :ly to be a totally ion. One way of butedlag model. : rate and letx, be Lshipbetweenthe r to 4 monthsin