Obtaining_Univariate_OLS_Estimates_Through_Elementary_Algebra
Obtaining univariate OLS parameter estimates
through elementary algebra
Consider the linear regression model
yt = α + βxt + ut, t = 1, . . . , T.
Given parameter estimates αˆ and βˆ the residuals are defined as
uˆt = yt − αˆ− βˆxt, t = 1, . . . , T.
The ...
Obtaining univariate OLS parameter estimates
through elementary algebra
Consider the linear regression model
yt = α + βxt + ut, t = 1, . . . , T.
Given parameter estimates αˆ and βˆ the residuals are defined as
uˆt = yt − αˆ− βˆxt, t = 1, . . . , T.
The OLS estimates of the parameters α and β are determined by minimizing the residual
sum of squares
RSS =
T∑
t=1
uˆ2t .
We will show that the OLS estimates can be obtained by elementary algebra, without
using calculus. First, express the residuals using the demeaned versions of yt and xt
(that is, yt − y¯ and xt − x¯) as
uˆt =
[
(yt − y¯)− βˆ (xt − x¯)
]
−
[
αˆ−
(
y¯ − βˆx¯
)]
and note that since
∑T
t=1 (yt − y¯) =
∑T
t=1 (xt − x¯) = 0 we have
T∑
t=1
uˆ2t =
T∑
t=1
[
(yt − y¯)− βˆ (xt − x¯)
]2
+ T
[
αˆ−
(
y¯ − βˆx¯
)]2
,
as the sum of cross products vanishes. The second term is minimized, and becomes
equal to zero, when
αˆ = y¯ − βˆx¯.
Hence, it suffices to minimize the term
∑T
t=1
[
(yt − y¯)− βˆ (xt − x¯)
]2
with respect to βˆ.
Note that
T∑
t=1
[
(yt − y¯)− βˆ (xt − x¯)
]2
= Syy + Sxxβˆ
2 − 2Syxβˆ
1
where
Syy =
T∑
t=1
(yt − y¯)2 ,
Sxx =
T∑
t=1
(xt − x¯)2 ,
Syx =
T∑
t=1
(yt − y¯) (xt − x¯) .
Since Syy does not depend on βˆ, it suffices to minimize the quantity Sxxβˆ2−2Syxβˆ with
respect to βˆ. We do so, by “completing the square”:
Sxxβˆ
2 − 2Syxβˆ = Sxx
(
βˆ2 − 2Syx
Sxx
βˆ
)
= Sxx
[(
βˆ − Syx
Sxx
)2
− S
2
yx
S2xx
]
.
It is clear that the expression above is minimized when the square
(
βˆ − Syx
Sxx
)2
becomes
zero, i.e., when βˆ = Syx
Sxx
. It follows that the OLS estimates of α and β are given by
βˆ =
∑T
t=1 (yt − y¯) (xt − x¯)∑T
t=1 (xt − x¯)2
=
∑T
t=1 ytxt − T x¯y¯∑T
t=1 x
2
t − T x¯2
,
αˆ = y¯ − βˆx¯.
2
本文档为【Obtaining_Univariate_OLS_Estimates_Through_Elementary_Algebra】，请使用软件OFFICE或WPS软件打开。作品中的文字与图均可以修改和编辑，
图片更改请在作品中右键图片并更换，文字修改请直接点击文字进行修改，也可以新增和删除文档中的内容。
该文档来自用户分享，如有侵权行为请发邮件ishare@vip.sina.com联系网站客服，我们会及时删除。
[版权声明] 本站所有资料为用户分享产生，若发现您的权利被侵害，请联系客服邮件isharekefu@iask.cn，我们尽快处理。
本作品所展示的图片、画像、字体、音乐的版权可能需版权方额外授权，请谨慎使用。
网站提供的党政主题相关内容(国旗、国徽、党徽..)目的在于配合国家政策宣传，仅限个人学习分享使用，禁止用于任何广告和商用目的。
浙公网安备 33021202002483