Differences

This shows you the differences between two versions of the page.

--- regression [2023/05/17 07:58] – hkimscil
+++ regression [2024/09/30 01:13] (current) – [E.g., Simple regression] hkimscil
@@ Line 23: / Line 23: @@
 상관관계에서 살펴 본것처럼, 관측된 데이터는 최소자승 (Least Squared) 법을 이용하여 회귀식을 유도할 수 있는데, 이때의 절편과 기울기 값은 각각 다음과 같이 구할 수 있다:
+\begin{eqnarray*}
-$$b = \displaystyle \frac{SP}{SS_X}$$
+b & = & \displaystyle \frac{SP}{SS_X} \\
-$$a = \displaystyle \overline{Y} - b \overline{X} $$
+a & = & \displaystyle \overline{Y} - b \overline{X}
+\end{eqnarray*}
+참조: [[deriviation of a and b in a simple regression|리그레션에서 a와 b 구하기]]
 [{{:f03x1b.gif }}] 최소자승이 의미하는 것은 옆의 그림과 같다. regression line (회귀선)으로 X 값에 해당하는 Y 값을 예측할 수 있는데, 이때에는 실측값과 차이가 날 수 있다. 이 차이가 위의 그림에서 녹색선인데, 이 녹색선의 합이 최소값을 갖도록하는 것을 최소자승(least squared)법이라고 한다.
@@ Line 283: / Line 285: @@
 ^  ANOVA(b)  ^^^^^^^
 |  Model   |      |  Sum of Squares   |  df   |  Mean Square   |  F   |  Sig.  |
-|  1.000    |  Regression   | @white: 18.934    | @blue: 1.000    |  18.934    |  13.687    |  0.006   |
+|  1.000    |  Regression   | @white: 18.934    | @lightblue: 1.000    |  18.934    |  13.687    |  0.006   |
-|     |  Residual   | @orange: 11.066    | @green: 8.000    |  1.383*    |     |    |
+|     |  Residual   | @orange: 11.066    | @lightgreen: 8.000    |  1.383*    |     |    |
 |     |  Total   | @yellow: 30.000    | @#eee: 9.000    |     |     |    |
 | a Predictors: (Constant), bankIncome  income \\ b Dependent Variable: bankbook  number of bank  |||||||
@@ Line 301: / Line 303: @@
 위의 표에서 (Anova table),
-| for SS   | for degrees of freedom   |
+| @grey: for SS   | @grey: for df    |
-| @white: white \\ = explained error (E) \\ = $SS{reg}$  | @blue: for regression \\ (number of variable -1) \\ = 1 (blue) |
+| @white: white \\ = explained error (E) \\ = $SS{reg}$  | @lightblue: for regression \\ (number of variable -1) \\ = 1 (light blue) |
-| @orange: orange \\ = unexplained error (U) \\ = $SS{res}$  | @green: for residual \\ (number of case - number of variable) \\ = 8 (green) |
+| @orange: orange \\ = unexplained error (U) \\ = $SS{res}$  | @lightgreen: for residual \\ (number of case - number of variable) \\ = 8 (green) |
 | @yellow: yellow \\ = total error $SS_{total}$ \\ = E + U \\ = $SS_{reg} + SS_{res}$ | @#eee: grey \\ = total df \\ = total sample # -1  |
@@ Line 350: / Line 352: @@
 {{:acidity.sav}} \\
 {{:acidity.sps}} \\
+<file csv acidity.csv>
+stream	spec83	ph83
+Moss	6	6.30
+Orcutt	9	6.30
+Ellinwood	6	6.30
+Jacks	3	6.20
+Riceville	5	6.20
+Lyons	3	6.10
+Osgood	5	5.80
+Whetstone	4	5.70
+UpperKeyup	1	5.70
+West	7	5.70
+Boyce	4	5.60
+MormonHollow	4	5.50
+Lawrence	5	5.40
+Wilder	0	4.70
+Templeton	0	4.50
+</file>
+<code>
+df <- read.csv("http://commres.net/wiki/_export/code/regression?codeblock=3", sep = "\t")
+</code>
 <code>stream         spec83 ph83
@@ Line 663: / Line 687: @@
-**__r-square:__**
+===== r-square =====
   * $\displaystyle r^2=\frac{SS_{total}-SS_{res}}{SS_{total}} = \frac{\text{Explained sample variability}}{\text{Total sample variability}}$
@@ Line 672: / Line 696: @@
-**__Adjusted r-square:__**
+===== Adjusted r-square =====
   * $\displaystyle r^2=\frac{SS_{total}-SS_{res}}{SS_{total}} = 1 - \frac{SS_{res}}{SS_{total}} $ ,
@@ Line 693: / Line 717: @@
       * R2 value goes down -- which means
       * more (many) IVs is not always good
-  * Therefore, the Adjusted r<sup>2</sup> = .367 / 1.5 = 0.756 (green color cell)
+  * Therefore, the Adjusted r<sup>2</sup> = 1- (.367 / 1.5) = 0.756 (green color cell)
-**__Slope test__**
+===== Slope test =====
 If we take a look at the ANOVA result:
@@ Line 706: / Line 730: @@
 | b Dependent Variable: y    |||||||
 <WRAP clear />
+F test recap.
   * ANOVA, F-test, $F=\frac{MS_{between}}{MS_{within}}$
-  * MS_between?
+    * MS_between?
-  * MS_within?
+    * MS_within?
-  * MS for residual
+  * regression에서 within 에 해당하는 것 == residual
    * $s = \sqrt{s^2} = \sqrt{\frac{SS_{res}}{n-2}} $
-   * random difference (MS<sub>within</sub> ): $s^2 = \frac{SS_{res}}{n-2} $
+   * 왜냐하면 이 ss residual이 random difference 를 말하는 것이므로 (MS<sub>within</sub> ): $s^2 = \frac{SS_{res}}{n-2} $
   * MS for regression . . . Obtained difference
    * do the same procedure at the above in MS for <del>residual</del> regression.
@@ Line 729: / Line 753: @@
   * Why do we do t-test for the slope of X variable? The below is a mathematical explanation for this.
-  * Sampling distribution of b:
+  * Sampling distribution of error around the slope line b:
    * $\displaystyle \sigma_{b_{1}} = \frac{\sigma}{\sqrt{SS_{x}}}$
+     * We remember that $\displaystyle \sigma_{\bar{x}} = \frac{\sigma}{\sqrt{n}}$ ?
    * estimation of $\sigma_{b_{1}}$ : substitute sigma with s
+만약에 error들이 (residual들) slope b를 중심으로 포진해 있고, 이것을 따로 떼어내서 distribution curve를 그려보면 평균이 0이고 standard deviation이 위의 standard error값을 갖는 normal distribution을 이루게 될 것이다.
   * t-test
    * $\displaystyle t=\frac{b_{1} - \text{Hypothesized value of }\beta_{1}}{s_{b_{1}}}$
+   * Hypothesized value of b 값은 (혹은 beta) 0. 따라서 t 값은
+   * $\displaystyle t=\frac{b_{1}}{s_{b_{1}}}$
+   * 기울기에 대한 표준오차는 (se) 아래와 같이 구한다
-   * Hypothesized value of beta 값은 대개 0. 따라서 t 값은
+\begin{eqnarray*}
+\displaystyle s_{b_{1}} & = & \sqrt {\frac {MSE}{SS_{X}}} \\
-   * $\displaystyle t=\frac{b_{1}}{s_{b_{1}}}$
+ & = & \displaystyle \sqrt { \frac{1}{n-2} * \frac{SSE}{SS_{X}}} \\
+ & = & \displaystyle \sqrt { \frac{1}{n-2} * \frac{ \Sigma{(Y-\hat{Y})^2} }{ \Sigma{ (X_{i} - \bar{X})^2 } } } \\
+\end{eqnarray*}
-   * $\displaystyle s_{b_{1}} = \sqrt {\frac {MSE}{SS_{X}}} = \frac{\sqrt{\frac{SSE}{n-2}}}{\sqrt{SS_{X}}} = \displaystyle \frac{\sqrt{\frac{\Sigma{(Y-\hat{Y})^2}}{n-2}}}{\sqrt{\Sigma{(X_{i}-\bar{X})^2}}} $
 ^ X  ^ Y  ^ $X-\bar{X}$  ^ ssx  ^ sp  ^ y<sub>predicted</sub>  ^ error  ^ error<sup>2</sup>  ^
@@ Line 754: / Line 781: @@
 Regression formula: y<sub>predicted</sub> = -0.1 + 0.7 X
-SSE = Sum of Square Error
+SSE = Sum of Square Error = SS_residual
 기울기 beta(b)에 대한 표준오차값은 아래와 같이 구한다.
 \begin{eqnarray*}
 se_{\beta} & = & \frac {\sqrt{SSE/n-2}}{\sqrt{SSX}} \\
@@ Line 764: / Line 792: @@
 따라서 t = b / se = 3.655631
-<code>x <- c(1, 2, 3, 4, 5)
-y <- c(1, 1, 2, 2, 4)
-mody <- lm(y ~ x)
-</code>
-<code>
-> x <- c(1, 2, 3, 4, 5)
-> y <- c(1, 1, 2, 2, 4)
-> mody <- lm(y ~ x)
-> summary(mody)
-Call:
-lm(formula = y ~ x)
-Residuals:
-          2          3          4          5
-.000e-01 -3.000e-01 -3.886e-16 -7.000e-01  6.000e-01
-Coefficients:
-            Estimate Std. Error t value Pr(>|t|)
-(Intercept)  -0.1000     0.6351  -0.157   0.8849
-x             0.7000     0.1915   3.656   0.0354 *
----
-Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
-Residual standard error: 0.6055 on 3 degrees of freedom
-Multiple R-squared:  0.8167,	Adjusted R-squared:  0.7556
-F-statistic: 13.36 on 1 and 3 DF,  p-value: 0.03535
->
-</code>
 ====== E.g., 4. Simple regression ======
 Another example of simple regression: from {{:elemapi.sav}} \\