This is an old revision of the document!
derivate of a and b in regression
dv for a
dv for b
to understand gradient descent
\begin{eqnarray*} \sum{(Y_i - \hat{Y_i})^2} & = & \sum{(Y_i - (a + bX_i))^2} \;\;\; \because \hat{Y_i} = a + bX_i \\ & = & \text{SSE or SS.residual} \;\;\; \text{(and this should be the least value.)} \end{eqnarray*}
\begin{eqnarray*} \text{for a (constant)} \\ \\ \dfrac{\text{d}}{\text{dv}} \sum{(Y_i - (a + bX_i))^2} & = & \sum \dfrac{\text{d}}{\text{dv}} {(Y_i - (a + bX_i))^2} \\ & = & \sum{2 (Y_i - (a + bX_i))} * (-1) \;\;\;\; \\ & \because & \dfrac{\text{d}}{\text{dv for a}} (Y_i - (a+bX_i)) = -1 \\ & = & -2 \sum{(Y_i - (a + bX_i))} \\ \\ \text{in order to have the least value, the above should be zero} \\ \\ -2 \sum{(Y_i - (a + bX_i))} & = & 0 \\ \sum{(Y_i - (a + bX_i))} & = & 0 \\ \sum{Y_i} - \sum{a} - b \sum{X_i} & = & 0 \\ \sum{Y_i} - n*{a} - b \sum{X_i} & = & 0 \\ n*{a} & = & \sum{Y_i} - b \sum{X_i} \\ a & = & \dfrac{\sum{Y_i}}{n} - b \dfrac{\sum{X_i}}{n} \\ a & = & \overline{Y} - b \overline{X} \\ \end{eqnarray*}
\begin{eqnarray*} \text{for b, (coefficient)} \\ \\ \dfrac{\text{d}}{\text{dv}} \sum{(Y_i - (a + bX_i))^2} & = & \sum \dfrac{\text{d}}{\text{dv}} {(Y_i - (a + bX_i))^2} \\ & = & \sum{2 (Y_i - (a + bX_i))} * (-X_i) \;\;\;\; \\ & \because & \dfrac{\text{d}}{\text{dv for b}} (Y_i - (a+bX_i)) = -X_i \\ & = & -2 \sum{X_i (Y_i - (a + bX_i))} \\ \\ \text{in order to have the least value, the above should be zero} \\ \\ -2 \sum{X_i (Y_i - (a + bX_i))} & = & 0 \\ \sum{X_i (Y_i - (a + bX_i))} & = & 0 \\ \sum{X_i (Y_i - ((\overline{Y} - b \overline{X}) + bX_i))} & = & 0 \\ \sum{X_i ((Y_i - \overline{Y}) - b (X_i - \overline{X})) } & = & 0 \\ \sum{X_i (Y_i - \overline{Y})} - \sum{b X_i (X_i - \overline{X}) } & = & 0 \\ \sum{X_i (Y_i - \overline{Y})} & = & b \sum{X_i (X_i - \overline{X})} \\ b & = & \dfrac{\sum{X_i (Y_i - \overline{Y})}}{\sum{X_i (X_i - \overline{X})}} \\ b & = & \dfrac{\sum{(Y_i - \overline{Y})}}{\sum{(X_i - \overline{X})}} \\ b & = & \dfrac{ \sum{(Y_i - \overline{Y})(X_i - \overline{X})} } {\sum{(X_i - \overline{X})(X_i - \overline{X})}} \\ b & = & \dfrac{ \text{SP} } {\text{SS}_\text{x}} = \dfrac{\text{Cov(X, Y)}} {\text{Var(X)}} = \dfrac{\text{Cov(X, Y)}} {\text{Cov(X, X)}}\\ \end{eqnarray*}
리그레션 라인으로 예측하고 틀린 나머지 error의 제곱의 합을 (ss.res) 최소값으로 만드는 선의 기울기와 절편값은 위와 같다 (a and b).
위는 증명을 통해서 a와 b값을 알아낸 것이고, R과 같은 어플리케이션에서 a와 b를 알아내는 방법은 없을까?