outliers
This is an old revision of the document!
Table of Contents
Outliers e.g.,
This is further reading for detecting outliers, adopted from http://www.ats.ucla.edu/stat/spss/webbooks/reg/chapter2/spssreg2.htm .
attachment:crime.sav
attachment:outlierCheck.sps
get file = "DirectoryOfYourComputer\crime.sav". descriptives /var=crime murder pctmetro pctwhite pcths poverty single.
Descriptive Statistics N Minimum Maximum Mean Std. Deviation violent crime rate 51 82 2922 612.84 441.100 murder rate 51 1.60 78.50 8.7275 10.71758 pct metropolitan 51 24.00 100.00 67.3902 21.95713 pct white 51 31.80 98.50 84.1157 13.25839 pct hs graduates 51 64.30 86.60 76.2235 5.59209 pct poverty 51 8.00 26.40 14.2588 4.58424 pct single parent 51 8.40 22.10 11.3255 2.12149 Valid N (listwise) 51
graph /scatterplot(matrix)=crime murder pctmetro pctwhite pcths poverty single .
GRAPH /SCATTERPLOT(BIVAR)=pctmetro WITH crime BY state(name) .
GRAPH /SCATTERPLOT(BIVAR)=poverty WITH crime BY state(name) .
GRAPH /SCATTERPLOT(BIVAR)=single WITH crime BY state(name) .
regression /dependent crime /method=enter pctmetro poverty single.
Model Summary Model R R Square Adjusted R Square Std. Error of the Estimate 1 .916a .840 .830 182.068 a. Predictors: (Constant), pct single parent, pct metropolitan, pct poverty ANOVA(b) Model Sum of Squares df Mean Square F Sig. 1 Regression 8170480.211 3 2723493.404 82.160 .000a Residual 1557994.534 47 33148.820 Total 9728474.745 50 a. Predictors: (Constant), pct single parent, pct metropolitan, pct poverty b. Dependent Variable: violent crime rate Coefficients(a) Unstandardized Coefficients Standardized Coefficients Model B Std. Error Beta t Sig. 1 (Constant) -1666.436 147.852 -11.271 .000 pct metropolitan 7.829 1.255 .390 6.240 .000 pct poverty 17.680 6.941 .184 2.547 .014 pct single parent 132.408 15.503 .637 8.541 .000 a. Dependent Variable: violent crime rate
regression /dependent crime /method=enter pctmetro poverty single /residuals=histogram.
Model Summary(b) Model R R Square Adjusted R Square Std. Error of the Estimate 1 .916a .840 .830 182.068 a. Predictors: (Constant), pct single parent, pct metropolitan, pct poverty b. Dependent Variable: violent crime rate ANOVA(b) Model Sum of Squares df Mean Square F Sig. 1 Regression 8170480.211 3 2723493.404 82.160 .000a Residual 1557994.534 47 33148.820 Total 9728474.745 50 a. Predictors: (Constant), pct single parent, pct metropolitan, pct poverty b. Dependent Variable: violent crime rate Coefficients(a) Unstandardized Coefficients Standardized Coefficients Model B Std. Error Beta t Sig. 1 (Constant) -1666.436 147.852 -11.271 .000 pct metropolitan 7.829 1.255 .390 6.240 .000 pct poverty 17.680 6.941 .184 2.547 .014 pct single parent 132.408 15.503 .637 8.541 .000 a. Dependent Variable: violent crime rate Residuals Statistics(a) Minimum Maximum Mean Std.Deviation N Predicted Value -30.51 2509.43 612.84 404.240 51 Residual -523.013 426.111 .000 176.522 51 Std. Predicted Value -1.592 4.692 .000 1.000 51 Std. Residual -2.873 2.340 .000 .970 51 a. Dependent Variable: violent crime rate
regression /dependent crime /method=enter pctmetro poverty single /residuals=histogram(sdresid).
regression /dependent crime /method=enter pctmetro poverty single /residuals=histogram(sdresid) id(state) outliers(sdresid).
see at http://www2.bc.edu/~stevenw/MB875/mb875_Analyzing%20Residuals.htm for sdresid (studentized deleted residuals).
Residuals Statistics(a) Minimum Maximum Mean Std. Deviation N Predicted Value -30.51 2509.43 612.84 404.240 51 Std. Predicted Value -1.592 4.692 .000 1.000 51 Standard Error of Predicted Value 25.788 133.343 47.561 18.563 51 Adjusted Predicted Value -39.26 2032.11 605.66 369.075 51 Residual -523.013 426.111 .000 176.522 51 Std. Residual -2.873 2.340 .000 .970 51 Stud. Residual -3.194 3.328 .015 1.072 51 Deleted Residual -646.503 889.885 7.183 223.668 51 Stud. Deleted Residual -3.571 3.766 .018 1.133 51 Mahal. Distance .023 25.839 2.941 4.014 51 Cook's Distance .000 3.203 .089 .454 51 Centered Leverage Value .000 .517 .059 .080 51 a. Dependent Variable: violent crime rate
regression /dependent crime /method=enter pctmetro poverty single /residuals=histogram(sdresid) id(state) outliers(sdresid) /casewise=plot(sdresid) outliers(2) .
Casewise Diagnostics(a) Case Number state Stud. Deleted violent crime Predicted Residual Residual rate Value 9 fl 2.620 1206 779.89 426.111 25 ms -3.571 434 957.01 -523.013 51 dc 3.766 2922 2509.43 412.566 a. Dependent Variable: violent crime rate
regression /dependent crime /method=enter pctmetro poverty single /residuals=histogram(sdresid lever) id(state) outliers(sdresid lever) /casewise=plot(sdresid) outliers(2).
Outlier Statistics(a) Case state Statistic Number Stud. Deleted Residual 1 51 dc 3.766 2 25 ms -3.571 3 9 fl 2.620 4 18 la -1.839 5 39 ri -1.686 6 12 ia 1.590 7 47 wa -1.304 8 13 id 1.293 9 14 il 1.152 10 35 oh -1.148 Centered Leverage Value 1 51 dc .517 2 1 ak .241 3 25 ms .171 4 49 wv .161 5 18 la .146 6 46 vt .117 7 9 fl .083 8 26 mt .080 9 31 nj .075 10 17 ky .072 a. Dependent Variable: violent crime rate
regression /dependent crime /method=enter pctmetro poverty single /residuals=histogram(sdresid lever) id(state) outliers(sdresid, lever) /casewise=plot(sdresid) outliers(2) /scatterplot(*lever, *sdresid).
regression /dependent crime /method=enter pctmetro poverty single /residuals=histogram(sdresid lever) id(state) outliers(sdresid, lever, cook) /casewise=plot(sdresid) outliers(2) cook dffit /scatterplot(*lever, *sdresid).
Casewise Diagnostics(a) Case Number state Stud. violent Cook's DFFIT Deleted crime Distance Residual rate 9 fl 2.620 1206 .174 48.507 25 ms -3.571 434 .602 -123.490 51 dc 3.766 2922 3.203 477.319 a. Dependent Variable: violent crime rate Outlier Statistics(a) Case Number state Statis Sig. F Stud. 1 51 dc 3.766 Deleted 2 25 ms -3.571 Residual 3 9 fl 2.620 4 18 la -1.839 5 39 ri -1.686 6 12 ia 1.590 7 47 wa -1.304 8 13 id 1.293 9 14 il 1.152 10 35 oh -1.148 Cook's 1 51 dc 3.203 .021 Distance 2 25 ms .602 .663 3 9 fl .174 .951 4 18 la .159 .958 5 39 ri .041 .997 6 12 ia .041 .997 7 13 id .037 .997 8 20 md .020 .999 9 6 co .018 .999 10 49 wv .016 .999 Centered 1 51 dc .517 Leverage 2 1 ak .241 Value 3 25 ms .171 4 49 wv .161 5 18 la .146 6 46 vt .117 7 9 fl .083 8 26 mt .080 9 31 nj .075 10 17 ky .072 a. Dependent Variable: violent crime rate
regression /dependent crime /method=enter pctmetro poverty single /residuals=histogram(sdresid lever) id(state) outliers(sdresid, lever, cook) /casewise=plot(sdresid) outliers(2) cook dffit /scatterplot(*lever, *sdresid) /save sdbeta(sdfb).
list /variables state sdfb1 sdfb2 sdfb3 /cases from 1 to 10.
state sdfb1 sdfb2 sdfb3 ak -.10618 -.13134 .14518 al .01243 .05529 -.02751 ar -.06875 .17535 -.10526 az -.09476 -.03088 .00124 ca .01264 .00880 -.00364 co -.03705 .19393 -.13846 ct -.12016 .07446 .03017 de .00558 -.01143 .00519 fl .64175 .59593 -.56060 ga .03171 .06426 -.09120 Number of cases read: 10 Number of cases listed: 10
VARIABLE LABLES sdfb1 "Sdfbeta pctmetro"
/sdfb2 "Sdfbeta poverty"
/sdfb3 "Sdfbeta single" .
GRAPH
/SCATTERPLOT(OVERLAY)=sid sid sid WITH sdfb1 sdfb2 sdfb3 (PAIR) BY state(name)
/MISSING=LISTWISE .
| Note | |
| Measure | Value |
| leverage | >(2k+2)/n |
| abs(rstu) | > 2 |
| Cook's D | > 4/n |
| abs(DFBETA) | > 2/sqrt(n) |
PRED Unstandardized predicted values. RESID Unstandardized residuals. DRESID Deleted residuals. ADJPRED Adjusted predicted values. ZPRED Standardized predicted values. ZRESID Standardized residuals. SRESID Studentized residuals. SDRESID Studentized deleted residuals. SEPRED Standard errors of the predicted values. MAHAL Mahalanobis distances. COOK Cook’s distances. LEVER Centered leverage values. DFBETA Change in the regression coefficient that results from the deletion of the ith case. A DFBETA value is computed for each case for each regression coefficient generated by a model. SDBETA Standardized DFBETA. An SDBETA value is computed for each case for each regression coefficient generated by a model. DFFIT Change in the predicted value when the ith case is deleted. SDFIT Standardized DFFIT. COVRATIO Ratio of the determinant of the covariance matrix with the ith case deleted to the determinant of the covariance matrix with all cases included. MCIN Lower and upper bounds for the prediction interval of the mean predicted response. A lowerbound LMCIN and an upperbound UMCIN are generated. The default confidence interval is 95%. The confidence interval can be reset with the CIN subcommand. (See Dillon & Goldstein ICIN Lower and upper bounds for the prediction interval for a single observation. A lowerbound LICIN and an upperbound UICIN are generated. The default confidence interval is 95%. The confidence interval can be reset with the CIN subcommand. (See Dillon & Goldstein
regression /dependent crime /method=enter pctmetro poverty single /residuals=histogram(sdresid lever) id(state) outliers(sdresid, lever, cook) /casewise=plot(sdresid) outliers(2) cook dffit /scatterplot(*lever, *sdresid) /partialplot.
regression /dependent crime /method=enter pctmetro poverty single.
Coefficients(a) Unstandardized Coefficients Standardized Coefficients Model B Std. Error Beta t Sig. 1 (Constant) -1666.436 147.852 -11.271 .000 pct metropolitan 7.829 1.255 .390 6.240 .000 pct poverty 17.680 6.941 .184 2.547 .014 pct single parent 132.408 15.503 .637 8.541 .000 a. Dependent Variable: violent crime rate
compute filtvar = (state NE "dc"). filter by filtvar. regression /dependent crime /method=enter pctmetro poverty single .
Coefficients(a) Unstandardized Coefficients Standardized Coefficients Model B Std. Error Beta t Sig. 1 (Constant) -1197.538 180.487 -6.635 .000 pct metropolitan 7.712 1.109 .565 6.953 .000 pct poverty 18.283 6.136 .265 2.980 .005 pct single parent 89.401 17.836 .446 5.012 .000 a. Dependent Variable: violent crime rate
e.g., 2
redirected from . . . [wiki:MultipleRegression#s-4 multiple regression].
attachment:elemapi2.sav
attachment:r.api00.OutlierDetection.sps
inspection
![plot spred by sresid)]]
===== Outlier dection =====
Let's say, we decide to opt out cases whose studentized deleted residual value exceed normal. We set the criterion as ABS(sdresid) > 2. These cases which meet this criterion will filtered out.
We need to save some residual statistics first, with regression method. Saved values include:
PRED
ZPRED
MAHAL
COOK
LEVER
RESID
ZRESID
SDRESID
DFBETA
Among them, we take a look at SDRESID, whose variable name will be SDR_1 in spss data set.
For the referece,
| Note: outlier detection |||
|Measure|Value | |
|leverage | >(2k+2)/n | 0.021108179 |
|abs(rstu) | > 2 | 2 |
|Cook's D | > 4/n | 0.01055409 |
|abs(DFBETA) | > 2/sqrt(n) | 0.102733099 |
<WRAP clear />
<code>REGRESSION
/DESCRIPTIVES MEAN STDDEV CORR SIG N
/MISSING LISTWISE
/STATISTICS COEFF OUTS R ANOVA COLLIN TOL CHANGE ZPP
/CRITERIA=PIN(.05) POUT(.10)
/NOORIGIN
/DEPENDENT api00
/METHOD=ENTER meals ell acs_k3 avg_ed
/residuals=histogram(sdresid lever) id(snum) outliers(sdresid, lever, cook) Durbin
/casewise=plot(sdresid) outliers(2) cook dffit
/SCATTERPLOT=(*ZRESID ,*ZPRED)
/SAVE PRED ZPRED MAHAL COOK LEVER RESID ZRESID SDRESID DFBETA.
</code>
Then, we need to filter out cases whose SDR_1 value exceed:
abs(SDR_1) > 2
with the below command.
<code>USE ALL.
COMPUTE filterVar=(abs(SDR)_1 < 2).
FILTER BY filterVar.
EXECUTE.
</code>
Then, we do regression again, excluding the suspicious cases. But, this time we do not save the residuals.
<code>REGRESSION
/DESCRIPTIVES MEAN STDDEV CORR SIG N
/MISSING LISTWISE
/STATISTICS COEFF OUTS R ANOVA CHANGE ZPP
/CRITERIA=PIN(.05) POUT(.10)
/NOORIGIN
/DEPENDENT api00
/METHOD=ENTER ell avg_ed acs_k3 meals
/SCATTERPLOT=(*ZRESID ,*ZPRED) .
</code>
Compare the ouptput between the previous and this regression.
| Model Summaryb ||||||||||
|Model | R | R[[br]]Square | Adjusted[[br]]R Square | Std. Error of[[br]]the Estimate | Change[[br]]Statistics | | | | |
| | | | | | R Square Change | F Change | df1 | df2 | Sig. F Change |
|1 | .938a | .880 | .879 | 49.914 | .880 | 649.458 | 4 | 353 | .000 |
| | | ANOVAb |||||
|Model | | Sum of[[br]]Squares | df | Mean[[br]]Square | F | Sig. |
|1 | Regression | 6472284.822 | 4 | 1618071.206 | 649.458 | .000a |
| | Residual | 879470.664 | 353 | 2491.418 | | |
| | Total | 7351755.486 | 357 | | | |
| Coefficientsa ||||||||||
|Model | | Unstandardized[[br]]Coefficients | | Standardized[[br]]Coefficients | t | Sig. | Correlations | | |
| | | B | Std. Error | Beta | | | Zero-order | Partial | Part |
|1 | (Constant) | 705.495 | 51.072 | | 13.814 | .000 | | | |
| | ell | -.915 | .170 | -.160 | -5.374 | .000 | -.789 | -.275 | -.099 |
| | avg_ed | 25.661 | 6.061 | .138 | 4.234 | .000 | .809 | .220 | .078 |
| | acs_k3 | 4.452 | 2.127 | .040 | 2.093 | .037 | .204 | .111 | .039 |
| | meals | -3.056 | .171 | -.683 | -17.868 | .000 | -.928 | -.689 | -.329 |
{{tag>statistics "multiple regression" regression "research methods"](/wiki/_detail/r.api.regression.predbyresi.01.jpg?id=outliers "plot spred by sresid)]]
===== Outlier dection =====
Let's say, we decide to opt out cases whose studentized deleted residual value exceed normal. We set the criterion as ABS(sdresid) > 2. These cases which meet this criterion will filtered out.
We need to save some residual statistics first, with regression method. Saved values include:
PRED
ZPRED
MAHAL
COOK
LEVER
RESID
ZRESID
SDRESID
DFBETA
Among them, we take a look at SDRESID, whose variable name will be SDR_1 in spss data set.
For the referece,
| Note: outlier detection |||
|Measure|Value | |
|leverage | >(2k+2)/n | 0.021108179 |
|abs(rstu) | > 2 | 2 |
|Cook's D | > 4/n | 0.01055409 |
|abs(DFBETA) | > 2/sqrt(n) | 0.102733099 |
<WRAP clear />
<code>REGRESSION
/DESCRIPTIVES MEAN STDDEV CORR SIG N
/MISSING LISTWISE
/STATISTICS COEFF OUTS R ANOVA COLLIN TOL CHANGE ZPP
/CRITERIA=PIN(.05) POUT(.10)
/NOORIGIN
/DEPENDENT api00
/METHOD=ENTER meals ell acs_k3 avg_ed
/residuals=histogram(sdresid lever) id(snum) outliers(sdresid, lever, cook) Durbin
/casewise=plot(sdresid) outliers(2) cook dffit
/SCATTERPLOT=(*ZRESID ,*ZPRED)
/SAVE PRED ZPRED MAHAL COOK LEVER RESID ZRESID SDRESID DFBETA.
</code>
Then, we need to filter out cases whose SDR_1 value exceed:
abs(SDR_1) > 2
with the below command.
<code>USE ALL.
COMPUTE filterVar=(abs(SDR)_1 < 2).
FILTER BY filterVar.
EXECUTE.
</code>
Then, we do regression again, excluding the suspicious cases. But, this time we do not save the residuals.
<code>REGRESSION
/DESCRIPTIVES MEAN STDDEV CORR SIG N
/MISSING LISTWISE
/STATISTICS COEFF OUTS R ANOVA CHANGE ZPP
/CRITERIA=PIN(.05) POUT(.10)
/NOORIGIN
/DEPENDENT api00
/METHOD=ENTER ell avg_ed acs_k3 meals
/SCATTERPLOT=(*ZRESID ,*ZPRED) .
</code>
Compare the ouptput between the previous and this regression.
| Model Summaryb ||||||||||
|Model | R | R[[br]]Square | Adjusted[[br]]R Square | Std. Error of[[br]]the Estimate | Change[[br]]Statistics | | | | |
| | | | | | R Square Change | F Change | df1 | df2 | Sig. F Change |
|1 | .938a | .880 | .879 | 49.914 | .880 | 649.458 | 4 | 353 | .000 |
| | | ANOVAb |||||
|Model | | Sum of[[br]]Squares | df | Mean[[br]]Square | F | Sig. |
|1 | Regression | 6472284.822 | 4 | 1618071.206 | 649.458 | .000a |
| | Residual | 879470.664 | 353 | 2491.418 | | |
| | Total | 7351755.486 | 357 | | | |
| Coefficientsa ||||||||||
|Model | | Unstandardized[[br]]Coefficients | | Standardized[[br]]Coefficients | t | Sig. | Correlations | | |
| | | B | Std. Error | Beta | | | Zero-order | Partial | Part |
|1 | (Constant) | 705.495 | 51.072 | | 13.814 | .000 | | | |
| | ell | -.915 | .170 | -.160 | -5.374 | .000 | -.789 | -.275 | -.099 |
| | avg_ed | 25.661 | 6.061 | .138 | 4.234 | .000 | .809 | .220 | .078 |
| | acs_k3 | 4.452 | 2.127 | .040 | 2.093 | .037 | .204 | .111 | .039 |
| | meals | -3.056 | .171 | -.683 | -17.868 | .000 | -.928 | -.689 | -.329 |
{{tag>statistics \"multiple regression\" regression \"research methods\"")
outliers.1462313111.txt.gz · Last modified: 2016/05/04 06:35 by hkimscil