schmidt & hunter approach to r bare bones. statistical artifacts extraneous factors that...
TRANSCRIPT
![Page 1: Schmidt & Hunter Approach to r Bare Bones. Statistical Artifacts Extraneous factors that influence observed effect Sampling error* Reliability Range restriction](https://reader035.vdocuments.us/reader035/viewer/2022070404/56649f335503460f94c5024c/html5/thumbnails/1.jpg)
Schmidt & Hunter Approach to rBare Bones
![Page 2: Schmidt & Hunter Approach to r Bare Bones. Statistical Artifacts Extraneous factors that influence observed effect Sampling error* Reliability Range restriction](https://reader035.vdocuments.us/reader035/viewer/2022070404/56649f335503460f94c5024c/html5/thumbnails/2.jpg)
Statistical Artifacts
Extraneous factors that influence observed effectSampling error*
Reliability
Range restriction
Computational error
Dichotomization of variables
*addressed in the (bare-bones) analysis
![Page 3: Schmidt & Hunter Approach to r Bare Bones. Statistical Artifacts Extraneous factors that influence observed effect Sampling error* Reliability Range restriction](https://reader035.vdocuments.us/reader035/viewer/2022070404/56649f335503460f94c5024c/html5/thumbnails/3.jpg)
Bare Bones r
Find weighted mean and variance:
Note sample size weight.
Note that for unit weights, the weighted variance estimator is the sample, not population, estimate.
i
ii
N
rNr
i
iir N
rrNs
22 )(
![Page 4: Schmidt & Hunter Approach to r Bare Bones. Statistical Artifacts Extraneous factors that influence observed effect Sampling error* Reliability Range restriction](https://reader035.vdocuments.us/reader035/viewer/2022070404/56649f335503460f94c5024c/html5/thumbnails/4.jpg)
Confidence Interval for Mean
i
iir N
rrNs
22 )(
k
srCI r
2
96.1%95
i
ii
N
rNr
There are k studies, with Ni observations.
This is not the only formula they use, but it’s the best one IMHO.
![Page 5: Schmidt & Hunter Approach to r Bare Bones. Statistical Artifacts Extraneous factors that influence observed effect Sampling error* Reliability Range restriction](https://reader035.vdocuments.us/reader035/viewer/2022070404/56649f335503460f94c5024c/html5/thumbnails/5.jpg)
Estimated Sampling Error Variance
The variance of r
Nr
222 )1(
1
)ˆ1(ˆ
222
N
iei
1
)1(ˆ
222
N
re
Estimated variance for a study.
Estimated sampling variance for a meta-analysis. Note mean r is constant. This is the variance of sampling error we expect if all the studies have a common effect estimated by r-bar.
![Page 6: Schmidt & Hunter Approach to r Bare Bones. Statistical Artifacts Extraneous factors that influence observed effect Sampling error* Reliability Range restriction](https://reader035.vdocuments.us/reader035/viewer/2022070404/56649f335503460f94c5024c/html5/thumbnails/6.jpg)
Variance of Rho
ETX 222ETX
er 222er
222er
Classical Test Theory
Sampling Error
A definition
![Page 7: Schmidt & Hunter Approach to r Bare Bones. Statistical Artifacts Extraneous factors that influence observed effect Sampling error* Reliability Range restriction](https://reader035.vdocuments.us/reader035/viewer/2022070404/56649f335503460f94c5024c/html5/thumbnails/7.jpg)
Estimated Variance of rho
222 ˆˆ ers
2ˆ
i
ii
N
rrN 2)(
1
)1( 22
N
r-
To find the variance of infinite-sample correlations, find the variance of r in the meta-analysis and subtract expected sampling error variance. Schmidt would be quick to add that part of the estimated variance of infinite-sample correlations is artifactual.
Note that the variance of rho will be called tau-squared by Hedges
![Page 8: Schmidt & Hunter Approach to r Bare Bones. Statistical Artifacts Extraneous factors that influence observed effect Sampling error* Reliability Range restriction](https://reader035.vdocuments.us/reader035/viewer/2022070404/56649f335503460f94c5024c/html5/thumbnails/8.jpg)
Credibility Interval
96.1%95 rCRThe credibility interval and the confidence interval are quite different things. The CI is a standard statistical estimate (intended to contain rho, or average of rho). The CR is intended to contain a percentage of the values of a random variable – infinite-sample effect sizes. The S&H value forgets that there is also uncertainty in the mean value; the two should be added. There are Bayesian programs that will do this; there is also an approximation called the prediction interval described in Borenstein et al.
![Page 9: Schmidt & Hunter Approach to r Bare Bones. Statistical Artifacts Extraneous factors that influence observed effect Sampling error* Reliability Range restriction](https://reader035.vdocuments.us/reader035/viewer/2022070404/56649f335503460f94c5024c/html5/thumbnails/9.jpg)
Bare-Bones Example (1)Study Ni r
1 200 .20
2 100 .20
3 150 .40
4 80 .40
Mean 132.5 .30 <- Unit weighted mean
![Page 10: Schmidt & Hunter Approach to r Bare Bones. Statistical Artifacts Extraneous factors that influence observed effect Sampling error* Reliability Range restriction](https://reader035.vdocuments.us/reader035/viewer/2022070404/56649f335503460f94c5024c/html5/thumbnails/10.jpg)
Bare-Bones Example (2)
r Ni rNi
.20 200 40
.20 100 20
.40 150 60
.40 80 32
sum 530 152
![Page 11: Schmidt & Hunter Approach to r Bare Bones. Statistical Artifacts Extraneous factors that influence observed effect Sampling error* Reliability Range restriction](https://reader035.vdocuments.us/reader035/viewer/2022070404/56649f335503460f94c5024c/html5/thumbnails/11.jpg)
BB Example (3)
2868.530
152
i
ii
N
rNr
Recall unwighted or unit weighted mean = .30.Why are they different?
![Page 12: Schmidt & Hunter Approach to r Bare Bones. Statistical Artifacts Extraneous factors that influence observed effect Sampling error* Reliability Range restriction](https://reader035.vdocuments.us/reader035/viewer/2022070404/56649f335503460f94c5024c/html5/thumbnails/12.jpg)
BB Example (4)r Ni
.20 200 1.507
.20 100 .753
.40 150 1.922
.40 80 1.025
Sum 530 5.208
2)( rrN i
![Page 13: Schmidt & Hunter Approach to r Bare Bones. Statistical Artifacts Extraneous factors that influence observed effect Sampling error* Reliability Range restriction](https://reader035.vdocuments.us/reader035/viewer/2022070404/56649f335503460f94c5024c/html5/thumbnails/13.jpg)
BB Example (5)
009826.
530
208.5)( 22
i
iir N
rrNs
2868.r 5.132N
006405.15.132
)2868.1(
1
)1(ˆ
22222
N
re
003421.006405.009826.ˆˆ 222 ers
4k
384,.190.4
009826.96.12868.96.1%95
2
k
srCI r
![Page 14: Schmidt & Hunter Approach to r Bare Bones. Statistical Artifacts Extraneous factors that influence observed effect Sampling error* Reliability Range restriction](https://reader035.vdocuments.us/reader035/viewer/2022070404/56649f335503460f94c5024c/html5/thumbnails/14.jpg)
Interpretation
Schmidt says this is a random-effects meta-analysis. It uses a sample of studies to represent a larger population of studies.
People interpret the Credibility Interval, but typically do not recognize that it is poorly estimated.