Design effect

In statistics, the design effect (or estimates of unit variance) is an adjustment used in some kinds of studies, such as ones that use cluster sampling or cluster randomised controlled trial, to allow for the design structure. The adjustment inflates the variance of parameter estimates, and therefore their standard errors, which is necessary to allow for correlations among clusters of observations.[1][2] It is similar to the variance inflation factor and is used in sample size calculations.[3] The term was introduced by Leslie Kish in 1965.[4]

Definition

The design effect is the ratio of two theoretical variances for an estimator:[4][5]

• the actual variance for a given sampling design;
• the variance assuming the same sample size, but using simple random sampling without replacement.

Kish's design effect, for the increase in variance of the sample mean, for when all observations have (at least approximately) the same expectation and variance in the response variable of interest. It also ignores potential variance in the weights themselves:[6][7]

${\displaystyle D_{eff}={\frac {\overline {w^{2}}}{{\overline {w}}^{2}}}={\frac {{\frac {1}{n}}\sum _{i=1}^{n}w_{i}^{2}}{\left({\frac {1}{n}}\sum _{i=1}^{n}w_{i}\right)^{2}}}={\frac {n\sum _{i=1}^{n}w_{i}^{2}}{(\sum _{i=1}^{n}w_{i})^{2}}}}$

Notice that this definition is tightly tied to the coefficient of variation (when using uncorrected sample standard deviation for estimation: ${\displaystyle D_{eff}={C_{V}}^{2}+1}$

[Proof]

${\displaystyle {C_{V}}^{2}={\frac {s_{w}}{\bar {w}}}^{2}={\frac {\frac {\sum _{i=1}^{n}(w_{i}-{\bar {w}})^{2}}{n}}{{\bar {w}}^{2}}}={\frac {\frac {\sum _{i=1}^{n}{w_{i}}^{2}-n{\bar {w}}^{2}}{n}}{{\bar {w}}^{2}}}={\frac {{\overline {w}}^{2}-{\bar {w}}^{2}}{{\bar {w}}^{2}}}={\frac {{\overline {w}}^{2}}{{\bar {w}}^{2}}}-1=D_{eff}-1\implies D_{eff}={C_{V}}^{2}+1}$

For data collected using cluster sampling with m observations in each cluster and intra-cluster correlation of ${\displaystyle \rho }$, the design effect. D_eff, is given by:[8][9]

${\displaystyle D_{\text{eff}}=1+(m-1)\rho .}$

See also

• Effective sample size

References

1. Alexander K. Rowe; Marcel Lama; Faustin Onikpo; Michael S. Deming (2002). "Design effects and intraclass correlation coefficients from a health facility cluster survey in Benin". International Journal for Quality in Health Care. 14 (6): 521–523. doi:10.1093/intqhc/14.6.521.
2. "Glossary - NCES Statistical Standards".
3. Heo, Moonseong; Kim, Yongman; Xue, Xiaonan; Kim, Mimi Y. (2010). "Sample size requirement to detect an intervention effect at the end of follow-up in a longitudinal cluster randomized trial". Statistics in Medicine. 29 (3): 382–390. doi:10.1002/sim.3806. Archived from the original on 2013-01-05.
4. Kish, Leslie (1965). "Survey Sampling". New York: John Wiley & Sons, Inc. ISBN 0-471-10949-5. Cite journal requires |journal= (help)
5. Everitt, B.S. (2002) The Cambridge Dictionary of Statistics, 2nd Edition. CUP. ISBN 0-521-81099-X
6. Kish, Leslie, and J. Official Stat. "Weighting for unequal Pi." (1992): 183-200. pdf link
7. Little, Roderick J., and Sonya Vartivarian. "Does weighting for nonresponse increase the variance of survey means?." Survey Methodology 31.2 (2005): 161. pdf link
8. Bland, M (2005), "Cluster randomised trials in the medical literature", Notes for talks, York Univ
9. Methods in Sample Surveys (pages 5-6)

Further reading

• The Impact of Typical Survey Weighting Adjustments on the Design Effect: A Case Study
• Estimating components of design effects for use in sample design