Sampling determines whether results can represent a target population. Random sampling methods reduce systematic error, while convenience sampling, nonresponse, and wording effects can introduce bias that no later formula can fully fix.
- A larger sample does not repair a biased sampling process.
- Define the target population before collecting data.
- Document sampling frame, response rate, and exclusions.
Common probability sampling methods
Simple random sampling gives each unit an equal selection chance. Stratified sampling samples within subgroups and can improve precision when groups differ. Cluster sampling samples groups first, often for logistical reasons.
A sampling frame is the operational list used to draw samples. Coverage gaps in that frame can systematically miss part of the target population.
Bias types to check early
Selection bias occurs when sampled units differ systematically from the target population. Nonresponse bias appears when responders differ from nonresponders on key variables.
Measurement bias can arise from leading question wording or inconsistent instruments. These errors can shift conclusions even with careful downstream analysis.
- Separate sampling error from non-sampling error in reports.
- Use pilot surveys to detect confusing items.
- Track who was excluded and why.
Plan a student commute survey
A college wants average one-way commute time for all enrolled students.
- Define target population as all currently enrolled students.
- Build a sampling frame from the registrar list and stratify by program type.
- Draw random samples within each stratum and send identical survey wording.
- Compare response rates by stratum and follow up where response is low.
Common mistakes
- Using volunteer responses as if they were random sample results.
- Ignoring nonresponse differences across subgroups.
- Changing survey wording mid-collection without documentation.
- Generalizing beyond the population actually sampled.
Try one
Why does increasing sample size not solve selection bias?
Because bias is systematic. More data repeats the same distortion more precisely.
Sources
- OpenStax: Introductory StatisticsOpen textbook covering descriptive statistics, probability, inference, and regression fundamentals.
- NIST/SEMATECH e-Handbook of Statistical MethodsAuthoritative reference on probability models, inference procedures, and practical statistical diagnostics.