11.2 Statistics · HSS.IC.A.1 · HSS.IC.B.3 · HSS.IC.B.6

Statistical Studies &
Sampling Methods

20 tricky questions designed to challenge your understanding. Read each scenario carefully — the devil is in the details.

Score: 0 / 20  ·  Answered: 0 / 20

⚡ Quick Memory Points — Memorize These First

ASSIGN = Experiment Researcher randomly assigns treatment to groups
ASK = Survey You ask people questions — no treatment given
OBSERVE = Observational Watch without interfering — subjects already affected
EQUAL CHANCE = Simple Random Everyone has same probability of selection
GROUPS → pick from each = Stratified Divide by trait, random from each stratum
GROUPS → pick whole = Cluster Randomly pick entire cluster (not individuals)
EASY = Convenience ⚠️ Nearby or accessible — high bias risk
RULE (every nth) = Systematic Start random, then apply fixed interval rule
VOLUNTEER = Self-Selected ⚠️ Participants choose themselves — extreme bias
BIAS = Misrepresents population One group more likely to be included than another
Section 1 — Types of Studies

Experiment vs. Survey vs. Observational

Q 01
Hard
A nutritionist monitors the eating habits of 200 adults over six months and records their cholesterol levels. No dietary changes are suggested. What type of study is this?
📖 Explanation
Key word: "monitors" and "no dietary changes suggested." In an observational study, researchers record what is already happening without assigning any treatment. If the nutritionist had told some people to change their diet, it would be an experiment. The fact that participants are already eating a certain way (already "affected") and nothing is changed makes this observational.
Q 02
Tricky
A school sends home a questionnaire asking parents whether they support a new homework policy. Parents fill it out and return it if they choose to. Which TWO issues apply?
Select the answer that correctly identifies both the study type AND the sampling method.
📖 Explanation
This is a Sample Survey because parents are being asked questions (no treatment assigned). The sampling method is Self-Selected because parents choose whether to respond. This creates extreme bias — only parents with strong opinions (usually those who strongly oppose or strongly support) tend to respond. Don't confuse this with Convenience sampling, which is about proximity, not voluntary participation.
Q 03
Hard
Researchers split 300 patients randomly into two groups. Group A takes a new blood pressure drug. Group B takes a sugar pill (placebo). After 3 months, blood pressure readings are compared. What is the control group?
📖 Explanation
The control group does not receive the real treatment — it's the baseline for comparison. Group B (placebo) is the control group. Group A is the experimental group. Note: the control group doesn't have to receive "nothing" — a placebo counts. The purpose is to isolate whether the treatment (drug) caused any difference by comparing to a group without it.
Q 04
Tricky
A biologist counts how many birds visit a feeder each morning for two weeks. She does not interact with the birds or change anything. This is MOST similar to which example from your notes?
📖 Explanation
This mirrors the wildlife researcher example exactly. Key signals: recording/counting natural behavior, no intervention, subjects (birds) are already behaving naturally. The researcher is purely observing. This is a classic Observational Study.
Q 05
Hard
A doctor notices that patients who drink more coffee seem to have higher anxiety. She publishes a report stating: "Coffee causes anxiety." What is the fundamental problem with this conclusion?
📖 Explanation
This is one of the most important concepts: observational studies show association, not causation. The doctor observed a pattern but didn't randomly assign coffee consumption — maybe anxious people drink more coffee, or a third factor (like work stress) causes both. Only a randomized experiment can establish cause-and-effect. This is why experiments are the gold standard in research.
Section 2 — Sampling Methods

Identify the Method Correctly

Q 06
Hard
A researcher numbers all 1,200 employees at a company from 001 to 1200. She then uses a random number generator to pick 60 employees to survey. What sampling method is this?
📖 Explanation
Simple Random Sampling: assigning a number to everyone and using a random generator is the textbook definition. Every individual has an equal probability of selection. Don't confuse "using numbers" with Systematic — Systematic requires a fixed interval rule (like every 5th person). Here, the selection is purely random with no interval pattern.
Q 07
Hard
A city divides its population into four age groups: under 18, 18–35, 36–55, over 55. Researchers then randomly select 25 people from each age group for a transportation survey. Which method is this?
Note: 100 people are surveyed total from a city of 80,000.
📖 Explanation
Stratified vs. Cluster is the most commonly confused pair. The key: in Stratified, you divide by a characteristic (age, gender, grade) and pick some individuals from each group. In Cluster, you pick entire groups at random. Here: age groups are formed → individuals randomly selected from each → Stratified. ✓
Q 08
Tricky
A school district has 30 elementary schools. Researchers randomly select 5 schools, then survey every student in those 5 schools. What method is this?
📖 Explanation
Cluster: the schools are the "clusters." The key word is "every student" in the selected schools — the entire cluster is included, not a random sample from each. If the researchers had surveyed only some students from each of the 5 schools, it might be Stratified. But including ALL students from randomly chosen schools = Cluster.
Q 09
Hard
A quality control inspector checks every 10th product coming off an assembly line, starting with product #7 (chosen randomly). What sampling method is this, and why does it start with a random number?
📖 Explanation
Systematic sampling = random start + fixed interval rule. The random start (product #7) ensures the beginning isn't biased, but after that, a rule takes over: check #7, #17, #27, #37… This is NOT Simple Random because the remaining selections are not random — they're determined by the rule once the start is set.
Q 10
Tricky
A reporter surveys people walking past city hall to ask about local government satisfaction. Which method is this, and what bias does it introduce?
📖 Explanation
Convenience sampling: surveying whoever is easy to access. The bias: people near city hall likely work in government or have strong opinions about it — not representative of the whole city. This is different from Self-Selected because the reporter is doing the selecting (approaching people), not participants volunteering themselves.
Section 3 — Bias & Advanced Reasoning

Spot the Flaw — Hardest Questions

Q 11
Hard
A survey asks: "Don't you agree that students deserve longer lunch periods?" This question introduces bias because:
📖 Explanation
This is question wording bias. The phrase "Don't you agree..." is a leading question — it implies the "correct" answer is yes. Bias in surveys can arise not just from who you ask, but how you ask it. A neutral version: "Do you support extending the lunch period?" Both the sample selection AND the question design can create bias.
Q 12
Hard
An online news site posts: "Vote in our poll: Should the speed limit be lowered?" 10,000 people respond. The result: 78% say NO. Why is this result unreliable?
📖 Explanation
Self-selected sampling = greatest chance of biased results. People who feel strongly (e.g., drivers who hate lower speed limits) are far more motivated to respond than those who are indifferent. The 10,000 responses sound impressive but represent a biased subset, not the general population. Larger sample size does NOT fix selection bias.
Q 13
Tricky
Researchers want to study the effect of sleep on test scores. They survey students about their sleep habits and compare scores. Why can they NOT conclude that "more sleep causes higher scores"?
📖 Explanation
Even though lying is a valid concern, the primary statistical reason is that surveys/observational studies cannot establish causation. A confounding variable (like study habits, stress level, or socioeconomic status) might cause both better sleep and better scores. To prove causation, you'd need a randomized experiment where sleep amount is actually controlled and randomly assigned.
Q 14
Hard
A hospital wants to survey patients about their care. They survey every patient in the ICU on Tuesday afternoon. Which type of bias does this introduce?
📖 Explanation
Surveying everyone in a conveniently accessible group (ICU on Tuesday) is Convenience sampling. ICU patients are seriously ill and have a very different care experience than other patients. The sample misrepresents the population (all hospital patients). Even if 100% of ICU patients respond, the sample itself is biased — coverage bias, not response bias.
Q 15
Hard
A researcher randomly selects 5 ZIP codes from a city, then interviews every household in those ZIP codes. Is this Cluster or Stratified sampling?
The city has 40 ZIP codes. 5 are randomly selected. All residents in those 5 are included.
📖 Explanation
Ultimate Stratified vs. Cluster test:
Stratified → divide into groups, select some from each group.
Cluster → select entire groups at random, include everyone.
Here: 5 ZIP codes chosen at random, ALL households included → Cluster. The "randomly chosen" ZIP codes are clusters, not strata.
Q 16
Tricky
In a randomized experiment, why is it important to randomly assign subjects to the experimental and control groups, rather than letting subjects choose?
📖 Explanation
Random assignment is what makes an experiment powerful. If subjects chose their own group, healthier/more motivated people might choose the treatment — making it look like the treatment works when it's actually just healthier people doing better. Random assignment spreads unknown confounders (age, health, lifestyle) approximately equally, isolating the treatment effect.
Q 17
Hard
A student survey at a school: students in the library during 3rd period are asked about study habits. Which of the following statements is TRUE?
📖 Explanation
Students in the library during 3rd period are selected because they are accessible, not because of any random process. This is Convenience sampling. The bias: library students during class time likely have stronger study habits than the average student — severely misrepresenting the school population. It's also a survey (asking questions), not an observational study (just watching).
Q 18
Hard
A gym assigns members to two groups by alphabetical order of last name: A–M gets the new workout plan, N–Z keeps the old one. Muscle gain is measured after 8 weeks. What is the problem?
📖 Explanation
This IS an experiment (treatment is assigned), and there IS a control group (old plan). But the assignment is NOT random. Alphabetical order can correlate with cultural background, which may relate to baseline fitness, diet, or other factors. For a valid experiment, groups must be formed by true random assignment, not any systematic method that could correlate with the outcome.
Q 19
Tricky
A national phone survey calls landline numbers only. Which population would be systematically under-represented?
📖 Explanation
This is an example of coverage bias — the sampling method systematically excludes certain groups. Young adults (especially college students and 18–35 age group) predominantly use mobile phones and don't have landlines. This skews survey results toward older demographics. This is why modern surveys must use multiple contact methods or random digit dialing including mobile numbers.
Q 20
Hard
A researcher studying exercise habits divides a city's population by neighborhood (Downtown, Suburbs, Rural). She randomly selects 3 neighborhoods, then randomly picks 50 residents from each selected neighborhood. What sampling method is this?
This is a two-stage process: first selecting groups, then sampling within those groups.
📖 Explanation
The hardest question! Here's the key: neighborhoods are used as strata (groups defined by a geographic characteristic). From each selected stratum (neighborhood), a random sample of individuals is taken — NOT all residents. This is Stratified sampling. If she had included ALL 50,000 residents of each selected neighborhood, it would be Cluster. The partial selection within each group = Stratified. ✓