Trap Zone 1
Experiment vs. Observational Study
The #1 Most-Missed Distinction
The word "random" appears in BOTH experiments and observational studies — it doesn't tell you which one you have.
The only question that matters: Did the researcher assign the treatment, or did subjects already have it?
If the researcher assigned it → Experiment. If subjects just happened to have it → Observational.
01
A university randomly selects 200 students from its enrollment list and surveys them. It finds that students who sleep more than 8 hours score higher on exams. A classmate says, "This proves that sleeping more causes better grades." Is this conclusion valid?
⚠ Watch: "randomly selects" sounds scientific. Does that make this an experiment?
Explanation
Random selection makes a good sample survey — but the researcher never assigned anyone to sleep more. Students chose their own sleep habits. This is an observational study, and observational studies can only show correlation (association), never causation.
Options A and B both use the word "random" as proof of validity. Random sampling ≠ causal proof. Only a controlled experiment where sleep time is assigned can establish cause-and-effect.
Observational study → correlation only. Experiment (assigned treatment) → can claim causation.
02
A gym assigns members to one of two groups using a random number generator: Group X follows a new high-intensity program; Group Y follows the standard program. After 12 weeks, muscle gain is measured. A student says this is an observational study because the gym just watched their progress. Is the student correct?
Explanation
The defining feature of an experiment is that the researcher randomly assigns subjects to treatment groups. The gym assigned members to programs — that's treatment assignment. "Watching progress" doesn't make it observational.
Option A is the trap: "they only measured results." Measuring outcomes is what every study does. The type of study is determined by how subjects were placed into groups, not how results were collected.
Random assignment of treatment → Experiment, even if you're "just watching" the outcomes afterward.
03
Two studies are conducted on the same topic — whether a new medication reduces headaches:
Study A: Researchers review hospital records of 1,000 patients who already chose to take the medication, comparing their outcomes to patients who did not.
Study B: Researchers randomly assign 500 patients to receive the medication and 500 to receive a placebo. Neither group knows which they received.
Study B: Researchers randomly assign 500 patients to receive the medication and 500 to receive a placebo. Neither group knows which they received.
Which study can more reliably conclude the medication causes headache reduction?
Explanation
Study A is observational — patients who chose the medication may differ in other ways (age, severity of headaches, other habits). These confounding variables make it impossible to isolate the drug's effect. Study B's random assignment distributes confounders evenly across both groups, making it possible to attribute differences to the medication alone.
Option A uses sample size as the deciding factor. Bigger samples reduce sampling error — but they do NOT eliminate confounding. A 1,000-person observational study is still weaker than a 100-person randomized experiment for causal claims.
04
In a randomized experiment, what is the purpose of the control group?
Explanation
The control group receives no treatment (or a placebo) and acts as the baseline. Without it, you'd have nothing to compare the treatment's effect against. You cannot know if a treatment "worked" unless you know what happens without it.
Option C uses the word "control" differently — "controlling" the randomization process. The control group is not about controlling selection; it's about receiving no treatment.
Experimental group = gets treatment. Control group = gets nothing (or placebo). Results are compared between the two.
Trap Zone 2
Stratified vs. Cluster Sampling
The Most Confusing Pair on Every Exam
Both methods divide the population into groups first. The difference is what you do next.
Stratified: pick a random sample from every group → members of each group are represented.
Cluster: randomly pick some whole groups → everyone in those groups is included, others are excluded entirely.
Ask: "Is someone from every group in the sample?" If yes → Stratified. If only some groups are represented → Cluster.
05
A hospital has 4 departments: Surgery, Pediatrics, Oncology, and Emergency. To survey staff satisfaction, a researcher randomly selects 10 doctors from each department. Which method is this?
⚠ Both Stratified and Cluster divide into departments first.
Explanation
10 doctors are randomly selected from each of the 4 departments → Stratified. All 4 groups are represented in the final sample.
Option A is the classic mistake. Cluster also divides into groups — but in cluster sampling, you'd randomly pick some departments (e.g., only Pediatrics and Emergency) and survey everyone in them. Here, all departments are sampled.
06
A school has 30 homeroom classes. A researcher randomly selects 3 homeroom classes and surveys every student in those 3 classes. Students in the other 27 classes are not surveyed. Which method is this?
Explanation
Homerooms are the clusters. 3 are randomly selected, and every student in those 3 is surveyed. Students in other classes are completely excluded → Cluster sampling.
Option A is a trap because homerooms could be considered "strata." But in stratified, you'd randomly pick a FEW students from EACH of the 30 classes. Here, 27 classes are completely ignored — that's the cluster signature.
Stratified = few from ALL groups. Cluster = ALL from FEW groups.
07
A researcher wants to survey opinions in a city. She divides the city into 50 neighborhoods. She then randomly selects 8 neighborhoods and randomly selects 20 residents from each of those 8 neighborhoods. What method is this?
💡 This one has two stages — think carefully about each step.
Explanation
Stage 1: Randomly select 8 clusters (neighborhoods) out of 50 — this is cluster logic.Stage 2: Randomly select 20 individuals from within each chosen cluster — this is SRS within the cluster.
This is called two-stage (or multi-stage) cluster sampling. Not everyone in the chosen neighborhoods is surveyed — only a random subset.
Option A (pure cluster) would mean surveying ALL residents of the 8 neighborhoods. Option B (stratified) would require sampling from ALL 50 neighborhoods. This question tests whether you catch both stages.
Trap Zone 3
Sources of Bias — The Subtle Ones
Bias Appears in 3 Ways
(1) Sampling bias — who is (or isn't) included in the sample.
(2) Response bias — how the question is worded influences the answer.
(3) Voluntary response bias — only people with strong feelings respond.
Teachers love to write scenarios that look like one type but are actually another.
08
A study on national reading habits mails surveys to households — but only households with a mailing address on file. Homeless individuals and people in rural areas without registered addresses are excluded. What type of bias is this?
Explanation
Undercoverage happens when some groups in the population have no chance of being selected. Homeless individuals and unregistered rural residents cannot receive a mailed survey — they are systematically excluded from the sampling frame itself.
Option B (voluntary response) is tempting — people do choose to return the survey. But voluntary response bias is about self-selection after being reached. Undercoverage occurs before anyone is asked — these groups never even receive the survey.
Undercoverage = excluded before the study starts. Voluntary response = chose not to respond after being invited.
09
A company surveys its customers by posting a link on its social media page: "How satisfied are you with our service?" Only customers who actively follow the company page see the survey. Which type of bias is most significant here?
⚠ There might be TWO types of bias present — identify the most significant one.
Explanation
Two biases compound here:1. Undercoverage — customers who don't follow social media are never exposed to the survey.
2. Voluntary response — among those who see it, only customers with strong opinions (very satisfied or very unhappy) bother clicking and responding.
Option B catches undercoverage but misses the voluntary response layer. Option D is a trap: "followers are the target" — but the company wants to know about ALL customers, not just social-media-active ones.
10
Two surveys ask the same question about screen time in teens. Compare:
Survey 1: "How many hours per day do you spend on your phone?"
Survey 2: "Research shows excessive phone use harms brain development. How many hours per day do you waste on your phone?"
Survey 2: "Research shows excessive phone use harms brain development. How many hours per day do you waste on your phone?"
Which survey is likely to produce more biased results, and why?
Explanation
Survey 2 has response bias from two elements: (1) the phrase "research shows...harms brain development" creates social pressure, and (2) the word "waste" is judgmental and loaded. Together, these push respondents to underreport screen time to appear less harmful.
Option D is a trap: "accurate scientific research" — factual accuracy does NOT neutralize bias. Even true statements can create response bias when included in question wording.
Trap Zone 4
Systematic Sampling · Self-Selected · Convenience
Three Easy-to-Confuse Methods
Systematic: start at a random point, then use a fixed rule (every nth). Has a random START.
Self-Selected: individuals volunteer themselves — researcher doesn't reach out.
Convenience: researcher picks whoever is easiest/nearby — researcher does the selecting, just lazily.
Key difference: In convenience, the researcher chooses. In self-selected, the subject chooses.
11
A researcher lists all 500 employees of a company. She randomly selects employee #4 as her starting point, then selects every 10th employee after that. Which sampling method is this, and approximately how many employees are selected?
Explanation
Random start (employee #4) + fixed interval (every 10th) = Systematic sampling. Sequence: 4, 14, 24, 34, ... up to ≤500. Number selected: approximately \(500 ÷ 10 = \mathbf{50}\) employees.
Option A (Simple Random) is appealing because "randomly selects the start." But SRS requires every individual to have an equal and independent chance of selection throughout the process — not just at the start. After employee #4 is chosen, employees #1–3 and #5–9 can no longer be selected.
12
A researcher stands at a mall food court and asks the first 30 people who walk past to fill out a survey. A classmate says this is self-selected sampling because people could refuse to participate. Is the classmate correct?
Explanation
Convenience sampling: the researcher approaches and selects subjects based on ease of access (whoever walks past). Self-selected (voluntary response): the researcher puts out an open invitation and subjects choose on their own to participate — the researcher never approaches individuals.
Options A and B both use logic about "choosing." Yes, subjects can refuse — but that's true of ANY study. The defining question is: who initiates contact? Researcher approaches them → Convenience. Subjects approach the study → Self-selected.
Trap Zone 5
Mixed Scenarios — Identify Everything
13
A study claims: "A survey of 10,000 people found that those who eat breakfast daily earn $12,000 more per year on average." A journalist writes: "Eating breakfast makes you richer." What is wrong with the journalist's statement?
Explanation
A survey is observational — it cannot assign breakfast habits to people. A confounding variable (like having a stable, organized lifestyle) could cause both eating breakfast AND earning more. You cannot conclude breakfast "causes" higher income without a controlled experiment.
Option A: sample size does NOT convert correlation into causation. A study of 10 million people is still observational if no treatment was assigned. Large ≠ causal.
No matter how large an observational study is, it cannot prove causation. Period.
14
A researcher wants to study student opinions on a new school policy. She emails every student in the school. Only 38 out of 600 students respond. Which of the following BEST describes the main concern with this study?
Explanation
All 600 students were invited — so there's no undercoverage. But only 38 (6.3%) responded. Those who chose to reply are likely students who feel strongly about the policy (very for or very against). The silent majority is unrepresented → voluntary response bias.
Option A (undercoverage) is the trap: undercoverage = groups excluded from the sample frame. Here, every student received the email — they just chose not to respond. That's voluntary response, not undercoverage.
Contacted but didn't respond → voluntary response bias. Never contacted at all → undercoverage.
15
A city wants to estimate the average household income in its 8 districts. Due to budget limits, surveyors can only visit 2 of the 8 districts. They randomly select districts 3 and 7, then survey every household in those two districts. Which sampling method is this, and what is a potential weakness?
Explanation
Entire districts (clusters) are randomly chosen and fully surveyed → Cluster sampling. The weakness: if districts 3 and 7 happen to be middle-income areas, the estimate won't reflect the city's full range of incomes. Cluster sampling is efficient but risks cluster-level bias.
Option B: "randomly selected districts = unbiased" — random selection reduces bias but doesn't eliminate it. Two districts out of 8 is a very small cluster sample; the chosen clusters may not represent the diversity of all districts.
16
Which of the following is an example of an experiment — not an observational study?
Explanation
Only option D involves randomly assigning a treatment (low-carb diet). Options A, B, and C all involve watching, recording, or noting — classic observational language. No one in A, B, or C is being told to do anything differently by the researcher.
Option A seems scientific and controlled ("tracks for 6 months") — but tracking is not treating. Tracking = observational. Assigning = experimental.
17
A study finds that children who own more books perform better in school. A parent concludes: "I'll buy my child 100 books and their grades will improve." What logical error is this parent making?
Explanation
The finding is an association (correlation) — both book ownership and performance may be caused by a confounding variable such as family income, parental education, or reading culture at home. Buying books alone doesn't guarantee the confounding variable comes with it.
Option A: "clearly leads to" — this is exactly the error. An observational association, no matter how strong, never "clearly" proves causation without a controlled experiment.
18
A study has bias if it:
Explanation
The key word in the definition of bias is "systematically" — results are skewed consistently in one direction due to a flaw in the study design. A random sample will always differ slightly from the true population value (that's called sampling error, not bias). Bias is a structural problem, not random variation.
Option B describes sampling error — normal, unavoidable, random variation that exists in every sample. Bias is different: it's a consistent, directional distortion. Option D describes every sample ever taken — not being a census doesn't make something biased.
Random variation from truth = Sampling Error (normal). Consistent distortion in one direction = Bias (structural problem).
19
A researcher wants to study whether a new reading program improves literacy. She allows teachers to choose whether their class uses the new program or the old one. After a semester, she compares scores. Why is this study design problematic?
Explanation
A treatment is being applied (new program vs. old), so this is intended to be an experiment. But allowing teachers to choose their group means random assignment is absent. More motivated or skilled teachers might choose the new program — and their students might score higher regardless of the program. This confounding variable makes any conclusion unreliable.
Option A: "scores were recorded, not assigned" — you can never assign scores. Experiments assign treatments. The recording of outcomes is separate from what makes a study an experiment.
20
A school wants to know what percent of students support a longer lunch period. Which approach would produce the MOST reliable estimate with the least bias?
Explanation
Option D is a Simple Random Sample — every student has an equal chance of selection, minimizing systematic bias.A → Voluntary response (only engaged/strong-opinion students respond)
B → Convenience bias (cafeteria students may be the ones who want longer lunch most)
C → The teachers are reporting, not the students — this adds a layer of inaccuracy and doesn't represent student opinion directly.
Option C seems "fair" because it covers all homerooms (like stratified sampling). But teachers guessing or approximating student opinions is NOT the same as students self-reporting — it introduces a different kind of bias (inaccurate proxy reporting).
For least bias: Simple Random Sample of the actual population being studied, using direct self-reporting.
—
— / 20
—