6 · Models and uncertainty
Wednesday, September 30, 2026
What you should be able to do after this meeting. Distinguish a parameter, an estimator and an estimate, and explain what a standard error measures. Compare estimators by their bias, efficiency and consistency. Choose a model and an estimation method for your outcome, and interpret its coefficients in the units of the problem. Read a confidence interval and a p-value correctly, and say whether your question is predictive or causal.
Handouts
Where to read more
Adams ch. 11 on descriptive statistics and ch. 12 on correlation and regression.
Session A · 2:00–3:15
Most projects in this course estimate an unknown parameter: a price premium, an elasticity, a probability, an effect. This session covers how that estimate is obtained from a sample and how its uncertainty is measured. Live simulations from a population with a known mean show the sampling distribution, the standard error, and why the standard error falls with the square root of the sample size. The German tank problem compares three estimators by their bias, efficiency and consistency. The Literary Digest poll opens the cases a larger sample does not fix: bias, dependent observations, little variation in the regressor, and heavy tails. The session then covers models for continuous, binary and count outcomes, and three estimation methods: least squares, maximum likelihood and the method of moments (GMM). It ends with standard errors from formulas and from the bootstrap, a p-value computed by permutation, confidence intervals read against the smallest effect that matters, and the difference between prediction and causal inference.
Session B · 3:30–4:30
Every session B opens with a ten-minute team check-in: what you did since last week, what is stuck, and who is doing what next. Written down, and handed in with that session’s work.
Ten minutes of simulation first, driven from the front. Draw from a world you control. Watch the p-value distribution under the null. Move the sample size and the variation, and watch it shift. Then fit the same world in and out of sample, and watch an excellent-looking model fail on data it has not seen. Nothing to install. Then the cards. Each team gets an analyst who ran something and said something about it, and every one overclaims differently. A count outcome fitted by OLS and predicted below zero. A 0.03% lift called highly significant across two million users. A wide interval called no effect. An elasticity estimated off two price changes in three years. A churn probability of 112%. Work out what they are actually entitled to claim. Then a team is drawn at random to defend that while the room pushes back. The last line on the sheet is the one that matters: is your own project at risk of the same error?
Your team’s output is submitted before you leave.
Due
Rate your teammates
Practice questions
These 10 questions practice the ideas of this lecture. Click an option to check it. A wrong pick lets you try again, and the right one links to the slide that covers it. The quiz on this lecture asks different questions on the same ideas.
1. Each quarter, mystery shoppers visit 40 of a wireless carrier's 600 stores, drawn at random. Service across the stores has not changed in two years, yet the quarterly average score has ranged from 7.6 to 8.2 out of 10. What explains the movement?
2. A market research firm has two unbiased estimators of average weekly streaming hours per household. With 100 households, the diary-only estimator has a standard error of 0.6 hours, and the one that adds set-top box data has 0.3. Both standard errors fall with the square root of n. How many households would the diary-only estimator need to reach 0.3?
3. A bank estimates each branch's default rate as a weighted average of the branch's observed default rate and the national rate. The weight on the national rate is 20/(n + 20), where n is the branch's number of loans. What happens to this estimator as n grows very large?
4. A consulting firm surveys its staff about padding billable hours. Each person privately flips a coin: heads, answer yes; tails, answer truthfully. The firm doubles the share answering yes and subtracts 100%. Across many surveys this estimator averages the true share. This year 48% said yes, giving an estimate of −4%. What should the firm conclude?
5. A hotel chain forecasts nightly room demand from 60 predictors and two years of daily data. An analyst prefers least squares to ridge regression, which shrinks coefficients toward zero, because least squares is unbiased. What is wrong with the argument?
6. A hospital system budgets next year's total cost of care for about 8,000 uninsured patients. Cost per patient is right-skewed: most costs are small and a few are very large. Which statistic of cost per patient should the budget build on?
7. A manufacturer ranks its plants each year by mean workers' compensation cost per injury. Plants have 10 to 80 injuries a year, mostly minor, but one severe injury can triple a plant's mean, and the ranking reshuffles every year. Which change would make the yearly ranking least sensitive to single cases?
8. A fitness-studio franchisor wants to tell prospective owners the profit a new studio can expect in its first year. It has opened 900 studios, and 600 are still open. Studios that lose money usually close within two years. Which figure should it report?
9. A beverage distributor regresses log(cases ordered + 1) for each bar account and week on a promotion indicator by OLS. Many accounts order nothing in a given week. An analyst reruns it counting bottles instead of cases, at 24 bottles per case. What happens to the promotion coefficient?
10. A utility mails home energy reports to 400 of 800 households chosen at random and compares their electricity use the next month with the other 400. An analyst wants the differences in mean use that would arise if the reports had no effect. How should she simulate them?