8 · Research designs I
Wednesday, October 14, 2026
What you should be able to do after this meeting. Draw the graph for an experiment, an instrument and a discontinuity. Say whether any of the three is open to your question.
Handouts
Where to read more
Remix ch. 4 on randomization, ch. 6 on regression discontinuity, ch. 7 on instrumental variables. In Mostly Harmless, the same three are ch. 2, 6 and 4.
Session A · 2:00–3:15
Three designs, each drawn before it is described. A research design is the plan for which units you compare and where the variation in their treatment comes from. The identification strategy is the argument that this variation is as good as random. LaLonde’s job-training study shows why the design matters more than the model: against survey comparison groups the standard estimators gave anything from −$15,578 to +$1,714, while the randomized comparison gave +$886. Experiments first, opened by eBay, which switched its Google search ads off for 60 days in 68 of 210 US market areas picked by a randomized draw. A regression had implied a return above 4,100%, and the experiment found −63%. A simulated retention email shows randomization recovering the true effect where targeting overstates it twentyfold. Then the decisions that make an experiment work: the unit you randomize, and the interference that breaks it, shown by an eBay reminder email that only moved sales between bidders; how many arms the sample affords; checking balance; pre-registration and pre-analysis plans; survey and choice experiments; and non-compliance, through the Oregon Medicaid lottery. Instrumental variables next, opened by the Vietnam draft lottery: relevance, exclusion and independence; a simulated bike-rental market where the fuel price recovers the demand slope; an exclusion violation that no statistic detects; weak instruments; and the compliers the estimate is about. Then regression discontinuity, opened by Yelp’s half-star rounding: continuity at the cutoff, local linear fits and the bandwidth, manipulation and the density test, fuzzy designs, and who the estimate is about.
Session B · 3:30–4:30
Every session B opens with a ten-minute team check-in: what you did since last week, what is stuck, and who is doing what next. Written down, and handed in with that session’s work.
Each team draws a setting nobody in the room owns. A bank with an email list. A scholarship cutoff. An NGO that gave filters to whoever asked. Fifteen minutes to draw the graph, name a design, and state the assumption carrying it. Then teams are drawn at random to defend for four minutes while the room attacks. What breaks this? Nobody knows in advance whether they are up. One card cannot be answered honestly at all, and the team holding it has to be willing to say so. Last: is an experiment, an instrument or a discontinuity actually open to your own project? Most teams find the answer is no. Finding that out today rather than in November is the point.
Your team’s output is submitted before you leave.