AEM 6850
Empirical Methods for Applied Economists
Prof. Ariel Ortiz-Bobea
Thursday, October 1, 2026
By the end of this session you can:
renv.Sources: Dewald, Thursby and Anderson (AER, 1986); Chang and Li (CFR, 2022); Herbert, Kingi, Stanchi and Vilhuber (CJE, 2024); Gentzkow and Shapiro (guide, 2014).
Takeaway: the error was found because the authors had posted their data and code.
Sources: Donohue and Levitt (QJE, 2001); Foote and Goetz (QJE, 2008); Donohue and Levitt (QJE, 2008).
Takeaway: the bug sat in the program that built a variable, where only someone rebuilding that variable could find it.
Sources: Feldstein (JPE, 1974); Leimer and Lesnoy (JPE, 1982); Feldstein (JPE, 1982), with the numbers in its working paper, Table 1.
Source of the definitions: National Academies (report, 2019), p. 6.
Sources: McCullough and Vinod (AER, 2003); Bernanke (AER, 2004); McCullough (EAP, 2009); AEA policy; datacodestandard.org.
| Journal | Asks for | When | Checked before publication |
|---|---|---|---|
| AJAE | Data and a README; code where needed | At acceptance | No data editor; partial compliance accepted |
| JAERE | Data, code, output and a README, on its Dataverse | After conditional acceptance | An editor checks the package against a checklist |
| JEEM | Data and code in a repository, cited in the paper, or a statement of why not | Not stated | Not stated |
Table1.tex, intextresults.log.Sources: AJAE author guidelines; JAERE data availability policy; JEEM guide for authors, “Research data”.
Sources: the AEA policy; Vilhuber (AER P&P, 2026), Table 2.
Takeaway: about 1 paper in 19 passes the Data Editor’s check with no changes.
A replication package is the data, code and README published with a paper so that others can reproduce its results. Below, a package for a two-page paper on income and life expectancy.
income-life/
├── README.pdf
├── income-life.Rproj
├── renv.lock
├── data/
│ └── gapminder.csv
├── code/
│ ├── run_everything.R
│ ├── 01_table1.R
│ ├── 02_figure1.R
│ └── 03_numbers.R
├── output/
│ ├── table1.tex
│ ├── figure1.pdf
│ └── numbers.tex
└── paper/
└── paper.tex
README.pdf: AEA journals ask for a PDF.data/: the data, or how to get them.run_everything.R: every script, in order.output/: every table, figure and number in the paper.renv.lock: R and package versions.| Section | What it says |
|---|---|
| Data availability and provenance | Where each dataset comes from, whether you may share it, how to get it |
| Dataset list | Each data file, its source, and whether it is in the package |
| Computational requirements | Software and versions, random seeds, memory, running time |
| Description of programs | What each script does; the license for the code |
| Instructions to replicators | The steps, in order |
| List of tables and programs | Which script makes each table and figure, and the file it writes |
| References | A citation for every dataset |
Start from the template README written by the data editors of economics journals; the AEA policy says its use “is strongly encouraged”.
The README section that says who made each dataset, under which license, where your file came from and what you did to it, and whether it is in the package. For the draft project:
The data are the Gapminder excerpt distributed in the R package
gapminder(Bryan 2025, doi:10.32614/CRAN.package.gapminder), released under CC0. The underlying data come from the Gapminder Foundation (gapminder.org/data) under a CC BY 4.0 license. The filedata/gapminder.csvis the package’sgapminderdata frame written withwrite.csv(); it is included in this package.
renv.lock, that lists the R version and each package’s version.renv::init(): creates the project library and writes renv.lock.renv::snapshot(): after you install or update a package, records the change in the lockfile.renv::restore(): on another computer, installs the versions the lockfile names.renv::status(): says whether the lockfile and the library agree.renv.lock, .Rprofile, renv/activate.R and renv/settings.json. renv writes its own .gitignore, which keeps the library folder out.| Approach | What it fixes | What it leaves open |
|---|---|---|
library() calls at the top of the script |
Nothing: the script stops if a package is missing | Which version is installed |
| Install the missing packages, then load them | Every package is installed | The version: it installs the newest |
remotes::install_version("gapminder", "1.0.1") |
One package’s version | The versions of the packages it needs |
| renv | Every package’s version, listed in renv.lock |
The R version, the operating system |
| A dated copy of CRAN, from Posit Package Manager | Every package as CRAN had it on that day | The R version, the operating system |
| A Docker image | The operating system, R and every package | Needs Docker on the computer that runs it |
checkpoint package used dated copies of CRAN kept by Microsoft. Microsoft closed that service on July 1, 2023, and checkpoint no longer works.code/run_everything.R from start to end with no manual steps. Compare every table, figure and number with the paper.code/run_everything.R. Nothing else.run_everything.R runs the numbered scripts in code/ in order."data/...", never "/Users/you/...".run_everything.R there.data/ or data_clean/: the code rebuilds them. The last commit on main at the deadline is the one graded.3159ecc. A tag gives it a name that stays put, v1-submitted; the branch main moves on.git log --decorate and in GitHub’s list of tags.-a makes an annotated tag: it records who made it, when, and the message.git push alone does not send tags to GitHub; push each tag by name.git log --oneline --decorate
#> 83b87f6 (HEAD -> main) README: add the list of tables and programs
#> 042175d (tag: v2-revised) Referee 2: describe the fitted line in the figure note
#> 9ae9460 Write the numbers in the text from code/03_numbers.R
#> 3159ecc (tag: v1-submitted) Correct five numbers in the text and Table 1
#> a228ee2 Draft: data, Table 1 and Figure 1 scripts, papermain.git diff --stat v1-submitted v2-revised: the files that changed between the two versions, for the response to the referees.git switch --detach v1-submitted: the folder as it was when you submitted, to rerun it. git switch main returns to the newest commit.git archive writes the files at a tag to a zip, with no history:git archive into the deposit, so the deposit lists every file.10.3886/E…V…, works once the deposit is published, at the end of review.Source: the AEA Data Editor’s guidance on the deposit.
https://doi.org/..., to one archived copy of the files.gapminder package’s releases: doi.org/10.5281/zenodo.594018.CITATION.cff file at the top of the repository adds a “Cite this repository” link on GitHub, and Zenodo reads it when it archives a release.A draft written for this session: the paper (PDF) · the project (zip).
~/github/income-life-draft.Rproj, then Tools → Terminal → New Terminal; or in Terminal, cd ~/github/income-life-draftclaude. The first time, choose Yes, I trust this folder. If it says the login expired, type /loginNeeds: R; Quarto, with LaTeX for PDF output (quarto install tinytex in a terminal, once); Claude Code, installed and logged in.
Prompt to paste into Claude Code
Read paper/draft.pdf. Check every number in it that reports a result:
in the text, in Table 1, and in the notes under the table and the
figure. For each number, find the script in code/ that computes it. If
no script computes it, compute it from data/gapminder.csv with R. Then
list only the numbers that disagree with the code or the data: the
number as printed, where it appears, and the value you get. End with how
many numbers you checked. Do not change any file.Claude Code’s reply in a test run before class, rearranged into a table:
| As printed | Where | What the code or data give |
|---|---|---|
| 0.73 more years per 10% | Abstract | 0.69 (7.203 × log 1.1); 0.72 if taken as 7.203 × 0.1 |
| 42.8 years | Results, first paragraph | 43.0 (82.603 − 39.613) |
| 69.2 (Asia) | Table 1 | 70.7, from 01_table1.R and table1.csv |
| 28,504 (Europe) | Table 1 | 28,054: two digits swapped |
| 0.73 | Results, third paragraph | the same as the abstract |
| Gapminder, 2002 | Note under Figure 1 | 2007: the figure plots 2007 |
.qmd file. Rendering runs the code, puts each computed value into the text, and writes an HTML, PDF or Word file..tex file. R scripts write each table to a .tex file and each number in the text to a file of LaTeX macros. The paper reads those files with \input.Prompt to paste into Claude Code
Convert paper/draft.tex into a Quarto document, paper/paper.qmd, that
renders to PDF. Every number in the text, the table and the notes must
come from R code that reads data/gapminder.csv or the files in output/;
type none by hand. Build Table 1 from output/table1.csv with
knitr::kable, and include output/figure1.pdf as Figure 1. Keep the
wording; do not try to match the layout. Render it. Then list every
number that differs from paper/draft.pdf. Do not change any other file.paper/paper.qmd, rendered paper/paper.pdf, and listed the five stale numbers with their new values.pct <- 10.`r ...` is computed in one R chunk at the top of the file, from data/gapminder.csv and output/table1.csv.`r ...`, the only digits left should be in the references.Top to bottom: the header, an R code chunk, text with inline R code. Rendered, the last two lines read: “In 2007, life expectancy was lowest in Swaziland, at 39.6 years.”
`r expression` inside a sentence: rendering runs the expression and replaces it with its value.| Code | Prints |
|---|---|
round(67.00742, 1) |
67 |
sprintf("%.1f", 67.00742) |
67.0 |
format(6124, big.mark = ",") |
6,124 |
```{r}
#| label: tbl-continents
#| tbl-cap: "Countries and life expectancy by continent, 2007"
knitr::kable(table1) # a data frame as a table
```
```{r}
#| label: fig-scatter
#| fig-cap: "Life expectancy and GDP per capita, 2007"
plot(d$gdpPercap, d$lifeExp, log = "x")
```
@tbl-continents reports averages by continent. @fig-scatter plots
life expectancy against income.tbl- or fig- makes Quarto number the table or figure, in order.@tbl-continents in the text prints “Table 1”, linked to the table. Add or reorder tables and the numbers change at the next render.quarto render paper.qmd.format: in the header picks the output: html, pdf or docx. A PDF needs LaTeX; install it once with quarto install tinytex in the Terminal.# Exercise 1 (5 minutes). RStudio.
#
# 1. File > New File > Text File.
# Paste the text on the right.
# Save As... exercise.qmd
#
# 2. Replace each ___ with inline
# R code, `r ...`: yr,
# low$country, and low$lifeExp
# rounded to one decimal.
#
# 3. Render. Read the sentence.
#
# 4. Change 2007 to 2002 in the
# chunk. Render again.
#
# Two answers to compare with the
# room: the country and the number
# for 2007, and for 2002.A LaTeX macro is a name that LaTeX replaces with its definition. R writes the definitions to output/numbers.tex:
The paper reads it once, in the preamble, and uses the macros:
# code/03_numbers.R: every number the text quotes
d <- read.csv("data/gapminder.csv")
d07 <- d[d$year == 2007, ]
numbers <- c(
nCountries = length(unique(d$country)),
lifeExpStart = sprintf("%.1f", mean(d$lifeExp[d$year == 1952])),
lifeExpEnd = sprintf("%.1f", mean(d07$lifeExp)),
lifeExpGap = sprintf("%.1f", max(d07$lifeExp) - min(d07$lifeExp))
)
writeLines(sprintf("\\newcommand{\\%s}{%s}", names(numbers), numbers),
"output/numbers.tex")sprintf("%.1f", x) prints exactly one decimal.\lifeExpEnd works, \lifeExp2007 does not.The first lines of output/table1.tex, for the draft’s Table 1:
booktabs = TRUE: three horizontal rules, no vertical lines.table environment holds the caption, the notes and \input{../output/table1.tex}.Source: the AER style guide.
Figure1a.pdf, Figure1b.pdf.cairo_pdf() embeds the fonts in the file; pdf() does not.Sources: the AER style guide (formats, dpi, panels); Elsevier’s sizing guidance (7 points).
Ten exercises on the session page, with solutions behind a click. All of them use the draft project, download it from the page.
numbers.tex and table1.tex from R; the Results section in Quarto; Figure 1 at one column’s widthEach number in the paper comes from one line of code, and the code writes it into the paper.
The commands, the README sections and the steps to a DOI, one line each, are on the session page.
AEM 6850 · Session 11