---
title: "Reproducibility"
subtitle: "How to rerun the analysis"
knitr:
opts_chunk:
comment: "#>"
---
```{r load}
#| include: false
source("R/00_params.R"); source("R/01_utils.R"); source("R/90_theme.R")
library(dplyr); library(ggplot2); library(gt)
diag <- read_result("diagnostics")
```
Every number on this site comes from synthetic inputs in `params.yaml` and `data/`, and from results computed by the scripts in `R/`. Nothing is downloaded, and every random draw is seeded, so running the same commit with the same package versions gives the same results. The pages never compute the heavy results themselves: they read cached results from `data/results/`. The decision-map slider recomputes the closed-form walk-away point in the browser, using a JavaScript file that the pipeline checks against R. This page explains how to rebuild the results, lists the inputs with their checksums, shows the tests, and records the software environment.
## Quick start {#quick-start}
1. Clone the [GitHub repository](https://github.com/patrick-lefler/ransomware-pay-or-restore-game-theory) and open the project folder in RStudio or Positron.
2. Restore the packages with `renv::restore()` (see [Software environment](#environment)). You need R 4.3 or later and Quarto 1.5 or later.
3. From a terminal in the project root, rebuild the results, run the tests and render the site:
``` bash
Rscript R/run_all.R
Rscript -e 'testthat::test_dir("tests/testthat")'
quarto render
```
The rendered site is written to `docs/`. Run `R/run_all.R` with `Rscript` from a terminal, not with `source()` in an R console: it ends with `quit()`, which would close your session. It exits with an error status if any diagnostic fails. To look at the site locally, run `quarto preview`, which renders it and opens it in your browser. The sliders need a web server, so they don't work when a page is opened straight from disk; on GitHub Pages they work as is.
## Pipeline {#pipeline}
`R/run_all.R` sources the scripts below in order, calls each script's `main_*()` function, runs the diagnostics and saves every result to `data/results/`. Later scripts use functions from earlier ones, so sourcing a later script on its own in a fresh session fails.
| Script | What it does | Main outputs |
|:-----------------|:-----------------------------------|:-----------------------|
| `R/00_params.R` | Reads `params.yaml` into constants, the base case and the three firm profiles; sets the project paths | `BASE`, `PROFILES`, `INPUT_SPEC`, seeds |
| `R/01_utils.R` | Closed-form costs and walk-away point, triangular and Beta samplers, formatters, result read/write helpers | Shared functions |
| `R/20_walkaway.R` | Reference cases, decision-map boundaries, the cost components of the three paths, the leak-term sensitivity and firm profiles | `walkaway.rds` |
| `R/30_mc_drivers.R` | Monte Carlo over all inputs; first-order variance shares for the walk-away point and the pay decision, on five seeds and at three demand spreads | `mc_drivers.rds` |
| `R/31_bargaining.R` | The crew's belief grid, its best demand and the Bayesian update after a payment or refusal | Shared functions |
| `R/32_reputation.R` | 10-year repeated game for every attack rate on the slider, the targeting-response sweep and the pay-once credibility test in present value | `reputation.rds`, `reputation_grid.rds` |
| `R/35_sensitivity.R` | One-at-a-time swings and the market-calibration run | `sensitivity.rds` |
| `R/80_diagnostics.R` | The checks shown on the Governance page, including the JavaScript–R parity check and the discounting and deviate-once checks | `diagnostics.rds` |
| `R/run_all.R` | Runs all of the above | All results |
`assets/js/walkaway.js` holds the walk-away formula used by the decision-map slider; the parity diagnostic runs this same file in R through the `V8` package. Every page sources `R/00_params.R`, `R/01_utils.R` and `R/90_theme.R` (labels, colors and the plot and table styles) in a hidden first chunk. `R/99_wordcount.R` is an internal prose-length check whose output is printed to the terminal only.
## Data {#data}
All inputs are synthetic and illustrative, written for this project. `params.yaml` holds every number: base values, ranges, seeds and simulation settings (see [Parameters and seeds](#parameters)). `data/inputs.csv` holds the labels, units and sources shown in the inputs table on [The Game](game.qmd#the-inputs) page; it contains no numbers. `data/assumptions_register.csv` holds the text of the register on the [Governance](governance.qmd#assumptions-register) page. The files in `data/results/` are generated by `R/run_all.R` and committed, so the site renders without rerunning the analysis.
The checksums below identify the exact files used in this render. A different checksum means a different file.
```{r}
#| label: tbl-checksums
#| tbl-cap: "Input files and cached results used in this render."
input_files <- c("params.yaml", "data/inputs.csv", "data/assumptions_register.csv", "assets/js/walkaway.js")
result_files <- file.path("data", "results", c(
"walkaway.rds", "mc_drivers.rds", "reputation.rds", "reputation_grid.rds",
"sensitivity.rds", "diagnostics.rds"
))
tibble(
Kind = c(rep("Input", length(input_files)), rep("Cached result", length(result_files))),
File = c(input_files, result_files)
) |>
mutate(
`Size (KB)` = file.size(file.path(PROJECT_ROOT, File)) / 1e3,
MD5 = unname(tools::md5sum(file.path(PROJECT_ROOT, File)))
) |>
gt() |>
tab_header(title = md("**Files used in this render**"), subtitle = paste0("Rendered ", Sys.Date())) |>
fmt_number(columns = `Size (KB)`, decimals = 1) |>
gt_board() |>
cols_align(align = "left", columns = c(Kind, File, MD5))
```
::: {.reader-note}
**How to read the table.** Each row is one file the site depends on: `r length(input_files)` hand-written inputs and `r length(result_files)` cached results produced by the scripts. The size is in kilobytes. The MD5 column is a checksum, a short fingerprint computed from the file's exact contents; changing even one character gives a completely different fingerprint. The subtitle gives the date this page was rendered.
**What the table tells you.** Anyone rerunning the analysis can compute the same fingerprints on their copies. Matching input fingerprints mean identical starting points, and the results should then match too. If a result fingerprint differs while the inputs match, check the package versions first, since a different version of R or a package can change random draws or rounding.
:::
## Parameters and seeds {#parameters}
Every setting lives in `params.yaml`, and each model input is labeled as an assumption, with its range. Every random stream is seeded explicitly: the Monte Carlo in `R/30_mc_drivers.R` uses seed `r SEED_MC`, and the seed-stability check reruns it with the next `r P$monte_carlo$stability_seeds - 1` seeds. The reputation simulation in `R/32_reputation.R` uses seed `r SEED_REP`. It draws one set of random numbers and reuses them for every attack rate, firm and policy, so the slider chart moves smoothly instead of jittering. The event-level check in `R/80_diagnostics.R` uses seed `r format(SEED_MC + 2, scientific = FALSE)`.
```{r}
#| label: tbl-params
#| tbl-cap: "Model inputs in params.yaml."
fmt_dist <- \(s) {
par <- unlist(s$par)
switch(s$dist, tri = sprintf("Triangular(%s)", paste(par, collapse = ", ")),
beta = sprintf("Beta(%s)", paste(par, collapse = ", ")), fixed = "Fixed",
ratio_of_p_np = sprintf("p_np × Triangular(%s)", paste(par, collapse = ", ")))
}
tibble(
Name = names(INPUT_SPEC),
Base = purrr::map_dbl(names(INPUT_SPEC), \(n) BASE[[n]]),
Distribution = purrr::map_chr(INPUT_SPEC, fmt_dist),
Status = purrr::map_chr(INPUT_SPEC, "status")
) |>
gt() |>
fmt_number(columns = Base, decimals = 2, drop_trailing_zeros = TRUE) |>
gt_board() |>
cols_align(align = "left", columns = -Base)
```
::: {.reader-note}
**How to read the table.** Each row is one model input from `params.yaml`, the single file that sets every number in the analysis. The base column is the value used for the worked example and the firm profiles. The distribution is the range drawn from in the Monte Carlo: a triangular range lists its lowest, most likely and highest values, and a Beta range lists its two shape values. The status marks each input as an assumption and names any supporting source.
**What the table tells you.** To test a different view, change a value in `params.yaml` and rerun the pipeline; no code needs editing. Simulation settings such as the number of draws, the crew's starting belief and the attack-rate grid sit in the same file, below the inputs. The seeds fix every random draw, so two people with the same file get the same numbers.
:::
## Tests {#tests}
Two layers of checks guard the results. Unit tests in `tests/testthat/` check the closed-form model against the seven reference cases and the worked example from the project outline, the indifference condition, the staged-path reference values, the samplers, the crew's opening demand and the share of incidents in which paying the base demand is cheaper. They run again each time this page is rendered, and the table below shows the results. The pipeline diagnostics (`r sum(diag$result == "Pass")` of `r nrow(diag)` passing in this build) are shown on the [Governance](governance.qmd#diagnostics) page.
```{r}
#| label: tbl-tests
#| tbl-cap: "Unit test results from rendering this page."
tests <- testthat::test_dir(
file.path(PROJECT_ROOT, "tests", "testthat"),
reporter = "silent", stop_on_failure = FALSE
) |>
as.data.frame() |>
transmute(
File = file, Test = test, Expectations = nb,
Result = if_else(failed == 0 & !error & !skipped, "Pass", "FAIL")
)
tests |>
gt() |>
gt_board() |>
cols_align(align = "left", columns = c(File, Test, Result))
```
::: {.reader-note}
**How to read the table.** Each row is one unit test. The expectations column counts the individual comparisons inside it, such as one computed walk-away point against its reference value. "Pass" means every comparison in the test held. Here `r sum(tests$Result == "Pass")` of `r nrow(tests)` tests passed, covering `r sum(tests$Expectations)` comparisons in total. Each test's name says what it checks, and the file column shows where to find it in `tests/testthat/`.
**What the table tells you.** The formulas behind every page reproduce the reference values to within one hundredth of a million dollars, and the walk-away point is exactly where paying and refusing cost the same. A failure here would mean the code no longer matches the documented model, and the site's numbers should not be used until it is fixed.
:::
## Caching {#caching}
- **Results cache (`data/results/*.rds`).** Written by `R/run_all.R` and read by every page. It is not invalidated automatically: after changing `params.yaml` or any script, rerun `Rscript R/run_all.R`. The files are committed to git so the site renders from a fresh clone.
- **Quarto freeze (`_freeze/`).** `_quarto.yml` sets `freeze: auto`, so a full `quarto render` re-executes a page only when that page's `.qmd` file changes. Edits to `R/` scripts or new results do not count as a change. After rerunning the pipeline, render each page explicitly (for example `quarto render governance.qmd`) or delete `_freeze/` first. `_freeze/` is not committed.
## Licenses {#licenses}
The code on this site is released under the [MIT License](license.qmd). The project redistributes no third-party data: every input is synthetic and written for this project. The publications cited on the [Governance](governance.qmd#references) page are quoted briefly and linked; they remain under their publishers' terms.
## Software environment {#environment}
The site was built with R and Quarto. The pages load `dplyr`, `ggplot2` and `gt`, and the scripts also call `purrr`, `tibble`, `tidyr`, `readr`, `scales`, `yaml`, `V8`, `testthat` and `sessioninfo`. The two sliders use Observable JS, which is built into Quarto. Package versions are pinned in `renv.lock`; run `renv::restore()` to install them.
::: {.callout-note icon=false collapse="true"}
## Full R session information
```{r}
#| label: session-information
#| echo: false
sessioninfo::session_info()
```
:::