Bootstrap resampling and Anscombe’s quartet: why a good summary still needs a plot

What the bootstrap actually resamples

The nonparametric bootstrap samples with replacement from the observations already available. Each replicate has the same size as the original sample, but some observations appear more than once and others disappear. Recomputing a statistic across many replicates provides an approximation to its sampling variability under conditions that need justification.

The bootstrap resampling board starts with six values: 1, 2, 2, 4, 7 and 9. Inspect individual resamples before looking at the histogram. You can see exactly how repeated draws create different means without creating new observed values. Efron’s original bootstrap paper introduced the general resampling approach; the board is a deliberately small example rather than a claim that six observations are sufficient for reliable inference.

A statistic can hide the feature that matters

A regression slope describes a particular linear summary. It does not say whether a relationship is curved, whether one point dominates the result, or whether the observations form a pattern that makes the model inappropriate. A correlation close to another correlation therefore need not imply that the datasets tell the same story.

Switch among the four datasets in Anscombe’s quartet. Their rounded numerical summaries are very similar, but the plots show a broadly linear pattern, curvature, an unusual response and a highly influential x value. Anscombe’s paper on graphical analysis is a direct argument for examining the observations graphically. The lesson is to inspect structure alongside summaries, rather than rejecting numerical analysis or treating every unusual point as an error.

Worked example: trace one estimate back to its observations

This sequence uses authored teaching data and the original rounded quartet coordinates. It is an audit of the calculation, not a statistical test that a real observation should be removed. Keep the original and modified datasets distinguishable throughout.

1. Open the bootstrap board and calculate the original mean: the six values sum to twenty-five, giving about 4.17. Inspect a resample to see how repeated values change its mean even though every draw comes from the observed list.
2. Toggle omission of nine and compare the new empirical distribution. The remaining sample mean is 3.2. Treat this as a sensitivity scenario, retaining the original sample rather than announcing that the largest observation was invalid.
3. Open dataset four in the quartet and omit its largest-x point. Every remaining x equals eight, so a slope cannot be identified. The interface should report undefined rather than displaying a confident but meaningless fitted number.

Use a plot to choose what deserves investigation

A useful review begins with what a row represents. Identify the unit, collection process and variables before fitting a model. Plot the relationship and inspect suspicious records in context: an extreme value could be a transcription mistake, a genuine rare event, or evidence of another regime. Those explanations call for different actions.

Once the dataset and estimand are clear, choose a statistic whose interpretation matches the task. Then use uncertainty analysis suited to the sampling process. Resampling individual rows assumes a different structure from resampling entire groups or time blocks. The simple Modic bootstrap deliberately assumes exchangeable observations; it does not solve clustered or dependent-data problems automatically. A sensible-looking histogram cannot supply a sampling argument that the analyst has not made.

Keep an inspectable analysis record

The goal is to make the answer auditable by a reader who did not construct the spreadsheet. Saving only the final coefficient makes it difficult to distinguish a robust pattern from a model choice, a data correction or a chance influential observation. A small record of decisions is often more useful than adding another decimal place.

• Keep a raw-data view and define precisely what each observation and variable represents.
• Plot the relationship before choosing a functional form or interpreting a correlation.
• Document every exclusion with a reason independent of whether the result becomes more attractive.
• Record the resampling unit, statistic, replicate count and interval construction used in uncertainty analysis.
• Compare sensitivity scenarios without silently replacing the original analysis with the most convenient one.

Uncertainty methods do not repair missing information

If the original sample excludes the people, events or tail outcomes relevant to the decision, repeatedly resampling it cannot recover them. A narrow bootstrap range can be a precise description of the wrong empirical distribution. Increasing the number of replicates reduces Monte Carlo roughness, but does not increase the number of independent observations collected.

Measurement can also distort a relationship before resampling begins. The regression dilution experiment explores one such mechanism: noise in an explanatory variable can weaken a fitted slope under its stated model. A useful report separates sampling variability, sensitivity to observations, model structure and measurement quality. These are complementary questions. Answering one carefully should make the remaining uncertainty more visible rather than giving the whole analysis an undeserved seal of reliability.