Technical Note 3: Is Claude A Better Economist Than I Am?
Statistical Significance Not Robust
Statistical Significance Not Robust
The authors do not spend much time focusing on the statistical significance of the climate variables they identify as producing economic damage, perhaps since the way they calculated the measures of statistical significance showed that most variables were statistically significant. The paper calculates the statistical significance of the estimated parameters by assuming that the unobserved influences on real income are correlated across geographical areas. That’s a sensible assumption. The technical term for that assumption is that the standard errors are “clustered by region.” However, the paper assumes that the unobserved influences on real income are not correlated over time, an assumption that is almost certainly false. To correct the unobserved factors affecting real income being correlated across regions and over time we “double cluster,” i.e., we cluster standard errors in the regression model over regions and time. Other academic papers that study the effect of climate change on economic growth double cluster their standard errors.
The table below shows my re-estimates of the paper’s regression model using the paper’s original assumption—clustering standard errors by region—and using the new double clustering of the standard errors. I’ve circled in red the one temperature variable that is responsible for almost all of the economic damage. As can be verified, the statistical significance of the important temperature variable and most of the other variables evaporates when we double cluster the standard errors. The statistical significance of the model is not robust to a reasonable alternative method.
