I’ve been calling myself a causal inference Luddite recently and it only just occurred to me now to look up what it means. Thankfully, I think it captures what I want to say.
A Luddite is someone who is against technological change. Like someone who refuses to get a smartphone or someone who refuses to file their taxes online resorting to paper. Causal inference and the fancy methods that have been developed to estimate causal effects are not technologies but I’ve become increasingly more and more skeptical of methods that rely on more and more causal assumption. I often say it like this: we’re skeptical of analyses that make one exchangeability assumptions but we’re not three times more skeptical of methods that require three exchangeability assumptions.
Now you might say that maybe some of the exchangeability assumptions require adjusting for the same variables and therefore we shouldn’t be three time more skeptical, but you get my point. Our skepticism doesn’t scale appropriately with the plausibility of the assumptions we’re making.
I guess I should make it clear that I’m not a Luddite about causal inference in general. How could I be? It helps us think so much more clearly about what we’re doing. But, like Charles Manski, I think even at our most skeptical, we’re too certain about our results. In his words, we are drawn by “incredible certitude”, literally impossible to believe certitude.
