I spend a lot time thinking about how much an applied epidemiologist should know about methods or statistics. I clearly don’t expect them to know everything a dedicated methodologist know but there are some important basic concepts that they should now. So I thought I’d approach this by making a list of topics I think every applied epidemiologist should know and be able to discuss comfortably. Here’s what I came up with:
Methodological topics
- The difference between descriptive, predictive and causal questions
- Awareness that adjustment in descriptive studies is not for confounding but to define the estimand (or question) of interest
- How to clearly state a research question
- That though journals require language about associations, many questions labelled as associations are actually causal in nature.
Causal inference
- The difference between a question about time-fixed and time-varying questions so they know they need different tools when answering the latter. - The concepts of effect measure modification, causal interaction and statistical interaction including explaining why these must appear on at least one scale (when both variables involved have an effect).
- A relatively nuanced understanding of confounding and selection bias
- The consistency assumption
- The positivty assumption
- Ability to use and think with directed acyclic graphs
- When running a mediation analysis
- Can define controlled and natural effects
- Knows the assumptions required to estimate each
- Not defining exposure based on things that will only be known in the future or, similarly
- The existence of methods such as instrumental variables and other “quasi-experimental” designs
Statistical topics
- The definition of a p-value and awareness that it assumes the null to be true
- When conducting a hypothesis test, the reason why they are doing so instead of simply reporting an estimate and confidence interval
- A non-significant result does not mean there is no effect. Ability to distinguish between non-significant but non-informative (wide confidence intervals) and non-siginificant and informative
- The difference between significant and not significant is not itself significant
- The difference between statistically significant and actually interesting
This may be a strong opinion but I think that an applied researcher that is not able to converse about these topics comfortably can easily make mistakes in their research. The literature is full of examples of errors on these topics which supports my idea, to some extent.
After making this list I tried to think about what proportion of a randomly selected sample of applied epidemiologists would meet these criteria. Based on my experience with peer-review, reading applied papers, etc, I’m not very confident that these concepts are well understood. Maybe I’m being pessimistic though?
This post might be somewhat similar to my Epidemiology Jail post but framed in a more positive way.
Feel free to hit me up with feedback if there’s something you think should or shouldn’t be on this list.
