Correlation vs. Causation: What They Actually Mean

Correlation does not imply causation is a phrase frequently heard in data science, but rarely explained in detail. Understanding the distinction between correlation and causation, why confusion frequently occurs, and how this can lead to mistakes is crucial.

Correlation simply measures how two variables move together. When one increases and the other tends to increase as well, that's a positive correlation. If one goes up while the other tends to go down, that's a negative correlation. No cause-and-effect relationship is suggested by correlation. The correlation coefficient, often denoted as r, quantifies the strength of this relationship, ranging from -1 (perfectly inverse) to +1 (perfectly parallel), with 0 indicating no relationship whatsoever. Correlation merely describes a pattern; it doesn't speculate on the cause of that pattern.

Causation, on the other hand, is a stronger assertion. It implies that one variable directly causes a change in another. If you intervene and change the first variable, and nothing else, the second variable will change as a result. Turning up your oven's temperature, for instance, directly causes food to cook faster. You can verify this by changing nothing else and observing the predictable outcome.

The mental test to ascertain causation is: could you reach in and change just one thing, and would the other thing change as a consequence? If the answer is yes, that signifies a causal relationship. If not, you're likely observing correlation.

The tendency to conflate correlation with causation arises from human nature - we're naturally inclined to notice patterns and jump to conclusions about causality. This instinct, while beneficial in survival contexts, can lead to errors in data analysis because correlation is easy to calculate, while causation is more challenging to establish. This often happens in four common traps: confounding variables, reverse causality, coincidence, and selection bias.

Confounding variables, also known as the third-variable problem, occur when an unseen third factor influences both variables under examination, making them appear linked when they're actually both responding to this hidden factor independently. A classic example is the correlation between ice cream sales and shark attacks, which both increase in summer.

However, it's not ice cream that causes shark attacks, or vice versa. The confounder here is warmer weather, which independently drives both increased ice cream purchases and more people entering the ocean simultaneously.

Reverse causality is another trap where the relationship might be causational, but in the opposite direction of what's assumed. For instance, a model might show that customers contacting support more frequently lead to more churn. However, it's equally plausible that customers who are already unhappy and on the verge of leaving the platform contact support more due to their frustration. The outcome is quietly driving the presumed cause, illustrating how reverse causality can be misleading if not properly identified.

Coincidence, or spurious correlations, happens because with enough variables analyzed over time, some unrelated trends will align purely by chance. The infamous examples of spurious correlations, like per-capita cheese consumption almost perfectly tracking with the number of people who died tangled in their bedsheets, highlight this.

There's no actual mechanism connecting them; such correlations are simply a statistical fluke. This underscores the danger of comparing many variables without a clear hypothesis (a phenomenon known as data dredging).

Selection bias, a trap that emerges from how data is collected, can also lead to manufactured correlations. If the sample isn't representative of the population being studied, it can create a false appearance of a relationship or conceal an actual one. An example is surveying only customers who responded to a marketing email, making the email appear more effective than it actually is, because non-respondents are systematically absent from the data.

The relationship observed is baked in by the way the sample was built, not by the actual interaction between the variables.