Data analysis methods suited to the NIPN set-up (2/3)

2. Population-based survey data or routine data do not permit a robust analysis of causal relationships or impact evaluation

Policy decision makers are understandably interested in the big questions, for example: “Has investment in this programme led to reduced anaemia levels?”. To be able to draw a definite conclusion and assert a causal relationship, a number of conditions need to be fulfilled through data analysis:

  • There is a statistically significant association between the implementation of the nutrition programme and the reduction of anaemia rates;
  • The investment in the relevant programme occurred prior to the measured reduction of anaemia (temporality).

This is however not sufficient to conclude that there is a causal relationship!

For example, a person drinking coffee every morning before sunrise could conclude that drinking coffee causes the sun to rise. There is a perfect association and coffee drinking precedes sunrise. Yet no one would draw this conclusion!
(Example provided by J. Leroy, IFPRI. Look at the video of his presentation made at the 1st NIPN Global Gathering, Paris, July 2018)

Other factors need to be considered, including:

  • Plausibility: the causal interpretation of the observed association needs to be coherent with existing knowledge. Knowing the laws of the universe, it is not meaningful to interpret the association between drinking coffee and the sun rising as causal.
  • Confounding factors (see below) that might provide an alternative explanation for the observed association need to be verified.
  • Other criteria for suggesting causality have been described by Hill and Bradford (1965): The Environment and Disease: Association or Causation? published in the Proceedings of the Royal Society of Medicine 58 (5): (295-300) (see below).
Using population-based survey data or routine data to conduct a causal analysis can be very misleading and is subject to criticism
The below example provides a real-life case. Many scientific studies use regression modelling methods to conduct a nutrition causal analysis using population-based surveys (Source: FEWSNET).
However, these studies:
  • can be misleading and subject to criticism because they do not control for all confounders;
  • can have a high incidence of collinearity (when determinants are co-associated);
  • rarely explain more than 20% of the variability observed;
  • rarely provide new information;
  • rarely provide actionable recommendations.
Additional information and examples

The Environment and Disease: Association or Causation? Bradford & Hill, 1965

We use cookies. By continuing to use our site, you agree to this. Details and objection options can be found in our privacy policy.
Use of third party offers

In addition to technically necessary ‘session’ cookies, this website uses Matomo tracking as well as video hosting from and

When you choose to play a video, your browser establishes a connection to third-party providers’ servers, which may automatically transmit your IP address as well as information about your browser, operating systems, date/time and the address of our website to them.

Tracking cookie from Matomo

Matomo is used in a GDPR-compliant manner, as it only collects and processes data within this website. It is used for non-personal tracking of user interaction.