Challenges and solutions (2/2)

Challenge 3: How to ensure data quality

It is critical that the NIPN provides reliable data to build the credibility of the initiative.
There is no magic bullet regarding data quality. Rather, a rigorous process is adopted to define whether the quality of the data is sufficient for it to be included in the analysis and to allow reliable and coherent interpretation.
This rigorous process needs to answer the following questions:

• Which data quality control mechanisms have been implemented for this indicator? Do these mechanisms consider multiple aspects of data quality (training, sampling design, differences between surveyors, outliers, etc.)?
• Is there a data quality section in the survey report? What does it say? (For example, National Nutrition Surveys following the SMART methodology should produce a data plausibility report with an overall score).
• Are there regional disparities in data quality? (For example, during a national survey, the access for logistic or security reasons to one region may have been difficult. The sampling may have been excellent in 9 regions and poor in 1. This is not a rare situation. For providing a national prevalence this may be good enough; for providing a regional prevalence this can be problematic for the region in question).
• Are the data coherent with other sources of information?
• Are the data over-interpreted given the data quality?

Always:

• Do the homework: never take data quality for granted. Rigorous data quality assessment allows the user to have confidence in the data analysis and the results when presenting them to policy makers, and to be able to respond rigorously to any doubts about data quality.
• Be consistent in any decision to drop certain indicators and keep others. Decide collectively on the basis of well-defined criteria.
• Be conservative: in the case of doubt regarding data quality, the best and only choice is to leave out that specific indicator from the dashboard.
• Be transparent: the dashboard should provide sufficient information (definition of indicator, geographic scope, year, source of information) to allow every user to reproduce the dashboard with the same data.

Refer to section 3.3 for guidance on anthropometric data quality.