Principles and guidance for data analysis

The strength of NIPN

The strength of the National Information Platforms for Nutrition lies in their ability to:

1. answer nutrition policy-relevant questions for which the starting point is the demand from policy makers.
2. make better use of existing national and sub-national quantitative data (depending on availability/accessibility/quality of data), for a better understanding of:

• progress and regional disparities in nutrition targets and determinants;
• coverage of nutrition-specific and nutrition-sensitive interventions and programmes;
• investments in these programmes.

The added value of the NIPN operational cycle is the creation of a continuous dialogue between policy experts and data analysts to decide on the priority policy-relevant questions, the type of analysis and how the results from the analysis will be communicated to policy makers as actionable recommendations for policy decisions. This differs from the usual approach in which data analysts, analyse data, prepare a report, and share the report with policy makers, without much interaction with them.

• Because the strength of NIPN lies in answering nutrition policy-relevant questions and making better use of existing data, the data analyses carried out by NIPN needs to follow specific principles.
• Because of these principles, and because of the nature of the data typically used by NIPN, some data analysis methods are better suited to providing actionable answers than others.

This guidance note clarifies these two points from a “data perspective”. It should be used in full coherence with the guidance note on the formulation of policy questions (see section 2).

The objectives of this guidance note are:

• To support the data teams in identifying or choosing the appropriate data analysis method, pertinent to the nutrition policy questions and data quality;
• To discuss the strengths and limitations of the platforms for different types of data analysis, illustrated by concrete examples and methodological argumentation;
• To identify analysis tools and methods that are particularly suited to leading to actionable recommendations to inform nutrition policy, programme and budget decisions.

A short technical note summarising this section is available and can be used for:

• Creating a common understanding amongst the data and policy units and with members of the Multisectoral Advisory Committee, on what NIPN can and cannot do in terms of data analysis.
• Supporting the NIPN teams in communicating what the information platform can and cannot do in terms of data analysis to other stakeholders (including policy decision makers, sectoral technical experts, donors, FAO-FIRST policy officers or partners active in SUN networks).

Formulating a relevant nutrition policy question

As described in section 2 on formulating nutrition policy questions, a nutrition policy-relevant question:

1. responds to a relevant policy need or decision maker’s interest;
2. can be answered using existing quantitative data and available capacity;
3. provides timely output for policy use or decision making;
4. provides answers that lead to actionable recommendations and decisions.

What are the implications for data analysis?

Implications for data analysis (1/4)

Principle 1: Question responds to a relevant policy need or decision maker’s interest

• The objective of the analysis is defined by decision makers, not by the data analysts. A dialogue between data analysts and decision makers is important to specify the objective and clarify the question.

Principle 2: Question can be answered using existing quantitative data and available capacity

• Data analysts need to evaluate whether it is feasible to answer a specific question given the data availability and its quality. This has also implications in terms of the data analysis methods that may be particularly suited or not (see pages 6 to 9 in this section).
• The idea behind the information platforms is to make better use of existing data. Therefore, questions that require the collection of new data cannot be answered by NIPN teams. It may however be possible to partially answer the question (by formulating sub-questions) or alternatively, to explore whether a third party (e.g. a research institute) would be able to conduct a specific study/research to collect these data.
• Qualitative data, though not the primary focus of the information platforms, are an important source of information to better interpret results from quantitative data analysis.

Principle 3: Question provides timely output for policy use or decision making

• The results of the analysis must be produced in a timely fashion, meaning that the results are available in the window of opportunity of decision-making by policy makers.
• For example, if decision makers need information to finalise the new Multisectoral Plan of Action for Nutrition by the end a given year, it is useless to produce a comprehensive analysis 3 months later. It is more important to produce a less comprehensive analysis which is ready in time to inform the decisions.
• The data analyst should not be guided by the question “How much time is needed to produce the results?” but by “What results can realistically be produced within the proposed timeframe?”.

Implications for data analysis (2/4)

Principle 4: Question provides answers that lead to actionable recommendations and decisions

• “In-house” or “country-owned” data analyses can have a higher impact on policy makers than sophisticated data analyses carried out by international organisations.
• Simple descriptive analyses are sometimes more telling than complex analyses because the message is clear and the data can be presented in a simple and visual way. A simple bar chart (Figure 1) is more easily understood and is more likely to provide actionable recommendations than a regression table (Figure 2).

(Also refer to section 4: Communicating and disseminating findings)

(source: REACH)

(source: FEWS NET)

Implications for data analysis (3/4)

However, simple does not mean quick and dirty. What is done must be done correctly

• To influence policy makers, the data analyses and results must be undisputable. Disseminating the findings of a shaky analysis can be detrimental to the reputation and credibility of the platform and the host organisations.
• The proposed analysis must be of high quality, given the available resources.
• This means coherence between the question, the data availability, the data quality, the data analysis method and the capacity of the data analysis unit. A well-designed, detailed data analysis plan (see below) ensures this overall coherence.
• High quality data analysis does not necessarily mean using complex data analysis methods.
• Recognising the need to rapidly establish the credibility of the platform, it is important to identify one initial question which requires analysis that can be rapidly produced (collecting the “low-hanging fruit” first).
• The NIPN nutrition dashboard (see section 3.5) is an example of a rapidly produced analysis. It is useful ONLY if it is used to establish the policy dialogue with decision-makers (see section 2).

Implications for data analysis (4/4)

The findings should “tell a story”

• The NIPN approach makes use of nutrition impact pathways to unpack broad policy questions which are often related to impact. The impact pathway is a logical way to organise the various elements leading to impact (inputs – activities – outputs – outcomes – impact). A broad impact question can be split into a number of sub-questions which are more likely to be answerable with existing data, such as questions related to inputs (financial and human resources), activities (the interventions) and outputs (intervention coverage of the target population)
• The analysis of data related to the different elements of the nutrition impact pathway should not just appear as a compilation of indicators. It can tell a great story which makes the logical flow of the pathway clear, and is thus more likely to influence policy makers.