Formulating relevant policy questions
- To engage relevant sectors to formulate policy questions
- To forge a common understanding of what questions can be answered within the platform’s scope
- To develop the capacity of sectoral government counterparts on question formulation in line with the contribution and role of sectors in the MPPA (impact pathways)
- To achieve a consensus on a set of policy-relevant questions
- Activity A: Organising and facilitating a consultative workshop
- Document with prioritised list of policy-relevant questions
The key activity in Step 2 is to convene a consultative workshop with sectoral government counterparts and non-government stakeholders with the following objectives:
- To formulate and prioritise a set of policy questions, based on questions identified in Step 1.
- To develop the capacity of sectoral government counterparts and non-government stakeholders in policy question formulation (especially important during the first cycle of question formulation).
- To enhance dialogue and build common expectations amongst key nutrition stakeholders.
There are a number of important points to take into account in the agenda setting and invitation to participants in this workshop:
- Who will facilitate and who will participate in the workshop?
- Setting the agenda of the workshop including:
- 1. Relevant framing of the workshop
- 2. Creating a common understanding
- 3. Which questions and data analyses best lend themselves to the NIPN approach?
- 4. Adopting an impact pathway approach to formulate relevant questions
- 5. Illustration of the use of the impact pathway to support the formulation of question
- 6. Characteristics of well-formulated policy questions
- 7. Preparing relevant and concrete country examples
1. Relevant framing in country context
The overall objective is to strengthen or develop the data-informed nutrition policy dialogue, and this workshop is a first step to engaging stakeholders in this process.
The workshop and the NIPN approach need to be framed to be coherent with the broader multisectoral nutrition policies and plans at country level.
It is important to highlight that NIPN does not replace existing Monitoring and Evaluation (M&E) systems that track the progress of MPPA, but that its added value is to analyse the data of such M&E systems to answer specific policy questions.*****
2. Create a common understanding
Before engaging in the process of question formulation, it is crucial that all key actors share a common understanding of what NIPN’s purpose and approach is. Even when several advocacy or awareness-raising activities have taken place prior to the workshop, it is important to start the workshop with a generic presentation of NIPN (section 1.1, advocacy and awareness raising) to create a common understanding amongst all participants.
The scope of NIPN and its added value can be mentioned briefly in this presentation in order to manage participants’ expectations with regard to the type of answerable questions, but this should be a specific and separate session on the agenda of the consultative workshop. This session can be facilitated with support from the Technical Note on Data Analysis.
3. Which questions and data analyses lend themselves best to the NIPN approach?
The NIPN data expert(s) will facilitate a session to explain what type of data analysis is particularly suited to providing solid answers to policy questions which can be translated into actionable recommendations. The choice of data analysis is based on data availability, and the quality and capacity of the team. The details of this information is explained in the guidance note on data analysis (section 3.4), as well as in the Technical Note on data Analysis.
It is important to illustrate the theoretical explanations with concrete examples: the guidance note on data analysis (section 3.4) provides a number of examples (summarised in the table below). It is recommended that the initial questions formulated by NIPN stakeholders are also used (see output of Step 1, section 2.2). The NIPN data team will need to review the list of initial questions prior to the workshop, in order to decide which data analysis method is best suited to providing an actionable answer.
In many cases, the initial question can be reformulated into answerable (sub-)questions. At all stages during question formulation, it is important to manage stakeholders’ expectations and be transparent about the questions which are prioritised and why, and what the next step will be.
Questions and data analyses which are less suited to providing actionable recommendations, and are therefore less of a priority, could become part of a ‘Question Bank’.*****
Suitability of questions for NIPN approach Questions which are answerable by suitable data analysis methods Partially answerable questions (after reformulation) Questions which are not suitable for analysis in the NIPN approach Are nutrition interventions reaching the target population? What are the causes of high levels of stunting in the semi-arid regions of the country? What is the impact of nutrition-sensitive interventions on stunting? What factors can explain the lower coverage of nutrition-specific interventions in some districts compared to others? Which nutrition-specific interventions should be prioritised to maximise impact on stunting? Is it more cost-effective to invest in food fortification or supplementation? What are the causes of stunting in my country and where should we focus our intervention efforts? Have investments in WASH contributed to stunting and mortality reduction? How can indicators for nutrition sensitive interventions at sub-national level be defined and tracked? Where should interventions to prevent obesity and overweight be focused? - -
4. Adopt an ‘impact pathway’ approach to help formulate policy questions
Using an impact pathway approach can be helpful for unpacking broad policy questions, which are often related to impact, into sub-questions which are more likely to be answerable with existing data.
For instance, it is difficult to answer the following question: “Did the investments in nutrition-specific interventions in the Multi-sectoral Nutrition Policy and Plan of Action, phase II (MPPA-II), reduce stunting significantly over the same period?”
The question asks whether the inputs at the start of the pathway resulted in impact at the end of the pathway. Yet, there are so many different factors along the pathway which could have intervened that it is impossible to answer this question without a controlled research set-up which measures all these (confounding) factors. It is, however, possible to break down this broad question into a set of smaller questions pertaining to each step of the impact pathway.
The impact pathway model is a logical way to organise the various elements (inputs - activities - outputs - outcomes - impact).
As shown in the animation below, the logic can be tested by moving from one element to the next. The relationship between two elements is based on underlying assumptions. For instance, at implementation stage the overall assumption is that if the multi-sectoral plan is well designed, and the interventions are implemented according to the planned coverage and with the desired quality, this would lead to measurable impact on the intended outcome (e.g. stunting).
It is important to test the relationship between elements as well as the assumptions themselves.
Following the impact pathway, policy-relevant questions and sub-questions can therefore be formulated with respect to each individual element, the relationship between elements, or the underlying assumptions.
Impact pathway models or a theory of change already adopted by the country to monitor the MPPA may be a good basis for triggering the discussion and preparing the workshop exercise.
These models will be particularly useful for identifying the intervention logics of the MPPA and the existence (or gaps) in indicators. However, the actual categorization chosen for the question formulation purpose and for the MPPA’s purpose may be slightly different.
In the absence of impact pathway models or a theory of change adopted to monitor the MPPA, impact pathways models of the REACH Compendium of Actions can serve as a good basis from which to start the exercise.*****
The impact pathway can guide the formulation of policy-relevant, answerable questions*****
5. Illustration of the use of the impact pathway to support the formulation of question
- Unpack what is happening at the level of inputs and activities, filling in each level of the impact pathway based on the information from the MPPA.
- Consider all the evidence available to inform on the degree of implementation of the MPPA.
- Consider whether changes took place during the MPPA implementation period or in comparison with the period before, in terms of availability and quality of implementation, or intervention reach (all population) and coverage (target population) (step 1, section 2.2).
- Consider differences in interventions coverage and nutrition indicators of population groups (e.g. by income quintile, or rural/urban) and geographies.
Refer to the examples 1 to 3 below.
6. Characteristics of a well-formulated policy question
Remind workshop participants that a well-formulated question has the following characteristics:
- It responds to a relevant policy need or demand from decision makers.
- It can be answered by analysing existing quantitative data with available capacity.
- It provides a timely answer for policy use or decision making.
- It leads to an actionable recommendation and decision.
- In addition, it should be specific in its formulation and specific with regard to the subject of investigation (see the visual below).
- A sixth characteristic is discussed in more detail in the guidance note on data analysis (section 3.4). In fact, each question needs to be formulated in such a way that an appropriate data analysis method can be used. The NIPN platform focuses on data analysis using existing data, mainly from population-based surveys and routine monitoring data, which are less suitable for studying causal relationships or cost-effectiveness of interventions. Questions about causal relationships may require robust study design to collect data, as well as more elaborate data analysis methodologies, which will not lead to clear-cut answers and actionable recommendations for NIPN.
Elements that need to be specified in the question
7. Prepare relevant and concrete country examples to trigger question formulation
Examples are key for helping the participants in the workshop: they need to go through the question formulation process and practise by themselves how impact pathways models can help. Thus, it is important that these have been prepared and tested by the person facilitating the process before the workshop itself.
Especially in the first NIPN cycle of ‘question-analysis-dissemination’, it has proven difficult for participants to grasp what are possible policy-relevant questions and how to formulate a good question. To initiate the process, it is recommended that workshop participants are provided with examples which are as concrete as possible.
Questions which have been proposed as priority questions in Step 1 may provide a good starting point for developing the examples for the workshop. In all cases, it is important to:
- Ensure that examples are available and prepared ahead for participants to practise with.
- Present and explain at least one unique example to all participants, independent of whether it relates to their own sector or not.
- Include sector-specific questions identified as relating to a nutrition-relevant issue in the MPPA, to help sector experts understand the importance of formulating policy-relevant questions.
The four steps described in this guidance note for formulating policy-relevant questions can be applied at any administrative level. In cases where the NIPN operational cycle is implemented at decentralized/sub-national level, it is important that all the steps described in this guideline are implemented at that level as well.
Thus, Steps 1 and 2 are crucial for engaging with stakeholders identified at decentralized level to create a policy and programme dialogue around their needs and to assess what their strategic priorities are for the question formulation.*****
Examples of strategic priority needs and policy-relevant questions at national and sub-national level
The matrix on key opportunities for influencing policy, programming and investment decisions (output of Step 1, section 2.2) will be used as a main input for the workshop. Also the NIPN sub-national nutrition dashboard (section 3.2) could serve as an entry point for discussion and further policy-relevant question formulation.*****
A priority list of well-formulated policy-relevant questions.
Based on the needs, priorities and time frame of decision makers identified in Step 1, it is important, once having formulated the questions, to agree on the final set of questions. The NIPN team will then finalise these questions, looking at the technical feasibility of answering them (Step 3, section 2.4).*****
Facilitation and participation
It is strongly recommended that one person is identified to take on the role of facilitator in the workshop preparation and during the workshop. This could typically be the senior policy advisor within the NIPN core team.
Prior to undertaking the consultative workshop, the NIPN country team should have decided whether to engage with a subset of priority sectors or whether to invite all of the nutrition-relevant sectors to this workshop and involve them in the full NIPN operational cycle (section 2.2: Step 1 – Scope of the question formulation process).
For each sector, several key people might be invited to this workshop who collectively bring relevant expertise to the table regarding:
- The policies and plans of the sector (for example, FIRST policy officer in Ministry of Agriculture).
- Routine M&E data availability and accessibility as well as surveys data.
- Nutrition: typically nutrition focal point, if existent.
Non-government partners, notably UN agencies, REACH, and members of the SUN networks (UN, civil society, donors) could also be invited as important resources to support preparation of and participation in the workshop.*****