Formulating relevant policy questions
- To engage relevant sectors to formulate policy questions
- To forge a common understanding of what questions can be answered within the platform’s scope
- To develop the capacity of sectoral government counterparts on question formulation in line with the contribution and role of sectors in the MPPA (impact pathways)
- To achieve a consensus on a set of policy-relevant questions
- Activity A: Organising and facilitating a consultative workshop
- Document with prioritised list of policy-relevant questions
The key activity in Step 2 is to convene a consultative workshop with sectoral government counterparts and non-government stakeholders with the following objectives:
- To formulate and prioritise a set of policy questions, based on questions identified in Step 1.
- To develop the capacity of sectoral government counterparts and non-government stakeholders in policy question formulation (especially important during the first cycle of question formulation).
- To enhance dialogue and build common expectations amongst key nutrition stakeholders.
There are a number of important points to take into account in the agenda setting and invitation to participants in this workshop:
- Who will facilitate and who will participate in the workshop?
- Setting the agenda of the workshop including:
- 1. Relevant framing of the workshop
- 2. Creating a common understanding
- 3. Which questions and data analyses best lend themselves to the NIPN approach?
- 4. Adopting an impact pathway approach to formulate relevant questions
- 5. Illustration of the use of the impact pathway to support the formulation of question
- 6. Characteristics of well-formulated policy questions
- 7. Preparing relevant and concrete country examples
The matrix on key opportunities for influencing policy, programming and investment decisions (output of Step 1, section 2.2) will be used as a main input for the workshop. Also the NIPN sub-national nutrition dashboard (section 3.2) could serve as an entry point for discussion and further policy-relevant question formulation.*****
A priority list of well-formulated policy-relevant questions.
Based on the needs, priorities and time frame of decision makers identified in Step 1, it is important, once having formulated the questions, to agree on the final set of questions. The NIPN team will then finalise these questions, looking at the technical feasibility of answering them (Step 3, section 2.4).*****
Facilitation and participation
It is strongly recommended that one person is identified to take on the role of facilitator in the workshop preparation and during the workshop. This could typically be the senior policy advisor within the NIPN core team.
Prior to undertaking the consultative workshop, the NIPN country team should have decided whether to engage with a subset of priority sectors or whether to invite all of the nutrition-relevant sectors to this workshop and involve them in the full NIPN operational cycle (section 2.2: Step 1 – Scope of the question formulation process).
For each sector, several key people might be invited to this workshop who collectively bring relevant expertise to the table regarding:
- The policies and plans of the sector (for example, FIRST policy officer in Ministry of Agriculture).
- Routine M&E data availability and accessibility as well as surveys data.
- Nutrition: typically nutrition focal point, if existent.
Non-government partners, notably UN agencies, REACH, and members of the SUN networks (UN, civil society, donors) could also be invited as important resources to support preparation of and participation in the workshop.*****
1. Relevant framing in country context
The overall objective is to strengthen or develop the data-informed nutrition policy dialogue, and this workshop is a first step to engaging stakeholders in this process.
The workshop and the NIPN approach need to be framed to be coherent with the broader multisectoral nutrition policies and plans at country level.
It is important to highlight that NIPN does not replace existing Monitoring and Evaluation (M&E) systems that track the progress of MPPA, but that its added value is to analyse the data of such M&E systems to answer specific policy questions.*****
2. Create a common understanding
Before engaging in the process of question formulation, it is crucial that all key actors share a common understanding of what NIPN’s purpose and approach is. Even when several advocacy or awareness-raising activities have taken place prior to the workshop, it is important to start the workshop with a generic presentation of NIPN (section 1.1, advocacy and awareness raising) to create a common understanding amongst all participants.
The scope of NIPN and its added value can be mentioned briefly in this presentation in order to manage participants’ expectations with regard to the type of answerable questions, but this should be a specific and separate session on the agenda of the consultative workshop. This session can be facilitated with support from the Technical Note on Data Analysis.
3. Which questions and data analyses lend themselves best to the NIPN approach?
Because the platforms operate on one key principle which is the use and analysis of existing data, the platforms may be best suited or less suited for certain type of questions, and thus analysis.
Existing data may be sufficient and appropriate to answer any type of question that helps better “understanding implementation progress” forward impact along an impact pathway approach, as shown in below table examples.
Existing data, however, are likely to be inappropriate to answer questions related to measuring causality, impact and cost-efficiency, because they need to be collected for that very purposes to ensure robust and appropriate analysis. Thus questions related to measuring causality, impact and cost-efficiency, though legitimate and of high importance to decision makers are not best suited to the NIPN approach.
However, in a context of increased accountability around the Multisectoral Plans of Actions to achieve national and global targets, NIPN is well positioned to answer questions about implementation along the impact pathway, contributing to understand and unpack progress forward impact.
The choice of data analysis is based on data availability, and the quality and capacity of the team. The details of this information is explained in the guidance note on data analysis (section 3.4), as well as in the Technical Note on data Analysis.*****
Suitability of questions for NIPN approach
4. Provide concrete examples
It is important to illustrate the theoretical explanations with concrete examples: the guidance note on data analysis (section 3.4) provides a number of examples.
It is recommended that the initial questions formulated by NIPN stakeholders are also used (see output of Step 1, section 2.2).
The NIPN data team will need to review the list of initial questions prior to the workshop, in order to decide which data analysis method is best suited to providing an actionable answer.
In many cases, the initial question can be reformulated into answerable (sub-)questions. At all stages during question formulation, it is important to manage stakeholders’ expectations and be transparent about the questions which are prioritised and why, and what the next step will be.
Questions and data analyses which are less suited to providing actionable recommendations, and are therefore less of a priority, could become part of a ‘Question Bank’.
5. Adopt an ‘impact pathway’ approach to help formulate policy questions
Using an impact pathway approach can be helpful for unpacking broad policy questions, which are often related to impact, into sub-questions which are more likely to be answerable with existing data.
For instance, it is difficult to answer the following question: “Did the investments in nutrition-specific interventions in the Multi-sectoral Nutrition Policy and Plan of Action, phase II (MPPA-II), reduce stunting significantly over the same period?”
The question asks whether the inputs at the start of the pathway resulted in impact at the end of the pathway. Yet, there are so many different factors along the pathway which could have intervened that it is impossible to answer this question without a controlled research set-up which measures all these (confounding) factors. It is, however, possible to break down this broad question into a set of smaller questions pertaining to each step of the impact pathway.
The impact pathway model is a logical way to organise the various elements (inputs - activities - outputs - outcomes - impact).
As shown in the animation below, the logic can be tested by moving from one element to the next. The relationship between two elements is based on underlying assumptions. For instance, at implementation stage the overall assumption is that if the multi-sectoral plan is well designed, and the interventions are implemented according to the planned coverage and with the desired quality, this would lead to measurable impact on the intended outcome (e.g. stunting).
It is important to test the relationship between elements as well as the assumptions themselves.
Following the impact pathway, policy-relevant questions and sub-questions can therefore be formulated with respect to each individual element, the relationship between elements, or the underlying assumptions.
Impact pathway models or a theory of change already adopted by the country to monitor the MPPA may be a good basis for triggering the discussion and preparing the workshop exercise.
These models will be particularly useful for identifying the intervention logics of the MPPA and the existence (or gaps) in indicators. However, the actual categorization chosen for the question formulation purpose and for the MPPA’s purpose may be slightly different.
In the absence of impact pathway models or a theory of change adopted to monitor the MPPA, impact pathways models of the REACH Compendium of Actions can serve as a good basis from which to start the exercise.*****
The impact pathway can guide the formulation of policy-relevant, answerable questions*****
6. Illustration of the use of the impact pathway to support the formulation of question
- Unpack what is happening at the level of inputs and activities, filling in each level of the impact pathway based on the information from the MPPA.
- Consider all the evidence available to inform on the degree of implementation of the MPPA.
- Consider whether changes took place during the MPPA implementation period or in comparison with the period before, in terms of availability and quality of implementation, or intervention reach (all population) and coverage (target population) (step 1, section 2.2).
- Consider differences in interventions coverage and nutrition indicators of population groups (e.g. by income quintile, or rural/urban) and geographies.
Refer to the examples 1 to 3 below.
7. Characteristics of a well-formulated policy question
Remind workshop participants about the definition of a nutrition policy relevant question in the context of NIPN. By “nutrition policy relevant question” it is means a question that:
- respond to a relevant policy need or demand from decision makers
- are answered through existing data, methods and capacities available
- provide timely output for policy use
- provides answers that lead to actionable recommendations and decisions.
In addition, to the definition of a ‘nutrition policy relevant question’, there are few characteristic that may be useful to bear in mind for the actual formulation of the question. These characteristics help mainly to refine and specify questions so that they can be well formulated so as to be answerable by data analysts.*****
Elements that need to be specified in the question*****
In fact, the question needs to be formulated in such as way that it can be answered by quantitative analysis. For example, questions formulated as ’what can we learn from’ may not be directly answerable by analysis and may need to be reformulated.
Questions started with ’why’ or ’what are the causes of’ due to the nature of the NIPN are also likely not to be answerable by NIPN; they may either be ‘not suited’ if they aim to measure causality, impact or cost-efficiency or they need to be broken down further or reformulated to become answerable (refer to the technical note on data analysis).
8. Prepare relevant and concrete country examples to trigger question formulation
Examples are key for helping the participants in the workshop: they need to go through the question formulation process and practise by themselves how impact pathways models can help. Thus, it is important that these have been prepared and tested by the person facilitating the process before the workshop itself.
Especially in the first NIPN cycle of ‘question-analysis-dissemination’, it has proven difficult for participants to grasp what are possible policy-relevant questions and how to formulate a good question. To initiate the process, it is recommended that workshop participants are provided with examples which are as concrete as possible.
Questions which have been proposed as priority questions in Step 1 may provide a good starting point for developing the examples for the workshop. In all cases, it is important to:
- Ensure that examples are available and prepared ahead for participants to practise with.
- Present and explain at least one unique example to all participants, independent of whether it relates to their own sector or not.
- Include sector-specific questions identified as relating to a nutrition-relevant issue in the MPPA, to help sector experts understand the importance of formulating policy-relevant questions.
The four steps described in this guidance note for formulating policy-relevant questions can be applied at any administrative level. In cases where the NIPN operational cycle is implemented at decentralized/sub-national level, it is important that all the steps described in this guideline are implemented at that level as well.
Thus, Steps 1 and 2 are crucial for engaging with stakeholders identified at decentralized level to create a policy and programme dialogue around their needs and to assess what their strategic priorities are for the question formulation.*****
Examples of strategic priority needs and policy-relevant questions at national and sub-national level