Formulating relevant policy questions
-
STEP 2: Formulate policy-relevant questions
Objectives:
- To engage relevant sectors to formulate policy questions
- To forge a common understanding of what questions can be answered within the platform’s scope
- To develop the capacity of sectoral government counterparts with regard to question formulation based on the contribution to and role of sectors in the MPPA (the use of nutrition impact pathways)
- To achieve consensus on a set of policy-relevant questions
*****Activities:
- Activity A: Organising and facilitating a consultative workshop
*****Outputs:
- Document with prioritised list of policy-relevant questions
***** -
Activity A: Preparation of a consultative workshop
The key activity in Step 2 is to convene a consultative workshop with sectoral government counterparts and non-government stakeholders with the following objectives:
- To formulate and prioritise a set of policy questions, based on questions identified in Step 1.
- To develop the capacity of sectoral government counterparts and non-government stakeholders in policy question formulation (especially important during the first cycle of question formulation).
- To enhance dialogue and build common expectations amongst key nutrition stakeholders.
There are a number of important points to take into account in the agenda setting and invitation to participants in this workshop:
- Who will facilitate and who will participate in the workshop?
- Which points should be on the agenda of the workshop:
- Relevant framing of the workshop
- Creating a common understanding
- Which questions and data analyses lend themselves best to the NIPN approach?
- Preparing relevant and concrete country examples
- Adopting an impact pathway approach to formulate relevant questions
- Illustration of the use of the impact pathway to support the formulation of question
- Characteristics of well-formulated policy questions
***** -
Activity A: Workshop inputs, outputs, facilitation and participation
Workshop inputs
The matrix on key opportunities for influencing policy, programming and investment decisions (output of Step 1, section 2.2) will be used as the main input for the workshop. In case the NIPN team has elaborated a (sub-national) nutrition dashboard (section 3.5), this could serve as a good entry point for discussion and further policy-relevant question formulation.
*****Workshop outputs
A priority list of well-formulated policy-relevant questions.
Based on the needs, priorities and the time frame of decision makers as identified in Step 1, it is important to agree on the final set of questions, as it is unlikely that all formulated questions can be answered with the existing capacity in due time. The NIPN team will then review the technical feasibility of answering these questions, which will lead to final list of reformulated questions (Step 3, section 2.4).*****Facilitation and participation
It is strongly recommended that one person is identified to take on the role of facilitator in the workshop preparation and during the workshop. This could typically be the senior policy advisor of the NIPN core team.
Prior to undertaking the consultative workshop, the NIPN country team should have decided whether to engage with a subset of priority sectors or whether to invite all of the nutrition-relevant sectors to this workshop and involve them in the full NIPN operational cycle (section 2.2: Step 1 – Scope of the question formulation process).
For each sector, several key people might be invited to this workshop who collectively bring relevant expertise to the table regarding:- Policies and plans of the sector (for example, the FAO-FIRST policy officer in Ministry of Agriculture).
- Routine M&E data availability and accessibility as well as surveys data.
- Nutrition: this could be the nutrition focal point, if such a person has been appointed.
Non-government partners, notably UN agencies, REACH, and members of the SUN networks (UN, civil society, donors) could also be invited as important resource persons to support preparation of and participation in the workshop.
***** -
Activity A: Setting the agenda of the workshop (1/7)
1. Relevant framing in country context
The overall objective of NIPN is to strengthen or create a data-informed nutrition policy dialogue, and this workshop is a first step to engaging stakeholders in this process.
The workshop and the NIPN approach need to be framed in order to be coherent with the broader multisectoral nutrition policies and plans at country level.
It is important to highlight that NIPN does not replace existing Monitoring and Evaluation (M&E) systems that track the progress of MPPA, but that its added value is to analyse the data of such M&E systems to answer specific policy questions.
*****2. Create a common understanding
Before engaging in the process of question formulation, it is crucial that all key actors share a common understanding of what NIPN’s purpose and approach is. Even when several advocacy or awareness-raising activities have taken place prior to the workshop, it is important to start the workshop with a generic presentation of NIPN (section 1.1, Advocacy and awareness raising) to create a common understanding amongst all participants.
The scope of NIPN and its added value can be mentioned briefly in this presentation in order to manage participants’ expectations with regard to the type of answerable questions, but this should be a specific and separate session on the agenda of the consultative workshop. This session can be facilitated by a Data analysis expert, using the Technical Note on Data Analysis.
***** -
Activity A: Setting the agenda of the workshop (2/7)
3. Which questions and data analyses lend themselves best to the NIPN approach?
Because one of the key principles of NIPN is to use and analyse existing data, certain type of questions, and thus analyses, add a lot of value to the policy dialogue in country and are therefore highly recommended.
In a context of increased accountability for the Multisectoral Nutrition Plans of Actions to achieve national and global targets, analyses of existing data will help to better understand "implementation progress” towards these targets. Progress can be measured at different levels of the impact pathway, which ranges from inputs (financial and human resources, activities (interventions), outputs (coverage) and outcomes (determinants) to nutrition impact (see page 7). Examples of appropriate questions are listed in the table below.
Existing data, however, are often less useful to answer questions related to measuring causality, impact and cost-efficiency. To answers such type of questions, data need to be collected for that very purpose, using a specific research set-up and data collection methods, to ensure robust and appropriate data analysis and correct interpretation. Use of existing data for modelling or other complex analyses will lead to results that cannot be easily translated into actionable recommendations for policy decision makers, and therefore do not add much value to the policy dialogue. Though questions regarding causality, impact and cost-efficiency are very legitimate and of high importance to decision makers, they are not recommended for analysis by NIPN.
The choice for data analysis methods is based on data availability , data quality and capacity of the team. The details of this information are explained in the guidance note on data analysis (section 3.4), as well as in the Technical Note on Data Analysis.
*****Type of questions in NIPN approach ***** -
Activity A: Setting the agenda of the workshop (3/7)
4. Provide concrete examples
The NIPN data team will need to review the list of initial questions prior to the workshop, in order to decide which questions are well-formulated and have the best potential to provide an actionable answer.
To facilitate good understanding of the participants, it is important to illustrate the theoretical explanations with concrete examples. It is recommended that the initial questions formulated by NIPN stakeholders are used as practical examples to get to well-formulated questions (see output of Step 1, section 2.2). The guidance note on data analysis (section 3.4) also provides a number of examples.
In many cases, the initial questions can be reformulated into answerable (sub-)questions. At all stages of question formulation however, it is important to manage stakeholders’ expectations and be transparent about the fact that not all questions may be answered, which questions will be prioritised and why, and what the next step will be.
Questions and data analyses which are not prioritised or cannot be answered by NIPN could become part of a ‘Question Bank’ (see section 2.5). The data collection or analyses of these questions might possible for other nutrition partners.
*****Example of policy-relevant initial questions At national level- Can the targets for reducing chronic malnutrition be reached within the timeframe of the National Plan of Action?
- Should a pilot project on community-based behaviour change interventions be scaled up to other districts?
- Which nutrition interventions should be prioritised to maximise impact on stunting?
- Are nutrition interventions of the National Plan of Action reaching the target population for all sectors?
- What can explain that some nutrition-specific interventions have lower coverage in some districts compared to other districts? In which districts can coverage be improved and what needs to change in terms of implementation to achieve this?
- How many cases of stunted children can be averted if national targets are to be achieved? Can these targets be achieved with the current intervention coverage levels? Thus, what needs to change in term of implementation?
At sub-national level
- What explains that some districts have lower malnutrition levels than others over past years of implementation of the National Multisectoral Plan of Action for Nutrition?
- How do districts perform on stunting, underweight and anemia among children under the age of 5?
- How are budget expenditures / budget allocations for nutrition-related interventions distributed across sectors?
- How do key outcome indicators (determinants) vary across these districts? (p.e. what are the levels and trends of timely initiation of breastfeeding or adequate household dietary diversity)
- How does the district perform on implementation of nutrition specific interventions along a continuum of care? Is coverage of services to women of reproductive age, pregnant women, new mothers and new-borns adequate ?
***** -
Activity A: Setting the agenda of the workshop (4/7)
5. Adopt an ‘impact pathway’ approach to help formulate policy questions
The impact pathway model is a logical way to organise the various elements (inputs - activities - outputs - outcomes - impact). As shown in the animation below, the logical flow can be tested by moving from one element to the next. The relationship between two elements is based on underlying assumptions. For instance, the overall assumption is that if the multisectoral nutrition plan is well designed:
- the funding and human resources (inputs) are allocated as planned,
- the interventions are implemented (activity) with the desired quality,
- this would lead to adequate coverage of the target population (output),
- thus improving the intended behaviour (outcome, such as breastfeeding) which ultimately will lead to
- impact on the target nutrition indicator (for instance stunting).
It is important to test whether the data confirm that the relationship between two elements indeed exists, as well as the assumptions themselves.
Using the framework of the impact pathway will help to unpack broad policy questions, which are often related to impact, into sub-questions about the earlier elements in the impact pathway, which are more likely to be answerable with existing data. Indeed, following the impact pathway, policy-relevant questions and sub-questions can be formulated with respect to each individual element, the relationship between elements, or the underlying assumptions.
For instance, it is difficult to answer the following question: “Did the investments in nutrition-specific interventions in the Multi-sectoral Nutrition Policy and Plan of Action, phase II (MPPA-II), reduce stunting significantly over the same period?”
The question asks whether the inputs (investments) at the start of the pathway resulted in impact on a key nutrition indicator at the end of the pathway. Yet, there are so many different factors along the pathway which could have influenced the implementation of the MPPA that it is impossible to answer this question without a controlled research set-up and measuring all confounding factors. It is, however, possible to break down this broad question into a set of smaller questions pertaining to each step of the impact pathway, such as:- Was the budget allocated and dispersed as planned in the MPPA?
- Were the interventions implemented as planned?
- Were the target populations reached by the interventions as planned?
Impact pathway models, or a theory of change already adopted by the country to monitor the MPPA, may provide a good basis for triggering the discussion on policy relevant questions and preparing this workshop exercise. These models will be particularly useful to identify the intervention logic of the MPPA and the existence (or gaps) of indicators.
In the absence of impact pathway models or a MPPA theory of change, impact pathways models of the REACH Compendium of Actions can serve as a good basis to start the exercise.
*****The impact pathway can guide the formulation of policy-relevant, answerable questions <video_responsive286|center>
***** -
Activity A: Setting the agenda of the workshop (5/7)
6. Illustration of the use of the impact pathway to support the formulation of question
Tips
- Unpack what is happening at the level of inputs and activities, filling in each element of the impact pathway based on the information from the MPPA.
- Consider all the evidence available to inform the degree of implementation of the MPPA.
- Consider whether changes took place during the period of MPPA implementation or compared with the period before, in terms of availability and quality of intervention implementation, in terms of reach (all population) or coverage (target population) of the intervention (step 1, section 2.2).
- Consider variation in intervention coverage or nutrition indicators across population groups (e.g. by income quintile, or rural/urban) and geographies.
Refer to the examples 1 to 3 below.
*****Examples -
Activity A: Setting the agenda of the workshop (6/7)
7. Characteristics of a well-formulated policy question
Remind workshop participants about the definition of a nutrition policy-relevant question. A “nutrition policy-relevant question” means a question that:
- responds to a relevant policy need or demand from decision makers
- is answerable with existing data, and for which data analysis methods and capacities are available
- allows a timely answer for policy use
- provides an answer that leads to actionable recommendations and decisions.
In addition to the definition of a ‘nutrition policy-relevant question’, there are a few characteristics that may be useful to bear in mind for the actual formulation of the question. These characteristics mainly help to refine the question, making it sufficiently specific in terms of target group, indicator, and time frame to allow appropriate data analysis. See the example below of question reformulation.
*****Elements that need to be specified in the question *****In fact, the question needs to be formulated in such as way that it can be answered by quantitative analysis. For example, questions formulated as ’What can we learn from...’ are not specific enough and need to be reformulated to be answerable by data analysis.
’Why..’ or ’What are the causes of...’ type of questions may also not be answerable with existing data. They can however be broken down further or reformulated to become answerable (for more detailed explanation, see page 5 or the guidance note on Data analysis (section 3.4) or the Technical Note on Data Analysis).
*****Example of question reformulation -
Activity A: Setting the agenda of the workshop (7/7)
8. Prepare relevant and concrete country examples to trigger question formulation
Concrete examples are key to a better understanding for the participants in the workshop: the participants need to go through the question formulation process and practise by themselves how impact pathways models can help. Thus, it is important that the facilitator prepares and practices relevant and clear examples prior the workshop.
Especially when participants engage for a first time in the NIPN cycle of ‘question-analysis-dissemination’, it has proven challenging for participants to grasp what are possible policy-relevant questions and how come to a well-formulated question. Concrete examples will help them to initiate and understand the process.The priority questions which are the output of Step 1 (see section 2.2) may provide a good starting point for developing the examples for the workshop. In all cases, it is important to:
- Ensure that examples are prepared prior to the workshop and are available on paper / on screen for participants to practise with.
- Present and explain at least one unique example to all participants, independent of whether it relates to their own sector or not.
- Include examples of sector-specific questions that are aligned to nutrition-sensitive topics in the MPPA, to facilitate understanding by the sector experts.
The four steps described in this guidance note for formulating policy-relevant questions can be applied at any administrative level.
In cases where the NIPN operational cycle is implemented at decentralised or sub-national level, it is important that the steps described in this guidance note lead to question formulation that allows a policy dialogue around the specific needs and specific strategic priorities at sub-national level.
*****Examples of strategic priority needs and policy-relevant questions at national and sub-national level *****
