Case study

  • What do we mean by “Sub-National” divisions in NIPN countries?

    Administrative sub-divisions differ across countries. For the purpose of this tool the following definitions are adopted:

    • The generic term “Region” is equivalent to the “Sub-National Administrative Level N-1”, which typically splits the country into 10-20 sub-divisions.
    • The generic term “District” is equivalent to the “Sub-National Level N-2”, which typically splits the country into 30-100 sub-divisions.

    “Region” is used as a generic term in the tool for “Sub-National Level 1”. The notable exception is Côte d’Ivoire where “Region” actually refers to “District”.

    Sub-national divisions in NIPN countries
    Generic term used in the guidance notesRegionDistrict
    CountriesLevel N-1Level N-2
    Bangladesh 8 divisions 64 districts
    Burkina Faso 13 regions 45 provinces
    Côte d’Ivoire 12+2 districts 31 regions
    Ethiopia 11 regional states 68 zones
    Guatemala 22 departments 335 municipalities
    Kenya 47 counties
    Lao PDR 17 provinces + Vientiane districts
    Niger 7 regions + Niamey 63 departments
    Uganda 15 sub-regions districts
    Zambia 10 provinces 103 districts
  • The SUN MEAL national and sub-national dashboards

    Link:
    https://scalingupnutrition.org/progress-impact/monitoring-evaluation-accountability-learning-meal/country-dashboards/

    National dashboards have been produced for the SUN countries. Sub-national dashboards are currently being produced, which are slightly different and designed to compare data across districts.

    Main features of the SUN MEAL dashboard:

    • Clear sub-sections that correspond directly to the SUN Theory of Change and also align in many respects with the UNICEF and Lancet nutrition frameworks.
    • Indicators with “no data” appear clearly (although not shown in this example).
    • Most recent year of when the indicator was collected is included.
    • Data sources are included.
    • Colour coding clearly shows performance compared with median performance of other countries (or national median for sub-national dashboard).
    • Sub-national dashboards that are not yet fully finalised are colour coded to compare one region with another.
    • A summary narrative is included.
    • A page to explain the colour classification is included.
    • 6 pages in total for national dashboards; 8 pages for initial draft sub-national dashboards (not yet published).

    The SUN MEAL dashboard contains the following sections:

    • Enabling environment
    • Finance for nutrition
    • Intervention and food supply
    • Enacted legislations
    • SDGs drivers of nutrition
    • IYCF and dietary Intake
    • Nutrition status
    • SDGs linked to nutrition
    Example

    Extract of the SUN MEAL national dashboard for Lao PDR

    JPEG - 98.1 kb
  • The POSHAN District dashboards (India)

    Link:
    http://poshan.ifpri.info/category/publications/district-nutrition-profiles/

    Main features of the POSHAN district dashboard:

    • District level dashboard
    • Clear sub-sections
    • Very clear and simple bar chart with average of the district and average of the state. One issue with this sort of bar chart is the scale as it is difficult to compare indicators ranging from 10 to 20% with those ranging from 40 to 50%.
    • Indicators with “no data” appear clearly.
    • Data sources are included.
    • A section with “Possible points of discussion” is included.
    • A page describing the determinants of malnutrition and sources of information
    • 4 pages in total for sub-national dashboards

    The POSHAN district dashboard contains the following sections:

    • Demographic profile
    • The state of nutrition
    • Immediate determinants of nutrition
    • Coverage of nutrition-specific interventions
    • Underlying and basic determinants of nutrition
    • Interventions that affect basic and underlying determinants
    Example

    Extract from the POSHAN dashboard of the Bokaro district

    JPEG - 85.1 kb
  • The Global Nutrition Report national dashboards

    Link:
    https://globalnutritionreport.org/

    Main features of the GNR natonal dashboard:

    • Clear sub-sections
    • Helpful visual lay out with variety of colours but quite dense
    • Some graphs difficult to interpret right away
    • Data sources are included
    • Analysis of inequalities as defined by wealth quintile included
    • Section on progress against Global Nutrition Targets included
    • 2 pages in total for national dashboards

    The GNR national dashboard contains the following sections:

    • Economics and demography
    • Child anthropometry
    • Adolescent and adult nutrition status
    • Progress against Global Nutrition Targets
    • Intervention coverage and child feeding practices
    • Underlying determinants
    • Financial resources and policy, legislation and institutional arrangements
    Example

    Extract from the GNR national dashboard of Uganda

    JPEG - 67.4 kb
  • Countdown 2030 maternal, new-born and child survival national profiles

    Link:
    http://countdown2030.org/country-and-regional-networks/country-profiles

    Main features of the Countdown 2030 national profiles:

    • Clear sub-sections
    • Helpful visual lay out with variety of colours but quite dense
    • Interactive web-based profile with a lot of potential
    • Equity plots that are quite easy to interpret are included
    • Data sources are included through a link

    The Countdown 2030 national profile contains the following sections:

    • Demographics
    • Continuum of care coverage
    • Equity
    • Maternal & newborn health
    • Women’s and children’s nutrition
    • Demographics
    • Child health
    • Policies, system, financing
    • Environmental
    Example

    Extract from the Countdown 2030 national profile of the Republic of Côte d’Ivoire

    JPEG - 93.5 kb
  • Nutrition Landscape Information System (WHO NLiS) country profiles

    Link:
    http://apps.who.int/nutrition/landscape/report.aspx

    Main features of the WHO NiLS country profile:

    • Clear sub-sections
    • Year of when the indicator was collected is included
    • Direct link to source of information (good for web readers, less for readers of printed dashboards)
    • Very clear information on the meaning and interpretation of each indicator (click window)
    • Section on policies and programmes listed in the GINA database included

    The WHO NiLS country profile contains the following sections:

    • Child malnutrition
    • Malnutrition in women
    • Vitamin and mineral deficiencies
    • Health services
    • Food security
    • Caring practices
    • Commitment
    • Capacity
    • Meta indicators
    • Policies and actions in the Global Database on the Implementation of Nutrition Action (GINA)
    Example

    Extract from the WHO NiLS country profile of Guatemala

    JPEG - 50.5 kb
  • Information system, dataset, indicator

    An information system is a system of interrelated components that work together for collecting, processing, storing and disseminating information amongst stakeholders/beneficiaries to support decision-making, coordination, control, analysis and visualization (e.g. DHIS-2; National Statistics Office repository; DHS platform).

    A dataset is a file that contains all the individual records of one specific survey (e.g. DHS survey in country “X” of 2005). A dataset can be available in one or more information systems. For example, the Niger DHS survey of 2012 can be found in the STATcompiler information system (DHS official platform) and in the information system of the National Statistical Office of Niger.

    An indicator is calculated based on one or more variables from a dataset.
    The same indicator can be found in different datasets but may or may not be comparable because:

    • The definition of the indicator is not exactly the same. For example, the definition of some indicators of DHS surveys have evolved over time.
    • The survey sampling methodology is different.
  • Two options to overcome challenge n°1

    Option 1: Limit the exercise to the level of the datasets (exclude the indicator matrix)

    The exercise is still vast and the scope of the exercise can be further reduced by considering the following:
    1) The range of datasets to investigate
    For example, Ivory Coast decided to investigate the datasets managed by key sectors involved in nutrition (health, education, gender, social affairs, agriculture, animal resources, water, economy, finance & planning) and other agencies known for managing databases (National Agency for Rural Development, National Water Offices, etc.)
    “Key sectors” are those involved in the Multi-sectoral Plan of Action for Nutrition. In some countries this list can be much more vast, including more than ten ministries. For the sake of the data landscape exercise, it is recommended to reduce the list of data providers to investigate to keep the exercise feasible. Other sectors could be investigated in a second stage using the experience of the first exercise.

    2) The level of detail of information to collect for each dataset
    NIPN teams can decide the level of detail necessary to collect for each dataset. One option is to request a detailed description of datasets only for some prioritised sectors.

    Option 2: Include datasets and indicator matrix in the data landscape exercise

    The scope of the data landscape exercise can be reduced by looking at:
    1) The number and range of indicators to investigate
    There are several ways to decide the number and range of indicators to include in the matrix:

    • Select key data providers instead of selecting indicators. For example, the Ivory Coast NIPN country team decided to investigate all the data available in key ministries (health, education, gender, social affairs, agriculture, animal resources, water, economy, finance & planning) and other agencies known for managing databases (National Agency for Rural Development, National Water Offices, etc.)
    • Refer to the M&E plan and list of multi-sectoral indicators that are attached to a national multi-sectoral plan of action for nutrition. If this list is still too long, key indicators can be identified and selected. For example, the Guatemala NIPN country team selected 70 key indicators from the M&E plan. This would mean that the indicators matrix is limited to those 70 indicators but the description of the datasets would need to have a wider scope.
    • Adapt the SUN MEAL system which includes a list of multi-sectoral indicators (about 300) and a sub-selection of key indicators (about 70) for nutrition.

    2) The level of detail of information to collect for each indicator
    There are several options to consider when deciding the level of detail for each indicator, which include:

    • Complete a full indicator matrix. The indicator matrix available here is an example of an in-depth assessment for each selected indicator.
    • Compile only basic information for each indicator. For example, the NIPN in Burkina Faso listed all the indicators available in the datasets and their definition only.
    • Complete the indicator matrix in full for key sectors only (Health, Agriculture, Education, etc.) and collate less detailed information for the sectors that are of lower priority. For example, the NIPN in Ethiopia decided to focus on nutrition & WASH in the first phase. This means that a detailed indicator matrix can be completed for these two domains with less detailed information for the other domains.
  • The National Evaluation Platform (NEP) project

    The NEP is an initiative of Johns Hopkins University in four African countries (Mali, Malawi, Mozambique and Tanzania) that was supported by the Government of Canada between 2014 and 2018.
    Its objective is to equip government decision makers with the tools and skills needed to critically evaluate the state of maternal, newborn, and child health and nutrition in their countries, and support sound decision making. It is built through a cycle-based approach that progressively adds new types of data, analytical tools and communications skills, and disseminates findings to policy makers concerned with maternal and child health and nutrition.

    NEP has been implemented through a country-owned and government-led approach. It works with multiple national stakeholders concerned with maternal and child health, nutrition data and decision making, who all have an interest in improving health and nutrition and decreasing mortality outcomes.
    To consult the NEP country experience, or to review the outputs generated and lessons learnt, you can consult the NEP website.

  • EXERCISE 1: Start with understanding the magnitude and the trends of the undernutrition problem and the trends in relation to national or sub-national targets

    Work with the data analysts to gain a picture of the magnitude and trends of the undernutrition problem: it is recommended that DHS and any other prevalence point from validated national surveys are used.
    Draw the scenarios 1) if trends continue as usual, and 2) if national targets are to be met.
    This exercise is to be repeated at the sub-administrative level (region, district) at which the NIPN cycle ‘questions-analysis-findings’ is strategically interested in focusing on. As data allow to do so, the NIPN cycle should strengthen the decentralized decision-making level as much as possible, and thus the questions identified also respond to that level.

    JPEG - 81.7 kb
  • EXERCISE 2: Map out key multi-sectoral policies and plans of action in relation to the undernutrition trend

    Present visually the main multi-sectoral policies and plans in relation to the undernutrition trend and consider how the trend evolved during the implementation period.
    Do it for the most relevant policies, programmes and/or changes in investment in nutrition that are believed to have occurred in previous years.

    JPEG - 100.4 kb

    The priorities and interests of stakeholders will differ depending on the stage of the policy or plan:

    • At formulation stage, the focus will be on the definition of targets, the selection of interventions, the required coverage, etc.
    • During implementation, the priorities will be around progress of implementation and probability of reaching targets.
    • At evaluation, the focus will be on impact: what worked, what did not.

    Each stage represents a strategic opportunity to improve the next one, should the corresponding information be available to decision makers.
    The assumption is that if the multi-sectoral policy, plan or programme is well designed, and the interventions are implemented 1) according to planned coverage and 2) with the desired quality, an impact should be seen on the intended outcome.
    Visualising the implementation period of the policy, plan or programme in relation to the undernutrition trend will provide an initial idea of the probability of these assumptions being correct.

  • EXERCISE 3: Create a timeline of the multisectoral policies and plans

    • Display the implementation periods of the multi-sectoral policies, plans and programmes on one timeline.
    • Add complementary information on the context of implementation, as shown in the fictional country example below.

    This chronogram can help to confirm the priorities of policy makers for the next 12-24 months and will help to identify possible windows of opportunity for influencing planning, formulation or evaluation cycles.

    The fictional example below shows that policy makers in this country need information in 2018 on the progress of implementation of the Multi-sectoral Plan of Action, phase II, and the probability that the plan will achieve its targets by 2020. The absence of a mid-term evaluation makes this information need, which can be filled by NIPN, more acute.
    With an upcoming political transition in 2019, the new policy makers will also need information regarding ‘implementation progress’ to inform the formulation of new policy by 2020.

    JPEG - 74.4 kb

    NOTE: The relationship between multi-sectoral and sectoral policies or plans can be presented visually in a similar way, to identify whether the latter could offer an entry point to strategically supporting multi-sectoral efforts.

  • EXERCISE 4: Identify which policies, programmes or investment decisions are more likely to be influenced and decide which administrative level to influence

    Ultimately, the review process will give the NIPN team an initial idea of the decision makers’ priorities and the corresponding time frame on which the ‘questions-analysis-findings’ cycle should focus.
    The initial selection of the priorities will be based on the information pulled together in the policy review, using the previous tips, and on the feedback from stakeholders.
    At the same time, the team will need to decide at which administrative level the NIPN cycle intends to strengthen the decision-making process. This decision will depend on how decentralized nutrition interventions are, the interests of policy makers and the availability of data at sub-national level. The demand for information is likely to be higher at sub-national level, where capacity to use collected data is often sub-optimal and a feedback loop is often lacking. The sub-national dashboard can support analysis at the decentralized level as long as the process is initiated by decision makers’ interests and question formulation.

  • EXERCISE 5: Identify and know the key ‘decision makers’ at the various administrative levels

    At this stage, the team should have good knowledge of the various decision makers along the implementation chain. Decision makers are not necessarily limited to policy makers: programme planners and implementation officers are other actors along the implementation chain who are also making decisions to improve nutrition actions at their level.
    Categorizing the decisions makers, according to the administrative level at which they intervene and according to the type of decisions they can make, will help to assess the diversity of the NIPN stakeholders’ needs.
    It will also define the target audiences and consequently help to fine-tune the policy relevance of the questions and align the ‘storyline’ of the findings with each target audience of decision makers.

  • EXAMPLE 2: Identify the different elements of a specific policy question against the impact pathway and break them down into sub-questions

    Have investments (input) in WASH interventions (activity) led to better access to WASH facilities (output), resulted in a reduction in % of children suffering from diarrhoea (outcome) and reduced child undernutrition (impact)?

    1. Break the question down into more specific questions to better understand the intermediate steps of the impact pathway.
    For example:

    • Have investments in WASH interventions changed over the past 5 years?
    • How are investments in WASH interventions distributed at sub-national level?
    • Has coverage in WASH interventions changed over the past 5 years?
    • Are WASH interventions reaching the target populations?
    • Have WASH interventions led to improved access to latrines/access to safe drinking water?

    2. Unpack the question to identify the indicators, the relationship between the indicators and any assumptions, in order to generate more specific questions that can be answered by the data available.
    For example:

    • Indicators can be investments or programme coverage.
    • A relationship is: “WASH investment leads to increased WASH programme coverage.”
    • An assumption is: “WASH investments translate directly into increased programme coverage.”

    3. Zoom in on a specific question and unpack it further.
    For example:

    • The latest assumption can in turn become a question to be unpacked into further questions: “WASH investments translate directly into increased programme coverage.”
    • Have investments in WASH translated into equal budget expenditures at regional/department level for WASH interventions?
    • In which WASH interventions have investments been made at regional level? What was the distribution of budgets across regions?
    • Has staff been hired, trained and supervised in each region equally? Were supplies and equipment equally available in each region to achieve planned coverage?
    • etc.
  • EXAMPLE 3: Formulating questions related to a social protection pathway for impact on nutrition

    Based on the SUN Movement Secretariat (SMS), 2015. The contribution of agriculture and social protection to improving nutrition; Scaling Up Nutrition in Practice. Geneva.

    JPEG - 170.3 kb

    Developing more detailed impact pathways than the ones shown in this example, which highlight specific assumptions and specific relationships between activities, outputs and inputs, will allow the formulation of more specific and detailed questions to help identify bottlenecks, such as:

    • “Has the budget allocated to the implementation of social protection interventions also been used to strengthen the human resource capacity, and has this in its turn led to a higher quality of services and better targeting?”
    • “What are the reasons for a lack of increase in coverage of social protection interventions? - Insufficient number of staff, low level of training, low quality of service delivery, insufficient number of distribution points, inadequate targeting, incomplete monitoring?”
  • The types of data that can be used by a NIPN

    NIPN is unique in that it brings together and values multiple data sources shared by the various sectors that influence nutrition: health, agriculture, water, sanitation and hygiene, social protection, and education, among others.
    Typical data that NIPN would use comes from:

    • National population-based surveys (which can be representative at sub-national level)
      • Demographic and Health Surveys (DHS); Multiple Indicators Cluster Surveys (MICS); National Nutrition Surveys (NNS); Household Incomes and Expenditures Surveys (HIES); Health and Nutrition Expenditures Surveys (HNES); Services Availability and Readiness Assessment (SARA), etc.
    • Local population-based surveys (representative of one district/region)
      • Nutrition Surveys (SMART); Household surveys; Intervention coverage surveys, etc.
    • Programme data
      • Routine data collected by each sector
      • Monitoring data collected by each sector
    • Early warning system data
    • Financial data
      • Evaluation of budget needed (Nutrition Plan of Action)
      • Official commitments for financial investments
      • Budget allocation at central and sub-national levels
    • Modelling data

    This list is not meant to be exhaustive. Each country should explore the wealth of information available. The data landscape exercise is useful for this purpose (see section 3.1).

  • Example 1: Trend analysis of stunting to track progress

    Using national survey data from DHS and MICS, a trend analysis can be used to estimate the Average Annual Reduction Rate of stunting (see section 3.6). The same method can be applied to other target indicators.
    With this method, NIPN can answer questions such as:

    • What is the current trend of stunting reduction?
    • Has the reduction of stunting accelerated over the period of implementation of the last Multi-sectoral Plan of Action for Nutrition?
    • Is the current trend in stunting reduction sufficient to reach the target?
    Global average annual reduction rate for stunting

    (Source: EU Action Plan on Nutrition)
    This analysis carried out at global level shows that the current AARR is not sufficient to achieve the planned target of 2025.

    JPEG - 86.1 kb
  • Example 2: Venn diagrams to visualise the coexistence of multiple forms of malnutrition

    This Venn diagram shows the population affected by one or more forms of malnutrition.
    It is taken from the Global Nutrition Report, which analysed national level data.
    Similar analysis can be carried out at sub-national level to show which regions are affected by which forms of malnutrition and what actions are needed.
    It allows the issue of the double burden of malnutrition to be highlighted.
    The same figure with the actual number of children affected by region can highlight where to invest.

    Coexistence of multiple forms of malnutrition in one African country

    (Source: Global Nutrition Report)

    JPEG - 36.9 kb
  • Example 3: Descriptive analysis to improve the targeting of a social assistance programme in Mexico

    A National Food Assistance programme has been implemented in Mexico for decades.
    In 1994, a descriptive analysis of the levels of incomes of the beneficiaries of the programme showed that the programme was not very effective at targeting the poorest households.
    After active measures were taken, the same analysis showed that in 2000 a much greater proportion of the programme’s beneficiaries were indeed those with the lowest incomes.

    Benefit incidence of Food Oriented Social Assistance, by income decile, 1994–2000

    Source: Levy, S. (2006). Progress against Poverty. Sustaining Mexico’s POP Programme. Washington, DC: Brookings Institution Press.
    Between 1994 and 2000, the Government of Mexico improved the targeting of the Food Oriented Social Assistance programme:

    • In 1994, less than 10% of the beneficiaries belonged to the lowest income decile.
    • In 2000, more than 30% of the beneficiaries belonged to the lowest income decile.
      JPEG - 37.6 kb
  • Example 4: Descriptive analysis to improve the targeting of hygiene interventions

    A descriptive analysis looking at regional disparities shows that the three regions with high levels of stunting also have high levels of open defecation practices.
    This can be a starting point to further investigate why open defecation is high in those three regions and whether other determinants are also showing high levels.

    Open defecation and stunting rates by region

    (Source: REACH, Example of Ethiopia MICS, 2011)

    JPEG - 91.9 kb
  • Example 5: Descriptive analysis of coverage of health interventions in India

    Iron-folic acid supplements are delivered through health services. This combined data suggests that most districts with low levels of iron-folic acid supplements also have low ANC visits coverage, suggesting an issue with access to health services.
    But it also shows that quite a few districts with good ANC visits coverage also have a low coverage of iron-folic acid supplements, suggesting issues with delivery.
    Further investigation can provide interesting insights into what is needed to improve the coverage of iron-folic acid supplementation.

    Coverage of interventions

    Source: POSHAN

    JPEG - 116.2 kb
  • Example 6: Equity analysis of child feeding practices

    This equity analysis of child feeding practices shows that some practices are more sensitive to the levels of incomes (minimum diet diversity) than others (early initiation).
    Equity analysis is useful for determining whether the progress achieved is benefiting all and, in particular, the most vulnerable.
    If most of the progress is only observed in households with higher incomes, this suggests a problem with the targeting or design of the interventions.

    Child feeding practices by Wealth Quintiles in one African country

    (Source: Global Nutrition Report)

    JPEG - 52.5 kb
  • Confounding

    The characteristics of a potential confounder are as follows:

    • It must be associated with the outcome;
    • It must be associated with the exposure of interest;
    • It must not be an intermediate step in the causal pathway between the study exposure of interest and the outcome.

    Example: Has investment in programme A led to reduced anaemia levels?
    Possible factors confounding a conclusion regarding the impact of one intervention on anaemia levels could, for instance, be:

    • the coexistence of another programme with a potential impact on anaemia reduction, targeting the same population groups;
    • lower malaria incidence in the intervention area due to reduced rainfall during the baseline measurement.

    Without controlling the data analysis for confounding factors, it is not possible to attribute the reduction of anaemia levels to the nutrition intervention.

    • While measuring an association and checking the temporality is relatively easy, controlling for all (known and unknown) confounding factors is very challenging, as data on all confounding factors is often not available in population-based surveys. This typically requires a randomised controlled trial in a research setting to measure and compare rates of anaemia and numerous potential confounding factors between the intervention group and a control group (not exposed to the intervention), before and after the intervention.
    • National population-based surveys and routine monitoring data typically do not have a control group.
    • Without a control group, trying to interpret an association or a causal relationship can be very misleading. There is a high risk of drawing an incorrect conclusion, such as the nutrition intervention having an impact on anaemia while in reality a confounding factor has caused the change in anaemia rates, and not the nutrition intervention.
      Data from this type of survey is not suitable for the analysis of a causal relationship and could lead to incorrect and misleading policy decisions.
  • Example: Why having a control group is essential for establishing a causal relationship

    This randomised controlled trial measured the impact of an intervention that was designed to reduce stunting.
    Between 2010 and 2014, it was observed that:

    • the control group (without the intervention) saw their stunting levels increase from 68.2% to 74.8%;
    • the stunting levels of the intervention group remained almost stable.

    Results from secondary analysis of Tubaramure, a food-assisted integrated health and nutrition programme in Burundi

    JPEG - 76.7 kb

    (Source: Leroy J.L., Olney D., Ruel M. 2016. Tubaramure, a Food-Assisted Integrated Health and Nutrition Program in Burundi, Increases Maternal and Child Hemoglobin Concentrations and Reduces Anemia: A Theory-Based Cluster-Randomized Controlled Intervention Trial. The Journal of Nutrition 146(8), p.1601–1608. https://doi.org/10.3945/jn.115.227462)

    • It is only by comparing the delta of pre- and post interventions for the control group and the treatment group that a robust conclusion can be reached: the intervention reduced stunting levels by 6.4 percentage points (74.8-68.2) – (64.3-64.1).
    • Without a control group, the measurements “before and after intervention” for the treatment group could be compared, leading to the conclusion that the intervention had no impact (increase of 0.2 pp).
    • Without baseline measurements, the populations “with and without” interventions could be compared, leading to the conclusion that the intervention had a very high impact (reduction of 10.5 pp).

    In the last two cases, an incorrect conclusion has been reached.

  • Data analysis plan

    1. Why a data analysis plan?
    A data analysis plan helps you think through the data you will collect, what you will use it for, and how you will analyse it. Analysis planning can be an invaluable investment of time” (Center for Disease Control and Prevention, 2013)
    The method for creating a data analysis plan in the context of a NIPN is not much different from the method used in a research context.
    In the context of NIPN, the process should be simpler because:

    • A data analysis framework is already produced (step 3 of question formulation process) and forms the basis for the more detailed data analysis plan (after step 4 of question formulation process).
    • In section 3.4, pages 7 to 9 data analysis methodologies are described.
    • NIPN is about the use of existing data, it is not about designing a protocol for new data to be collected.

    The next section describes briefly the content of a data analysis plan focusing on what is a bit specific to the NIPN.

    As general recommendations:

    • Don’t panic!
    • Use the advice and experiences from colleagues and experts
    • Quickly contact an expert when necessary

    Recommended sources to read:
    Centers for Disease Control and Prevention (2013) Creating an analysis plan. Atlanta.
    Simpson, S.H. Creating a data analysis plan: what to consider when choosing statistics for a study (2015).

    2. What is a data analysis plan?
    Main sections of a data analysis plan (based on CDC module):

    • Main question and sub-questions
    • Dataset(s) to be used
    • Inclusion/exclusion criteria
    • Variables to be used in the main analysis
    • Statistical methods and software to be used
    • Table shells
      => Estimation of time and resources needed

    3. Main question and sub-questions
    At this stage, the policy relevant question (and, in some cases, its sub-questions) is already well defined (section 3.4, page 11).
    Answering all the sub-questions will provide a full answer to the main question.

    4. Dataset(s) to be used
    The dataset(s) needed is(are) listed. In the context of the NIPN, particular attention may be needed on data management: as the dataset(s) may come from different sources and/or may not have been designed for the main question, there could be quite some work to be done to harmonise / append / clean the raw dataset(s).

    • Are the datasets comparable?
    • Are the indicators harmonized?
    • Is there a need to transform the data for the analysis?
      To answer these question, you need to have accessed the datasets in question.

    5. Inclusion/exclusion criteria
    In this section population sub groups, geographic scope, timeframe… are very precisely defined.
    You also need to clarify the data quality level required for the analysis.
    Indeed, depending on the analysis, you may need to be more or less strict on data quality level required.
    This is detailed in the Data Quality training module (section 3.3).

    6. Variables to be used in the main analysis
    In this section you define precisely variables/indicators to be used in the analysis.
    For example, if you analyse “obesity”, you need to precise if you refer to the Body Mass Index (BMI) and if you are going to use different categories of BMI or the mean or both.
    In the context of NIPN, the harmonization of the definition of indicators across datasets will be important.

    7. Statistical methods and software to be used
    Ensure coherence with section 4 of the guidance notes on data analysis.
    Also, to provide only undisputable analysis (principle 3 section 3.4, page 4), make sure that the statistical method used is coherent with the datasets available and the data quality of these datasets. The choice of the statistical method is key to avoid overinterpretation of the data that could lead to misleading conclusions.
    Does the NIPN team has the technical capacity to handle the statistical method and the software identified?

    8. Table shells
    Nothing specific to NIPN.

    9. Estimation of time and resources
    At this stage, a precise estimation of the time and resources needed to conduct the analysis should be made.
    If this estimation lead to more time than the initial estimation made during the data analysis framework, you may adjust the question/s to be addressed first.

  • Question 1: What is the spatial distribution of obesity and overweight in Bangladesh? Who are the most affected groups?

    The underlying objective behind this question is to be able to improve targeting of the intervention.

    WHAT are the characteristics of a well-formulated policy question?Comments for Question 1
    1. The question must respond to a relevant policy need The policy relevance is ensured by going through steps 1 and 2 of the question formulation process.
    2. The question must be answerable with existing quantitative data Data experts need to identify the main datasets available that measure this indicator.
    Population-based surveys seem to be a good source of data for this question.
    Modelled data also exists on BMI (www.ncdrisc.org) that can help answer the question.
    This has pros and cons that need to be carefully balanced when developing the data analysis plan.
    3. The question must provide a timely answer for policy use The timeliness of the answer is discussed during steps 1 and 2 of the question formulation process. Note that only the data analysis plan will be able to define precisely the time needed for the analysis.
    4. The analysis of the question must lead to actionable recommendations and decisions This is ensured by going through steps 1 and 2 of the question formulation process.
    5. The question must specify:
    a) the population groups
    b) the type of intervention
    c) the objective
    d) the time frame
    e) the expected outcomes
    a) The age groups need to be specified: Adults? Children? Adolescents? All?
    The data analysis plan will specify what is meant by “most affected groups”.
    Should the analysis look at overweight and obesity by:
    - Region? (need for representative data at sub-national level). Use the data where it is actionable. If decisions are taken at regional level, it makes sense to use regional subgroups.
    - Income quintiles?
    - Rural vs urban?
    - Incomes quintiles AND region?
    A brief literature review can help to define those groups in the context of Bangladesh. The data available can also limit the choice of the groups.
    b) This question is about a nutrition target, not a specific intervention.
    c) The underlying “policy objective” could be made more specific as this would guide the development of the data analysis plan.
    d) The question seems to suggest looking at the most recent data. If data is only available for 2005, is the analysis still relevant? The time frame can also be chosen based on official targets, or a strategic year for the launch of a new plan.
    e) BMI is usually used as the outcome indicator of overweight and obesity. The data analysis plan will detail whether the analysis will use categories of BMI or the mean BMI as the outcome indicator.
    6. The question must imply data analysis methods which are suitable for use with the NIPN approach The question implies the use of a descriptive data analysis method.
    The data analysis plan should detail the methods to be used, e.g. t-test to compare mean BMI among the different groups, or a graphic display of the mean BMI with confidence intervals.
  • Question 2: What are the determinants of stunting by region?

    WHAT are the characteristics of a well-formulated policy question?Comments for Question 2
    1. The question must respond to a relevant policy need The policy relevance is ensured by going through steps 1 and 2 of the question formulation process.
    2. The question must be answerable with existing quantitative data Data experts need to identify the main datasets available that measure stunting and determinants of stunting. Population-based surveys seems to be a good source of information for this question.
    3. The question must provide a timely answer for policy use The timeliness of the answer is discussed during steps 1 and 2 of the question formulation process. Note that only the data analysis plan will be able to define precisely the time needed for the analysis.
    4. The analysis of the question must lead to actionable recommendations and decisions This is ensured by going through steps 1 and 2 of the question formulation process.
    5. The question must specify:
    a) the population groups
    b) the type of intervention
    c) the objective
    d) the time frame
    e) the expected outcomes
    a)Need to specify the age groups: Children? Adolescents? All?
    b) This question is about a nutrition target and determinants, not a specific intervention.
    c) The underlying “policy objective” could be made more specific as this would guide the development of the data analysis plan.
    d) The question seems to suggest looking at the most recent data. If data is only available for 2005, is the analysis still relevant? The time frame can also be chosen based on official targets or a strategic year for the launch of a new plan.
    e) Stunting is the outcome indicator. The data analysis plan will detail whether the analysis will use prevalence of stunting (global or severe or both) or the mean Height- for-Age Z-score as the outcome indicator. The data analysis plan needs to describe precisely which “determinants” should be included in the analysis.
    6. The question must imply data analysis methods which are suitable for use with the NIPN approach The question implies the use of causal data analysis methods which do not lend themselves to the NIPN approach, as causal analysis using population-based data can be disputed and may not lead to actionable recommendations. Knowing that the levels of education, income, diarrhoea and exclusive breastfeeding are associated with stunting does not bring new information. A stronger association between one determinant and stunting in one region than elsewhere does not necessarily imply that this determinant should be a priority for intervention. Idem, in case a determinant is not associated with stunting.
    However, reformulating the question as follows: “What is the magnitude and severity of the prevalence of known determinants of stunting by region?” allows for descriptive analyses which are well suited here, and may lead to actionable recommendations and partly answer the policy objective. Comparing determinants as well as coverage of nutrition interventions between regions can lead to even more interesting recommendations. A sub-national nutrition dashboard could be helpful here (section 3.2).
  • Question 3: Have investments (input) in WASH interventions (activity) led to better access to WASH facilities (output), resulted in a reduction in % of children suffering from diarrhoea (outcome) and reduced child undernutrition and mortality (impact)?

    WHAT are the characteristics of a well-formulated policy question?Comments for Question 3
    1. The question must respond to a relevant policy need The policy relevance is ensured by going through steps 1 and 2 of the question formulation process.
    2. The question must be answerable with existing quantitative data Data experts need to identify the main datasets available that measure the indicators mentioned. Budget data, routine data, monitoring of progress indicators, and population-based surveys seems to be a good source of information for this question.
    3. The question must provide a timely answer for policy use The timeliness of the answer is discussed during steps 1 and 2 of the question formulation process. Note that only the data analysis plan will be able to define precisely the time needed for the analysis.
    4. The analysis of the question must lead to actionable recommendations and decisions This is ensured by going through steps 1 and 2 of the question formulation process.
    5. The question must specify:
    a) the population groups
    b) the type of intervention
    c) the objective
    d) the time frame
    e) the expected outcomes
    a) “Children”: age groups must be specified, for instance, 0-59m or 6-59m of age.
    b) This question is about an intervention in the WASH sector, but the exact interventions need to be described more specifically.
    c) The underlying “policy objective” could be made more specific, as this would guide the development of the data analysis plan.
    d) The question does not specify the time frame. Is it based on the time frame of the official action plan on WASH intervention?
    e) Diarrhoea is the outcome indicator. The data analysis plan will detail whether the analysis will use prevalence or incidence of diarrhoea. The data analysis plan needs to describe precisely the other indicators that should be included in the analysis (WASH indicators, but also impact indicators, namely stunting and mortality).
    6. The question must imply data analysis methods which are suitable for use with the NIPN approach The question can be broken down into more specific questions to gain a better understanding of the intermediate steps of the impact pathway:
    - Have investments in WASH interventions changed over the past five years?
    - How are investments in WASH interventions distributed at sub-national level?
    - Has coverage in WASH interventions changed over the past five years?
    - Are WASH interventions reaching the target populations?
    - Have WASH interventions led to improved access to latrines/access to safe drinking water?
    These questions can be answered by suitable descriptive data analysis methods using existing data. The last part of the pathway regarding the impact of repeated diarrhoea episodes on stunting can be tackled with a literature review and/or using a modelling tool such as LiST.
  • Question 4: How much impact can we achieve with nutrition-specific interventions on stunting?

    WHAT are the characteristics of a well-formulated policy question?Comments for Question 4
    1. The question must respond to a relevant policy need The policy relevance is ensured by going through steps 1 and 2 of the question formulation process.
    2. The question must be answerable with existing quantitative data Data experts need to identify the main datasets available that have estimates on stunting and coverage of nutrition-specific interventions.
    3. The question must provide a timely answer for policy use The timeliness of the answer is discussed during steps 1 and 2 of the question formulation process. Note that only the data analysis plan will be able to define precisely the time needed for the analysis.
    4. The analysis of the question must lead to actionable recommendations and decisions This is ensured by going through steps 1 and 2 of the question formulation process.
    5. The question must specify:
    a) the population groups
    b) the type of intervention
    c) the objective
    d) the time frame
    e)the expected outcomes
    a) Need to specify the age groups: Children? Under-two or under-five?
    b) The data analysis plan needs to specify the list of interventions that should be part of the analysis. The list of Essential Nutrition Actions or the interventions prioritised in the Nutrition Action Plan can be used.
    c) The underlying “policy objective” could be made more specific as this would guide the development of the data analysis plan.
    d) The time frame is not very precise. Is it based on official targets (e.g. 2025)?
    e) Stunting is the outcome indicator. The data analysis plan will detail whether the analysis will use prevalence of stunting (global or severe or both) or the mean of the Height for Age Z-score. The data analysis plan needs to describe precisely the coverage indicators that should be included in the analysis.
    6. The question must imply data analysis methods which are suitable for use with the NIPN approach The question implies the use of causal data analysis methods which are not suited because population-based data cannot provide a robust measure of the effect of an intervention on stunting, since there is no comparison possible with a control group.
    However, some research studies have measured the effect of interventions on stunting, especially nutrition-specific interventions, and, on the basis of these studies, LiST has modelled how a coverage increase of nutrition-specific interventions may result in the number of stunted children prevented. LiST requires data on the coverage of interventions.
    After reformulation, this question can be answered using the LiST tool:
    What is the number of children prevented from being stunted and from dying if the coverage of a package of nutrition-specific interventions is increased by 20% between 2019 and 2025?
  • Experience of NIPN management in Guatemala

    In Guatemala, a Steering Committee provides clarity and transparency on plans and arrangements to the main NIPN partners. It comprises representatives of the EU Delegation, as the main donor supporting the NIPN; the Secretariat for Food and Nutrition Security (SESAN), as the government host for the NIPN; and CATIE (Centro Agronómico Tropical de Investigación y Enseñanza), as the organisation managing the grant and providing technical assistance.

    The Steering Committee meets regularly (three times a year), meetings are well attended and comprise in-depth discussions on progress, barriers to progress and how to overcome them.

    CATIE has assigned an experienced coordinator, an assistant and a financial administrator to manage NIPN. The CATIE team meets weekly to discuss progress and issues and are in regular (daily) contact with the government’s NIPN host, SESAN, about project implementation.

    CATIE has developed an elaborate project management tool in Excel, which tracks goals, activities and spending, with a simple traffic light system to signal progress or issues to the Project Steering Committee.

    Download the Excel project monitoring template from CATIE.

    *****
  • Examples of documents to review

    Documents that can be reviewed during the desk study include:

    • Government resources such as: national policy, plans and programmes for nutrition (multisectoral and sector-specific with nutrition objectives).
    • Non-governmental resources such as:
      • Scaling Up Nutrition (SUN) movement UN Network online country reports and SUN country Joint Annual Assessments.
      • UN Network/REACH supported documents: policy review, stakeholder and action mapping, common narrative or situation analysis overview and nutrition barometer.
      • Specific NIPN policy review
  • Interview with the NIPN Niger team about the data landscape exercise

    November 15th, 2018

    • Issiak Balarabé Mahamane, Assistant to the General Secretary, INS
    • Guillaume Poirel, Head of Mission for NIPN Technical Assistance, SOFRECO

    Recap
    The NIPN Niger team conducted a data mapping study between November 2017 and May 2018, during the start-up phase of the project, using an external consultant. The aim of the study was to carry out “an inventory and analysis of the information and data systems for nutrition in Niger”. The study was carried out by the INS (National Statistical Office), under the strategic direction of the High Commissioner to the 3N “Nigeriens Nourish Nigeriens” initiative. This interview highlights:

    1. How the Niger NIPN team has used the results to develop a capacity building plan for sectors
    2. How the Niger NIPN team dealt with the lack of official definition of key indicators for nutrition
    3. Methodological recommendations for collecting the information

    How did the study go? What were the main challenges encountered?
    The structures to investigate were identified based on the institutions named as “responsible” or “collaborative” in the eight commitments of the National Policy for Nutrition Security (PNSN).
    Questionnaires were given to the sectors, resulting in a significant loss of time. The best method is to meet with the institutions directly and carry out this mapping work with them. The necessary information was obtained after just two or three visits to each institution and only the institutions at the central level were investigated. This was a limiting factor because certain information, notably on the data quality monitoring mechanism, is available at sub-national level.
    Due to the sheer volume of information collected, there was an enormous amount of simplifying and restructuring work to be done which mobilised the NIPN teams. The ‘sector sheets’ were systematically created, consolidating the various elements.
    This led to considerable delays: instead of being finalised by the end of December 2017, the study was finalised in May 2018 and pushed forward until February 2019.

    How did you use the results of the study?
    Firstly, the study provided objective information about the multi-sectorial data available.
    The results allowed us to draw up an initial list of indicators available in each sector in an Excel file.

    1. The study also produced ‘Sector Sheets’ which outline:
    2. The institutional framework;
    3. The organisational framework;
    4. The collection mechanism;
    5. The mechanism for the validation and quality assurance of data;
    6. The data management mechanism;
    7. The output;
    8. The dissemination and exploitation of information;
    9. The indicators and data available;
    10. The nutrition-sensitive indicators;
    11. The indicators selected for the NIPN.

    Following the data mapping, it became apparent that there was an urgent need to put together a referential database of NIPN indicators. In fact, in Niger there is no existing framework providing an official list of the multi-sectoral indicators for nutrition. We are working to adopt nutrition-sensitive indicators in each sector. These indicators will serve as a basis for the NIPN platform.

    Taking into account the institutions’ capacities, we decided to recruit ‘Sectoral Study Officers’ for four months.

    The Sectoral Support Officers have access to clearly identified indicators and will have to ensure that the values for each indicator are collected (15 October 2018 – 15 February 2019), which is a necessary prerequisite for organising the data and building the ‘Nutrition Info’ module. The work of the Sectoral Support Officers will enable us to:

    1. Consolidate the database of nutrition indicators. As a result, for example, we have determined that in the health sector 27 nutrition indicators were missing in the initial mapping. To date, there are several stages to the consolidation: 1. Verifying the comprehensiveness of the indicators, 2) Sorting the indicators according to their link with or ‘sensitivity’ to nutrition, 3) Completing the 23 fields for each indicator (definition, frequency, method of calculation, etc.) and 4. Validating the set of fields for the indicators selected for the NIPN. Also, 239 indicators have been selected for the 3 sectors currently covered (123 indicators for health, 23 indicators for education, 93 indicators for farming and agriculture);
    2. Analyse the nutritional statistics situation in the beneficiary sectors;
    3. Create and validate data sheets for each indicator;
    4. Collect information and documents;
    5. Update the contact list.

    The ‘Sectoral Study Officers’ also make it possible to strengthen capacities in each sector and build relationships that will facilitate access to multi-sectoral data at a later stage.

  • Difference between communication and visibility

    Visibility of a specific organisation, project or initiative can be created by using their logo and acknowledging their contribution in the different communication events that are being organised.

    For instance during a launch event of NIPN in a country, the logo of the implementing organisations and the donors should be displayed, and if available the logo of the NIPN (either the global NIPN logo or the one which has been created specifically by the country NIPN).

    The NIPN countries which receive funding from the European Union are requested to develop a visibility plan following specific EU guidelines.

    Communication and visibility are not the same. Communication is about specific messages that one wishes to convey, to achieve a well-defined objectives, reaching specific target audiences, using appropriate communication channels and tools. A communication plan is more elaborate, but can include a specific chapter on how it intends to create visibility for the NIPN.

    Creating a specific identity and branding for the NIPN, including the design of a logo and the use of consistent templates/formats for reports and presentations, is part of the visibility plan.

  • How to do a stakeholder analysis?

    It can be very useful to do a stakeholder analysis of the overall nutrition audience to better understand which actor has which attitude towards the project and how influential that person is.

    Someone who is very positive and influential could become a champion for the project in the communication approach, whereas someone who is largely negative about the project’s approach, but also influential needs to be managed carefully and communication with this person should be geared towards changing her/his attitude from negative to neutral.

    If a person with a negative attitude is perceived as having very limited to no influence, less efforts need to be made to communicate with him/her.

    A stakeholder mapping is usually done along 2 dimensions: the level of interest in the topic/project (negative to positive) and the level of influence or power the stakeholder has in the target community (in the case of NIPN, the multisectoral nutrition system).

    Map stakeholders according to level of interest and influence
    JPEG - 40.3 kb