Measurement and Evaluation
An Integrated Monitoring, Learning, and Evaluation Approach for Climate-Resilient SRHR
To our knowledge, there is no existing publicly available guide or toolkit that specifically provides guidance on an monitoring, evaluation, and learning (MEL) approach and indicators for tracking climate impacts and resilience of SRHR.
However, there are many resources and literature available about MEL for climate-health and the climate-gender intersections more broadly. We have distilled some key relevant points and recommendations from existing relevant literature as a starting point for approaching climate-SRHR measure and evaluation.
Assessing an intervention or portfolio’s success in improving SRHR outcomes amid increasing climate stresses requires a modified approach compared to conventional health systems monitoring, evaluation, and learning (MEL). We outline a few ways that you and your grantee partners may need to think differently about your M&E approach.
When thinking about climate-health interactions (including SRHR), there are three main elements to track:
Vulnerability, risk, and exposure for both communities and health systems
Impacts of climate change on community health and on health systems
Adaptation and resilience in communities and health systems, across scales from local to national
Challenge 01
Standard health system M&E frameworks typically do not incorporate information about environmental shifts or other intensifying hazards.
Recommendations
Establish a relevant baseline, including environmental data.
Ensure that the baseline and initial data gathering is, at least, disaggregated by sex, but preferably also by other key areas of social difference, such as age and (dis)ability. The baseline data should include:
Current environmental and climatic data
Current health outcomes: A qualitative or quantitative description of the current distribution and burden of climate-sensitive health outcomes by vulnerable population or region.
Current health system condition: What is the current capacity of the health system to respond to the challenges posed by climate change?
1
Track data about environmental hazards, exposure, and systems resilience– not just health outcomes.
These hazards have serious implications for health system operations and population health, and are essential to track in order to anticipate and effectively respond to climate change’s evolving impacts on health, including SRH. Integrate environmental and climate datasets with health datasets in order to understand interrelated trends over time:
Indicators of exposure (e.g., ambient surface temperature, precipitation) are monitored by national weather and climate services
Some indicators of system resilience, particularly those related to infrastructure, are monitored by sectors such as electrical power or transportation departments, and thus can be integrated into your monitoring system. Infrastructure conditions affect the vulnerability of women and girls and their ability to access services, and they affect the ability of health systems to deliver services (e.g., electricity grid resilience to heat waves; local road infrastructure’s ability to withstand multiple flooding events in one season).
Proactive prioritization using environmental information (e.g., projected changes in temperature and precipitation) can reduce the likelihood of future morbidity and mortality. Relevant indicators to track relate to processes, (e.g. utilization of environmental information) as well as outcomes, (e.g. early warning of a vector-borne disease outbreak). (Ebi 2018)
2
Use both absolute and relative measures when examining climate hazards.
This is key because the frequency and severity of climate hazards is projected to increase – but if we effectively build resilience and adaptive capacity, the relative impact of these hazards should lessen.
For example, while exposure to heatwaves may increase over time, the rate of adverse health outcomes for pregnant people and young children should decrease with effective interventions and increased awareness. If we only track the absolute hazard (i.e., increasing temperature) without incorporating data about behavior change (i.e., an increase in pregnant people utilizing cooling centers during heatwaves), we would have an inaccurate picture of the risk and vulnerability.
3
Challenge 02
Conventional health systems M&E frameworks are typically more outcome-focused than process-focused, and often have pre defined timeframes for impact.
Recommendations
Use indicators to measure outcomes and process.
Monitoring process indicators is a key part of assessing health system resilience. Process indicators are needed to assess the extent to which sufficient human and financial resources are available to support adaptation programs and projects (Ebi 2018).
In addition, conventional health system MEL approaches typically have clear timelines for measurement of the objective success of an investment (e.g., Number of new contraceptive users in 2 years; % increase in antenatal care visits over X time).
The time horizon for climate impacts, however, is not certain, so measuring resilience to those impacts based on observed outcomes alone in a pre-set time period is not sufficient
For example, you might have an investment focused on gender-based violence (GBV) prevention for women and girls in the wake of extreme typhoon events. Scientific evidence indicates with high confidence that the frequency and severity of typhoon events will increase in your focal region, but of course cannot predict when exactly or how frequently they will occur. Your grant investment horizon might be two years, but if a typhoon does not happen within the two year period of the grant, you would not be able to say whether the grant was “successful” based on observed post-typhoon conditions alone.
Process indicators should be used to track implementation of a GBV-prevention preparedness before a typhoon occurs while outcome indicators will track GBV rates before and during the future extreme storm events; Both types of indicators are necessary to help determine how well the investment might protect and then actually protects health during/after an extreme weather event.
In this example case, process indicators might include
Inclusion by national government of GBV considerations in disaster preparedness plan and emergency worker curriculum
District government budgeting for creating gender segregated spaces within the emergency shelters
Production and inclusion of rape crisis kits as part of storm preparedness stockpiles.
1
Implementing effective SRH interventions amid a changing climate inherently means that environmental conditions are changing during the evaluation period, often in ways that cannot be fully predicted.
Challenge 03
Recommendations
Adaptive MEL and implementation is key.
Due to the complexity of climate-environment systems, there is inherent uncertainty in the rate, magnitude, and pattern of climate change impacts for any one location.
Set up your MEL systems with on-ground partners with regular learning review cycles built in so that you can collaboratively reflect and adjust the data collection and monitoring approach. Expect to change your implementation pathway and be flexible about the stringency of your evaluation framework.
1
Qualitative data isn’t an afterthought.
In addition to quantitative indicators, budget time and resources to collect complementary qualitative data, which is important for understanding the possible reasons behind a perplexing trend in the quantitative data and identifying gaps in what’s being measured and identifying adaptive adjustments in intervention approach or MEL approach that as needed due to changing climatic and/or social conditions.
2
Where possible, measure across scales.
Bring together context-specific metrics with more widely used metrics and indicators when possible. This can be done by integrating available national or regional datasets. Measuring across scales helps contextualize the trends that you are observing in one community or region, and gives insight into whether it’s a scalable/generalizable trend or one that is hyperlocal to one location.
3
Considering climate mitigation metrics, while advancing a rights-based approach to SRHR.
Challenge 04
Recommendations
Track and reduce health systems emissions, without equating improving SRHR access directly with emissions reduction.
For effective collaboration between health and climate actors, it is important for health actors to demonstrate some understanding of, and commitment to, quantified progress around greenhouse gas mitigation strategies and metrics pertaining to the health sector. An exclusive focus on climate adaptation may not be sufficient in achieving the mitigation goals that climate sector funding often prioritizes.
However, for the SRHR sector specifically, any quantified linkage between greenhouse gas mitigation and SRHR needs to be handled very carefully, due to the history of eugenics and forced sterilization of impoverished people conducted in the name of “environmental protection.”
To approach this challenge, we recommend 1) focusing on the monitoring and reducing the biggest sources of emissions in the supply chain, such as product manufacturing and transportation, and 2) looking for low-hanging fruit that advances greenhouse gas mitigation and health system resilience simultaneously (e.g., installing solar power micro-grids on rural clinics). Include metrics in your MEL framework to capture the avoided emissions from these changes.
When women have full and voluntary access to contraception, evidence indicates that fertility rates do typically decline (i.e., they have fewer children), which often is a supportive factor for household resilience to the impacts of climate change. Fewer people also can mean fewer lower emissions, depending on per capita consumption rates.
However, we do not recommend quantifying contraceptive uptake in low and middle income countries (LMIC) in terms of avoided greenhouse gas emissions in your MEL or impact frameworks as it implicitly creates a dangerous and unethical incentive structure, and diverts attention away from the biggest historical and contemporary contributors to climate change (i.e., industrialized countries in the Global North and a small handful of fossil fuel companies). Voluntary and informed choice and bodily autonomy should always be at the core of any discussion about expanding contraceptive access and usage.
1
