Learning from Evaluation: A meta-analysis of the Dialogue & Dissent Strategic Partnerships with DSH

This working paper explores the central question: “does adaptive programming contribute to better results compared to more traditional approaches to programming?”

It uses as case study five Strategic Partnerships (SPs) under the Dialogue and Dissent (D&D) funding window (2016-2020) of the Ministry of Foreign Affairs of the Netherlands (MFA) for which the Department for Stabilization and Humanitarian Aid (DSH) has been the contact point.

Main conclusions

The implementation of the adaptive approach by the five SPs thus far has been enough to have generated better results, at least in some countries inside the SPs. The list below highlights the pathways through which the adaptive programming contributes to better results.

1. When contextualized and frequently reviewed from a learning perspective, the Theory of Change approach delivered adaptations to intervention strategies that
   1.  got country programs unstuck after contextual events prevented them from delivering outputs and
   2. strengthened the translation between interventions’ outputs and broader social and political behavioral change.
2. A combination of informal interactions with local actors and formal studies allowed the SPs to adapt their approaches well to the contexts.
3. SPs also avoided wasting resources in highly political and volatile spaces by using flexible processes of annual planning and reporting with their country partners.

Nevertheless, none of the SPs has implemented the adaptive programming approach in its entirety, which has likely prevented the adaptive approach from delivering even better results.

1. None of the SPs has rolled out rigorously the Problem Driven Iterative Adaptation (PDIA) or similar processes for systematically exploring problems and testing and recording the results of different interventions.
2. SPs aimed to enable the actions on local partners, but barriers and limitations persist. Dilemmas around power imbalances between North CSOs (N-CSOs) and local partners remained in models ostensibly founded on collaboration. One such example is the N-CSOs’ practice of contracting local partners on rigid annual contracts based on defined outputs.
3. Few of the SPs conducted annual, participatory reviews of their Theory of Changes, preferring instead to review their Theory of Change only at the mid-term and at the end of the program. Few of the SPs had made explicit the entire chain of theories that sustained their work: from the organizational level, to the program level, to the country level. Even fewer SPs implemented experiments that tested empirically different solutions to a problem before rolling out a solution.

Main recommendations

1. Lead N-CSOs in the SPs should conduct an internal review to identify and overcome the remaining administrative constraints to adaptation and to the localization agenda.
2. The SPs should continue implementing and learning on the TOC approach at multiple levels: organizational, programme-wide, and country level.
3. The SPs should continue trying out (and be given the space to try out) experimental approaches to learning.
4. The SPs should recognize and value the complementarity between the formal learning processes mentioned in recommendations 2 and 3 and informal learning taking place at the level of programmatic staff.
5. The SPs should define and co-create partnerships together, explore coordination platforms with the MFA outside the usual annual planning and reporting, and avoid over-ambitious programs that spread funding over too many countries.
6. The SPs should continue to improve the practice of Outcome Harvesting, strengthening the evidence that supports their outcome claims and feeding back better the evidence into practice.
7. DSH should build the capacity of the DSH contact point for coordinating multiple embassies and MFA departments, reporting against multiple results framework effectively, and establishing, where relevant, ad hoc, high level of effort coordination structures for joint planning on DSH priorities so that embassies and central MFA departments can also be involved in the informal learning.
8. DSH should consider how to utilize external learning facilitators in inter-consortia /cross-programmatic learning (external to both the consortia and DSH).
9. DSH and DSO should review the contractual space allowed by its other tender and delivery modalities for implementing the adaptive approach.