Developmental Evaluation as a

method for systemic change.

Where does it come from?

In this case, Developmental Evaluation is the solution. This method takes on an active role and focuses on repeating trends in the areas of hypothesis, the sharing of knowledge, communication and project management.

Where are the complexities within social innovation?

The complicated aspect about social intervention and innovation is that solutions are not immediately known, and leading group members may have conflicting views about how to move forward.

How does one manage and evaluate a process of innovative change?

In traditional evaluation the focal points are designed for projects that follow linear trajectories in constant environments aimed at finding solutions to defined problems using age-old methods.

In this case, the results are evaluated after solutions are implemented, focusing on questions such as: did we do what we said we would do? And has this generated the impact we thought it would generate?

 

But how to we think about this complexity? How do we consider the value of a social innovation program that is not liner, or one that addresses a multifaceted problem? We are operating in a context characterized by the unknown and in which solutions emerge over time rather than being proposed straight away from the outset.

 

Developmental Evaluation offers an approach that supports innovation processes using feedback loops to help capture emerging impacts.

Its core value lies in the fact that it uses an active evaluation process and allows questions to be asked which guide leaders in decision making roles.

Characteristics of Developmental Evaluation

Developmental Evaluation goes beyond accountability and focuses on:

• Creating hypotheses and asking questions like, is what we are doing helping us to achieve our goals?

• Education: What is working and what is not? What should be changed? What should we continue doing? Spaces for education and reflection are essential to continually compare and adapt our actions to achieve the desired outcome.

• Communication: What has worked? What impact are we having? Internal communication keeps teams functioning smoothly, and external communication is key for involving groups in broader movements towards systemic change.

• Management: What hypotheses do we have about initiating a transformation? What methods will we put to use? How can we compare the initiatives currently being used? This final element deals with analyzing and tracking how systemic change is facilitated by progressive evaluation.

 

Traditional versus Developmental Evaluation

Reflection of Traditional Evaluation versus Developmental Evaluation by Michael Quinn Patton:

Traditional Evaluation	Developmental Evaluation
Objectives: Supporting incremental improvement and assessment.	Support the process of innovation and adaptation into dynamic environments.
Roles and responsibilities: Evaluation takes place outside of the program to ensure independent and objective analysis	Functioning as an internal group as part of the project to implement and test new solutions in real time.
Data gathering: focused on explicit and predetermined criteria	Focused on program values and committed to long-term impact.
Results: Formal reports and best practice case management.	Real-time feedback centered on the education process in addition to reporting.
Complexity: The evaluator’s objective is to control the evaluation process.	The evaluator aims for immediate responsiveness without having full control over the process
Fundamental criteria: rigor, independence, credibility with external players, and critical analysis	Adaptability, thinking through complex systems, ambiguity, openness and flexibility, group work

        (PATTON, 2006) 

Learning with Developmental Evaluation. Changing our perception of evaluation

In traditional evaluation approaches, the evaluator is called in at the end of the project, the project’s impact is analyzed and the results are presented. In fact, this evaluation cycle hinders learning, as the information arrives late to support decision-making on key elements such as strategy and vision.

The use of Developmental Evaluation required a specific mindset and an understanding about what evaluation brings to innovation initiatives.

 

Developmental Evaluation is ongoing. It starts at the beginning of the Project and continues throughout. The aim is to situate evaluation within the complexity of social change in order to support decision making even in the early stages of an intervention.

 

Nevertheless, this approach comes with its own challenges: how to consider the added value of a process when the next steps are unknown?

 

The introduction of Developmental Evaluation as an approach implies a change of mindset in relation to the way evaluation is normally perceived. We often hear: "We don’t yet have results to share, and we feel it is too early to evaluate."

Therefore, the explanation of the different elements that make up a progressive evaluation and the resulting learning have had to be explained throughout the process.

 

Developmental assessment assumes that one learns by doing, or that one understands the value that Developmental Evaluation brings through exploration, experience and experimentation. In this process we see new questions emerge.

 

In terms of data collection and its exploitation, Developmental Evaluation can work with data that have been collected through other processes while establishing specific data collection mechanisms to support decision making.

 

Different roles within Developmental Evaluation

Developmental Evaluation does not separate roles in the same way as traditional assessment techniques.

The work of evaluation is a team effort in which each member plays a role.

The evaluation team combines the internal knowledge of the organization, the ability to incorporate knowledge from outside, and the team asks questions to inspire reflection.

 

Therefore, in systemic change initiatives, the roles of the designers and the evaluators are interconnected. We strike this balance to foster and maintain a healthy innovation relationship. Too much change and too much uncertainty, and the team will be lost; too little change, you are repeating what has been done before.

 

Systematize and integrate Developmental Evaluation

Systematize and integrate Developmental Evaluation in to the workflow with:

1. LISTENING

Through structured interviews and observations, the members involved collect information from all levels. This information is integrated into a matrix.

 

2. ANALYSIS

The matrices are integrated into the evaluation tool, such as Sensemaker, and the analysis is performed for each of the levels.

 

3. COLLECTIVE INTERPRETATION

In order to contrast and enrich the information gathered and analyzed, group sessions are organized involving many actors who are directly or indirectly involved.

These sessions are important for going beyond the analysis, and identifying and ensuring the implementation of actions that mark a route towards the desired impact. In these sessions it will be important to identify and reflect on the following:

 

What are the opportunities that could be acted upon, considering the barriers hindering our impact?

What connections could be generated?

What actions that we are taking are not necessary?

What actions that we are not taking that should be taken?

 

4. ITERATE AND ANALYZE ACTIONS

Based on the reflections of the collective interpretation sessions, iterate the analysis of the materials developed and the actions of the actors involved.

 

In a Developmental Evaluation process, opportunities are brainstormed to generate spaces for co-creation and to explore solutions with participants. Based on what was collected during the collective interpretation, it is possible to begin to glimpse opportunities will arise in the future.