evaluation-conference.de - Panel Discussion 2

Panel discussion 2: Conclusions

From left to right:
Ruerd Rubens:Director Policy and Operations Evaluation Department (IOB) at the Ministry of Foreign Affairs, the Netherlands
Michaela Zintl:Head of Division for Evaluation of Development Cooperation; Auditing, German Federal Ministry for Economic Cooperation and Development (BMZ)
Matthias Lanzendörfer:Evaluation and Quality Management, Misereor
Martina Vahlhaus:Director Evaluation Department, Deutsche Gesellschaft für Internationale Zusammenarbeit (GIZ)

In the conclusions, the four panelists were asked about the lessons learned during the conference. The following aspects were discussed and summarized.

Systemic approaches are particularly useful for learning and improvement
There was an agreement that systemic approaches can encourage learning and innovation, and enhance adaptive management during program implementation. As they are particularly helpful for looking into the "why" questions, systemic approaches in evaluation support the improvement of development interventions. The panelists did not agree on the usefulness of applying systems thinking when it comes to accountability: While Ruerd Rubens raised doubts, Michael Zintl said that systemic tools can help to "measure results" to a certain degree because systems thinking broadens our view (e.g. to include unintended results).

Systemic approaches are particularly useful in complex settings
During the conference, it was stressed that in every evaluation, appropriate methods for the given situation and evaluation questions should be used. No single approach fits every setting. In this sense, the audience was reminded that systems thinking is a means to an end and not an end in itself. It should only be applied when useful. It was concluded that systemic approaches are particularly useful in complicated and complex situations.

Methods and tools are available
Matthias Lanzendörfer echoed the findings from Session B and C of the conference: A lot of (old) tools are already available. There is not necessarily a need to reinvent tools. Evaluators can "steal" methods and tools from the systems field.

The debate is vivid but heterogeneous, and there are quality issues
According to Martina Vahlhaus, the conference showed that the debate about systemic approaches in evaluation is very heterogeneous. There is no fixed stock of knowledge and no clear guidelines exist as yet. Still, there is a huge interest in applying systems thinking in evaluations. The challenge is to assure the quality of evaluations that (claim to) use systemic approaches ⤓ especially because these approaches are "fashionable" nowadays.
In the German context, the connection and exchange between the evaluation field and the management/organisational development field needs to be strengthened to further develop the application of systems ideas in evaluation.

Our partners need to be involved in the debate
It is crucial to involve our partners more in these methodological debates. Systemic approaches must not be discussed over our partners' heads. In this context, it was regrettable that there were only a few participants from the South.

Pictures & Illustrations

R. Rubens and M. Zintl on the panel
M. Lanzendörfer on the panel
M. Vahlhaus on the panel
M. Zintl on the panel
Panelists of panel discussion 2
Woman from the audience asking a question
Woman from the audience asking a question