II-a: The Science and Art of Evaluation Design
description
There are no hard and fast rules for the design of an evaluation, it is built on innovation and experimentation, but like the design of a beautiful building, it must also have a solid foundation and fulfill its function. Much like how architects begin their work with a blank canvas to tackle a seemingly simple task hiding much complexity, evaluators start their work with a blank page to address seemingly simple questions hiding much complexity. Questions like “What interventions work, for whom, under what circumstances, and how?” motivate most evaluations.
The answers to this question often require developing purposeful design that weave together several methodological approaches, to enhance the evaluation’s potential utility while ensuring rigor. Just like any good architectural design, a good methodological design requires creativity and collaboration, rigor, and pragmatism.
Building on the experience of real-world evaluation examples, the workshop will cover the following topics:
- Principles of methodological quality in evaluation: focus, consistency, reliability, validity
- Scope of the evaluation: program theory, boundaries, criteria, and questions
- Methodological design: using a design matrix for simple and complex evaluations
- Overview of key methodological approaches
- Sampling, Selection and Generalizability
- Practical tips for managing and designing evaluations under different institutional conditions
The workshop will have a strong practical orientation. Examples from evaluations in the field of international development will be used throughout the workshop to illustrate the content. The workshop will use a combination of short lectures, group exercises, and plenary discussions.
objectives
After completing this course, participants will be able to:
- Apply the fundamental principles of evaluation design to their work
- Describe the sequencing and logic of evaluation design
- Know how to set scope boundaries, use design matrices, and select and combine methods to answer evaluation questions
- Recognize the importance of informed sampling and selection, and how to apply analytical generalizability principles
- Identify various ways to gauge the methodological quality and soundness of evaluations
recommended for
The workshop is intended to provide essential knowledge to professionals who conduct or commission complex evaluations. Emerging evaluators will benefit from the workshop’s introduction of foundational principles, frameworks, and guidance. At the same time, the workshop may also serve more experienced evaluators who seek to deepen their understanding of the topic and gain exposure to newer methodologies.
level
The workshop is mid-level. Participants should have some familiarity with the evaluation field (i.e. how to work with theories of change, exposure to various evaluation methods and criteria). But no advanced training in evaluation is required.
Prerequisites
Having attended IPDET core course or any similar training before taking this workshop would be recommended.
