I came across a very interesting and concise article by Fred Carden from RTI in the December 2013 American Journal of Evaluation, which makes plain and simple some important points—whether you’re working from the perspective of sustainability, systems strengthening, country ownership, or learning and adaptation.
Carden’s introductory quote captures his key point about evaluation succinctly: “it’s not about your project, it’s about my country.”
While I would prefer critiquing the emphasis for “international development aid evaluation” instead of Carden’s “development evaluation” (which I think is always relevant—aid or no aid*), Carden makes three simple but important points:
(1) Evaluation will endure but “[international] development [aid] evaluation” is not a permanent field of practice.
(2) Evaluation needs to look at systems, not projects; and
(3) Evaluation requires local expertise.
I think our work on the Sustainability Framework has certainly taken this look at systems before projects to heart, even if it has been a challenge. We have used local expertise, but whether we have given sufficient leadership to local expertise is more questionable. We’ve recently had another experience with this type of local system perspective:
Ilona recently assisted a Gates Grantee in laying the foundation for an evaluation approach looking at the local and national systems rather than just the project, in this case by developing a theory of change. A theory of change (TOC) is a type of logic model that articulates an expected outcomes pathway, the causal relationships, and the underlying assumptions that relate to the broader social, cultural, political, economic, or institutional environment behind the process of change. TOCs are under-utilized tools that can be very helpful in mapping out, in Carden’s words, the “constellation of activities [and we would argue also, of actors] that create change and betterment” in a system. If conceived with that perspective in mind, a TOC encourages a broader view of change beyond the immediate project that encompasses (and is grounded in) the realities of the context and therefore allows key elements outside the project boundaries to be explored and included in the evaluation of a change process. By clarifying how particular activities are expected to produce particular outputs and outcomes (and the relevant assumptions), the TOC helps in framing the evaluation questions to provide meaningful insights or evidence of program success and identify possible counterfactual explanations, as relevant.
Such approaches and the logic behind them are going to continue to grow in relevance. Carden articulates this logic very concisely and clearly—we recommend you take a look at his short (less than 3 pages!) article. http://intl-aje.sagepub.com/content/early/2013/07/12/1098214013495706.full
[*] I would suggest just one language adjustment to the first point and state that “international development evaluation” is not a permanent field of practice. Development—as in social and human development-- needs to continue here, there, everywhere and at all times; and all stakeholders from government to civil society need to rely on good evaluation to learn and adapt.