Systems Approaches to Development Planning and Evaluation: Fad, Reinvention of the Wheel, or Necessity?

Everywhere you turn these days, there are discussions of ‘systems approaches’, ‘complexity’, ‘holism’ and ‘holistic approaches’.  And that’s not just in global health, that’s in development and sustainable development, that’s in food security and livelihoods, that’s in evaluation.

What is this all about? 

Are people just bored with inputs-outputs, and looking for a different flavor? 

Here’s how I figure we got to where we are, and I’ll attempt a conclusion on the fad vs. necessity debate... [Read more in this light-hearted discussion paper, which also provides a substantial number of interesting references (2,000 words)...]

How the Family Planning Sustainability Checklist Helped Identify FP Challenges in Ethiopia

Last fall, a blog post by Sharon Arscott-Mills introduced the new Family Planning Sustainability Checklist, a tool that helps project designers, implementers, and evaluators ensure long-term sustainability for their community-based family planning program. Now, Adrienne Allison describes World Vision's recent use of this tool in the field in her blog entry titled, "How the Family Planning Sustainability Checklist Helped Identify FP Challenges in Ethiopia".

Several points from Adrienne’s blog are summarized here:

• The Checklist was used as part of a family planning workshop with Ministry of Health and World Vision staff in rural Ethiopia.
• The Checklist guided participants to think through issues and questions they had not considered before.
• The WV team recommends the use of the community mobilization module as being particularly useful to elucidate divergent views between male and female community members. 
• The availability of the checklists in two formats (1) by Health Care Providers and (2) by Program Elements was also found to be useful and provided more flexibility.
• The Checklist assisted the WV team to pinpoint needed FP activities that could be immediately integrated into existing programs. 

View Adrienne's full blog entry on the Knowledge for Health (K4H) blog.

Complexity and Development lecture at CGD

On February 5 I had the pleasure of hearing a lecture given by Owen Barder at the Center for Global Development, entitled “Complexity Theory and Development Policy.”  This blog comprises the notes I took during the lecture.

Overview: The lecture and presentation were a Complex Adaptive System (CAS) Theory  101 for the uninitiated and applies it to development at the high theory level (with hints to policy).  There were many book references throughout, shared below.  The audience was full of SAIS professors as well as the World Bank, former (retired) USAID staff, and some interns at various NGOs. CGD streamed the lecture live because there was so much interest in this topic.  Main point: we need to be able to fail and to document and share the anatomy of each failure, recognizing system-wide effects.

Intro: The first slide was the famous chart depicting GDP growth in South Korea and Ghana over the past 40 years or so.  Why did South Korea rise spectacularly and Ghana flatline? (Spoiler alert: EVERYONE can list a few reasons why this is so…and then he didn't return to this chart to tie it to CAS so…we’re still wondering about that.) He referenced Why Nations Fail by Acemoglu and Robinson who posit that corruption leads to bad policies; to change institutions, you need to change politics first.  Owen believes this book would explain everything if it included just one more chapter…and I’ll make you wait for that just like he did in the lecture.

Toaster project: a guy tries to build a toaster from scratch and succeeds somewhat; this experiment illustrates the benefits of economies of scale and the value of trial and failure.  “Development” has not followed this model.

The past 50 years of development saw: 1) fastest progress; 2) no explanation for difference; 3) questioned the existence of a missing ingredient; and 4) led to thinking that all is endogenous—a function of a system that cannot be affected from outside.

Adapt by Harris emphasizes learning from failure.  Testing through trial and error, adaptation and iteration is a better way to solve problems than trying to engineer a solution from the outset.  And an observant audience member pointed out that in that case, success cannot be separated from failure, to which Owen agreed.  Models and engineering solutions are not appropriate for development.

Industries, not firms, adapt.  Eighty percent of innovation results from firms going bust and new firms starting. Witness the fall of Barnes and Noble to Amazon, which then leads to Congress rethinking interstate commerce.  Firm goes bust and institutions adapt.  Adaptations affect each other – interlocking adaptive-ness yields a complex system.

Complexity overview: Origin of Wealth by Beinhocker describes an economy as a CAS.  Characteristics of a CAS:

1.       Butterfly Effect

2.       Predictable at Scale

3.       Emergent Properties – like thunderstorms; these are non-linear systems with system-wide properties

4.       Tend to complexity

5.       State of perpetual disequilibrium – periods of time look like they are relatively stable then there is seemingly a sudden change because all units are constantly adapting to each other

These are familiar traits in the study of economics.

Development: slight adaptation to Amartya Sen’s description—there is a need for a system that makes it likely for people to live out their life choices.

Discussion: Development is an emergent property of a CAS. What are the properties of the system that bring about this emergent property?

What can we do to accelerate the evolution of systems? To affect the rules that govern relationships of elements in the system?

1.       Resist engineering.  Instead, iterate, adapt and learn through trial and error rather than designing answers

2.       Resist fatalism

3.       Promote innovation and variation – equity promotes innovation, e.g. a social safety net gives people the freedom to innovate (leftist view).  The right side would say, eliminate regulation in order to get out of the way of entrepreneurs.

4.       Embrace creative destruction (Schumpeter’s term) – e.g. the evolution of the music industry

5.       Shape development – what is the fitness function.  Create selection pressures.

6.       Embrace experimentation

7.       Act global – open markets to poor countries, encourage migration, etc.

We want to accelerate adaptation.

 If development is an emergent property of a CAS then development policy should promote adaptation.  Does thinking about it this way help us?

How can we fail safely?  We should expect successes in portfolios, not individual projects (Remember—he’s an economist!).  We should package projects in portfolios (CSHGP, anyone?).  Take the implementation risk out of the public sector and put it in local groups, like “cash on delivery” aid.

Aside: DFID is an interdisciplinary agency and adopts interdisciplinary approaches.

Support of microfinance initiatives has blunted financial environmental pressures in some markets, creating a protected class.  Development should do the opposite—it should strengthen adaptive pressures.  And failure needs to generate feedback and learning.

Big question:  measurement (coming from the Bank, no less). Owen admits this is tricky and requires more thought: In CAS collect less data—focus on the part of the system you are working on.  Editorial note: I don’t agree with the second part.  Less data, of the right kind, is appropriate but to focus on measuring/documenting just the part of the system you’re working on is exactly where we are now.  I think we need to describe systemic effects as best we can.  That kind of documentation is significant for iterations and learning.

Ah, you’re wondering about his reference to the missing chapter in Why Nations Fail – it’s the identification of politics as an endogenous property that co-evolves with everything else.

Let's invite him to a CEDARS happy hour!

Food Security Revisited

We recently posted a short 2-pager on the opportunities and challenges, notably for monitoring of new donor efforts with food security approaches, integrating livelihood as well as nutrition and health. (See Sudhir Wanmali’s Food Security Revisited.)

We asked Sudhir to answer three questions for this blog.

Q- Why is food security becoming fashionable again?

The Global Food Security Initiative was launched at L’Aquila, Italy in 2009 at the G-8 Summit in which the global leaders agreed to reverse the decade long decline in investment in agriculture, and to “do business differently”. Amongst others, this was to be achieved by aligning their efforts with country owned processes and plans, by paying specific attention to immediately tackle the hunger of the most vulnerable, and by initiating medium and long-term sustainable, agricultural, food security, nutrition security, and rural development programs. The United States Government, through USAID, is at the forefront of these global efforts with its Feed the Future Initiative and its Global Health Initiative.

Q- What is the new thinking behind what is happening in food security?

Food security is now being seen by all donor agencies at once as a multi-sectorial, mutli-level, and multi-disciplinary exercise in rural development with a view to enhancing food, nutrition, and livelihood security of vulnerable households in Sub-Saharan Africa, South Asia, and Latin America. It is also being seen as an integral part of the national plans of development with the emphasis of engaging all stakeholders, and strengthening their capacities with a view to sustaining this exercise. Rigorous tracking of its progress, from the beginning, is one of the salient features of this new thinking.

Q- What is the focus of your note?

Having studied issues of agriculture driven rural economic development in Sub-Saharan Africa and South Asia since 1981, having noted the decline of the idea of agriculture as the “engine of growth” in these two most poor regions of the world during 1995 to 2010, and having witnessed the emergence of this idea again on the center stage of the development agenda of the world recently, the note briefly delineates the opportunities and challenges that lie ahead in understanding, analyzing, and monitoring the intricacies of food, nutrition, and livelihood security, and learning from that experience, in order to make this exercise sustainable and transferable, thereby making the vulnerable rural population of these regions more resilient than what it is today.

Publication in Globalization and Health -- Zambia HIV Sustainability Framework Application

A heads up about an interesting publication by a Dublin-Lusaka-London team, which uses the Sustainability Framework to look at CBO work in HIV/AIDS in Zambia.

Apart from stroking my fragile ego when I saw such an interesting use of our model, this paper achieves a number of things:

• substantially, anyone interested in HIV work at community level, will be interested in the case study it presents;
• methodologically, this is a great qualitative and systematic use of the Sustainability Framework. The qualitative emphasis is particularly interesting in that we often get caught in the production of spider diagrams, and the quantitative computations behind them. This is a good illustration of the fact that a framework is here to organize (systematize) our thinking. When numbers help, let's use them and organize them in a way which helps our intelligence. But here's a good illustration that qualitative analysis can be deep and meaningful, and that it is certainly not second-rate to good quantitative data;
• I personally really liked the way in which the authors used the definition of sustainability to structure some of their discussion. They also explicitly present the need for adaptation of the model for this specific study in the methodology.

Finally-- and this may betray my own bias as much as the authors' intent--the study findings are used to develop recommendations, which I see as scenarios (from a complex systems' methodological perspective) for sustaining CBO efforts for HIV in rural areas. [Some of our ICF colleagues in Atlanta have been doing work on this same issue with NACA and the Bank in Nigeria.]

Click here to access the provisional PDF of this paper. You can also find it in our Resources' page, with the authorization of the authors.

Have a great 2013!


Eric 

NEW RESOURCE: Family Planning Sustainability Checklist

The Family Planning Sustainability Checklist is a tool that has evolved over several years and was field tested in multiple countries during the process.  The goal of the Checklist is to provide a tool that enables family planning project designers, implementers and evaluators to think through all of the elements that need to be in place to sustain community-based family planning services over the long term.  The Checklist can be used at multiple points during a project cycle—from the initial design phase through regular staff meetings, annual reviews, to midterm and final evaluations. 

The Checklist does not provide a score but it does provide a method for consistently reviewing all essential program elements to ensure long term sustainability of services—and guides the user through  action planning to address identified weaknesses.  Although it is targeted toward NGO programs working with a national district management team, the “essential program elements” are common to most, if not all, community-based programs.  The essential program elements are: 1) Reliable supply of contraceptive methods, 2) Training, 3) Maintaining a network of quality service providers, 4) Supervision, 5) Demand creation, 6) Reporting and integration of data.  

The guide includes two versions of the Checklist which contain the same elements formatted in two ways to improve the usefulness of the tool.  It also includes guidance for 1) a facilitated one-day workshop for use with project partners and 2) a two hour meeting for when shorter timeframes are available.

We would love to hear feedback from those who use the tool.  Please post a blog or write us directly at sharon.arscott-mills@icfi.com  

In addition to being available on the CEDARS Resources page, the Checklist can be downloaded at the following locations:

1) Flex Fund website

2) K4Health

Best,

Sharon

CEDARS

Fellow in International Health

ICF International

Peak Oil and Global Health – Eric's reflection on an interesting paper by Winch and Stepnitz

There’s a lot of talk about systems and boundaries in our recent discussions about sustainability (mostly in health, but generally in social development).

We always emphasize that, no matter how compact, the social systems we work with in developing countries*are connected to a larger environment (“open systems” in Systems Thinking parlance).

Here’s a piece of thinking and analysis which looks at the rather Big System of Global Health work and the potential effects and scenarios following peak oil. Beyond Jesuitical hair splitting about “what do we really mean by sustainability?”, one part of sustainability thinking is to help actors consider the options in front of them to preserve progress for the future.

Peter Winch and Rebecca Stepnitz’s paper Peak Oil and Health in Low- and Middle-Income Countries: Impacts and Potential Responses offers an interesting analysis. As an interesting focus, they take the example of delays in receiving care for maternal emergencies (the three delays) and look at pathways for effects of peak oil and factors affecting utilization and outcome. The analysis is grounded in the contextual reality of poorer countries and looks at the links of energy and economy to people’s mobility, food security, and trade-offs with health (which we have documented for example for the 2008 food price crisis in the Middle East for example).

Winch and Stepnitz conclude with recommendations at a global level. I was particularly tickled by the fact that their conclusion takes them back to Alma-Ata, not out of nostalgia, but maybe because sound principles for local development and governance for now are also the sounder principles for preserving tomorrow’s options.

An interesting read  (access article on CEDARS website or through this link).

Eric

* I more and more think that this “developing country” distinction is making less and less sense. What we’re talking about are poor communities lacking essential resources—a situation which our “developed nations” are seeing more and more of as [pick your position] (1) the economy is slowing down, (2) safety nets are shrinking, or (3) all of the above. At the same time, the wealthy are getting wealthier, even in “developing countries.”