Not for the first time, and why we might foul it up again

Owen Barder is keeping a nice list of major public figures’ claims that “For the first time ever, we have a real opportunity to end extreme poverty within a generation.” in the words of World Bank president, Jim Kim, the latest to so pronounce. The list goes right back to Woodrow Wilson addressing the League of Nations in 1919. Obviously previous generations have not lived up to such lofty aspirations. Why not? And why should we be any different? I present some wild speculation …

Back in 1919, perhaps for the first time, the western powers could say they truly knew most of the world. The major features had been well mapped, and many distant peoples had been ‘civilized’ (aka colonised by racist imperialists). The sun never set on the British Empire and the industrial revolution had made some Americans fabulously wealthy. There were a lot of poor people in the world, but not so many, and Westerners had a surfeit of confidence as to their capacity to achieve great things. Moreover, in a world before the widespread existence of welfare states, it is possible they were not aiming that high.

What happened? Two major changes. Developing countries won their independence from the colonial yoke. This hugely increased their welfare in one important dimension (political freedom), but possibly impeded progress on technocratic goals such as raising average incomes due, in part, to the need to first concentrate on building the capacity of those new states. One signal success, nonetheless, does stand out: the drastic decline in infant and maternal mortality rates. So while economic development was stalling in many countries, populations were exploding. Suddenly it became a lot harder to eliminate poverty.

Fast forward to 2013 and we have passed an important inflexion point: now the number of desperately poor people in the world is declining in absolute terms, and not just in China. The zero goals some people are suggesting should follow the Millennium Development Goals when they expire in 2015 appear tantalisingly in reach.

So why might we fail again? What new issue might once again expose our hubris? I give you two words: climate change.

Aid project selection & implementation

Some great quotes in a new working paper proposing a different approach to M&E by Lant Prichett et al. My eye was particularly caught by these two from the conclusion.

“The reality of the project selection process, inside government organizations and between government organizations, tends to be an adversarial process of choosing among projects, which puts project advocates in the position of making much stronger claims for project benefits than can be supported, and being more specific than they would like to be.”

I’m relatively relaxed about the tendency to make over-ambitious claims of expected project impact since everyone does it, and is thus likely to fairly well factored into how projects are viewed. The problem of over-specificity in design is, I think, a bigger problem since it leads to significant wasted effort during the project proposal stage developing ridiculously over-detailed action plans and budgets. Most donors like to think they are flexible when it comes to plan and budget changes mid-grant, but the simple requirement to obtain approval is a deterrent to project managers and a source of risk: what if they do not approve the changes?

The issue of over-specified designs has other implications for implementation too:

“Organizations like the World Bank perpetually over-emphasize, over-reward, and over-fund ex ante project design over implementation. This is because in the standard model, implementation is just faithful execution of what has already been designed, whereby the thinking is done up front and the implementation is just legwork. However, de facto many successful project designs are discovered when project implementers are given the flexibility to learn, explore and experiment.”

As I wrote before: good strategies need good implementation. If the implication – that big donors like the World Bank already know this basic fact – is correct then it really makes me question the whole competitive grant awarding process that dominates NGO involvement in conservation and development. Donors could save everyone a lot of trouble by awarding grants on much shorter project outlines combined with a good track record of delivery (which needs to be much more robustly assessed). Good NGOs would be strongly incentivised to deliver good outcomes since otherwise they would lose their future funding. An entry level system would still allow new players to prove themselves, and also those fallen stars to re-establish themselves.

I will blog again tomorrow on the core proposal of the paper when I’ve had longer to digest it.

Hat tip: the Blattman

The importance of choosing the right tool for the job

Duh! I mean obvious or what? Except, as is well known, people equipped with only hammers often see too many problems as solvable purely by use of a hammer. International donor engagement with Community-Based Natural Resource Management (CBNRM) has long struck me a case in point.

The international donors come armed with their hammer, which is money to support the government in the recipient country. This is primarily what big international donors do. They work often through diplomatic or pseudo-diplomatic channels. Aid should be channelled to the host government; anything else could be construed as meddling in someone else’s country. This is a pretty good hammer for things that the host country government is already good at, e.g. building new schools all over the country: the more money you pour in, the more new schools get built.

CBNRM has been just one of the sexiest concepts in conservation for about 30 years now. Even Payments for Ecosystem Services (PES), including REDD, hasn’t fully displaced it, just been added to the mix, since rich folk much prefer to pay those poster children of poverty porn – poor rural communities of farmers – for protecting their local environment than a bunch of shapeless bureaucrats. So plenty of donor investment in conservation in recent decades has focused on CBNRM. Except that, as I pointed out previously, the same government officials who are charged with enforcing local environmental laws do not really make the most sensible agents of change when it comes to facilitating the development of CBNRM projects.

Now along comes a World Bank report critiquing participatory approaches to development. It includes a substantial section on CBNRM, which is certainly pleasing to see. The main conclusion is that the majority of CBNRM projects do not appear very successful; depending on how they interact with existing policies and laws they may even be counter-productive, helping the rich at the expense of the poor. Now I am familiar to some extent with some of the CBNRM projects they are talking about (or citing others talking about), and I noticed one thing in common: they were all classic international donor funded efforts, working through local government.

So yeah, I’m not particularly surprised by Messrs Mansuri and Rao’s findings, but I’m not convinced that it greatly undermines the theory of CBNRM so much as the practice in the context of international development aid. (Which one presumes was very much Mansuri and Rao’s starting point, given their employer, and is supported by their associated blog post. All of which is not to say the report is not worth a read.) Major donors, of course, employ some very bright people, some of whom have come to realise the same thing. Alas, while they would choose to fund us – a small NGO – if they could, the only thing their employer ever gives them is a metaphorical hammer.

Public v private, principles v practice

Over at Duncan Green’s Poverty to Power blog there’s a bit of a ding-dong (parts 1, 2 and 3) between Justin Sandefur and Kevin Watkins over whether donor money for education in developing countries should be invested in state or private schools. Both have data and research findings to support their positions, and both make the point that a bit of both should have a place in any sensible education donor’s portfolio. So far so good, so why the all the fuss and strident opinions? One doesn’t have to be a psychoanalyst to suspect that attitudes to private education in our own countries – a fraught subject if ever there was one – are spilling over into the international development sphere.

If that is the case then I should at least declare my own loyalties; I was lucky enough to go to a private school, and very possibly would not be working in conservation and development today if I had not had that chance. But except for the economists and the religiously inspired, development is a decidedly left-of-centre vocation, and so the balance of commenters and voters in Duncan’s poll appear to be against donor funding for private schools.

My only experience of education systems is as a consumer and employer of other consumers, but you do not need any special insights to agree with the primary assertion made by both that there is a big problem of education quality in many developing countries, and a significant issue of access in at least some countries, e.g Pakistan. The essential quandary is how to practically improve the quality of education without compromising on principles of access and fairness. Private schools by definition cost people money and so are likely to be less accessible to poorer people.

But there are lots of essential public goods for which we are accustomed to paying for. Many, e.g. power, water and telecoms, are tightly regulated as a result, and rightly so. They do not, however, come for free. Yet access to information on market prices via a mobile telephone might potentially be more useful and important to a poor farmer’s children than a questionable education would be. Food is an even more fundamental requirement for life than education, and yet just about everybody has to pay for what they eat.

So just because education is a vitally important public good, I do not buy the argument that it needs to be provided directly by the state. As with, say, generating electricity, you may find that private operators are able to do so both better and more efficiently. On the other hand, given the situation in most countries that the state is the dominant provider of education, and that this generally embodies principles of universal access (at least in theory), I am quite open to the argument that funds for education might best be spent improving that system …

… if such funding can actually lead to significant and cost-effective improvements. Lots of newly-built but empty classrooms (paid for by donors) coupled with miserable educational achievement in rural areas around here even when there are teachers present suggests there can be a flaw in this assumption. So if better results can be achieved by supporting low cost private schools then I am all for them. The same ‘if’, of course, applies, and outside of American tea partiers I imagine there are few people who would support private schooling as a matter of principle; the evidence base if their favour needs to be there. (In the absence of sufficient evidence either way funding some pilot projects would be reasonable.)

Ultimately, in my view, it is better to educate some people well, rather than everyone badly. For all the problems with elites and elitism in developing countries, without an elite you do not have anyone to run your country. (Something, it seems, South Sudan is struggling with.) The principle of universal access only becomes a concern when the thing being accessed is actually worth something, or at least when we are able to fix the problems so that it is worth something. As stated many times before donor funds are sometimes quite good at delivering concrete outputs (more classrooms) but much less frequently successful at delivering less tangible outcomes (better educational achievement).

Is opposition within the development sector to private schooling based upon prejudices founded in developed country politics or actual realities in developing countries? I have no expertise to say one way or the other, but I do find similar biases in tropical conservation, in which many so-called experts insist on the gold standard of protected areas despite plentiful evidence that they often do not work very well on the ground. (This is not the same as saying they do not work at all.) As with education, rich country principles seem to occupy the moral high ground over practical solutions that might actually suit poor countries better. That is a shame for both the people affected and the wildlife that might otherwise be conserved.

Whither the Western Grey Whales – might sacrificing a sub-species be justified?

This is part five of a seven part series on my views on the philosophy of conservation and the case of the Western Grey Whales off Sakhalin in particular – see Richard Black’s article for an introduction. If you are coming to this blog new, before you read this and other posts in this series please consider reading my earlier one and voting in the poll.

In that poll I offered the option that: “Conservation and development should be balanced, but we’d be asking Russia and the local communities in Siberia to sacrifice a lot of money and this is just a sub-species.”

This statement is essentially asking us to explicitly weigh the value of this sub-species* population. Different people have different values and so will answer differently as to which side of the fence they sit on. Regardless as to which side you sit upon, I think it is important to acknowledge the different perspectives and not to assume that one or other is necessarily morally superior to the other. It may be that in the future such questions are resolved with the same clarity as other moral questions, in much the same way that slavery, once an accepted part of life, is now regarded as barbarous. But for now there is not such a universality of views, and basic pragmatism requires us to recognise that, however strongly held may be our views, we need to be able to talk to and engage with others who do not see things the same way.

Moreover if we are to properly answer this question we need to ground our arguments properly. Exactly how much biodiversity does a sub-species represent? Many other species and sub-species of beetles and microscopic organisms which we do not even know about are going extinct every day with destruction of precious, highly diverse habitats such as tropical rainforests. Just because the whales are mammals does that make them somehow more special?

Of course such arguments are in themselves not a reason to do nothing. If it was going to be easy or cheap to save the whales we might reasonably conclude that we should do it anyway. But it appears that saving these particular whales could be very expensive. Think of all the other sub-species we could potentially save with the money that it would cost to save these whales!

This all might seem a bit cold and calculating in comparison to the moral simplicity of other stances, but the reality is that we live in a resource-constrained world, and conservation organisations are having to determine their priorities in this way all the time. I just hope that when it comes time to weigh my soul such calculations will not feature on the negative side of the ledger.

* I am not sure whether, strictly speaking, the Western Grey Whale is a full sub-species or some other taxonomic category. Depending upon ones values, determination of such might be critical to answering the specific question this blog post poses, but is not relevant to the wider issues I raise.

Dimensions of Sustainability

One of the big disappointments of Rio+20 was the evisceration of the Sustainable Development Goals initiative which now looks like it is going nowhere. I hope that some of the ideas underlying that can find some other outlet; I particularly liked Kate Raworth’s notion of the sustainable development doughnut, in which economic activity is constrained by social minima and environmental maxima.

But, as with my previous post, there is no reason to wait for international politics to sort itself out; we can get implementing these ideas in our own efforts right now. When it comes to designing conservation projects I like to use another visual metaphor, what I call the dimensions of sustainability.

Typically a conservation project will start with a problem statement along the lines of habitat A or species B is severely threatened and something must be done. This may be expressed as a target to prevent the area of habitat shrinking below a given carrying capacity or a population declining below the Minimum Viable Population. Thus a red line is drawn. Everything else must fit around that line, and the further away from the line the better, so not only do we start with a massive constraint, but the whole project design is oriented towards pushing the target variable as far as possible from that minimum.

The problem is that this monocular vision of sustainability greatly constrains the range of solutions which might be considered, and can also lead to blinkered project management with negligible attention paid to other variables. Instead I like to start with a general consideration of the ‘sustainability space’. This space has three primary axes of environmental, social and economic sustainability (ref the three chambers of FSC). Each axis, however, may be a composite of various measures (or sub-axes, if you like), e.g. the environmental axis may list habitat protection, biodiversity and carbon as important issues, the social axis may consider issues of equity and cultural propriety, and the economic axis returns on investment and ability to meet the needs of the market (as opposed to the project logframe).

For a project to be truly sustainable we need to keep all of these variables within sustainable bounds. Excessively prioritising one or two over the others will rarely be constructive. Moreover consideration of this wider picture may help one to understand how a little shift in that initial red line might in fact make the whole difference between project feasibility and miserable failure.

Moving the red line can create space to find a workable solution

For the hard-core conservationists out there it is important to note that this is not about compromising on important principles; if a habitat fragments too much it ceases to function as God intended. But many of those red lines we like to draw are based upon questionable data, and may be somewhat precautionary. Neither point invalidates the need for a line, but they do suggest a certain amount of flexibility. This is important when considering the range of practical interventions. It might be that without such flexibility no project is likely to succeed. (Alas such unsolvable equations are too often not sufficient to stop investment in the project.)

Instead my approach of considering the various dimensions of sustainability is intended to define the problem space properly. It is only within that space that we are going to find any solutions.

Disclaimer: I don’t claim any great originality for the insights above so I would be interested to hear if anyone has similar or alternative frameworks. Equally please do let me know if you ever find the above useful in designing a project yourself.

Accounting for failure

Project failure is far too common in conservation and development for anyone’s comfort. Many agencies and practitioners regrettably seek to hide their poor records behind euphemism and by redefining success radically downwards after that fact. Last year an aid bloggers forum considered the question of admitting failure although the consensus was not very positive (see my two contributions: here and here).

Yesterday I blogged about an alternative solution: pushing aid projects to obtain insurance against failure. It’s a nice idea, but probably quite a few years away at best from wide-scale implementation. But, it occurred to me, there is an intermediate solution which would take very little change to implement. Put simply donors would account rigorously for their projects’ success rate. It would work as follows.

Already most donors demand clear statements of project aims and expected outcomes before committing funds. Good donors will also ask for a risk assessment. All we need to do is quantify those risks. Sure putting a number on the likelihood that you will get the necessary buy in from local government officials is an exercise in extreme subjectivity, but we can live with that. Multiply all your risk percentages together and you should get an indication of the likelihood of project success. (I bet often it will be substantially lower than the project’s proponents would like, but if they try to massage it up they’ll get caught out later …)

Then, come evaluation time, the reviewers should explicitly assess what proportion of the original aims and outcomes have been attained, and what risk factors in actual fact came into play to the detriment of project impact. This assessment would have to be extremely robust and refer only to the original estimates of impact, so as not to allow project managers to redefine success downwards mid-project. (A subsidiary assessment could consider revised aims that were formally set out and agreed.)

Project proponents and implementers who consistently underestimate risks will be shown up (although, obviously the feedback loop will take a few years to generate much in the way of information). Imagine if donors pooled all this information so that they could look up organisations records. Imagine further if such estimates were linked to specific key people who worked upon the projects, and you had to justify both your risk assessment accuracy rate and actual project success rate in your next job interview. (Sensible employers would be tolerant of those who have only worked on a few projects and just got unlucky, but could talk intelligently about what went wrong, the lessons they learned and how they would put right such situations in future.)

As well as improving accountability and honesty about the very real risks involved in most conservation and developing projects, as the data built up, donors could also use it for their own internal evaluations. How successful were their projects? Which types of risk factors proved to be the most dangerous? Which types of risk factor were consistently under-estimated and which might have been over-estimated? Donors wanting specifically to target riskier projects with some or all of their money should not be discouraged; we all know that the pay-off from such initiatives can be that much bigger than me-too carbon copies of established models. This way the risk would simply be more explicitly acknowledged.

The total guesswork inherent in the original risk estimation would limit the data’s utility in evaluating individual performance, but for donors and BINGOs, when aggregated across the organisation, these errors would start to average out, and analysis of later results would help agencies to refine their estimates. E.g. typical political risk factors could be classified according to severity, with information for project proponents on actual failure rates in previous projects to help them gauge the likely risk in the new project they are proposing.

I can see plenty of resistance from many in the aid industry to such crude quantifications, but the move to increase transparency in the sector is gathering momentum. Perhaps it is the sort of thing that could be considered in the next iteration of IATI?

Insuring against failure

With Rio+20 about to open I have to give a shout out to the idea of iREDD advanced by Corey Bradshaw et al. They propose to address some of the major challenges of REDD by requiring sellers of REDD credits to purchase insurance (that oh-so-cool i on the front of iREDD). One very good point in its favour is that this should massively increase the buyers’ confidence, although, it should be noted that voluntary carbon market standards such as VCS already require project developers to retain substantial buffers of carbon offsets for just such an eventuality. However, I think financial insurance is a stronger option, since one catastrophe – e.g. a massive forest fire – could wipe out a project’s entire carbon achievements with no recourse. Insurance provided by a highly capitalised third party delivers much better cover for such events.

Thus insurance appears a very good way of dealing with the problem of permanence. It should also be fairly easy; there are pretty good data available on wild fires, whilst forest owners who themselves cleared the forest (or allowed someone else to do so) would be guilty of insurance fraud. (Following the logic of my post yesterday, insurers would probably also have to consider political risk, in which the forest owner suddenly finds they are no longer the forest owner.)

But in tackling the other challenges in REDD in demonstrably delivering real net additional reductions in carbon emissions (see here for my previous observations) iREDD seems mostly just to come down to the idea of insuring against the risk that a proposed action does not result in the desired outcome. Here I see a much greater challenge for insurers. How should they quantify such risks? Every project will be unique in some way or another. I foresee great difficulties in developing standard metrics by which these things can be assessed, and where such guidelines could be determined I fear they would tend to favour cookie-cutter style projects and incentivise against innovation.

On the other hand, if these hurdles could be overcome, this seems to me like an idea that has great potential way beyond the REDD arena. How about if donors required every aid project except for the riskiest (which would therefore obviously stand out) to obtain such insurance? That would force project developers to confront major operational risks in a much more explicit manner than at present. Furthermore it could open up innovative approaches to project funding (donors would expect to get some of their money back when projects fail), and could allow private sector operators taking a greater responsibility for the entire project cycle and assumption of risk. This would be in contrast to the current model for involving the private sector in which big consultancies make fat profits for running flawed projects designed by a donor who should have but didn’t know any better.

Owen Barder recently blogged about attempts to extend the still experimental idea of Social Impact Bonds into the international development space*. These also invite private sector players to assume some of the risk of delivery. Insuring projects against failure could be another option to add into the mix.

In practice I can see that it is going to take quite a while for these various different instruments to be put into practice, during which time we can hope – indeed reasonably expect! – that significant numbers of poor people and poor countries will have developed to the point where they are rather less poor and less in need of development assistance. But that could also work in these ideas’ favour: the least developed countries have the least capacity to engage in these kind of risk-sharing models involving the private sector.

So all in all I think it is great to see innovative thinking around conservation and development finance, and I hope that at least some can come to serious fruition. And I look forward to one day filling out an insurance registration for a project I have helped design!

* The various comments echo my points above about some of the practical challenges that would need to be overcome to introduce these as financial instruments worthy of the name.

Oxymoron of the day

Despite years of designing and implementing incentive-based strategies for preventing deforestation in the tropics, the factors that hinder and/or promote effectiveness of incentives in changing people’s behaviours for sustainable forest management are yet to be systematically understood.

That is from a research proposal recently sent to me. It was shared in confidence so the authors will have to remain anonymous. That shall spare their blushes. For much tropical deforestation is driven primarily by local issues and economics so even were the instigators’ behaviour to be systematically understood, such an ‘understanding’ would be of little practical relevance to any given situation. The authors appear to grasp this since they were proposing to conduct their research in just one part of one country. One can hope the findings from their work may usefully inform local efforts to reduce deforestation in that area.

Social entrepreneurs and M&E

Bill Easterly apparently wants to see faster determination of successful and failing aid projects than is provided by traditional monitoring and evaluation. That is according to Tom Murphy’s report on the DRI annual conference from back in March.* Tom commented:

“… strong and open monitoring and evaluation practices can ensure to the trial and error of the Easterly ‘searcher.’” (sic)

I also seem to recall – but cannot now find the right quote or link – the suggestion that aid practitioners could seek inspiration from how success and failure business is determined in the business world. (The DRI debates considered whether an RCT would be a suitable evaluation mechanism for the IPhone game Angry Birds – not the best example imho!) (Update 27/06/12: found that link, it was Ian Thorpe blogging here.)

As someone who played a major role in setting up a local NGO and then leading it for several years, I guess I can be classed as something of a social entrepreneur. Thus I have a few thoughts on how this can or cannot work in practice.

Like too many organisations in the development and conservation sectors I wouldn’t put down M&E as one of our strong points. We’re not terrible at it, and we’re getting better, but it wouldn’t be too difficult to poke plenty of holes in what we’ve done so far.

As ever with M&E, getting the budget balance right is tricky. If we had as much money as the MVP we could do some amazing M&E. (Hopefully a lot better than the MVP has achieved in practice!) But we do not have anything like that amount. Plus we have a problem that many of the impact indicators we are targeting take a long time to move in a positive direction (e.g. biodiversity). So we have to wait quite a while just to see whether our monitoring programmes are capable of detecting the kind of change we are seeking to achieve.

These challenges are universal. More particularly, taking an entrepreneurial approach involves a lot of flexibility and adaptation to changing circumstances. Most people seem to agree this kind of approach is a good thing. Which is all fine and dandy, but it does make it hard to set up your M&E baselines, because we’re continually adapting how the project will work, and thus what impacts it will have. As the project matures it settles down better, but for our earliest pilot sites the opportunity to establish a firm M&E baseline has long gone.

So how do we track our own progress? Mostly I would say through a number of milestones, which may or may not be well set out on paper. Conceptually we started with a pretty good idea of what we wanted to achieve. As things progressed we developed a number of internal targets (usually with very flexible timelines if they are specified at all) against which we could measure ourselves.

The analogy with modern business is perhaps something like Facebook. At first it was just an idea – something cool to do – then it became an enterprise, but one not particularly focused on bottom-line impact (profit) as on other metrics and milestones (e.g. market share and time spent on site by users). Only latterly has the focus at Facebook shifted increasingly to monetising their substantial achievements. (If recent share performance is anything to go by, this is proving tricky.)

Similarly our own project is being subjected to increasingly robust M&E assessment, as indeed it should do. But, in my mind, most M&E approaches and entrepreneurial innovation apply to quite different stages of project development. This is also why starting small is so important; it allows time for KISI. Alas too many people in conservation and development are often in such a rush that they want to spend $100 million first, and ask questions later.

* Yes I’m finally back blogging again. I have a number of posts queued up in my head responding to the news over the last couple of months. Hopefully it won’t all seem too much like yesterday’s left-overs.