Aid project selection & implementation

Some great quotes in a new working paper proposing a different approach to M&E by Lant Prichett et al. My eye was particularly caught by these two from the conclusion.

“The reality of the project selection process, inside government organizations and between government organizations, tends to be an adversarial process of choosing among projects, which puts project advocates in the position of making much stronger claims for project benefits than can be supported, and being more specific than they would like to be.”

I’m relatively relaxed about the tendency to make over-ambitious claims of expected project impact since everyone does it, and is thus likely to fairly well factored into how projects are viewed. The problem of over-specificity in design is, I think, a bigger problem since it leads to significant wasted effort during the project proposal stage developing ridiculously over-detailed action plans and budgets. Most donors like to think they are flexible when it comes to plan and budget changes mid-grant, but the simple requirement to obtain approval is a deterrent to project managers and a source of risk: what if they do not approve the changes?

The issue of over-specified designs has other implications for implementation too:

“Organizations like the World Bank perpetually over-emphasize, over-reward, and over-fund ex ante project design over implementation. This is because in the standard model, implementation is just faithful execution of what has already been designed, whereby the thinking is done up front and the implementation is just legwork. However, de facto many successful project designs are discovered when project implementers are given the flexibility to learn, explore and experiment.”

As I wrote before: good strategies need good implementation. If the implication – that big donors like the World Bank already know this basic fact – is correct then it really makes me question the whole competitive grant awarding process that dominates NGO involvement in conservation and development. Donors could save everyone a lot of trouble by awarding grants on much shorter project outlines combined with a good track record of delivery (which needs to be much more robustly assessed). Good NGOs would be strongly incentivised to deliver good outcomes since otherwise they would lose their future funding. An entry level system would still allow new players to prove themselves, and also those fallen stars to re-establish themselves.

I will blog again tomorrow on the core proposal of the paper when I’ve had longer to digest it.

Hat tip: the Blattman

Line by Line

Justine Greening the new UK minister for International Development has apparently ordered a “line by line” review of DFID’s programmes. Setting aside snide remarks about that being what an accountant who knows next to nothing about international development would ask, here are some questions she should ask of each line, and a couple she shouldn’t ask.

What to ask

1. Is this programme compatible with your overall strategy? I.e. decide what it is you want to fund and, just as importantly, what you do not. But be careful of ordering a change in strategy just because you’re the new minister. Too much chopping and changing by bilateral donors can be a real problem.
2. Is it achieving good outcomes? Answering this question needs either hard evidence and/or independent evaluations. Note that not all programmes, e.g. those aimed at governance reform, can produce easily measurable outcomes, but it may be too early in their life to evaluate them properly. Do not confuse outcomes with outputs: shiny new classrooms are not evidence of rising educational standards even if the children in the picture are smiling.
3. Is it delivering value for money? An absolutely critical question. Obviously it is difficult to compare outcomes between different sectors: how many vaccines delivered equals one species saved from extinction? But you can ask yourself is the ‘good’ quantified in (2) worth the money spent on it.
4. Are improper people benefiting? This covers two eventualities. The first is whether the recipient government is using the aid programme to support its own naked political goals. The second case concerns ordinary corruption. This is detected by auditors. If you haven’t set up proper audit arrangements or agreed them with the grantee that is your fault.

What not to ask

5. Are the accounting procedures correct? If this programme is about improving accounting procedures or if a programme to improve accounting procedures was recently implemented with this grantee then this is a reasonable question to ask. In all other cases simply refer to question 3: if value for money is being delivered who cares about the accounting procedures?
6. Is anyone else funding it? Have the courage to believe in your convictions. Matching funding is for pussies!

And finally, if Justine Greening wants to make a difference, I suggest that she looks at all the new projects due to be funded and all the projects whose funding periods just came to an end and are now without. Better the devil you know, than the devil you don’t, as they say, and if you want to avoid charges of wasting tax-payers’ money then not cutting funding to proven projects just because they’ve reached the end of the originally planned grant period would be a really good place to start.

Whither the Western Grey Whales – might sacrificing a sub-species be justified?

This is part five of a seven part series on my views on the philosophy of conservation and the case of the Western Grey Whales off Sakhalin in particular – see Richard Black’s article for an introduction. If you are coming to this blog new, before you read this and other posts in this series please consider reading my earlier one and voting in the poll.

In that poll I offered the option that: “Conservation and development should be balanced, but we’d be asking Russia and the local communities in Siberia to sacrifice a lot of money and this is just a sub-species.”

This statement is essentially asking us to explicitly weigh the value of this sub-species* population. Different people have different values and so will answer differently as to which side of the fence they sit on. Regardless as to which side you sit upon, I think it is important to acknowledge the different perspectives and not to assume that one or other is necessarily morally superior to the other. It may be that in the future such questions are resolved with the same clarity as other moral questions, in much the same way that slavery, once an accepted part of life, is now regarded as barbarous. But for now there is not such a universality of views, and basic pragmatism requires us to recognise that, however strongly held may be our views, we need to be able to talk to and engage with others who do not see things the same way.

Moreover if we are to properly answer this question we need to ground our arguments properly. Exactly how much biodiversity does a sub-species represent? Many other species and sub-species of beetles and microscopic organisms which we do not even know about are going extinct every day with destruction of precious, highly diverse habitats such as tropical rainforests. Just because the whales are mammals does that make them somehow more special?

Of course such arguments are in themselves not a reason to do nothing. If it was going to be easy or cheap to save the whales we might reasonably conclude that we should do it anyway. But it appears that saving these particular whales could be very expensive. Think of all the other sub-species we could potentially save with the money that it would cost to save these whales!

This all might seem a bit cold and calculating in comparison to the moral simplicity of other stances, but the reality is that we live in a resource-constrained world, and conservation organisations are having to determine their priorities in this way all the time. I just hope that when it comes time to weigh my soul such calculations will not feature on the negative side of the ledger.

* I am not sure whether, strictly speaking, the Western Grey Whale is a full sub-species or some other taxonomic category. Depending upon ones values, determination of such might be critical to answering the specific question this blog post poses, but is not relevant to the wider issues I raise.

Small may be beautiful but is easily overlooked

CGD’s Connie Veillette and John Norris have some proposals for rationalising (and trimming) US AID’s budget. Included in some eminently sensible suggestions was the vignette that Belize apparently receives ~$20k each year from US AID, and, in Norris and Veillete’s opinion, this probably does not justify the overheads involved in managing it, and so it would make sense to cut it to focus on bigger things.*

If I were a senior manager at US AID I expect I would be pretty convinced by this argument for rationalisation. Except that I know I’ve also heard from time to time talk amongst BINGO staff to the effect that while a certain project may be one of their smallest / cheapest but it may also be one of their best. Such small projects often have very limited ambitions but may fulfil them exceedingly well. Not trying to be something more than you are can be a real virtue in such circumstances. Thus my message to US AID is that that $20k you send each year to Belize may just be some of the best $20k you spend each year, so if you’re thinking about cutting it in the name of rationalization please ensure there is someone else to take on funding the project before you pull out.

This brings me on to a related point about project sustainability. Some projects nearly but never quite succeed in graduating from donor / BINGO support. This can be frustrating for the donor / BINGO who want to move on, and always intended their project should reach self sufficiency. Indeed this may have been central to the original concept. If the project has not succeeded in its wider aims then it should certainly be canned, but I have also heard the odd suggestion from donor / BINGO staff that patience is running out with more successful projects. I find this attitude rather myopic. Given the failure rate amongst conservation and development projects it seems to me crazy to junk a project just because it still needs a few thousand dollars a year in support and technical advice.

So this is a plea to all those aid planners out there in search of the big win: please do not forget the various little ways you may have already discovered to make a small difference. They may not add up to the end of poverty as we know it, but they all count, and are probably a helluva lot more cost effective than those big daring programmes that too often fail for trying to be just too ambitious.

* I am amazed the US doesn’t fund Belize more, but it doesn’t matter to my argument if this particular detail is wrong.

The man in the street donor

Caveman Tom reckons I was too kind on donors with my recent post on how many NGOs we might need.

To be clear for a moment, normally when I talk about ‘donors’ on this blog I am referring to the big institutions (government aid departments, UN agencies, philanthropic trusts, and the like) who recruit professional staff to help them determine how they should spend their money. But in this case actually I was more talking about the man in the street, Joe Public if you will, who is persuaded to stick his hand in his pocket for some apparently worthy cause, usually based upon some NGO’s publicity. Almost without exception this literature grossly simplifies the actual problems that the NGOs are trying to address to convert it into the only message that arguably matters at that point: by getting out his wallet or chequebook the man in the street is making a difference. Even if it’s a small difference, is the implication, if enough men in the street do it then they’ll make a big difference.

Tom says of these donors’ ignorance:

“No, it is not their fault, but they need to start asking more questions and received better education.”

To which I say: carry on dreaming! There are so many issues out there in the world that we are called upon to adjudicate, that is frankly not possible to come close to a decent understanding on more than just a few. Just think about all those ethical product labels one encounters in an average shopping trip: e.g. fair trade bananas or organic bananas? I’d take the fair trade ones any day, but there’s a whole tonne of issues to unpack in making just that one decision.

If many people vote in elections (if they vote at all!) simply for the party that their parents voted for (with little further analysis) what hope can we have that they will want to read up on effective means to alleviate poverty? All these NGOs marketing departments might be immensely frustrating to field staff (ref any one of J from the ‘Hood’s rants), but the guys working for them aren’t stupid; they know that a simple picture of a starving child works far better than a bucketful of prose on the developmental tensions between settled crop cultivators and semi-nomadic pastoralists.

So yes, while it infuriates me no end that big institutional donors continue to make the same basic mistakes time after time, I am rather more phlegmatic over the man in the street donor. If we want to influence their giving then, it seems to me, then we have to make it as simple as possible to differentiate between the quality of NGO’s work. Sam Gardner’s suggestion that the Sphere standards could form the basis for such a ranking in the humanitarian aid sector is thus particularly intriguing.

If we could truly all agree on such a rigorous method for comparing the achievements of different NGOs (with a cost-benefit ratio factored in), and then widely publicise that as a basis for making funding decisions, then that would be fantastic. But is it really in the interests of BINGOs who already have very successful fundraising campaigns? One can be optimistic, and appeal to the idealism of most NGO staff, and suggest that they should commit their NGOs to this now, but what will happen when the first big NGO gets a significantly sub-par score? Suddenly the excuses about the complexity of development will rain down thicker than ever.

Joe Public is often quite happy to be sold stuff that s/he knows is bad for her/him, e.g. a Big Mac, on the basis that it might taste good and any way all their peers are eating them. So I think one faces an insuperable task in attempting to persuade them that their do-gooding donation might actually do some harm. I would keep up the campaign against SWEDOW – as it has a simple message – but apart from that I wouldn’t get our hopes up too high. Sometimes we just have to accept things as they are. Institutional donors, on the other hand, deserve all the excoriation they get and more.

ps. That I previously suggested that appealing to the voter in achieving the necessary reforms to bilateral donors is another one of the contradictions in which we live.

How many NGOs do we need?

Around this time of year one sees lots of ‘Away On Vacation’ signs up on blogs. Alas the reason for my lack of recent posts is just that I’ve been insanely busy. Plus a temporary internet outage for ~5 days interrupted my blog reading, and I’ve been struggling to catch up ever since. But I couldn’t let this piece from J of the ‘hood just pass by. He says:

“We need fewer NGOs.”

I suspect he is probably right, but I think there is a lot more to it than just a numbers game. I think we need both more NGOs and less NGOs. We need the useless ones to die as quick and as painless a death as possible, and then we need new NGOs to keep the existing ones on their toes. As David Week points out in the comments:

“We tend to forget that Oxfam, WV, STC and MSF all started out as corner stores, a meeting in someone’s living room, a zero-budget, zero-org with not much clue to begin with.”

J works in emergency humanitarian relief where the NGO scene is several orders of magnitude more crowded than that in tropical conservation and development, so my heart goes out to him for sitting in interminable cluster meetings in a vain attempt to coordinate who is doing what. I’m also broadly with him on the need for professionalism in conservation and development. If some random guy knocked on your door and suggested you send your child to a new school he was starting up you’d be mighty suspicious and have a list of questions longer than your arm, so why anyone would think poor people in developing countries should be any different beats me.

But for me it’s the whole attempt at coordination that is partly wrong. Not that coordination is a bad thing – we do it around here, and with a much smaller number of NGOs it actually works reasonably well – but there is a limit to what you can coordinate without strong metrics for determination of success and failure. Communism had lots of faults, but one of them was the fallacy that some bunch of bureaucrat planners in Moscow could effectively and efficiently oversee the economies of whole countries. In contrast, in the West for the most part we contented ourselves with letting the market sort things out. The market works on the simplest metric of all; profit. It didn’t always work perfectly (Betamax was a better technical solution that VHS), but it sure worked out better than the Soviet Union (no consumer video recorders at all).

The reason we have so many NGOs is that rich dupes keep on funding the incompetent ones. That they may then deliver the equivalent of the Soviet Union’s cars that no-one wanted to buy is as impossible for the donors to determine as it was for Moscow’s Politburo apparatchiks.

But it’s hard to blame the donors as most NGOs’ fund-raising literature is all much of a muchness; the same themes keep on cropping up and there is little basis on which the non-expert can use to choose. This works for the useless DIY aiders but also serves to protect the established BINGOs against effective upstarts (DIYers who actually hit on a better solution). It can take a long time to get much recognition in the Aid world; no from-nowhere-to-world-conquering-heroes like Google. Neither, apart from the satisfaction of a job well done, is there much in the way of reward for non-profit social entrepreneurs: BINGOs are not in the business of buying out their competition.

Too often it seems people expect too much from Aid / Development / Conservation, such that reducing their myriad outputs to one or more simple metrics would be almost impossible, ref my recent plea for simplicity in environmental certification. Simple metrics will always be distorting, but the merits of simplicity for comparing two rival service providers are substantial. Hence why I was most tickled by Sam Gardner’s recent vision of development finance in 2021, in which he posited the Sphere standards as the basis for competition amongst humanitarian relief agencies. Sam imagined this as being used by big institutional donors like DFID, but why not by the man in the street too?

How many NGOs do we need? As many as can effectively compete for rigorously apportioned funding! No cartel for the BINGOs and no room for crassly ignorant distributors of SWEDOW.

Numbering off

Bill Easterly and Laura Freschi lament the tendency to “make up” international development statistics. Though done from the best of intentions the result does rather seem to resemble a counting system that goes: one, two, many … pick a number, any number.

Thankfully, I don’t think too much attention is paid to such imaginary numbers at the national and sub-national level since clearly they have only a passing relation to reality. That presents development planners with a new problem: without relevant, up-to-date information how can ‘we’ plan our next round of interventions?

Leaving aside the question as to who are ‘we’ to be planning anything of the sort, the obvious answer is to fill the data vacuum pronto, and in turn this involves collating a whole load of statistics from the provinces. This generally requires a significant effort, and development organisations respond to this challenge in one of two ways.

One option is to ignore everyone else; go out and collect the information ‘we’ need and nothing more. Then, since no-one else publishes their data, we won’t publish ours. (And hence the endless of  cycle of household questionnaires asking almost identical questions of people.)

Alternatively ‘we’ can recognise that many others might be interested in answering similar questions and attempt something rather more systematic. Unfortunately it turns out that everyone has subtly different questions they want answered, and so, as we talk to each new set of wannabe development planners, new questions / dimensions get added to our information gathering initiative. It all takes a long time, but eventually we produce the master plan to undertake the best ever information gathering exercise ever; talk about “best practice” – ‘we’ are going to completely redefine best practice, yeah! Such talk is very exciting to donors, so even though this initiative is going to be very expensive, they readily cough up the cash. The developing country government is also pretty keen because this is a big budget investment that politicians can crow about and from which local officials can earn lots of per diems.

Of course, because ‘we’ are not stupid, someone will have realised quite early on the importance of developing this best ever survey into an on-going monitoring programme for which the initial big push will provide the base line. That big push will train all the local officials in everything they need to do the job, and so after that they’ll be able to continue it under their own steam, and the local politicians will continue to fund it because everyone will acknowledge how important this kind of information is so that we can plan development interventions properly …

Except that in practice that is often precisely what doesn’t happen, because although ‘we’ might think this is absolutely central to efficient aid planning, that is not the priority of our local country partners. For a start while ‘we’ may think in terms of development interventions, they are trying to run a country that has run just fine(-ish) up to now without that data, and there are always other priorities. Moreover a couple of years without updates won’t hurt, right?

At the end of the day I’m not certain which is worse: the original problem or the prescribed solution. But, in my limited experience, it does appear to be recurrent and until donors can learn to be more modest in their aspirations and less fickle with their money, I suspect it is unlikely to go away.

Should the biggest problems get the most money?

Last week Ranil over at Aid Thoughts discussed the new big thing in international aid: fragile states. The powers that be appear to have decided that these are the biggest problem out there in the aid world, and, as such, they deserve to get the biggest slice of the aid pie. I long have noticed a similar approach to prioritisation within conservation: the rarest species and the most threatened habitats get the most money.

Despite the fact that it incentivises everyone to talk up exactly how bad their problem is, on the face of this is in an eminently sensible approach to determine how to divide up a pot of money that is never big enough to solve all the problems in the world.

This, however, pre-supposes two things:

a) that there are solutions to be found to these, the biggest problems, and

b) that these solutions need a lot of money to succeed,

Neither of which are necessarily true. The biggest problems tend to me the most wicked problems, and therefore the least tractable to ordinary problem solving. And we seem to have conveniently forgotten about that old chestnut, absorptive capacity. Whilst I know little about how to ‘cure’ a fragile state (if such a thing is possible), the horror stories coming out of Afghanistan of wholesale corruption of aid flows suggest that the absorptive capacity conundrum hasn’t gone away, and is probably magnified in fragile states.

In both conservation and development I would like to see more money being devoted to solutions, especially proven solutions. That is not to say we should ignore the biggest problems, but if you don’t have a workable solution, then a lighter touch with smaller amounts of supporting funds seems a more sensible approach. Pouring money at a problem is unlikely to achieve very much more than reducing your pot of money for tackling other problems and increasing public and political scepticism of aid due to low success rates.

Great businesses, like great sporting teams, play to their strengths. International aid and conservation too often seems to play to its weaknesses, and that is much to our loss.

Bad news brings in the wonga

We’re all familiar with the unfortunate fact of life that bad news tends to be much more newsworthy than good news, e.g. this recent example.

This is doubly true for NGOs, as Karen Rothmeyer points out. (H/T: Tom Murphy) We depend upon bad news to bring in the money. So, although we do like to trumpet our successes, the mantra is always about how much more there is to do.

Got a choice of statistics to use? Just use the one which portrays matters the worst, and a molehill can be turned into a mountain over night. Have I ever done this? Yes, although I don’t think ever as badly as the case of the inflating population statistics for Kibera slum in Nairobi that Rothmeyer reports. Do I feel guilty? A bit. Would I do it again? Probably, since all our competitors are doing it, we need to do so too if we are going to continue to receive funds for our programmes. This is one time where being principled doesn’t get you very far.

It is a tragedy of the commons of information: we’d all be better off if everyone was scrupulously honest, but at the individual or organisational level the incentives are all in favour of scare-mongering. Africa, and other supposedly ‘impoverished’ places are caught in the middle: impoverished of good news stories about them.

The same is true in conservation: the IUCN Red List catalogues all species that are “facing a high risk of global extinction” (description provided by Google), and yet its lower risk categories, Near Threatened and Least Concern, cover species that are not exactly at a high risk of extinction. Still, I regularly hear people bandying around numbers of ‘red-listed’ species in their particular habitat, and I always wonder how many of them are classified NT or LC.

This bias towards bad news, I think, can be particularly problematic when it drives global prioritising and funding decisions: the result is that money ends up chasing problems rather than solutions.

Expectations management (the donor version)

Expectations management is a big part of any community development work; no project will make everyone rich over night, but just like rich world dieters in search of no-effort weight loss, that’s what many poor people will hope for.

Donors also need their expectations managed. All donors are ambitious for what they can achieve with their money, and when applying for funds we frequently have to over-promise what we can do with the budget. (I’m still sufficiently naïve and green behind the ears that this generates considerable stress when, unsurprisingly, we struggle to meet the ‘stretch’ (!) targets we set ourselves.) Bigger INGOs, perhaps, suffer less from this as donors can feel more confident in the decision to give them money anyway, but they still appear more than capable of dreaming up ridiculously grandiose project concepts.

Most donor money, however, does not get spent through NGOs but through direct engagement with the recipient government. This can give donors something of a challenge as they struggle to come up with enough good ideas on to which to unload their barrel-loads of cash. Given their failure rate, you can be sure that their definition of success is pretty low.

I was reminded of this sorry state of affairs by the recent posts Aid Thoughts hosted on the Malawi agricultural inputs subsidy programme. These provided an interesting counterpoint to the prevailing view of the programme as a big success, and a home-grown one at that, although the degree of donor opposition appears uncertain. Both posts suggested the positive assessments of the programme might have been a bit simplistic and did not necessarily ask all the right questions. I don’t know enough about the detail of the situation to comment in more detail, but the queries posed in the posts all seemed worthy of attention, and the sustainability of such a large budget outlay certainly does concern me.

However, I wonder to a certain extent whether Aid Thoughts’ contributors were asking the right questions? If you’re a donor with a huge pile of wonga to push out the door, then something that soaks up a big chunk of it whilst doing a reasonable amount of good is going to look pretty attractive, especially one that reaches a large number of rural poor, doesn’t appear too leaky or to have many immediate down sides, is relatively simple and, best of all, is championed by the recipient government.

For those jaded donor bureaucrats going for the least wasteful option might seem like a pretty good criterion for choosing what to support. Expectations of achieving good – who needs them?