Aid project selection & implementation

Some great quotes in a new working paper proposing a different approach to M&E by Lant Prichett et al. My eye was particularly caught by these two from the conclusion.

“The reality of the project selection process, inside government organizations and between government organizations, tends to be an adversarial process of choosing among projects, which puts project advocates in the position of making much stronger claims for project benefits than can be supported, and being more specific than they would like to be.”

I’m relatively relaxed about the tendency to make over-ambitious claims of expected project impact since everyone does it, and is thus likely to fairly well factored into how projects are viewed. The problem of over-specificity in design is, I think, a bigger problem since it leads to significant wasted effort during the project proposal stage developing ridiculously over-detailed action plans and budgets. Most donors like to think they are flexible when it comes to plan and budget changes mid-grant, but the simple requirement to obtain approval is a deterrent to project managers and a source of risk: what if they do not approve the changes?

The issue of over-specified designs has other implications for implementation too:

“Organizations like the World Bank perpetually over-emphasize, over-reward, and over-fund ex ante project design over implementation. This is because in the standard model, implementation is just faithful execution of what has already been designed, whereby the thinking is done up front and the implementation is just legwork. However, de facto many successful project designs are discovered when project implementers are given the flexibility to learn, explore and experiment.”

As I wrote before: good strategies need good implementation. If the implication – that big donors like the World Bank already know this basic fact – is correct then it really makes me question the whole competitive grant awarding process that dominates NGO involvement in conservation and development. Donors could save everyone a lot of trouble by awarding grants on much shorter project outlines combined with a good track record of delivery (which needs to be much more robustly assessed). Good NGOs would be strongly incentivised to deliver good outcomes since otherwise they would lose their future funding. An entry level system would still allow new players to prove themselves, and also those fallen stars to re-establish themselves.

I will blog again tomorrow on the core proposal of the paper when I’ve had longer to digest it.

Hat tip: the Blattman

Specificity killed the cat

The Climate & Development Knowledge Network innovation fund has opened to a second round of applications (with an African focus). In order to apply you need to meet the following criteria:

• Are an applicant group that includes at least two (2) partners; Lead Applicants must be African (and not international institutions based in Africa), with additional partners also being African.
• Are an applicant group which includes government institutions; if not, you must demonstrate significant government buy-in.
• Are proposing activities that support an existing network/community of practice to ensure buy-in and sustainability.
• Have game-changing project ideas/concepts focused on Africa that require development and shaping through an innovation process; provide a forum in which active (cross-sector and cross-regional) engagement, consultation and collaboration can directly take place and catalyse the direction, design and structure of a future project or initiative.
• Are aligned with the evaluation criteria.
• Can set up a sustainable uptake pathway for the results of the innovation process, for which a relevant target audience is clearly articulated.

I have two observations. Firstly, this is jargon overkill à la EU for what I imagine should be a wide-ranging call for proposals accessible to as many people as possible. What is an “innovation process” when it’s at home? Opaque language like this tends to restrict applications to the usual suspects who can navigate the linguistic contortions, the very opposite of what innovation is supposed to be about.

Secondly, these are very specific criteria. Why does it have to be 2+ partners, 100% African? Why demand government buy-in when civil society can be a more powerful agent of change? Why should applicants be working through a network and have to provide a forum for engagement etc? The problem with very specific funding criteria is that you encourage applicants to dream up projects specifically to meet your preconceived notions rather than to meet their own priorities.

This gets a thumbs down from me.

Donors: show us your scorecards

A few months ago Lant Pritchett hectored the World Bank Executive Directors to publish their scorecards on the three candidates for the WB presidency. It was a nice idea which unfortunately didn’t get a lot of traction, presumably because WB presidential selection is essentially a political process, despite all the publicly claimed desire to simply find the best candidate.

But why limit this idea just to the WB president’s selection? A lot of aid funding for NGOs comes through competitive calls for proposals. Some donors, the EU comes to mind, define tediously exact criteria by which proposals will be judged, others are rather woollier. Wooliness doesn’t have to be a bad thing – it allows extra weight to be given to proposals which contain that hard-to-define X factor – but for all the fact that judgements are necessarily hugely subjective – often favouring the known, established applicants – precision has a lot of merits in clarifying ranking of proposals. When the number of proposals is high it very probably becomes the only way of effectively comparing the full range of applications, so I assume most donors practice something along these lines.

So why not publish the results? In the age of openness and transparency, why are donor decisions shrouded in mystery? The EU may tell you the score allotted to your proposal, but do not tell you the threshold that was necessary to obtain funding. (When your proposal has averaged over 80% it does leave you wondering exactly how good it needs to be?)

Donor decision-making needs to be final, there is no question of that. Endless appeals against the system, however it is rigged, won’t do anyone any good. But it would still make the lives of NGO fundraisers easier if we knew both what had and had not been successful in the past, and what were the various scores. (It might even save us from writing the odd wasted application.) So donors, how about a bit more sunlight on your decision making? Go on, show us your scorecards!

Inflexion points

Various thoughts have collided in my head over the last week concerning value for money* in tropical conservation and development. I can see two possible inflexion points, which are clearly related in some sense, but also reflect different concerns linked to the issue of scaling up.

The first I’ll call the So What? point. At the bottom end of development, if someone spends $1,000, say, on donating school books, then most people’s reaction is simply to celebrate the fact that now some kids in a poor country have some books that they would not have had otherwise. It would seem churlish to question whether, in fact, there might be a better way to spend the money.

However, if someone else spends $10m on school books, then one is naturally inclined to question what is the eventual impact of this donation. How much better educated are the kids? How many are lifted out of poverty – or at least become less poor – as a result? They spent $10m on school books, but so what?

The second inflexion I’m calling the Get out of Bed point, and is the minimum grant size likely to interest a development practitioner. It is to a large part set by the organisation for which you work. BINGOs may not chase grants smaller than $1m, while the smallest NGOs will happily jump through several hoops for just a few thousand dollars.

The problem comes when a would-be donor perceives one as having a lower Get out of Bed threshold than one in fact does. One does not want to seem ungrateful, and maybe if there were less hoops  to jump through or less strings attached (hint to donors: these make small grants much more worthwhile), and/or if the grant would be easily renewable (dependable funding is always appreciated!) then we might go for it, but every application form takes up senior staff time when they might be better off just managing what they’ve already got on their plates. This can be a tricky issue for small but growing NGOs to navigate.

I haven’t had the time to really think this through, but I suggest that the best conservation and development action probably takes place when the two inflexion points are roughly aligned for both the implementing agency and its main donors. If the Get out of Bed threshold is much lower than the point at which the So What? test becomes applicable then the organisation concerned is desperately short of money (or fixated on raising money rather than achieving notable results). Conversely if  the Get out of Bed point is significantly higher than the  So What? question level, then the agency is probably swimming in cash and may well be more concerned about spending its great dollops of wonga than achieving value for money.

Contrary thoughts or suggested examples of the different cases are welcomed in the comments section.

* Note this discussion is not about the quality of service provided, efficiency of the activities undertaken, or the level of overheads applied. For the purposes of this post please assume such variables can be controlled for, although there may be an inevitable degree of confounding factors.

My ideal donor

I just received a delightful email from a chap who was setting up a new grants scheme for somewhere in Sub-Saharan Africa. He had read my rant entitled Preposal Prepostery, and wanted to know what advice I would give to a donor wanting to do the right thing. My dream come true: a chance to design the ideal donor!

Here are ten tips I gave him. Please suggest more in the comments, and I’ll pull the best together into a second piece.

1. Offer funding that is presumed to continue (unless there is a bad evaluation or something) rather than presumed to stop (but where you can apply for follow-on funds). Obviously your ability to do this will be constrained by your own funder(s) – you don’t want to make promises you cannot keep – but maybe this is the kind of innovation which might make you more attractive if sold to them in the right way. This doesn’t mean promising to fund projects in perpetuity, but there should be a sensible strategy to achieve long term sustainability (handing over to the local government does not count!) either because need for the external assistance will eventually subside (e.g. farmers will have learned the new techniques being promulgated) or because alternative funding can be found. But these should be long term issues to be reviewed and gradually refined over the length of the project, not something to be dreamt up for the initial proposal.
2. Give 100% funds upfront at the beginning of each year. If you feel the need to incentivise report production, do it through personal perks like invites to lesson-sharing workshops. Never let grantees suffer funding gaps!
3. Keep budgets short. I reckon 10 different budget lines should generally be enough. Don’t demand ridiculous levels of detail in financial reporting, but do demand proper audits. Even ask to be consulted over the auditing process so you can ask auditors to look for specific issues of concern to you. (You might have to pay extra for that.)
4. Don’t get too hung up on large proportions of budgets going on certain things. Maybe the grantee needs that. In particular a high % going on salaries might mean they are employing high calibre staff which is about the best indicator of success I can think of.
5. Consider funding 6 month inception phases as a matter of course. After 3 months the grantee should submit a full project document setting out their plans in a lot more detail. This can be further negotiated, until satisfactory. Funds release beyond the 6 month inception phase is dependent upon a satisfactory agreement over the detailed project document. This can be repeated every year so long as the grantee has at least 3 months funds carry-over so they can keep on working in the meantime. Major asset purchases should be held off until after the inception phase. Inception phases are really useful where you’re not sure about an applicant: maybe they just don’t have the writing skills to prepare a good application, and you can offer support to the applicant to this phase, assessing for yourselves what needs to happen. Conversely don’t demand highly detailed proposals before the inception phase.
6. Require applicants to complete a risk analysis together with how they intend to address these risks. This should be reviewed annually.
7. Focus on impact / outcomes over outputs / immediate objectives. Grantees should clearly communicate what’s important; that way when things go wrong (as they always do) you can hope they focus on the impact and not on which outputs they promised to deliver.
8. Beware of box ticking exercises. A logframe is no substitute for a clear strategy, and if you have a clear strategy the log frame is relatively unimportant. (Tho some people may find it useful in both formulating their strategy and communicating it.) Equally, SMART objectives are all very well but sometimes can get a bit silly, e.g. time-bound should simply be assumed to be by end of project unless otherwise stated. On the other hand they do force the indisciplined to get properly specific.
9. Allow for free-form proposals with some suggested headings, making it clear applicants should feel free to add their own, and exclude any that are not applicable.
10. Be very careful about your call for proposals: restrictive calls simply force people into straitjackets, writing proposals for what you want to fund rather than what they want to do. Even quite general restrictions can have this effect. E.g. I recently review applications for a small conservation grant which required applicants to include a local communications / outreach component. There were some good proposals which suffered from having this bolted on where it didn’t make full sense.

And here’s an eleventh I just thought of:

11. If you must demand certain issues be mainstreamed, then allow applicants to mark them as not applicable without that being a black mark against them.

Proposal Prepostery

We recently got a consultant’s report appraising a project proposal in which we’re a partner. The good news is that the donors are going to fund it, but they want a revised proposal. Apparently some fairly important issues were not clear to the donors which is fair enough. However, appraising consultants clearly believe they’re not earning their (very generous slice of) daily bread if they don’t write a long, involved report. So we’ve got a loooong list of issues to respond to, some of which are plain daft*, some querying things we were asked to do by the donors after they received the first draft, and others just go to show the consultants didn’t spend enough time checking issues, rehashing old concerns that were addressed a while ago.

But the biggest problem we had with the consultants’ approach was that it is VERY BUREAUCRATIC. We are constantly being told that they sense potential, innovative thinking amongst the partners etc., but then complain that it is not fully reflected in the project proposal. This seems to me to miss the wood for the trees. Surely, the point about this exercise is not to produce the perfect proposal, but to get the best project possible. If they think the partners have the right ideas who really cares about the project proposal? The consultants’ approach was exemplified in the following recommendation:

“Writing proposals for funding, and delivering to donors requirements, is an acquired skill, requiring recognition of the donors’ needs for accountability and monitoring. This proposal has clearly worked hard to achieve the necessary standard but some gaps remain. External support appears necessary to complete this proposal, and would also help [the  lead partner] and the partners to better manage future donor applications, reporting and partner relationships.”

Eh? Let me just get this right: you’re recommending we spend further lavish sums on you and your ilk so all the donors can wow themselves at our amazing use of the latest jargon and buzzwords? There was plenty more along these lines, e.g. suggesting one of our partners lacks the wherewithal to chair a meeting neutrally (so the consultants could do it for us!) which is just utter tosh.

On the other hand, we do have reason to be concerned as to the consultants’ expertise. One has negligible previous experience of the country in which we work whilst the other has never been involved in a project remotely like the one we were proposing.

So let’s be clear about this: if we want someone to give us some money then we need to explain clearly what it is for, that is self evident. Furthermore they have the right to hire an independent external evaluator to check on their behalf for any red flag issues. But, if a project has taken us months to plan, and incorporates lessons from a previous pilot phase, funded by one of the same donors, then in one week’s running around the country you are unlikely to understand every aspect sufficiently to offer detailed criticism. And it’s just plain rude to try making extra work for yourself when you’re paid so much more than the real innovators. Advice is welcome, self-promotion  and unfounded criticism not.

However, an equal responsibility lies with the donors who are responsible for the context in which said consultants are operating; if they didn’t demand such ridiculous levels of paperwork the consultants wouldn’t write reports like this. Donors could give more freedom and responsibility to in-country (i.e. embassy) staff to approve and support projects which they know are good without endless proposal tinkering. (Though for this to really work well, said embassy staff would have to stay for longer stints than 2-3 years.) They should also stop trying to influence small details in such projects (the consultants advocated a role for the donors on the project steering committee which is absolutely not going to happen), and instead have greater confidence in those they have chosen to fund. (Give them the rope to hang themselves if so be it.) No wonder developing country officials get so exasperated with donor meddling.

Sadly, I cannot imagine this situation changing very fast, so it’ll be back to that proposal in the new year …

* I’ve previously written about gender mainstreaming and conservation. Now, apparently, we have to mainstream health issues too despite the complete lack of relevance to the proposed project.

A useful lesson

When I was about 15 I had an English teacher who didn’t much care for dissenting views. A critical or negative essay about one of our set texts would be marked down as, or so it seemed, showing insufficient appreciation for the magnificent literary powers of the author. In quasi-Orwellian fashion we were supposed to like whatever it was we were given to read. Conversely, I found, if I wrote an essay praising the text and full of flowery phrases like “innate symbolism” (still damned if I know what that one meant, >20 years after I wrote it) then I got an A or A+ regardless as to whether or not my essay corresponded to my actual views on the book in question. At the time all this did was put me off all literary criticism and associated arts studies for the rest of my life (so far). Years later I regard it as one of the most important formative experiences of my school years, and critical to resolving quandaries like the need to mainstream gender issues into a proposal that has little to do with gender.

ps. For those who like their critics not to hold back on the sarcasm, the full-frontal demolition job on the **** one has to put into funding applications is here.

pps. Actually I liked the Orwell!