Scraps from big brother’s table

“How do you find working with [name of BINGO withheld]? How could the relationship be improved?”

These questions, and others in a similar vein, were asked of me and my colleagues last week by consultants hired by a major donor of a BINGO that supports us with some of said donor’s money. The donor wanted to see whether they were getting fair value for money from the relationship.

Equal partners, right?

As with so many other ‘partnerships’ in the aid world, the relationship between a BINGO and small local NGO is far from an equal one whatever anyone claims. Indeed, just like recovering drug addicts, so for BINGOs: the first step in dealing with the problem is to recognise that it exists and that simply mouthing endless platitudes will not make it go away.

There is also no avoiding the fact that societal pressures (aka donor preferences) mean that both sides, the BINGOs and the local NGOs, are addicted to such relationships, with little hope of being weaned off. Moreover, for both sides there is often very little choice in who they partner with: at least in tropical conservation BINGOs largely try to stay out of one another’s territories, whilst vice versa the choice of capable local partners for a BINGO to work with may often be quite small. So to a significant degree partners are stuck with one another.

A more reflexive, self-critical approach will certainly help; being honest about the nature of the relationship does not have to mean being arrogant about it. As with so many things in life, that understanding, personal touch can make all the difference to the relationship. But is there anything more that could be done?

Ultimately I think it comes down to a question of how and to whom the BINGO are accountable. This echoes one of the main themes in development aid in recent years to which a leading response has been a call for more transparency, and the emergence of things like the International Aid Transparency Initiative. I think the same trick could work again here.

One of the things we found frustrating about our relationship with the BINGO was being drip fed small amounts of money that covered only a few months and were then micro-managed, e.g. budget lines of only $1-2,000. (The BINGO did not have systems that could differentiate between capable, trusted local partners, and others about whom they were more cautious.) The micro-management was a pain, but in many ways it was the drip-feeding of funds that caused the bigger problem, because they were almost impossible to plan for.

Partly, I think, the BINGO did this to remain in control, but partly also I think because it kept us in Oliver Twist mode – always asking for more but rarely seeing the bigger picture – and thus allowed them to prioritize their own management costs and strategies. I would like to see BINGOs being much more open about their own funding streams; how they are divvied up, managed and reported. The middle-man role is an important one, and for our part, those of us in smaller NGOs need to recognise that performing this role does not come cheap.

So when a big chunk of the donor money goes on BINGO bureaucracy (no doubt disguised as technical assistance) we shouldn’t bleat too much. Proposing standards for what is an appropriate slice to take is likely to be counterproductive (in the same way that misdirected focus on overheads has proven). Instead the BINGO becomes accountable to some degree to their small NGO partners (who are already have to do lots of accounting to their bigger brethren), and both sides of the partnership can better evaluate whether or not they are getting value for money from the relationship. Longer lasting commitments of funds would also help the smaller NGOs to plan better.

Advertisement

Donors: show us your scorecards

A few months ago Lant Pritchett hectored the World Bank Executive Directors to publish their scorecards on the three candidates for the WB presidency. It was a nice idea which unfortunately didn’t get a lot of traction, presumably because WB presidential selection is essentially a political process, despite all the publicly claimed desire to simply find the best candidate.

But why limit this idea just to the WB president’s selection? A lot of aid funding for NGOs comes through competitive calls for proposals. Some donors, the EU comes to mind, define tediously exact criteria by which proposals will be judged, others are rather woollier. Wooliness doesn’t have to be a bad thing – it allows extra weight to be given to proposals which contain that hard-to-define X factor – but for all the fact that judgements are necessarily hugely subjective – often favouring the known, established applicants – precision has a lot of merits in clarifying ranking of proposals. When the number of proposals is high it very probably becomes the only way of effectively comparing the full range of applications, so I assume most donors practice something along these lines.

So why not publish the results? In the age of openness and transparency, why are donor decisions shrouded in mystery? The EU may tell you the score allotted to your proposal, but do not tell you the threshold that was necessary to obtain funding. (When your proposal has averaged over 80% it does leave you wondering exactly how good it needs to be?)

Donor decision-making needs to be final, there is no question of that. Endless appeals against the system, however it is rigged, won’t do anyone any good. But it would still make the lives of NGO fundraisers easier if we knew both what had and had not been successful in the past, and what were the various scores. (It might even save us from writing the odd wasted application.) So donors, how about a bit more sunlight on your decision making? Go on, show us your scorecards!

Accounting for failure

Project failure is far too common in conservation and development for anyone’s comfort. Many agencies and practitioners regrettably seek to hide their poor records behind euphemism and by redefining success radically downwards after that fact. Last year an aid bloggers forum considered the question of admitting failure although the consensus was not very positive (see my two contributions: here and here).

Yesterday I blogged about an alternative solution: pushing aid projects to obtain insurance against failure. It’s a nice idea, but probably quite a few years away at best from wide-scale implementation. But, it occurred to me, there is an intermediate solution which would take very little change to implement. Put simply donors would account rigorously for their projects’ success rate. It would work as follows.

Already most donors demand clear statements of project aims and expected outcomes before committing funds. Good donors will also ask for a risk assessment. All we need to do is quantify those risks. Sure putting a number on the likelihood that you will get the necessary buy in from local government officials is an exercise in extreme subjectivity, but we can live with that. Multiply all your risk percentages together and you should get an indication of the likelihood of project success. (I bet often it will be substantially lower than the project’s proponents would like, but if they try to massage it up they’ll get caught out later …)

Then, come evaluation time, the reviewers should explicitly assess what proportion of the original aims and outcomes have been attained, and what risk factors in actual fact came into play to the detriment of project impact. This assessment would have to be extremely robust and refer only to the original estimates of impact, so as not to allow project managers to redefine success downwards mid-project. (A subsidiary assessment could consider revised aims that were formally set out and agreed.)

Project proponents and implementers who consistently underestimate risks will be shown up (although, obviously the feedback loop will take a few years to generate much in the way of information). Imagine if donors pooled all this information so that they could look up organisations records. Imagine further if such estimates were linked to specific key people who worked upon the projects, and you had to justify both your risk assessment accuracy rate and actual project success rate in your next job interview. (Sensible employers would be tolerant of those who have only worked on a few projects and just got unlucky, but could talk intelligently about what went wrong, the lessons they learned and how they would put right such situations in future.)

As well as improving accountability and honesty about the very real risks involved in most conservation and developing projects, as the data built up, donors could also use it for their own internal evaluations. How successful were their projects? Which types of risk factors proved to be the most dangerous? Which types of risk factor were consistently under-estimated and which might have been over-estimated? Donors wanting specifically to target riskier projects with some or all of their money should not be discouraged; we all know that the pay-off from such initiatives can be that much bigger than me-too carbon copies of established models. This way the risk would simply be more explicitly acknowledged.

The total guesswork inherent in the original risk estimation would limit the data’s utility in evaluating individual performance, but for donors and BINGOs, when aggregated across the organisation, these errors would start to average out, and analysis of later results would help agencies to refine their estimates. E.g. typical political risk factors could be classified according to severity, with information for project proponents on actual failure rates in previous projects to help them gauge the likely risk in the new project they are proposing.

I can see plenty of resistance from many in the aid industry to such crude quantifications, but the move to increase transparency in the sector is gathering momentum. Perhaps it is the sort of thing that could be considered in the next iteration of IATI?

Donor befuddlement: half measures & contradictory aims

Two posts recently have brought to my attention how far donors still have to go in designing a more adaptable and hence effective and efficient aid system.

Half Measures

First, Lindsay Morgan (of Dispatches) wrote a piece about performance-based contracting for health services in Southern Sudan.  Those with a serious interest should read the whole piece yourself, but in essence it appears that a coalition of donors is trying to inject a little discipline of the market into healthcare provision in one of the most under-developed soon-to-be-countries of the world. Except that they still don’t seem to entirely get some of the most important aspects of how the private sector works. So NGOs are getting performance-based contracts, but have to work with GOSS staff, over whom, presumably, they have pretty low influence (no hire or fire, no disciplinary measures), which to my mind rather diminishes the level of achievement you can expect from management. The NGOs also had to work through some kind of central procurement system, which apparently caused no end of problems.

Whilst the GOSS health ministry staff may perhaps have learned something from this kind of system (NGO working practices and the like), I fail to see how just putting in a team of technical advisers couldn’t have just achieved the same thing. On the other hand if the donors want to get private sector / market-based practices embedded in healthcare provision in Southern Sudan, then they need to act a bit more like ordinary customers and stop making silly restrictions as to how their contractors work.

Contradictory Aims

Then Madeleine Bunting wrote about DfID’s new contradictory policies (HT: Matt @ Aid Thoughts). I am with many of the commenters in having little sympathy for the worries of DfID staff about Daily Mail journalists digging through the data and trashing one or more aid projects. The Daily Mail may be more than a little bit biased, but a good dose of daylight is the best prescription in my opinion, and, very probably, there are one or two aid projects which deserve a good trashing.

But I do think that the plan to axe up to half the work force is crazy, and even crazier when one considers the anticipated large increases in DfID’s budget. (Disclosure: I have a friend who works there.) Openness and transparency requires more effort to put documents into the necessary formats, especially if the data is to be made available in a manner actually useful for general analysis. These kinds of overheads cannot be just wished away. Moreover the relentless focus on minimising transaction costs is, in my opinion, mostly illusory: someone is bearing these costs, if not DfID then someone they are funding. I know of at least two international consulting firms currently doing very well out of managing DfID grants round where I work.

Not a good week

If anybody out there reading this works for DfID and/or with healthcare in Southern Sudan can correct me or at least give a better picture, then please do. But, unless I’m completely barking up the wrong tree, all in all it doesn’t look a good week for donors.