Social entrepreneurs and M&E

Bill Easterly apparently wants to see faster determination of successful and failing aid projects than is provided by traditional monitoring and evaluation. That is according to Tom Murphy’s report on the DRI annual conference from back in March.* Tom commented:

“… strong and open monitoring and evaluation practices can ensure to the trial and error of the Easterly ‘searcher.’” (sic)

I also seem to recall – but cannot now find the right quote or link – the suggestion that aid practitioners could seek inspiration from how success and failure business is determined in the business world. (The DRI debates considered whether an RCT would be a suitable evaluation mechanism for the IPhone game Angry Birds – not the best example imho!) (Update 27/06/12: found that link, it was Ian Thorpe blogging here.)

As someone who played a major role in setting up a local NGO and then leading it for several years, I guess I can be classed as something of a social entrepreneur. Thus I have a few thoughts on how this can or cannot work in practice.

Like too many organisations in the development and conservation sectors I wouldn’t put down M&E as one of our strong points. We’re not terrible at it, and we’re getting better, but it wouldn’t be too difficult to poke plenty of holes in what we’ve done so far.

As ever with M&E, getting the budget balance right is tricky. If we had as much money as the MVP we could do some amazing M&E. (Hopefully a lot better than the MVP has achieved in practice!) But we do not have anything like that amount. Plus we have a problem that many of the impact indicators we are targeting take a long time to move in a positive direction (e.g. biodiversity). So we have to wait quite a while just to see whether our monitoring programmes are capable of detecting the kind of change we are seeking to achieve.

These challenges are universal. More particularly, taking an entrepreneurial approach involves a lot of flexibility and adaptation to changing circumstances. Most people seem to agree this kind of approach is a good thing. Which is all fine and dandy, but it does make it hard to set up your M&E baselines, because we’re continually adapting how the project will work, and thus what impacts it will have. As the project matures it settles down better, but for our earliest pilot sites the opportunity to establish a firm M&E baseline has long gone.

So how do we track our own progress? Mostly I would say through a number of milestones, which may or may not be well set out on paper. Conceptually we started with a pretty good idea of what we wanted to achieve. As things progressed we developed a number of internal targets (usually with very flexible timelines if they are specified at all) against which we could measure ourselves.

The analogy with modern business is perhaps something like Facebook. At first it was just an idea – something cool to do – then it became an enterprise, but one not particularly focused on bottom-line impact (profit) as on other metrics and milestones (e.g. market share and time spent on site by users). Only latterly has the focus at Facebook shifted increasingly to monetising their substantial achievements. (If recent share performance is anything to go by, this is proving tricky.)

Similarly our own project is being subjected to increasingly robust M&E assessment, as indeed it should do. But, in my mind, most M&E approaches and entrepreneurial innovation apply to quite different stages of project development. This is also why starting small is so important; it allows time for KISI. Alas too many people in conservation and development are often in such a rush that they want to spend $100 million first, and ask questions later.

* Yes I’m finally back blogging again. I have a number of posts queued up in my head responding to the news over the last couple of months. Hopefully it won’t all seem too much like yesterday’s left-overs.

Advertisement

Small may be beautiful but is easily overlooked

CGD’s Connie Veillette and John Norris have some proposals for rationalising (and trimming) US AID’s budget. Included in some eminently sensible suggestions was the vignette that Belize apparently receives ~$20k each year from US AID, and, in Norris and Veillete’s opinion, this probably does not justify the overheads involved in managing it, and so it would make sense to cut it to focus on bigger things.*

If I were a senior manager at US AID I expect I would be pretty convinced by this argument for rationalisation. Except that I know I’ve also heard from time to time talk amongst BINGO staff to the effect that while a certain project may be one of their smallest / cheapest but it may also be one of their best. Such small projects often have very limited ambitions but may fulfil them exceedingly well. Not trying to be something more than you are can be a real virtue in such circumstances. Thus my message to US AID is that that $20k you send each year to Belize may just be some of the best $20k you spend each year, so if you’re thinking about cutting it in the name of rationalization please ensure there is someone else to take on funding the project before you pull out.

This brings me on to a related point about project sustainability. Some projects nearly but never quite succeed in graduating from donor / BINGO support. This can be frustrating for the donor / BINGO who want to move on, and always intended their project should reach self sufficiency. Indeed this may have been central to the original concept. If the project has not succeeded in its wider aims then it should certainly be canned, but I have also heard the odd suggestion from donor / BINGO staff that patience is running out with more successful projects. I find this attitude rather myopic. Given the failure rate amongst conservation and development projects it seems to me crazy to junk a project just because it still needs a few thousand dollars a year in support and technical advice.

So this is a plea to all those aid planners out there in search of the big win: please do not forget the various little ways you may have already discovered to make a small difference. They may not add up to the end of poverty as we know it, but they all count, and are probably a helluva lot more cost effective than those big daring programmes that too often fail for trying to be just too ambitious.

* I am amazed the US doesn’t fund Belize more, but it doesn’t matter to my argument if this particular detail is wrong.

Small still is beautiful

The beautiful bee hummingbird is the smallest bird in the world.

Last year I blogged about how I believe that in conservation and development small is often beautiful, remarking:

“small projects and organisations are a lot more personal; I think this element of personal endeavour can do a lot to ameliorate the charge of development work being patronising.”

I stand by this, and I think the recent small kerfuffle over Community-led total sanitation (CLTS) is as good an indication of this as any. First, Robert Chambers, who I genuinely respect, praised it in a piece of on the Guardian and Duncan Green’s Poverty to Power blog. On that blog I posted a comment querying how well this translated from a small pilot project into a big donor-funded programme (see also my recent post on the Scaling Up Fallacy):

“It’s great to hear of new bottom up approaches like this. I have a question about roll out … I can see how this kind of approach could work at a pilot level with well-trained *sensitive* facilitators. But what about when it is scaled up? How do communities react when a local govt health official comes along and shames them all? From my experience I can see how the whole approach might suffer from institutionalization – and the different relationship people tend to have with officialdom – but seek enlightenment from my cynicism.”

Alas I received no enlightenment.* Then Liz Chatterjee responded with a comment piece in the Guardian that raised many of the ghosts that I had speculated at. Whilst I should be cautious about assuming that cause and effect align with my own prejudices, I cannot help thinking that at least some of the more degrading aspects of CLTS might be alleviated in a pilot project, and/or where you have a small NGO with dedicated, caring staff who are alert to the possibility that things could go wrong.

Big organisations, and especially big government bureaucracies, are inherently clunkier than their smaller, nimbler cousins. They tend to resort to top-down, even when implementing a supposedly bottom-up approach. Staff subjected to a couple of weeks’ training will try to adhere to a perceived blue-print, rather than have the confidence to innovate and respond flexibly to the impacts they achieve in the field.

Unfortunately, these conclusions are pretty depressing for anyone expecting international aid to have large-scale positive impacts. It suggests big economies of scale are really hard to achieve. And I think this is true of most community development oriented projects. More than anything else these depend on the quality of the staff implementing them, and the talent pool in developing countries can be worryingly shallow.

But Aid can and does support many other types of projects which may be more susceptible to rapid expansion. If donors have the right kind of relationship with the host country government, as Owen Barder implies with many of his posts on aid successes in Ethiopia, then maybe you can improve services at a larger scale. That is not to say that we should give up on community projects – quite the opposite (as otherwise I’d be out of a job!) – but we need to understand their limitations, and who are the best agents to support them.

* Hint: even if you are an aging giant of development studies, if you’re going to use new media, then you should also make an effort to utilise its capacity for rapid interaction with your readers that more traditional academic publishing largely lacks.

Small is Beautiful

J over at the Tales from the Hood blog has recently been reminding us about how important professionalism is in development (1, 2, 3, 4). For the most part I agree with him: I find it incredibly frustrating dealing with well-meaning amateurs whose suggestions are mostly the opposite of helpful. But I believe that the drive for professionalism (including proper professional standards) needs to be balanced with consideration for what can be lost through taking such a focus.

Exhibit number one is to note that all the ‘professionalism’ of the donors, multi-lateral agencies and the big NGOs has not got us very far to date. The conservation and development industries may well have made life more bearable for millions of poor people around the world and mitigated some of the worst environmental practices, but both have fallen a long way short of all the promises they made.

It is true that many flaws of international aid have been pointed out by various commentators over time (see my blog roll for a small selection), and perhaps if all these flaws were addressed, the professional approach of all these various agencies would suddenly bear more fruit. But right now, we don’t know that for sure.

What does seem clear to me is that ‘professionalism’ generally seems to be associated with the established players who have the resources to hire the right people and do all the proper evaluations before embarking on a new course of action. I think this omits an important class of conservation and development initiatives.

Small is beautiful. I know it is a cliché, but there is a lot of truth to it. Firstly small projects can be a lot easier to manage; lack of complexity is certainly a virtue. Secondly small projects and organisations are a lot more personal; I think this element of personal endeavour can do a lot to ameliorate the charge of development work being patronising.

Small also allows for experimentation; where the sums are low, there often won’t be much lost if a project collapses for having failed to follow one or more pieces of best practice. Some of these holes can, and should, be filled in later before scaling up (if that is the goal). Bill Easterly constantly reminds us of the power of many different people making their own separate attempts to achieve their goals over a centrally-planned system. I think the aid industry could benefit from a lot more disruption from nimble, radical-thinking start-ups.

All in all, despite the manifest problems of DIY aid, if I had a donation to make, I’d far rather give it to a small local organisation I know well with relatively modest objectives and a long-term commitment to the communities it supports than to a BINGO.