Admitting it’s not good enough

The call to be open about failure in development projects has much to be said about it, as I have blogged before. But between success and failure there is a middle ground in which many conservation and development projects cluster. Sometimes acceptance of this result is appropriate; it may not be appropriate to expect rich country levels of achievement in some of the poorest countries. And such compromise is de rigeur in any policy processes, whatever a country’s level of economic development.

But when it comes to project implementation I think that too often we are too ready to accept this half-baked mediocrity, write up our ‘success’ reports and move on. Unfortunately short term papering over of the cracks can lead to long term failure, although by that point usually the main protagonists have long since moved on. Many times this takes the form of an initially successful project that has been poorly scaled up into a programme that grinds on for years based on its initial fanfare, before eventually donors get tired off the lack of progress and pull the plug, often one at a time so it limps on for quite a while with ever-diminishing financial support. My guess is that this kind of failure rarely even gets noticed as anything other than a sense of regret amongst those who were involved that so much early promise should amount to so little in the end.

But sometimes the failure can be more dramatic, such as the drastic short-comings both morally and militarily that have been brutally exposed in the Malian army over the last 12 months, despite years of capacity building from the US previously. Todd Moss laments the tendency to see those policies and results through rose-tinted glasses. I’m no military man, but allow me to guess a little at what might have happened: the junior officers on the ground would have reported the good start they made whilst making their reservations clear that there was a long way to go. These reservations were subsequently air-brushed out by senior officials and politicians keen to declare success and move on. Doh!

The even bigger difficulty occurs when that conversation needs to take place across the cultural boundary. How do you tell the local partner that while their efforts are nice and appreciated they do not, ultimately, deliver on the requirements? That technically their output is lacking a necessary level of sophistication? Arrogance does not become one, and us oh-so-enlightened Westerners are guilty of that far too often. Smooth diplomacy, however, can only get you so far: either you need to accept the product delivered with all its flaws, or you need to risk giving offence in pushing for improvements, whoever is tasked with delivering them.

At this point budget strictures can come into play. Few project designs incorporate budget for doing anything twice (although a contingencies budget can help). So as well as having the courage to reveal the hard truths to local partners one faces the challenge of finding the budget and/or fessing up to the donor how you stumbled. Little wonder then that many project managers opt for the easy way out. A little less neo-colonialism, may come at the cost of a lot less development. It doesn’t take much imagination to guess which outcome the target beneficiaries would prefer.

---

Before I get deluged in a pile of hate mail, I should point out the big but in this. There is obviously a huge slippery slope starting with robust and honest assessment of technical quality of local partner outputs and descending to rampant neo-colonialism, and at the bottom of which one is likely to find the target beneficiaries having very different views. Alternatively you can just call the bottom of the slope charter cities, on which it seems the jury is still out, and will almost certainly stay out, until one is actually attempted.

I should also like to add that such eventualities as I describe above are not the rule. Many times I have seen excellent outputs produced by local partners. But neither, unfortunately, it is as rare an exception round here as one would like, especially when dealing with quasi-governmental institutions who do not have a meritocratic culture.

Leapfrogging is hard with only one leg

“Environmental systems are easier to set up when development is still in its early stages” claims the FT. This arguably is a misquote from the article (blame the sub-editor?) by Sarah Murray who cites the old chestnut of the leap over fixed line telephony straight to mobile phones in Africa. With the possible exception of mobile money, building, of course, on the earlier leap, and still mostly confined to Kenya, I struggle to think of any other big technological leaps made that have significantly fast-forwarded development on the national or international scale.

That is not to say that small leaps are not being made by various businesses across the developing world, but Ms Murray was talking about the big systemic leaps, especially to greener technologies in such things as agriculture and power generation. Here I feel the picture is less rosy.

Mobile phones first brought benefits to the elite, benefits they had not previously been able to access in other ways, so I imagine that less regulatory hurdles were put in their way. In contrast there are lots of powerful vested interests in agriculture and power generation, and elites have little problem putting food on the table or fuel in their private generators. Moreover significant reform to the farm sector inevitably involves wrestling with that most sensitive of subjects, and one in which elites are particularly attached to their privileges (and thus potential for abuse): land.

Most eco-friendly farming techniques either require a certain amount of upfront investment (organic certificates don’t come cheap) and/or have a long term payoff, e.g. better soil nutrient retention. Without security of tenure such innovations will always be restricted to small pilot projects.

Alas most farmers in Africa (Ms Murray was specifically talking about Africa) are smallholders with negligible if any land tenure security. Large commercial farms will at least have the relevant pieces of paper, but may have had to trample upon a few rights just to get them in the first place, and thus can be a lot more vulnerable to changing political winds then they might like, i.e. their tenure security ain’t so great either.

So although I would like to be more optimistic, I would caution anyone expecting a great leap forward in agricultural technologies in Africa or other developing countries. It’s hard to go leapfrogging when one leg is nailed to the floor.

Hat tip: my Dad.

The development tortoise and the donor-fuelled hare

Now my blog has been going for a little while, one of the great pleasures is getting unexpected comments on posts several months old. (Comments on new posts are similarly gratifying – I love all commenters equally! – but not so unexpected.) Thus I’ve recently enjoyed my little debate with David on my post from May: The Scaling-up Fallacy. We touched on several issues related to project scale, but one seemed to need a fuller response, hence this post.

David suggested that small pilot projects that are not designed to be scaled should never happen on the assumption (my inference) that they will never represent value for money. He went on to say:

“I’ve seen a lot of projects that work really well with a ratio of 10 staff to 300 participants from communities of 10,000, without any suggestion that the delivering company can grow to be 100 people and deliver the same service to 100,000 people – and this is often in a country of 10,000,000 or more. It’s hard to see how that is ‘good development’ as opposed to advanced humanitarian relief.”

I think he’s got some pretty good points, and, as a general principle, I do think we should be looking to take things to scale. E.g. I fully support the ambition of Jeffrey Sachs et al. in the effort to make poverty history, I’m just sceptical about their proposed means. But, as I’ve blogged before, I do think there is a lot to be said for small projects that do not try to be something more than they are or can be. So how do we square this circle?

First I think it is worth noting that not every Western company grows into a world-conquering behemoth, sometimes this is not for lack of ambition of the directors, but many other companies stay small out of choice; less hassle for the boss. The same also applies to development, particularly amongst the smaller NGOs. If their donors are happy to keep the money flowing then they clearly represent value for money to someone.

Many small company bosses want to keep their companies small because of the challenges of managing staff. These challenges are multiplied in developing countries where the small pool of educated talent is often a major constraint. I suggest that might be what is stopping many of those projects with 10 staff serving 10,000 community members from growing further.

Unfortunately, albeit for the best of reasons, many donors, especially institutional donors, are not satisfied with this. They want to reach the 10 million. So they push the accelerator pedal as hard as they can … and then the wheels come off because capacity to deliver is just not there.

One response might be simply to invest in capacity development – e.g. focusing on education, but that turns out to be just as dependent on internal capacity. I think we need to be more pragmatic. If, as most do, donors want to achieve tangible results then they need to face up to the reality on the ground, rather than swinging wildly between over-optimistic up-scaling programmes, and white flag exit strategies. A project that is going just as big and as fast as it can is better than a car crash. As in so many other cases, it turns out that Aesop’s old fable is extremely relevant to conservation and development: nine times out of ten, the tortoise wins in the end.

Inflexion points

Various thoughts have collided in my head over the last week concerning value for money* in tropical conservation and development. I can see two possible inflexion points, which are clearly related in some sense, but also reflect different concerns linked to the issue of scaling up.

The first I’ll call the So What? point. At the bottom end of development, if someone spends $1,000, say, on donating school books, then most people’s reaction is simply to celebrate the fact that now some kids in a poor country have some books that they would not have had otherwise. It would seem churlish to question whether, in fact, there might be a better way to spend the money.

However, if someone else spends $10m on school books, then one is naturally inclined to question what is the eventual impact of this donation. How much better educated are the kids? How many are lifted out of poverty – or at least become less poor – as a result? They spent $10m on school books, but so what?

The second inflexion I’m calling the Get out of Bed point, and is the minimum grant size likely to interest a development practitioner. It is to a large part set by the organisation for which you work. BINGOs may not chase grants smaller than $1m, while the smallest NGOs will happily jump through several hoops for just a few thousand dollars.

The problem comes when a would-be donor perceives one as having a lower Get out of Bed threshold than one in fact does. One does not want to seem ungrateful, and maybe if there were less hoops  to jump through or less strings attached (hint to donors: these make small grants much more worthwhile), and/or if the grant would be easily renewable (dependable funding is always appreciated!) then we might go for it, but every application form takes up senior staff time when they might be better off just managing what they’ve already got on their plates. This can be a tricky issue for small but growing NGOs to navigate.

I haven’t had the time to really think this through, but I suggest that the best conservation and development action probably takes place when the two inflexion points are roughly aligned for both the implementing agency and its main donors. If the Get out of Bed threshold is much lower than the point at which the So What? test becomes applicable then the organisation concerned is desperately short of money (or fixated on raising money rather than achieving notable results). Conversely if  the Get out of Bed point is significantly higher than the  So What? question level, then the agency is probably swimming in cash and may well be more concerned about spending its great dollops of wonga than achieving value for money.

Contrary thoughts or suggested examples of the different cases are welcomed in the comments section.

* Note this discussion is not about the quality of service provided, efficiency of the activities undertaken, or the level of overheads applied. For the purposes of this post please assume such variables can be controlled for, although there may be an inevitable degree of confounding factors.

Why I’m a Millennium Villages sceptic

Last week Jeffrey Sachs set out a robust defence of his brainchild, the Millennium Villages Project, although, as Tom Murphy pointed out, it was somewhat low on detail. I’ve never knowingly been near a Millennium Village, but my own experience causes me to doubt the lasting legacy of the MVP, at least in countries with similar problems to where I work.

First the good news, Sachs took on some of this detractors by saying that the MVP was as much about developing systems to improve service delivery (and hence attainment of the Millennium Development Goals) rather than just,  per se, achieving the MDGs in the targeted villages. I’m a big fan of systems approaches, so this gets the thumbs up from me. Systems are definitely more easily replicable and scaled up than individual projects that focus simply on the needs of its target area.

Now for the bad news, systems are not automatically and by definition scalable. A system that works well at one level may not work well a wider scales due to unanticipated problems and bottlenecks. Ben Ramalingam recently blogged on exactly some of the new challenges that occur as one scales up. This doesn’t mean that the original system was designed badly, but simply that good systems management takes an iterative approach, making tweaks and improvements as we go along (what I term the KISI approach).

But that is not the biggest problem that I see. Even the best designed systems need to interact with things outside their control, in particular people; indeed I suspect that the MVP has people playing integral roles at every step in the way (i.e. that mostly what we’re talking about here is systems for organising human work). A system’s output is constrained by the quality of these interactions. In short, as any good businessman knows, you need competent and motivated staff to deliver a high quality of service. And that is where so much service delivery in developing countries falls down, with last mile service delivery particularly badly managed. Unfortunately short-term, local solutions to this are not scalable.

The problems are legion, and not all a result of poor education amongst the workforce. I know some excellent and (when you consider what they are up against) surprisingly motivated local civil servants. But the overall system drags everyone down. Sure you can tinker at the margins with systems to improve paper flow (mostly in local government around here, we’re talking about paper flow), but the elephant in the room is an unmotivated and unsackable workforce.

Of course this problem will apply at the MVP sites, but there you also have a massive aid effort with lots of expat technical advisers and a high level of political interest. I’ve noticed around here, normally sloth-like civil servants who won’t even sit in a meeting without a generous per diem rush around like lauded socialist workers striving manly (or womanly) in the name of their country when a bigwig is due to visit, working into the night and through weekends, all without any per diems.

Thus I fear all the achievements of the MVP will wash up against the great brick wall that is a change resistant bureaucracy. Once the high level of funding, all the expat TAs, and the high level political interest have withdrawn we’ll be back to business as usual, and the MVP will be neither sustainable in the selected pilot villages nor scalable. Maybe this will not apply everywhere, but I would wager a decent sum that it will happen here. The community contributions which Sachs highlights may also be much harder to be elicit when it’s just government staff doing the asking.

The MVP has a laudable goal, and even as an experiment, the idea of resolving various systemic problems in service delivery is a worthy one that definitely deserves some experimentation; marginal changes can lead to marginal improvements, and, as a by product, perhaps a marginal improvement in government staff morale. But if Sachs wants to take a systems approach to achieving the MDGs maybe he should have looked at HR management reform in developing country civil services. It’s a Herculean task to be sure, that, around here at least, the World Bank has been striving at vainly for some time. But until you resolve that problem I fear these sorts of big push attempts to transform service delivery and hence quality of life in developing countries will always be at least one more big push away from succeeding.

Big fish swim in bigger ponds

The ever excellent Ben Ramalingam posts on the complexity challenges of scaling up aid projects. He and the paper he reports have many interesting and valid points, and yet the dry analysis glosses over I think one of the biggest challenges of all: increasing political interference. It is relatively easy to run a successful small project largely under the radar; those few people who know about it won’t feel threatened by it. In such instances I have vaguely sensed what seems to me a rather patronising attitude from local officials; that it’s quite wonderful that the great unwashed should have their little aid project play thing to keep them happy, while they continue in their ways very much as before, and as long as things stay that way everyone else is happy too.

But as you scale up and the numbers become bigger two things happen. Firstly some people start to feel threatened by your project. You may be directly stepping on their toes or constraining their ability for future manoeuvre, or it may simply be that your success risks putting their rather more modest achievements in the shade, leading to their bosses (and ultimately the donors) to question why they are throwing so much money at regular system when there appears to be a better way of delivering said results. Secondly, the sharks will smell the big money and start to circle. Big development programmes are patronage opportunities and per diem cows to be milked to the utmost. Additionally, and as I’ve previously remarked, you lose the tight management team you had before and instead are confronted with trying to manage a big programme through the leviathan of government bureaucracy.

I think this explicit political dimension is captured to a certain extent in some of the figures that Ben reproduces (trade union organising is listed as an example), but I think it unwise simply to relegate it to one aspect of systems complexity. To be fair to Ben, I think he is not arguing that politics is unimportant, merely presenting a systems-based analysis of the challenges faced. Anyone interested in trying to scale up projects should read his blog post and take due note of the recommendations. Just don’t forget the politics!

Establishing and maintaining a supportive political environment is a major challenge and critical to the success of any aid project beyond the very smallest scale. As I sometimes joke to friends: in my job as a conservation project manager in reality I’m a part time conservationist, but a full time politician. Unfortunately that’s not out of choice.

Small still is beautiful

The beautiful bee hummingbird is the smallest bird in the world.

Last year I blogged about how I believe that in conservation and development small is often beautiful, remarking:

“small projects and organisations are a lot more personal; I think this element of personal endeavour can do a lot to ameliorate the charge of development work being patronising.”

I stand by this, and I think the recent small kerfuffle over Community-led total sanitation (CLTS) is as good an indication of this as any. First, Robert Chambers, who I genuinely respect, praised it in a piece of on the Guardian and Duncan Green’s Poverty to Power blog. On that blog I posted a comment querying how well this translated from a small pilot project into a big donor-funded programme (see also my recent post on the Scaling Up Fallacy):

“It’s great to hear of new bottom up approaches like this. I have a question about roll out … I can see how this kind of approach could work at a pilot level with well-trained *sensitive* facilitators. But what about when it is scaled up? How do communities react when a local govt health official comes along and shames them all? From my experience I can see how the whole approach might suffer from institutionalization – and the different relationship people tend to have with officialdom – but seek enlightenment from my cynicism.”

Alas I received no enlightenment.* Then Liz Chatterjee responded with a comment piece in the Guardian that raised many of the ghosts that I had speculated at. Whilst I should be cautious about assuming that cause and effect align with my own prejudices, I cannot help thinking that at least some of the more degrading aspects of CLTS might be alleviated in a pilot project, and/or where you have a small NGO with dedicated, caring staff who are alert to the possibility that things could go wrong.

Big organisations, and especially big government bureaucracies, are inherently clunkier than their smaller, nimbler cousins. They tend to resort to top-down, even when implementing a supposedly bottom-up approach. Staff subjected to a couple of weeks’ training will try to adhere to a perceived blue-print, rather than have the confidence to innovate and respond flexibly to the impacts they achieve in the field.

Unfortunately, these conclusions are pretty depressing for anyone expecting international aid to have large-scale positive impacts. It suggests big economies of scale are really hard to achieve. And I think this is true of most community development oriented projects. More than anything else these depend on the quality of the staff implementing them, and the talent pool in developing countries can be worryingly shallow.

But Aid can and does support many other types of projects which may be more susceptible to rapid expansion. If donors have the right kind of relationship with the host country government, as Owen Barder implies with many of his posts on aid successes in Ethiopia, then maybe you can improve services at a larger scale. That is not to say that we should give up on community projects – quite the opposite (as otherwise I’d be out of a job!) – but we need to understand their limitations, and who are the best agents to support them.

* Hint: even if you are an aging giant of development studies, if you’re going to use new media, then you should also make an effort to utilise its capacity for rapid interaction with your readers that more traditional academic publishing largely lacks.

The Scaling-up Fallacy

Last month Justin Sandefur at CGD lamented the regrettable failure of the Kenyan government to sustain a successful school-based de-worming programme after donor funding was withdrawn due to corruption in the Education Ministry. This is another good example of the sustainability paradox: despite clear evidence that this programme was extremely cost-effective it was cut when the donor funding was withdrawn. I assume that this was as much a political act intended to hurt the donors – who lost something they cared about – but as such is clearly rather callous. But, more than anything, it is another example of the phenomenon that what the donors want and what the recipient country government want are often not the same thing.

This, however, is not exactly news around here. More interestingly Sandefur also suggests that this raises questions about “the feasibility of turning small NGO pilots into manageable national policies”, although he failed to elaborate much on that idea in the rest of his post. This is something I’ve been thinking about a bit recently, and I think there is an important additional argument to be made here.

Whether a pilot is being developed by an NGO or a bespoke, direct donor-funded project, it will have its own management structure. It will also have a significant investment of technical advice and support that is inevitably diluted when a project is transformed into a national programme. However, I can live with that; if we want aid to be cost efficient, then we need to be able to realise economies of scale on techniques that have been shown to work.*

My beef is with the management. Because, to the international aid industry, scaling up nearly always means launching a nationwide government programme. In doing so the donors discard the effective management that produced the initial successes in favour of a dysfunctional government bureaucracy. Not only do you lose some basic management nous, but you also lose the driving vision, the leadership that got the pilot project to where it did.

When Larry Page and Sergey Brin founded Google, they didn’t show some initial promise and then hand their genius idea over the government. Instead they secured some outside investment including big business management expertise (Eric Schmidt) – thus addressing their ‘absorptive capacity’ – and grew the company to the multinational search behemoth it is today. More to the point, Google isn’t just big; it continues to be incredibly successful.

I’ve blogged before (here and here) about the importance of the quality of management in delivering conservation and development results. The standard donor approach to scaling up suggests that donors remain stuck in a rut that emphasises technical barriers (leading to misdiagnoses of project failure) over management constraints, combined with the belief that all you need is a bit of capacity-building in profoundly dysfunctional institutions to turn it around.

The next time donors are seeking to scale up a successful programme, I hope they will remember the Google story, the Grameen Bank story, and the countless other examples of private sector efficacy in turning innovation into successful business models. After all, most donors are capitalist countries, not socialist ones, and there’s a reason that communism collapsed.

* There is another argument to be made here that many projects are scaled up before the jury has properly returned a verdict, leaving key issues still unresolved. But, conversely, if an approach does appear to be working, I can understand how funders, desperate for new solutions, may pile in prematurely.