Monitoring Our Assumptions
Mar 4, 2015
For many of the world’s poor, usable land is in short supply. In Africa, for example, more than 90 percent of rural land is undocumented and vulnerable to land-grabbing and expropriation. This leaves smallholder farmers and pastoralists—people trying to eke out an existence—vulnerable, especially in countries where land governance is weak.
At DAI, we manage Land Tenure Regularisation (LTR) projects designed to kick-start administrative changes that lead to surety of tenure. This surety enables the poor to buy, rent, trade, or receive land and brings a level of stability into their lives. But what happens when one of these projects, which typically last several years, runs adrift from plan early on?
DAI recently concluded projects in Rwanda and Mozambique for the U.K. Department for International Development (DFID) and Mozambique Millennium Challenge Account (MCA), respectively. I took it on myself to embrace and reveal, by way of this note and on my own time, some useful lessons from a mix of self and independent reviews of these two projects.
Support to Land Tenure Regularisation in Rwanda established a formal system and identified and registered more that 10.3 million land parcels. It won the 2014 British Expertise award for outstanding international development project. The other project, “Consultancy and Technical Assistance to the Mozambique MCA Land Component,” improved the efficiency of the land rights management and registration system in four provinces. To know what differences we made (or not), I set out to answer three interrelated questions:
- How good were we at delivering the aid project?
- How could we get better?
- What changes did we set in motion, among whom, and will these changes outlive the contract periods?
The theory goes that land surety will stimulate behaviors by farmers and other investors that lead to more productive land ownership and equitable economic growth. If the assumptions held, improved land administration and its resulting benefits would outlive our projects’ presence. But what if our assumptions did not hold? In gleaning information from independent reviews, reports, and interviews, I confirmed that the veracity of our assumptions was more important to the success of these projects than indicators, and monitoring as important as evaluation.
Here are some of the lessons learned in Rwanda and Mozambique, lessons that would help inform DAI’s ongoing land tenure work in Liberia, Peru, Ethiopia and elsewhere:
Critical Success Factors
Tell communities about their rights again and again—A dedicated, systematic, and ongoing outreach program improves awareness of laws and rights, how citizens can secure them, and who is responsible for providing services.
Build consensus from the start—Anticipate establishing and carrying out ongoing consensus-building activities. Include civil society groups and local and national officials from the outset.
Keep your approach inclusive—In Rwanda, for example, we recruited and trained parasurveyors from the villages where LTR was operating; in Mozambique, we established a governance-cum-oversight structure that can settle disputes and hold local government to account. Ensure back-office systems and skills—Strong finance and administrative skills proved valuable on both projects as our clients demanded ever more detail.
Follow your instincts but be flexible—We increased the level of detail in policy and associated regulations after finding confusion regarding responsibilities and technical system requirements.
Find well-placed champions—The commitment of the Rwandan government bolstered by an exceptional director general helped facilitate our work, as did the MCA in Mozambique, which helped overcome an ambivalent national government.
Things To Improve On Next Time
Clarify roles and responsibilities early—Negotiating expectations and clarifying who is responsible for what is equally important as negotiating financial concerns. The advent of results-based payments—which require pre-agreement of results and independent verification, among other concerns—increases the exactitude needed to the agreed-upon services and products as well as changes to work and goals and among whom, where, and, most importantly, by when these changes will occur. This is especially important for projects that promise to build the capacity of government systems and institutions while simultaneously making a difference to the lives and livelihoods of its citizens (the government’s clients).
Improve understanding and management of logframes—Incoherence within the Rwanda logframe remained unresolved, and though it proved to be an award-winning project, our rickety logframe challenged DFID Rwanda’s technical advisers and caused misunderstandings among stakeholders over what defined the project’s end-of-contract status.
Instill discipline and balance in monitoring—Blindly tracking ex ante indicators does not constitute effective monitoring. The monitoring process must clearly align with the logframe, including monitoring of the assumptions on which workplans are based. Too often, a management information system and personnel, to include the “monitoring and evaluation” function, do not effectively integrate financial and nonfinancial performance. DAI is less responsible than our clients for evaluating our projects. Hence, we need to lose the ubiquitous phrase “monitoring and evaluation” and concentrate on management-focused monitoring processes that answer three questions:
- On Effort. Where are we spending money and effort, and how inclusive are we, using basic principles of activity-based costing?
- On Outreach. How do beneficiaries initially respond to and rate our products and services, and why does this vary by place and people?
- On Change. How are beneficiaries and others responding to our assistance over time, and why does this vary among places and people? Answering these questions deliver results that matter to managers as they lead their projects over time.
In the end, neither the Rwanda nor Mozambique project could well articulate how they changed their beneficiaries’ lives (the investment period was too short) or host-nation government systems (there was limited interest), but the prospects for both in realizing benefits look good given the legacy both laid down.
Because of the relatively short investment periods of aid projects, it is impossible to predict how things will play out in the long run. Such institutional and personal behaviors defy prediction. But for projects such as the one in Rwanda (delivering titles to their citizens on behalf of the government) or the one in Mozambique (to transform institutions), good practice in monitoring assigns more importance to the periodic testing of assumptions and less to the measuring of pre-defined, ex ante indicators of behavioral change.