Gauging the impact of development work presents many challenges, not least the one of attribution. It is difficult to demonstrate that particular programmes result in significant and lasting changes in the well-being of intended beneficiaries, when a range of agencies and external factors are involved. No single agency can realistically claim full credit for observed social changes, something that is equally the case for communication and information work in development. This key list provides key resources that address the challenges of assessing the impact of development interventions. There is a focus on the impact of information products, services and projects. An additional selection of materials looks at development impact from a 'network' evaluation perspective - by tracking changes in relationships over time, and comparing this to what might be expected from a project's objectives.
This list of key resources was compiled for the Exchange Lunchtime Discussion on The challenge of impact assessment in complex development situations. Source welcomes details of additional resources and accounts of how they are useful - please send these to firstname.lastname@example.org.
This book is the result of the Evaluating Capacity Development (ECD) Project, begun by ISNAR in January 2000, which aimed to improve capacity development efforts in research and development organizations through the use of evaluation. It explains how the project used an action-learning approach, bringing together people from various countries and different types of organisations. Six evaluation studies were conducted over the course of three years: exploring capacity development in a rural development NGO in Bangladesh; towards strategic management in a Cuban agricultural research institute; understanding capacity development in a plant genetic resources centre in Ghana; assessing organisational change in an agricultural faculty in Nicaragua; strengthening participatory research capacities in a Philippines root crops research centre; and expanding capacities in a rural development institute in Vietnam. Chapter 1 provides background reading on the ECD project that gave rise to the book. Chapter 2 discusses basic concepts of organisational capacity, capacity development and evaluation. Chapter 3 addresses two fundamental issues: why managers should be concerned with organisational capacity development and why they should evaluate capacity development efforts. Chapter 4 discusses issues related to the 'how' of capacity development. Chapter 5 discusses partnerships for capacity development while 6 outlines approaches and methods for evaluating organisational capacity development. Chapter 7 discusses how to utilize evaluation processes and results to advance capacity development and performance in an organisation
This is a letter written to managers of information projects. Its aim is to provide the reader with a series of guidelines, checklists and practical suggestions for carrying out an evaluation of an information project
A useful handbook looking at the basics of monitoring and evaluation through the examples of SCF projects internationally. Deals with all stages of planning monitoring and evaluation for project design and in particular participatory methods, which it identifies as most appropriate to interventions with young people
This report documents an initiative that, through its methods and focus, aimed to provide a realistic starting point to understanding the impact of information on development. It consisted of a electronic conference, followed by a workshop through which ideas about information and evaluation were shared and built upon. The report weaves together the input of dozens of information users and providers, policymakers, information scientists, and others from the South and the North, and presents a preliminary assessment framework as a starting point to future work in this area
This scoping study has attempted to identify and document how various agencies and institutions have approached the assessment of advocacy. It sets out a number of frameworks that look at similar issues from different perspectives and, instead of promoting one framework as the 'correct' one, allows the reader to pick and choose what elements are most useful to them. The work was limited in scale, and focused in particular upon the approaches of NGOs. The insights and ideas from this study will contribute to a three-year action research project to be undertaken by ActionAid and partners in Africa, Asia and Latin America.
This paper argues for the use of a network perspective in representing and evaluating aid interventions. Commonly used linear evaluation tools such as the logical framework (logframe) or problem tree selectively represent what an agency is trying to do, and then assess particular aspects of the intervention. Social network analysis describes social relationships which, the author contends, is what development is about. The wide range of methods and theories for network analysis means that there are lots of ways of thinking about and describing expected outcomes of interventions. Social network analysis is also very flexible in terms of the scale of the project, and can accomodate non-linear processes of change. This paper presents these five arguments for social network analysis, and then describes next steps for developing a coherent approach to evaluation based on a network perspective
Assessing the impact of the efforts of a single external development agency is problematic because changes in the well-being of beneficiaries occur as a result of a ‘confluence of events’. Outcome mapping is a dynamic methodology that looks at the contribution an agency has made at project, programme or organisational level to influence the processes leading to changes in the behaviour, relationships, actions and activities of people and organisations. According to the authors, development is about people relating to each other and their environment. Outcome mapping is therefore a monitoring and evaluation system for current and completed activities and provides a framework and vocabulary for understanding changes and assessing efforts. It is based on principles of participation, iterative learning and evaluative thinking throughout.
This book includes a thorough explanation of the outcome mapping approach, and provides detailed information on workshop design and facilitation, as well as numerous worksheets and examples
Gives an overview discussion of the key characteristics of networks, noting that international development increasingly takes a network form. Networks typically put an emphasis on: facilitative leadership; building relationships and trust; light co-ordinating structure that allows decentralisation, autonomy and voluntary participation. However, these are charactersitics that traditional evaluation approaches have not been developed to address. Provides a useful check-list for evaluating networks and suggests some useful practical tools to approach the evaluation of networks, such as: ‘Contributions assessment’ to guage how effectively the network facilitates the circulation of resources, and enables people to make the contribution that they are capable of; ‘Clarification of aims and activities’ (adapted Weaver’s Triangle) tool to clarify how participants perspectives and activities diverge or converge over time; ‘Channels of Participation’ tool to assess how and were members interact in a network and look at changes over time; ‘Monitoring the edges’ to track independent networking stimulated but not through the centre
Presents the findings of a research project that focused on the information needs of the urban poor, and the information sources they access. The study found that the main sources of information were social networks, key informants, and infomediaries. It considers the role of information and communication technologies (ICTs), and impact assessment methods. It ends with suggestions for development agencies seeking to share their knowledge and information with the urban poor
This field-tested toolkit has been designed to measure the extent to which programmes make a difference. The 2003 edition of Toolkits has been extended with contributions from SCF and beyond. It describes participatory methodologies, such as mapping and focus groups. It is divided into three sections: underlying principles, practical questions and tools. This new edition brings these up to date and discusses the implications of adopting a human rights approach to development and the increased emphasis on partnership. There are new chapters on impact assessment, monitoring and evaluating advocacy, as well as two new tools - one for improving planning, evaluation, and impact assessment and one for stakeholder analysis
The Learning and Evaluation Action Program (LEAP) is an initiative to promote a coordinated and comprehensive approach to learning about the use of knowledge and ICTs in international development. It hosts an e-mail discussion list called LEAP-Impact, a community of practice open to all individuals and organisations interested in the evaluation of information services, projects and products. The website includes posted messages, documents and links to further resources on knowledge management, learning and evaluation for development
Outlines the principles of 'Outcome Mapping' a method developed by IDRC to assess impact of development work. The approach recognises that the impact of an organisations work is difficult to disentangle from the range of other factors influencing development in a particular place. It focuses on the effect the development project or organisation has on the behaviour, relationships, activities and actions of key people and partners it works with, or that are within its sphere of influence. It begins with a workshop to design a programme and monitoring system, followed by a series of self-assessment workshops to monitor change and refine strategies
A gateway to methods, tools and manuals, indicators, case studies, discussion lists and bibliographies relevant to participatory monitoring and evaluation. Includes some background to the issues, with links to relevant full-text documents and websites
This valuable service includes an electronic discussion forum, numerous papers and resources on evaluation, news and upcoming events, and links to other evaluation resources and practitioners. There is a special section on network evaluation and the use of social analysis in development projects