Independent Magazine - Issue 8, 2024

EVALUATION

20 YEARS OF INDEPENDENCE

IFAD

Fine-tuning

It is a special privilege to be at the helm of an esteemed office at the historic juncture of its 20th anniversary. In order to look ahead we must look backwards. IOE has been built on the efforts and commitments of many stakeholders, key being the Executive Board of IFAD, which insisted that the evaluation function be made independent. The formative leadership of this endeavour had to translate a vision into practice, and confront the resistance that is spawned from the establishment of any independent evaluation office.

This edition has historic lessons for any evaluation function that seeks to establish and entrench independence, which is contested along many lines. The interviews of the founding Directors, Luciano Lavizzari and Oscar Garcia, reflect on specific challenges and opportunities which continue to be applicable in the global evaluation community. Building a reputation moves beyond passing a policy that enshrines the evaluation triad of independence, credibility and utility. It requires delivering products that are generated from such principles which improve organizations. The independent function must be structurally and operationally independent to cast judgement. This is only possible if the head has behavioural independence and resources to do so. Fabrizio Felloni, current Deputy Director, complements these perspectives, sharing with us his historic memory of IFAD’s journey towards an independent evaluation function.

The unwavering support of the IFAD Executive Board, to which IOE reports directly and is directly accountable to, has been a consistent factor that has helped cement the role and grow the status of our office. The words of Ms Michelle Winthrop, Ambassador of Ireland to the Republic of Korea, speak volumes in this regard. In her interview, the former Evaluation Committee member makes a clear call that the credibility of IFAD is enhanced by partners having faith in the validity of IOE products.

Independence is a principle, and should not be confused with isolation. Work captured in this edition shows some golden threads in IFAD evaluative practice. The first is foresight from evaluative hindsight, and the design of a workplan that learns from the past for a better IFAD. The

In my own words

multi-year evaluation plan, which is broadly consulted with the governing bodies and Management, serves this purpose. The second is persistent engagement and building the context for empowering dialogues. IOE has delivered a global evaluation first in its neuroscience approach and module to communicate more effectively with stakeholders. The third is customising products to audiences, and making evaluative work accessible. The fourth is selecting evaluative work that helps management to course-correct. The fifth is generating evaluative transparency, which we do through our independent website and suite of products that add organizational credibility. Finally, we must look at the evaluation function which is beyond IOE, including self-evaluation done by Management.

This is a process to build a reflective evaluation culture within IFAD, as it seeks to help millions around the world. Our first EAP has helped us, and gratitude goes to the member states who value evaluation not as a stand-aside, but also a foundational support to IFAD as it strives to achieve its mandate. The magazine shows this in the positive response to the evaluation of decentralization.

Once again, the flagship IOE Annual Report of the Independent Evaluation of IFAD provides a big-picture view of performance around key indicators. The IOE-sponsored book on policy evaluation in the era of COVID-19 stimulates global debates on our function in a changing world, as we look inwards to remain relevant and impactful. We continue to engage internationally, most recently with the University of Arizona and the European Institute of Innovation and Sustainability, feeding IOE practice into the academic area.

I do hope you find the magazine engaging and thank you for your support to IOE.

Independent Office of Evaluation of IFAD

INDEPENDENT OFFICE OF EVALUATION

IOE PILLARS

MISSION

Our mission is to promote accountability and learning through independent, credible and useful evaluations of IFAD's work.

GUIDING PRINCIPLES

Accountability, learning, independence and partnership are the principles that guide our work

HISTORY

The IFAD’s evaluation function has evolved over more than 40 years of existence, from being discharged by a unit internal to the IFAD management into a fully-fledged independent evaluation outfit.

QUALITY ASSURANCE

We have supported two external peer reviews of IFAD’s evaluation function conducted by independent professionals in the field of evaluation (2009 and 2019).

1978

Monitoring and Evaluation Unit

Monitoring and Evaluation Unit

04 1982

02

1994

Office of Evaluation and Studies

Fabrizio Felloni Acting Director | 2020-2021

Kees Tuinenburg Officer in Charge| 2013-2014

Ashwani Muthoo Acting Director | 2012-2013

Luciano Lavizzari

Director

1999-2012

FORGE CORPORATE CULTURE

Contribute to forging IFAD’s corporate culture as a transparent, learningoriented and accountable organization.

IMPROVE COVERAGE

03

IOE JOURNEY

2011 Independent Office of Evaluation

Office of Evaluation

05

2003

IOE DIRECTORS

Oscar A. Garcia Director

2014-2020

Indran A. Naidoo Director

2021

BUILD EVALUATION DIALOGUES

Engage with Management, Member States and external partners to support evaluation capacity and use within and outside IFAD.

Improve evaluation coverage and promote transformative evaluations that reflect the scale and scope of IFAD operations.

RETAIN & DEEPEN LEADERSHIP

Retain and deepen IOE’s position as an internationally recognized leader in the evaluation of rural development programmes, policies and strategies.

EVALUATION OF IFAD (IOE)

IOE EVALUATION PRODUCTS

[click on icons to access database]

Aggregation level

Sub-regional Evaluations

Primary objective

Assess strategy, common intervention approaches and IFAD organizational set up in a set of countries that share salient characteristics.

Main users

Regional and country director(s), technical advisors, operational staff, and government counterparts.

Aggregation level

Project Completion Report

Validations

Primary objective

Validate the project completion reports prepared by IFAD Management.

Main users

IOE and IFAD Management for reporting (ARIE and RIDE) and feedback.

Evaluation Synthesis

Primary objective

Contribute to knowledge generation by consolidating findings from past evaluations.

Main users

Senior management, Directors, staff of regional and technical divisions, and members of governing bodies.

Thematic Evaluations

Primary objective

Provide evidence of development effectiveness, performance and results of operations in a thematic topic.

Main users

Senior management, Directors, staff of regional and technical divisions, and members of governing bodies.

Corporate Level Evaluations

Primary objective

Assess the organizational performance and institutional effectiveness of IFAD.

Main users

Senior management, Directors, staff of regional and technical divisions, and members of governing bodies.

Aggregation level

Annual Report of Independent Evaluation

Primary objective

Report all of IOE’s evaluation activities in a given year, and presents a synthesis of IFAD’s performance, lessons and challenges.

Main users

Senior management, Directors, staff of regional and technical divisions, and members of governing bodies.

Aggregation level

Project Performance Evaluation

Primary objective

Assess the performance and results of projectlevel operations funded by IFAD.

Main users

Regional and country director(s), technical advisors, operational staff, and government counterparts.

Impact Evaluation

Primary objective

Provide a rigorous quantitative assessment of the impact on rural poverty of selected IFAD’s operations.

Main users

Regional and country director(s), technical advisors, operational staff, and government counterparts.

Project Cluster Evaluations

Primary objective

Assess the experience of several projects that have a common theme or common major component.

Main users

Regional and country director(s), technical advisors, operational staff, and government counterparts.

Country Strategy and Programme Evaluation

Primary objective

Assess performance and results of country strategy and operations and provide lessons and recommendations to guide preparation of next country strategy

Main users

Divisional and country director, country team, and government.

INDEPENDENCE

EVIDENCE-BASED MANAGEMENT

Guarantees the avoidance of conflicts of interest by upholding the provisions that safeguard the behavioral, organizational and structural independence of the Office.

Reflects IFAD’s increasing focus on embracing a culture of evidence-based management to maximize development effectiveness, in which evaluation has a critical role to play

Clarifies that accountability and learning are objectives of evaluation, and emphasizes the effective use of evaluation products to this end

Presents the roles and responsibilities of the Evaluation Committee and, of its Chair, in the evaluation function on behalf of the Executive Board

Editorial Board

Writing

Graphic design Publishing

Profile

Proofreading

Revisioning

Independent Magazine brings to the forefront of the global development dialogue the major efforts undertaken by the Independent Office of Evaluation of IFAD, while seeking to advance the organization’s vision of vibrant, inclusive and sustainable rural economies, where people live free from poverty and hunger. To present the richness of rural life, and detail facets of local community lifestyle, Independent Magazine also zooms in on cultural activities and landmark occurrences in countries featured by IOE’s evaluations.

DIFAD LEADERSHIP ACTS UPON RECOMMENDATIONS ON DECENTRALIZATION TO ACHIEVE

ecentralization done right is essential to improve results achieved on the ground. To this end, the recent corporate-level evaluation (CLE) of the Fund’s decentralization experience carried out by the Independent Office of Evaluation (IOE) recommended more strategic planning, adequate resource allocation and fit-for-purpose country presence. IFAD Senior Management has taken immediate and far-reaching action to address each and every recommendation. Management presented these important new developments in detail during a learning event organized to discuss the CLE, on 7 November 2023.

Organized by IOE, the event was held in a hybrid modality with in-person participation of staff at IFAD headquarters and online participation of over one hundred and twenty-five field staff world-wide. The content was managed by IOE with active participation of IFAD Management. A very high number of senior officials partici-

pated and took the floor, including four AVPs, namely Donal Brown, AVP Programme Management Department; Jyotsna Puri, AVP Strategy and Knowledge Department; Guoqi Wu, AVP Corporate Services Department; and Alvarado Hernan, AVP, Chief Financial Officer and Chief Controller, Financial Operations Department.

The event aimed to promote better understanding across IFAD of the key evaluation findings and recommendations, and organization-wide discussions around how the evaluation informs the ongoing and future efforts to strengthen decentralization. The workshop featured two main sessions. During the first, the IFAD AVPs presented the key changes to the decentralization approach introduced in response to this evaluation. During the second, the IOD Director moderated a panel discussion around the steps to strengthen the value-added by IFAD field presence, the efforts to improve adaptive management and course correction, and how

RECOMMENDATIONS OF EVALUATION ACHIEVE RESULTS ON THE GROUND

the well-being of staff is being addressed in response to the evaluation recommendations.

Decentralization is widely recognized at the Executive Board, Management and staff levels as a necessary step to improve development results on the ground. Country presence brings IFAD closer to beneficiaries and governments, enhances partnerships with other development actors, and improves IFAD’s relevance and development effectiveness. However, the evaluation found that increasing outposted staff does not automatically translate into better support to development effectiveness. More to the point, the decentralization process was top down, not fully responsive to staff concerns and not adequately informed by the decentralization experiences of other international financial institutions and United Nations agencies as well as IFAD’s own earlier decentralization efforts.

“Inadequate time was available for effective

“I appreciate the concrete actions that are being taken by IFAD leadership. This demonstrates the strong commitment of Senior Management to addressing the issues captured by IOE’s evaluation report”

adaptive management and learning. Actions did not adequately reflect the lessons from earlier decentralization experiences”, noted Dr S. Nanthikesan, IOE Lead Evaluation Officer, and lead author of the CLE, who helped conceptualize

and organize this learning event.

Dr Nanthikesan also noted that the findings, conclusions and recommendations of the CLE were based on inputs received from a range of sources including an econometric analysis of all 588 projects completed by IFAD since 2003, as well as 15 country case studies (Bangladesh, Brazil, Burkina Faso, Cambodia, Côte d’Ivoire, Cuba, Djibouti, Egypt, Eritrea, Ethiopia, Kenya, Niger, Panama, Sudan, and Viet Nam). Inputs were also received from 686 interviews with IFAD stakeholders in client countries, 807 online survey responses, and a comparative study of the decentralization experience of 8 other international financial institutions and UN agencies.

To ensure the existing momentum of the decentralization efforts continues, the Fund’s Senior Management has decided to immediately address the shortcomings of IFAD’s decentralization process identified by the evaluation with concrete actions.

Speaking at the opening of the event, Dr Brown provided an overview of how IFAD has responded to the specific evaluation recommendations. These include, among several others, increasing the budget for country programme delivery, increasing capacity building of country directors and operation staff, adopting measures to improve staff well-being and reducing vacancy rates, revising the reassignment exercise timelines to improve both personal and business planning, and taking proactive steps to provide the Executive Board with comprehensive updates on progress being made vis-à-vis IOE’s recommendations.

Dr Puri complemented Dr Brown’s remarks by providing insights into the strategic implications of decentralization changes in the organization. In particular, she highlighted that IFAD has developed a ‘decentralization recalibration plan’, which is already under implementation, and which is largely based on CLE recommendations.

The CLE brought to the fore many limitations related to human resource management, including the high vacancy rates and ensuing unsustainable workload for existing staff. Dr Wu addressed these over the course of the event.

“The plan maintains the decentralization commitment. Key elements are review and further refinement of the global, technical anchor locations. It also includes comprehensive transition plans for HQ-based general service staff, and addresses the number of country offices”

- Dr Donal Brown

Regarding recruitment, the AVP explained that, since IFAD President Alvaro Lario took office in October 2022, the Fund has developed an action plan to reduce the high level of vacancy rates. One year later, the rate has fallen from 17%, at the beginning of 2023, to 11.4% in September 2023. Vacant positions in the field have witnessed a similar trend, with a drop in the ratio from 21.9% to 13%.

“These actions represent our commitment to improving our decentralization approach, and to addressing the recommendations from the recent evaluation because we, as Management, want to do this right. We appreciate your support and cooperation”

- Dr Jyotsna Puri

“With these lowered vacancy rates, we hope that IFAD will be better equipped to deliver in this challenging context. We are confident that we will be able to keep the vacancy rates even lower in the coming year“, Dr Wu affirmed.

Another topical issue that the CLE captured was that adaptive management was insufficiently integrated into the decentralization process. The overly ambitious timeline and targets limited the opportunities to course correc. Taking the floor to address these challenges, Mr Hernan clarified that IFAD finds itself in a dynamic phase, in which adaptive management has now started to play an important role and outlined several actions aimed at enhancing the Fund’s decentralization approach.

“Today, IFAD is delegating more decision-making authority to empower its global staff, including country directors and national officers. This delegation underscores our commitment to a more responsive and efficient organizational structure. IFAD is taking important steps to enhance monitoring of decentralized operations, and providing necessary support to standardizing business processes to reduce risks of error and fraud”, Mr Hernan highlighted.

The discussion panel that followed the first session with the AVPs further delved into the concrete steps outlined by IFAD’s leadership to address the CLE’s recommendations. Panel members were chosen to represent the key stakeholder units spearheading the decentralization efforts and featured Sara Mbago-Bhunu, Regional Director of the East and Southern Africa Division, which the first regional office to be decentralized; Juan Diego Ruiz Cumplido, Head of the Multi-Country Office in Peru; Juan Carlos Mendoza Casadiegos, Director of the Environment, Climate, Gender and Social Inclusion Division; Advit Nath, Director of the Financial

The learning event established this CLE as one of the most influential recent evaluations that has led to far-reaching management actions in a number of areas, ranging from human resource management and budget allocation to ensuring fit-for-purpose field presence. The event signalled the ongoing commitment of IFAD’s leadership to take onboard the findings and recommendations of the CLE report, and to strengthen the decentralization efforts under the current challenging circumstances.

Access report [here]

BUILDING INFRASTRUCTURE AND MARKET DEMANDS PROVES

IFAD’s integrated value chain approach has strengthened agro-processing and market linkages for select commodities in Uganda. Reduced transport costs and higher market prices due to improved road access, plus added value through bulking and processing, have contributed to higher household incomes. These findings relate to the Uganda Country

Strategy and Programme Evaluation (CSPE), which Dr Naidoo presented to a group of distinguished representatives of the IFAD Executive Board (EB) during a country visit to the Republic of Uganda.

Organized by the Office of the Secretary of IFAD, from 26 November to 2 December 2023, the visit enabled the EB representatives to witness IFAD’s

work on the ground, and observe first-hand the opportunities and challenges that the Fund faces in carrying out its mandate. Delegates included Lamia Ben Redouane, Permanent Representative of Algeria to the Rome-based UN agencies; Gerson Mateus Dos Santos Francisco, Third Secretary of the Embassy of Angola to Italy; Veronika BaumgartnerPutz, Senior Advisor of Austria

AND SUPPORT SERVICES AROUND

SUCCESSFUL IN UGANDA

to International Financial Institutions; Zeng Shiyang, Second Secretary of the Permanent Mission of the People’s Republic of China to the Romebased UN agencies; Lisbeth Jespersen, Minister Counsellor of The Royal Danish Embassy in Rome; Ronald Meyer, Alternate Permanent Representative of Germany to the International Organizations in Rome; Erma Rheindrayani,

Counsellor of the Embassy of Indonesia to Italy; Miguel Jorge García Winder, Permanent Representative of Mexico to the Rome-based UN agencies; Yaya O. Olaniran, Permanent Representative of Nigeria to the Rome-based UN agencies; Petter Nilsson, Deputy Permanent Representative of Sweden to IFAD; and Krisztina Bende, Permanent Representative of Switzerland to the

Rome-based UN agencies.

The mission promoted dialogue with state and local government officials regarding IFAD’s role in Uganda, allowed Board representatives to meet beneficiaries, and provided an insight into the importance of public policy as a means of promoting rural development. In addition, EB members also gained a better understanding

of the lessons emerging from the 2021 CSPE carried out by IOE. Dr Nadioo’s presentation offered an opportunity to discuss what the Government and IFAD have learned from this CSPE, and how they have used it to prepare a new strategy and to enhance the portfolio.

He explained that sufficient evidence indicates that IFAD-funded programmes have contributed alongside other factors to growing productivity and incomes, although some of these increases derive from the success of past investments and to better weather conditions in recent years. IFAD’s investments in rural finance have been effective in terms of outreach, building linkages between local savings and credit groups and service providers, and in terms of regulatory reforms. Sustainability now depends

on the ability of groups to pay for services, while apex organizations still face challenges. It is recommended that IFAD expand its effective value chain approach to other commodities with greater beneficiary outreach potential.

These findings were complemented by the information that the EB members gathered first-hand during their field visits to IFAD-supported operations in Nwoya, Gulu and Kalangala districts. During the visits, Board representatives held discussions

with beneficiaries, farmers’ groups, representatives of small and medium enterprises and local government authorities. In Nwoya District, the group met with representatives of KAMP Feed, an animal feed producer that sells to both distributors and directly to farmers, using local raw materials.

In Gulu District, meetings took place with beneficiaries of the Project for the Restoration of Livelihoods in the Northern Region (PRELNOR). The project aims to increase sustainable production, productivity and climate resilience of smallholder farmers and provide increased and profitable access to domestic and export markets. In Kalangala District, the EB members visited the National Oil Palm Project (NOPP) and the Vegetable Oil Development Project 2 (VODP2), including an on-site visit Wto

an oil palm mill. The projects aim at increasing the domestic production of vegetable oil and its byproducts, thus raising rural incomes for smallholder producers and ensuring the supply of affordable vegetable oil products to Ugandan consumers.

Following these on-the-ground visits, back in Kampala, the EB group and the IOE Director held discussions with high-level government authorities and key donors at the national level. Interactions included meetings with the Ministry of Agriculture Animal Industry and Fisheries, the Ministry of Finance and Economic Planning and Development, and the Ministry of Local Government. The group also engaged with the United Nations Resident Coordinator in Uganda, as well as with other UN Romebased agencies.

Looking ahead, the EB representatives will report on the country visit at a future session of the Board, and in doing so will provide informed guidance on strategic and operational matters to the rest of the EB to enhance their critical oversight and strategic role.

Uganda’s economy has grown strongly in the past 20 years, though it has slowed to 5.2 per cent in the past decade, while per capita growth has slowed to 1.6 per cent due to high population

growth. Agriculture provides just a quarter of national GDP but employs 72 per cent of the labour force. Imports of processed food and particularly vegetable oil remain high. Poverty has worsened in recent years with a fifth of Ugandans living below the poverty line, and inequality persists particularly in rural areas, the north, amongst women, youth and refugees.

IFAD projects continue 10-year positive performance in climate change adaptation

Downward trends in effectiveness and impact require closer review

Performance of IFAD projects in environment and natural resource management and climate change adaptation (ENRM and CCA) has continued to improve over the past 10 years. This is one of the main findings of the 2023 Annual Report on the Independent Evaluation of IFAD (ARIE), published by the Independent Office of Evaluation of IFAD (IOE).

The 2023 ARIE presents a comprehensive account of the evaluation activities undertaken by IOE, by analysing the performance ratings from 288 project-level evaluations and 45 country strategy and programme evaluations. It also draws from the findings of corporate-level and thematic evaluations, as well as project cluster evaluations, a new IOE product. The report analyses the evaluative evidence pertaining to three thematic areas: rural enterprise development, rural extension and advisory services, and institutional efficiency of IFAD. These themes were selected based on their relevance to strengthening the organizational and development performance of IFAD. The ARIE also includes a

Access

trend analysis of performance ratings for projects completed between 2012 and 2021. The report summarizes the useful lessons emerging from these analyses.

considerations in all its interventions, and providing necessary guidance.

For instance, IFAD declared environment and climate change adaptation as a corporate priority in 2010; committed an increasing share of its programme of loans and grants to finance climate interventions; launched the Adaptation for Smallholder Agricultural Programme in 2012 to learn lessons and help mainstream climate change adaptation across all IFAD’s operations and country strategic opportunities programmes; mandated the Social, Environment, and Climate Assessment Procedures in all projects since 2015; instituted a dedicated unit with capacities to support mainstreaming themes including climate change and environment in 2018; developed the new IFAD Strategy and Action Plan on Environment and Climate Change (2019-2025) and continued to develop mechanisms and tools to guide climate responses, such as the Adaptation Framework (2020).

The ARIE finds that relevance (93 per cent) and innovation (87 per cent) are the other two

criteria that have the largest shares of projects performing well, out of the 71 evaluated between 2019 and 2021.

From a geographical perspective, the 10-year average of project performance ratings shows that Asia and the Pacific has the largest share of projects performing well, while West and Central Africa has the smallest share in all criteria. In this regard, it must be noted that factors beyond the control of any IFAD country presence – for instance, the political, institutional and development context in which projects operate and the implementation capacity and ownership of projects by government counterparts – heavily influence project performance.

The ARIE 2023 also presents downward trends, which may warrant further analysis. The most notable refers to the rural poverty impact and effectiveness criteria, which were not rated as ‘highly satisfactory’ across any of the 288 projects completed during 20122021. Moreover, effectiveness and rural poverty impact performance have declined since 2018. The share of effective projects fell from 83 per cent during 2017-2019, to 75 per cent during 2019-2021. For rural poverty impact, the share of well-performing projects fell from 89 per cent during 2012-2014 to 79 per cent during 2015-2017, plateaued and again fell, beginning in 20172019, ending at 72 per cent during 2019-2021.

This decline in effectiveness performance was more no -

ticeable under conditions of fragility and conflict, where the share of well-performing projects fell from 67 per cent to 42 per cent during this period. In this regard, projects in non-fragile contexts unambiguously outperformed those in fragile contexts in effectiveness, efficiency and sustainability of benefits. This analysis compared the performance of the 73 projects that operated under conditions of fragility and the 215 completed projects during the period that did not.

Based on these analyses, the report notes that IFAD

has underinvested in country programme delivery at a time when project sizes and complexity were increasing. Though IFAD has taken steps to reverse this decline, the report highlights the need for IFAD to track the longer-term impact of underinvestment practices in country programme delivery.

The assessment of how fit-forpurpose IFAD Country Offices were to deliver on their mandate following the accelerated decentralization efforts and reassignment exercises undertaken since 2017 needs closer scrutiny.

LEVERAGING SCIENCE TO ENSURE EFFECTIVE COMMUNICATION OF EVALUATION FINDINGS

The ability to effectively communicate findings can make or break the impact of an evaluation report. However, achieving successful communication requires an unintuitively unique set of skills, which has little or nothing to do with ‘mincing words’. The Independent Office of Evaluation of IFAD (IOE) has recognized these skills to be nested in the seemingly intricate realm of brain science, and found the key to navigating said realm in the person of Dr Srini Pillay. On 7-8 November 2023, he hosted a series of individual coaching sessions for IOE members and select IFAD staff at the Fund’s headquarters, in Rome. The tailor-made trainings equipped the team with nothing short of a mindset shift.

For over a year, IOE has worked alongside Dr Pillay to produce a flurry of products aimed at raising awareness regarding the added value of embracing synergies between the disciplines of neuroscience and evaluation. Firmly rooted in the conviction that by applying brain science-based principles into their daily work evaluators can improve the impact of findings and recommendations, IOE has provided practitioners world-wide with the concrete tools and resources necessary to embrace this synergy – a first in the history of the evaluation landscape. To culminate this year-long campaign, IOE decided follow the

well-known saying “be the change you want to see in the world”, and initiated the evaluation mindset shift in-house. This involved a collective team training session, which was followed by a series of individual coaching sessions

The two-day series of one-on-one clinics on ‘communicating evaluation findings’ afforded evaluators the unprecedented opportunity to engage directly with the neuroscientist, in a safe and confidential environment. IOE members brought to the table numerous communication-related issues that affect their evaluative work. For each issue, Dr Pillay provided practical advice and very tangible solutions that addressed the specific needs.

The trainings moved from the premise that there is a need to breakdown three myths that permeate the evaluation world, namely: evaluators always deliver objective findings; evaluands listen to these findings and understand them; and evaluands act upon said findings. To unpack these myths, it is necessary to recognize that human communication is largely inferential and unconscious. Moreover, between 55% and 65% of communication is nonverbal, and is largely affected by the juxtaposition between feelings of dominance versus submission, and affection vs. hostility.

Against this backdrop, the session emphasized the importance of understanding and applying seven new brain-based goals. These include first, preventing habitual thinking by creating a warm and non-stressful environment to facilitate change. Second, balancign facts and figures with a more open discussion by encouraging genuine curiosity about an ideal solution. Third, synchronizing intentions in order to be perceived as a leader and achieve mutual understanding, cooperation, coordinated execution of tasks and collective creativity. Fourth, delivering informative feedback by discussing potential reasons why something went wrong and encourage mutual problem-solving around this. Fifth, spending less time on what should not be done and focus the conversation on what should be done instead. Sixth, encouraging intrinsic motivation by encouraging self-discovery and individualized approaches to solutions, as well as by emphasizing the competence that you see in the evaluand. Seventh, facilitatign focus and attention by overcoming the potential fear, anger and cynicism of evaluands. To achieve this, evaluators should break down any complicated problem and focus on solutions, helping the evaluand to look past any mistakes after having learned from them.

Dr Pillay is a world-renowned Harvard-trained psychiatrist, brain researcher, keynote speaker, lecturer, author and consultant. He is known for combining “head and heart” (figuratively and literally) in an approach to personal development and goal mastery that blends science, spirituality, and horns-grabbing joie de vivre to combat the stresses faced by ambitious and high-achieving people in academia, business, and life. He has been voted one of the Top 20 movers and shakers in leadership development in the world.

Access

Access

Independent evaluation makes IFAD a more credible institution

“It is IFAD, not just IOE, that becomes a more credible institution, with an independent evaluation”, explained Dr. Naidoo in his opening statement during an event organized to celebrate the 20th anniversary of the independence of IOE. IFAD was the first UN agency with an independent evaluation office and one of the first in the international financial institutions to have a fully-fledged evaluation outfit.

Held at IFAD headquarters,

on 6 October 2023, the commemorative gathering saw the active involvement of a host of high-level attendees. Participants featured the distinguished representatives of the IFAD Evaluation Committee (EC), including Ms Sandra Paola Ramirez Valenzuela, First Secretary, Mulitlateral Affairs, Rome-based UN agencies, Mexico, and Dr Yaya Adisa Olaitan Olaniran, Permanent Representative of Nigeria to Rome-based UN agencies. Nigel Brett, Director of the Operational Policy and Re-

sults Division at IFAD, and Ms Claudia ten Have, Director of the Office of the Secretary and Secretary of IFAD, engaged in the event, as did members of IFAD Senior Management. IOE staff also partook in the celebrations.

“Our office has served IFAD well. Just to mention a few, IOE was instrumental in fostering major institutional changes of IFAD, such as direct supervision, field presence, decentralization and the reform of IFAD’s financial architecture.

At the operational level, we provide a wealth of analysis that can lead to far stronger country strategies and project design, if there is commitment to follow up”, highlighted the IOE Director.

Members of the EC praised IOE for the dedication with which it has upheld the value of independence over the past two decades, and congratulated the Office for the rigour of its outputs, which has seen a further surge in quality and quantity over the last two years. Delegates also applauded IFAD for putting

in place solid safeguards that protect the independence of IOE, and for diligently upholding them. Maintaining the independence construct that permeates the functioning of the Office is essential to ensuring the continued production of credible, high-quality and useful evaluations.

By the same token, Mr Brett and Ms ten Have were equally appreciative of IOE’s work, and spoke highly of the professionalism, commitment and integrity with which the Office pursues its mandate to promote both accountability

and learning at IFAD. To this end, IOE successfully balances its independence with the ability to engage fruitfully with all its stakeholders, including IFAD Management, throughout the evaluative process.

“Independence is not synonymous with isolation. We do not want to live in an ivory tower. Evaluation is most fruitful when it addresses real problems and helps solve them. We want to preserve independence of analysis and judgment but we want to engage with our stakeholders. Our evaluation process has many opportunities for them to signal their priorities and share their experience”, clarified the IOE Director.

The roots of IOE’s independence can be traced back to 2003, when the Executive Board (EB) approved the first IFAD Evaluation Policy. From that time, the Office reported to the EB directly, with the support of the EC. Independence means that the responsibility for the conduct of in-

dependent evaluations rests exclusively with IOE. It also implies that the Director and staff of IOE have the authority to communicate and interact directly with members of the Board, with in-country partners, and with others outside the Fund, as appropriate for the undertaking of evaluations. The Director has the authority to issue and publish final evaluation reports without prior clearance from anyone outside IOE. Furthermore, in light of the importance of the independence of IOE, the selection, appointment and possible dismissal of the Director is the sole responsibility of the EB.

Two decades since the historic first Evaluation Policy, today, more than ever, independence remains the unmovable bedrock of IOE, as the Office continues to grow and evolve, seeking to enhance its international reputation. Over recent years, IOE has succeeded in asserting itself as a key player in global networks such as the Evaluation Coop -

eration Group, the UN Evaluation Group and the Global Evaluation Initiative. The work delivered meets the highest international methodological standards, and produces reports with recommendations to optimize the impact of IFAD operations. In April 2021, all these reports became publicly accessible through IOE’s own website, in a move that reflects a radical shift of the Office’s approach to communication and outreach. IOE not only produces independent documents, but is now also able to independently manage the sharing and dissemination of evaluation findings in a continuously creative, engaging and user-friendly fashion.

In years to come, the focus of IOE’s renewed efforts will be on forging IFAD’s corporate culture as a transparent, learning-oriented and accountable organization; promoting transformative evaluations that reflect the scale and scope of IFAD operations; cementing evaluation dialogues with all stakeholders; and re-

taining and deepening IOE’s position as an internationally recognized leader in the evaluation of rural development programmes, policies and strategies.

STORY MORE THAN 20 years

THAN A JOURNEY

years of independence

Twenty years have passed since the independence of the evaluation function was enshrined in IFAD’s first evaluation policy. Since then, IOE has worked at the service of the rural poor, producing poignant reports and unwavering efforts that have helped IFAD to become an evermore credible institution. These undertakings have been instrumental in fostering major institutional changes across the Fund, such as direct supervision, field presence, decentralization and the reform of IFAD’s financial architecture, among others.

The IFAD Executive Board bestowed independence on what was known as the ‘Office of Evaluation’, back in 2003. This made IFAD the first UN agency with an independent office and one of the first of the international financial institutions to have a fully-fledged evaluation outfit. But the winding road that led to said independence started long before.

When IFAD began its operations in 1978, its evaluation function was vested in the recently created Monitoring and Evaluation Unit, which focused on monitoring the performance of the IFAD project portfolio. Initially, the Unit did not take part in full scale evaluations because IFAD’s projects were in the early stages of implementation. It was subsequently transformed into the Monitoring and Evaluation Division, reporting to the Assistant President, Economic Policy Department, an arrangement that lasted until 1994.

The evaluation and monitoring functions in IFAD were separated following the 1994 recommendations of the Rapid External Assessment of IFAD. The Office of Evaluation and Studies was created shortly afterwards. The assessment also recommended that the Director of the Office should report directly to the President and to the Board. Fast forward eight years, and the 2002 Consultation on the Sixth Replenishment of IFAD’s Resources urged IFAD Management to create an independent evaluation function because it would provide value and contribute lessons learned from evaluations, thereby enhancing learning and accountability. The rest, as they say, is history.

Today, IOE stands proudly among the most reputable evaluation outfits within the multilateral development institution world. The Office’s efforts and strides forward in leadership, strategy, guidance, research, and – most recently – communication and outreach have seen it surge to the rank of innovator and pioneer. Partner of the rural poor, key player in international fora, IOE has found a way to marry independence and engagement, escaping the misconceived notion that evaluation is synonymous with isolation, and shattering the false dichotomy between learning and accountability.

With ground breaking work ongoing in areas such as the application of neuroscience-based principles to evaluation, the future holds

great promise for IOE. There is much to look forward to. So, as we get ready to witness what the next 20 years have in store, the time is ripe to sit down and have a chat with those who made all this possible: former directors Luciano Lavizzari and Oscar A. Garcia, Deputy Director Fabrizio Felloni, and the captain currently steering the ship at IOE, Dr Indran A. Naidoo.

Good afternoon, esteemed colleagues.

Good afternoon, Alexander

During your time with IOE, you have witnessed the evolution of the independence of the office. In your opinion, what were the key elements that drove the impetus for independence back in 2003, and how has the evaluation policy evolved since then?

Indran

Since I joined IOE three years ago, I have been pleased to inherit and build upon the good leadership and work of former heads of this office: Luciano Lavizzari and Oscar Garcia, both of whom have provided messages to this special issue of Evaluation News. The result of a strong and independent evaluation function follows a challenging journey, as independent evaluation is not naturally accepted by potential evaluands. Starting this journey towards what we have today at IOE is the strong call from the Board, made in 2002, that IFAD develop an independent evaluation function. Several iterations of policies and reviews during this period have resulted in our fully independent and credible office that guides the Board in its deliberations, while providing real-time feedback to Management and Members States of their programmes. The consistent support by the Board, which selects the IOE Director, oversees the office and approves the workplan and budget, and reflects on Management responses to evaluations, is a key factor that has sustained and strengthened the independence of the office.

Over time, other aspects have also improved, such as a common understanding that evaluation is for the organization’s good, with the new policy delineating roles and responsibilities in independent and self-evaluation. Interventions by both IOE and Management in preparing the revised evaluation policy of 2021, and in a joint evaluation manual has helped strengthen mutual respect and collaboration.

The investment in more engaging evaluation processes, to institutionalize evaluation and mainstream it as part of everyone’s work, has yielded benefits, with joint learning and other collaborative events

helping to build evaluation culture at IFAD. Our IOE professional team (we use an IOE-led or IOE-conduct model, as opposed to a contract evaluation model) helps build internal capacity and knowledge, is cost-effective and helps mainstream the function and activities. The office has come a long way, with our first fully independent website, allowing us to communicate IFAD’s results globally.

IOE has been at the forefront of global evaluation thought leadership, attested in our global engagement on matters from evaluation policy, to strategy, methodology and engagement. The centrality of IOE can be seen in that most engagements around IFAD operations globally and nationally always reference IOE’s work. The strength of IFAD has been linked to it having a fully independent evaluation function, as recently reinforced by a positive rating of IOE by the Multilateral Organisation Performance Assessment Network (MOPAN) review. IOE gives assurance to donors and governments that their investments are under constant independent scrutiny, with IOE being obliged to report on results. Further attention can be seen in its regular policy reviews, the most recent having been completed in 2021. The last review addressed evaluation as an ‘all of IFAD’ responsibility, and included both independent and self-evaluation. IOE continues to evaluate not only the programmatic work of IFAD, but also the quality of all work that falls under self-evaluation.

In summary, a combination of clarity on evaluation brought about by the joint manual and a multi-year evaluation strategy has ensured evaluation relevance. We have advanced a more engaging evaluation process and have a global first in terms of a neuroscience-designed communication module in the manual. We also place strong emphasis on jointly planned knowledge events, at global and country level. I continue to recruit the best, as our IOE conduct model necessitates high internal capacity. We do not use a contract-evaluation model at IOE.

Fabrizio

In fact, I was here at IFAD when independence first came in 2003, and independence at IFAD came together with a request by the boards, not just of IFAD, but of many other international financial institutions, to allow more independence, more objectivity, and more professionalized evaluation units. In other words, IOE’s independence was not an isolated case. It was a request that the boards of many IFIs put out when these IFIs came either to replenishment or an increase of capital.

“IOE assures governments that their investments are under scrutiny, and is able to talk truth to power in a way that is unprecedented.”

- Dr Indran A. Naidoo

In 2003, the main concern was that evaluation should be independent of Management. The main features of the policy then were to clarify how we should be independent, what it means to be institutionally independent, to have a separate budget, to have a separate decision process for the programme, have the freedom to publish a report without the approval by Management or by any other external body, and also to manage the internal human resources. That was a main focus at the time.

Once the independence was established, then there were several evolutions of the policy. We had a new policy in 2011, and at that time, for example, there was a bit more mention of self-evaluation by management, although it was very limited . Then, when the latest version of the policy came in 2021, the new document followed the trend set by many other international financial institutions that tried to have a common corporate-level evaluation function and, within that function, differentiate the roles of independent evaluation and self-evaluation by Management. Independence was not questioned at all. The main concerns in 2021 were how are Management and independent evolution going to collaborate, and what are the synergies between self-evaluation and independent evaluation.

Oscar

IFAD, as the only international financial institution within the UN system, has a very important role to play to promote rural development. The Executive Board of IFAD is very much interested in fully exercising its role and discharging its oversight function, ensuring that the organization is accountable for the resources that it gets, which are public resources, and that it learns from its experiences to improve the quality of its programmes on the ground. Therefore, establishing the independence of IOE was a significant step in improving accountability and learning within the organization, and strengthening governance for the credibility and use of these resources towards donors and partners. The composition of the EB by lists, and the way in which they are represented in the Evaluation Committee, which is led by countries of list B and C, while the Audit Committee is led by countries of list A, is a very good arrangement for strengthening governance that promotes accountability.

IOE plays a very significant role in helping strengthen IFAD’s governance. There are different levels of accountability. One is towards the EB, and the other is towards the stakeholders at the ground level, including farmers and rural workers. This accountability is taken into account when the reports of IOE are available to the public and are fully disclosed, allowing for lessons to be taken at the ground level.

Luciano

I remember really well that the impetus for the independence of IOE came from the 2002 replenishment of IFAD resources, when the United States – which was the largest IFAD donor at the time – requested that the then Office of Evaluation be transformed into an independent evaluation outfit. The US saw an

independent office as being an instrument to refine the accountability of IFAD. In this context, it is worth recalling that at the time a number of donors were not overly convinced about the quality of the results achieved and the accuracy of the results presented by IFAD. The establishment of an independent office of evaluation was seen as an important step towards getting a better understanding of the performance and the value-for-money of the organization. This also explains why, a few months later, after the 2003 evaluation policy was approved, the EB entrusted the Director of IOE – which was me at that time – with the overall supervision of the independent external evaluation of IFAD. This was a very critical exercise, because it was the first of its kind. The objective was to review IFAD’s performance in the last decade.

The 2003 policy contains not only a statement of purposes and objectives, but also a large number of procedural and process details related to human resources, budgetary aspects and the process leading up to the yearly work programme. These details were deemed necessary because, at that time, the concept of an independent evaluation outfit was quite a novelty. Not everyone was in favour of it in IFAD, at the Management level but also among certain Board members and even within the Office of Evaluation. We soon realized that even a detailed policy was not sufficient and we needed to develop operational guidelines to implement the policy. These guidelines were developed painstakingly together with almost all IFAD divisions and approved by the President, one year later – that’s how long it took to anchor independence in the day-to-day operations.

In the subsequent years, the policy evolved to accommodate the changes in IFAD’s operating models, such as the introduction of direct supervision and country presence. Interesting to note is that these two fundamental changes were introduced in IFAD based on the recommendations of two IOE evaluations, which were among the first produced by the new independent office. A subsequent change that also occurred in the policy was the integration of the aspect of self-evaluation given that, in 2003, there was no formal self-evaluation system at all in IFAD.

Looking at the future, I think that the challenge for IFAD will be to be able to promptly enhance its evaluation policy to respond and adapt proactively to changes which will take place in the environment in which the Fund operates.

Based on your experience, how do you believe that IOE compares to other international financial institutions, multilateral development banks, and UN offices?

Indran

“Evaluation is people with people, people. In the IFAD, for people farmers and ducers in

We are in a position where we are both a UN agency and an international financial institution (IFI). I believe that IOE compares extremely well with other members of both our key networks, the ECG of the IFIs and the UNEG of the UN agencies. External reviews have confirmed this fact, and also that it is possible to be independent and address accountability obligations whilst also promoting learning as part of the evaluation process. There is no dichotomy between the two objectives, and conducting evaluations without the key principle of judgement of past performance, only credible through independent evaluation functions, is an issue of principle, and practice needs to catch up.

There are various reasons for this high performance, the key being that it is an office that is 20 years old, and as such has a long track record. Secondly, the Board has been very consistent in its support and affirmation of IOE’s independence, which is essential for the office to not become a spokesperson for Management. Thirdly, IOE is able to talk truth to power in a way that is unprecedented. Fourthly, we have our own dedicated website and retain full control of all aspects of our communication endeavours.

In addition, not only have we evolved in terms of the quality of our products, but we have also succeeded in entering the various international debates pertaining to the field of evaluation.

Fabrizio

I think that the concept of independence here at IFAD is very strong. Certainly, when we compare to the UN family, the concept of independence at IFAD and IFIs is much stronger and better embraced. It is clear that it means reporting to the Board, not to Management. Also, all the elements and the features of institutional and functional independence are clearer in IFIs, rather than in UN agencies.

is done by people, for the case of people means small proin the field.”

Within the IFIs, I would say that the evaluation policy of IFAD is a little bit more elaborate in a number of elements of institutional independence. Our policy, even the previous versions, were used as a model for other IFIs and evaluation offices to prepare their own evaluation policy, and even for the evaluation cooperation group of the multilateral development banks to prepare a paper, a sort of guideline on institutional independence in IFIs.

One aspect which has particular attention in our policy is human resources. Sometimes people ask us, why is it so important? The answer is that evolution in the end is done by people, even nowadays, with artificial intelligence, you still have people that conduct evaluations. The moment people think that there’s a chance that they might be subject to retaliation because of a critical evaluation, they will not write an evaluation in the same way, with the same objectivity, with the same rigour that

they do when they feel protected. The other reason is that it’s important to ensure that the evaluation staff have certain qualities. So, the analytical skills which are not really common in all the functions within an organization. Also, evaluation has become itself a profession or almost a discipline over the past 30 years, and therefore it also reflects the specificity of skills that are required for evaluation.

Oscar

I think, compared to other IFIs, IFAD has a very strong evaluation function that is know for its credibility and the rigour with which it conducts its evaluative work. It is also known for ensuring the independence of its work. Independence has many faces. First, in determining the programme of work, IOE follows a consultative process with Member States and Senior Management, and rolls out a multi-year programme of work. In this consultation, the office keeps its independence, having the last say in what it will and will not evaluate. This is a very important feature to exercise. Second, IOE has delegated authority to manage human and financial resources, without which independence could be undermined. Third, IOE has the ability to publish the reports unfiltered without any interference from Senior Management. The ability to reach out to the broader public and to the EB members with reports ensures that IOE is a stallion of independence.

In the case of IOE, independence is not associated with isolation. This is very important. The office has managed to strike a balance between recognizing the problems and challenges encountered by Management, on the one hand, and promoting a space for reflection where lessons can be shared to improve the quality of decision making, on the other. In this regard, it is important for the decision-making process to be participatory. Its not just about the evaluation report, but also the process that leads to the findings, conclusions and recommendations.

Luciano

I do believe that IOE compares quite well with other evaluation offices of both the IFIs and the UN. This was also confirmed by two peer reviews conducted by the Evaluation Cooperation Group of the multilateral development banks. In my view, IOE contributed to the evaluation functions of other development agencies, particularly concerning the aspect of independence. It is worth recalling IOE’s participation in the peer review of UNDP conducted by the DAC evaluation network in 2004. This was important because this exercise opened the door to the establishment of an independent evaluation function also at UNDP, which

happened a few years later.

The word ‘evaluation’ often conjures feelings of resistance. How would you address the cultural subtexts that hinder receptiveness to critical findings when they emerge from independent evaluation reports?

Indran

“The ability to reach out to the broader public and to the EB members with reports ensures that IOE is a stallion of independence.”

- Oscar A. Garcia

I have been the head of an evaluation function for over twenty-five years, and I must say that the word ‘evaluation’ continues to conjure forms of resistance, irrespective of where or by whom it is conducted. The resistance is even higher when you have an independent evaluation function. This is because there is a direct reporting line to the highest level of authority, such as the governing board, over Management. Many of the international evaluation offices are not structured this way, and report to Management. In those cases, levels of resistance will most likely not be so high.

To address the issue of resistance, I have always started by focusing on how we communicate and unpack the evaluation process. The words I use with my staff are ‘engage’, ‘listen’ and ‘exercise empathy’. We brought in an international expert on neuroscience, Dr Srini Pillay, to help us enhance and refine our approach to engagement. As a result, we are the first evaluation office in the world that has a module on brain science in the context of our communication work.

The second point is that we need to raise the bar and raise the conversation. This means recognizing that we are all working for the same organization. There needs to be a great amount of humility in recognizing that any policy or programme will, by the time that it is implemented, always have a discrepancy between what is intended and what happens on the ground. I always underscore that we are evaluating the programme not the individuals, that we are all working in the interest of IFAD, and that we are here to help the people on the ground. Giving a positive assessment of results when they are suboptimal would not help the citizens at that grassroots. If we take the optic that our clients are the intended beneficiaries across the world, then all these other factors related to hierarchy, attachment and personality become unimportant. I have been very clear and consistent on this.

In summary, we are certainly trying to increase our capacity to engage. By the same token, we remain very firm in terms of the principle whereby independent evaluation is in place to improve IFAD. This means testing and ensuring that results actually do happen on the ground.

Fabrizio

This is certainly true, and I think evaluation, the word evaluation in any culture, in any language is associated with a sort of tension somehow. In particular, for those that are being evaluated and for the programmes that are being evaluated. Interestingly, in our office, we are now developing a programme together with a specialist in neuroscience to address this problem. This means training ourselves in ways to better present evaluations, and then in the future, this training will also be extended to other users outside IOE, in IFAD, for example, as well as those outside IFAD, on how to conduct evaluation, but also how to accept evaluation in a way that entices collaboration.

I think that, at the end of the day, evaluation is done by people with people, for people, who are the big stakeholders. In the case of IFAD, for people means farmers and small producers in the field. Therefore, we

need to calibrate the way we express ourselves in a way that is culturally sensitive, so that we don’t offend people. We have to learn to listen. We need to have capacity to listen to what people are saying, what they mean, what is valuable to them. We need to have a strong sense of respect and understand that what we do is, in the end, for the benefit of the people in the field. It’s important that we create a culture of respect and that we communicate in a way that entices collaboration. There will always be some tension, but we can avoid doing too much damage.

Oscar

The culture of transparency, to me, is increasingly important in this inter connected world where information is out there. It’s by embracing a culture of transparency that you can take advantage of having a robust internal mechanism to do a systematic evaluation of your performance in a way that you have agreed upon. As the evaluand, you know what are the criteria for the evaluation, you know that the methodology is rigorous, and so you should take advantage of this opportunity for learning. This requires a level of maturity from the Management side, being able to be the subject of scrutiny without fear. It is through this culture of transparency and participation of stakeholders throughout the evaluation process that you can ensure that learning takes place.

Learning is really at the heart of the modern evaluation function. There has been a historic transition from accountability to learning. Its not a trade-off, it’s not an ‘either/ or’. Nowadays, the emphasis is tilting towards learning more than accountability, because the systems are very well established and respected. IOE must continue to leverage these opportunities, through direct engagement, but also by making full use of the ICT tools available. We did this in 2017, when we organized a conference to explore the use of ICT for the conduct of development evaluation. We found great opportunities for integrating several means for data validation and verification, including geo-spatial analysis of satellite imagery.

We also ventured into impact evaluation quite significantly, which is very important. IFAD knows who the farmers are, where they are located, and therefore can assess what was the situation before and after its intervention, and really try to establish the counter-factual evidence about what has been its contribution. Having the ability to assess the impact of interventions on the ground, in a credible manner, also strengthens accountability and learning. It is not about implementing a blanket approach. Impact evaluations need to be carefully applied to the situations where you want to learn more, because these are costly exercises. We ventured into some methodological advances for that. One of these was sensemaking, which was an approach that collected primary data within rural communities. This data was presented to the beneficiaries of IFAD’s programmes, who helped to signify the data through simple visualizing tools, such a diagrams. As a result, instead of being passive recipients of assistance, the beneficiaries became agents of change. This was a powerful way to demonstrate how a participatory approach leads to deeper and more transformative change, which is sometimes necessary to advance rural development.

Luciano

“The establishment of IOE was an important getting a better standing of the ue-for-money

Especially tough evaluations are likely to encounter resistance. This resistance would take the form of what I used to call the ‘denial – lamenting – acceptance – agreement’ process. This process was usually quite painful and conflictual, and not easy to avoid. It is, however, possible to minimize and reduce the intensity and duration of it. There are four considerations to consider in this regard.

First, concerning the evaluators themselves, they must be seen as honest and upright. There must be a full understanding and acceptance of the methodology used by the evaluation. For this reason, it is critically important to invest in discussing the methodology of the approach prior and during the evaluation process itself.

Second, the acceptance of evaluation results depends on the quality of the evaluation findings. Evaluators

must understand that they are not judges, and must base their findings on facts and evidence to the maximum extent possible.

Third, evaluators should show empathy towards those that are being evaluated and be able to effectively convey the sentiment. The reality on the ground is often complex and difficult, and evaluators should show an appreciation for the many possible causes and reasons for poor results, and be very prudent before assigning responsibilities. They should always remain as constructive as possible.

establishment was seen as important step in better underthe the val-

of IFAD.“

- Luciano Lavizzari

Fourth, evaluation findings are more likely to be accepted if they are feasible, doable and if they take into account the actual capacity available on the ground to implement all the recommendations.

Looking ahead, how important will it be for IOE and IFAD to continue to have a robust evaluation policy?

Indran

This is critically important, although a policy on its own is not sufficient to ensure success. Firstly, I am pleased to note that IFAD has an Evaluation Committee (EC), which represents the EB, and which has understood very clearly the imperative of IOE.

The EC and the EB fully recognize that we are here to provide them with critical information for their oversight. The second issue is behavioural in-

dependence. We can have structures and policies in place, but if there are elements that undermine the integrity and behaviour of the IOE Director, then the independence quickly becomes something in name but not in practice.

IOE is clear about the fact that its role is to provide recommendations. We do not expect Management to agree with all. We would prefer that Management disagrees and explains reasons thereof on a recommendation, than accept and not act upon it. A key indicator of the success of this office is that IFAD has a very high uptake on Management responses, one of the highest that I’ve ever seen. Any new programme, project or policy is always pre-empted by what the relevant evaluation said and the consequent actions that are taking place.

The sweet spot that we want to hit is when you bring together the highest quality of analytical work that serves the purpose of critically but constructively supporting the mandate of the organization. We need to remember that IFAD is over forty years old, and some of its processes and practices may be outdated. For the organization to renew and rejuvenate itself, it needs constant, real-time feedback that feeds into decision making, so that it becomes agile, response and fit for the challenges that lie ahead.

Fabrizio

Having a strong evaluation policy and an independent department is good for IFAD, not just for IOE. A few years ago, there was a MOPAN assessment of IFAD which said that ‘one of the reasons why we have confidence in IFAD is because there is an independent evaluation department. We know that there is a unit that will always provide an objective assessment of what IFAD does’. That’s good not only for IOE, but also for IFAD, because it gives credibility to the organization.

I think the policy that we have now is sufficiently strong, but as you know, a policy needs to be kept alive, meaning that we need to implement and achieve what the policy wants to do, which is help IFAD become a stronger organization, be more effective in the field, and promote rural development. That’s what we want. We also want to promote dialogue and cooperation with other parts of IFAD. The idea of having an ivory

tower in the evaluation office is not what the policy is trying to promote. It wants to protect independence, but it also wants to foster collaboration.

In the future, we might discover that some additional aspects in the evaluation function are important, as has happened in the past. Frist, we have to guarantee, strengthen and bolster independence, then we have other aspects. The same might happen in the future. But I think for the moment we have a strong policy and we need to implement it and make clear what it is for, which is improving IFAD, it’s operations and its results.

Oscar

I am a very strong admirer of the office and feel very proud about its work. I really think that it’s one of the best within the multilateral system. Moving forward, I think it is time to take the evaluation function out of the conference room, and to bring it to the field. This means making all of the evaluation findings much more accessible to the beneficiaries of IFAD’s programmes, making them available in the local languages that beneficiaries understand. In Africa, for example, there are so many important lessons that beneficiaries could share throughout the continent on issues such as value chains. These could really enhance their ability to connect to markets and improve their livelihoods much quicker. Unfortunately though, the content of our evaluations is still in our formal systems, and does not reach the ultimate beneficiaries of IFAD’s projects. The same reasoning should be applied to making videos and other visual products. Today, two-thirds of the content online is moving images. However, we are still focused on reading, while a large proportion of our beneficiaries are illiterate. You can improve methodologies and have more transformative approaches, dealing with complexity, trying to get out of a liner logic and understanding multiple causalities: these are all interesting fields to pursue. However, the innovation – in my view – will not come from breakthroughs on the methodological side. Making a bold move to bring findings to the ground level, I see this as being the next frontier, rather than finding new ways to better refine knowing what we know. For IFAD and IOE, this could be a real niche going forward.

Luciano

It will obviously be very important because it brings reassurance to the Board that IOE continues to bring state-of-the-art knowledge in evaluation, and also continues to possess the ability to speak the truth to all its stakeholders and hence promote accountability and learning. Retrospectively, we have seen that the evaluation policy has been strengthened and evolved. It is easy to imagine that this will continue in the future, and that the policy will evolve even further.

However, the outlook is not so positive when we consider the capacity of recipient and borrowing countries to undertake evaluations. Improvements are noticeable in some countries, but for the most part countries have an evaluation capacity that remains quite weak. In addition, these countries also remain only moderately interested in evaluations and in the implementation of findings. The truth is that, for many countries that receive loans and grants from IFAD, evaluation remains a necessary evil, something that they have to accept in order to receive further resources. In such a context, the challenge consists in developing an evaluation policy that is owned not only by IFAD, but also by its borrowers and recipient countries. The only option that we have is focusing on capacity-building in terms of providing resources and training, and supporting institutional development to strengthen the evaluation capacity of partners and governments. This is something that IFAD cannot achieve alone, and will require the support of other development agencies.

Thank you, esteemed colleagues.

You are welcome, Alexander.

The first Evaluation Advisory Panel concludes on a positive note

“We believe that the IOE evaluation processes are among the best that we can find in international development organizations”, stated Dr Gonzalo Hernández Licona, Director of the Multidimensional Poverty Network at the University of Oxford, during the annual meeting of the Evaluation Advisory Panel (EAP) of the Independent Office of Evaluation of IFAD (IOE).

In 2021, Dr Naidoo established the Office’s first EAP to further IOE’s contribution to IFAD’s organizational and development effectiveness. The panel provides the Director with systematic advice, by reviewing and commenting various aspects of IOE’s work to enhance the professionalism of the evaluation function. The panel also serves as a critical friend, drawing on its substantive experience and expertise to help improve our independence, credibility and utility.

IOE hosted the 2nd EAP meeting at IFAD headquarters, in Rome, on 14 and 15 November 2023. The event featured active contributions from all EAP members, namely Prof Donna M. Mertens, Professor Emeritus at Gallaudet University; Rob D. Van den Berg, Visiting Professor at King’s College London; Prof Bagele Chilisa, Professor at the University of Botswana; Hans E. Lundgren, former Head of the Evaluation Secretariat at the DAC Evaluation Network of OECD; and Dr Hernández Licona. EAP members paid special attention to following up on the areas for methodological development identified in the first annual meeting, in 2022. In addition, panel

members provided strategic advice to IOE for further fine-tuning its approaches and strategic directions.

Distinguished country representatives of the IFAD Evaluation Committee attended the event, which also saw the participation of Dr Claudia ten Have, Director of the Office of the Secretary and Secretary of IFAD, who attended on behalf of the Fund’s Management. Dr ten Have commended IOE for the value-added that it brings to IFAD, helping the Fund improve its operations on the ground. The Director also emphasized that she finds the commitment with which IOE delivers its work to be truly inspiring, and applauded the Office’s dedication to continuously seeking to improve the quality of its outputs.

IOE staff members also played a key role, engaging with EAP members on the first day of the event, which featured three presentations. In their joint presentation, Dr Hernández Licona and Mr Lundgren reviewed the formulation of IOE recommendations and follow-up. The EAP members noted that IOE and IFAD Management have a clear and transparent process for producing important recommendations and a strict follow-up process. Recommendations are well-formulated, backed by solid empirical evidence, and substantiated by robust findings and conclusions. Equally important, recommendations are formulated in an action-oriented manner, with clear target groups. The combination of these positive factors has led to a very high Management uptake on recommendations, culminating in a 100% agreement rate in 2023.

Other EAP presentations included those jointly discussed by Prof Chilisa and Prof Mertens, as well as that delivered by Mr Van den Berg. Prof Chilisa and Prof Mertens put the spotlight on ‘leaving no knowledge behind’. Discussions centred on the importance of promoting testing of innovative techniques, frameworks and approaches, including those based on indigenous people’s experiences. Equally important is to decolonize research and evaluation paradigms,

In his session, Mr Van den Berg delved into the facets of climate finance. He noted that CCA receives less funding than mitigation. However, in light of the worsening of the climate crisis, mitigation would need to include appropriate adaptation and vice versa. In this context, he noted that IOE’s evaluations have scope for climate finance and may benefit from further fine-tuning in order to address transboundary issues and to incorporate a broader contextual analysis.

and embrace a gender and disability transformative approach to evaluation, underscored the two panel members.

“

POST COVID-19 REALITY CALLS FOR REORIENTATION OF EVALUATION PROFESSION

Evaluators no longer have exclusivity on performance appraisal. There are new actors that have moved quietly into the space, and have done so quite successfully. Against this backdrop, evaluators need to ask themselves if they can they answer the big and urgent questions of our times, or if they have been overtaken by reality in terms of their own relevance. Being fit-for-purpose requires rethinking approaches and crafts”, affirmed Dr Naidoo during the webinar titled ‘What did we learn? Policy evaluation in the era of COVID-19’, on 6 December 2023.

Hosted by the Max Bell School of Public Policy, at McGill University, the virtual event brought together international experts, practitioners and established academic thinkers in the field of policy evaluation, and drew on insights from Australia, Canada, the UK and the United Nations. The webinar marked Routledge’s open-access release of the book ‘Evaluation in the Era of COVID-19’. Event speakers featured the co-editors of the volume, namely Dr Naidoo; Prof Pearl Eliadis, Associate Professor at McGill University; and Dr Ray C. Rist, former President of the International Program for Development Evaluation. Several chapter authors also intervened, including Mariel Aramburu, Partnerships Officer at the UN World Food Programme; Steve Montague, Partner at Performance Management Network Inc.; Robert E. Lahey, President of REL Solutions Inc.; Dorothy Lucks, Executive Director of Sustainable Development Facilitation Global, Australia; and Jeremy Lonsdale, former Director of the UK National Audit Office.

In a frank take on the shortcomings of evaluation during the pandemic, contributors shared

notable stories of learning, innovation and successes, using institutional, national and disciplinary perspectives. Discussants touched upon a number of questions, such as whether evaluation met the challenges of the COVID-19 crisis, if evaluators were caught flat-footed by the restrictions and challenges of the pandemic, and how evaluation practices, architectures and values were affected.

Traditionally, evaluators have worked in predictable environments, often in silos, with long time horizons available to generate results and findings. The COVID-19 pandemic called for a change of pace and of approach. Unfortunately though, evaluation outfits were largely unable to respond by working in a holistic manner and at speed, especially in the UN. As a result, the IOE Director explained that during the crisis the input of evaluators was limited. Instead, the critical monitoring data generated and used for recovery came from non-traditional actors and sectors, such as academia and research outfits. This was a missed opportunity.

He further highlighted that, today, the demand from governments is for action-oriented solutions, which often cannot be delivered through traditional evaluation models. Moreover, traditional evaluation ecosystems, trainings and methodologies focus too much on methods in the sense of research, and not enough on the need to engage and co-create. Furthermore, they revolve around project-based thinking, assume linear trajectories, and do not reflect dynamic and new development paradigms, which are not easy to measure due to interconnectivities. For this reason, new forms of working need to emerge, including closer collaboration with

think tanks and coalitions, in an effort to move away from the view that evaluators possess the exclusive right to judge.

In line with these considerations, Prof Eliadis underscored the need to think about human rights as a broad transformative framework in support of the Sustainable Development Goals (SDGs). Evaluation, historically, has not done a great job about understanding or integrating human rights into its framework, except as one of many

principles and ethical ideas that evaluators consider. As a result, Prof Eliadis argued that it needs to be frontloaded, and given a much higher rank in the hierarchy of evaluation practice.

Looking ahead, Dr Rist stressed the need to recognize that the crisis is still underway. Broad pronouncements are neither warranted nor required. Instead, brief and careful evaluation studies are mandated. These should start by recognizing that an emerging trend from the pandemic is that it is strengthening the institutionalization and reach of big government. This stands in contrast to several previous decades, in which the role of government had been reduced in size and scope, and societies had increasingly relied on the market.

‘Policy Evaluation in the Era of COVID-19’ is the first book to offer a broad canvas that explores government responses and ideas to tackle the challenges that evaluation practice faces in preparing for the next global crisis. Contributors examine the impacts of evaluation on socioeconomic recovery planning, government innovations in pivoting internal operations to address the crisis, and the role of parliamentary and audit institutions during the pandemic. Chapters also example the SDGs, and the inadequacy of human rights-based approaches in evaluation, while examining the imperative proposed by some authors that it is time that we take seriously the call for substantial transformation.

Better awareness of local nutrition practices necessary to tackle malnutrition in Laos

Awareness of local nutrition practices and solid coordination among development partners are necessary to tackle the multiple, complex causes of malnutrition in Laos. This is one of the main findings that emerged from the project performance evaluation (PPE) of the Southern Laos Food and Nutrition Security and Market Linkages Programme (FNML) in the Lao People’s Democratic Republic, published by the Independent Office of Evaluation of IFAD (IOE).

The PPE report found that while FNML’s support to agricultural production groups contributed to increased production and crop incomes, this did not translate into improved nutrition. Moreover, besides implementing nutrition-related activities through different partners, there was no clear coordination with other nutrition programmes in the target area, which would have been crucial to address the multiple and complex causes of malnutrition. Consequently, malnutrition and diet diversity remain serious problems for the target families.

FNML was designed to ensure sustainable food and nutrition security and incomes of households in 175 villages in the target districts in the south of the country.

The total programme cost was US$18.86 million, funded by an IFAD Debt Sustainability Fund grant, an Adaptation for Smallholder Agriculture Programme grant, the Government of the Lao PDR and beneficiaries.

The FNML is the sixth project evaluation carried out by IOE in Lao PDR to date.

The IFAD-supported programme proved innovative in a number of areas, including the creation of agricultural production groups (APGs), the provision of training, matching funds and adaptation to climate change practices. With regard to the latter, concerted efforts were made by FNML to mainstream climate change adaptation throughout the programme. These resulted in the implementation of climate change adaptation learning

activities within the country and region, and the integration of such practices into project interventions.

An activity that farmers highly valued was the provision of seedlings for cultivation in the forest production zone. This intervention added a clear production value to the forest and enhanced forest cover. FNML also improved farmers’ access to markets through the setting up of agroshops along main roads, providing an additional market for nearby villages to sell produce. Beneficiaries perceived these outlets as useful for increasing sales.

Overall, the provision of inputs was well-adapted to the needs of APGs owing to their close involvement in the process. Increases in the farming area cultivated, in the volumes produced and yields increased productive capacity.

Nevertheless, although all target villages were reached, the remoteness of many villages with poor road conditions meant they generally received less support. The different livelihood strategies and needs of people across villages in different rural contexts was not fully recognized or accounted for. The planning processes in some villages also meant that some voices were less heard or never heard, such as those

of young men and women with limited or no access to land, as well as poor women and men who do not speak Lao.

Furthermore, APG members benefited from the project in proportion to their rates of literacy and means of production. This means that the benefits have been relatively higher for people in villages close to district centres and men and women with more arable land, labour and literacy skills.

In addition to the aforementioned nutrition-related lessons, the experience of FNML highlights the need for a better balance between the breadth of activities, local implementation capacities and the number of poor and remote villages that can be reached effectively. IFAD and Government should ensure a strong diagnostic study of local conditions and different poor and vulnerable people’s needs and priorities during design and early implementation to apply tailored solutions to different locations and groups.

Laos Receives Accolades at ASEAN Tourism Standards Awards Ceremony*

Access report [here]

@Pixabay/142253

Laos received 20 ASEAN Tourism Standard Awards at a ceremony held in Vientiane Capital on 26 February to officially seal the end of the ASEAN Tourism Forum. In total, 192 entities in the bloc received this award.

Under the umbrella of the ASEAN National Tourism Organization, the bloc has established seven tourism criteria aimed at standardizing tourism services. These criteria are crucial in elevating the quality of tourism products and services while fostering responsible management practices among tourism businesses and stakeholders across the region. Recognizing adherence to these benchmarks, awards are bestowed upon facilities in the region that meet ASEAN’s criteria for excellence in tourism services.

Vientiane Capital and Kaysone Phomvihane City in Savannakhet Province received the ASEAN Clean City Tourist Standard award, which is aimed at encouraging major urban tourist destinations to improve the quality of tourism in their cities, increase their marketing competitiveness, and also improve the situation of local residents and their livelihood by alleviating poverty.

This year’s ASEAN Tourism Forum, from 22-27 January, focused on sustainability, with the bloc’s members committing to prioritize environmentally responsible practices in tourism.

Tourism projection for the region revealed a year-on-year increase of 153% in international arrivals for the region, signaling a strong trajectory for post-pandemic recovery.

*https://laotiantimes.com

HOLISTIC CLIMATE ADAPTATION SOLUTIONS CALLED FOR AT COP 28 SIDE EVENT

There is a tendency to first mobilize climate adaptation finance, then to think about how to support the governments to have climate adaptation policies, and then to worry about the solutions on the ground. Instead, we need to adopt a holistic approach to climate change adaptation solutions, explained Dr S. Nanthike-

san, Lead Evaluation Officer of the Independent Office of Evaluation of IFAD (IOE), during a side event at the UN Climate Change Conference, COP 28.

“We are thinking about climate change adaptation by looking at finance, national policies and capacities, and where farmers are going to produce agricultural solutions, as silos. These cannot be separated like this - they have to be integrated. You have to make sure that the chain is completely analysed and that you are able to produce a solid argument for why you need that dollar and how you are spending it at the ground level”, Dr Nanthikesan underscored.

Titled ‘advancing adaptation and climate resilience through evidence and opportunities lenses’, the COP 28 side event took place in Dubai, on 3 December 2023, co-organized by the Inter-American Development Bank and the German Institute for Development Evaluation (DEval). The event discussed the fact that increasing funding and supporting efforts to achieve the global goal of adaptation requires evidence, solutions, alliances, and good practices of international climate institutions. In particular, the event presented research and evaluation findings regarding progress, challenges and gaps as well as opportunities, solutions and good practices of effective and just climate finance as well as loss and damage action. In addition to Dr Nanthikesan, panellists included Andreas Reumann, Head of the Independent Evaluation Unit at the Green Climate Fund; Dr Sven Harten, Deputy Director at DEval; and Debbie Menezes, Chair of the Technical Evaluation Reference Group of the Adaptation Fund.

“In terms of solutions that we have, good is no longer good enough. We have come to a stage where a ‘doing no harm’ approach is not going to solve the problem. We have to focus on restoring the damaged ecosystems. In this context, the urgent question is how do we convince those governments that are reluctant to make investments in adaptation solutions that ‘do no harm’ to now invest in restoring. That will be a challenge that needs international collaboration to address”, said Dr Nanthikesan.

These considerations stem from the fact that national commitment and capacities for promoting climate adaptation are uneven. In an era of debt distress, pandemic and wars, it is challenging for governments to prioritize sound climate investments that are perceived to take resources away from poverty reduction. Often, they have to choose between saving people from immediate hunger and disease today, versus saving them from risks that may materialize in the future. Moreover, governments have to deal with multiple priorities, and recognizing climate adaptation as one of the key priorities can be a challenge. In this context, how can evaluations be useful to convince governments of the need for adaptation investments?

“To convince the governments, you have to show

that your programme can not only solve the climate resilience problem, whilst addressing the environmental resilience problem, but also give economic resilience. Resilience in all three dimensions - economic, climate and environment - needs to be achieved together in order to have win-win solutions. There are concrete examples of this, which we encountered in our thematic evaluation of IFAD’s support for smallholder farmers’ adaptation to climate change”, noted Dr Nanthikesan.

Moreover, it’s not only the commitment of the governments that needs to be focused upon, but also that of the smallholder farmers. There may be brilliant solutions at hand, but why should smallholder farmers uptake them when there are financial risks involved? To best change behaviours, IOE’s climate change adaptation evaluation found peer-to-peer learning and strong agricultural extension services to be effective in convincing smallholders of the benefits of the proposed initiatives. Naturally, all these initiatives require funding. The United Nations Environment Programme estimated that, globally, climate adaptation would require between US$ 140 and US$ 360 billion annually to succeed. At the present time, there are approximately US$ 50 billion available, at most.

In addition to this side event, IFAD’s participation at COP28 featured a series of other events that addressed key aspects of climate change and sustainable agriculture. Events included shedding light on global climate finance challenges, launching the Africa Rural Climate Adaptation Financing Mechanism (ARCAFIM), and addressing climate vulnerability in Small Island Development States. IFAD collaborated with civil society to investigate innovative strategies for a sustainable food future, and explored the impact of climate change on local cuisines with Senegalese chef Pierre Thiam. The series concluded with discussions on scaling nature-based solutions, ensuring sustainable food systems, and exploring digital innovations for climate-resilient agriculture.

Targeting must be embedded in multi-faceted development programmes

Multi-faceted development programmes target poor people in complex realities. For this reason, development agencies need to embed targeting as a value throughout the project cycle, from project design to implementation, and ensure adequate resources are made available in terms of time, staff and funds. Fabrizio Felloni, Deputy Director of the Independent Office of Evaluation of IFAD (IOE), and Jeanette Cooke, IOE Evaluation Officer, underscored the importance of addressing this need during the international conference titled ‘Targeting of the poor’, on 8-9 November 2023.

The Initiative for Agency and Development at the University of Arizona hosted the conference, co-organized with IOE,

at its premises in Tucson, Arizona. The event provided a forum for presentation and discussion of the cutting-edge applied evaluation research on targeting of the poor and ultra-poor. The line-up of participants and discussants featured practitioners, technical experts and academics from international financial institutions, United Nations entities, Government agencies, universities and think tanks.

The conference’s academic leads were Tauhidur Rahman, Associate Professor at the University of Arizona, Mr Felloni and Dr Naidoo. Mr Felloni represented IOE in-person during the event. In particular, Mr Felloni offered introductory remarks, following which he delivered a presentation alongside Ms Cooke during

a session titled ‘How to know better what works in targeting: lessons from a multi-country evaluation of targeting challenges and practice’. Mr Felloni also participated in the final panel discussion titled ‘Targeting of the poor’, while Ms Cooke acted as a discussant in a session titled ‘Evidence from traditional and new targeting approaches: Evidence from Norway’, led by the evaluation office of NORAD. The typology of programmes reviewed included development projects as well as national social protection programmes. A keynote speech by Professor Price Fishback, Regents Professor at the University of Arizona and one of the leading scholars in the USA on the history of social security, provided an overview of the history of social protection

programmes in high-income countries and lessons learned and implications for similar programmes in low-income countries.

Throughout the two-day event, participants reiterated that effective and efficient targeting of the poor is a critical step for successful anti-poverty and social protection programmes, particularly in rural areas. However, they also recognized that targeting of the poor is challenging, especially in developing countries. This is because poverty status is dynamic, with people and households moving in and out of poverty, multidimensional, since it involves income, nutrition, human and social capital, among other dimensions, and challenging to measure.

In its first evaluation synthesis note titled ‘Targeting in IFAD-supported projects’, IOE confirmed that targeting poor rural people is central to IFAD’s mandate and to realizing its comparative advantage.

Presenting the report during the conference, Mr Felloni and Ms Cooke explained that IFAD’s comparative advantage is its engagement in targeting poor rural people. This distinguishes it from other financing institutions. Moreover, IFAD has piloted and increasingly adopted targeting innovations in and alongside its loan programmes. Nevertheless, target groups are sometimes unclearly defined or defined in multiple ways in the Fund’s interventions.

Against this backdrop, the IOE report notes the importance of a distinction between tar-

get groups and the principle of inclusion. Target groups are those that the project is mainly intended to benefit. Inclusion is a principle that can be applied across project interventions and addresses the issues of access and equity. Project design and implementation can make core activities more inclusive, instead of creating parallel components for unreached groups as separate target groups.

Many discussants who intervened during the conference also noted that, traditionally, anti-poverty and social protection programmes have relied on household consumption expenditure survey data to identify the poor. However, such surveys are expensive and suffer from various measurement errors. As a result, a significant amount of interdisciplinary literature has emerged on alternative methods of targeting the poor, including studies that utilize qualitative and quantitative techniques, case studies, randomized experiments, and impact evaluation studies conducted by international development institutions. The conference also reviewed the importance of process evaluation, in order to assess policy and organizational capacity and incentives for targeting, at the donor agency and national government agency level.

Today, alternative methods of targeting have been proposed and utilized, including means testing, proxy means testing, geographic targeting, demographic targeting, self-targeting, and community-based targeting. The conference

also discussed the opportunities and limitations of using GIS-based imaging and related meta-data for geographic targeting, as well as to complement large sample surveys that assess targeting effectiveness, with examples from Central and South America.

A selection of academic papers discussed during the conference will be considered for publication in an edited book.

@Pixabay/monika1607

Geospatial tools address evaluation challenges

The Independent Office of Evaluation of IFAD (IOE) successfully deployed GIS tools to overcome challenges encountered in conducting the country programme and strategy evaluation (CSPE) of IFAD’s support in Ethiopia. The tools provided a spatial framework to overlay project activities, monitor progress, and analyse geographical patterns.

The Ethiopia CSPE covered the period 2015-2022. The evaluated portfolio amounted US$ 1.8 billion with an IFAD financing of US$ 654.1 million, in support of nine projects, five of which have been completed. These projects included the Participatory Small-scale Irrigation Development Programme II (PASIDP II), which rehabilitated 51 irrigation schemes. Specifically related to PASIDP II, the CSPE assessed the extent to which the project contributed to improving livelihoods of smallholder farmers and to enhancing their resilience.

Monitoring and evaluation challenges encountered in evaluating PASIDP II included difficulty in tracking activities across diverse agricultural landscapes, and limited ability to identify specific land-

scape areas for intervention. Against this backdrop, IOE investigated the spatiotemporal variation of rainfall, vegetation, temperature, and evapotranspiration within 51 irrigated areas, from 2009 to the present, using remote sensing technology to assess changes or variations in vegetation greenness and health.

The application of GIS tools helped IOE to obtain accurate measurement and visualization of key indicators such as land use changes and crop yields. This enhanced the accuracy of reporting and provided stakeholders with clear visualizations of project impacts. In particular, this approach allowed the evaluation to confirm the positive results

achieved in improving and strengthening the ecosystem resilience through soil and water conservation measures in degraded areas, community-based watershed, and rangeland management.

IFAD is investing in the use geospatial tools. GIS software allows users to produce maps and other geographic displays to analyse and present information. The displays typically include points, lines, areas or raster images from photos or scanned images. The Fund’s ICT team provides in-house training for staff on how to build data dashboards, leverage advanced analytics, and work with GIS and artificial technology. In addition, IOE has compiled a list of external

training materials and information facilities on GIS tools and databases. These allow staff to harness the power of complex geospatial analytics with easy-to-use tools. IFAD has also established a community of practice for geospatial applications, where staff can get advice on geospatial tools, data and methodologies.

Access report [here]

Independent Development Evaluation office of the African Development Bank hosts ECG meeting

Enhancing collaboration between independent evaluation and Management

In recent years, the Independent Office of Evaluation of IFAD (IOE) and the Fund’s Management have made notable strides forward in building synergies between independent and self-evaluation products, whilst avoiding overlaps or duplication of efforts. The evolution and current status of this productive relationship is captured in the third edition of the IFAD Evaluation Manual, which was jointly published by IOE and Management. Fabrizio Felloni, IOE Deputy Director, explained that this was an unprecedent-

ed step in the collaboration between the two entities, during a panel presentation at the Fall 2023 meeting of the Evaluation Cooperation Group (ECG), on 20 October 2023.

Held in Abidjan, on 18 – 20 October, the Fall 2023 meeting of the ECG brought together the heads of the evaluation offices of the African Development Bank, Asian Development Bank, Black Sea Trade and

Development Bank, Central American Bank for Economic Integration, the European Bank for Reconstruction and Development, the European Investment Bank, the Inter-American Development Bank, the World Bank Independent Evaluation Group, IFAD, the International Monetary Fund, and the Islamic Development Bank. As current Chair of the ECG, AfDB hosted the group of directors in its office premises. A number of senior evaluation officers and technical specialists attended the event remotely. Dr Naidoo represented IFAD in

person, and moderated panel discussions. Fabrizio Felloni, IOE Deputy Director, and Dr S. Nanthikesan, Lead Evaluation Officer, joined on-line, and delivered presentations.

The event was structured around four main discussion themes, namely: (i) ‘quality assurance in and of the evaluation process’; (ii) ‘capacity development in and through our own evaluation work’; (iii) ‘integrating a climate perspective in evaluation’; and (iv) ‘synergies, collaboration or duplication between independent evaluation and evaluation by management’. Each theme featured a series of panel presentations, followed by group discussions.

Speaking during the fourth discussion theme, Mr Felloni presented the working relationship between IOE and IFAD Management. A major milestone in said relationship was the joint publishing of the 2022 Evaluation Manual. This marked the first time in IFAD’s history that methodological guidance and standards for evaluations were applied across the whole organization. The Manual provides details on the complementarities of IFAD’s self- and independent evaluation systems and related evaluation products with a view to strengthening accountability, learning and the overall utility of evaluations efforts. In particular, the Manual establishes common quality standards for the two, enhances their mutual consistency and, ultimately, fosters a stronger results and evaluation culture at IFAD.

The IOE Deputy Director also

participated during the third discussion theme. In his presentation, Mr Felloni spoke about the importance of integrating a climate perspective in evaluation, and explained the efforts undertaken by IOE in this regard. He noted that the Evaluation Manual has merged the ‘natural resource management’ and ‘climate change adaptation’ criteria under ‘sustainability’, which now applies to all projects and country -evel evaluations. Elements focused upon include climate-resilient technology and practices, climate-proof infrastructure, forest conservation or regrowth, renewable energy, big data for new services, and engagement in policy dialogue. Furthermore, IOE published a thematic evaluation of IFAD’s support to smallholder farmers’ adaptation to climate change, earlier this year. Currently, IOE is carrying out a subregional evaluation on the Dry Corridor in Central America.

The ECG event also included two webinars. IOE and the AfDB organized the first, which was titled ‘Evaluating Organizational Decentralization: Insights from IFAD and AfDB’. the IOE Director chaired the session, which saw presentations from Dr S. Nanthikesan and Oswald Agbadome, Principal Evaluation Officer, AfDB. Dr S. Nanthikesan presented the main findings and summary recommendations of IOE’s recent ‘Corporate-Level Evaluation of IFAD’s Decentralization Experience 2023’. IOE’s Lead Evaluation Officer clarified that decentralization is widely recognized at the Executive Board, Management and staff levels as a necessary step to

improve the development results achieved on the ground. However, although IFAD’s former President and Senior Management were committed to decentralization, the decentralization process was implemented without sufficient time for adaptive management, was not adequately informed by the decentralization experiences of other IFIs and UN agencies and human resource allocation did not reflect enough lessons from past IFAD experience.

The importance of distinguishing between fact and fantasy

Keynote address by Dr Naidoo at EIIS event

Students that are set to enter the multilateral development workforce will need to be critically minded in order to distinguish between facts and fantasies, and to recognize that there is no ‘one truth’ in this line of work. The ability to understand different perspectives, linking theory to practice and working in different contexts, is essential for navigating the present and foreseeable future job market. Dr Naidoo put these strong messages on the table during the ‘Food & Sustainability On-site Experience’ of the ‘Food and Sustainability Certificate Program’, on 17 January 2023.

IFAD and the European Institute of Innovation for Sustainability (EIIS) jointly organized the certificate programme, which has a community of 300 alumni from over 80 countries and is designed to provide students from around the

world with a comprehensive understanding of food sustainability issues. The Food & Sustainability On-Site experience represented the last step of the programme, on the eve of the graduation ceremony, and was held under the auspices of IFAD’s Change Delivery and Innovation Unit (CDI).

Coordinated by Gladys Herminia Morales Guevara, Senior Officer and Global Head of Innovation, the on-site experience welcomed to IFAD approxi-

mately 50 students from 30 different countries to give them an opportunity to learn more about the work of the Fund, and interact with some of their favourite faculty members. The IOE Director was joined by a group of distinguished experts who constituted a panel on ‘youth, climate and innovative approaches for food systems’. Presenters featured Ilaria Firmian, Senior Technical Specialist, Indigenous Peoples, Environment, Climate, Gender and Social Inclusion Division, IFAD; Thin Yu Mon, Global Indigenous Youth Caucus Representative, Asia; and Cortney Price, Lead for Behavioural Science, FAO. Ann-Kathrin Beck, Junior Adviser in the Office of the President and Vice President of IFAD provided closing remarks.

In his opening keynote address, the IOE Director noted the opportunities and challenges that the rapid rise of artificial intelligence will likely pose to employment prospects for young graduates. These may include a shift in jobs from urban and peri-urban realities to rural areas. This will call for a flexible workforce, equipped with an adaptive skill set. Equally important is the recognition of the permacrisis that we are living in, which presents the world with multiple challenges, including climate change, economic downturns and the long-term effects of the COVID-19 pandemic. The latter are described in the recently published volume ‘Policy Evaluation in the Era of COVID-19’, co-authored and co-edited by Dr Naidoo.

In the midst of this complex and fluid global environment, he explained how IOE has been proactive in shifting organizational mindsets at

IFAD, in an effort to move from evaluation phobia to evaluation embracing. To this end, the Office has mainstreamed neuroscience-based principles into its approach to communication – a strategy which has paid dividends, as engagement with stakeholders across the board is fast-improving in qualitative and quantitative terms.

The food and sustainability certificate programme also afforded students the possibility to be protagonists of their very own innovation challenge, designed and coordinated by CDI. The challenge was aimed at fostering the identification of sub sets of problems and potential food-value chain solutions that contribute to building resilience in smallholders and Indigenous People to climate change. Participants’ proposals were presented on the occasion of the course Graduation Ceremony, which was held in Rome, at Palazzo Taverna, on 18 January.

The EIIS is an academic company that designs innovative solutions for companies, academia, institutions and entrepreneurs to facilitate and foster innovation for sustainability. In EIIS, people connect to reach environmental, social and economic sustainability, through education, research and continuous experimentation.

A new evaluation collaboration is taking shape

The Memorandum of Understanding (MoU) between the Department for Evaluation of the Norwegian Agency for Development Cooperation (NORAD) and the Independent Office of Evaluation of IFAD (IOE) is coming into full force, as joint efforts will soon be getting underway. Speaking at a seminar hosted by IOE to kick-start implementation of the agreement, Helge Østtveiten, Director of the Department for Evaluation at NORAD, noted that the MoU marks an important milestone insofar as it brings together the evaluation offices of a bilateral agency and of a multilateral organization – something which has rarely happened before. The agreement will be mutually beneficial for both partners, given overlaps and synergies in key thematic areas of research and countries of interest.

Collaboration on country evaluations, joint workshops, publications, trainings, seminars and staff exchanges are but some of the opportunities that the MoU opened back in June 2023, when Dr Naidoo and Mr Østtveiten signed the agreement. Moreover, with their

common interest to plan, implement and report evaluations, and wishing to coordinate their efforts within the scope of their respective mandates, IOE and NORAD’s Department for Evaluation agreed to collaborate in a number of areas. These areas were the focus of attention during the joint seminar held on 4 December 2023 in Rome, which laid the groundwork for operationalizing the agreement between the two partner offices.

During the seminar, Mikal Khan, IOE Evaluation Officer, Dr Ida Lindkvist and Anne Asselin de Williencourt, NORAD Senior Advisers, presented opportunities for tangible collaboration. These included methodological exchanges related to recent high-level evaluations of mutual interest, in areas such as climate change adaptation and knowledge management. Furthermore, the offices discussed the possibility of joint studies and other types of collaborative products related to specific countries that they are scheduled to work on. Countries may include Ghana, Egypt and Madagascar, as well as Small Island Developing States. NO-

RAD also expressed a keen interest to learn more about IOE’s innovative work in the field of brain science, and its efforts to apply neuroscience-based principles to the work of evaluation. These and other topics may form the basis for future webinars, during the course of 2024.

The event also offered an opportunity to discuss and exchange lessons learned related to communication and outreach practices. In this regard, Dr Alexander Voccia, Coordinator of IOE’s Evaluation Communication Unit, and Anja Øvregaard, NORAD Advisor, presented three pathways for future collaboration. These included, efforts to promote joint outputs between the two offices, systematic sharing of

best practices, and mutual support for growing respective audience bases.

The Department for Evaluation, located in NORAD, is mandated to initiate and carry out independent evaluations of any activity financed over the Norwegian aid budget. In 2022, NORAD’s overseas development assistance amounted to US$ 5.2 billion. The evaluation function contributes, in part, to learning and gathering experience from Norwegian participation in international development cooperation, and in part to keeping the actors in Norwegian development policy accountable for its administration.

“Anybody with skin in the game in IFAD has to have faith in the credibility of the products that

IOE generates”

Ms Michelle Winthrop is the current Ambassador of Ireland to the Republic of Korea, prior to which she served as Development Policy Director, in the Department of Foreign Affairs and Trade of Ireland. It was in this capacity that she partook in the IFAD Executive Board and Evaluation Committee (EC). Her role within the EC has been greatly appreciated, as have been the thoughts, perspectives and feedback that she provided IOE over the years. Recognizing this notable contribution, Independent Magazine reached out to Ms Winthrop, who graciously agreed to take time out of her busy schedule.

Good morning, Michelle. Good morning, Alexander.

In light of your experience as a member of the EC, what are your considerations regarding the extent to which IFAD uptakes and implements evaluative recommendations?

As a member of the EC, I always really appreciated the manner in which IOE sets out recommendations very clearly. The consistency of structure across all the reports clearly links the findings to the corresponding recommendations, and to what IOE advises that Management should do. In turn, Management always give a very structured response following the same structure. This makes it very easy to track how Management is or is not picking up on those recommendations. I’ve also always found the EC to be a very helpful forum. I find that the tone of the discussion is always very honest between IOE and Management. For the most part, Management do agree with the recommendations and propose specific actions to follow up on them. Occasionally, of course, they don’t agree. This may be due to how IOE has come to a particular recommendation as a result of the findings, or to a difference of opinion about what Management should do in response. In my experience, these kinds of discussions have always been very open, very constructive and very respectful. As a member of the EC you have a sneak preview of these discussions, you can see where there are tensions. To be honest, though, tensions are more often not there.

I know that IOE, particularly since Indran has joined, has really invested in making sure that there’s a constructive and mutually respectful relationship with Management. This really shows in the process of working through recommendations, particularly when there is disagreement. In those cases, there may be an agreement to disagree, or compromises might be found. Nonetheless, at all times, I found that a foundation of trust and respect underpins everything. This is what makes the clarity of follow-up by Management easy to track and interpret on the rare occasion that disagreement does arise.

What do you believe is the contribution that evaluation makes to transparency and accountability, as well as to a democratic ethos at the institutional level?

I think that, by contrast to many other sister UN agencies, IFAD has a very dogged focus on impact on

the ground. Again, by contrast to agencies that are delivering slightly more linear type results, a lot of the development challenges that IFAD is seeking to address are particularly complex. They are often in some of the most challenging locations in the world, with people who have really quite limited capacity and face very real challenges. Because of that, all IFAD stakeholders are very focused on following the impact on the ground, it is almost like the north star that guides all of our engagement. That’s important for donors, because we make the case to our government for putting taxpayers’ money into IFAD. We have to do that on the basis of where there is impact. Even more importantly, however, clients put their own resources into IFAD operations. Farmers and rural communities are contributing their labour, their time, their efforts, their land into bringing about the delivery of impact.

We all have to agree that the delivery of impact is the priority and we all have to agree on what that impact looks like. Importantly, we all have to have confidence in the evidence that is being presented to demonstrate that impact. The ability of an independent, robust and credible entity to provide this evidence is the glue that is holding all these relationships together. It is mutual accountability that is based on evidence. Anybody with skin in the game in IFAD has to have faith in the credibility of the products that IOE generates, otherwise the whole thing falls apart, because we are not willing to enter into this pact of trust that holds it all together.

What hand over insights and messages do you feel like leaving to your successor vis-à-vis the role, function and impact of evaluation at IFAD?

The evaluation function that IOE performs is something that Executive Board members are very proud of and should continue to be very proud of. There was always rigour in the way in which IOE conducted its work. Where IOE has now really come into its own, in recent years, is in the relationship-building and cost-effectiveness of the evaluation function. IOE has really focused down on where the resources need to be deployed the most. IOE has also built internal capacity. This is a great way to save resources and to make sure that the learning, the cross-fertilization and the capacity stay in-house.

The one area where I would like to see a little bit more progress, in the coming years, is using that capacity to reach out and invest in the food systems and agriculture data community at large. IOE could invest in the ecosystem of agriculture statistics organizations and of the M&E world related to food systems, to build the capacity of the ecosystem as a whole. That then reinforces good policy and broader investments in client countries.

I have found working with IOE a real pleasure. They’ve got the model right in terms of the balance between rigor, independence, engagement and all of those things that you want from an evaluation function with, quite frankly, a bit of humanity and understanding. There is always a sense that the findings of IOE have been put through the prism of what is an acceptable recommendation that reflects where Management is at and the specific context in which each recommendation is made. That’s what makes IOE a really solid evaluation outfit, one which delivers work to the true benefit of IFAD.

Are there any final thoughts that you would wish to share with our readers?

There are bumpy times ahead. We are in a poly crisis, with growing conflict, climate impacts and markets all over the place. I don’t predict the next few years to be the ‘great leap forward’ in terms of impact in building sustainable food systems. We are all firefighting at the moment on a number of fronts, which is a real shame.

All of the knowledge, perspective, operational wisdom, humanity, communication skills and relationship building that IOE has heavily invested in over the last few years is going to be really needed. We need IOE even more, we need more evidence. It could be a rocky few years ahead, so I hope that the Office continues to deliver and that its morale continues to be high. I wish the whole IOE team the very best of luck.

Divisional Retreat based on ‘Lego Serious Play’ Method

Who says that work can’t be playful? Evidence says that it certainly can, as the IOE team found out during their staff retreat. Held at Villa Tuscolana on 2627 October, the retreat brought back childhood memories for many team members, suddenly tasked with visually building their thoughts through the use of LEGO.

The overarching objectives of the retreat were to build alignment on key issues and move towards actions, whilst improving team spirit. To achieve this goal, the overarching approach was to build an aspirational narrative (i.e. “what does ‘good’ look like for this team if we are working at our best”) and link that through to the actions required to achieve it. Key activities included skills-building sessions, solutions-focused exercises of articulating the team’s aspirations, and identifying challenges to achieving the aspirations.

The LEGO Serious Play (LSP) method was at the core of the retreat. LSP is a facilitated thinking, communication and problem-solving method for use with groups. It uses LEGO bricks to build stories about the intangible ideas people work with in their business and brings them into the physical world through the use of metaphor and play. LSP taps into new neuroscience discoveries and the link between the hand and the brain.

Richard Gold facilitated the retreat.