UNHCR Evaluation Office
Annual Report 2022

Evaluating what works for the people we serve

2022 in review:

  • 19 evaluations completed - a record high since 2018. Overall evaluation quality remained high. Evaluation expenditure remained stable at an estimated 0.17% of UNHCR's budget

  • Initiated 20 new evaluations – an important number being commissioned by Regional Bureaus and Country Offices

  • Adopted a new Evaluation Policy, which presents a forward-looking vision for the evaluation function in UNHCR and reflects organization-wide reforms related to regionalization and decentralization and changes in roles, responsibilities and authorities

  • Continued to expand our regional presence to better support the decentralization process. From one position in the Americas in 2020, as we go into 2023 we now have five regional Senior Evaluation Officer positions established - in East and Horn of Africa and Great Lakes; West and Central Africa; Asia and the Pacific, Middle East and Northern Africa and the Americas regions

  • Clear uptake of evaluation learning in new organizational policies and strategies with increasing demand for country strategy evaluations to inform multi-year strategic plans.

2022 was marked by significant instability. Many crises have unfolded in the past year, hitting vulnerable communities the hardest, including in Ukraine, Afghanistan, Ethiopia and – at the time this report is being written – southern Turkey and northern Syria.


In light of these unprecedented times, the role of evaluation as a critical tool to inform evidence-based decision-making is more important than ever. Through evaluation, we can see what works and what doesn’t, shine a light on good practice, and take corrective action where needed.


Despite the challenges of the last year, the Evaluation Office at UNHCR remained committed to its vision of supporting UNHCR with evidence, data, and learning for better results for the people we serve. Evaluation is a critical part of any results-based management (RBM) system, providing accountability and strategic analysis to inform agile management and decision-making.


The new Evaluation Policy, approved by the High Commissioner in October 2022, creates an enabling environment for an organization-wide approach to boost the use of evaluation as a routine management tool at all levels. One of the main changes in the policy is the establishment of coverage targets for different types of evaluations. The new policy introduces a requirement for management to report on the implementation of accepted recommendations starting from 2023.


As we move into 2023, we would like to take the time to share with you the progress we have made last year towards achieving our strategic goals under the four pillars of our work:


  1. Increasing evaluation coverage and quality
  2. Improving the relevance and utilization of evaluation findings
  3. Building capacity to undertake and use evaluations
  4. Strengthening linkages with other functions
1. Increasing evaluation coverage and quality
The number of evaluations has grown steadily from 4 evaluations completed in 2016
to 19 evaluations (13 centralized and 6 decentralized) in 2022.
Evaluations Geographical Coverage 2018-2022​
Evaluations Coverage of Strategic Directions Results Areas
2018-2022​
External assessments have shown that the quality of our evaluations has improved from an aggregate rating of 58% in 2018-19 to 70% in 2021. The quality assessment for evaluations completed in 2022 is underway.

In order to ensure that UNHCR’s evaluations consistently meet UN quality standards, the Evaluation Office has revised its external quality assurance guidance and tools. The revision of the evaluation quality assurance mechanism is expected to benefit all evaluations conducted by UNHCR, including those commissioned at the regional and country level.

In terms of coverage, and in line with the new Evaluation Policy, all major policies, themes, strategic results areas, and geographies of the organization’s operational work will now be evaluated at least once over a 5 to 10-year period.  In addition, a recent commitment has been made by Evaluation Directors from across the UN system – to consider systematically in our evaluations, how climate change impacts the people we serve, and how displacement impacts the environment.

OVERVIEW OF EVALUATIONS COMPLETED IN 2022

2. Improving the relevance and utilization of evaluation findings 


Evaluations bring value when they inform thinking and programming, provide assurance of alignment with the organization’s strategic directions, as well as generate new knowledge and evidence that, in some cases, may be contrary to commonly held views. Real change can sometimes only be visible several years after the evaluation has been completed.


Some examples of how evaluations are having impact include:


  • The newly revised Policy on Emergency Preparedness and Response was informed by the lessons distilled from the recent evaluations of UNHCR's L3 emergency responses. The evaluation findings provided lessons learned on the design and implementation of policies, guidance and systems used in large-scale rapid-onset emergencies.


UNHCR's Global Learning and Development Center has made notable progress toward strengthening a learning culture within UNHCR. It has reduced the number of its in-person courses and workshops, focusing more on in-the-flow-of-work informal learning and development activities. Regional Talent and Development teams are being established in the Regional Bureaux. These and other changes were informed by the Evaluation of UNHCR's Approach to Learning and Development for Workforce and Partners (2020)

Evaluations are helping UNHCR to capitalize on innovation and new opportunities but also to navigate through difficult times. Clearly one of the biggest challenges faced over 2021 - 2022 continued to be COVID-19 and a range of evaluations provided some valuable lessons.

Our joint evaluation, which looked at the protection of the rights of refugees during COVID-19, was carried out with the OECD DAC and the Governments of Colombia, Uganda and Finland. Recommendations were made, not only to UNHCR, but also to States – calling, for example, for Governments to make exceptions for asylum seekers in future large-scale emergencies and pandemics, and to leave their borders open.

We continue to invest in strategic communications. In 2022, we scaled up the use of global Brown Bag lunches for major evaluations. The Brown Bags bring together on average, over 100 participants from UNHCR offices around the globe and lead to useful knowledge exchange.

In 2022, we had over 11 700 visitors to our web page, with some of our reports reaching over 800 hits.
3. Capacity Building
The Evaluation Office remains committed to partnering with UNHCR management to develop internal capacity for improved commissioning and utilization of evaluations and other robust sources of evidence in their planning and priority setting.

To support the scale-up of our work, particularly at decentralized levels, we are increasing our presence on the ground. Through dedicated regional positions, evaluations are having an enhanced contribution to multi-year operational planning and contributing to Regional Bureau oversight. A global M&E community of practice, co-championed by the Division of Strategic Planning and the Evaluation Office, now offers mentoring support for 217 colleagues around the world. At the country operations level there are now a dozen officers across the countries we work in who have an explicit role in managing and supporting evaluation activities.

We continue to offer on-the-job learning and coaching for evaluation managers. Alongside experiential learning and coaching, flexible learning approaches such as a virtual monitoring and evaluation community of practice for the UNHCR workforce, have been developed, providing experience-sharing opportunities, including during the global “learn@work” week and the ‘Welcome Days’, a 3-day virtual onboarding for new UNHCR staff, led by Department of Human Resources and Global Learning and Development Center. Finally, we delivered an innovative ‘PeerXchange’ session with UNHCR decentralized evaluation managers to enable experience sharing and draw lessons to feed the update of UNHCR’s evaluation operational guidelines.
4. Linkages: evaluation, results-based management and oversight

The Evaluation Office continued to collaborate with operations, regional bureaux and headquarters divisions to support the monitoring and evaluation component of multi-year strategies which form an integral part of COMPASS, the UNHCR results-based management system for planning and budgeting. This was undertaken in several ways, including by supporting multi-functional teams in the development of regional and country-level monitoring and evaluation plans; contributing to COMPASS guidance and administrative instructions; providing resources for online training; revising the online course on multi-year monitoring and evaluation plans; and co-delivering virtual sessions to over 200 members of the monitoring and evaluation community of practice.


In 2022, a new consolidated recommendation tracker was designed and launched by the IGO. Starting from 2023, management will be able to see all “open recommendations” coming from audits and evaluations in a user-friendly dashboard intended to prompt timely follow-up action. One of the features of the new UNHCR evaluation policy includes a requirement for management reporting on evaluation recommendations for a two-year period after the completion of evaluations.

External Partnerships - The Evaluation Office plays an important role and engages with evaluation standard-setting efforts under the auspices of the UN Evaluation Group (UNEG) - the Head of Evaluation is Vice-Chair of the Executive Steering Group and EvO staff contribute to a number of ongoing working groups and system-wide evaluation efforts. The Head of Evaluation has been asked to lead a UNEG/OECD-DAC peer review of UNFPA's evaluation function during the first half of 2023.
5. Selected Evaluation Highlights

UNHCR's Repatriation Programmes and Activities

The 2022 Evaluation of UNHCR's Repatriation Programmes and Activities found that reflecting the voice and aspirations of refugees and better capturing the complexity of their decision-making on return remains key in both the design and implementation of UNHCR’s voluntary repatriation activities.


UNHCR’s operating models and guidance in support of voluntary repatriation are outdated and lack adaptiveness to complex environments. UNHCR’s effectiveness in the coordination of repatriation operations is widely recognized. Long-term reintegration support, however, still poses significant challenges. Challenges remain in UNHCR’s consistent and systematic collection and sharing of operations-level data on repatriation and reintegration activities. Area-based approaches offer promising avenues for reintegration programming, as they can facilitate multi-stakeholder engagement and increase the sustainability of returns.


Informed by the evaluation, UNHCR has updated its guidance documents on repatriation, including the Handbook for Repatriation and Reintegration Activities (2004) and the Voluntary Repatriation handbook for International protection (1996).


Read Full Report

Asylum Capacity Development

The Evaluation of UNHCR’s support to strengthen national asylum systems found that UNHCR has made good progress over the last few years, with its work on asylum capacity development becoming more comprehensive and developmental, in recognition that supporting national systems development is inherently multidimensional and complex – political, as well as technical, institutional as well as individual – and involving multiple actors, requiring sustained support over a long period of time.


UNHCR, though, still has some way to go. In some situations, the assumption has been made that substituting or adding capacity, for example, embedding contract staff in Government agencies has a capacity development effect or outcome.


The strength of UNHCR’s advocacy for asylum capacity development greatly increases when part of an intergovernmental process. The evaluation team found that the organization is good at using political openings but less adept at building or joining networks or partnerships that can exert more influence on asylum capacity development than by us acting alone.


UNHCR accepted all the recommendations and developed a full set of actions in the management response approved in November 2022 to be carried out over the next two years, including consolidating a capacity development approach, increasing engagement with partners, ensuring capacitated staff, managing risks and increasing the evidence base.


Read Full Report

I want to convey our appreciation for the very consultative process and the excellent recommendations that came out of the Asylum Capacity Development evaluation. Very relevant recommendations on a more strategic and holistic view to asylum looking at root causes of backlogs including galvanizing opportunities to work with other actors in advocating for legal pathways for admission for those not needing international protection which in turn alleviates the asylum system for those who really needed it.

— UNHCR Senior Protection Coordinator
UNHCR’s preparedness and response to the COVID-19 pandemic: what the evaluative evidence says

Evaluating UNHCR’s response to COVID-19 has been critical to understanding how the organization mitigated the negative impacts of the pandemic on displaced and stateless persons. Three evaluative exercises have been undertaken: a synthesis of evidence on UNHCR’s COVID-19 response drawn from 27 UNHCR evaluations over the period 2020-22; and two joint evaluations – one led by UNHCR, the Governments of Colombia, Finland and Uganda, the Active Learning Network for Accountability and Performance, and the COVID-19 Global Evaluation Coalition focused on the protection of the rights of refugees during the COVID-19 pandemic; and the other, an inter-agency humanitarian evaluation of the COVID-Global Humanitarian Response Plan, where UNHCR engaged as part of the management group.


These evaluations revealed how protection actors made extraordinary efforts in support of refugee rights in the face of an unprecedented global health emergency. Despite many challenges, the humanitarian community provided a safety net for many millions of people who otherwise would have likely gone without assistance. Furthermore, working under difficult circumstances, UNHCR and other humanitarian agencies, particularly national and local NGOs, showed remarkable adaptability, courage and tenacity in delivering a coordinated response at an unprecedented scale. There was consensus that refugees and refugee-led organizations played a key part in the response.


Despite these efforts, and sustained contact with persons UNHCR serves, under the pressures of remote delivery, not all the needs of vulnerable groups were met. Registration and refugee status determination were challenging to carry out, while resettlement met with, not surprisingly, major barriers. The remote approach to delivery placed those with specific needs at greater risk. Delicate or sensitive issues such as trauma, mental health issues, and sexual and gender-based violence were difficult to address remotely, and it was also harder to assess participants’ well-being via video or telephone calls. However, in many other areas, the remote approach stimulated innovation, allowing the maintenance of many services which previously relied on face-to-face contact. It has also created new delivery modalities that could be used in future emergencies (e.g., child protection and gender-based violence case management and mental health and psychosocial support, telehealth for health responses).


Overall, the commitment by UNHCR to ‘Stay and Deliver’ to those in need was identified to be integral to the organizational DNA, recognized by partners and valued by the people UNHCR serves. The double burden was for staff, challenging, navigating operational delivery amid pandemic conditions, but the lessons from the evaluations are being taken by the organization as it refines and adapts its approaches.

UNHCR’s Response to Multiple Emergencies in the Central Sahel Region: Burkina Faso, Niger, Mali

Faced with unprecedented challenges, UNHCR was largely effective in responding to the needs of diverse populations in accessible areas of the Central Sahel Region. Stakeholders positively viewed the scale-up of the response following the emergency declaration, corresponding to an increased budget, protection assistance capacity, and partner coordination. However, UNHCR’s response faced several major challenges, including insufficient resources for the scale of the response needed, the insecurity and volatility of the Sahel context, and the COVID-19 pandemic, which led UNHCR to declare a global L2 emergency one month after the Sahel declaration.


Significant differences in coverage emerged across countries per population group. There was a lack of consistency in defining UNHCR’s responsibilities toward internally displaced people (IDPs). The evaluation found that the IDP Policy and the Emergency Policy lack concrete guidance on how to achieve goals or mechanisms to evaluate effectiveness, offering broad objectives rather than specific directives.


The evaluation recommends the Sahel operations to be guided by a multi-year regional strategy considering the scale and trend of the Sahel crisis. The Sahel operations must be supported by systems and tools that facilitate and promote the roll-out, adoption, and implementation of emergency and IDP policies. The evaluation further recommends that emergency declarations must be accompanied by a strengthening of information systems, monitoring and evaluation, and knowledge management adapted to the operational environment.


Read Full Report

Mexico Country Strategy Evaluation

Evaluation of Mexico country operation found that UNHCR has been instrumental in expanding the protection space and contributing to the perception of Mexico as a host country for populations in need of international protection (rather than just one of transit and emigration). UNHCR and its partners have helped transform the national asylum system; there is broad consensus on the improvements achieved in access, registration, refugee processing and status determination, and local integration, despite systemic barriers and persistent challenges.


The evaluation recommends reinforcing coordination and regional analysis of population movements, capitalizing on existing data generation mechanisms and networks to drive evidence-based government and UN plans and responses. It also recommends promoting the strategic vision of the role to be played by the national asylum system among national institutions, within a context of intense mixed flows, in order to promote new migratory alternatives and preserve and broaden progress made in international protection and local integration.


Read Full Report

UNHCR’s Approach to Accountability to Affected People

Synthesis of Evaluative Evidence

This report consolidates findings from independent evaluations for evidence of UNHCR’s multi-faceted approaches to Accountability to Affected People. This report aims to contribute to learning and reflection on a central tenet of UNHCR’s protection role and responsibilities: the participation by, and accountability to, the people UNHCR serves.


The evaluations find there is prolific use of a varied range of mechanisms and instruments by UNHCR operations to engage with and consult the beneficiary populations. Evaluations identify UNHCR’s wide presence in crisis-affected locations, and the operational need to engage directly with the people it serves, as important enabling factors for participatory activities. The evaluations revealed uneven levels of participatory practices in different operations and found limited evidence that the results of the participatory mechanisms resulted in changes in programming, implementation or prioritization. At the same time, evaluations note that UNHCR operations have made progress in establishing communication and information channels for affected populations, including those tailored to specific groups. UNHCR’s efforts and role are widely recognized by partners and counterparts.


Read the Full Report

6.The New Evaluation Policy

Our new Evaluation Policy builds on the Evaluation Policy of 2016. The vision going forwards is that:


“Evaluation informs choices made at all levels of the organization in strategic planning, programming and decision-making based on timely, credible and impartial evidence.”


The Policy:


  • Outlines clear operational model for the evaluation function – global evaluations carried out by the central Evaluation Office, and thematic, regional and country-level evaluations carried out by divisions, bureaux and country offices.
  • Clarifies the role of persons of concern in the evaluation process, from a selection of topics to design, implementation and use 
  • Establishes coverage norms for evaluation at global, regional and country levels, with all major policies / strategic results areas to be evaluated at least once 10 years; and all country offices to be covered by some form of evaluation over a strategic planning cycle or every five years. The norms will be phased in over the life of the policy 
  • Clarifies the role, accountabilities and authorities of the High Commissioner, Senior Executive Team, Directors of Divisions, Bureaux and Representatives to advocate for, resource, use and follow-up on evaluations 
  • Outlines the potential sources of funding for evaluation for different types of evaluation. 
  • Incorporates risk management concepts and processes into evaluation 
  • Simplifies the typology of evaluation into the main types which are defined by the subject rather than the methodology. 
7. Year Ahead

In 2023, we will be evaluating UNHCR’s work in a number of important global strategic areas, such as engagement with internally displaced people, gender-based violence, decentralization and regionalization.


In addition, we plan to start the following country strategy evaluations in 2023:


Country Strategy Evaluations:

  • Ecuador
  • Mozambique
  • Mauritania
  • Tajikistan
  • Peru
  • Honduras
  • Mali

L3 Emergency response countries hosting refugees from Ukraine (Hungary, Moldova, Poland, Romania, and Slovakia) and some form of evaluative exercise inside Ukraine and for the Turkey and Syria earthquake response.


Examples of evaluations to be commissioned by Divisions/Regional Bureaus/Country Offices:


  • DFAM: Green Financing Facility
  • Rwanda: Climate-smart agriculture and market development for enhancing livelihoods of refugees and their host communities
  • Burkina Faso: Cash-Based Interventions for Livelihood/Socioeconomic Inclusion
  • Southern Africa: Joint Project on Mixed Migration
  • Philippines: Protection and Solutions Programme for Internally Displaced Persons in Mindanao

We are currently working on developing an Evaluation Strategy, which will support the implementation of the new policy. Our rolling centralized evaluation work plan 2024/25 will be prepared, approved and published soon.
I am pleased to see the growing interest in UNHCR commissioned evaluation – both internally and externally. It points to the relevance that evaluation can have in helping the organization, at different levels, to prioritize and adapt as well as to be more visibly accountable for the way in which the organization uses its resources.

As we approach the second Global Refugee Forum in 2023, it is an opportunity for us all to take stock in a collaborative way of the collective achievements of the UN, Government, resource partners and civil society actors towards commitments made under the Global Compact. The Evaluation Office remains at the service of the mandate of the organization and the refugees, internally displaced, host communities and stateless that we are committed to support and protect.

Lori Bell

Head of the Evaluation Office

Evaluation Office
Team at the retreat
in geneva,
December 2022
To learn more about the Evaluation Office at UNHCR, visit our website or contact us at hqevaser@unhcr.org
Made on
Tilda