Consultant – Final Evaluation for PRODUCT project: Vietnam

ACTION ON POVERTY IN VIETNAM

Consultant – Final Evaluation for PRODUCT project: Vietnam

Effective date:             May 2024

Work location:             Based in Hanoi, Vietnam with frequent travels to the fields

Reports to:                  Product Manager

Application

All Applications must be sent to HR Department ([email protected] [1]) by 10 May 2024.  

Please limit the CV included with the Expression of interest to a maximum of three (3) pages.

Project Background

In 2021, Action on Poverty (AOP) in Vietnam completed its three-year project named “Building Capacity and Access for Resilient Communities”. The learnings from the project recommended that i) it is an urgent need for local communities, especially women, to equip with digital knowledge and skills to apply digital technologies in their daily lives and businesses so that they could not only cope with the travel-restricted situation such as COVID-19 pandemic and or natural disasters but also actively participate in an emerging business trend, that is ecommerce/online trading; ii) it is necessary to engage more market actors, to mobilise resources from private sectors and government agencies, and proactively involve in advocacy efforts at the national level; (iii) and most importantly, product approach would be applied to scale up successful models at community and used for policy advocacy at the national level.

To address the emerging issues, AOP in Vietnam implemented a project Promoting Rural Opportunities with Digital technologies and Upscaling Community-based Tourism (PRODUCT) that aimed to improve the resilience, and social and economic status of women and ethnic minorities living in rural and mountainous areas of Vietnam from 2021 to 2024. To achieve this goal, AOP in Vietnam determined the following outcomes:

(1) Outcome 1 – Social and economic status of rural women is improved thanks to access to financial services, business models, and the use of digital technologies: This includes increased knowledge and skills of ethnic minority women in financial literacy, business, digital tools, stronger networks, influencing their decision-making in the family, and the community. Key activities include: developing and implementing ICT4WE products including platform, tools to provide training on financial literacy, digital literacy, business and e-commerce related skills, and reinforcing the organisation of existing products and design, implementing new products to promote Financial Inclusion for local women and men.

(2) Outcome 2 – Established inclusive business partnerships on CBT and rural products: This includes improved incomes and more jobs created for women through viable private sector-led service delivery models in rural areas, committed partnership with private partners and other market actors of tourism and agriculture, promoting increased employment opportunities for women and female entrepreneurship. Key activities include reinforcing existing CBT model following a product-based approach, scaling up successful models, establishing CBT alliances, and private sector engagement.

(3) Outcome 3 - Better and inclusive policies and national programs on CBT and ethnic minorities: This includes changes in policies, local and national programs, guidelines on CBT, and representatives of women at local and national dialogues on policy to promote evaluation framework socially inclusive and responsibly sound tourism and for ethnic minorities. Key activities include conducting research and surveys, policy dialogues, networking, documenting and disseminating policy recommendations.

Under the project, various activities were implemented to support thousands of low-income people in Hoa Binh, Dien Bien, Phu Tho, Son La, Thai Nguyen, Lai Chau, Ha Tinh and Soc Trang provinces. In these areas, AOP Vietnam continued to reinforce its expertise in the following models: AOP microfinance (APM), inclusive community-based tourism (CBT).

Scope and Focus

The final evaluation will scope the PRODUCT project for the period of 2021-2024 in its impact areas including Hoa Binh[1], Dien Bien, Son La[2], and Lai Chau. The evaluation will provide (1) a comprehensive review of the technical activities, partnerships, approaches implemented by AOP in Vietnam throughout the PRODUCT and (2) a report to ANCP funding donor (DFAT) by the end of June 2024.

This final evaluation will focus on, but not exhaustive to:

• Assessing to what extends the project(s) met the stated goals and objectives of at all levels (output, outcome, impact);
• Reviewing the effectiveness of the technical approach including APM and CBT models
• Developing of the overarching lessons learned from the project and opportunities for AOP in the next 3-year programming

In addition, it is required to emphasise on a participatory and collaborative approach with gender, ethnicity and disabilities concerns. To be able to report to DFAT as part of ANCP’s requirement at the end- of – term period, the final evaluation with focus on:

• Verification and assessment of implementation and results
• Identification of project’s successes in order to create replicability
• Actions necessary for consolidation and sustainability of results
• Emphasis on Lessons learned
• Improve design of other future projects based on the deeds of stakeholders and beneficiaries

Project implementation is monitored through on-site collection of data, regular project reports from partners as well as through periodic site visits.  In line with AOP practice we also undertaken more comprehensive and qualitative analysis by undertaking evaluations, either internally or led by an external consultant in order to draw broader outcome and impact conclusions and validate results.

Evaluation Objectives

The objective of this evaluation, as mentioned earlier partially, will be to review and evaluate the project results to date against the agreed results framework indicators, approach/model, lessons learned and propose recommendations for adaptive management and development. The project evaluation will refer to the OECD evaluation criteria: Relevance, Coherence, Effectiveness, Efficiency, Equity, Impact, and Sustainability, by reviewing available data and documentation, conducting in-person and remote interviews, and triangulating and verifying information as required.

The overall evaluation objectives are: 

1. To determine efficiency, effectiveness, equity, relevance, and sustainability of the results towards project targets including CBT and APM model in targeted sites;
2. To assess the Value for Money of the project (efficiency, effectiveness, economy, partnerships and equity), particularly of the CBT model;
3. To Identify lessons learned and best practices from the project which could be applied to improve the project;
4. To provide information to further plan and improve the project.

To achieve the evaluation objectives, we will engage a suitably qualified person to undertake and coordinate the work. This person will be expected to follow a participatory and consultative approach ensuring close engagement with relevant stakeholders that will include government counterparts, research institutions, and project staff.  We envisage that members of the country program team will participate in the evaluation as a learning exercise in itself. 

The consultant will be expected to facilitate evaluation tools with input from the team. The evaluation should correlate with the five evaluation criteria outlined in AOP’s design, monitoring, evaluation and learning framework and defined terms:

1. Relevance: whether the objectives of the project are in accordance to locally defined needs and priorities as well as to partner government priorities.
2. Effectiveness: the extent to which project objectives are being achieved or can be achieved
3. Ownership and sustainability:  The likely ability of project activities to continue to deliver benefits for an extended period after completion. Potential for the project to be owned by smallholder farmers themselves.  The resilience to risk of the net benefit flows over time.
4. Equity: how the project involves and benefits different genders, people with disability and other often marginalised people throughout the project cycle, including the appropriateness of design and monitoring systems against project and program objectives to date.
5. Efficiency/ Value for Money: to determine the extent to which the project has managed resources to balance economy, efficiency and effectiveness in ways that are both equitable and ethical.

The conclusion should be comprehensive and balanced, and highlight the strengths, weaknesses and emerging outcomes of the project, with a special focus on what and how to continue the PRODUCT project in the next phase considering possible integration of interventions in Climate Change and Adaption and Disability.

Methodology

The evaluation will use a mixed methodology approach that endeavours to draw together a mix of qualitative and quantitative conclusions.

Primary research will be conducted in Hoa Binh, Dien Bien, Son La, and Lai Chau where the project is being implemented in Vietnam. The specific sites will be agreed upon with the project team and partners before commencing the evaluation.  A detailed methodology will be agreed between the consultant and AOP, in country partners before evaluation activities commence.

Evaluation tools will need to be developed and must be able to undertake both quantitative and qualitative data and be appropriate to be administered at the community level. The proposed methodology should be clear and relevant to the context of the project location as well as the specific indicators in the project design. Please note that tools may require to be translated into tested before data collection.

Key informants:

• Households participating in projects
• Project partners, including government, civil society, and business partners
• Enterprises and cooperatives that were established during the project implementation
• Local authorities and mass organisations as needed
• AOP program management team
• Project management units
• Others as needed

Key research questions:

1. To what extent has each project achieved its planned objectives? Any positive/negative changes brought by the projects out of plans? Why did those changes happen?
2. How consistent was the project with AOP and donor’s strategy? How did the project contribute to AOP and donor’s strategic objectives?
3. How have possible collaborations been utilised and how have these contributed to the project results? Recommendations to foster more shared value partnerships in the projects and alike.
4. To what extent can the project activity implementation considered to be cost-efficient, while not compromising quality? Were there opportunities within projects to create more value with the available budget or to reduce costs without compromising quality and reach?
5. Analyse the impacts of unforeseen risks such as COVID-19 or natural disasters/climate change and the organisation’s responses. What lessons can be brought to the organisation’s project management?
6. What is the satisfaction level of project beneficiaries with the interventions and results? Also analyse the feedback of other key stakeholders regarding AOP’s project management.
7. How was the participation of stakeholders during all project phases?
8. To what extent was inclusion including gender inclusion, considered and achieved in the projects?
9. Are the benefits brought by the project likely to continue after the project has been completed and no more donor funding is available? Identify opportunities for replication and scaling-up of the project. Include examples of replication if any.
10. What are the conclusions in relevance, adaptiveness, effectiveness, efficiency, sustainability, and replicability of each project? Identify lessons to improve the performance and impacts of the projects?

Sampling:

Quantitative method sampling:

The method proposed by the consultants should meet the level of satisfaction in statistics so the results of the sample can be representative and manipulated to the study frame.

The sample size, proposed by the consultants, will be agreed upon between AOP and consultants during the negotiation process and finalized in the study plan which is a critical part of the technical and financial proposal. Please refer to the project’s performance indicators to develop the quantitative evaluation approach (Annex 1).

Qualitative method sampling:

The method of selecting how many respondents, who and where they should be presented will be proposed by the consultant and based on the nature of the PRODUCT’s key interventions and target sites.

Expected Deliverables and Reporting Format

1. An evaluation framework clearly indicating how the consultant plans to reach the objectives of the evaluation and, how the overall exercise will be conducted
2. A detailed work program clearly identifying the activities to be undertaken during the evaluation
3. A draft report that meets the standards and requirements as agreed upon between AOP, its partners and the consultant.  The reports will undergo a comprehensive quality appraisal and approval progress by AOP and its partners before it is accepted as meeting the agreed standards
4. A final evaluation report reflecting comments received from selected AOP and partner staff and other stakeholders
5. All materials used for data collection and management, e.g. questionnaires, data entry tools, etc., and
6. All full clean dataset used for data analysis as well as the output tables and syntaxes created during the process.
7. A powerpoint presentation on key findings and recommendations

Required Qualifications

• A degree with relevant post-graduate studies in Economics, Rural Economics, Development Studies, or related field
• Prior experience in undertaking microfinance, community-based tourism, ethnicity, baseline studies, project monitoring, evaluations and documentation of lessons learnt in the country.Demonstrated experience in communicating stories, production of case studies and best practicesDemonstrated experience coordinating and working with partners
• Good quantitative and qualitative data analysis skills. Excellent analytical and reporting writing and presentation skills

Tentative Timeline

Please note that the evaluation will be used as a learning exercise. Therefore, the methodology that the consultant should take this into consideration, especially mindful of how partner staff can meaningfully participate in data collection and analysis process.

Management and Reporting Arrangements

The consultant(s) will report to the PRODUCT’s manager with support from MEL Coordinator.  An internal consultative group will be established to provide oversight to the implementation of the study and to review findings.

All reports must:

• be written in English
• be provided in an electronic format (Microsoft Word; Times New Roman 12; Normal page margin, line space 1.15)
• be no more than 19 pages

The annex (excluded from the report) must include data collection materials, list of participants, raw and cleaned data, photographs, actual evaluation timeline, consent language.

Accountabilities and Responsibilities

AOP’s International Program Director will support the PRODUCT manager and MEL Coordinator to coordinate the evaluation with support from the in-country project team.  AOP will have overall responsibility for the evaluation and in the final approval of all deliverables.  The AOP Monitoring, Evaluation and Learning Coordinator will support the in-country review process in consultation with the team.

Annex 1- Project indicators