Terms of Reference for
Final Evaluation of the Shelter Program in Haiti
Final Evaluation of the Shelter Program in Haiti
- Introduction
1.1. Reasons/Rationale
The shelter program of Canadian Red Cross/CRC in Haiti was implemented from March/April 2010 to September 2012. In line with the CRCS evaluation policy, a final evaluation of the program will be undertaken for accountability and learning purposes.
1.2. Purpose
The purpose of this review is as follows:
a) To assess achievements of the program, in terms of effectiveness (achievements of the program’s outcomes), efficiency (cost effectiveness and timeliness), partnership/coordination with implementing partners, appropriateness/sustainability/impact (including disaster risk reduction and environmental sustainability), and targeting/coverage (geographic targeting versus needs, allocation of budget based on needs).
b) To assess achievements of the program in integrating cross-cutting issues in the overall program cycle management, namely gender, capacity building of the targeted communities, the role of CRCS in advocacy on shelter related issues in Haiti, as well as protection of vulnerable beneficiaries.
c) To draw lessons learned for design and implementation of CRC shelter program in the future based on the findings in the points (a) and (b).
a) To assess achievements of the program, in terms of effectiveness (achievements of the program’s outcomes), efficiency (cost effectiveness and timeliness), partnership/coordination with implementing partners, appropriateness/sustainability/impact (including disaster risk reduction and environmental sustainability), and targeting/coverage (geographic targeting versus needs, allocation of budget based on needs).
b) To assess achievements of the program in integrating cross-cutting issues in the overall program cycle management, namely gender, capacity building of the targeted communities, the role of CRCS in advocacy on shelter related issues in Haiti, as well as protection of vulnerable beneficiaries.
c) To draw lessons learned for design and implementation of CRC shelter program in the future based on the findings in the points (a) and (b).
1.3. Estimated dates
15 November – 15 December 2012 for the field visit and the final report by January 31, 2013.
1.4. Location of consultancy
Field visits to Port-au-Prince, Jacmel and Leogane in Haiti and interviews with stakeholders in Montreal and Ottawa, Canada. Telephone calls to Germany regarding the Diakonie Project may also be necessary.
1.5. Users and Intended Use:
Users:
Director of Haiti Operations and senior management at CRC National Office, Haiti Program Managers, Haiti Country Representative and senior management team, Construction Management Unit team, IFRC Haiti, Haitian Red Cross
Director of Haiti Operations and senior management at CRC National Office, Haiti Program Managers, Haiti Country Representative and senior management team, Construction Management Unit team, IFRC Haiti, Haitian Red Cross
Intended Use:
• To document good practices for the CRC Haiti team
• To share key lessons learned to Movement partners – especially with regards to coordination and communication
• To transfer lessons learned to other on-going Haiti construction projects.
• To report outcomes/impact and be accountable to senior management, and other stakeholders
• To document good practices for the CRC Haiti team
• To share key lessons learned to Movement partners – especially with regards to coordination and communication
• To transfer lessons learned to other on-going Haiti construction projects.
• To report outcomes/impact and be accountable to senior management, and other stakeholders
Users:
Knowledge Management focal points for Haiti Program and CRC-Ottawa, (i.e.Planning Quality and Learning Team); Senior Shelter Advisor
Knowledge Management focal points for Haiti Program and CRC-Ottawa, (i.e.Planning Quality and Learning Team); Senior Shelter Advisor
Intended Use:
• To document and collect good practices, approaches and tools of the shelter program for internal and external knowledge sharing.
• To document and collect good practices, approaches and tools of the shelter program for internal and external knowledge sharing.
Users:
NAFC and Senior Management of CRC
NAFC and Senior Management of CRC
Intended Use:
• To obtain evidence-based information about achievements and gaps in the shelter program and how the program results can improve future CRC shelter and construction programs.
• To obtain evidence-based information about achievements and gaps in the shelter program and how the program results can improve future CRC shelter and construction programs.
- Background of the Program
Needs
The earthquake that occurred on 12 January 2010 caused significant damage to housing and public infrastructure, with the resulting need for emergency and transitional shelter and longer-term reconstruction. Provisional estimates of the usability of damaged buildings suggested that 60% of buildings could be repaired, 28% would require retro-fitting and 12% would need to be demolished and reconstructed (shelter/NFI cluster, March 28 2010).
The earthquake that occurred on 12 January 2010 caused significant damage to housing and public infrastructure, with the resulting need for emergency and transitional shelter and longer-term reconstruction. Provisional estimates of the usability of damaged buildings suggested that 60% of buildings could be repaired, 28% would require retro-fitting and 12% would need to be demolished and reconstructed (shelter/NFI cluster, March 28 2010).
Commitment
CRC, with CIDA support, committed funds for the fabrication and implementation of up to 15,000 Shelters in Haiti (out of a Movement commitment of 30,000). The CRC agreed to build approximately 7,500 shelters through its own programming and up to 7,500 through IFRC programming. In terms of financing, the CRC construction budget for 7,500 shelters was established at $63 million out of which $19.15 million was a CIDA IHA Grant Agreement contribution. It was agreed that the CRC construction should be focused on Leogane and the South East Department (Jacmel).
CRC, with CIDA support, committed funds for the fabrication and implementation of up to 15,000 Shelters in Haiti (out of a Movement commitment of 30,000). The CRC agreed to build approximately 7,500 shelters through its own programming and up to 7,500 through IFRC programming. In terms of financing, the CRC construction budget for 7,500 shelters was established at $63 million out of which $19.15 million was a CIDA IHA Grant Agreement contribution. It was agreed that the CRC construction should be focused on Leogane and the South East Department (Jacmel).
The key outcomes of the program were defined as follows:
Ultimate Outcome: Improved housing condition of beneficiaries of CRC shelter in Jacmel and Leogane
Intermediate/Medium-Term Outcome 1 Strengthened relationship and participation of community members in planning and implementation of the shelter project; and sustained involvement of local authorities.
Intermediate/Medium Term Outcome 2
Enhanced occupation of the shelter by CRC beneficiaries
Enhanced occupation of the shelter by CRC beneficiaries
Immediate or Short-term Outcome 2
Improved skills and knowledge of CRCS staff, volunteers, and community members as well as systems to enable active role of community members in decision-making processes and activities that affect them.
Improved skills and knowledge of CRCS staff, volunteers, and community members as well as systems to enable active role of community members in decision-making processes and activities that affect them.
Immediate or Short-term Outcome 2
Improved access to safe hurricane and earthquake resistant shelters, as well as secure tenure for at least 3 years for CRC eligible beneficiaries
Improved access to safe hurricane and earthquake resistant shelters, as well as secure tenure for at least 3 years for CRC eligible beneficiaries
The following criteria were considered during the design phase:
a) cost
b) speed of construction
c) Maintenance of quality
d) level of economic impact on the affected community (e.g. sourcing & inflation of materials, construction crews, increased level of skills provided, etc…)
e) environmental impact (source of materials and construction)
f) sustainability (e.g. can it be easily repaired? How long will it last?)
g) appropriateness to local conditions
h) equity (should be a Movement-wide design, should not be vastly better than pre-disaster housing stock)
i) vulnerability to risk (e.g. earthquake, strong winds, flooding, security, land tenure)
j) flexible to allow for changes according to individual sites and family sizes
k) portability (does it need to be moveable to allow for land tenure issues?)
a) cost
b) speed of construction
c) Maintenance of quality
d) level of economic impact on the affected community (e.g. sourcing & inflation of materials, construction crews, increased level of skills provided, etc…)
e) environmental impact (source of materials and construction)
f) sustainability (e.g. can it be easily repaired? How long will it last?)
g) appropriateness to local conditions
h) equity (should be a Movement-wide design, should not be vastly better than pre-disaster housing stock)
i) vulnerability to risk (e.g. earthquake, strong winds, flooding, security, land tenure)
j) flexible to allow for changes according to individual sites and family sizes
k) portability (does it need to be moveable to allow for land tenure issues?)
Separately, the Netherlands Red Cross (NLRC) agreed to provide water and sanitation to the 7,500 families receiving shelters directly from Canadian Red Cross (refer MOU signed May 2010).
Beneficiary Selection Process
Assessments were undertaken by the CRC in the targeted communities to determine whether a house had been either fully destroyed by the earthquake or damaged beyond repair. In addition to the condition of the dwelling, the particular circumstances of the individual or family was also assessed through a questionnaire with a weighting given based on vulnerability. For example, child headed households, those with disabilities, the old and infirm, female headed households. In order to select beneficiaries for a shelter the CRC worked with validation committees’ composed of members of the community. The validation committee reviewed the findings of the CRC survey and questionnaire, confirmed the circumstances of the potential beneficiary and prioritised who would receive a shelter based on vulnerability. After a final confirmation by CRC community mobilization officers, the file was handed over to a legal officer who prepared documentation to ensure that the intended beneficiary was entitled to live in the shelter at the site determined. As Leogane was the most severely affected, most of the beneficiaries (4,996) were identified in this area. A total of 2,520 beneficiary families were identified in the South East Department.
Assessments were undertaken by the CRC in the targeted communities to determine whether a house had been either fully destroyed by the earthquake or damaged beyond repair. In addition to the condition of the dwelling, the particular circumstances of the individual or family was also assessed through a questionnaire with a weighting given based on vulnerability. For example, child headed households, those with disabilities, the old and infirm, female headed households. In order to select beneficiaries for a shelter the CRC worked with validation committees’ composed of members of the community. The validation committee reviewed the findings of the CRC survey and questionnaire, confirmed the circumstances of the potential beneficiary and prioritised who would receive a shelter based on vulnerability. After a final confirmation by CRC community mobilization officers, the file was handed over to a legal officer who prepared documentation to ensure that the intended beneficiary was entitled to live in the shelter at the site determined. As Leogane was the most severely affected, most of the beneficiaries (4,996) were identified in this area. A total of 2,520 beneficiary families were identified in the South East Department.
Shelter Response
The final shelters constructed were comprised of the following types:
• 195 x 18m2 IFRC-sourced ‘stick built’ shelters – retroactively clad with CRC exterior treated plywood
• 3,750 x 18m2 panelized ‘Type A’ shelters (designed for wind speeds of up to 145mph) primarily for Jacmel and some parts of Leogane.
• 2,955 x 18m2 panelized ‘Type B’ shelters (designed for wind speeds of up to 112 mph) only for Leogane area
• 600 x 24m2 permanent shelters built by our external partner, Diakonie, in La Vallee, Jacmel for CRC beneficiaries
The final shelters constructed were comprised of the following types:
• 195 x 18m2 IFRC-sourced ‘stick built’ shelters – retroactively clad with CRC exterior treated plywood
• 3,750 x 18m2 panelized ‘Type A’ shelters (designed for wind speeds of up to 145mph) primarily for Jacmel and some parts of Leogane.
• 2,955 x 18m2 panelized ‘Type B’ shelters (designed for wind speeds of up to 112 mph) only for Leogane area
• 600 x 24m2 permanent shelters built by our external partner, Diakonie, in La Vallee, Jacmel for CRC beneficiaries
Diakonie
As part of the 7,500+ shelters being constructed directly by the CRC, the CRC funded the construction of 600 shelters through a German INGO, Diakonie. These shelters were 24 sq. m. permanent shelters built of concrete block. The shelters were built for beneficiaries identified through the CRC beneficiary assessment process. They also received latrines through the Netherlands RC, as did other CRC beneficiaries. The rationale behind this initiative is to increase the build rate to allow the CRC to have 7,500 shelters complete before the onset of the 2011 hurricane season.
As part of the 7,500+ shelters being constructed directly by the CRC, the CRC funded the construction of 600 shelters through a German INGO, Diakonie. These shelters were 24 sq. m. permanent shelters built of concrete block. The shelters were built for beneficiaries identified through the CRC beneficiary assessment process. They also received latrines through the Netherlands RC, as did other CRC beneficiaries. The rationale behind this initiative is to increase the build rate to allow the CRC to have 7,500 shelters complete before the onset of the 2011 hurricane season.
Final Statistics (# of shelters built and locations)
A total of 7,520 households were selected according to the above established beneficiary selection criteria.
A total of 7,514 shelters were constructed are as follows:
• Leogane: 4,996 CRC panelized shelters constructed.
• Jacmel: 1,920 CRC panelized shelters constructed.
• Total: 6,916 CRC panelized shelters completed (not including Diakonie)
• Diakonie (La Vallee only) – 598 constructed out of 601 to be built. The final 3 almost completed shelters were not formally completed due to unresolved problems with theft of construction materials. • Grand Total (including Diakonie): 7,514 completed out of the original 7,500 target.
A total of 7,520 households were selected according to the above established beneficiary selection criteria.
A total of 7,514 shelters were constructed are as follows:
• Leogane: 4,996 CRC panelized shelters constructed.
• Jacmel: 1,920 CRC panelized shelters constructed.
• Total: 6,916 CRC panelized shelters completed (not including Diakonie)
• Diakonie (La Vallee only) – 598 constructed out of 601 to be built. The final 3 almost completed shelters were not formally completed due to unresolved problems with theft of construction materials. • Grand Total (including Diakonie): 7,514 completed out of the original 7,500 target.
All shelters were completed by September 6, 2012.
Water and Sanitation
To September 7, 2012, NLRC had constructed a total of 7,178 latrines and 2 communal water points for 800 families:
To September 7, 2012, NLRC had constructed a total of 7,178 latrines and 2 communal water points for 800 families:
Leogane:
Number of latrines built: 4,563 (4,538 built and 25 latrines on progress)
The gap between 4,996 shelters and 4,563 latrines is 433 and (according to NLRC) is due to the following reasons:
• Number of beneficiaries who had stolen the construction materials: 87
• Number of beneficiaries who did not have a space in their compound for latrine: 89
• Number of beneficiaries who did not want latrine: 90
• Number of beneficiaries moved or not found in the project area: 133
• Number of beneficiaries NLRC could not account for (status unknown): 34
+ 1 communal water point for 400 families
Number of latrines built: 4,563 (4,538 built and 25 latrines on progress)
The gap between 4,996 shelters and 4,563 latrines is 433 and (according to NLRC) is due to the following reasons:
• Number of beneficiaries who had stolen the construction materials: 87
• Number of beneficiaries who did not have a space in their compound for latrine: 89
• Number of beneficiaries who did not want latrine: 90
• Number of beneficiaries moved or not found in the project area: 133
• Number of beneficiaries NLRC could not account for (status unknown): 34
+ 1 communal water point for 400 families
Jacmel:
Number of latrines built: 2,615 latrines (all completed). Locations: • Cayes_Jacmel: 535 • Jacmel: 593 • Marigot: 560 • La_Vallee: 917 • Various: 10 (this includes 65 latrines for IOM and 10 for Oral Rehydration Points (Cholera)) + 1 communal water point for 400 families
Number of latrines built: 2,615 latrines (all completed). Locations: • Cayes_Jacmel: 535 • Jacmel: 593 • Marigot: 560 • La_Vallee: 917 • Various: 10 (this includes 65 latrines for IOM and 10 for Oral Rehydration Points (Cholera)) + 1 communal water point for 400 families
Previous Evaluations on this Project In February 2011, an internal review of the community mobilization process and the approach to beneficiary accountability within the shelter program (e.g., beneficiary communication, participation of community throughout the program cycle management, beneficiary selection process and risk management, complaint and feedback mechanisms), was conducted with recommendations for improvements. The report is available.
Two independent reviews were undertaken as part of a Mid Term Evaluation process for this project: the first was undertaken by the Global Emergency Group, an independent evaluation team over a two-month period (May to June 2011). The final findings and recommendations below were based on over 57 stakeholder interviews and visits to affected communities in Jacmel and Leogane. The second review was a beneficiary satisfaction survey arranged by CRC. It was conducted in Leogane and Jacmel throughout May and June 2011 in order to gather relevant information to complement and corroborate the Review findings. The sample size and selection of beneficiaries was based on random sampling with 100% of selected beneficiaries (majority are original beneficiaries above 18 years old) in Leogane (159 households) and Jacmel (135 households) responding. The key findings of the survey were also referenced in the main evaluation. Two reports were released: Review of the Canadian Red Cross Shelter Program- Haiti: Final Review Report (GEG, June 2011) (“Review”) and Findings of the Beneficiary Survey in Leogane & Jacmel (CRC, June 2011) (“Survey”). It should be noted that, the Diakonie project was not evaluated in this MTE.
According to NLRC, an assessment of the water and sanitation program was undertaken earlier in 2012. We have not yet seen the results.
- Evaluation Scope
Scope of this review: a. Scope of evaluation criteria: effectiveness, efficiency, partnership/coordination with implementing partners appropriateness/sustainability/impact (including environmental sustainability and disaster risk reduction), and targeting/coverage. In addition the evaluation will also look at the cross cutting issues, namely gender, capacity building of the targeted communities, the role of CRCS in advocacy on shelter related issues in Haiti, as well as protection of vulnerable beneficiaries. b. Scope of the components of the shelter: construction of shelter, community mobilization, implementation strategy (CRCS and other implementing partners), and implementation model of the CRC directly implemented shelter program (e.g., issues of efficiency) c. Scope of time-frame of intervention: from the beginning to the time of the evaluation d. Scope on stakeholders: relevant former or existing staff and volunteers of the shelter program in Ottawa/Montreal and Haiti; implementing partners (e.g. the Netherlands Red Cross, Diakonie), operating partners (e.g., Shelter Cluster), and other external stakeholders (e.g., local authorities, community committees). e. Scope of program sites: Jacmel, Leogane, Port-au-Prince, Ottawa and Montreal
- Evaluation Criteria and Key Evaluation Questions (But not limited to these)
Effectiveness: • To what extent were the program expected outcomes achieved, especially whether or not: - Adequate proportion of original beneficiary is still occupying the shelter - Satisfaction of beneficiary with the shelter provided to them and the engagement between them and the program personnel and volunteers • How do the achieved results compare against the specified quality standards?
Efficiency – Cost Effectiveness and Timeliness
• What was the ratio between the operational cost and the cost of deliverable for the end-beneficiaries? How does this compare with similar shelter programs in Haiti? • What lessons can be drawn from variance in planned vs. actual timelines?
• What was the ratio between the operational cost and the cost of deliverable for the end-beneficiaries? How does this compare with similar shelter programs in Haiti? • What lessons can be drawn from variance in planned vs. actual timelines?
Partnership and Coordination and Efficiency
• Has the decision to partner with the Netherlands Red Cross on the water and sanitation and Diakone on permanent shelters led to achievement quality, timely, and more cost efficient delivery of shelter to end-beneficiaries? • How has the management/coordination structure between CRC and their partners facilitated or hindered achievements of quality, timely and cost efficient implementation of the overall shelter program?
• Has the decision to partner with the Netherlands Red Cross on the water and sanitation and Diakone on permanent shelters led to achievement quality, timely, and more cost efficient delivery of shelter to end-beneficiaries? • How has the management/coordination structure between CRC and their partners facilitated or hindered achievements of quality, timely and cost efficient implementation of the overall shelter program?
Appropriateness/Sustainability/Impact, including environmental sustainability and disaster risk reduction
• What are advantages and disadvantages of the design of the shelter that attempted to balance the concept of ‘transitional shelter’ and ‘core shelter’, land issue, cost efficiency and timeliness for the CRC’s operational capacity and end-beneficiaries in terms of meeting their immediate and long-term shelter needs? • Has the program adequately adopted environmentally sustainable practices such as in selecting of building materials and disposal of unused building materials? • Has the program adequately adopted practices that facilitate long-term use of the shelter provided to the end-beneficiaries through integration of disaster risk reduction, locally available skills and materials for repair, and other practices? • To what extent the program has implemented mitigation measures to minimize its harmful impact, including conflict between the staff and the community as well as among community members?
• What are advantages and disadvantages of the design of the shelter that attempted to balance the concept of ‘transitional shelter’ and ‘core shelter’, land issue, cost efficiency and timeliness for the CRC’s operational capacity and end-beneficiaries in terms of meeting their immediate and long-term shelter needs? • Has the program adequately adopted environmentally sustainable practices such as in selecting of building materials and disposal of unused building materials? • Has the program adequately adopted practices that facilitate long-term use of the shelter provided to the end-beneficiaries through integration of disaster risk reduction, locally available skills and materials for repair, and other practices? • To what extent the program has implemented mitigation measures to minimize its harmful impact, including conflict between the staff and the community as well as among community members?
Targeting/Coverage, Gender and Protection of Vulnerable Population:
• Was the decision to select Jacmel/South East Province and Leogane for the shelter program in line with the unmet needs of shelter in Haiti in general? • Was the allocation of shelters for Jacmel/South East Province and Leogane in proportion of unmet shelter needs in these two locations? • To what extent did the program reach the most vulnerable population (e.g., poorest family, family with disabled family members and/or elderly) and ensure equal access of the shelter kits for female and male headed households?
• Was the decision to select Jacmel/South East Province and Leogane for the shelter program in line with the unmet needs of shelter in Haiti in general? • Was the allocation of shelters for Jacmel/South East Province and Leogane in proportion of unmet shelter needs in these two locations? • To what extent did the program reach the most vulnerable population (e.g., poorest family, family with disabled family members and/or elderly) and ensure equal access of the shelter kits for female and male headed households?
Impact and capacity building of the targeted communities
• What are positive and negative impact of the shelter program on the targeted communities • Are there aspects of the shelter program that the communities should have been more involved?
• What are positive and negative impact of the shelter program on the targeted communities • Are there aspects of the shelter program that the communities should have been more involved?
Advocacy:
• What has been contribution of the CRC in the shelter cluster in Haiti with regards to advocacy on quality standards and other shelter major issues?
• What has been contribution of the CRC in the shelter cluster in Haiti with regards to advocacy on quality standards and other shelter major issues?
Lessons-Learned and Good Practices
• What are the most and least ineffective practices in designing, implementing and, and monitoring, as well as managing different aspects of the shelter program (e.g., partnership strategy, community mobilization, construction of the shelter, involvement of the host national society, and the operational model) in Haiti for learning and replication? Among the lessons is whether the adopted concept of core shelter appropriate given that decision on the design needs to be made at early stage while operating in very dynamic operational context.
• What are the most and least ineffective practices in designing, implementing and, and monitoring, as well as managing different aspects of the shelter program (e.g., partnership strategy, community mobilization, construction of the shelter, involvement of the host national society, and the operational model) in Haiti for learning and replication? Among the lessons is whether the adopted concept of core shelter appropriate given that decision on the design needs to be made at early stage while operating in very dynamic operational context.
- Review Methodology and Process
The review will be managed by a review management team, which will comprise of Senior Program Manager Haiti Operations, a representative of the CRC delegation in Haiti, Senior Shelter Advisor in Ottawa, and Ottawa-based Program Manager. The management team will oversee the review and with the review team to ensure that it upholds the expected quality of the review. It will make a decision on the final TOR, selection of reviewer(s), review matrix, tools and methodologies, and the final report.
The review will be led by an external reviewer(s) He or she will be responsible for the overall review from design, preparation of tools, data collection, coordination with relevant parties, and finalising the report (to be revised accordingly).
The specific review methodology will be detailed in close consultation with the review management team, but will draw upon the following methods (but not limited to): • General desktop review of operational background documents, relevant organizational background and history and any relevant sources of secondary data, and the existing reviews (e.g., mid-term review, the internal community mobilization review) • Separately, it is anticipated that analysis will be undertaken on the ongoing occupancy household survey in Jacmel and Leogane. This final evaluation will take into account the results of this analysis. The occupancy household survey data was collected from the shelter beneficiaries over the period from September 2011- August 2012 starting with the oldest shelters first. This data was collected by interview at nearly every shelter site (>7000). Interviews typically took place several months to a year and half after the occupants moved in. The survey covered several a range of issues including number/age/gender/disability status/relationship to beneficiary/shelter condition/latrine. • Primary data collection: the consultant should propose appropriate methodology based on where there is need for information complementary to the secondary data, in order to respond to the evaluation questions. One consideration should be collecting information from representative sampling of end-beneficiaries who have lived in the CRC provided shelter for more than one year in order to obtain insight into the impact and sustainability of the shelter program. CRC requests this include measures of beneficiary satisfaction as to the relevance and appropriateness of the shelters from the beneficiaries’ perspective. • Briefing at the beginning of the review and debriefing at the end of the data collection with key personnel of CRC Haiti Program in Canada and Haiti.
Key milestones of the review are:
Submission of Final Report by the consultants to CRC - January 31, 2013 Submission of comments to the consultants by CRC - January 24, 2013 Submission of the first draft of the Final Report - January 11, 2013
Travel and field mission in Port-au Prince, Leogane and Jacmel - a max of 12 working days of field visit, inclusive of travel time) - December 3-14, 2012 Comments from relevant parties on the Inception Report - November 29, 2012 Desk-reviews and submission of the inception report (e.g., methodology and workplan) - November 21, 2012 Selection of consultants, contract signing with consultants - November 15, 2012 Deadline of the submission of the proposal from consultants - October 26, 2012
The evaluation report will be shared and commented by relevant stakeholders while maintaining the integrity and independence of the evaluation report according to the following lines.
• Inaccuracy. Inaccuracies are factual, supported with undisputable evidence, and therefore should be corrected in the evaluation report itself. • Clarifications. A clarification is additional, explanatory information to what the evaluators provided in the report. It is the evaluators’ decision whether to revise their report according to a clarification; if not, the evaluation management response team can decide whether to include the clarification in their management response. • Difference of opinion. A difference of opinion does not pertain to the findings (which are factual), but to the conclusions and/or recommendations. These may be expressed to the evaluators during the evaluation process. It is the evaluators’ decision whether to revise their report according to a difference of opinion; if not, the evaluation management response team can decide whether to include the clarification in their management response.
- Review Deliverables
Inception Report – The inception report will include the proposed methodologies, data collection and reporting plans with draft data collection tools such as interview guides, a timeframe with firm dates for deliverables, and the travel and logistical arrangements for the team – by November 21, 2012
The First Draft report – A draft report, consolidating findings from the evaluation, identifying key findings, conclusions, recommendations and lessons for the current and future similar program, will be submitted within 1 week of the evaluation team’ return from the field to the Review Management Team on January 11, 2013.
Final report – The final report will contain a short executive summary (no more than 5 pages) and a main body of the report (no more than 60 pages) covering the background of the intervention evaluated, a description of the evaluation methods and limitations, findings (to be presented by evaluation criteria), conclusions, lessons learned, clear recommendations. A section should outline recommendations that the program staff and the reviewer(s) have in common or different views based on the workshop to discuss the findings. The report should also contain appropriate appendices, including a copy of the ToR, an annex with the result of the summary of census database, a cited resources or bibliography, a list of those interviewed and any other relevant materials (e.g., tools). The final report will be submitted one week after receipt of the consolidated feedback from the Review Management Team on January 31, 2013.
- Evaluation Quality & Ethical Standards
The evaluators should take all reasonable steps to ensure that the review is designed and conducted to respect and protect the rights and welfare of the people and communities involved and to ensure that the review is technically accurate and reliable, is conducted in a transparent and impartial manner, and contributes to organizational learning and accountability. The evaluator(s) will sign and adhere to the Canadian Red Cross Code of Conduct and the Evaluation Standards below.
The evaluation standards are:
1. Utility: Evaluations must be useful and used. 2. Feasibility: Evaluations must be realistic, diplomatic, and managed in a sensible, cost effective manner. 3. Ethics & Legality: Evaluations must be conducted in an ethical and legal manner, with particular regard for the welfare of those involved in and affected by the evaluation. 4. Impartiality & Independence; Evaluations should be impartial, providing a comprehensive and unbiased assessment that takes into account the views of all stakeholders. 5. Transparency: Evaluation activities should reflect an attitude of openness and transparency. 6. Accuracy: Evaluations should be technical accurate, providing sufficient information about the data collection, analysis, and interpretation methods so that its worth or merit can be determined. 7. Participation: Stakeholders should be consulted and meaningfully involved in the evaluation process when feasible and appropriate. 8. Collaboration: Collaboration between key operating partners in the evaluation process improves the legitimacy and utility of the evaluation.
1. Utility: Evaluations must be useful and used. 2. Feasibility: Evaluations must be realistic, diplomatic, and managed in a sensible, cost effective manner. 3. Ethics & Legality: Evaluations must be conducted in an ethical and legal manner, with particular regard for the welfare of those involved in and affected by the evaluation. 4. Impartiality & Independence; Evaluations should be impartial, providing a comprehensive and unbiased assessment that takes into account the views of all stakeholders. 5. Transparency: Evaluation activities should reflect an attitude of openness and transparency. 6. Accuracy: Evaluations should be technical accurate, providing sufficient information about the data collection, analysis, and interpretation methods so that its worth or merit can be determined. 7. Participation: Stakeholders should be consulted and meaningfully involved in the evaluation process when feasible and appropriate. 8. Collaboration: Collaboration between key operating partners in the evaluation process improves the legitimacy and utility of the evaluation.
- Qualifications
Selection of the external consultants will be based on the following qualifications:
The minimum requirements for the team are the following expertise:
• Excellent expertise in different components of the TOR (cost efficiency analysis, quality assurance of the shelter construction, management of a large shelter program, community mobilization and partnership between implementing partners and host national society).
• Excellent expertise in different components of the TOR (cost efficiency analysis, quality assurance of the shelter construction, management of a large shelter program, community mobilization and partnership between implementing partners and host national society).
Requirements of the Team Leader:
Minimum of 4 previous assignments of leading a similar complex evaluation, covering multi-functions issues such as partnership, construction, management and community mobilization. Minimum of 15 years of professional experience in international development Demonstrated capacity for strategic thinking, team-management and writing skills as he or she is responsible in integrating various analysis and findings from different team members in a coherent manner.
Minimum of 4 previous assignments of leading a similar complex evaluation, covering multi-functions issues such as partnership, construction, management and community mobilization. Minimum of 15 years of professional experience in international development Demonstrated capacity for strategic thinking, team-management and writing skills as he or she is responsible in integrating various analysis and findings from different team members in a coherent manner.
Requirements of the Team Member: Minimum of 10 years of professional experience in his or her area of expertise (e.g., construction, management, partnerships with implementing partners and host national society, cost efficiency analysis, and community mobilization) Minimum of 2 previous evaluation assignments, preferably in a large scale disaster
General Requirements for All:
Ability to manage relations with representatives from national societies, government, and the community, including diplomacy and consensus building skills. Good knowledge of the RC/RC movement systems Excellent skills in quantitative and qualitative analysis Familiarity with Haiti context or previous experience of working in large-scale disaster affected countries. A sound knowledge of cross-cutting issues in shelter (e.g., gender, environmental sustainability and disaster risk reduction) is essential. Excellent spoken and written command of French and English (at least one Team Member or the Team Leader needs to be fully bi-lingual). All must have as minimum excellent spoken and written English.
Ability to manage relations with representatives from national societies, government, and the community, including diplomacy and consensus building skills. Good knowledge of the RC/RC movement systems Excellent skills in quantitative and qualitative analysis Familiarity with Haiti context or previous experience of working in large-scale disaster affected countries. A sound knowledge of cross-cutting issues in shelter (e.g., gender, environmental sustainability and disaster risk reduction) is essential. Excellent spoken and written command of French and English (at least one Team Member or the Team Leader needs to be fully bi-lingual). All must have as minimum excellent spoken and written English.
How to apply:
Interested candidates can apply as a team or a team member of specific aspect(s) of the TOR. He or she should submit their application materials by October 26, 2012 to the following emails: wartini.pramana@redcross.ca and cc:careers_in@redcross.ca . Application material is non-returnable, and we thank you in advance for understanding that only short-listed candidates will be contacted for the next step in the application process.
Application materials should include:
1. A maximum of 5 pages of proposal outlining key evaluation questions that would be done and proposed methodology to answer the relevant questions for a sectoral evaluator and 15 pages for a team who is interested in carrying all aspects of the TOR.
2. Curricula Vitae (CV)
3. Cover letter clearly summarizing your experience as it pertains to this assignment, your daily rate, and three professional references.
4. At least one example of an evaluation report most similar to the candidate’s proposed role (e.g., Team Leader, etc.).
1. A maximum of 5 pages of proposal outlining key evaluation questions that would be done and proposed methodology to answer the relevant questions for a sectoral evaluator and 15 pages for a team who is interested in carrying all aspects of the TOR.
2. Curricula Vitae (CV)
3. Cover letter clearly summarizing your experience as it pertains to this assignment, your daily rate, and three professional references.
4. At least one example of an evaluation report most similar to the candidate’s proposed role (e.g., Team Leader, etc.).
