Improving Development Evaluation

Hosted by David Wand

Business EducationInterviews guests

Website RSS feed

Episodes

Latest episode

Jun 2026

Language

About the show

Join Dr. David Wand as he takes you on a provocative, investigative journey challenging Canadian & American international development organizations as well as international organizations to improve their performance measurement frameworks (PMFs) or Activity Monitoring and Evaluation (MEL) Plans. In Season 1, part 1 of the episode, Dr. Wand will introduce you to an international development organization and one of its projects where flaws in their PMF will be identified. In part 2 of the episode, the international development organization and evaluation experts will be invited to discuss Dr. Wand's critique and offer solutions to improving the evaluation of the project. For Season 2, Dr. Wand expands the scope of his podcast to include USA government funded international development projects by acquiring Activity Monitoring Evaluation Learning Plans (MEL Plans) or their equivalent as well as continue acquiring Canada government funded international development projects that require PMFs or their equivalent. For Season 2 the focus of the podcast has also been improved whereby several projects from the same international development organization are covered in the same episode exposing how the international development organization fails to properly measure and evaluate whether its expected outcomes have been achieved. This is despite these organizations falsely claiming on their websites that they have achieved these outcomes at the expense of the taxpayer.

Listen to episodes

31 recent

June 24, 202635 min

GCERF post evaluation fails to show effectiveness in countering violent extremism in Nigeria

In this episode, Dr. Wand explains how a poorly designed evaluation framework with invalid outcome indicators leads to a post-project evaluation that fails to demonstrate effectiveness in countering violent extremism in Nigeria. Dr. Wand reviews the evaluation report produced by the Global Community Engagement Resilience Fund (GCERF) available at https://www.gcerf.org/wp-content/uploads/2015/12/Nigeria-Results-Paper-Round-2-Funding-2019-21.pdf . Dr. Wand's review covers the following findings with detailed examples: Expected outcomes cannot be claimed to be achieved as GCERF claims as the outcome indicators depend on biased self-reporting rather than objective measures. Examples include GCERF claims of Community Action Response Teams being 'fully functionable' but with no objective measures of functionality. GCERF claims of high levels of confidence by beneficiaries are questionable as methods of measuring these confidence levels are not provided. Outcome indicators measuring skill acquisition for concrete skills such as the ability to mediate or manage conflicts or the ability to identify and report early warning signs of conflict are absent yet GCERF assumes without data that this skill acquisition occurred and that conflict mediation or management was successful. GCERF makes 'impact' claims beyond the project despite no evidence such as reductions in violence and alcohol in specific communities. Recommendations to include data sources and methods of data collection and analysis are provided to enable GCERF to support their claims that currently cannot be supported as well as ensuring that outcome indicators actually measure the expected outcomes. To receive discounts for the purchase of sunglasses or polo shirts or travel bags/backpacks/wallets designed for the Global South use the following link: https://mydeals.page/1hjl Donate here: https://www.paypal.com/ncp/payment/ZAQD8888DEDXL Or at buymeacoffee.com/davidwand to help the podcast.

June 19, 202641 min

Humber Polytechnic International Development Institute fails to properly evaluate its $2 Million Kenya & Ethiopia STEM Education for Empowerment project

In this episode, Dr. Wand reviews the Humber Polytechnic International Development Institute ( https://humber.ca/global/international-development-institute-idi ) $2 Million Kenya & Ethiopia STEM Education for Empowerment project's Performance Measurement Framework (PMF). He concludes that Humber Polytechnic International Development Institute cannot make the claim that it has achieved 8 of the 9 expected Outcomes for the project since 18 out of the 19 outcome indicators are not valid measures of the Outcomes , or for the 1 valid outcome indicator fail to show the necessary statistical analysis that would support the project's claim that the outcome was achieved statistically significantly rather than due to chance. Details about the project can be found here: https://w05.international.gc.ca/projectbrowser-banqueprojets/project-projet/details/p007925001 The PMF and Dr. Wand's critique are available by emailing evaluatecanadaaid@gmail.com. The PMF and the critique will be shared with Humber Polytechnic along with an invitation for them to respond to the critique by attending a future podcast. It will also be shared with the Secretary of State for International Development as well as the shadow critics for all the opposition parties. To receive discounts for the purchase of sunglasses or polo shirts or travel bags/backpacks/wallets designed for the Global South use the following link: https://mydeals.page/1hjl Donate here: https://www.paypal.com/ncp/payment/ZAQD8888DEDXL Or at buymeacoffee.com/davidwand to help the podcast.

June 1, 202628 min

GCERF fails to validly measure rehabilitation of Returning Foreign Terrorist Fighters - Western Balkans

The Global Community Engagement Resilience Fund (GCERF) ( gcerf.org ) has been funding organizations in Bosnia & Herzegovina, Kosovo, Albania, and North Macedonia to deliver psychological services to Returning Foreign Terrorist Fighters to enable them to become first, rehabilitated, and then reintegrated into their communities. Unfortunately, GCERF has failed to provide any valid measures of rehabilitation for these foreign terrorist fighters despite making claims of achieving subsequent reintegration for these fighters as shown by the absence of any rehabilitation outcome indicators in their "Results from Round 1 and Round 2 Funding in the Western Balkans (2020-2025)" report. In this episode, Dr. Wand recommends that GCERF draw upon 4 valid measures of rehabilitation specific to Returning Foreign Terrorist Fighters, including the Violent Extremism Risk Assessment VERA and the TRAP-18 (Terrorist Radicalization Assessment Protocol) psychometric tools. He also identifies flaws in statistical analysis such as aggregating outcome indicators across different projects within a country as well as aggregating outcome indicators across different countries that hinders GCERF's ability to use evaluation results to improve service delivery at the project level. He also notes the improper use of comparing averages of percent indicators rather than using medians that fails to recognize extreme percent values that can distort the overall percent indicator. For receiving a discount on the products recommended when visiting the Global South, use the following link: https://mydeals.page/1hjl Donate here: https://www.paypal.com/ncp/payment/ZAQD8888DEDXL Or at buymeacoffee.com/davidwand to help the podcast.

May 26, 202625 min

GCERF reports percentage increases but fails to do the statistical analysis to support statistical significance: Ghana Results Paper Emergency Grant

In this episode, Dr. Wand thanks the Global Community Engagement Resilience Fund (GCERF) ( gcerf.org ) for 'improving development evaluation' by, six years later, finally publishing their results frameworks, including outcome statements, outcome indicators, as well as actually data on these indicators. A marked increase in data transparency!!! In this episode, Dr. Wand reviews the GCERF report Ghana Results Paper: Emergency Grant and identifies 4 areas of weakness in the results paper as follows: Poor design of Outcome Indicators that leads to invalid measures of whether the expected outcome has been achieved. Failure to describe psychometric tool used to measure the Outcome Indicators along with suggested psychometric tools to measure levels of trust. Failure to measure the expected outcome of increased vocational skill specific to tailoring, parboiling rice, and processing shea butter. Failure to show the inferential statistical analysis needed to test whether reported percentage increases in the report have achieved statistical significance or have not achieved statistical significance. For receiving a discount on the products recommended when visiting the Global South, use the following link: https://mydeals.page/1hjl Donate here: https://www.paypal.com/ncp/payment/ZAQD8888DEDXL Or at buymeacoffee.com/davidwand to help the podcast.

April 24, 202652 min

Humber Polytechnic International Development Institute fails to properly evaluate its $4.9 Million Bhutan Education and Skills Training project

In this episode, Dr. Wand reviews the Humber Polytechnic International Development Institute ( https://humber.ca/global/international-development-institute-idi ) $4.9 Million Bhutan Education and Skills Training project Performance Measurement Framework (PMF). He concludes that Humber Polytechnic cannot make the claim that it has achieved 9 of the 11 expected Outcomes for the project since the outcome indicators are either not valid measures of the Outcomes and/or contain conflicts of interest for 25 out of 29 Outcome Indicators where project staff rather than independent evaluators have been responsible for data collection on these Outcome Indicators. Even for the remaining 2 Outcomes that contain 4 valid Outcome Indicators, there is no inferential statistical analysis p<.05 to support the claim that observed percentage increases on the indicators would be statistically significant to support the claim that the project was effective in achieving its outcomes. Details about the project can be found here: https://w05.international.gc.ca/projectbrowser-banqueprojets/project-projet/details/p010696001 The PMF and Dr. Wand's critique are available by emailing evaluatecanadaaid@gmail.com. The PMF and the critique will be shared with Humber Polytechnic along with an invitation for them to respond to the critique by attending a future podcast. It will also be shared with the Secretary of State for International Development as well as the shadow critics for all the opposition parties. To receive discounts for the purchase of sunglasses or polo shirts use the following link: https://mydeals.page/1hjl Donate here: https://www.paypal.com/ncp/payment/ZAQD8888DEDXL Or at buymeacoffee.com/davidwand to help the podcast.

March 31, 202647 min

World Vision Canada fails to show Impact in its $41 Million Gender Equality Nutrition program but potential to show project effectiveness is there

In this episode we review the $41 Million World Vision Canada project ( www.worldvisioncanada.ca ) entitled Realizing Gender Equality, Attitudinal Change, and Transformative Systems in Nutrition. More information about this project can be found at https://w05.international.gc.ca/projectbrowser-banqueprojets/project-projet/details/p011791001. A critique of the Performance Measurment Framework (PMF) for this project reveals 13 expected Outcomes for this project with 38 Outcome Indicators developed to examine whether those Outcomes were indeed achieved. This critique concludes that 10 Outcome Indicators are designed to enable to examine whether those Outcomes were achieved while the remaining 28 Outcome Indicators were considered to be poorly designed with no possibility of supporting the claim that their related Outcomes were achieved. Data on these Outcome Indicators are not available to the public which would enable the public to see if Outcomes were achieved. Flaws in Outcome Indicator design include using self-reporting bias instead of external objective measures as well as Outcome Indicators not related to the Outcome. Even for the 10 Outcome Indicators considered to well designed in measuring the achievement of its related Outcome, many of these Outcome Indicators failed to offer statistical significance analysis to show that the achievment of the Outcome could be attributed to the $41 Million project where that achievement was significantly different compared to the Outcome Indicator moving in the desired direction anyway in the absence of the $41 Million project or moving in the desired direction over time but with negligible effect (i.e. lacks significant Impact). Please sign the petition here https://www.change.org/EvaluateCanadaAid to have the Government of Canada require all Canadian taxpayer funded international development organizations release on their websites their Performance Measurement Frameworks (PMFs) or its equivalent and data for all output and outcome indicators listed in the PMF. For a 15% discount on sunglasses, and cotton polo shirts when visiting the Global South use this discount link: https://mydeals.page/1hjl Donate here: https://www.paypal.com/ncp/payment/ZAQD8888DEDXL Or at buymeacoffee.com/davidwand to help the podcast

December 17, 202551 min

Failure of CUSO International to properly evaluate its 2 projects worth $7.5 Million

Please sign the petition here https://www.change.org/EvaluateCanadaAid to have the Government of Canada require all Canadian taxpayer funded international development organizations release on their websites their Performance Measurement Frameworks (PMFs) or its equivalent and data for all output and outcome indicators listed in the PMF. CUSO International ( www.cusointernational.org ) has failed to properly measure and evaluate its 2 projects costing the Canadian taxpayer $7.5 Million. CUSO International cannot make the claim that it has achieved its outcomes for their 2 projects. This is based upon a review and critique of the Performance Measurement Frameworks (PMFs) for these 2 projects. This critique is explained for each outcome and outcome indicator from these 2 PMFs in this episode. These PMFs and PMF critique can be requested by emailing evaluatecanadaaid@gmail.com. Details on these 2 CUSO International projects is available here: Blueprints for Gender Equality https://w05.international.gc.ca/projectbrowser-banqueprojets/project-projet/details/p010478001 and Building Bricks Caribbean https://w05.international.gc.ca/projectbrowser-banqueprojets/project-projet/details/p013586001 For the Blueprints for Gender Equality CUSO International project, the PMF had 9 Outcomes with 13 Outcome Indicators. Only 1 of those Outcome Indicators was properly designed to support the claim that the Outcome was achieved. Even if it was achieved, no data was provided to support that claim. The remaining 12 poorly designed Outcome Indicators were poorly designed due to either self-reporting bias or conflict of interest when an external consultant should have been used instead to objectively measure whether the outcome was achieved. They used an external consultant for the 1 good Outcome Indicator but oddly refused to do it for the other Outcome Indicators!! Other poorly designed Outcome Indicators were due to measures that had nothing to do with whether the Outcome was achieved. For the Building Bricks CUSO International project, the PMF had 8 Outcomes and 15 Outcome Indicators. 4 of those Outcome Indicators were properly designed to support the claim that some of the Outcomes were achieved. Even if they were achieved, no data was provided to support that claim. The remaining 11 poorly designed Outcome Indicators were poorly designed due to either self-reporting bias or due to measures that had nothing to do with whether the Outcome was achieved. CUSO International has been invited to attend the podcast to provide a response to this critique of their project PMFs. The PMF critique and PMFs have been sent to CUSO International as well as the Secretary of State for International Development and the shadow critics for International Development. For a 20/15% discount on sunhats, sunglasses, and cotton polo shirts when visiting the Global South use this discount link: https://mydeals.page/1hjl Donate here: https://www.paypal.com/ncp/payment/ZAQD8888DEDXL Or at buymeacoffee.com/davidwand to help the podcast

November 13, 202521 min

GCERF fails to show it is achieving its outcomes from spending $1.1 Million USD from Global Affairs Canada

In this episode, the Global Community Engagement Resilience Fund (GCERF) ( www.gcerf.org ) fails to evaluate the effectiveness of its $ 1.1 Million USD received from Global Affairs Canada. GCERF claims to be achieving its 4 outcomes by funding organizations in developing countries to deliver services to achieve those outcomes. But GCERF provides no measures on these outcomes through outcome indicators nor any data related to those outcome indicators in the document they sent to Dr. Wand. This document was a response to his request for the outcome indicators and their data used to measure changes in these outcomes. The 4 outcomes that GCERF claims to be achieving with this $1.1 Million USD Government of Canada funding are: Increased social cohesion Increased ability to mobilize, organize, represent their own interests (agency) Increased knowledge, skills, and subsequent increases in income/livelihoods. Increased confidence, critical thinking, life skills, self-worth, resilience (sense of purpose). GCERF claims on its website, "At grant level, each project is guided by a theory of change and a results framework with outcome and output indicators , requiring grantees to collect baseline data, report quarterly, and conduct final assessments." GCERF is falsely claiming to Canadians that they have achieved these 4 outcomes because GCERF provides no outcome indicators nor data from these outcome indicators showing that social cohesion, agency, skills & income, or sense of purpose have increased. Dr. Wand has sent his critique of this GCERF document to the Secretary of State for International Development as well as the shadow critics for International Development. He has recommended that all funding to GCERF be suspended until outcome indicators and the data used to measure the achievement of these outcomes is provided on their website for all grants including this grant, if such outcome indicators and data exist. Sign the petition here to require GCERF to put all its PMFs and related data on its website: www.change.org/EvaluateCanadaAid GCERF has also been sent the critique of their response as well as an invitation to be on the podcast to respond and offer actions to address the absence of outcome measures for their expected outcomes or to respond in writing to evaluatecanadaaid@gmail.com. https://mydeals.page/1hjl Donate here to increase the number of organizations that receive performance audits like this one: https://www.paypal.com/ncp/payment/ZAQD8888DEDXL Or at buymeacoffee.com/davidwand

September 19, 202545 min

Oxfam Canada (www.oxfam.ca) fails to evaluate its 2 projects worth $13,140,000: Care Connect and Camino Verde

Sign the petition here to require Oxfam to put all its PMFs and related data on its website: www.change.org/EvaluateCanadaAid In this episode, Dr. Wand reviews the Performance Measurement Frameworks (PMFs) for 2 Oxfam Canada ( www.oxfam.ca ) projects: Care Connect https://w05.international.gc.ca/projectbrowser-banqueprojets/project-projet/details/p012493001 at a cost of $5,950,000 and Camino Verde https://w05.international.gc.ca/projectbrowser-banqueprojets/project-projet/details/p005232001 at a cost of $7,190,000. He concludes from his review that Oxfam Canada has failed to design proper outcome indicators (with the exception of 2 outcome indicators out of 18 outcome indicators for the Care Connect project) to meet the Development Assistance Committee (DAC) criteria of achieving effectiveness or impact for these 2 projects. In addition, Oxfam Canada has failed to provide any outcome indicators in its PMF for 10 of the 14 outcomes listed in its Camino Verde project. Therefore, Oxfam Canada cannot make the claim that their projects are achieving their expected outcomes of effectiveness and impact. Dr. Wand has sent this episode along with his critique of the PMFs and the 2 original PMFs to the Secretary of State for International Development as well as the shadow critics for the Conservative party, the New Democratic Party, and the Bloc Quebecois party. He has recommended to all of these individuals that Government of Canada funding for these 2 projects be stopped immediately until proper outcome indicators are designed so that proper evaluation of the effectiveness and impact of these projects can be achieved. Oxfam Canada has also been sent this episode as well as the critique of the PMFs and the 2 original PMFs inviting them to respond to the critique, including an invitation to attend a future podcast to discuss their flaws in their project PMF outcome indicators as well as their proposed actions to address these flaws. If you wish to receive your discount on the purchase of sun hats, sunglasses, or polo shirts designed for the Global South, select the link below: https://mydeals.page/1hjl If you wish to make a donation to this podcast click on either of the links below: https://www.paypal.com/ncp/payment/ZAQD8888DEDXL https://buymeacoffee.com/davidwand Listeners can receive the PMFs and my critique of the PMFs by sending an email to: evaluatecanadaaid@gmail.com

February 7, 202514 min

Global Community Engagement Resilience Fund (www.gcerf.org) fails to evaluate its $5 Million USAID funding

In this episode, the Global Community Engagement Resilience Fund ( www.gcerf.org ) fails to evaluate its $ 5 Million USD received from the United States Agency for International Development. Although GCERF claims on its website to be achieving its 4 expected outcomes through funding its implementing partners in developing countries to deliver services to achieve those outcomes, they provide no evidence in the form of outcome indicators and any data related to those outcome indicators on its website. The 4 expected outcomes that GCERF claims to be achieving with this $5 Million USAID funding are: Increased social cohesion Increased ability to mobilize, organize, represent their own interests (agency) Increased knowledge, skills, as well as subsequent increases in income/access to livelihoods. Increased confidence, critical thinking, life skills, self-worth, resilience (sense of purpose). In addition, letters from the USAID and U.S. State Department indicate that no records exist specific to results frameworks/outcome indicators and related data for these expected outcomes for this $5 Million USD given to GCERF from the US AID. GCERF is falsely claiming to the American people on their website with their taxpayer funds that they are achieving these expected outcomes because they provide no evidence on their website of achieving these outcomes in the form of outcome indicators and data to show that these expected outcomes have been achieved. As a result, the Improving Development Evaluation podcast has mailed a letter along with copies of the letters from the USAID and U.S. State Department to the Chairman of the Foreign Affairs Committee, Brian Mast, requesting that he request GCERF to post all their results frameworks, including all outcome indicators and data for those outcome indicators that measure the achievement of their 4 expected outcomes, from all the grants/projects produced by GCERF in collaboration with their implementing partners that have come from the original $5 Million USD that GCERF received from USAID up to September 15, 2023 specific to Activity ID; 77266; Project #: SCTOOP21VC004; International Sector: Government & Civil Society; USG Sector: Counter-Terrorism; Managing Agency: Department of State; Funding Agency: USAID https://mydeals.page/1hjl Donate here to increase the number of organizations that receive performance audits like this one: https://www.paypal.com/ncp/payment/ZAQD8888DEDXL Or at buymeacoffee.com/davidwand