Improving Development Evaluation

Hosted by David Wand

Business EducationInterviews guests

Website RSS feed

Episodes

Latest episode

Jun 2026

Language

About the show

Join Dr. David Wand as he takes you on a provocative, investigative journey challenging Canadian & American international development organizations as well as international organizations to improve their performance measurement frameworks (PMFs) or Activity Monitoring and Evaluation (MEL) Plans. In Season 1, part 1 of the episode, Dr. Wand will introduce you to an international development organization and one of its projects where flaws in their PMF will be identified. In part 2 of the episode, the international development organization and evaluation experts will be invited to discuss Dr. Wand's critique and offer solutions to improving the evaluation of the project. For Season 2, Dr. Wand expands the scope of his podcast to include USA government funded international development projects by acquiring Activity Monitoring Evaluation Learning Plans (MEL Plans) or their equivalent as well as continue acquiring Canada government funded international development projects that require PMFs or their equivalent. For Season 2 the focus of the podcast has also been improved whereby several projects from the same international development organization are covered in the same episode exposing how the international development organization fails to properly measure and evaluate whether its expected outcomes have been achieved. This is despite these organizations falsely claiming on their websites that they have achieved these outcomes at the expense of the taxpayer.

Listen to episodes

29 recent

June 1, 202628 min

GCERF fails to validly measure rehabilitation of Returning Foreign Terrorist Fighters - Western Balkans

The Global Community Engagement Resilience Fund (GCERF) (gcerf.org) has been funding organizations in Bosnia & Herzegovina, Kosovo, Albania, and North Macedonia to deliver psychological services to Returning Foreign Terrorist Fighters to enable them to become first, rehabilitated, and then reintegrated into their communities. Unfortunately, GCERF has failed to provide any valid measures of rehabilitation for these foreign terrorist fighters despite making claims of achieving subsequent reintegration for these fighters as shown by the absence of any rehabilitation outcome indicators in their "Results from Round 1 and Round 2 Funding in the Western Balkans (2020-2025)" report.In this episode, Dr. Wand recommends that GCERF draw upon 4 valid measures of rehabilitation specific to Returning Foreign Terrorist Fighters, including the Violent Extremism Risk Assessment VERA and the TRAP-18 (Terrorist Radicalization Assessment Protocol) psychometric tools.He also identifies flaws in statistical analysis such as aggregating outcome indicators across different projects within a country as well as aggregating outcome indicators across different countries that hinders GCERF's ability to use evaluation results to improve service delivery at the project level. He also notes the improper use of comparing averages of percent indicators rather than using medians that fails to recognize extreme percent values that can distort the overall percent indicator.For receiving a discount on the products recommended when visiting the Global South, use the following link: https://mydeals.page/1hjlDonate here: https://www.paypal.com/ncp/payment/ZAQD8888DEDXLOr at buymeacoffee.com/davidwand to help the podcast.

May 26, 202625 min

GCERF reports percentage increases but fails to do the statistical analysis to support statistical significance: Ghana Results Paper Emergency Grant

In this episode, Dr. Wand thanks the Global Community Engagement Resilience Fund (GCERF) (gcerf.org) for 'improving development evaluation' by, six years later, finally publishing their results frameworks, including outcome statements, outcome indicators, as well as actually data on these indicators. A marked increase in data transparency!!!In this episode, Dr. Wand reviews the GCERF report Ghana Results Paper: Emergency Grant and identifies 4 areas of weakness in the results paper as follows:Poor design of Outcome Indicators that leads to invalid measures of whether the expected outcome has been achieved.Failure to describe psychometric tool used to measure the Outcome Indicators along with suggested psychometric tools to measure levels of trust.Failure to measure the expected outcome of increased vocational skill specific to tailoring, parboiling rice, and processing shea butter.Failure to show the inferential statistical analysis needed to test whether reported percentage increases in the report have achieved statistical significance or have not achieved statistical significance.For receiving a discount on the products recommended when visiting the Global South, use the following link: https://mydeals.page/1hjlDonate here: https://www.paypal.com/ncp/payment/ZAQD8888DEDXLOr at buymeacoffee.com/davidwand to help the podcast.

April 24, 202652 min

Humber Polytechnic International Development Institute fails to properly evaluate its $4.9 Million Bhutan Education and Skills Training project

In this episode, Dr. Wand reviews the Humber Polytechnic International Development Institute (https://humber.ca/global/international-development-institute-idi) $4.9 Million Bhutan Education and Skills Training project Performance Measurement Framework (PMF). He concludes that Humber Polytechnic cannot make the claim that it has achieved 9 of the 11 expected Outcomes for the project since the outcome indicators are either not valid measures of the Outcomes and/or contain conflicts of interest for 25 out of 29 Outcome Indicators where project staff rather than independent evaluators have been responsible for data collection on these Outcome Indicators.Even for the remaining 2 Outcomes that contain 4 valid Outcome Indicators, there is no inferential statistical analysis p<.05 to support the claim that observed percentage increases on the indicators would be statistically significant to support the claim that the project was effective in achieving its outcomes.Details about the project can be found here: https://w05.international.gc.ca/projectbrowser-banqueprojets/project-projet/details/p010696001The PMF and Dr. Wand's critique are available by emailing evaluatecanadaaid@gmail.com.The PMF and the critique will be shared with Humber Polytechnic along with an invitation for them to respond to the critique by attending a future podcast. It will also be shared with the Secretary of State for International Development as well as the shadow critics for all the opposition parties.To receive discounts for the purchase of sunglasses or polo shirts use the following link:https://mydeals.page/1hjlDonate here: https://www.paypal.com/ncp/payment/ZAQD8888DEDXLOr at buymeacoffee.com/davidwand to help the podcast.

March 31, 202647 min

World Vision Canada fails to show Impact in its $41 Million Gender Equality Nutrition program but potential to show project effectiveness is there

In this episode we review the $41 Million World Vision Canada project (www.worldvisioncanada.ca) entitled Realizing Gender Equality, Attitudinal Change, and Transformative Systems in Nutrition. More information about this project can be found at https://w05.international.gc.ca/projectbrowser-banqueprojets/project-projet/details/p011791001.A critique of the Performance Measurment Framework (PMF) for this project reveals 13 expected Outcomes for this project with 38 Outcome Indicators developed to examine whether those Outcomes were indeed achieved. This critique concludes that 10 Outcome Indicators are designed to enable to examine whether those Outcomes were achieved while the remaining 28 Outcome Indicators were considered to be poorly designed with no possibility of supporting the claim that their related Outcomes were achieved. Data on these Outcome Indicators are not available to the public which would enable the public to see if Outcomes were achieved.Flaws in Outcome Indicator design include using self-reporting bias instead of external objective measures as well as Outcome Indicators not related to the Outcome. Even for the 10 Outcome Indicators considered to well designed in measuring the achievement of its related Outcome, many of these Outcome Indicators failed to offer statistical significance analysis to show that the achievment of the Outcome could be attributed to the $41 Million project where that achievement was significantly different compared to the Outcome Indicator moving in the desired direction anyway in the absence of the $41 Million project or moving in the desired direction over time but with negligible effect (i.e. lacks significant Impact).Please sign the petition here https://www.change.org/EvaluateCanadaAid to have the Government of Canada require all Canadian taxpayer funded international development organizations release on their websites their Performance Measurement Frameworks (PMFs) or its equivalent and data for all output and outcome indicators listed in the PMF.For a 15% discount on sunglasses, and cotton polo shirts when visiting the Global South use this discount link: https://mydeals.page/1hjlDonate here: https://www.paypal.com/ncp/payment/ZAQD8888DEDXLOr at buymeacoffee.com/davidwand to help the podcast

December 17, 202551 min

Failure of CUSO International to properly evaluate its 2 projects worth $7.5 Million

Please sign the petition here https://www.change.org/EvaluateCanadaAid to have the Government of Canada require all Canadian taxpayer funded international development organizations release on their websites their Performance Measurement Frameworks (PMFs) or its equivalent and data for all output and outcome indicators listed in the PMF. CUSO International (www.cusointernational.org) has failed to properly measure and evaluate its 2 projects costing the Canadian taxpayer $7.5 Million. CUSO International cannot make the claim that it has achieved its outcomes for their 2 projects. This is based upon a review and critique of the Performance Measurement Frameworks (PMFs) for these 2 projects. This critique is explained for each outcome and outcome indicator from these 2 PMFs in this episode. These PMFs and PMF critique can be requested by emailing evaluatecanadaaid@gmail.com.Details on these 2 CUSO International projects is available here: Blueprints for Gender Equality https://w05.international.gc.ca/projectbrowser-banqueprojets/project-projet/details/p010478001 and Building Bricks Caribbean https://w05.international.gc.ca/projectbrowser-banqueprojets/project-projet/details/p013586001For the Blueprints for Gender Equality CUSO International project, the PMF had 9 Outcomes with 13 Outcome Indicators. Only 1 of those Outcome Indicators was properly designed to support the claim that the Outcome was achieved. Even if it was achieved, no data was provided to support that claim. The remaining 12 poorly designed Outcome Indicators were poorly designed due to either self-reporting bias or conflict of interest when an external consultant should have been used instead to objectively measure whether the outcome was achieved. They used an external consultant for the 1 good Outcome Indicator but oddly refused to do it for the other Outcome Indicators!! Other poorly designed Outcome Indicators were due to measures that had nothing to do with whether the Outcome was achieved.For the Building Bricks CUSO International project, the PMF had 8 Outcomes and 15 Outcome Indicators. 4 of those Outcome Indicators were properly designed to support the claim that some of the Outcomes were achieved. Even if they were achieved, no data was provided to support that claim. The remaining 11 poorly designed Outcome Indicators were poorly designed due to either self-reporting bias or due to measures that had nothing to do with whether the Outcome was achieved.CUSO International has been invited to attend the podcast to provide a response to this critique of their project PMFs. The PMF critique and PMFs have been sent to CUSO International as well as the Secretary of State for International Development and the shadow critics for International Development.For a 20/15% discount on sunhats, sunglasses, and cotton polo shirts when visiting the Global South use this discount link: https://mydeals.page/1hjlDonate here: https://www.paypal.com/ncp/payment/ZAQD8888DEDXLOr at buymeacoffee.com/davidwand to help the podcast

November 13, 202521 min

GCERF fails to show it is achieving its outcomes from spending $1.1 Million USD from Global Affairs Canada

In this episode, the Global Community Engagement Resilience Fund (GCERF) (www.gcerf.org) fails to evaluate the effectiveness of its $ 1.1 Million USD received from Global Affairs Canada. GCERF claims to be achieving its 4 outcomes by funding organizations in developing countries to deliver services to achieve those outcomes. But GCERF provides no measures on these outcomes through outcome indicators nor any data related to those outcome indicators in the document they sent to Dr. Wand. This document was a response to his request for the outcome indicators and their data used to measure changes in these outcomes. The 4 outcomes that GCERF claims to be achieving with this $1.1 Million USD Government of Canada funding are:Increased social cohesionIncreased ability to mobilize, organize, represent their own interests (agency)Increased knowledge, skills, and subsequent increases in income/livelihoods.Increased confidence, critical thinking, life skills, self-worth, resilience (sense of purpose).GCERF claims on its website, "At grant level, each project is guided by a theory of change and a results framework with outcome and output indicators, requiring grantees to collect baseline data, report quarterly, and conduct final assessments."GCERF is falsely claiming to Canadians that they have achieved these 4 outcomes because GCERF provides no outcome indicators nor data from these outcome indicators showing that social cohesion, agency, skills & income, or sense of purpose have increased.Dr. Wand has sent his critique of this GCERF document to the Secretary of State for International Development as well as the shadow critics for International Development. He has recommended that all funding to GCERF be suspended until outcome indicators and the data used to measure the achievement of these outcomes is provided on their website for all grants including this grant, if such outcome indicators and data exist.Sign the petition here to require GCERF to put all its PMFs and related data on its website:www.change.org/EvaluateCanadaAidGCERF has also been sent the critique of their response as well as an invitation to be on the podcast to respond and offer actions to address the absence of outcome measures for their expected outcomes or to respond in writing to evaluatecanadaaid@gmail.com.https://mydeals.page/1hjlDonate here to increase the number of organizations that receive performance audits like this one: https://www.paypal.com/ncp/payment/ZAQD8888DEDXLOr at buymeacoffee.com/davidwand

September 19, 202545 min

Oxfam Canada (www.oxfam.ca) fails to evaluate its 2 projects worth $13,140,000: Care Connect and Camino Verde

Sign the petition here to require Oxfam to put all its PMFs and related data on its website:www.change.org/EvaluateCanadaAidIn this episode, Dr. Wand reviews the Performance Measurement Frameworks (PMFs) for 2 Oxfam Canada (www.oxfam.ca) projects: Care Connect https://w05.international.gc.ca/projectbrowser-banqueprojets/project-projet/details/p012493001 at a cost of $5,950,000 and Camino Verde https://w05.international.gc.ca/projectbrowser-banqueprojets/project-projet/details/p005232001 at a cost of $7,190,000. He concludes from his review that Oxfam Canada has failed to design proper outcome indicators (with the exception of 2 outcome indicators out of 18 outcome indicators for the Care Connect project) to meet the Development Assistance Committee (DAC) criteria of achieving effectiveness or impact for these 2 projects. In addition, Oxfam Canada has failed to provide any outcome indicators in its PMF for 10 of the 14 outcomes listed in its Camino Verde project. Therefore, Oxfam Canada cannot make the claim that their projects are achieving their expected outcomes of effectiveness and impact.Dr. Wand has sent this episode along with his critique of the PMFs and the 2 original PMFs to the Secretary of State for International Development as well as the shadow critics for the Conservative party, the New Democratic Party, and the Bloc Quebecois party. He has recommended to all of these individuals that Government of Canada funding for these 2 projects be stopped immediately until proper outcome indicators are designed so that proper evaluation of the effectiveness and impact of these projects can be achieved.Oxfam Canada has also been sent this episode as well as the critique of the PMFs and the 2 original PMFs inviting them to respond to the critique, including an invitation to attend a future podcast to discuss their flaws in their project PMF outcome indicators as well as their proposed actions to address these flaws.If you wish to receive your discount on the purchase of sun hats, sunglasses, or polo shirts designed for the Global South, select the link below:https://mydeals.page/1hjlIf you wish to make a donation to this podcast click on either of the links below:https://www.paypal.com/ncp/payment/ZAQD8888DEDXLhttps://buymeacoffee.com/davidwandListeners can receive the PMFs and my critique of the PMFs by sending an email to: evaluatecanadaaid@gmail.com

February 7, 202514 min

Global Community Engagement Resilience Fund (www.gcerf.org) fails to evaluate its $5 Million USAID funding

In this episode, the Global Community Engagement Resilience Fund (www.gcerf.org) fails to evaluate its $ 5 Million USD received from the United States Agency for International Development. Although GCERF claims on its website to be achieving its 4 expected outcomes through funding its implementing partners in developing countries to deliver services to achieve those outcomes, they provide no evidence in the form of outcome indicators and any data related to those outcome indicators on its website. The 4 expected outcomes that GCERF claims to be achieving with this $5 Million USAID funding are:Increased social cohesionIncreased ability to mobilize, organize, represent their own interests (agency)Increased knowledge, skills, as well as subsequent increases in income/access to livelihoods.Increased confidence, critical thinking, life skills, self-worth, resilience (sense of purpose).In addition, letters from the USAID and U.S. State Department indicate that no records exist specific to results frameworks/outcome indicators and related data for these expected outcomes for this $5 Million USD given to GCERF from the US AID.GCERF is falsely claiming to the American people on their website with their taxpayer funds that they are achieving these expected outcomes because they provide no evidence on their website of achieving these outcomes in the form of outcome indicators and data to show that these expected outcomes have been achieved.As a result, the Improving Development Evaluation podcast has mailed a letter along with copies of the letters from the USAID and U.S. State Department to the Chairman of the Foreign Affairs Committee, Brian Mast, requesting that he request GCERF to post all their results frameworks, including all outcome indicators and data for those outcome indicators that measure the achievement of their 4 expected outcomes, from all the grants/projects produced by GCERF in collaboration with their implementing partners that have come from the original $5 Million USD that GCERF received from USAID up to September 15, 2023 specific to Activity ID; 77266; Project #: SCTOOP21VC004; International Sector: Government & Civil Society; USG Sector: Counter-Terrorism; Managing Agency: Department of State; Funding Agency: USAIDhttps://mydeals.page/1hjlDonate here to increase the number of organizations that receive performance audits like this one: https://www.paypal.com/ncp/payment/ZAQD8888DEDXLOr at buymeacoffee.com/davidwand

November 13, 202431 min

Doctors of the World: Flaws in its Performance Measurement Framework - Access to health rights Burkina Faso

Sign the petition here to require Doctors of the World to put all its PMFs and related data on its website:www.change.org/EvaluateCanadaAidIn this episode, Dr. Wand reviews the Government of Canada-funded $14 Million Doctors of the World https://medecinsdumonde.ca/ project entitled Access to health rights in Northern Burkina Faso. Details about the project can be found here: https://w05.international.gc.ca/projectbrowser-banqueprojets/project-projet/details/P008979001From a review of the project's Performance Measurement Framework (PMF), he concludes that Doctors of the World cannot claim that their project is responsible for achieving any of the 8 expected outcomes due to the absence of control or comparison groups as well as invalid or missing outcome indicators for some of the 8 expected outcomes for the project.Dr. Wand has emailed the link to this episode as well as the PMF and summary Excel critique to the Minister of International Development, Doctors of the World, and the shadow critics for international development for the Conservative Party, the NDP, and the Bloc Quebecois.Dr. Wand has also invited Doctors of the World to respond to his critique in writing and/or be attending his podcast.Listeners can receive the PMF and summary critique by emailing evaluatecanadaaid@gmail.com.https://mydeals.page/1hjlDonate here to increase the number of organizations that receive performance audits like this one: https://www.paypal.com/ncp/payment/ZAQD8888DEDXLOr at buymeacoffee.com/davidwand#international development#evaluation

October 25, 202426 min

UNFPA Somalia reproductive health project: Flaws in the Performance Measurement Framework

Sign the petition here to require UNFPA to put all its PMFs and related data on its website:www.change.org/EvaluateCanadaAidIn this episode, Dr. Wand reviews the Government of Canada-funded $10 Million UNFPA Somalia project entitled Strengthening Sexual Reproductive Health and Rights through Midwives. Details about the project can be found here: https://w05.international.gc.ca/projectbrowser-banqueprojets/project-projet/details/p007483001He reviews the Performance Measurement Framework (PMF) for this project and notes flaws in the outcome indicators whereby the UNFPA Somalia country office cannot claim that they have achieved the following project outcomes:Reductions in maternal mortality;Increased quality of midwifery trainingIncreased skill of midwifery associations to lead and advocate;Increased knowledge and awareness of maternal nutrition.However, properly designed outcome indicators in the PMF do support the opportunity to show the achievement of increased access to and use of sexual reproductive health services, although actual data needs to be provided.Listeners can receive the PMF and the full summary of the Excel outcome indicators by emailing evaluatecanadaaid@gmail.com. Those documents and this podcast will be shared with the Minister for International Development as well the shadow critics for the Conservative party, the NDP, the Bloc Quebecois, and the Green Party.https://mydeals.page/1hjlDonate here to increase the number of organizations that receive performance audits like this one: https://www.paypal.com/ncp/payment/ZAQD8888DEDXLOr at buymeacoffee.com/davidwand#international development#UNFPA Somalia#evaluation