Don't scale the excitement; evaluate what's working.
The smartest move in AI today is to evaluate what works before you scale on excitement alone. What happens when Low and Middle Income Countries start pulling investments away from traditional health, judicial, or extension systems in order to build AI solutions instead? It can have huge upsides, but if you're only testing the AI and not the impact for people, it could lead to "irreversible losses." Listen to Attaullah Abbasi from J-PAL talk about their new AI Evidence Playbook. AI is moving faster than the evidence can keep up, and that means people are often making decisions about if an AI "works" with engagement metrics like user adoption and message volume. Those can be dangerously misleading. The success of many AI tools drops dramatically when you release them into the wild, because if people don't have the bandwidth to use them well, or the system can't act on what the AI found, your investment isn't delivering real change. Atta talks about the difference between the speed of evidence, the speed of technology, and the speed of change. Your AI tool might evolve hundreds of times weeks, but it's still going to take a whole school year to see if kids passed the end of year reading exam. The evidence playbook gives you practical tools to understand what is working and what's not, and what methods will give you good enough information to move to the next phase of investment.
