
Why AI Compliance Agents Need a Harness, Not Just a Model With Peter Piatetsky, CEO of Castellum
Peter Piatetsky is the CEO and co-founder of Castellum.AI , which started in 2019 as a primary source for sanctions, PEP and adverse media data and now builds AI agents that resolve compliance alerts for banks and fintechs. A former intelligence analyst, OFAC analyst and bank compliance head, he has seen the alert overload problem from every side. We talk about what actually happens when an alert fires, why the real question today is build and maintain versus buy, and what Castellum's benchmark shows about LLMs with and without a harness. What We Covered From intelligence analyst in Afghanistan to OFAC Becoming lead Russia analyst when Russia first invaded Ukraine Running US compliance for a major Asian bank Why Castellum chose to be a primary data source Going modular to meet clients where they are What happens when an alert hits the Arbiter agents Training an agent on a client's policies and procedures in 10 business days Why the AI reliability debate is over Build and maintain versus buy Benchmarking nine models with and without a harness Running inside LexisNexis Bridger and Fircosoft How criminals are using AI at scale Key Takeaways The harness matters more than the model. In Castellum's testing of nine models, top LLMs on their own were accurate about 70% of the time when asked to read policies, screen, create and adjudicate an alert. With Castellum's harness of specialized data, training, regression testing and QA/QC, accuracy rose to 99% and consistency got close to 100%. The question is no longer whether AI can do this. Within a year, buyers have moved from doubting AI agents to asking whether they have budget this year or next. The common objection now is "why not just use Claude," which Piatetsky frames as build and maintain versus buy, including retesting every new model on hundreds or thousands of cases. Sit on top of what banks already have. Castellum's agents run inside existing screening systems such as LexisNexis Bridger and Fircosoft, so clients avoid new model validations and tuning reports. His analogy: a water filter on the tap instead of ripping out all the pipes. L1 is going away, and that could be good news. Piatetsky expects L1 alert review to be fully automated and much of L2 to follow, freeing analysts to focus on alerts that matter and giving large institutions real visibility into what is happening inside them for the first time. About Peter Piatetsky Peter Piatetsky began his career as an intelligence analyst in Afghanistan, then joined the US Treasury Department, where he worked at OFAC on Iran and later became the lead Russia analyst. He went on to run US compliance for a major Asian bank before co-founding Castellum.AI in 2019. Castellum's clients include a top four US bank, a top three global payments provider and a top five global crypto exchange. Connect with Fintech One-on-One: Tweet me @PeterRenton Connect with me on LinkedIn Find previous Fintech One-on-One episodes















