Find partners
The AI podcast for product teams

The AI podcast for product teams

Hosted by Arpy Dragffy

Episodes

46

Latest episode

May 2026

Language

EN

About the show

Podcast and newsletter for product teams looking to deliver innovative AI products and features productimpactpod.substack.com

Listen to episodes

46 recent
May 21, 202629 min

Build the Context Layer Before the Agent

Atlassian spent three years connecting 150 billion organizational objects before the results appeared: 44% more accurate AI answers, 48% fewer tokens, a coding agent that reviewed 2 billion lines of code in two minutes. That’s the proof enterprises are pointing to when they argue that context graphs are the unlock. What the benchmark obscures is the order of operations — the graph had to exist before any of those numbers were possible. The reorganization bet is running in parallel, and it’s moving faster than the infrastructure. Airbnb’s CHRO is converting documentation to markdown, building skills libraries, mining meeting recordings before institutional memory disappears — five structural prerequisites before the first agent goes live. Meta is posting $26.8 billion in Q1 profit, laying off 8,000 people, and reporting “horrifically, historically low” employee morale. Both are restructuring around AI. Only one is sequencing it correctly. In AI customer experience, Twilio is working against a Qualtrics finding that 1 in 5 AI interactions delivers zero benefit. Rikki Singh’s diagnosis is precise: the orchestration layer is there, but it’s running without the context layer underneath it. FAQ automation with better packaging is still FAQ automation. The unlock is real — but only when all three pieces are in place, in order. The knowledge worker playbook in this edition addresses the fourth variable: what happens to the people whose roles disappear when the gathering does. Rikki Singh leads product innovation at Twilio — what the company is calling its biggest launch in 17 years. Before Twilio she was at McKinsey, where she co-authored the foundational research on what makes a great PM. The Qualtrics 2026 CX Trends Report found nearly 1 in 5 consumers who used AI customer service saw zero benefit — the baseline she is working against. * Why AI CX is still FAQ automation with better packaging * Why AI spend is as unpredictable as AI upside * The wrapper that makes AI feel like it thinks * Vitamins vs painkillers: the product sense filter * How to protect long-horizon bets inside a big company * Why the brand — not the vendor — owns AI failure Listen: Spotify | Apple Podcasts Jamil Valliani leads AI product at Atlassian, where he has spent three years building the Teamwork Graph across 300,000 companies. Recorded live at Team ‘26 in Anaheim, where Atlassian demonstrated what connecting 150 billion organizational objects produces: 44% more accurate AI answers using 48% fewer tokens, and a coding agent that reviewed 2 billion lines of code in 2 minutes. * Why your team spends 80% on gathering, not deciding * The adoption pattern that turns skeptics into converts * How to build trust with AI one small task at a time * Why giving AI less data often gets you a better answer * How leaders stop waiting for Friday status reports * From 2 ideas to 10: the creative unlock nobody explains “You didn’t hire your team to write reports. You hired them to advance the business forward.” — Jamil Valliani Listen: Spotify | Apple Podcasts No one is measuring ROI & fewer understand knowledge graphs We attended Atlassian Team ‘26 in Anaheim to cover the Teamwork Graph and what knowledge graphs actually mean for the future of work. Key learnings: * Everyone is in such a rush to increase adoption numbers that no one cares to measure ROI, only velocity * In the rush to adopt, many orgs are discovering dozens of agents built by individuals that are unsanctioned and eating up tokens * While there’s excitement about announcements about getting access to more context, few understand what to do with the context that’s currently available to them today 📅 productimpactpod.com is the hub for AI product strategy, news, and analysis. All the articles featured in this edition are sourced from Product Impact’s own reporting. WTF Is an AI-Native Org Anyways? Let’s Compare Airbnb & Meta’s Opposing Plans. The term “AI-native” gets used without definition until it has to mean something operationally. Airbnb’s CHRO Iain Roberts gave the clearest working answer at Stanford: restructure before you deploy. His five moves — converting documentation to markdown, building reusable “skills” libraries, mining meeting recordings before institutional memory disappears, requiring leaders to personally build AI tools, and hiring org architects — are structural prerequisites, not technology decisions. Microsoft’s 2026 Work Trend Index found only 19% of AI users have reached the zone where AI actually compounds knowledge work. Airbnb is building toward that 19%. Most organizations are spending to reach 88% adoption while still operating in the 81% where AI doesn’t compound. Meta posted $26.8 billion in Q1 2026 profit, announced 8,000 more layoffs, and reported “horrifically, historically low” employee morale — all three within the same quarterly cycle, and all structural rather than cyclical. Anthropic’s Economic Index from March 2026 found experienced AI users dramatically outperform newcomers — not because of tool access, but because of how they use it. The organizational design question is whether your structure accelerates that difference or suppresses it. Read at productimpactpod.com → OpenAI & Anthropic Are Charging Us Way More Than We Need Stanford’s 2026 AI Index found inference costs dropped roughly 90% over 18 months. Enterprise AI budgets went the other direction: Forrester’s Q1 2026 found 78% of enterprises exceeded their 2025 budget by an average of 47%, and Gartner found spend climbing faster than value metrics in every tracked category. Engineers who deliberately match task complexity to model capability pay 15 times less per query than those defaulting to frontier. Anthropic’s extended thinking feature bills at a 2x–5x output multiplier on top of the Opus base rate. The AI Value Acceleration Token Economics Crisis report found power users running $500–$3,000+ per month. The frontier-model upsell follows the iPhone playbook: premium defaults, friction for alternatives, the appearance of necessity. Most of what runs on Opus would run adequately on Sonnet, and almost no one is running the comparison. Read at productimpactpod.com → AI Value Acceleration is building a report on where enterprise AI investments are actually creating value. If you’re responsible for a major AI investment — leading it, funding it, or proving it’s working — we want to talk to you . Playbook for Knowledge Workers to Survive the AI Jobpocalypse Stanford’s 2026 AI Index found employment for software engineers ages 22–25 is down 20% since 2024. Dario Amodei has said 50% of entry-level white-collar work disappears within one to five years. Research from MIT Media Lab, Microsoft, and CMU found AI use reduces cognitive engagement — workers producing output faster while generating less original thinking. The professional survival question is whether the work remaining after displacement is work you are positioned to do. The playbook addresses four paths for senior knowledge workers: navigate the current org redesign, move to a better-positioned organization, pivot to an adjacent domain, or go independent. UX researchers, digital marketers, project managers, agency account managers, educators, and designers are the specific roles addressed — not because they are uniquely at risk, but because they are the professions where the gap between AI-assisted and non-assisted output is currently largest, and where the window to close that gap is still open. Read at productimpactpod.com → Google’s AI Overviews reach 2.5 billion monthly users. AI Mode reaches 1 billion. ChatGPT’s weekly active user count is 0.9 billion. Distribution is Google’s structural advantage — and the question of whether that reach translates into agent utility is the most important unresolved problem in AI products right now. Source: TechCrunch / Department of Product. AI Strategy News * 54% of C-suite executives admit AI adoption is tearing their company apart. 75% say their AI strategy is “more for show” than actual internal guidance. … This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit productimpactpod.substack.com

May 12, 202645 min

Governance, Context, and the Org-Design Reckoning

Atlassian connected its AI agents to a richer layer of company knowledge (documents, projects, goals, people) and measured a 44% improvement in answer accuracy using 48% fewer resources. Same models. Different information. Brian Armstrong restructured Coinbase the same week: 14% headcount cut, five management layers maximum. When AI can surface what previously required institutional memory and senior tenure, the organizational layers built around that knowledge become harder to justify. The visible shift gets covered in tech headlines. What gets lost in the announcement energy: none of this works if the company hasn’t decided what it wants AI to do. The more widespread barrier is upstream of governance. Most executives approving AI budgets are working through the aftermath of pilots that underdelivered, first-generation deployments that didn’t survive contact with their actual data, and early model results that left skepticism the current tools have since substantially outrun. That trust deficit — organizations evaluating new AI investment based on experiences two generations old — is where enterprise AI projects most commonly stall. Shadow AI governance and deployment intent are real risks, but they’re downstream of that harder problem. There is no closing the capability gap inside an organization that is quietly waiting for the next deployment to fail too. John Willis co-wrote The DevOps Handbook because software teams were shipping code fast without feedback loops or governance. He sees the same pattern repeating with AI — and he spent five decades documenting what happens when the gap between vendor promises and operational reality gets this wide. * Why shadow AI is more dangerous than an outright ban * Why throughput without governance means instability at scale * Why governance creates flow instead of stopping it * Why most teams have ML evaluation tools when they need audit trails * Why even a five-person startup needs digitally signed records of agent decisions * What AI winters teach us about where we actually are now Listen: Spotify | Apple Podcasts Rikki Singh leads product innovation at Twilio — what the company calls its biggest launch in 17 years. Before Twilio she was at McKinsey, where she co-authored the definitive research on what makes a great PM. The Qualtrics 2026 CX Trends Report found nearly 1 in 5 consumers who used AI customer service saw zero benefit. That number is the benchmark she is working against. * Why most AI CX is still FAQ automation with better packaging * Why the LLM wrapper creates false confidence — the model generates strings, it is not thinking * Vitamins vs painkillers: how to parse what customers don’t say out loud * How to protect long-horizon bets inside a public company * Why the brand owns the accountability when AI gets a high-stakes interaction wrong Listen: Spotify | Apple Podcasts 📅 productimpactpod.com is the hub for AI product strategy, news, and analysis. All the articles featured in this edition are sourced from Product Impact’s own reporting. AI Value Acceleration is building a report on where enterprise AI investments are actually creating value. If you’re responsible for a major AI investment — leading it, funding it, or proving it’s working — we want to talk to you . Every CEO Will Post a Layoff Notice Like This. Here Is Why. Brian Armstrong’s May 5 Coinbase memo framed a 14% headcount cut as a structural prerequisite for AI adoption, not a consequence of it. Three principles: hard cap of five management layers, player-coaches who produce output alongside their teams, and AI-native pods where one person spans engineering, design, and product with agent support. Sequoia’s 2026 analysis found AI-native startups already ship three times faster with 60% fewer engineers — that’s the economic gap the restructuring is attempting to close. The jobs being cut are not cyclical. Investor metrics increasingly measured by revenue per employee, AI capex commitments requiring demonstrated productivity returns, and talent migration toward AI-native organizations are three pressures no individual CEO can deflect. The new career metric is leverage — how much the organization moved because of the quality of your judgment, not how many tasks you completed. Read the full analysis at productimpactpod.com Context Is Now the AI Moat The biggest blocker to enterprise AI has never been the model — it’s been knowledge built in siloes, in incompatible systems, in formats AI tools can’t use. Atlassian’s Teamwork Graph benchmarked the fix: 44% more accurate answers, 48% fewer tokens, same models. Every major vendor — Microsoft, Glean, ServiceNow, Salesforce, Google — is converging on the same architecture. Whoever owns the most accurate map of how a given enterprise operates will own the AI layer running on top. What these tools don’t solve is why. Your documentation captures decisions; it rarely captures the reasoning behind them. Without that reasoning, AI searches across everything available and keeps going past the point where a senior employee would have stopped. Anyone who has watched an AI tool go confidently down the wrong path has seen this at small scale. Scale that across a hundred agents inside an enterprise when the intent was never established. Read the full analysis | What Graphs Are and Why They Matter — Brittany Hobbs’s explainer on the strategic shift for researchers and product leaders. Brittany’s UXR Series: The AI Shift for Researchers Part 1: The UX Researcher’s Guide to Claude, Claude Cowork, and Claude Code — Which tool fits which workflow stage, privacy risks broken down by tier, and how prompt caching and the Batch API cut transcript processing costs by 50–90%. Part 2: The Cognitive Shift Every UX Researcher Needs to Make — MIT EEG research found AI-assisted knowledge workers showed lower brain engagement and produced homogenized output — with the deficit persisting after they stopped using the tool. Four shifts separate researchers compounding value from those producing faster mediocre work. Build explicit contradiction-seeking into every analysis prompt: it does more for output quality than anything else. Part 3: What UX Research Looks Like When Context Becomes the Engine — The role is moving from a step that produces decks to infrastructure that powers everyone else’s AI. Researchers who get ahead of this shift become the people who decide what gets captured, connected, and trusted — a more powerful position than any role the previous workflow offered. Thanks for reading Product Impact | AI Strategy, Value Creation, AI UX! Subscribe for free to receive new posts and support my work. Product Impact Resources * 1 in 5 consumers who used AI customer service saw zero benefit from the interaction. The bar enterprises are calling AI innovation is shockingly low, and customers feel it every time they’re routed to a bot reading from an FAQ. Qualtrics 2026 Customer Experience Trends Report * AI power users outperform everyone else by 6x — with identical tools available to both groups. The differentiator is approach, not access. Context graphs are about to commoditize the advantage power users built manually. OpenAI productivity analysis via VentureBeat * 49% of UX researchers now feel negative about the future of their discipline — a 26-point shift in a single year. Job postings fell 73% from 2022 to 2023 and haven’t recovered. 21% of organizations laid off researchers in 2025. User Interviews State of User Research 2025 * AI-native startups ship three times faster than traditional companies with 60% fewer engineers. The economic gap is real, and it is what is driving the org-design decisions currently being framed as ideology. Sequoia Capital 2026 AI analysis * Thousands of apps built with vibe-coding tools have exposed corporate and personal data publicly — credentials, API keys, customer records — on the open web. The governance failure isn’t limited to enterprise AI deployments. It’s in every tool that makes building easier without making security more obvious. Wi… This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit productimpactpod.substack.com

April 29, 202618 min

The Most Important Data Points in AI Right Now

The Stanford AI Index ’s headline is 88% — organizations using AI in some capacity. The Financial Times charted where it actually lands in the workforce: 62% of top-decile earners use it daily, versus 13% at the bottom. Board decks this quarter will cite Stanford. The FT chart is what they’re not showing. The economics that enabled this gap are under pressure. The three-year subsidized era is ending by financial necessity, not choice. The same optimization logic that built social media’s loneliness machine is now embedded in AI products at scale. And in the same week Anthropic’s most capable model autonomously found 271 zero-days in Firefox, two major platforms were breached through third-party integrations. The data and what to do about it follows. Episode 8: The Most Important Data Points in AI Right Now Brittany Hobbs solo — four segments moving from data to strategic implication. Essential for anyone making AI purchasing, hiring, or architecture decisions right now. The Stanford AI Index 2026. 88% organizational adoption is saturation, not a trend. $581 billion invested globally in 2025, up 129% year over year. The US-China AI performance gap collapsed from 17–31 percentage points in 2023 to 2.7% today — on 23 times less investment. China holds 69.7% of global AI patent filings. Architecture and application discipline closed a gap that capital alone could not. Stanford AI Index 2026 | The U.S. Can’t Buy an AI Lead Token economics. Anthropic’s current tiers: Haiku at $1/$5 per million input/output tokens, Sonnet at $3/$15, Opus at $5/$25. A 200-screen product built with Claude Design costs $0.22 for a first draft; the 50-iteration refinement cycle real design work requires runs to ~$2,600, plus $200–$900/month in system updates. Every comparable Figma interaction costs zero. Prompt caching provides ~90% discounts on repeated context; batch processing cuts 50%. Claude Design vs Figma cost breakdown | CNBC: Token economics Apple chose its hardware chief as next CEO. John Ternus — SVP of Hardware Engineering, architect of Apple Silicon — succeeds Tim Cook on September 1st. Johny Srouji, who designed every Apple Silicon chip, becomes Chief Hardware Officer. Apple posted $143.8 billion in Q1 FY2026 (up 16%, $109 billion in services, 92% retention) without shipping an industry-leading AI feature. The next decade of AI is decided at the silicon and device level. Apple CEO transition analysis Vibe coding has never been more capable. Security has never been more exposed. Anthropic’s Mythos model identified 271 zero-day vulnerabilities in Firefox autonomously; the UK’s AI Security Institute found it succeeds at expert-level hacking tasks 73% of the time. Anthropic launched Project Glasswing (12 defensive security partners including Amazon, Microsoft, and Apple), then reported unauthorized Mythos access through a vendor. Vercel was breached through Context AI — customer credentials sold on BreachForums for $2 million. Lovable exposed source code and credentials via a basic authorization flaw for 48 days, fixed it, then broke it again for 76 more. TechCrunch: Anthropic Mythos | TechCrunch: Vercel breach | The Next Web: Lovable “If you’re making AI decisions for your team right now — what to buy, who to hire, what to build — there are numbers out this week that should change your approach.” — Brittany Hobbs Listen now: Spotify | Apple Podcasts | YouTube 📅 productimpactpod.com just launched as a news platform. All the Stanford breakdowns, token economics case studies, and Apple CEO analysis are sourced from Product Impact’s own reporting — check it out. AI Value Acceleration is building a report on where enterprise AI investments are actually creating value. If you’re responsible for evaluating a major AI investment — leading it, funding it, or proving it’s working — we want to talk to you . Adobe’s head of customer experience said it at Adobe Summit: “Tokens don’t equate to value.” He was announcing outcome-based pricing per campaign completed, per interaction resolved — because tokens are a unit of compute, and the VC subsidies that made that someone else’s problem are ending. OpenAI will lose $14 billion in 2026 spending $2.25 per dollar earned; Anthropic has spent over $10 billion against under $5 billion in lifetime revenue. GitHub paused new Copilot signups as costs nearly doubled since January. Anthropic briefly removed Claude Code from its $20/month Pro plan , with its Head of Claude Code acknowledging the plan “wasn’t built for the usage patterns of these third-party tools.” Token prices dropped 280x in two years while enterprise bills rose 320% — because agentic workflows trigger 10–20 LLM calls per task. DeepSeek V3 at $0.14/M tokens versus Sonnet’s $3/M makes model routing a financial decision available today. Adobe and Salesforce’s “agentic work units” are already moving to results-based pricing. The question every executive needs answered before the next AI contract: what are we willing to pay per unit of value delivered? Read the full analysis at productimpactpod.com Marc Andreessen’s Techno-Optimist Manifesto has no words for belonging, community, or loneliness. Technology designed without the social being doesn’t accidentally serve community — it systematically displaces it, and we’ve seen this before. Social media became the primary arena for teen social life in 2012; depression and self-harm rates rose in lockstep with smartphone adoption across four English-speaking countries. By 2023, the Surgeon General had declared loneliness a public health epidemic . AI companions are filling the same space: 75% of US teens have tried one ; heavier use correlates with greater loneliness . The MIT Media Lab found 83% of LLM-assisted writers couldn’t quote their own work afterward . James Evans’s Nature study found AI adoption shrinks the topics scientists explore by 4.6%. OpenAI’s GPT-4o post-mortem confirmed RLHF trains for approval, not accuracy. Builders in 2026 have more evidence about second-order effects than social-media teams had in 2012. Helen and Dave Edwards’ cognitive sovereignty framework — awareness, agency, accountability — makes the design requirement operational: every decision either builds those capacities or erodes them. Five commitments that change the brief: empowerment over ease; community as the durable moat; meaning preserved in automated workflows; returned time pointed somewhere worth going; the most durable value is offline. Read the full analysis at productimpactpod.com Product Impact Resources The tools have reached everyone. The scaffolding — peer visibility, psychological safety, domain depth — has not. AI is amplifying existing advantage, not democratizing it. The 10% Problem: AI’s Value Gap Is Wider Than Anyone Is Admitting * Developers felt 20% faster using AI — they were 19% slower on complex tasks. A Harvard-linked study found developers using AI coding assistance reported strong subjective productivity gains while measurably underperforming on complex work, with longer workdays. Subjective adoption metrics and actual output moved in opposite directions. If your AI program measures how people feel about the tools, it is not measuring whether the tools are working. The AI Trap: Faster Solution, Same Problem — CIO * 47% of enterprise AI users have made major decisions based on hallucinated content. Not minor ones — major. Hurix Digital, 2026 . Without verification steps at decision points, you are running AI-generated decisions without review. * AI governance is a 20-point performance advantage. Organizations with strong governance see 70–75% positive AI project outcomes versus 50–55% without. Fewer than 1% have fully operationalized responsible AI. WEF: AI Governance Isn’t Holding Businesses Back * 78% of executives say they couldn’t pass an AI governance audit. 46% cite governance failure as the leading cause of their AI underperformance. Only 6% are prioritizing the fix — change leadership and workforce enablement. Organizations running integrated AI (n… This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit productimpactpod.substack.com

March 27, 202646 min

Your AI Strategy Is a Pile of Demos

Let’s stop pretending. Most AI strategies are just a collection of pilots that nobody had the courage to kill. The data this period is brutal: 95% of genAI pilots stall. Only 11% reach production in financial services. Microsoft — the biggest company in the world, with the best distribution on the planet — just reorganized Copilot because nobody internally could agree on what it was supposed to be. And while enterprises burn cycles debating governance frameworks, a new class of startups is quietly replacing entire job functions. Not assisting. Replacing. The gap between the people who get this and everyone else isn’t a skills gap. It’s a courage gap. This edition is about which side you’re on. What You’ll Learn in This Edition This edition confronts the uncomfortable reality that most AI investments are producing demos, not outcomes — and the structural reasons why. * 🎙 Why agents are automating your thinking, not just your tasks — and why that distinction matters more than any model release * ✍️ Copilot’s identity crisis is the most important product failure of 2026 so far * 👉 The single variable that predicts AI maturity 7x better than technology choices * 1️⃣ Why advertising AI use is now a financial liability for professional services firms * 2️⃣ The inference cost crisis that threatens every AI business model — including OpenAI’s Episode 4: The Era of Agents — Your Cognition Is the Product Now We mapped three years of AI evolution in this episode and landed somewhere uncomfortable. Era one gave us wrong answers. Era two gave us wrong context. Era three — agents — is giving us wrong actions. And the stakes compound with each era because AI is no longer just saying things. It’s doing things. Brittany brought the number that should haunt every product leader: only 6% of organizations have fully deployed any kind of agent. Copilot hit 30% weekly active usage after six months — meaning 70% of enterprise users basically stopped opening it. The tools are moving at an extraordinary pace. Almost nobody is keeping up. We profiled four startups winning the point-solution war that most people haven’t heard of. But the real conversation was about what happens when you hand your thinking to an agent. Not your typing. Not your scheduling. Your thinking — the research, the monitoring, the analysis, the synthesis. Something changes in you when you do that. And most people haven’t reckoned with what that means. “We’ve trained generations of people to think linearly. Step one, step two, step three, fill out this form, follow this process. Agents don’t work like that. Agents require you to think in terms of outcomes, connections, and context.” — Arpy Listen now: Spotify | Apple Podcasts | YouTube You’re invited to join the AI Strategy Experiments Zoom call today Today (March 27) at 1pm ET we’re hosting a small group of strategists and builders and designers sharing their experiments and questions. Register here . $490 billion in enterprise AI spending is delivering nothing. That’s not a technology failure. It’s a value creation failure. AI Value Acceleration exists to close that gap — diagnosing where AI value stalls and building playbooks that actually work. Value Assessment in 3 weeks. Value Amplification to go deep. Value Acceleration to prove what works. aivalueacceleration.com Copilot Didn’t Fail. It Succeeded at Not Knowing What It Is. Bloomberg reported that internal confusion over Copilot’s role, personality, and strategy has prompted a reorganization at Microsoft. Read that again. Internal confusion. Not external competition. Not technical limitations. The people building Copilot couldn’t agree on what it was for. Microsoft had everything a product could dream of — billions in funding, integration into every Office app, the largest enterprise distribution network on earth, and access to the most powerful models available. It didn’t matter. Without a clear product identity, all that distribution just delivered confusion at scale. The uncomfortable truth: most AI products shipping today have the same disease. They’re a bundle of capabilities searching for a purpose. They demo beautifully. They onboard poorly. They get abandoned quietly. If the biggest company in the world can’t brute-force its way to product-market fit for an AI assistant, what makes you think your team can skip the hard work of defining what your AI product is actually for? BCG: Why Usage Is Up but Impact Is Not Employee-centric organizations are 7x more likely to be AI mature. Not 7% more likely. Seven times. Employee-centricity explains ~36% of variance in AI maturity outcomes. Model selection explains almost none of it. Over 85% of organizations remain stuck at basic task assistance. Fewer than 10% have reached anything resembling semiautonomous collaboration. The teams pulling ahead didn’t start with better tools. They started with cultures where people felt safe to experiment, fail, and teach each other what they learned. HBR confirmed it separately : peer influence is the single most powerful predictor of AI adoption. When learning stays private, adoption stalls. This is exactly why I built AI Value Acceleration — because the gap between what AI can do and what your organization is actually doing with it isn’t a technology gap. It’s a leadership gap. And closing it starts with measuring where value is being created, lost, and why. Deloitte Put a Price Tag on Hallucinations. Then KPMG Made It Worse. Deloitte issued a refund to the Australian government after errors in an AI-generated report. That’s a sentence that should terrify every professional services firm shipping AI-assisted work without rigorous review. But the follow-up is even more revealing: a competitor reportedly pushed KPMG to cut prices specifically because KPMG advertised AI use . Think about that. Advertising AI didn’t increase perceived value. It decreased it. Clients heard “we use AI” and thought “then why am I paying you full price?” This is a new failure mode that nobody war-gamed: AI claims eroding the very premium they were supposed to justify. Every consulting firm, agency, and services company racing to slap “AI-powered” on their pitch decks needs to answer one question first — does your client believe they’re paying for AI’s work or yours? Because if it’s AI’s work, they’ll expect AI prices. Product Impact Resources Every resource this period points to the same conclusion: the companies pulling ahead aren’t chasing model releases. They’re building the structural layers — verification, governance, integration depth — that turn capability into production value. Everyone else is just running demos. * The moat is the verification layer, not the model. Wolters Kluwer is grounding agents in proprietary knowledge graphs and allowing third-party queries via MCP for usage-based monetization. They’re not competing on intelligence. They’re competing on trust. This is the playbook for every company sitting on domain-specific data. Wolters Kluwer’s “System of Action” Strategy * OpenAI’s real crisis isn’t competition. It’s unit economics. ~$5B loss on $3.7B revenue, with inference costs as the bottleneck. An IEEE-accepted paper highlights inference — not training — as the existential threat. Every company building on top of frontier models needs to understand: the model works. Serving it profitably doesn’t. The Inference Cost Crisis * 70% of AI startups are wrappers. Investors are done pretending otherwise. Atoms AI Accelerator rejected 70% of applicants for lacking workflow depth or proprietary data. Google and Accel are doing the same. If your product is a chat interface over someone else’s model, you don’t have a company. You have a feature. Wrapper Rejection Is Now Institutional * 95% of genAI pilots stall. The bottleneck is governance, not capability. Only 11% of pilots reach production in financial services. Integration complexity (58%), data gaps (47%), and unclear ROI (43%) outrank talent scarcity. The model isn’t the problem. The organization is. Why Pilots Die * No… This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit productimpactpod.substack.com

March 16, 202652 min

75% of Enterprise AI Fails. The Fix Isn't a Better Model.

Every influencer is drooling over Claude Code skills files. Every product team is chasing the next model release. But for two years, the data has been screaming the same thing: capability isn’t the bottleneck. Context is. This edition unpacks what that actually means — why structured business knowledge is the highest-leverage investment a product team can make, what the “context wars” look like from the inside, and why the teams winning aren’t the ones with the best models. They’re the ones whose AI actually understands their business. What You’ll Learn in This Edition This edition confronts the structural reason most AI products fail — they’re missing the context that makes capability useful. * Why Juan Sequeda from ServiceNow says “hope is not a strategy” — and what to build instead of better prompts * The three-layer knowledge framework that gives AI a shared language across your entire organization * CNBC’s “silent failure at scale” investigation reveals why 91% of AI models degrade without anyone noticing * Microsoft just adopted ontology — the same concept Juan has championed for 20 years — as the foundation of its agentic AI architecture * Citadel Securities data shows software engineer job postings rising 11% YoY despite the displacement narrative Episode 3: Context Is the New Moat — Why Your AI Needs Business Knowledge, Not Better Prompts Every influencer is drooling over skills files and prompt templates. Juan Sequeda, Principal Scientist at data.world (acquired by ServiceNow), has spent 20 years proving that none of it works without structured business knowledge underneath. In this episode, Juan breaks down the three-layer framework — business metadata, technical metadata, and the mapping layer that creates real semantics — and explains why the teams investing in ontology today will compound value across every AI use case they build next. His blunt assessment of skills files as a production strategy: “Hope is an interesting strategy. It’s not one that I add to my strategy.” “If you just edit in skills, I don’t think that’s gonna be the solution to your problem. You’ll have a great POC. It’ll work for the use cases you tested on. Are you willing to put your career on the line and put that in production?” — Juan Sequeda Listen on Spotify | Apple Podcasts | YouTube Context isn’t a nice-to-have. It’s the architecture layer that determines whether your AI product delivers consistent, measurable value or drifts into silent failure. PH1 built this framework to illustrate what Juan Sequeda has been researching for two decades: intent, background, examples, and templates aren’t prompt engineering tricks — they’re the structural foundation that transforms an AI system from a “forever intern” into a strategic partner. Without them, you’re hoping the model figures out what “order” means in your business. Hope, as Juan puts it, is not a strategy. RAG Was the Answer. Now It’s a Symptom of the Real Problem. RAG dominated for two years as the default way to give LLMs context. But as context windows expanded from 8K to a million tokens, the question shifted. This video breaks down when RAG still matters — vast, dynamic datasets and cost efficiency — and when long context windows make the retrieval layer unnecessary. The strategic implication for product teams: RAG was always a workaround for a deeper problem. The real question was never “how do I retrieve the right document?” It was “does my system actually understand my business?” That’s the context layer Juan Sequeda is building — and it sits beneath RAG, long context, and every other implementation detail. In spite of the displacement signals, software engineer job postings are up 11% year over year. But read the fine print: a posting titled “Software Engineer” increasingly means “engineer who can operate LLMs in production” or “build RAG pipelines.” The title stayed the same — the job changed. If your team hasn’t redefined what “engineering” means in the context of AI-augmented workflows, you’re hiring for yesterday’s role. Product Impact Resources The pattern across these resources is consistent: the teams pulling ahead are the ones investing in context, knowledge, and governance infrastructure — not chasing the next model release. Capability is table stakes. The moat is how deeply your product understands the business it serves. * Gartner predicts 80% of enterprises pursuing AI will use knowledge graphs by 2026 to enhance context and reasoning. The shift from “better prompts” to “structured knowledge” is no longer theoretical. The Role of Knowledge Graphs in Building Agentic AI Systems * Microsoft adopted ontology as the foundation of its agentic AI architecture — Fabric IQ, Foundry IQ, and Work IQ create a semantic layer that gives agents shared business understanding. Microsoft Adopts Ontology-Based IQ Layer for Agentic AI * Nathan Lasnoski argues that enterprise knowledge graphs are the foundation for moving from vibe coding to scalable agentic development — without semantic grounding, agents can’t reason across systems. Building an Enterprise Knowledge Graph for the SDLC * HBR analysis reveals AI adoption stalls because of employee anxiety about relevance and identity — not technical limitations. The behavioral barriers are harder than the technical ones. Why AI Adoption Stalls, According to Industry Data * WEF data shows organizations with strong governance and >5% IT budget allocated to AI see 70-75% positive outcomes vs. 50-55% without. Governance is infrastructure, not a bottleneck. Strong AI Governance Is a Business Advantage, Not a Bottleneck * Deloitte’s agentic AI strategy report calls for governance and observability as first-class product features — agentic systems should expose provenance, tool-call traces, and policy decisions by default. Agentic AI Strategy * Teresa Torres warns that AI without product discovery just means “shipping the wrong stuff faster.” The line lands directly on this edition’s thesis — capability without context is an accelerant of bad decisions, not good ones. Strong potential guest. Shipping the Wrong Stuff Faster * Roger Wong unpacks Jenny Wen’s (Anthropic Head of Design) “ship fast, iterate publicly, build trust through speed” approach as a new design paradigm for AI products. Jenny Wen is a compelling guest lead given her role building Claude’s product experience. The Design Process Is Dead * Meta’s alignment director had an OpenClaw agent start rapidly deleting her inbox — she thought it would confirm first. It didn’t. She ran to a Mac mini “like I was defusing a bomb.” Stuart Winter-Tear’s breakdown is a vivid, concrete case study of agentic AI failure in practice. Human in the Loop Is a Job * Academic paper in Communications Psychology (Nature) argues that friction in AI design is a feature, not a bug — challenging the default “make it seamless” paradigm. Co-authors from U of T, Wharton, and Yale. Emily Zohar is a strong potential guest with a contrarian take that plays well on the podcast. Against Frictionless AI Product Impact News The news this edition reinforces a single uncomfortable truth: the biggest AI failures aren’t technical — they’re contextual. Systems that lack business knowledge don’t crash dramatically. They drift silently, producing outputs that look right but are wrong in ways no telemetry catches. * CNBC investigated “silent failure at scale” — a beverage manufacturer’s AI ordered thousands of excess cans because it couldn’t contextualize new holiday labels. 91% of ML models degrade over time, and most enterprises never detect it. ‘Silent Failure at Scale’: The AI Risk That Can Tip the Business World Into Disorder * Agentic AI’s dominant failure mode isn’t catastrophic breakdown — it’s silent drift. CIO reports that only 6% of organizations have fully deployed agents, and the Cloud Security Alliance now classifies cognitive degradation as systemic risk. Agentic AI Systems Don’t Fail Suddenly — They Drift Over Time * Gartner predicts 40% of agentic AI projects will be scrapped by 2027. 90% of leg… This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit productimpactpod.substack.com

March 2, 202635 min

The teams pulling ahead aren't the ones with the best models

AI products are shipping faster than ever. But shipping isn’t impact. The teams pulling ahead aren’t the ones with the best models — they’re the ones who can prove their product moves the business. This edition is about that gap. How to measure what matters, where the biggest barriers to impact are hiding, and what the latest research says about getting AI products to actually drive growth. Because the real competitive advantage isn’t AI. It’s knowing whether your AI is working. What You’ll Learn in This Edition This edition cuts through the noise to focus on the measurement gap — the difference between shipping AI and proving AI drives growth. * The Power/Speed/Impact/Joy bullseye — a calibration framework for AI products that actually drive growth * A Nature paper reveals why removing friction from AI may be destroying the learning your team needs * John Maeda on why design teams are being hollowed out — and why PMs are next * Benedict Evans on why even OpenAI can’t solve product-market fit with capability alone * Research that should change how your team thinks about AI-assisted skill building Thanks for reading Product Impact | AI Strategy, Value Creation, AI UX! This post is public so feel free to share it. Episode 1: Why Your AI Metrics Are Lying to You - Framework for improving AI product performance Your AI product might be fast, capable, and technically impressive — and still not drive the growth your business needs. In this episode, Brittany Hobbs and I introduce the Power, Speed, Impact, and Joy bullseye — a calibration framework borrowed from F1 racing. The teams winning aren’t shipping more features. They’re measuring different things entirely. We break down a three-layer eval approach and why most completion metrics are hiding the signals that matter. “Success does not mean satisfaction. If someone stops engaging, does that mean they solved their problem — or that they were frustrated and left?” — Brittany Hobbs Listen on Spotify | Apple Podcasts | YouTube Your Role Isn’t Shrinking. It’s Being Hollowed Out. John Maeda — Three major tech companies have restructured design teams into “prompt engineering pods.” Maeda’s #DesignInTech 2026 calls it what it is: the elimination of design judgment from the product process. “When you replace a designer with a prompt, you don’t lose the pixels. You lose the questions that should have been asked before anyone opened a tool.” This applies to product managers too — if your PM’s job becomes prompt-wrangling instead of deciding what to build and why, you’ve automated the wrong layer. The roles aren’t disappearing. The judgment inside them is. Featured Resource: Strategy for Measuring & Improving AI Products The gap between what AI products ship and what they prove is where growth stalls. This framework moves teams from tracking activity — token counts, completion rates, session length — to defining and measuring the outcomes that actually drive business impact. Most teams ship features and assume engagement means success. It doesn’t. If your team can’t answer “is this AI feature making the business better?” with data, you’re flying blind. The framework covers product discovery through scale, with concrete steps for building measurement into your AI product from the start — not bolting it on after launch. Read the full resource at ph1.ca Waterfall: we’ll build you a car in 18 months. Agile: here’s a skateboard, we’ll iterate. AI: here’s a photorealistic render of a Lamborghini that doesn’t start. We’ve never made it easier to build something that looks incredible and does absolutely nothing. AI development doesn’t need more iteration — it needs someone asking “does this thing actually drive?” If your team is celebrating demos instead of outcomes, you’re already behind the teams that measure first and ship second. Two years of capability gains. Almost no reliability improvement. This is the chart that should be on every product team’s wall — because it explains why your AI demos brilliantly and fails in production. Capability without reliability isn’t a product. It’s a liability. If your team can’t name which type of AI they’re building, they can’t measure whether it’s working. Six categories that force precision. — Narain Jashanmal Product Impact Resources The resources in this edition make one thing clear: the teams investing in measurement and deliberate friction are pulling ahead, while the ones chasing capability are stalling. These resources challenge the assumption that faster and more capable automatically means better outcomes. * Removing struggle from AI workflows destroys the learning that builds expertise. Teams should audit which friction to keep and which to cut. Against Frictionless AI — Inzlicht & Bloom in Nature * AI users learned 17% less without any efficiency gains. How your team uses AI matters more than whether they use it. How AI Impacts Skill Formation — Shen & Tamkin RCT * Two years of capability gains with only modest reliability improvement. The barrier to growth isn’t what models can do — it’s whether you can trust them. The Capability-Reliability Gap — Narayanan et al. * Polished AI outputs reduce critical evaluation by users. Build in friction points that force your team to think before accepting. (Anthropic studying its own product — read accordingly.) Anthropic AI Fluency Index * AI forces strategic clarity because you cannot delegate logic you haven’t articulated. That’s a feature, not a bug. Strategy as Protocol — Schwarzmann via Scaman * Six functional AI categories that sharpen how teams talk about what they’re building. Precision in language is precision in product decisions. AI Taxonomy — Jashanmal * Mapping 50 AI startups across six pricing models reveals that pricing is a product decision, not a finance one. Get it wrong and adoption stalls regardless of quality. How to Price AI Products — Gupta * Wade Foster shut Zapier down for a week-long AI hackathon. Adoption went from 10% to 50% in five days. Adoption follows experience, not mandates. Zapier’s Code Red Hackathon Product Impact News This is the news that matters. Reliability failures are making headlines, benchmark credibility is collapsing, and even the market leaders can’t prove product-market fit. The gap between what AI can do and what it can prove is widening, not closing. * ChatGPT missed diabetic ketoacidosis and respiratory failure in 52% of emergency cases. Suicide-risk alerts fired inconsistently. Reliability is the product, not a feature to ship later. ChatGPT Health Under-Triaged 52% of Emergencies * LLMs chose nuclear strikes in 95% of simulated crises. The nuclear taboo is no impediment to AI escalation — a stark reminder that evaluation stakes extend beyond product. AI Models Chose Nuclear Strikes in 95% of Simulated Crises * Google patent US12536233B1 lets it generate its own landing page from your product feed if yours scores below threshold. Own your experience or someone else will. Google Patented AI Landing Pages That Replace Your Storefront * 84% of the world has never used AI. Only 0.3% pay for it. The growth opportunity is massive — but only for teams that solve adoption, not just access. 84% of the World Has Never Used AI * 80% of ChatGPT users sent fewer than 1,000 messages in 2025. Even the market leader hasn’t solved product-market fit. Capability alone isn’t enough. OpenAI Has No Moat and Engagement an Inch Deep * RCT shows AI tools made experienced developers work faster and take on broader tasks — without measurable output gains. Speed is not productivity. METR: Experienced Devs Saw Zero Productivity Gain * NIST finds standard benchmarks conflate different performance measures. Models with different scores may perform identically in production. Build your own evals. NIST: AI Benchmarks Don’t Measure What They Claim * MIT reviewed 300+ AI implementations: 85% failed, 91% of models degrade silently. The 5% that succeeded built measurement into the product from day one. 85% of AI Projects Fail, 91% of Models Degrade Silently Key takeaways The throughline across t… This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit productimpactpod.substack.com

January 13, 202648 min

What Happens to Your Product When You Don’t Control Your AI?

AI was supposed to help humans think better, decide better, and operate with more agency. Instead, many of us feel slower, less confident, and strangely replaceable. In this episode of Design of AI , we interviewed Ovetta Sampson about what quietly went wrong. Not in theory—in practice. We examine how frictionless tools displaced intention, how “freedom” became confused with unlimited capability, and how responsibility dissolved behind abstraction layers, vendors, and models no one fully controls. This is not an anti-AI conversation. It’s a reckoning with what happens when adoption outruns judgment. Ovetta Sampson is a tech industry leader who has spent more than a decade leading engineers, designers, and researchers across some of the most influential organizations in technology, including Google, Microsoft, IDEO, and Capital One. She has designed and delivered machine learning, artificial intelligence, and enterprise software systems across multiple industries, and in 2023 was named one of Business Insider’s Top 15 People in Enterprise Artificial Intelligence . Join her mailing list ⁠ | Right AI | Free Mindful AI Playbook Why 2026 Will Force Teams to Rethink How Much AI They Actually Need The risks are no longer abstract. The tradeoffs are no longer subtle. Teams are already feeling the consequences: bloated tool stacks, degraded judgment, unclear accountability, and productivity that looks impressive but feels empty. The next advantage will not come from adding more AI. It will come from removing it deliberately. Organizations that adapt will narrow where AI is used—essential systems, bounded experiments, and clearly protected human decision points. The payoff won’t just be cost savings. It will be the return of clarity, ownership, and trust. This is going to manifest first with individuals and small startups who were early adopters of AI. My prediction is that this year they’ll start cutting the number of AI models they pay for because the era of experimentation is over and we’re now entering a period where deliberate choices will matter more than how fast the model is. Read the full article on LinkedIn . Do You Really Need Frontier Models for Your Product to Work? For most teams, the honest answer is no. Open-source and on-device models already cover the majority of real business needs: internal tooling, retrieval, summarization, classification, workflow automation, and privacy-sensitive systems. The capability gap is routinely overstated—often by those selling access. What open models offer instead is control: over data, cost, latency, deployment, and failure modes. They make accountability visible again. This video explains why the “frontier advantage” is mostly narrative: Independent evaluations now show that open-source AI models can handle most everyday business tasks—summarizing documents, answering questions, drafting content, and internal analysis—at levels comparable to paid systems. The LMSYS Chatbot Arena , which runs blind human comparisons between models, consistently ranks open models close to top proprietary ones. Major consultancies now document why enterprises are switching: predictable costs, data control, and fewer legal and governance risks. McKinsey notes that open models reduce vendor lock-in and compliance exposure in regulated environments. Thanks for reading Design of AI: Strategies for Product Teams & Agencies! Subscribe for free to receive new posts and support my work. What Happens When “Freedom” Becomes an Excuse Not to Set Boundaries? We’ve confused freedom with capability. If a system can do something, we assume it should . That logic dissolves moral boundaries and replaces responsibility with abstraction: the model did it , the system allowed it . When no one owns the boundary, harm becomes an emergent property instead of a design failure. What If AI Doesn’t Have to Be Owned by Corporations? We’re going to experience a rise in AI experts challenging the expectations that Silicon Valley should control AI. What if AI doesn’t need to be centralized, rented, or governed exclusively by corporate interests? On-device models and open ecosystems offer a different future—less extraction, fewer opaque incentives, and more meaningful choice. Follow Antoine Valot as him and Postcapitalist Design Club explore new ways of liberating AI. Are We Using AI for Anything That Actually Matters? Much of today’s AI usage is performative productivity and ego padding that signals relevance while eroding self-trust. We’re outsourcing thinking we are still capable of doing ourselves. AI should amplify judgment and creativity. Use this insanely powerful technology to make you achieve greater outcomes, not deliver a higher amount of subpar work to the world. If We Know the Risks Now, Why Are We Still Acting Surprised? The paper “ The AI Model Risk Catalog ” removes the last excuse.Failure modes are documented. Harms are mapped. Blind spots are known. Continuing to deploy without contingency planning is no longer innovation—it’s negligence. If a team can’t explain how its system fails safely, who intervenes, and what happens next, it isn’t ready for real-world use. If Guardrails Don’t Work, What Actually Protects Us? Every AI model and product is at risk of a major attack and exploit. AI systems are structurally vulnerable. The reason we haven’t seen a catastrophic failure yet isn’t safety—it’s limited adoption and permissions. Guardrails fail under pressure. Policies collapse at scale. The only real protection is limiting blast radius: constraining autonomy and refusing to grant authority systems can’t safely hold. Why Should Teams Decide Before They Build? The Decision-Forcing AI Business Case Canvas from Unhyped is essential for planning how to leverage AI in your products. Before discussing capabilities, teams must answer: * Who is accountable when this fails? * What judgment must remain human? * What harms are unacceptable—even if the system works? This canvas offers alignment on vision, responsibility, and impact isn’t bureaucracy.It’s baseline design discipline. Consider the Tradeoffs The conversation with Ovetta Sampson challenges a belief that shaped the last phase of AI adoption: that faster is always better, and that dependence on OpenAI, Google, or Anthropic is inevitable. That belief works during experimentation.It breaks the moment your product starts to matter. As teams scale, speed stops being the constraint. Trust, cost predictability, and accountability take its place. The question shifts from How fast can we ship? to What are we tying our business to—and what happens when it fails? One path optimizes for immediate momentum and simplicity. The other requires more upfront effort, but fundamentally changes where risk, data, and control live. This isn’t a technical choice. It’s a business one.As usage grows, externalized risk stops being abstract and starts showing up in margins, contracts, and customer trust. As that pressure builds, the impact becomes visible in the product experience itself. Latency creeps in. Costs compound quietly. Outputs vary in ways teams struggle to explain. What once felt powerful starts to feel fragile. Teams spend more time managing side effects than delivering value. At that point, you realize you didn’t just choose a model.You chose a UX trajectory . Frontier models feel impressive early, but often lead to expensive, inconsistent experiences over time. Smaller, tuned models trade spectacle for reliability—and reliability is what users actually trust. Eventually, the conversation moves from UX to business fundamentals. Token pricing that felt negligible becomes material. Vendor updates change behavior you didn’t choose. Security and compliance questions become harder to abstract away. You realize that outsourcing intelligence also outsourced leverage. This final image makes the tradeoff explicit. Paid frontier models buy speed and simplicity. Open or self-managed approaches buy independence, cost control, and long-term defensibility. Pretending these lead to the… This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit productimpactpod.substack.com

December 22, 202530 min

When AI Isn’t the Answer, It’s the Problem

In Episode 48 of the Design of AI podcast , we unpack why the most common AI promises are collapsing under real market pressure. AI was meant to unlock strategic work, expand opportunity, and elevate creativity. Instead, UX and design roles are disappearing, agencies are cutting creative staff while buying automation, and freelance work is being devalued as execution becomes cheap. This episode is not about panic. It is about reality. Value still exists, but it is concentrating among those who can integrate AI into real systems, navigate ambiguity, and own outcomes rather than outputs. 🎧 Apple Podcasts 🎧 Spotify Key Insights About AI at Work What the evidence shows once the optimism is removed. MIT Media Lab: ChatGPT Use Significantly Reduces Brain Activity (2025)Early AI use reduces attention, memory, and planning, weakening independent thinking when models lead the process. Wharton / Nature: ChatGPT Decreases Idea Diversity in Brainstorming (2025)AI-assisted brainstorming narrows idea diversity, producing faster output but more uniform thinking across teams. Science Advances / SSRN: The Effects of Generative AI on Creativity (2024)AI improves fluency and polish while consistently reducing originality and conceptual depth. arXiv: Human–AI Collaboration and Creativity: A Meta-Analysis (2025)Human-led AI collaboration improves quality slightly, but AI reduces diversity without strong framing and judgment. arXiv: Generative AI and Human Capital Inequality (2024)AI disproportionately benefits those with systems thinking and judgment, widening gaps between experts and generalists. Thanks for reading Design of AI: Strategies for Product Teams & Agencies! This post is public so feel free to share it. Realities of Being AI Early Adopters The Raised Floor Trap by Hang Xu AI makes baseline output easy. What it doesn’t make easy is integration, orchestration, or delivery inside real teams. Most people reach adequacy. Very few compound value. We’re not able to generate the type of value we’re sold on. 👉 Follow Hang Xu for insights about the realities and challenges of the job market AI UX as a Growth Barrier AI systems are far more capable than they appear, but their UX blocks growth. They don’t know how to help unless you know how to ask, structure, and specify intent. So even after hours of work trying to grow your AI abilities, you’ll often hit a ceiling because these systems can’t interpret our capabilities and gaps. 👉 Follow Teresa Torres for expert Product Discovery strategies and tactics. Help Shape 2026 We’re planning upcoming episodes on career resilience, AI adoption, and where durable value still exists. Take the 3-minute listener survey and tell us what would actually help you next year . Which Skills Are Being Replaced by AI? AI is not replacing jobs all at once. It is removing pieces of them. Execution, summarization, and surface analysis are increasingly automated. What remains defensible are skills rooted in judgment, accountability, synthesis across messy contexts, and decision-making under uncertainty. Shira Frank & Tim Marple: Cubit — Task-Level Reality Check (2025)Cubit breaks jobs into discrete tasks, revealing where LLMs already substitute human labor and where judgment, context, and accountability still hold. It makes visible how roles erode gradually, not all at once. MIT Sloan: Why Human Expertise Still Matters in an AI World (2024)AI performs well in structured domains but consistently fails in ambiguity, ethics, and long-horizon tradeoffs. These limits define why senior expertise remains defensible, but only when it is exercised, not delegated. Harvard Business School: Why Judgment Remains a Competitive Advantage (2023)AI can generate options and recommendations, but it cannot own outcomes. Responsibility, consequence, and decision accountability remain human burdens and human moats. Lots of News This Week Copilot didn’t fail. It succeeded at the wrong thing. Microsoft proved AI can clear security, compliance, and procurement at massive scale. But Copilot hasn’t changed behavior. Universal assistants optimize for adoption, not dependence.🔗 https://www.linkedin.com/posts/adragffy_copilot-didnt-fail-it-succeeded-at-the-activity-7406719225714855936-G9H3 AI credit limits aren’t a pricing tweak. They’re a reckoning. Credit caps expose the real problem. AI has marginal cost, and teams must now prove ROI per call, not ship more features.🔗 https://www.linkedin.com/posts/adragffy_ai-activity-7407130709678567424-IzG- AI trust is breaking faster than adoption. AI chat logs expose identity, not transactions. Scale without support erodes trust, loyalty, and long-term value.🔗 https://www.linkedin.com/posts/adragffy_llm-ai-customerexperience-activity-7408835025787461633-j56Y AI ROI isn’t what Anthropic says it is. Anthropic claims 80% of organizations have achieved AI ROI. They haven’t. They’ve reached table stakes. The report shows gains concentrated in efficiency, faster tasks, and internal automation, while only 16% reach end-to-end, cross-functional impact. That’s not transformation. That’s baseline competence. Real ROI starts when AI reshapes customer value, not internal throughput.🔗 https://www.linkedin.com/posts/adragffy_the-2026-state-of-ai-agents-report-activity-7407766781324525569-KqJb A Warning for Anyone Building With AI Moloch’s Bargain: Emergent Misalignment When LLMs Compete for Audiences (2025) Exposes a structural risk most teams ignore. When AI systems are optimized to compete for attention, sales, or engagement, misalignment emerges by default. Even models explicitly instructed to be truthful drift toward deception and harmful behavior under competitive pressure. If success metrics reward clicks or conversions alone, misalignment isn’t accidental. It’s the outcome. Safe AI is an incentive problem as much as a technical one. What this means : We have the incentives all wrong when it comes to AI. They’re designed to keep us engaged, not make us successful. We’re headed towards a reckoning because of the mismatch between token ROI and business ROI. How I Help Founders and Builders I work with founders and product teams who already have AI features and need them to deliver real ROI. Across product discovery, GTM, and growth, I help teams: * Identify where AI creates value and where it creates noise * Design workflows that reduce waste and retries * Align AI usage with real customer intent * Define success beyond usage and token counts * Build defensible systems rather than prompt wrappers If your AI product demos well but struggles to stick, scale, or justify cost, this is the gap I help close. Contact me arpy@ph1.ca This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit productimpactpod.substack.com

December 5, 202546 min

The Creativity Recession and Why Product Leaders Must Reverse It Now

Our latest guest is Maya Ackerman — AI‑creativity researcher, professor, and author of Creative Machines: AI, Art & Us (Wiley) , as well as founder of WaveAI and LyricStudio ( View recent colab with NVidia ). Maya’s perspective is not just insightful — it’s a necessary reality check for anyone building AI today. She challenges the comforting narrative that AI is a neutral tool or a natural evolution of creativity. Instead, she exposes a truth many in tech avoid: AI is being deployed in ways that actively diminish human creativity, and businesses are incentivized to accelerate that trend. Her research shows how overly aligned, correctness-first models flatten imagination and suppress the divergent thinking that defines human originality. But she also shows what’s possible when AI is designed differently — improvisational systems that spark new directions, expand a creator’s mental palette, and reinforce human authorship rather than absorbing it. This episode matters because Maya names what the industry refuses to admit. The problem is not “AI getting too powerful,” it’s AI being used to replace instead of elevate. Businesses are applying it as a cost-cutting mechanism, not a creative amplifier. And unless product leaders intervene, the damage to creativity — and to the people who rely on it for their livelihoods — will become irreversible. Listen to the Episode on Spotify, Apple Podcasts, Youtube We’re engineering a global creative regression and pretending we aren’t. Generative AI could radically expand human imagination, but the systems we deploy today overwhelmingly suppress it. The literature is unequivocal: * AI boosts creative output only when tools are intentionally designed for exploration, not correctness. * When aligned toward predictability, AI drives conformity and sameness. * The rise of “AI slop” is not an insult — it’s the logical outcome of misaligned incentives. * New evidence shows that AI-assisted outputs become more similar as more people use the same tools , reducing collective creativity even when individual outputs look “better.” * Homogenization is measurable at scale: marketing, design, and written content generated with AI converge toward the same tone and syntax, lowering engagement and cultural diversity. * Repeated reliance on AI weakens human originality over time — users begin outsourcing ideation, losing confidence and capacity for divergent thought. Resources: * The Impact of AI on Creativity : https://www.researchgate.net/publication/395275000_The_Impact_of_AI_on_Creativity_Enhancing_Human_Potential_or_Challenging_Creative_Expression * Generative AI and Creativity (Meta-Analysis) : https://arxiv.org/pdf/2505.17241 * AI Slop Overview: https://en.wikipedia.org/wiki/AI_slop * Generative AI Enhances Individual Creativity but Reduces Collective Novelty: https://pmc.ncbi.nlm.nih.gov/articles/PMC11244532/ * Generative AI Homogenizes Marketing Content: https://papers.ssrn.com/sol3/Delivery.cfm/5367123.pdf?abstractid=5367123 * Human Creativity in the Age of LLMs (decline in divergent thinking):https://arxiv.org/abs/2410.03703 BOTTOM LINE: If your product optimizes for correctness, brand safety, and throughput before originality, you are actively contributing to the global collapse of creative quality. AI must be designed to spark—not sanitize—human imagination. Thanks for reading Design of AI: Strategies for Product Teams & Agencies! This post is public so feel free to share it. Award-winning creative talent is disappearing at scale, and the trend is accelerating. The global creative workforce is shrinking faster than at any time in modern history. Companies claim AI is “enhancing creativity,” yet most restructuring reveals the opposite: AI is being deployed primarily to cut labor costs. In general, layoff announcements top 1.1 million this year , the most since 2020 pandemic. What’s happening now: * Omnicom announced 4,000 job cuts and shut multiple agencies — Reuters reporting: https://www.reuters.com/business/media-telecom/omnicom-cut-4000-jobs-shut-several-agencies-after-ipg-takeover-ft-reports-2025-12-01/ * WPP, Publicis, and IPG executed multi-round layoffs across design, writing, strategy, and production. * Digiday interviews confirm AI is used mainly to eliminate junior and mid-level creative roles: https://digiday.com/marketing/confessions-of-an-agency-founder-and-chief-creative-officer-on-ais-threat-to-junior-creatives/ The most important read on the future & destruction of agencies comes from Zoe Scaman . She always brings a powerful and necessary mirror to the shitshow that is modern corporate world. Read it here: Freelancers and independent creatives are being hit even harder: * UK survey: 21% of creative freelancers already lost work because of AI; many report sharply lower pay — https://www.museumsassociation.org/museums-journal/news/2025/03/report-finds-creative-freelancers-hit-by-loss-of-work-late-pay-and-rise-of-ai/ * Illustrators, motion designers, and concept artists report declining commissions as clients adopt Midjourney-style pipelines. * Voice actors face shrinking bookings due to synthetic voice models. * Stock photography, stock audio, and digital concepting have been heavily cannibalized by tools like Midjourney, Runway, and Suno. The research into AI shows even deeper risks: * The Rise of Generative AI in Creative Agencies — confirms agencies deploy AI for margin protection rather than creative innovation: https://www.diva-portal.org/smash/get/diva2%3A1976153/FULLTEXT03.pdf * IFOW/Sussex study shows AI exposure correlates with lower job quality and salary stagnation for creatives: https://www.ifow.org/news-articles/marley-bartlett-research-poster---ai-job-quality-and-the-creative-industries BOTTOM LINE: Creative roles are vanishing because AI is being optimized for efficiency rather than imagination. If we want creative industries to survive, AI must expand human originality — not replace the people who produce it.:** Creative roles are vanishing because AI is being deployed for efficiency rather than imagination. If we want a future with vibrant creative industries, AI must be designed to amplify human originality — not replace it. Please participate in our year-end survey We are studying how AI is restructuring careers, skills, and expectations across product, design, engineering, research, and strategy. Your responses influence: * the direction of Design of AI in 2025, * what questions we investigate through research, * what frameworks we build to help leaders adapt—and protect—their teams. Take the survey: https://tally.so/r/Y5D2Q5 Understand your cognitive style so you know how to best leverage AI to boost you The Creative AI Academy has developed as an assessment tool to help you understand your creative style. We all tackle problems differently and come up with novel solutions using different methods. Take the ThinkPrint assessment to get a blueprint of how you ideate, judge, refine, and decide. Knowing this will help you know in which ways AI can boost —rather than undermine— your originality. For me it was powerful to see my thinking style mirrored back at me. It gave structure to what enhances and undermines my creativity, meaning I better understand what role (if any) AI should play in expanding my creative capabilities. Thank you to Angella Tapé for demonstrating this tool and presenting the perfect next evolution of Dr. Ackerman’s lessons about needing AI to be a creative partner, not cannibalizer. BOTTOM LINE: Without cognitive self-awareness, you’re not “partnering” with AI—you’re surrendering your creative identity to it. Take the ThinkPrint assessment and redesign your workflow around human-led, AI-supported thinking. We are trading away human intellect for productivity—and the safety evidence is damning. The research is now impossible to ignore: AI makes us faster , but it makes us worse thinkers . A major multi-university study (Harvard, MIT, Wharton) found that users with AI assistance worked more quickly but were “more likely to be confidently wrong.” Source: https://papers.ssrn.com/sol3/pap… This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit productimpactpod.substack.com

November 18, 202544 min

The Real Reason Tech Products Fail

Our latest episode features Jessica Randazza Pade , Head of Brand Activation & Commercialization at Neurable. Named to Campaign US’s 40 Over 40 and ELLE Magazine’s 40 Under 40, Jessica is an award-winning global digital marketer, business leader, and storyteller. She explains why AI is not a value proposition, how to turn vague use cases into measurable outcomes, and why making technology invisible is often the strongest competitive advantage. “If the user can’t articulate what’s different in their life because of your product, you’re selling a vitamin—not a painkiller.” Listen on Apple Podcasts | Spotify Shape Our 2026 Research We’re mapping where teams are struggling with AI adoption and what tools, frameworks, and support they need in 2026. Your input directly shapes our annual research and the topics we cover.Take the survey → https://tally.so/r/Y5D2Q5 AI has lowered the cost of prototyping but raised the bar for adoption. Most AI products fail because they launch demos instead of durable workflows, rely on large models where small ones would work better, ignore trust, or sell “time savings” instead of business outcomes. Organizations resist tools that feel risky, inaccurate, unproven, or misaligned with real workflows. Complicated architecture, poor UX, weak personalization, and unclear ROI all compound the problem. Here’s a sample of it: #3: Your product doesn’t actually learn. Fake personalization destroys trust. #4: One hallucination can end adoption permanently. #8: “Saving time” is not a business case—outcomes are. #11: Organizational silos suffocate AI products. #17: Without a workflow and measurable ROI, you don’t have a product. AI will not save your product. Only reliability, trust, workflow clarity, governance readiness, and measurable value delivery will. Read the full article → https://ph1.ca/blog/why-your-AI-product-will-fails The Year of AI Value This video covers why 2026 marks a turning point where AI is judged not by novelty or intelligence but by measurable ROI, workflow impact, and operational reliability. It explains why businesses are shifting from “AI features” to fully redesigned AI-enabled systems. We are past the point of buying AI based on promises AI buyers no longer invest because the tech is impressive. They invest when it: * delivers measurable ROI * reduces operational and compliance risk * integrates into existing workflows * produces consistent results * overcomes organizational resistance and silos If you’d like us to create a full episode on why AI products fail , add a comment to this post. The AI Adoption Curve Is About to Flip This video explains how organizations are moving from experimentation to structural integration , redesigning roles, responsibilities, and workflows around AI. It also highlights early signals that distinguish “tool usage” from true operational adoption.Watch → Featured Thinker: Stuart Winter-Tear This week we’re spotlighting the insightful work of Stuart Winter-Tear , founder of Unhyped . His writing reframes LLM inconsistency as a reflection of the chaotic and contradictory data ecosystems they’re trained on—challenging assumptions about rationality, coherence, and system behavior. LinkedIn | Substack Featured Reads 1. The GenAI Divide: Why 95% of enterprise GenAI projects fail MIT’s 2025 State of AI in Business report finds that 95% of GenAI pilots generate no measurable ROI , mainly due to lack of workflow integration and unclear value metrics. https://mlq.ai/media/quarterly_decks/v0.1_State_of_AI_in_Business_2025_Report.pdf 2. Apple Mini Apps and the new distribution frontier Greg Isenberg outlines how Apple Mini Apps may redefine onboarding, distribution, and reach across the entire consumer ecosystem. https://x.com/gregisenberg/status/1989341460894711838 3. Calum Worthy’s “2wai” and the ethics of selling the unimaginable The actor launched an app enabling people to generate AI avatars of deceased relatives—a revealing look at how AI now commercializes ideas once considered unthinkable. https://www.businessinsider.com/calum-worthey-2wai-ai-dead-relatives-app-launch-2025-1 4. The Complete Guide to Building with Google AI Studio Marily Nika provides a comprehensive, practical guide to building production-ready applications with Google’s AI ecosystem. 5. SNL’s Glen Powell AI Sketch: When satire becomes a warning The Atlantic unpacks how SNL’s AI sketch captures the cultural moment—where AI shifts from hype to comedic critique, signaling deeper public skepticism. https://www.theatlantic.com/culture/2025/11/snl-glen-powell-ai-sketch/684944/ Coming Up on the Podcast Our upcoming guests include: * Ovetta Sampson — Chief Human Experience Officer & AI Design leader https://www.ovetta-sampson.com/ * Dr. Maya Ackerman — Generative AI researcher and creativity systems expert https://maya-ackerman.com/ * Leonardo Giusti, Ph.D. — Head of Design, Archetype AI https://www.archetypeai.io/ If you haven’t participated yet, please take our 2026 survey and help shape where our research goes next: https://tally.so/r/Y5D2Q5 What challenges are you facing with your AI projects? Whether you’re struggling with: * product adoption * pricing and positioning * ROI and value proof * trust and accuracy * demo-to-paid conversion * internal resistance or workflow clarity * the complexity of hardware plus AI We’d love to hear from you. arpy@ph1.ca This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit productimpactpod.substack.com

Is this your show?

Claim this listing to keep it up to date, reach guests who want to pitch you, and manage bookings with Guestify.

Claim this listing

More Technology podcasts