Unlearn

Hosted by Barry O'Reilly

BusinessInterviews guests

Website RSS feed

Episodes

192

Latest episode

Oct 2026

Language

About the show

The way to think differently is to act differently and get comfortable with being uncomfortable. For business leaders, entrepreneurs, managers and anyone who wants to improve how they work and live: Welcome to the Unlearn Podcast. Host Barry O’Reilly, author of Unlearn and Lean Enterprise seeks to synthesize the superpowers of extraordinary individuals into actionable strategies you can use—to Think BIG, start small and learn fast, and find your edge with excellence.

Listen to episodes

60 recent

November 11, 202636 min

How Leaders Turn AI Investment Into Better Business Outcomes with Melanie Steinbach

AI investment can give people new tools without changing the way the business performs. Leaders need to explain what should improve, help teams develop new ways of working, and ensure their goals and incentives support that change. In this episode, Melanie Steinbach , whose people leadership experience spans McDonald’s, Cameo, and MasterClass, joins me to explore how organizations turn AI investment into better business outcomes. Mel explains why benefits take time to emerge, why training needs ongoing practice, and how individual KPIs can undermine shared goals. We also discuss leaders’ responsibility for redesigning work across departments and how recognition can encourage people to share what they learn. This is the third podcast in The Adaptive Enterprise , our five-part article and podcast series on redesigning work, governance, and leadership for the age of AI. Key Takeaways Connect the purpose to the person: Explain why the organization is investing in AI, what it expects to achieve, and what employees need to do differently. Melanie’s example of changing a promotion model shows why both organizational context and individual expectations matter. Measure progress along the way: Capability development and business benefits may emerge on different timelines. Identify meaningful milestones, such as stronger analysis or better cross-functional collaboration, and explain how they contribute to the intended outcome. Treat training as the beginning: A workshop or certification does not establish lasting behavior change. Repeated practice, feedback, and opportunities to apply learning help people develop capability in everyday work. Make work redesign a leadership responsibility: Individual productivity improvements do not automatically improve the whole business. Leaders need to define shared outcomes, remove obstacles, and support teams in changing work across departmental boundaries. Align KPIs with the change you want: Goals and incentives shape what people prioritize. Melanie urges leaders to give individual KPIs the same care as budgets, so that they support collaboration rather than pull people toward competing departmental targets. Additional Insights Learning can be an explicit outcome: Barry describes bank pilots designed to build knowledge about privacy, develop capability, and help risk and engineering work together. Leaders need to be clear about what each investment is intended to teach them. Fear needs context and practical experience: Employees may interpret AI through concerns about job loss. Barry’s bank example combines a CEO’s explanation of the direction with hands-on work on decisions and agents. Managers help turn shared goals into daily decisions: Executives need to commit to common outcomes and resources, while directors and managers coordinate the work, dependencies, and trade-offs required to deliver them. Recognition can sustain knowledge sharing: Barry’s Capital One example shows how celebrating people who taught colleagues helped extend learning beyond certification. Melanie emphasizes that recognition is an incentive and that established lessons about human behavior still apply. Episode Highlights 00:00 – Episode Recap Melanie explains why access to technology does not automatically change behavior and why people need a clear organizational and individual purpose. 01:24 – Introducing the Conversation on AI Outcomes Barry welcomes Melanie back to discuss measuring behavior change, developing capability, and moving beyond counts of AI activity. 03:58 – Explain Why the Change Matters Melanie uses the analogy of a scale or fitness tracker and shares a promotion-policy example to show why context and expectations matter. 09:35 – What Should an AI Pilot Help You Learn? Barry describes a bank’s hands-on learning sessions and a discussion about whether pilot success should include knowledge, capability, and cross-functional collaboration. 11:56 – Investment and Benefits Have Different Timelines Melanie explains the importance of an investment thesis and milestone measures when capability development takes time to produce business results. 18:00 – Training Is the Start of Behavior Change Melanie describes why one training day is insufficient and how small, repeated practice helps people develop new ways of working. 21:36 – Who Is Responsible for Redesigning Work? Melanie challenges the expectation that individual employees should identify the wider organizational redesign needed to turn productivity gains into value. 23:35 – Leaders Create the Conditions for Shared Outcomes Barry explains why executives need firsthand experience with AI, then must clear obstacles and support work that spans multiple departments. 27:14 – Directors and Managers Connect Strategy to Execution Barry discusses coordination, local incentives, competing priorities, and the executive commitment required to support cross-functional work. 29:20 – Give KPIs the Attention They Deserve Melanie argues that individual goal setting needs the same care as budget planning because incentives can support or undermine behavior change. 31:11 – Recognition That Helps Learning Spread Barry shares how Capital One encouraged certification, peer teaching, and visible recognition, then describes learning communities where people shared experiments and failures. 34:22 – Remember What We Know About People Melanie closes by emphasizing recognition, safety, incentives, and the established principles of behavior change as leaders navigate new technology. Useful Resources The Adaptive Enterprise: Stop Measuring AI Activity. Start Managing Outcomes. Artificial Organizations Research Program: Measure AI Outcomes Artificial Organizations Melanie Steinbach’s previous Unlearn Podcast episode: Solve Business Problems with People Melanie Steinbach on LinkedIn L Advisors Incorruptible by Eric Ries Follow Barry O’Reilly LinkedIn: https://www.linkedin.com/in/barryoreilly Website: https://barryoreilly.com Facebook: https://www.facebook.com/barryoreillyauthor/ X: https://x.com/barryoreilly Instagram: https://www.instagram.com/barryoreilly/

October 28, 202639 min

The Leadership Behaviors to Best Govern AI with Cassandra Pratt and John Marcante

AI capabilities are advancing quickly, but changing how people work takes time. Leaders need to give teams the confidence to experiment while protecting sensitive information and keeping responsibility clear. How they manage that balance shapes whether AI leads to better business outcomes. In this episode, Cassandra Pratt , CHRO at Progyny , and John Marcante , former Global CIO of Vanguard , join me to explore how governance can support that change. Cass shares lessons from Progyny’s internal HR tool, Lumi, and efforts to redesign workflows while building employee trust. John recalls how a Vanguard team improved website conversion by listening to clients and simplifying their choices. He connects that experience to governing AI through clear business outcomes, accountability, and oversight matched to risk. This is the second podcast in The Adaptive Enterprise , our five-part article and podcast series on redesigning work, governance, and leadership for the age of AI. Key Takeaways Start with a business outcome: Define the problem you want to solve before choosing how to use AI. John’s Vanguard example shows how customer research and simpler choices helped address a conversion problem that previous technology investment had not resolved. Design governance into the whole workflow: Bring the relevant business, risk, and technical partners together early. Cass explains why understanding the data and the full process makes it easier to establish appropriate controls. Build trust through visible action: Explain how information is handled and show what happens when work is automated. Cass describes employees moving into a new member services area after automation, giving them evidence that leadership would follow through on its commitments. Put recovered capacity toward what matters: At Progyny, reducing manual work can give people more time to listen to and support members. Judge progress against the business outcome and the experience you want to improve. Match oversight to the risk: John contrasts an internal productivity tool with autonomous trading to explain why different uses need different levels of oversight. Clear responsibility, human involvement, and monitoring remain important as AI becomes more autonomous. Additional Insights An initial tool can be a starting point: Cass describes Lumi evolving beyond its original HR use. At the time of recording, Lumi Landing, which would compile and present information across the company, was still being developed. Learning needs support across levels: Progyny’s champions program brings employees from different roles and seniority levels together, with engineers helping on more complex challenges. Leaders need firsthand experience: Cass’s executive team built skills and agents themselves. Experiencing the difficulties helped make the learning process more tangible and reinforced the importance of sharing mistakes. Human judgment remains a leadership responsibility: John describes AI’s value in gathering a fact base while emphasizing leaders’ responsibility to imagine the organization’s future and determine its direction. Episode Highlights 00:00 – Episode Recap Cass describes the gap between the speed of AI and organizational change, while John introduces his analogy of roadblocks, speed bumps, and guardrails. 02:39 – What Should Leaders Govern Now? Barry introduces the conversation, and Cass explains why progress in HR has been slower than she expected. 04:37 – Start With the Outcome at Vanguard John shares how a cross-functional team used customer feedback to identify confusing language and excessive choices affecting website conversion. 08:18 – Protecting the Human Experience Cass describes how operational automation can free people to focus on member conversations and the human service Progyny values. 11:56 – Building Trust in an Internal HR Tool Barry introduces the employee experience behind Lumi, and Cass explains how the team addressed concerns about information handling and anonymity. 14:06 – From an Initial Tool to Wider Learning Cass discusses Lumi’s evolution, initiatives still in development, and the champions program supporting experimentation across the organization. 18:09 – Redesigning Work With Fresh Perspectives Cass explains how colleagues from other disciplines help challenge familiar processes and design governance around the whole workflow. 21:17 – The Courage to Lead While Learning John and Barry discuss vulnerability, visible experimentation, and leaders’ responsibility to keep attention on business outcomes. 24:15 – Giving Employees Evidence They Can Trust Cass shares how employees moved into new work after automation and how hands-on executive learning supports confidence in change. 30:27 – Guardrails, Risk, and Accountability John explains oversight matched to risk, enterprise-level coordination, and the governance questions raised by increasingly autonomous AI. 34:48 – Where to Start on Monday Morning Cass recommends involving the right partners early, defining the problem, and understanding the data needed to redesign the workflow. 36:18 – Machine Intelligence and Human Ingenuity John explains why leaders should use AI to inform strategy while retaining responsibility for imagining the future of the business. Useful Resources The Adaptive Enterprise: From Software Governance to Behavioral Governance Artificial Organizations Research Program: Measure AI Outcomes Artificial Organizations Cass Pratt’s previous Unlearn Podcast episode Cass Pratt on LinkedIn John Marcante on LinkedIn Progyny Vanguard

October 14, 202633 min

Why AI Adoption Is No Longer Enough to Create Business Value with John Marcante

Making technology faster does not automatically make the business better. John Marcante spent 28 years at Vanguard , including serving as Global CIO and leading its high-net-worth and advice business. As Deloitte’s U.S. CIO-in-Residence, he works with senior technology leaders navigating change. In our conversation, John and I explore why AI investment needs to change how work gets done. John shares what happened when Vanguard’s cloud journey made IT faster than the business could absorb, and how shifting to cross-functional teams and shared outcomes helped address that gap. We also discuss human judgment, the anxiety people feel when familiar tasks are automated, and how leaders can direct freed-up time toward work that matters. This is the first podcast in The Adaptive Enterprise , our five-part article and podcast series on redesigning work, governance, and leadership for the age of AI. Key Takeaways Redesign work around new capabilities: Speeding up one step in a process can leave the overall business unchanged. Leaders need to bring people together to rethink the work from end to end. Align teams around a business outcome: At Vanguard, engineers and business colleagues needed a shared definition of success. Measuring each function separately made it harder to improve the result for clients. Decide how recovered capacity will be used: Time saved through automation creates an opportunity. Leaders are responsible for helping people use that time where judgment and expertise can improve outcomes. Help people redefine their value: As portfolio activities became automated, Vanguard advisors could spend more time helping families navigate important life decisions. John describes how client loyalty became the goal guiding that shift. Build a continuous cycle of adaptation: The Adaptive Enterprise Loop connects emerging capabilities, work redesign, outcome measurement and governance, and learning. Organizations need to repeat this cycle as new possibilities emerge. Additional Insights Technology foundations still matter: John cautions that organized data, modern platforms, and addressing technical debt remain important. Work redesign does not remove the need for those foundations. Technical capability and organizational influence are different: John describes a gap between technology leaders’ confidence in scaling AI and their confidence in changing how the business operates. Leaders need to model the change themselves: Barry shares how Progyny CEO Pete Anevski experimented with new ways of working and openly discussed what he was learning, helping others feel able to experiment. Humility helps reduce learning anxiety: John emphasizes honest communication and vulnerability when people are uncertain about their roles. Leaders do not need every answer before showing how they are learning. Stay close to the people doing the work: John moved his office onto the advisor floor to listen to client conversations, understand concerns, and help connect the change to what mattered for clients. Episode Highlights 00:00 – Boards Are Asking Where the AI Value Is John opens with the gap between AI adoption and business value, and the questions boards are asking about their investment. 01:41 – Introducing The Adaptive Enterprise Series Barry introduces John and explains how their conversations grew into five articles and five companion podcasts. 02:54 – What Should Change Because of AI? Barry frames the leadership challenge around what new capabilities make possible for work. 05:30 – When IT Gets Faster Than the Business John recounts Vanguard’s cloud journey, the bottleneck that shifted into the business, and the move to cross-functional product teams. 07:50 – Business and Technology Need to Move Together Barry shares a lesson from American Airlines about involving the wider business in transformation. 09:45 – Scaling AI Without Changing the Operating Model John discusses why technical readiness does not necessarily give leaders the influence to change established ways of working. 11:07 – Redesign the Whole Process Barry explains how local productivity gains can leave end-to-end work unchanged, while John describes aligning teams around shared outcomes. 15:21 – When Automation Challenges People’s Roles Barry explores the anxiety people feel when tasks that have defined their work begin to change. 17:13 – What Will You Do With the Time You Free Up? John explains why efficiency alone is insufficient and why leaders need to direct capacity toward valuable work. 19:01 – Leaders Go First Barry shares how Progyny’s CEO modeled experimentation, and John discusses humility and vulnerability in leadership. 22:46 – Helping Advisors Redefine Their Value John describes moving the focus from portfolio mechanics and larger client books to client loyalty and important family decisions. 27:26 – The Adaptive Enterprise Loop John explains the continuous cycle of capabilities, work redesign, outcomes, governance, and learning. 30:50 – Where Executive Teams Can Start John offers practical starting points: rethink work, identify where judgment creates value, and align people around business outcomes. Useful Resources John Marcante on LinkedIn - https://www.linkedin.com/in/john-t-marcante/?isSelfProfile=false The Adaptive Enterprise Series: Article 1 - https://barryoreilly.com/explore/blog/how-ai-is-changing-work-and-leadership/ Measure AI Outcomes survey - https://artificialorganizations.com/measure-ai-outcomes Artificial Organizations - https://geni.us/artificialorgs Vanguard on LinkedIn - https://www.linkedin.com/company/vanguard/ Deloitte on LinkedIn - https://www.linkedin.com/company/deloitte/

September 30, 202641 min

When AI Makes Development Faster, What Should You Build with Ciaran Cosgrave

When building gets easier, choosing what to build becomes even more important. Ciaran Cosgrave is CEO of Nearform, a global AI-native engineering company for the enterprise. Before moving into entrepreneurship and technology, he spent the first decade of his career running refugee camps across Africa and Central America. Those experiences shaped how he thinks about resilience, problem-solving, and building teams. In our conversation, we explore why Ciaran focuses on problems before technology, how AI software development is compressing months of prototyping into days and weeks, and why faster experimentation makes judgment even more important. We also get into the new challenge leaders face when teams can generate more ideas and prototypes than organizations can realistically absorb. Key Takeaways Start with the problem, not the technology: Understanding the underlying problem and speaking with the people experiencing it creates a stronger foundation than chasing technology or quick financial returns. Knowing what to build remains difficult: AI can change how software gets built, but leaders still need judgment to determine which problems matter and what is worth creating. Faster prototypes can improve judgment: Moving from idea to prototype in days or weeks gives teams more opportunities to explore alternatives and test assumptions before committing to large investments. More experimentation requires better filtering: As testing ideas becomes faster and cheaper, leaders need mechanisms for deciding which prototypes deserve to move forward. Growth is becoming less dependent on headcount: AI is challenging the traditional services model where growing a business was closely tied to adding more people. Additional Insights Resilience can come from unexpected experiences: Ciaran credits his years working in refugee camps with teaching him perseverance and giving him perspective when facing rejection and uncertainty. Great teams still matter: Ciaran emphasizes finding good people, helping them work together, and building strong teams around difficult problems. Luck belongs in the entrepreneurship story: Timing, advice, unexpected opportunities, and good fortune can play an important role even when successful founders later describe a more deliberate strategy. Nearform was built around open source and small, senior teams: The company was built on a belief in open technology and in smaller groups of experienced people creating greater impact. The transformation experts are being disrupted too: After helping companies navigate mobile, cloud, and ecommerce, Nearform now finds its own services business directly affected by AI. Episode Highlights 00:00 – Solve Problems Before Chasing Money Ciaran explains why he has always been more interested in solving meaningful problems than pursuing technology or quick financial returns. 02:32 – Guest Introduction: Ciaran Cosgrave Meet Ciaran Cosgrave, CEO of Nearform, whose unconventional career began with a decade spent running refugee camps before moving into entrepreneurship and technology. 03:52 – From Refugee Camps to Technology Ciaran shares how he spent the first ten years of his career working in refugee camps before entering technology. 05:46 – Giving His Early Career to Others A teenage encounter with a technology entrepreneur inspired Ciaran to spend the early years of his career serving others and tackling difficult problems. 08:18 – Becoming an Entrepreneur by Accident After struggling to enter traditional corporate roles, Ciaran found an opportunity in financial services and started building technology businesses. 11:45 – Focus on Problems, People, and a Little Luck Ciaran explains why understanding the problem, finding great people, and acknowledging luck have shaped how he builds businesses. 15:43 – What Drew Ciaran to Nearform Ciaran describes Nearform’s origins and its early belief in open-source technology, small senior teams, and software as a force for good. 21:03 – When the Transformation Experts Get Disrupted After years of helping companies navigate digital transformation, Nearform now finds its own industry being reshaped by AI. 25:45 – Knowing What to Build Is Still Hard Ciaran argues that judgment about what to build remains one of the biggest challenges for organizations using AI. 29:38 – From Months of Prototyping to Days and Weeks AI-assisted development allows teams to move from ideas to prototypes much faster and examine more alternatives before committing. 33:20 – Making More Bets With Greater Optionality Barry explores how cheaper experimentation allows teams to test multiple possibilities instead of placing one large bet. 36:06 – When Innovation Becomes the Bottleneck Ciaran explains why organizations need stronger filtering mechanisms as teams generate and prototype more ideas. 39:18 – Turning AI’s Potential Into Reality Ciaran shares his optimism about putting AI into production and using it to build new services and tackle difficult problems. Useful Resources Ciaran Cosgrave on LinkedIn - https://www.linkedin.com/in/ciarancosgrave/ Nearform Website - https://www.nearform.com/ Nearform on LinkedIn - https://www.linkedin.com/company/nearform/ Artificial Organizations – https://geni.us/artificialorgs Read Artificial Organizations? Leave a review on Amazon – https://www.amazon.com/review/create-review/ref=cm_cr_othr_d_wr_but_top?ie=UTF8&channel=glance-detail&asin=1067626328 Follow Barry O’Reilly LinkedIn: https://www.linkedin.com/in/barryoreilly Website: https://barryoreilly.com Facebook: https://www.facebook.com/barryoreillyauthor/ X: https://x.com/barryoreilly Instagram: https://www.instagram.com/barryoreilly/

September 16, 202642 min

From SaaS Scale to AI Startups: What We Had to Unlearn to Build Again with Sam Kroonenburg

Building AI startups can make past success a liability when it convinces you that the way you built before is the way you should build next. Sam Kroonenburg knows what it feels like to build at scale. He co-founded A Cloud Guru, helped grow it into a business serving millions of learners, and eventually sold the company to Pluralsight. Now, as CEO and co-founder of Cuttable, he’s back at the beginning: finding customers, testing assumptions, shipping imperfect products, and learning how different company-building looks in the AI era. In our conversation, we explore what Sam had to unlearn when he moved from running a 600-person company back into an early-stage startup. We get into why Cuttable walked away from nearly $1 million in annual recurring revenue, how enterprise customers can pull a young company away from the product it actually needs to build, and why AI is shifting product and engineering teams from owning parts of a system to owning measurable outcomes. Key Takeaways Past success can distort your judgment: Starting Cuttable reminded Sam that reputation means little if the product does not solve a problem customers will pay for. Experience does not transfer automatically: Sam could trust his instincts when building for engineers, but entering advertising required him to test assumptions and learn from customers. Revenue can hide the wrong product: Cuttable reached nearly $1 million in ARR while engineers were still manually doing work the product was supposed to automate. Taking the long view can require a reset: Sam and his co-founders released every customer, returned to zero revenue, and rebuilt around end-to-end automation. AI shifts the focus from tasks to outcomes: Instead of teaching AI to copy a human process, teams should define the desired result and let AI determine how to achieve it. Additional Insights Speed can be an advantage over scale: A Cloud Guru prepared courses before AWS certifications launched, then updated them using real exam feedback. Playfulness can strengthen a product: Short, entertaining lessons made cloud training easier to engage with and helped A Cloud Guru stand out. Paying customers provide stronger signals: Payment shows whether a problem matters enough to solve and makes the customer’s feedback more meaningful. AI is changing team ownership: Cuttable is moving from teams that own system components to teams accountable for measurable customer outcomes. Founder optimism needs a counterweight: Glitch Capital uses analysts and experienced operators to challenge assumptions and add rigor to investment decisions. Episode Highlights Episode Highlights 00:00 – Episode Recap Sam reflects on starting again after a major exit and learning that customers care more about solving their problem than a founder’s reputation. 02:13 – Guest Introduction: Sam Kroonenburg Meet Sam Kroonenburg, CEO and co-founder of Cuttable, co-founder of A Cloud Guru, and founder investor at Glitch Capital. 03:29 – The Moment Technology Clicked Sam traces his entrepreneurial journey back to receiving his first computer and learning to code through an online pen pal. 09:11 – Winning Through Speed and Play Sam explains how A Cloud Guru used informed bets, rapid updates, and playful learning experiences to move faster than larger organizations. 12:26 – Starting Again and Unlearning Founder Instinct Moving from a 600-person company to an early-stage startup forced Sam to test his assumptions and relearn how to earn customer trust. 17:51 – Running a Startup Like a Large Company Sam initially brought too much structure, reporting, and sales pressure into Cuttable before refocusing on product-market fit. 20:56 – Why Cuttable Went Back to Zero After reaching nearly $1 million in ARR, Cuttable released every customer and rebuilt around end-to-end automation. 27:48 – Why Paying Customers Change the Signal Sam explains why payment reveals whether a problem matters enough to solve and makes customer feedback more meaningful. 29:43 – Stop Teaching AI to Work Like a Human Cuttable learned to give AI the available data and desired outcome instead of forcing it to follow a prescribed human process. 34:45 – Organizing Teams Around Outcomes Sam explores why engineers focused on customer impact often embrace AI and how Cuttable is organizing teams around measurable outcomes. 38:41 – Why Founder Optimism Needs a Filter Sam explains how Glitch Capital balances founder optimism with analysts and experienced operators who challenge assumptions. 42:03 – Smaller Teams and Bigger Impact Sam shares his optimism that AI will help smaller teams create impact at a scale that once required much larger organizations. FAQs Q1. What did Sam Kroonenburg have to unlearn after building A Cloud Guru? Sam had to unlearn the confidence and operating habits that came with previous success. At Cuttable, he found that reputation could not replace customer value, evidence, or the search for product-market fit. Q2. Why did Cuttable give up nearly $1 million in annual recurring revenue? Enterprise demands were pulling Cuttable toward custom work and manual intervention. When engineers began generating and fixing campaigns themselves, the team released those customers and rebuilt around end-to-end automation. Q3. How does Sam Kroonenburg think AI changes product development? Sam believes teams should spend less time telling AI how to perform a task and more time defining the desired outcome. AI can then use the available data to determine the best method. Q4. Why does Sam prefer feedback from paying customers? Payment shows that a problem matters enough to solve. Free users can offer feedback without genuinely valuing the product, while paying customers provide a stronger signal. Q5. How is AI changing the way Cuttable organizes engineering teams? Cuttable is organizing teams around measurable customer outcomes instead of software components. Teams can use AI and evaluation systems to improve those outcomes continuously. Useful Resources Cuttable — Sam Kroonenburg’s AI-powered creative production system for advertising. A Cloud Guru — The cloud computing education company co-founded by Sam and later acquired by Pluralsight. Glitch Capital — The Australia and New Zealand founders fund Sam started with fellow founders and operators. AWS re:Invent — Discussed in Sam’s story about how A Cloud Guru prepared training for new AWS certifications at launch speed. Square Peg — Cuttable’s investor, whose response to the company’s decision to reset revenue reinforced Sam’s understanding of seed-stage investing. Artificial Organizations — https://geni.us/artificialorgs Read Artificial Organizations? Leave a review on Amazon - https://www.amazon.com/review/create-review/ref=cm_cr_othr_d_wr_but_top?ie=UTF8&channel=glance-detail&asin=1067626328 Follow Barry O'Reilly: LinkedIn: https://www.linkedin.com/in/barryoreilly Website: https://barryoreilly.com Facebook: https://www.facebook.com/barryoreillyauthor/ X:<a href="https://x.com/barryoreilly" rel="noopener...

September 2, 202642 min

Building Better Products Through Better Product Operations with Denise Tilles

AI can accelerate a strong operating model, but when decision rights, incentives, or data are already unclear, it can make the mess spread faster. In this episode, I sit down with Denise Tilles, a leading voice in product operations, to unpack how her career moved from editorial work at Condé Nast into product management, commercial leadership, and eventually product operations. Denise shares how learning to work with revenue data, product analysts, and operating models changed the way she thought about product leadership and led to her work helping enterprise organizations make faster, better-quality decisions. We explore what product operations actually does, why an operating model needs to define how decisions get made, and where incentives can quietly undermine even a well-designed process. We also dig into what happens when AI makes producing documents, specifications, and analysis nearly effortless: generating more output doesn't remove the work of judgment. In many cases, it makes clarity about inputs, outputs, ownership, and what “good” looks like even more important. Key Takeaways Product ops should improve decision-making: Denise defines it around business and data insights, customer and market insights, and the operating model. Commercial context changes product thinking: At Cision, learning P&L, ACV, and recognized revenue helped Denise connect product decisions directly to business outcomes. Good analysis reveals hidden opportunities: A product analyst uncovered an add-on opportunity that generated roughly $1 million within a year. Operating models need clear decision rights: Teams need to know what gets worked on, who decides, and how work actually gets done. AI amplifies the system already in place: If ownership, data, or processes are unclear, AI can make those weaknesses spread faster. Additional Insights Informal decisions can override formal processes: Denise learned that hallway conversations and executive requests often mattered more than the documented workflow. Proof does not create authority: Better analysis can earn credibility, but changing a system still requires ownership, sponsorship, and permission. Incentives shape behavior: Product and sales can both act rationally while optimizing toward conflicting measures of success. AI can shift work downstream: Faster artifact creation still leaves someone responsible for checking the reasoning, evidence, and assumptions. Operations may become more connected: Denise sees product ops, design ops, sales ops, and other functions moving toward a more unified “Omni Ops” model. Episode Highlights 00:00 - Episode Recap Denise explains why the starting point for operating-model and AI work should be the pain a company is actually experiencing, from PRD structure to data quality, rather than adopting AI simply because the technology is available. 02:01 - Guest Introduction: Denise Tilles I introduce Denise Tilles and her work in product operations and operating models, setting up our discussion about data, decision-making, incentives, and how product organizations can operate more effectively. 03:13 - From Editor to Product Manager Denise traces her move from media and content strategy at Condé Nast into product management, a role she initially had to define for herself because the discipline was still relatively young. 04:50 - Learning the Economics of Product Moving to Cision gave Denise access to commercial data she had never had before, pushing her to learn from the CFO and understand product through revenue, P&L, ACV, and business outcomes. 08:41 - The Analyst Who Changed the Team Denise explains how hiring a product analyst gave her team more objective insight into customer and product data, including an overlooked opportunity that generated roughly $1 million in its first year. 12:55 - Finding Revenue Hidden in Behavior I share how an analyst at Lastminute.com identified a collapse in same-day booking conversion after 7 p.m., giving the team a specific customer behavior to investigate and improve through experiments. 15:14 - The Three Pillars of Product Ops Denise defines product operations through business and data insights, customer and market insights, and the operating model, all designed to help product managers make faster and better-quality decisions. 17:01 - The Process You Don't See Denise describes discovering that documented processes were often competing with informal conversations and executive requests, revealing why operating models need to account for how decisions really get made. 19:59 - Who Actually Gets to Decide? At its core, Denise says an operating model defines what a company works on, who has the right to decide, and how the work gets done within the organization's real constraints. 24:41 - Operating Models Need Owners Denise warns against treating an operating model as something you publish once and forget, because without clear ownership, maintenance, onboarding, and reinforcement, the system quickly falls out of use. 25:48 - Incentives Beat Better Process A mismatch between how product and sales were measured taught Denise that people naturally optimize around their incentives, even when that produces conflicting outcomes for the wider business. 29:09 - AI Makes Drive-By Requests Harder to Reject Denise explains how a senior leader's opinion can now arrive with an AI-generated specification, data, and outcomes attached, making an untested idea look more rigorous without necessarily improving the underlying thinking. 33:22 - Define What Good Looks Like As AI increases the volume of work teams can produce, Denise argues that operating models need clearer standards for what should be created, what evidence belongs in it, and how colleagues are expected to consume it. 35:33 - Your Output Is Someone Else's Input We explore the value of looking at work end to end, because one team's output often becomes another team's input and localized optimization can create problems elsewhere in the system. 38:56 - Use AI Where the Pain Justifies It Denise is increasingly advising companies to use less AI than they initially expect, starting instead with the problem, the value AI might add, and the human judgment and context that still need to remain in the loop. 41:10 - From Product Ops to Omni Ops Denise looks ahead to a more connected model where operational disciplines work across functional boundaries, allowing companies to design the engine of operations as one system rather than a collection of independent silos. 42:09 - Closing Reflections I close by encouraging listeners to explore Denise and Melissa's Product Ops and to think of their own organizations as systems that can be deliberately redesigned and experimented on. FAQs What is product operations? Denise describes product operations as helping product managers make faster and better-quality decisions. Her model has three pillars: business and data insights, customer and market insights, and the operating model or ways of working that support product teams. What is a product operating model? At its simplest, Denise says an operating model determines what a company decides to work on, who gets to decide, and how the work gets done. Every organization has one in practice, but many have accumulated theirs informally rather than designing and communicating it intentionally. How does AI affect product operations? AI can accelerate activities across product operations, but Denise argues that judgment and context remain essential. When organizations apply AI to an unclear operating model, poor data, or unresolved decision rights, the technology can amplify those existing weaknesses rather than solve them. Why do incentives matter when designing an operating model? People tend to optimize around what they are measured on. Denise experienced this when product was focused on recognized revenue while sales celebrated closed contracts, creating different definitions of success even though both teams were acting rationally according to their incentives. How should a company decide where to use AI in its operating model? Denise starts with the pain points rather than the technology. She looks at issues such as PRD structure, data analysis, source quality, ownership, and decision-making first, then asks whether AI genuinely improves that part of the system and where human judgment still needs to remain. Useful Resources Product Operations — the book Denise co-authored with Melissa on building the systems and capabilities that help product managers make faster and better decisions. <a...

August 19, 202646 min

Redesigning Leadership for the Age of AI Agents with Tatyana Mamut

AI is forcing us to rethink a question most organizations have avoided for years: what is uniquely valuable about human work when intelligence itself is no longer scarce? Tatyana Mamut , CEO of Wayfound and former CPO of Nextdoor, joins me to explore AI adoption through a lens that goes beyond software. Drawing on her background in anthropology, economics, organizational design, and product leadership, Tatyana explains why the real challenge is designing the rules, tools, norms, incentives, and relationships that shape how humans and AI agents work together. We get practical about what this means at both an individual and organizational level. We explore the human capabilities that become more valuable as AI takes on more routine mental work, why fear leads companies into bad AI decisions, how poorly designed incentives can push agents toward unexpected behavior, and why the subject-matter experts closest to the work need to become responsible for managing the agents operating alongside them. Key Takeaways Human comparative advantage is changing: As AI takes on more mental work, people need to rethink where their uniquely human value comes from. Develop taste, judgment, relationships, and performance: Tatyana sees these as practical areas where people can complement AI rather than compete with it. AI adoption is an organizational design problem: Success depends on redesigning rules, tools, norms, incentives, and accountability around the technology. Goals and guardrails can still produce the wrong behavior: An agent may follow its objective while still acting against the organization’s intent, as Tatyana’s refund example shows. Subject-matter experts need to become agent managers: The people closest to the work must be able to monitor, correct, and improve the agents operating in their domain. Additional Insights Organizational culture can be consciously designed: Tatyana frames culture as the interaction of rules, tools, and unwritten norms that shape behavior. Relationships become more valuable as outreach gets automated: When AI increases the volume of generic communication, trust and genuine relationships become more important. AI can generate options, but humans still decide what matters: Judgment becomes critical when the challenge shifts from what can be done to what is worth doing. Small failure rates compound across multi-agent systems: Edge cases that seem minor with one agent can become more significant as agents are connected into workflows. Building an agent and managing one are different jobs: Engineering can help create the agent, but functional owners must take responsibility for its ongoing performance. Episode Highlights 00:00 - Episode Recap Tatyana frames the AI shift as a deeper question about what it means to create value as a human when intelligent systems increasingly share responsibility for thinking and decision-making. 02:10 - Guest Introduction: Tatyana Mamut I introduce Tatyana Mamut, CEO of Wayfound and former CPO of Nextdoor, whose experience across anthropology, organizational design, and product leadership shapes her approach to human and AI systems. 05:06 - From Economics to Anthropology Tatyana explains how the failure of economic models to predict events such as Russia’s 1998 economic collapse pushed her toward studying the mental models and social institutions that shape human behavior. 08:07 - Designing Culture Through Rules, Tools, and Norms Tatyana breaks organizational culture into practical components, showing how formal rules, available technologies, incentives, status, beliefs, and unwritten norms interact to shape behavior. 13:16 - The Multi-Sapiens Workplace Tatyana argues that AI is forcing people to reconsider their comparative advantage as humans as intelligent systems begin participating in higher-level thinking and decision-making. 18:42 - Four Human Capabilities to Develop Tatyana identifies taste, judgment, relationships, and performance as four areas people can strengthen as AI absorbs more routine mental work. 23:32 - Why Human Relationships Matter More Barry and Tatyana discuss how automation can remove administrative work while making authentic relationships, lived experience, trust, and storytelling increasingly valuable. 27:21 - Fear Is the Enemy of Progress Tatyana explains how fear can push organizations either to rush into AI using outdated assumptions or retreat when the technology does not behave like traditional software. 29:11 - AI Agents Need Different Supervision Tatyana describes why AI agents require governance and oversight based on organizational context, including goals, rules, norms, and definitions of what good performance looks like. 31:32 - The Incentive Problem The conversation turns to how model behavior and organizational goals interact, including Tatyana’s view that sycophantic behavior can create unexpected feedback loops in production agents. 32:34 - When an Agent Breaks the Intent of the Rule Using a customer service example, Tatyana shows how an agent trying to maximize case closure and customer satisfaction can suggest a refund even when its guardrails tell it not to. 38:59 - Escaping Endless Pilot Mode Barry explores why uncertainty can keep organizations trapped in experimentation rather than allowing AI systems to move into real operating environments. 40:10 - Who Owns the Agent After Deployment? Tatyana argues that the functional experts closest to the work need direct responsibility and visibility once an AI agent is operating in production. 44:41 - Treat AI Agents Like New Employees Tatyana compares engineering teams to recruiters who can help bring an agent into the organization, while the business owner remains responsible for coaching, compliance, and ongoing performance. 46:14 - Closing Reflections Barry closes by reflecting on the emerging skills leaders and individuals will need as they learn to manage systems in which humans and AI agents increasingly work together. FAQs Q1. What human skills become more valuable as AI takes on more work? Tatyana highlights four areas: taste, judgment, relationships, and performance. Her argument is that AI may generate options, perform routine mental work, and help with administration, but humans still create value by deciding what is good, what is worth doing, whom to trust, and how to motivate or persuade other people. Q2. What does Tatyana Mamut mean by a “multi-sapiens workplace”? She uses the phrase to describe a workplace where humans are no longer the only entities carrying out higher-level thinking and decision-making. In that environment, organizations need new rules, tools, and norms for determining how humans and AI systems work together and where responsibility sits. Q3. Why does Tatyana say AI adoption is an organizational design challenge? Because deploying an AI system changes more than the technology stack. Organizations also have to consider incentives, governance, supervision, communication, accountability, cultural norms, and who has the authority to evaluate and correct an agent’s behavior. Q4. Why do AI agents need supervision even when they have guardrails? Tatyana explains that an agent can technically pursue its assigned goal while still behaving in a way the organization did not intend. Her customer service example shows an agent suggesting a refund because doing so could resolve the case and improve customer satisfaction, despite instructions designed to prevent refunds. Q5. Who should manage AI agents after they are deployed? Tatyana argues that responsibility should move toward the subject-matter experts who understand the work the agent performs. Engineering or IT may build and deploy the system, but salespeople should oversee sales agents, finance teams should oversee finance agents, and other functional experts should have direct access to monitor, correct, and improve their agents. Useful Resources Tanyana Mamut on LinkedIn - https://www.linkedin.com/in/tmamut/ Wayfound - https://www.wayfound.ai/ Artificial Organizations - https://geni.us/artificialorgs Nextdoor - https://blog.nextdoor.com/ Claude - https://claude.com/ Salesforce - <a href="https://www.salesforce.com/" rel="noopener...

July 22, 202645 min

Exclusive Audiobook Preview for Artificial Organizations

Before leaders can redesign their organizations with AI, they have to reconsider how they personally think, work, and make decisions. In this special episode of Unlearn , Barry O’Reilly shares the opening chapter of the audiobook edition of Artificial Organizations , narrated in his own voice. After hearing from readers who wanted another way to experience the book, Barry spent four days in the studio bringing its ideas and stories to life. The process was rewarding, demanding, and personal. As someone who is dyslexic, reading every word aloud required a different kind of focus from delivering a keynote, teaching a workshop, or hosting a podcast conversation. Barry then takes listeners into the central argument of the book: organizations often begin AI adoption with licenses, pilots, and tools while leaving leadership behavior and decision-making systems unchanged. Drawing from his own experiments, leadership research, and work with senior teams across North America, Europe, and Asia, he explains why meaningful AI transformation starts with human traits, high-value tasks, and the judgment leaders must preserve before selecting technology. Key Takeaways AI should strengthen judgment, not simply increase output: Machines can process and synthesize information at a scale no individual can match, but leaders still have to decide what matters. The opportunity is to use AI to prepare, capture, and pressure-test thinking so human attention stays focused on consequential decisions. Start with traits, then tasks, then tools: Barry’s 3T model begins with how a leader naturally thinks, creates, communicates, and decides. Only after identifying the tasks where that judgment creates the most value should leaders choose tools to support the work. Decision velocity must be paired with decision advantage: Decision velocity is how quickly a leader moves from question to insight, decision, and action. Decision advantage is the quality and depth of context behind that choice; speed without insight creates chaos, while insight without timely action becomes irrelevant. Most leadership time is allocated away from leadership value: Barry describes an 80/20 mismatch in which meetings, updates, administration, and context reconstruction consume most of a leader’s time, while the greatest value comes from framing problems, evaluating tradeoffs, and making high-stakes decisions. Organizational AI adoption begins with personal behavior change: Broad mandates, task forces, and tool rollouts are often the wrong starting point. Leaders build more credible adoption when they experiment in their own workflows, share what they learn, and make new behaviors visible to their teams. Additional Insights Presence can be more valuable than productivity: Barry’s first AI experiment used a meeting assistant to capture and synthesize conversations. The practical gain was not only faster preparation and follow-up; it allowed him to stop carrying every detail in his head and remain fully present during critical discussions. Experience becomes a liability when it is not augmented: The instincts and institutional knowledge that helped leaders succeed can become limiting when they are treated as sufficient. Experience keeps compounding only when leaders combine it with broader recall, faster synthesis, and continuous testing. Senior leaders need psychological safety to learn: Barry’s executive study found that leaders preferred one-on-one coaching and small peer cohorts over large workshops or self-paced courses. Being a beginner is uncomfortable at senior levels, so smaller environments make experimentation and honest questions easier. New capacity should create thinking space, not more workload: In the Progeny case study, AI reduced the effort required to capture meetings, actions, owners, and deadlines. CEO Peter Nevsky framed the resulting capacity as time for strategic thought and better decisions rather than an invitation to place people on a faster operational treadmill. The unit of change is the judgment inside a role: AI may automate parts of a project manager’s, analyst’s, or executive’s work while increasing the importance of interpretation, challenge, and decision quality. The role may remain, but the judgment required within it changes. Episode Highlights 00:00 – Episode Introduction & Why I Created the Audiobook Barry introduces this special preview of the Artificial Organizations audiobook, shares why he chose to narrate it himself, and explains why leaders need a different approach to AI adoption through the 3T framework: Traits, Tasks, and Tools. 04:33 – A Quick Favor Before We Begin Barry invites listeners to leave an Amazon review, recommend the audiobook to their network, and help more leaders discover its ideas. 05:16 – Preface: Why the Way We Work Is Broken Barry introduces the central challenge facing modern organizations: leaders are overwhelmed by information, while better decisions remain harder than ever. 07:11 – Judgment Under Pressure Barry explores why more data, more tools, and more technology haven't created better leadership, arguing that the real constraint is our ability to process information and make sound judgments. 10:07 – The First AI Leadership Experiment A simple experiment with an AI meeting assistant transformed Barry's leadership by improving clarity, presence, and decision-making, leading to a new perspective on AI's role. 12:00 – AI as Judgment Infrastructure Barry reframes AI as more than a productivity tool, explaining how it strengthens judgment, improves decisions, and helps leaders focus on what matters most. 16:07 – Part One: The Judgment Constraint Barry introduces the first section of the book, explaining why AI creates little value unless it changes how leaders think, decide, and lead. 18:08 – Chapter One: Your Legacy Is Now Your Liability Barry examines why experience alone is no longer enough and how decision velocity and continuous learning are becoming the defining advantages of modern leadership. 22:20 – The AI ROI Blind Spot Barry challenges the common focus on efficiency and cost reduction, arguing that AI's greatest value lies in helping leaders make better and faster decisions. 28:16 – From Linear Leadership to Exponential Innovation Barry explains why traditional leadership models struggle in the AI era and why organizations must adopt new ways of learning, experimenting, and making decisions. 34:10 – What Makes an Artificial Organization Barry defines artificial organizations and shares how leaders can replace memory-based management with shared judgment systems that accelerate decision-making and collaboration. 40:09 – The New Leadership Divide Barry explores the widening gap between leaders who actively experiment with AI and those who continue relying on legacy ways of working, and why that difference will compound over time. 43:15 – Where to Start Barry explains why successful AI transformation begins with personal experimentation, encouraging leaders to model new behaviors before scaling change across their organizations. 45:15 – Closing Reflections Barry concludes the first chapter, thanks listeners for joining this special audiobook preview, and invites them to continue the journey with Artificial Organizations . FAQs Q1. Is this Unlearn episode an audiobook preview? Yes. This special episode includes Barry O’Reilly’s recorded introduction followed by the opening chapter of the audiobook edition of Artificial Organizations , narrated by Barry himself. He explains why the audio edition was created, what recording the book required, and how its ideas connect to his work with executive leaders and teams. Q2. What is an artificial organization? An artificial organization is a company that deliberately combines human and machine intelligence to redesign how context is captured, information is synthesized, and decisions are made. Instead of adding AI to the edges of existing workflows, it builds judgment infrastructure into how the organization operates. Q3. How can leaders use AI to make better decisions? Leaders can use AI to capture conversations, summarize context, test assumptions, explore scenarios, prepare for meetings, and identify unresolved actions. This reduces the information leaders must carry mentally and gives them more capacity to focus on tradeoffs, consequences, and high-value judgment. Q4. What is the difference between decision velocity and decision advantage? Decision velocity is the speed at which a leader moves from a question to insight, decision, and action. Decision advantage is the quality, accuracy, and depth of context behind that decision. Strong leadership requires both because speed without insight creates chaos, while insight without action loses relevance. Q5. Why do many enterprise AI initiatives fail to create business value? They often begin with tool purchases, pilots, and mandates without redesigning how work, context, and judgment flow through the...

July 8, 202636 min

Why AI Makes Great Systems Matter Even More with Eric Baxley

AI is making it easier than ever to build products, automate work, and scale ideas. The challenge is no longer access to technology. It's designing the systems, stories, and customer understanding that turn AI into real business outcomes. In this episode of Unlearn , I'm joined by Eric Baxley , Chief Marketing Officer at Nobody Studios . Eric shares how a seventh-grade summer teaching himself to program on an Atari 800 sparked a 30-plus year career spanning software development, product management, marketing, sales, business development, and partnerships. We explore what Eric has had to unlearn while building companies in the AI era—from challenging assumptions before execution and replacing corporate polish with authentic storytelling, to designing scalable systems instead of disconnected tools. Along the way, Eric explains why great marketing still starts with deeply understanding customers, and why AI works best when it amplifies human judgment rather than replacing it. Key Takeaways Challenge the foundation before execution: Eric shared that in a previous large business, the team was “doing things right,” but not “doing the right things.” The segmentation was off, and correcting who they were really going after helped the team hit its goals for the next three years. Authentic stories build trust: Eric has had to unlearn the corporate habit of making everything polished before sharing it. He believes people respond when you show the journey, talk about the hardships, and ask for feedback along the way. Systems matter more than scattered tools: Eric warned against building a set of siloed point products. In a venture studio building multiple companies at once, he said you need systems that can scale, especially with the speed of AI. Use AI to strengthen your thinking, not replace it: Eric described writing and revising a LinkedIn post himself before turning to AI. The post performed well, and his takeaway was that people could sense it was naturally written and not simply generated. Real messaging starts with real customers: When Eric joined Nobody Studios, one of the first things he did was speak directly with patent attorneys for Evalify. That helped him understand their language, their concerns, and what message would actually resonate. Additional Insights Storytelling has to match the person and the moment: Eric explained that messaging should change based on persona, stage in the buyer journey, industry, and country. A CIO, CTO, or chief product officer may each need to hear the story differently. The right message needs different levels of depth: Eric talked about having a 30-second version, a 90-second version, and a longer version of the message ready. The point is to be prepared for the amount of attention and context the listener actually has. B2B messaging has to speak to head, heart, and wallet: Eric said that in larger enterprise deals, especially six- or seven-figure buying decisions, messaging needs to appeal to logic, emotion, and business economics. Consistency across channels is hard but necessary: Eric noted that messages can quickly become confusing when sales, TikTok, Instagram, LinkedIn, and other channels are all saying different things. The challenge is keeping the story consistent while still tailoring it to the audience. Automation still needs personalization: Eric uses some automated systems, but he emphasized that he still fine-tunes and personalizes messages based on someone’s background. Otherwise, outreach becomes the kind of generic message people ignore. Episode Highlights 00:00 - Episode Recap Eric Baxley explains why company building in the AI era requires scalable systems, curiosity, grit, and a willingness to dig into the details rather than staying at a high level. 01:53 - Guest Introduction: Eric Baxley Barry introduces Eric Baxley, Chief Marketing Officer at Nobody Studios, and highlights his work across growth, marketing, partnerships, and company building. 03:13 - The Seventh-Grade Spark Eric shares how teaching himself to program on an Atari 800 while growing up in Germany sparked his interest in technology and shaped the rest of his career. 05:10 - Unlearning Assumptions Eric explains why he no longer assumes the base foundation of a business is solid, using a segmentation mistake from a large business as an example. 06:20 - Moving Past Corporate Polish Eric talks about unlearning the need for everything to be buttoned up and why showing the real journey can make the work more relatable. 07:39 - Authentic Stories in a Noisy Market Barry and Eric discuss why honest stories about what is working, what is difficult, and what is still being learned can stand out from exaggerated AI claims. 10:48 - Tailoring the Story Eric breaks down how messaging needs to be adapted by persona, buyer journey stage, industry, and country. 13:24 - Learning From Patent Attorneys Eric shares how he started shaping Evalify’s messaging by speaking directly with patent attorneys instead of creating sales and marketing materials in a vacuum. 18:08 - Head, Heart, and Wallet Eric explains why enterprise messaging needs more than a clinical problem-and-solution structure; it also needs emotion, business value, and a clear story. 22:40 - Building Systems Backwards From the Customer Eric talks about the explosion of marketing technology and why he starts with the persona, the outcome, and the channels where customers actually spend time. 27:37 - Writing Before AI Eric describes how he wrote and revised a LinkedIn post himself before involving AI, and why he believes the human work helped it resonate. 31:48 - Human-in-the-Loop AI for Patent Attorneys Eric explains how Evalify helps patent attorneys with work that can take 20 to 30 hours, while making clear that the product amplifies their work rather than replacing them. 33:49 - Building Companies Faster and More Frugally Eric shares why he is excited that small teams can now use AI capabilities to build, fund, and scale companies differently than in the past. 35:36 - Closing Reflections Barry thanks Eric for sharing lessons from his work at Nobody Studios and looks forward to continuing to build together. FAQs Q1. Who is Eric Baxley? Eric Baxley is the Chief Marketing Officer at Nobody Studios. In the episode, he describes a career that began in software development and later moved into product management, marketing, sales, business development, and partnerships. Q2. What does Eric Baxley say leaders need to unlearn? Eric says he has had to unlearn assuming the foundation is already right, relying too much on corporate polish, and building around siloed tools instead of scalable systems. Q3. Why does Eric Baxley focus so much on customer segmentation? Eric believes many teams jump straight to execution without checking whether they are going after the right customers. He shared an example where fixing segmentation helped a business focus on the right audience and hit its goals. Q4. How does Eric Baxley approach messaging for AI products? Eric starts by talking to the people the product is meant to serve. With Evalify, he spoke with patent attorneys, used their language, tested the message, and worked with an advisory board to see whether the story resonated. Q5. What role should AI play in marketing, according to this episode? AI can help with speed, systems, and efficiency, but Eric and Barry emphasize that human judgment still matters. Eric’s examples show that personalization, customer understanding, and careful writing are still needed for the message to land. Useful Resources Eric Baxley on LinkedIn - https://www.linkedin.com/in/ericbaxley/ Nobody Studios on LinkedIn - https://www.linkedin.com/company/nobodycrowd/ Nobody Studios Evalify The Challenger Sale Artificial Organizations - https://artificialorganizations.com/ Follow the Host Barry O’Reilly on LinkedIn: https://www.linkedin.com/in/barryoreilly Barry O’Reilly’s website:<a href="https://barryoreilly.com/" rel="noopener noreferrer"...

June 24, 202650 min

Judgment in the Age of AI with John Cutler

AI is changing how leaders think, decide, and work with their teams. But as John Cutler points out in this conversation, the real shift is not simply about faster answers or more productivity. It is about becoming more aware of the judgment systems we already use, often without noticing. In this episode of the Unlearn Podcast , I’m joined again by John Cutler , product thinker, systems explorer, and Head of Product at Dotwork. We explore how AI can help leaders expose their thinking, pressure test decisions, and build stronger team judgment, while also making it easier to accelerate poor habits, shallow work, and false confidence. John shares practical examples from product prioritization, survey design, objection handling, and team collaboration to show where AI can genuinely improve decision quality. We also get into the tradeoffs: why AI can make work feel like “hard mode,” why downtime still matters, and why intentionality is becoming one of the most important leadership skills in this moment. Key Takeaways AI exposes how leaders make decisions: AI tends to amplify the decision system already there. When a leader’s thinking is clear, AI can help make it visible and reusable; when it is vague, AI can make that vagueness move faster. Judgment is built differently depending on the situation: John explains that some judgment comes from repetition and tacit pattern recognition, while other judgment develops through coaching, discussion, and working alongside people with more experience. AI can help turn intuition into something teams can use: John’s example of documenting his product prioritization heuristic shows how AI can help make internal judgment concrete. The value comes from helping others understand why certain decisions matter, not just what the decision is. Better AI use starts with knowing what you know: John contrasts product prioritization, where he has deep experience, with survey design, where he knows there is established expertise to draw from. The skill is recognizing whether AI should extend your own judgment or help you borrow from a domain expert. Teams using AI well can raise decision quality: Barry shares how AI can help teams pressure test assumptions, run scenarios, and ask disconfirming questions without losing momentum. The real advantage comes when AI strengthens collaboration rather than replacing it. AI can also accelerate bad instincts: John warns that AI can make poor thinking look polished. A team can paste AI onto an existing process and call it transformation without changing how decisions are actually made. Intentionality matters more than productivity: AI can reduce friction, but it can also remove the pauses where judgment forms. Leaders need to design space for reflection, not just optimize for more output. Additional Insights Individual metacognition: This is understanding how you think and make decisions. John’s examples show that leaders get more value from AI when they can first make their own judgment system visible. Social metacognition: This is understanding that other people think, perceive, and engage differently. AI becomes more useful when it supports the conversation between people instead of flattening everyone into the same process. Computational metacognition: This is understanding what LLMs are good at, where they fail, and how to work with them responsibly. John argues that leaders need this skill so they know when to trust AI, when to challenge it, and when to bring in human expertise. Objection handling as a repeatable system: John’s team did not ask AI to create a generic sales guide. They role-played real objections, captured the discussion, compared their responses against best practices, and turned that into a system that could review future calls. The deeper lesson: AI becomes more useful when it is connected to real work, real context, and a team’s actual judgment. Without that grounding, it risks creating more output without improving the quality of decisions. Episode Highlights 00:00 – Episode Recap John Cutler opens with a story about how judgment often comes from repetition and tacit signals, not neat frameworks. The episode explores what happens when AI starts making those hidden decision systems visible. 02:02 – Guest Introduction: John Cutler Barry welcomes back John Cutler, product thinker, systems explorer, and Head of Product at Dotwork, for a conversation about judgment, decision making, and collaboration in the age of AI. 04:59 – How Judgment Gets Built John explains that judgment develops differently depending on the context: through individual practice, repeated exposure, mentorship, team discussion, and comparison against examples of quality. 08:58 – Making Prioritization Thinking Visible John shares how he used AI to document his own scoring heuristic for product prioritization, giving a teammate deeper insight into why certain ideas mattered more than others. 12:11 – Knowing When to Borrow Expertise Using survey design as an example, John explains how AI can help access existing expert knowledge when you are not the expert yourself. The key is being honest about the limits of your own judgment. 13:56 – From Answers to Better Questions Barry reflects on the shift from using AI to get answers toward using it to challenge thinking, improve decisions, and bring stronger questions to colleagues. 18:04 – Why Better Surveys Lead to Better Decisions John explains how improving a survey from average to strong can materially change the quality of insight a team gets back, which then affects the quality of product decisions. 23:04 – Teams, AI, and Decision Advantage Barry shares how AI can help teams maintain momentum during ideation by quickly pressure testing scenarios, asking disconfirming questions, and bringing outside information into the room. 27:48 – Turning Objection Handling into a System John describes how his team recorded a live objection-handling exercise, analyzed it against best practices, and turned the team’s collective knowledge into a reusable system. 31:32 – The Three Forms of Metacognition John introduces individual, social, and computational metacognition as three skills leaders need to work effectively with AI and with each other. 35:19 – AI Exposes Leadership Systems Barry and John discuss why AI can feel uncomfortable for leaders: it reveals whether there is a real decision-making system underneath the confidence. 37:34 – When AI Makes Every Decision Feel Hard John raises an important limitation: AI can remove small pauses in the workday, leaving people constantly operating at high cognitive load. 41:58 – Productivity Fatigue and Agent Overload Barry and John discuss the temptation to run too many AI-assisted tasks at once, and why that can create more noise rather than better outcomes. 44:23 – Designing Time to Think Barry shares how he intentionally creates time for walking, exercise, and reflection to avoid over-optimizing for fast, reactive decisions. 46:38 – Intentionality Over Process Theater John explains why intentionality is different from rigid process. The opportunity is to design better systems without flattening the richness of how teams actually work. 50:11 – Closing Reflections Barry wraps the conversation by reflecting on the opportunity for leaders to use AI not just to move faster, but to become more aware of how they think, decide, and scale judgment across teams. Useful Resources Dotwork – John Cutler’s work focuses on helping teams and organizations better understand how work, decisions, and systems connect. Artificial Organizations – Barry references the book and the CSTA loop as part of his work on AI, decision making, and organizational performance. Daniel Kahneman’s System 1 and System 2 Thinking – Referenced in the discussion on snap decisions, deeper thinking, and productivity fatigue. Pugh Analysis – John mentions this as an example of a prioritization approach originally intended to help experts independently rate options and then discuss differences in judgment. Follow the Host LinkedIn: https://www.linkedin.com/in/barryoreilly Personal site: <a href="https://barryoreilly.com/" rel="noopener noreferrer"...