How I Tested That

Hosted by David J Bland

Business TechnologyInterviews guests

Website RSS feed LinkedIn Facebook

Episodes

Latest episode

Aug 2026

Language

About the show

Testing your ideas against reality can be challenging. Not everything will go as planned. It’s about keeping an open mind, having a clear hypothesis and running multiple tests to see if you have enough directional evidence to keep going. This is the How I Tested That Podcast, where David J Bland connects with entrepreneurs and innovators who had the courage to test their ideas with real people, in the market, with sometimes surprising results. Join us as we explore the ups and downs of experimentation… together.

Listen to episodes

55 recent

August 19, 2026Episode 5547 min

Barry O’Reilly | How I Tested an Artificial Organization

Summary In this episode, I’m joined by Barry O’Reilly, he’s an entrepreneur, advisor, author and a friend of mine who works with executives to redesign how their organizations perform. He’s also the co-founder of Nobody Studios, an AI venture studio building and launching over 100 companies. Barry and I chat about what he’s learned from years of helping corporations experiment, redesign systems, and make better decisions under extreme uncertainty. We talk about why so many AI transformations start in the wrong place, why leaders should focus on the work before the tools, and how capturing conversations can create a kind of organizational memory that gets smarter over time. Barry also shares his lessons from building Nobody Studios and how that work influenced his new book Artificial Organizations. If you want to learn more about testing with AI to redesign how work gets done while making better decisions, you’ll love this episode. Takeaways Start with the work, not the AI tools. The biggest gains come from redesigning how work and decisions happen, then choosing technology that supports that flow. Human judgment becomes more important, not less. AI can capture, synthesize, and model information at scale, but leaders still need to decide what matters and what action to take. Treat conversations and decisions as data assets. Capturing meetings, transcripts, decisions, and outcomes creates organizational memory that can be reused instead of constantly recreating context. Better systems can outperform experience alone. Deep domain expertise still matters, but rigorous decision-making systems combined with machine intelligence can challenge gut instinct as the default. AI transformations fail when they are treated as tool rollouts. Buying Copilot or another platform does not change how an organization works unless behaviors, processes, and operating systems change with it. Leaders can accelerate adoption by role-modeling experimentation. Admitting “I don’t know,” trying tools in real work, and openly sharing what works and what does not creates permission for others to learn. Start with one decision or workflow. Rather than attempting a company-wide AI transformation, pick a recurring decision or process, test a new way of working, and learn from the result. Guest Links Barry’s Website: https://barryoreilly.com/ Barry’s LinkedIn: https://www.linkedin.com/in/barryoreilly/ Artificial Organization: https://artificialorganizations.com/ Nobody Studios: https://nobodystudios.com/

August 5, 2026Episode 5451 min

Mark Graban | How I Tested the Smallest Change

Summary In this episode, I’m joined by Mark Graban, he’s an expert in Lean Leadership, Psychological Safety, and Continuous Improvement. Mark and I discuss the idea of the “smallest test of change,” a principle rooted in lean thinking that can be applied far beyond manufacturing. Speaking of manufacturing, we sort of geek out on what we’ve learned from being around Toyota and seeing how they work in practice, not just in theory. We also dive into why organizations reward people for sounding certain, how leaders unintentionally punish experimentation, and why psychological safety is essential if you expect people to admit what they don’t know. Mark shares how he’s applying these ideas to his own work by testing a new book through iterative publishing. If you want to learn more about creating a culture where people feel safe to speak up and use smaller tests to drive meaningful change, this episode is for you. Takeaways Start with the smallest test of change - something quick, inexpensive, and focused on learning. Separate knowledge from assumptions by asking, “How do we know this is true?” before acting with certainty. Psychological safety is essential for experimentation because people must feel safe admitting mistakes, asking for help, and challenging ideas. Leaders shape culture through their reactions . Saying failure is acceptable means little if people are punished when a test does not work. Even experienced practitioners can overbuild . Mark’s AI coaching experiment reinforced the need to test demand and usefulness before adding more features. Guest Links Website: https://www.markgraban.com/ LinkedIn: https://www.linkedin.com/in/mgraban/

July 22, 2026Episode 5341 min

Mike Yarbrough | How I Tested a Jewelry Business

Summary In this episode, I’m joined by Mike Yarbrough, he’s the founder of Rustic & Main, a custom jewelry company that’s known for crafting unique, story-driven wedding bands and engagement rings. They specialize in combining traditional heavy metals with unconventional, nostalgic inlays. Mike shares how his handmade wooden wedding band accidentally started it all by generating unexpected customer demand, while still working his day job. We discuss how the business grew from a garage operation into a team producing handcrafted rings with deeply personal stories. We also get into the product failure that led Mike and his team to create its durability-testing process (appropriately called “the ringer”) and how he uses prototypes and limited launches to test out new revenue opportunities. This is a story about following market signals, learning from what breaks, and building a company without losing the craftsmanship and meaning that made customers care in the first place. Takeaways Customer pull can be one of the strongest early signals that an idea is worth pursuing. Starting small gave Mike time to test demand, refine the product, and understand the business before leaving software development. Product failures led Rustic & Main to create a more rigorous durability-testing process for every new design. Customer requests helped the company expand from wooden rings into titanium, gold, custom inlays, and new product categories. Testing a new business model, such as retail, requires patience because not every opportunity grows as quickly as the original business. Guest Links Rustic and Main: https://rusticandmain.com/

July 8, 2026Episode 5240 min

Maggie Baker | How I Tested Clothing Subscriptions

Summary In this episode, I’m joined by Maggie Baker, she’s the Founder of Threadeco, an eco-friendly clothing company rethinking how people discover, buy, and keep fashion. Maggie talks about how she started with a high-end fanny pack business that landed some serious celebrity traction, but still failed when the economy shifted in 2008. That experience taught her hard lessons about pricing, channel risk and adaptability. Years later, she turned those insights into Threadeco, when she was inspired by imperfect produce boxes. Maggie shares how she tested her early MVP with a Google Form, friends and family, hand-packed boxes, and no real payment system. Fast forward to today, business is thriving and we discuss how she is now blending retail, styling and subscriptions together. It is a great conversation about learning the hard way, starting scrappy, listening to customers, and building a business model that can actually adapt. Takeaways A past failure became the foundation for a better business model. Maggie’s high-end fanny pack business got retail traction and celebrity attention, but the 2008–2009 downturn exposed pricing, channel, and adaptability risks she had not fully tested. Testing does not have to be sophisticated to be useful. Threadeco started with a Google Form, friends-and-family beta testers, hand-curated clothing boxes, and manual feedback loops. The first version of the model revealed the wrong customer. Early testing showed that customers who only wanted to try on many items and return most of them were not a good fit for Threadico’s economics. Threadeco changed the model based on learning, not imitation. Instead of copying Stitch Fix, Maggie shifted to a prepaid five-piece box with limited exchanges, creating a model that better fit her inventory, cash flow, and customer behavior. The brick-and-mortar store became more than a retail channel. The store functions as a warehouse, styling space, discovery engine, and subscription acquisition channel. Location testing mattered. Moving within Old Sacramento changed the business dramatically, showing how even a two-minute difference in storefront location can create a completely different retail outcome. The next big test is integrating physical retail, subscriptions, styling, and tech. Maggie is exploring an app that connects customer style profiles, boxes, in-store experiences, and stylist recommendations into one ecosystem. Guest Links Maggie’s LinkedIn: https://www.linkedin.com/in/getajobla/ Threadeco: https://www.threadeco.com/

June 24, 2026Episode 5137 min

Eric Lowe | How I Tested Mental Hardening

Summary In this episode, I’m joined by Eric Lowe. He’s the CEO and Co-founder of Aptiva Health, an outpatient group that offers specialized care in orthopedics, physical therapy, pain management, and mental wellness. Eric shares how solving one problem at a time helped him scale to 14 locations and why he believes great businesses are built by developing people first. We dig into launching an ambitious healthcare marketplace that had all the ingredients of a great product, only to run headfirst into the brutal realities of marketplace dynamics, timing and customer education. Eric also details how his team uncovered a breakthrough business by integrating mental health directly into orthopedic recovery. He shares how they experiment with new healthcare models before overinvesting, and why listening closely to patients and providers often reveals opportunities you never could have imagined. If you’re building in healthcare and trying to figure out how to test bold ideas, this episode is for you. Takeaways Great businesses come from solving the next problem . Long-term growth often compounds from consistently identifying and solving the most pressing issue in front of you. Not every customer problem is yours to solve . Sustainable businesses focus on problems they are uniquely positioned to solve while aligning value across all stakeholders. Marketplaces are brutally hard to build . Even strong solutions can struggle when adoption requires changing deeply ingrained customer behavior. Sometimes the customer isn’t who you think it is . Success often comes from recognizing when the original target market needs to shift entirely. Unexpected opportunities can outperform your biggest bets . Some of the strongest growth opportunities come from adjacent problems you discover while serving customers. Adoption depends on reducing friction . New solutions succeed when they fit naturally into how people already behave rather than forcing entirely new habits. Culture is built through actions, not slogans . Teams buy into organizations when leadership consistently demonstrates the values they claim to stand for. Big investments require proving the economics first . When capital commitments are high, getting the financial assumptions wrong can become incredibly expensive. Guest Links Eric’s LinkedIn: https://www.linkedin.com/in/ericloweaptiva Aptiva Health: https://www.aptivahealth.com/ Broker Driven: https://brokerdriven.com/

June 10, 2026Episode 5041 min

Courtney Honda and Slava Borisov | How I Tested Pet Retail

Summary In this episode I’m joined by Courtney Honda and Slava Borisov co-founders of Puptqe. What started as a deeply personal response to a health scare involving their dog Champ has grown into a unique retail experience built around community and creating memorable moments for dogs and their owners. We explore how they tested their way through one of the most competitive retail categories imaginable, discovering that success wasn’t about competing with big-box stores on price or selection. Instead, they focused on creating experiences that customers couldn’t get anywhere else, from dog-friendly events and memberships to immersive in-store moments designed to turn visitors into loyal advocates. We also get into the realities of building a brick-and-mortar business, including lessons around site selection, customer retention, community-driven marketing, and the surprising acquisition channels they tested to help them grow. Courtney and Slava share how they learned to stop trying to serve every pet owner, focus on their ideal customer, and transform retail from a transaction into an ongoing relationship. If you’ve ever wondered how to test a physical retail store in a crowded market, this episode is for you. Takeaways Differentiate by owning a niche, not by competing on price - Puptqe succeeded by becoming a destination for dog experiences and hospitality instead of trying to out-discount larger pet retailers. Make the experience the product; sales will follow - Customers come for the events, community, and memories, which naturally drives purchases and loyalty. Stop marketing to everyone and focus on your ideal customer - Growth accelerated once they identified who their best customers were and tailored their offerings around them. Retention matters more than acquisition - Long-term success came from creating reasons for customers to return again and again, not just making the first sale. Community and word-of-mouth outperform paid advertising - Loyal customers and local advocacy generated more sustainable growth than trying to outspend competitors on ads. Understand customer behavior, not just demographics - Knowing how customers spend their time and make decisions proved more valuable than basic age, income, or location data. Design every customer touchpoint intentionally - Every interaction, from the greeting to the checkout experience, was crafted to create memorable moments. Consistency beats constantly chasing new tactics - Small improvements executed repeatedly created stronger results than jumping from one growth idea to the next. Guest Links Courtney’s LinkedIn: https://www.linkedin.com/in/courtney-honda/ Slava’s LinkedIn: https://www.linkedin.com/in/slavaborisov/ Puptqe Website: https://puptqe.com/

May 27, 2026Episode 4949 min

Chris Butler | How I Test AI Agents at GitHub

Summary In this episode I’m joined by Chris Butler. He’s a longtime product leader and operator whose career spans companies such as Microsoft, Google, Facebook, and now GitHub, where he works on agentic workflows across the organization. We explore how AI is reshaping the way modern product teams think, collaborate, ship and its ripple effects on how we manage process and decision making. Chris and I chat about the messy realities behind agentic systems such as why removing too much friction can actually hurt decision quality and why qualitative research matters more now than ever before. Chris gives a candid behind the scenes look into what’s working, what’s failing, and why experimentation itself may become one of the most important capabilities in the AI era. If you’ve been wondering what testing AI Agents actually looks like inside a cutting edge company, this episode is for you. Takeaways AI is collapsing traditional product development workflows, but not necessarily eliminating the need for product managers, engineers, or designers. Instead, roles are decomposing into smaller tasks where humans and machines each handle different types of work. Removing all friction from product development can actually reduce decision quality. Chris argues that tension between desirability, viability, and feasibility perspectives is still critical because reasoning often happens through human discussion, not just inside individual minds or AI systems. AI-generated “rude feedback” tools can help teams improve ideas faster because people are often more receptive to harsh critique from a machine than from another human. GitHub experimented with sarcastic AI Q&A systems that surfaced weak assumptions and missing details without the reputational risk of peer criticism. The future of AI inside organizations may be less about autonomous agents replacing humans and more about “process as code.” GitHub is experimenting with natural-language policy documents that both humans and agents can read to automate operational workflows, release management, and risk detection. Product teams are at risk of building faster without learning faster. Chris warns that vibe coding and rapid prototyping can unintentionally reduce time spent talking to customers and conducting qualitative research, which still remains essential for understanding mental models and uncovering hidden assumptions. Agentic workflows become most valuable when they reduce operational toil instead of replacing human judgment. GitHub is using AI to automate repetitive coordination tasks like release tracking, documentation generation, and status updates so teams can spend more time on strategic thinking and collaboration. I nternal experimentation matters just as much as customer-facing innovation. Chris emphasizes that many AI workflow experiments inside GitHub are intentionally small, lightweight tests designed to explore possibilities quickly before deciding whether to scale, modify, or abandon them. The biggest long-term challenge for enterprise AI adoption may not be model capability, but integration, governance, and organizational coordination. Authentication, permissions, fragmented tooling, disconnected workflows, and siloed information remain major barriers to making agentic systems truly useful at scale. Guest Links LinkedIn: https://www.linkedin.com/in/chrisbu/ GitHub Next: https://githubnext.com/

May 13, 2026Episode 4844 min

Chad Holdorf | How I Tested Pull Requests

Summary In this episode I’m joined by Chad Holdorf, longtime product and technology leader whose career spans John Deere, Salesforce, Pendo, and now Demandbase, where he leads AI initiatives across the company. We explore how AI is fundamentally reshaping the way modern product teams test, ship, and learn, from debugging customer issues directly against live codebases to product managers and support teams submitting pull requests themselves. Chad shares how tools like Cursor and Claude are collapsing traditional handoffs between product, engineering, and support, creating a much faster feedback loop between customer problems, experimentation, and shipped solutions. We also get into the messy reality behind enterprise AI adoption, including data quality, hallucinations, trust, evals, and why testing AI products inside real customer environments is much harder than most demos make it look. Chad gives us a peek into how his own workflow has changed, how his teams are learning by building in real time, and why this moment reminds him of the early days of Lean Startup, where he and I first met. If you’ve been wondering what AI-native product development actually looks and feels like inside a real company, this episode is for you. Takeaways AI is collapsing traditional handoffs between product, engineering, and support teams. Chad described customer support teams going directly into code repositories with AI tools to investigate issues, understand root causes, and eventually submit merge requests themselves. Most enterprise AI demos fall apart when connected to messy real-world customer data. Chad emphasized that “just putting Claude on top of the data” failed quickly without extensive labeling, validation, testing, and human feedback loops. Customers could detect hallucinations within a few prompts. AI systems expose hidden data inconsistencies inside organizations. One example showed AI selecting a custom CRM field that technically produced better targeting results than the field support teams were trained to use, creating confusion about which “truth” was actually correct. Trust has become the critical success factor for enterprise AI adoption. Chad explained that once customers encounter inaccurate outputs, confidence in the system drops immediately, which forces teams to spend enormous time improving prompts, SQL queries, evals, and validation workflows before broader rollout. Product managers are increasingly becoming hands-on builders again. Instead of relying entirely on engineering handoffs, Chad now spends large portions of his week inside Cursor and AI coding agents investigating bugs, generating tickets, reviewing repos, and shaping product direction directly through code conversations. AI-native workflows dramatically compress feedback loops. Problems that previously took days of back-and-forth between support, product, and engineering can now move from customer issue to deployed fix in under an hour through AI-assisted workflows and automated merge requests. The biggest organizational bottleneck is shifting away from engineering speed toward enablement and adaptation. Chad compared this moment to early Agile adoption, where downstream teams like sales, support, and training struggled to keep pace with accelerated shipping cycles. AI is now amplifying that challenge even further. Continuous learning and experimentation matter more than formal process mastery right now. Chad repeatedly compared the current AI moment to the early Agile movement: the people progressing fastest are the ones willing to try tools, build things, stay curious, and learn in public rather than waiting for established best practices or certifications. Guest Links LinkedIn: https://www.linkedin.com/in/chadholdorf/ Demandbase: https://www.demandbase.com/

April 29, 2026Episode 4751 min

Büşra Coşkuner | How I Test B2B vs B2C

Summary In this episode, I’m joined by Büşra Coşkuner, a product management coach and trainer who helps teams move from project thinking to product thinking. We explore how she approaches product discovery and testing. From building zero-to-one products at Doodle to coaching teams across B2B and B2C environments, Büşra shares how to actually operationalize experimentation beyond just A/B tests. We also dig into how to test when you don’t have much data, how to combine qualitative and quantitative insights, and why many teams get stuck thinking they’re doing product work when they’re really just managing tickets. If you’re trying to build a stronger testing culture or just want to make better decisions this episode will challenge how you think about product metrics and experimentation. Takeaways Product transformation is a leadership decision - If leadership isn’t backing the shift from projects to products, it won’t happen, bottom-up enthusiasm isn’t enough. Most “product orgs” aren’t actually product orgs - Adopting Scrum and calling someone a product owner doesn’t mean you’re doing product, many teams are still just managing tickets. You can’t test what you can’t measure - Without proper data instrumentation, teams fall into a “build, build, build” loop instead of build–measure–learn. Metrics frameworks are a starting point, not the system - Pirate metrics (AARRR) or customer factory models help, but real insight comes from adapting them to your actual business model. Qualitative data is not optional - Quant tells you what is happening, qual tells you why. In low-data environments, qual becomes your primary signal. “No data” is usually an excuse - Even in B2B, you can extract directional insights, from sales teams, customer conversations, and patterns across feedback. A/B testing is over-indexed and often misused - Experimentation goes beyond A/B testing. Many teams default to it even when it’s impractical or irrelevant. Sometimes building is the test - For low-risk features, the fastest way to learn is to ship and observe behavior, treat the release itself as the experiment. B2B testing requires creativity, not scale - From sales-assisted experiments to prototype validation and even WhatsApp groups, testing in small markets is possible if you rethink the approach. AI changes the cost of being wrong - When building becomes cheap, you don’t always need heavy upfront validation, you can test the problem through the solution, as long as you’re willing to kill what doesn’t work. Guest Links LinkedIn: https://www.linkedin.com/in/busra-coskuner/ Website: https://busra.co/

April 15, 2026Episode 4630 min

Gil Vaisman | How I Tested ADU’s

Summary In this episode I’m joined by Gil Vaisman. He’s the founder of GoADU, a construction company focused on building accessory dwelling units. We explore how he went from a 15-year career in film editing to building a construction business that helps homeowners unlock equity and create new living spaces. What started out as a personal project in his own backyard turned into a growing business built through trial, error, and constant iteration. Gil shares how he tested his way into the market, from helping friends navigate permitting to evolving into a guaranteed fixed pricing model. We also dig into how he qualifies customers, avoids costly mistakes, and thinks about what to test next in an industry that’s rapidly changing. If you’re trying to turn a personal pain point into something scalable, this episode is a great look at how testing can lead to a viable business. Enjoy my conversation with Gil Vaisman. Takeaways Great businesses often start as personal pain points - Gil’s ADU company emerged from building one in his own backyard and helping friends navigate the same confusing process. Transferable skills matter more than industry experience - His background in film production translated directly into construction, both require coordination, budgeting, timelines, and managing complex teams. Early traction came from education, not selling - In the beginning, most customers didn’t even know what an ADU was, growth required teaching the market before capturing it. Trial and error built the real expertise - Navigating difficult permitting processes and making costly mistakes early on became the foundation for a repeatable, refined system. Pre-qualification is critical in complex services - Gil now asks 20–25 upfront questions before taking on a client, ensuring alignment and reducing downstream risk. Competing on price is a starting point, not a strategy - The business initially won work by being the cheapest, but evolved into a premium, top-20% offering focused on quality and service. A strong value proposition can reduce industry fear - Guaranteed fixed pricing became a key differentiator, directly addressing customer anxiety around hidden costs and change orders. Future innovation is constrained by feasibility, not demand - Customers clearly want faster, cheaper builds (prefab, SIPs), but adoption is limited by execution risk, expertise gaps, and inconsistent quality. Guest Links GoADU: https://www.goadu.com/ Vaisman Construction: https://www.vaismanconstruction.com/