Alexa's Input (AI)

Hosted by Alexa Griffith

TechnologyInterviews guests

Website RSS feed LinkedIn YouTube

Episodes

Latest episode

Jun 2026

Language

About the show

Alexa’s Input is a podcast about how technology actually moves forward. Hosted by Alexa Griffith, it features conversations with engineers, founders, CEOs, and leaders shaping today’s tech landscape. Each episode digs into the decisions behind the systems — what’s being built, what’s being questioned, and why it matters now. Opinions are my own Linktree: https://linktr.ee/alexagriffith Website: https://alexagriffith.com/ LinkedIn: https://www.linkedin.com/in/alexa-griffith/ X: @lexal0u

Listen to episodes

60 recent

June 15, 2026Episode 111 hr 17 min

David Aronchick on Distributed Data Orchestration with Expanso

In this episode of Alexa's Input (AI), I sit down with David Aronchick, co-founder and CEO of Expanso and former product lead for Kubernetes at Google.Data is growing everywhere outside your data center. Solar panels in remote across a country. Security cameras at retail stores. IoT sensors across factory floors. And moving that data to the cloud for processing? It's expensive, slow, and often restricted by compliance.David is an expert when it comes to solving distribution problems. He led Kubernetes product at Google, co-founded Kubeflow to bring ML to production, and now he's building Expanso to tackle a difficult constraint: when your data can't move, how do you process it where it lives?We discuss:- The need for distributed data orchestration-Upstream data control: filtering and transforming at the source- Three forces making edge computing inevitable (physics, regulations, economics)- How to build successful open source infrastructure projects- Customer discovery and finding real pain points- His transition from Protocol Labs to founding Expanso- ETL pipelines: moving the first four steps closer to the data- Context loss and lineage in distributed systems- Processing 400,000 signals per second with 150MB agents- AI observability: attaching source metadata to training data- Running ML pipelines at the edge- Real-world deployment challenges (bandwidth, regulations, cost)Expanso is rethinking how we process data in an AI-native world—moving compute to data instead of data to compute. If you want to understand where distributed systems and edge computing are heading, this is a deep dive into the infrastructure layer beneath modern AI applications.General Podcast LinksWatch: https://www.youtube.com/@alexa_griffith Read: https://alexasinput.substack.com/ Listen: https://creators.spotify.com/pod/profile/alexagriffith/ More: https://linktr.ee/alexagriffithLearn more about the host atWebsite: https://alexagriffith.com/ LinkedIn: https://www.linkedin.com/in/alexa-griffith/Find out more about the guest atLinkedIn: https://www.linkedin.com/in/aronchick/ Twitter/X: https://x.com/aronchick GitHub: https://github.com/aronchick Expanso Website: https://expanso.io/ResourcesExpanso Website: https://expanso.io/ Kubernetes: https://kubernetes.io/ Kubeflow: https://www.kubeflow.org/ CNCF (Cloud Native Computing Foundation): https://www.cncf.io/ Protocol Labs: https://protocol.ai/KeywordsDavid Aronchick, Expanso, Kubernetes, Kubeflow, distributed systems, edge computing, data pipelines, ETL, upstream data control, Google Kubernetes Engine, open source, CNCF, observability, log processing, data lineage, provenance, schema enforcement, IoT, edge AI, distributed data, machine learning infrastructure, Protocol Labs, IPFS, Filecoin, data governance, compliance, GDPR, bandwidth optimization, data aggregation, AI infrastructure, multi-cloud, hybrid cloud, real-time processing

June 3, 2026Episode 101 hr 42 min

How vLLM and llm-d Changed AI Inference with Rob Shaw

In this episode of Alexa’s Input (AI), I sat down with Rob Shaw from Red Hat to talk about how AI inference evolved from a simple model serving problem into a large-scale distributed systems problem.We explored the infrastructure shifts behind modern LLM serving, including how vLLM and PagedAttention changed the economics and efficiency of inference, why KV cache management became one of the most important bottlenecks in production AI systems, and how orchestration layers like llm-d are emerging to coordinate distributed inference.We also discuss:how LLM inference differs from traditional model serving runtimesKV cache, prefix caching, and cache-aware routingwhy throughput and latency became major infrastructure challengeslong-context agents and repeated inference callsdistributed inference on Kubernetesintelligent routing, flow control, and load balancingprefill/decode disaggregationenterprise AI deployment realitiesvLLM has become one of the most important open-source projects in AI infrastructure, and llm-d represents a newer shift toward treating inference as a coordinated distributed system rather than just a single runtime problem.If you want to better understand the systems layer beneath modern AI applications, this episode is a deep dive into where inference infrastructure is heading next.General Podcast LinksWatch: ⁠⁠⁠⁠⁠⁠https://www.youtube.com/@alexa_griffith⁠⁠⁠⁠⁠⁠Read: ⁠⁠⁠⁠⁠⁠⁠⁠https://alexasinput.substack.com/⁠⁠⁠⁠⁠⁠⁠⁠Listen:⁠⁠ ⁠⁠https://creators.spotify.com/pod/profile/alexagriffith/⁠⁠⁠⁠More: ⁠⁠⁠⁠⁠⁠https://linktr.ee/alexagriffith⁠⁠⁠⁠⁠⁠Learn more about the host atWebsite: ⁠⁠⁠⁠⁠⁠https://alexagriffith.com/⁠⁠⁠⁠⁠⁠LinkedIn: ⁠⁠⁠⁠⁠⁠https://www.linkedin.com/in/alexa-griffith/⁠⁠⁠⁠⁠⁠Find out more about the guest at:LinkedIn: https://www.linkedin.com/in/robert-shaw-1a01399a/ Red Hat Articles: https://developers.redhat.com/author/robert-shawGithub: https://github.com/robertgshaw2-redhat ResourcesvLLM Website: https://vllm.ai/vLLM GitHub Repository: https://github.com/vllm-project/vllmllm-d Website: https://llm-d.ai/llm-d GitHub Repository - https://github.com/llm-d/llm-d KeywordsAI inference, VLLM, LMD, distributed inference, GPU optimization, open source AI, Kubernetes, multi-cluster deployment, AI infrastructure, enterprise AI AI infrastructure, Kubernetes, model optimization, speculative decoding, mixture of experts, AI deployment, performance tuning, AI systems, neural network scaling Key TopicsEvolution of vLLM and llm-dDistributed inference and routingGPU utilization and performance optimizationOpen source AI infrastructureEnterprise deployment challenges and solutions Standardization in Kubernetes for NIC exposurePerformance optimizations: quantization and speculative decodingMixture of experts architecture and parallelism strategiesFlow control and request scheduling in AI systemsEmerging hardware for AI inference, Cerebras processorReinforcement learning and AI system supportModular architecture of vLLM and ecosystem projects

May 24, 2026Episode 101 hr 13 min

Intelligence Per Watt with Emilio Andere

On this episode of Alexa’s Input (AI), I sit down with Emilio Andere, co-founder and CEO of Wafer, to talk about the future of AI infrastructure, inference optimization, and the economics driving the AI compute race.We discuss:why “intelligence per watt” may become one of the defining metrics of the AI erathe current GPU and accelerator landscape across NVIDIA, AMD, TPUs, and emerging hardware startupswhy software optimization is becoming just as important as hardware itselfinference optimization strategieswhy AI infrastructure companies are racing up the stackwhat it’s actually like building an AI infrastructure startup todayand more!Emilio also shares lessons from founding Wafer, thoughts on the future of open-source AI infrastructure, and why he believes optimizing intelligence itself could become one of the most important engineering problems.General Podcast LinksWatch: ⁠⁠⁠⁠⁠⁠https://www.youtube.com/@alexa_griffith⁠⁠⁠⁠⁠⁠Read: ⁠⁠⁠⁠⁠⁠⁠⁠https://alexasinput.substack.com/⁠⁠⁠⁠⁠⁠⁠⁠Listen:⁠⁠ ⁠⁠https://creators.spotify.com/pod/profile/alexagriffith/⁠⁠⁠⁠More: ⁠⁠⁠⁠⁠⁠https://linktr.ee/alexagriffith⁠⁠⁠⁠⁠⁠Learn more about the host atWebsite: ⁠⁠⁠⁠⁠⁠https://alexagriffith.com/⁠⁠⁠⁠⁠⁠LinkedIn: ⁠⁠⁠⁠⁠⁠https://www.linkedin.com/in/alexa-griffith/⁠⁠⁠⁠⁠⁠Find out more about the guest at:LinkedIn: https://www.linkedin.com/in/emi-andere/Wafer Website: https://www.wafer.ai/Wafer AI / Y Combinator Article: https://www.ycombinator.com/companies/waferChapters00:00 Exploring AI Conversations and Recent Podcasts02:14 Intelligence per Watt: A New Metric for AI07:35 The Manifesto: Efficiency in Civilization12:40 Founding Wafer: The Journey Begins18:08 The GPU Hardware Landscape and Market Dynamics23:07 AMD's Growing Presence in the GPU Market24:07 Emerging Competitors in the AI Hardware Space26:04 Comparing TPUs and GPUs27:21 Acquisition and Availability of TPUs28:33 Navigating the GPU Marketplace30:05 Understanding Neo Cloud Economics33:30 The AI Bubble Debate36:25 Optimizing AI Models for Performance44:46 Bottlenecks in AI Model Performance48:08 Future Directions in AI Hardware Optimization54:39 Balancing Speed and Cost in AI Performance56:54 Kernel Arena: Benchmarking AI Performance01:03:45 Lessons from Founding: Sales and Emotional Resilience01:07:38 The Future of AI: Trends and Predictions01:13:03 Outro KeywordsAI hardware, inference optimization, intelligence per watt, GPU market, AI infrastructure, Wafer, AI bubble, TPU, GPU bottleneck, AI efficiency AI optimization, large language models, AI hardware, quantization, speculative decoding, benchmarking, AI infrastructure, model training, AI startups

May 17, 2026Episode 954 min

Building Reliable Systems at Bloomberg with Sal Furino

In this episode of Alexa’s Input (AI), I sit down with Sal Furino to explore the hidden engineering work that keeps modern systems reliable.We break down what Service Level Objectives, Indicators (SLOs/SLIs), and error budgets actually mean in practice, why reliability is as much a cultural problem as a technical one, and how teams can better measure real user experience instead of just infrastructure health.Sal also explains reliability engineering and the challenges of reliability at scale, like:Why latency and correctness become harder to measure with GenAIThe difference between a bad incident and a fundamentally bad systemHow observability and telemetry shape modern engineering organizationsWhy most teams focus too much on infrastructure metrics and not enough on user happiness Why “the best systems are the ones nobody notices.”If you work in AI infrastructure, distributed systems, platform engineering, observability, or SRE, this episode is a must listen!SRECon Talk Dashboards & Dragons: Reliability Magic for AI Platforms by Alexa Griffith and Sal Furino: https://youtu.be/aWMB_7ksbkc?si=S49nPyAl_hCUIH7yGeneral Podcast LinksWatch: ⁠⁠⁠⁠⁠https://www.youtube.com/@alexa_griffith⁠⁠⁠⁠⁠Read: ⁠⁠⁠⁠⁠⁠⁠https://alexasinput.substack.com/⁠⁠⁠⁠⁠⁠⁠Listen:⁠⁠ ⁠https://creators.spotify.com/pod/profile/alexagriffith/⁠⁠⁠More: ⁠⁠⁠⁠⁠https://linktr.ee/alexagriffith⁠⁠⁠⁠⁠Learn more about the host atWebsite: ⁠⁠⁠⁠⁠https://alexagriffith.com/⁠⁠⁠⁠⁠LinkedIn: ⁠⁠⁠⁠⁠https://www.linkedin.com/in/alexa-griffith/⁠⁠⁠⁠⁠Find out more about the guest at:LinkedIn: https://www.linkedin.com/in/salvatore-furino/Rootly Interview: https://rootly.com/humans-of-reliability/salvatore-furinoReliability at Scale Talk: https://youtu.be/J-VrU5JHPlk?si=8aV8acy57NWX30KABloomberg Careers: https://bloomberg.avature.net/careers/SearchJobsChapters00:00 - Introduction: Reliability in a world reshaped by generative AI02:22 - The importance of seamless, background system design04:41 - Becoming a Customer Reliability Engineer at Bloomberg05:17 - Clarifying the CRE role and its customer focus08:02 - The importance of observability and high-scale performance in finance09:00 - Balancing technical and cultural aspects of reliability10:19 - Coaching teams to be proactive using error budgets and SLIs12:21 - The social-technical system: People, processes, and tools13:06 - Mediation of differing opinions on reliability practices15:06 - The nuanced approach to alerting and incident response17:08 - The significance of tiered SLOs and the concept of error budgets21:08 - Using signals like latency, correctness, availability, saturation in system measurement22:53 - The impact of service level "nines" on system design and resilience28:00 - Handling non-determinism and trust in AI responses33:01 - Error budgets and their role in managing deployments34:10 - The challenge of achieving five nines and data durability considerations40:03 - Adapting SLOs for GenAI systems: core principles remain intact42:23 - Measuring non-deterministic AI responses and quality proxies44:41 - The ongoing importance of reliability even in AI/ML contexts47:25 - Reacting to error budget exhaustion and proactive mitigation50:42 - The significance of involving cross-functional teams during outages55:36 - Advocating reliability investment to leadership56:24 - The customer perspective: reliability as a fundamental feature58:42 - Connecting with Sal Furino: where to follow his work and learn more about Bloomberg's engineering culture59:20 - Final advice: Focus on user happiness to avoid common pitfalls in adopting SLOs

May 10, 2026Episode 854 min

Laila: Reinventing Dating as a Social Marketplace with Kaan Divitoğlu

In this episode of Alexa’s Input (AI), I sit down with Kaan Divitoğlu, founder of Laila — a New York based startup rethinking online dating as a social marketplace centered around real plans instead of endless swiping.We talk about why traditional dating apps struggle to create real-world connection, how marketplace dynamics shape modern dating behavior, and why Kaan believes the future of dating products is less about “matching soulmates” and more about helping people actually get out on first dates.Kaan shares what he’s learned building a product around something emotional, unpredictable, and deeply human: connection. We also get into:• The metrics behind dating products and user behavior• Why most matches never turn into real dates• Designing around human psychology and social incentives• AI in dating apps — where it helps and where it shouldn’t• The process of building Laila• Social media growth, creator strategies, and startup distribution• Why Kaan thinks apps themselves may eventually disappearLinksWatch: ⁠⁠⁠⁠https://www.youtube.com/@alexa_griffith⁠⁠⁠⁠Read: ⁠⁠⁠⁠⁠⁠https://alexasinput.substack.com/⁠⁠⁠⁠⁠⁠Listen:⁠⁠ https://creators.spotify.com/pod/profile/alexagriffith/⁠⁠More: ⁠⁠⁠⁠https://linktr.ee/alexagriffith⁠⁠⁠⁠Learn more about the host atWebsite: ⁠⁠⁠⁠https://alexagriffith.com/⁠⁠⁠⁠LinkedIn: ⁠⁠⁠⁠https://www.linkedin.com/in/alexa-griffith/⁠⁠⁠⁠Find out more about the guest at:LinkedIn: https://www.linkedin.com/in/kaan-divitoglu-152779105/Laila Website: https://laila.nycLaila Instagram: https://www.instagram.com/laila.socialChapters00:00 Introduction to Layla and Its Concept04:10 The Journey of Building Layla08:43 User Feedback and Validation13:35 Metrics of Success in Dating Apps18:23 Differentiation in the Dating App Market22:54 Understanding User Behavior and Expectations27:37 Challenges in the Dating Landscape29:50 Loneliness and Social Skills in Modern Dating30:51 AI's Role in Dating Apps34:20 The Future of Dating Apps and User Experience38:19 Building Community Through Events and Social Media42:54 Navigating Social Media Marketing46:00 Rapid Fire Insights on Dating and Relationships53:33 OutroKeywordsdating app, AI, product design, real-world connections, marketplace, user engagement, social media, social tech, startup, innovation

March 19, 2026Episode 758 min

The Creative Founder Mindset with Brady Jordan

In this episode, Alexa Griffith interviews Brady Jordan, a creative director and entrepreneur, who shares his journey from aspiring software engineer to the founder of Clip Play Media and the photo app Y2Cam. Brady discusses the intersection of creativity and technology, the importance of storytelling in video production, and the challenges of self-employment. He emphasizes the need for resilience, adaptability, and a consumer-first approach in product development, while also exploring the significance of networking and community building in achieving success.Podcast LinksWatch: ⁠⁠⁠⁠⁠⁠⁠https://www.youtube.com/@alexa_griffith⁠⁠⁠⁠⁠⁠⁠Read: ⁠⁠⁠⁠⁠⁠⁠⁠⁠https://alexasinput.substack.com/⁠⁠⁠⁠⁠⁠⁠⁠⁠Listen:⁠⁠⁠⁠⁠ https://creators.spotify.com/pod/profile/alexagriffith/⁠⁠⁠⁠⁠More Links: ⁠⁠⁠⁠⁠⁠⁠https://linktr.ee/alexagriffith⁠⁠⁠⁠⁠⁠⁠Find out more about the host, Alexa Griffith, at:Website: ⁠⁠⁠⁠⁠⁠⁠https://alexagriffith.com/⁠⁠⁠⁠⁠⁠⁠LinkedIn: ⁠⁠⁠⁠⁠⁠⁠https://www.linkedin.com/in/alexa-griffith/⁠⁠⁠⁠⁠Find out more about the guest at:Website: https://www.bradyjordan.com/Chapters00:00 Introduction to Brady Jordan and His Journey06:45 The Birth of Clip Play Media14:58 Quality vs. Consistency in Content Creation24:51 Y2Cam: A Solution to Frustration30:51 Cost and Infrastructure of App Development35:30 Navigating the Challenges of Self-Employment42:51 Marketing Strategies for App Success49:04 The Value-Based Approach to Creation

February 17, 2026Episode 648 min

Securing the Software Supply Chain with Justin Cappos

Modern software is built on layers and layers of code. So how do we know we can trust it?In this episode of Alexa’s Input (AI), Alexa Griffith sits down with Justin Cappos, professor of computer science at NYU and a leading expert in software supply chain security, to unpack what trust really means in today’s digital infrastructure.From package managers and dependency chains to large-scale outages and AI systems built on inherited code, Justin explains why many security failures aren’t random accidents, they’re predictable consequences of weak process, misaligned incentives, and insecure design.They discuss:Why security only becomes visible when something breaksThe difference between unavoidable failure and negligenceHow modern software supply chains amplify small mistakesThe role of leadership and culture in preventing breachesWhy verification systems like TUF and in-toto matter more than everAs AI accelerates development and increases system complexity, the need for verifiable trust only grows. This episode is a practical look at the invisible infrastructure that keeps modern software, and increasingly, modern AI, from collapsing under its own complexity.Podcast LinksWatch: ⁠⁠⁠⁠⁠⁠https://www.youtube.com/@alexa_griffith⁠⁠⁠⁠⁠⁠Read: ⁠⁠⁠⁠⁠⁠⁠⁠https://alexasinput.substack.com/⁠⁠⁠⁠⁠⁠⁠⁠Listen:⁠⁠⁠⁠ https://creators.spotify.com/pod/profile/alexagriffith/⁠⁠⁠⁠More: ⁠⁠⁠⁠⁠⁠https://linktr.ee/alexagriffith⁠⁠⁠⁠⁠⁠Website: ⁠⁠⁠⁠⁠⁠https://alexagriffith.com/⁠⁠⁠⁠⁠⁠LinkedIn: ⁠⁠⁠⁠⁠⁠https://www.linkedin.com/in/alexa-griffith/⁠⁠⁠⁠⁠Find out more about the guest at:Website: https://engineering.nyu.edu/faculty/justin-capposNYU page: https://ssl.engineering.nyu.edu/personalpages/jcappos/Wikipedia: https://en.wikipedia.org/wiki/Justin_CapposChapters00:00 Introduction to Justin Cappos and His Work01:17 The Importance of Security in Software Systems03:50 Understanding Security Breaches: Mistakes vs. System Design Problems06:34 Cultural Factors in Security Failures09:25 Justin's Journey in Software Security12:03 The Role of Academia in Enterprise Security14:10 Evaluating Enterprise Security Systems16:58 Foundational Projects in Software Security19:21 AI Security Concerns and Future Directions24:59 The Need for MCP 2.028:57 Security Challenges with LLMs32:33 Designing Secure AI Systems37:14 Ethical Dilemmas in AI Decision-Making40:17 The Role of AI in Open Source43:44 Trust and Mindset in AI Security

February 16, 2026Episode 51 hr 6 min

The Artificial Immune System with Wendy Chin, PureCipher CEO

As AI systems grow more autonomous, the question is no longer just what they can do, but whether we can trust the data and models behind their decisions. In this episode of Alexa’s Input (AI), Alexa Griffith talks with Wendy Chin, CEO of PureCipher, about building what she calls an artificial immune system for AI, a framework designed to make data, models, and inference tamper-evident across the AI lifecycle.They unpack what data poisoning really means (training data, weights and biases, inference inputs), why small amounts of targeted poison can create outsized model misbehavior, and how generative AI lowers the barrier to sophisticated malware. The conversation expands into the security implications of agent-to-agent communication via MCP, digital twins, and why we don’t have the luxury of “shipping now and securing later.” It’s a wide-ranging discussion that moves from practical threat models to the philosophical frontier of what happens as AI becomes more human-like, and more autonomous.Podcast LinksWatch: ⁠⁠⁠⁠⁠⁠https://www.youtube.com/@alexa_griffith⁠⁠⁠⁠⁠⁠Read: ⁠⁠⁠⁠⁠⁠⁠⁠https://alexasinput.substack.com/⁠⁠⁠⁠⁠⁠⁠⁠Listen:⁠⁠⁠⁠ https://creators.spotify.com/pod/profile/alexagriffith/⁠⁠⁠⁠More: ⁠⁠⁠⁠⁠⁠https://linktr.ee/alexagriffith⁠⁠⁠⁠⁠⁠Website: ⁠⁠⁠⁠⁠⁠https://alexagriffith.com/⁠⁠⁠⁠⁠⁠LinkedIn: ⁠⁠⁠⁠⁠⁠https://www.linkedin.com/in/alexa-griffith/⁠⁠⁠⁠⁠Find out more about the guest at:LinkedIn: https://www.linkedin.com/in/wendy-chin-ctg/Website: https://www.purecipher.com/Chapters00:00 Introduction to AI Security01:16 Understanding Data Poisoning04:38 The Dangers of Malware in AI07:46 AI's Moral Dilemmas and Decision Making08:45 Building Empathy in AI13:07 The Role of Good Data in AI Training17:02 PureCypher's Artificial Immune System22:34 Digital Twins and Their Implications25:22 Nurturing AI Like a Child30:53 Data Therapy for AI36:13 The Future of AI and Human Interaction38:45 The Dark Side of AI: Hacking and Security45:03 Global Perspectives on AI Security48:11 MCP Agents and Security Concerns51:41 Philosophical Implications of AI and Human Connection01:00:04 The Sci-Fi Future of AI and Humanity

February 16, 2026Episode 445 min

Shipping Agents, Not Vulnerabilities with Ian Webster, PromptFoo CEO

As LLM apps evolve from simple chatbots to tool-using agents, the attack surface explodes, and the old security playbooks don’t hold. In this episode of Alexa’s Input (AI), Alexa Griffith sits down with Ian Webster, co-founder and CEO of PromptFoo, to break down what AI security actually looks like in practice: automated red teaming, prompt injection and jailbreak testing, evaluation workflows that scale, and why “guardrails alone” is not a security strategy.Ian shares how PromptFoo grew from a side project into a widely adopted open-source standard, what it means to raise multi-millions in a fast-moving market, and how enterprises are approaching the full vulnerability lifecycle, from finding issues to triage, remediation, and validation. Ian also discusses the “lethal trifecta” that makes agents fundamentally risky (untrusted input + sensitive data + exfil path), and why MCP security isn’t just about users and tools, it’s about dangerous tool combinations and rogue servers.Podcast LinksWatch: ⁠⁠⁠⁠⁠https://www.youtube.com/@alexa_griffith⁠⁠⁠⁠⁠Read: ⁠⁠⁠⁠⁠⁠⁠https://alexasinput.substack.com/⁠⁠⁠⁠⁠⁠⁠Listen:⁠⁠⁠ https://creators.spotify.com/pod/profile/alexagriffith/⁠⁠⁠More: ⁠⁠⁠⁠⁠https://linktr.ee/alexagriffith⁠⁠⁠⁠⁠Website: ⁠⁠⁠⁠⁠https://alexagriffith.com/⁠⁠⁠⁠⁠LinkedIn: ⁠⁠⁠⁠⁠https://www.linkedin.com/in/alexa-griffith/⁠⁠⁠⁠Find out more about the guest at:PromptFoo Website: https://www.promptfoo.dev/Github: https://github.com/promptfoo/promptfooIan’s LinkedIn: https://www.linkedin.com/in/ianww/Chapters00:00 Introduction to AI Security Challenges02:06 Funding and Growth of PromptFu06:16 The Genesis of PromptFu11:05 Career Journey and Lessons Learned12:53 Understanding AI Red Teaming17:36 Recent AI Security Vulnerabilities19:46 The Dual Nature of AI in Security21:47 Understanding the Lethal Trifecta in AI Security24:22 Exploring Model Context Protocol (MCP) and Its Security Implications26:22 Common Security Issues in MCP Systems28:17 The Role of Identity and Permissions in AI Security30:00 Practical Implications of Using PromptFoo for Developers31:33 Evaluating Language Models: Challenges and Techniques36:34 The Limitations of Guardrails in AI Security38:25 Best Practices for Engineers in AI Development39:58 Future Trends in AI and Security42:28 Everyday Applications of AI and Language Models

February 6, 2026Episode 345 min

Inside the Future of AI Infrastructure with Marc Austin

Most AI infrastructure today is hitting a breaking point. Marc Austin, CEO of Hedgehog, reveals how open source networking and cloud-native solutions are revolutionizing how enterprises build and operate AI at scale. This episode addresses issues many building AI infrastructure today are facing — expensive proprietary systems, overwhelming complex network configurations, and ways to make on-prem AI infrastructure feel just like the public cloud.We discuss how networking is the hidden bottleneck in scaling GPU clusters and the surprising physics and hardware innovations enabling higher throughput. Marc shares the journey of building Hedgehog, an open source, cloud-native platform designed for AI workloads that bridges the gap between complex hardware and seamless, user-friendly cloud experiences. Marc explains how Hedgehog's software abstracts and automates the networking complexity, making AI infrastructure accessible to enterprises without dedicated networking teams.We break down the future of AI networks, from multi-cloud and hybrid environments to the rise of Neo Clouds and the open source movement transforming enterprise AI infrastructure. If you're a CTO, data scientist, or AI innovator, understanding these network innovations can be your moat. Listen to this episode to see how open source, cloud-native networking, and physical innovation are shaping the AI infrastructure of tomorrow.Podcast LinksWatch: ⁠⁠⁠⁠https://www.youtube.com/@alexa_griffith⁠⁠⁠⁠Read: ⁠⁠⁠⁠⁠⁠https://alexasinput.substack.com/⁠⁠⁠⁠⁠⁠Listen:⁠⁠ https://creators.spotify.com/pod/profile/alexagriffith/⁠⁠More: ⁠⁠⁠⁠https://linktr.ee/alexagriffith⁠⁠⁠⁠Website: ⁠⁠⁠⁠https://alexagriffith.com/⁠⁠⁠⁠LinkedIn: ⁠⁠⁠⁠https://www.linkedin.com/in/alexa-griffith/⁠⁠⁠⁠Find out more about the guest at LinkedIn: https://www.linkedin.com/in/austinmarc/Website: https://hedgehog.cloud/Github: https://github.com/githedgehogChapters00:00 Rethinking AI Infrastructure02:49 The Role of Networking in AI05:54 Marc's Journey to Hedgehog08:46 Lessons from Big Companies11:38 Requirements for AI Networks14:48 Advancements in AI Networking17:33 Future Challenges in AI Infrastructure20:46 Creating a Cloud Experience On-Prem23:32 The Shift to Hybrid Multi-Cloud28:10 Evolving AI Infrastructure and Efficiency30:57 AI Workloads and Network Configurations32:41 Zero Touch Lifecycle Management35:12 Support for Hardware Devices35:45 Networking Paradigms and Vendor Lock-in38:42 The Rise of Neo Clouds41:31 Demand for AI Infrastructure43:57 Open Source and Cloud-Native Networking47:27 Challenges of Building a Networking Startup50:46 Proud Accomplishments at Hedgehog52:41 Future Excitement in AI Inference