Adventures in DevOps

Hosted by Will Button, Warren Parad

Technology Careers EducationInterviews guests

Website RSS feed

Episodes

285

Latest episode

Aug 2026

Language

About the show

Join us in listening to the experienced experts discuss cutting edge challenges in the world of DevOps. From applying the mindset at your company, to career growth and leadership challenges within engineering teams, and avoiding the common antipatterns. Every episode you'll meet a new industry veteran guest with their own unique story.

Listen to episodes

60 recent

August 7, 2026Episode 28242 min

Why does anyone use Crossplane?

Share Episode Pushkar Gopalakrishna, Senior Staff Software Engineer at Snap, previously Cruise and AWS, joins to explore why engineering organizations pivot away from Terraform and HCL toward Kubernetes-native tools even when they might not be better. We unpack how developer friction, copy-pasted control structures, and misaligned organizational incentives create massive tech debt—often forcing SREs to manually update infrastructure repositories for compliance, resulting in broken pipelines and severe operational friction. We debate over the mechanics of Crossplane, detailing how its continuous reconciliation loop and Custom Resource Definitions (CRDs) allow teams to express cloud infrastructure as YAML alongside their application manifests at potentially the cost of async validation. Pushkar pulls back the curtain on how Cruise managed infrastructure at scale using a custom internal platform called 'Juno' to bootstrap GCP projects, repositories, and permissions, while leveraging Crossplane for application-level resources. We also dive into the dangers of using CI tools for continuous deployment, detailing a terrifying incident where a pipeline bug accidentally marked three production Kubernetes namespaces for deletion, and how moving to ArgoCD and Argo Rollouts helped prevent future outages for autonomous vehicles. Finally, we touch on the realities of non-production environment isolation, testing against live APIs, and why platform teams must balance providing a seamless developer experience without stripping away developer accountability. 💡 Notable Links: Crossplane Podcast Guest Request for Principal Engineer — What work are you doing? Amazon Multi-level fullyment center for drones ✨ Episode: Terraform vs OpenTofu 🎯 Picks: Warren - Books: The Murderbot Diaries Pushkar - DJI mini drone

July 31, 2026Episode 28141 min

Building Observability for the Innovators

Share Episode Co-Founder of Grafana Labs, Anthony Woods, joins to share his perspective on how open source solutions are thriving despite the deluge of pull requests being thrown at them through their open source repositories. Most importantly, he outlines how observability is no longer being done by users looking directly at dashboards. The data rarely makes to LLMs or automation, it rarely makes sense even to humans looking at them, without the context. The context is critical component, and having a model that was built on the semantic concepts relevant to your use cases. And of course we can't stay away from asking on the record the current state of security of open source repositories from a vendor side. Given how their was an inevitable incident with some of the Grafana open source repos, we dig in to figure out how they are dealing with the real world impacts of malware being spread throughout the ecosystem. 💡 Notable Links: Coinbase's ridiculous spend on observability Book: Crossing the Chasm Silicon Valley Show: Hot Dog or Not Hot Dog Shai Hulud — Grafana open source compromise Podcast Guest Request for Package Manager Security Expert ✨ Episode: Productivity 🎯 Picks: Warren - Best Starfleet Captain: Pike Anthony - The Bitter Lesson

July 24, 2026Episode 2801 hr 9 min

When knowledge is free but the infrastructure isn't

Share Episode As it turns out, the entire artificial intelligence boom is essentially running on Wikipedia's free labor, but while knowledge is free, physical server infrastructure definitely is not. We sit down with Moriel Schottlender, Principal Systems Software Engineer at the Wikimedia Foundation, to dissect how public systems survive an endless onslaught of high-volume AI scrapers and aggressive crawlers. Because 65% of the resource-heavy requests originate from automated bots, we explore how Wikimedia navigates this traffic without blocking legitimate users. We skip the approaches of IP-banning which doesn't work in practice and discuss actual mature architectural strategies, by focusing on the users' needs. From structured database dumps and high-volume enterprise APIs to rate-limiting and CDN caching trade-offs. It's a mind-bogglingly complex ecosystem of open-source, a 25-year-old PHP monolith supporting over 900 distinct site instances across 300 languages and 11 unique projects. It's an immense engineering challenge to modernize infrastructure while serving 250,000 active volunteer editors who build custom workflows via Toolforge—Wikimedia's internal, open-source mini-AWS. Finally, we have to tackle the philosophical divide between artificial statistical models and human creativity. Because LLMs are trained to predict the statistical mean, they inherently miss the edge cases where real human value, internationalization, and accessibility actually reside. And even if they did, we managed to squeeze out every last bit of AI creativity that early models had until what we are actually left with is the most boring result. We also commiserate over the gratuitous low-quality AI pull requests flooding open-source repositories, drawing parallels to the chaotic Hacktoberfest spam of years past. 💡 Notable Links: Frodo project Impact of crawlers on Mediawiki's infrastructure Book: The Platform Revolution Moriel's LLM experiments ✨ Episode: 🎯 Picks: Warren - Video: Are all flags Drawable in PowerPoint Moriel - Audiobook: Dungeon Crawler Carl

July 10, 2026Episode 2791 hr 0 min

Technically We Have Code Reviews and the LLM Semantic Layer

Share Episode We are joined this week by Mark Hay, CTO and co-founder of TextQL and former lead of Text Classification Infrastructure at Meta, to uncover the hidden complexities behind massive-scale machine learning. Mark explains why the most crucial features for identifying abusive behavior, like drug dealers or scammers on Facebook and Instagram, rarely rely on the content itself but instead analyze the underlying behavioral graphs, such as abnormal friend requests or messaging patterns. Of course we review the adversarial nature of spam detection, where bad actors constantly evolve from simple regex evasion to embedding messages inside images or even utilizing pure symbolic communication, like comparing different sized cucumber emojis to evade text filters. That requires diving into the evolution of database querying and the rise of the semantic layer. Mark unpacks why relying on raw LLMs to write complex SQL is a recipe for hallucinations, and how implementing a "correct by construction" semantic layer guarantees structurally sound queries by restricting outputs to a strictly defined configuration. However, this rigid structure fundamentally stifles the creative flexibility of LLMs. Lastly, we can't avoid exploring the tension between these approaches and how new tools aim to bridge the gap by dynamically balancing raw SQL generation with structured ontological constraints, providing rapid time-to-value for analytical workflows. Finally, we discuss the controversial philosophical shift occurring within software engineering, particularly the tension between the "Don't Repeat Yourself" principle and "Locality of Behavior". 💡 Notable Links: ✨ Episode: Semantic Search ✨ Episode: Formal Verification ✨ Episode: Subjective Model Embeddings 🎯 Picks: Warren - Article: I Left Port 22 Open on the Internet for 54 Days Mark - Hotel Room Exercise: Burpies

June 26, 2026Episode 27854 min

Who Needs Testers Anyway?

Share Episode We sit down with Itacama CEO Pia Wiedermayer to discuss the absurdity of siloed QA, the disaster of AI-generated API tests, and why developers hate the word "quality." This time we are asking the age-old question: Who needs testers anyway? Pia and Warren discuss how to dismantle the toxic culture of isolated quality assurance. We explore how the ghosts of waterfall development still haunt modern teams, creating silos where developers blindly throw unverified code over the wall and expect a separate QA department to magically inject quality. Included is the inevitable discussion on the psychological safety of hiding behind narrow job titles and why refusing to take collective ownership of a product is a guaranteed recipe for architectural failure. Of course we can't adoiv commenting on the terrifying reality of replacing human intuition with automated hype. Pia shares a case study of a scale-up that aggressively pivoted to "full steam AI development," intentionally excluding both their Product Owner and QA from the entire experiment. Predictably, it did not end well, but we were able to laugh at the painful irony that an AI-accelerated project scheduled for four weeks ended up taking eight weeks, proving that simply generating code without human oversight just creates more sophisticated bottlenecks. 🎯 Picks: Warren - Wason Selection Task on The Rest Is Science Pia - Book: The Culture Map

June 19, 2026Episode 27752 min

You Wouldn't Implement A Database

Share Episode We talk with Ragic CEO Jeff Kuo about Semantic Web origins, dodging DDoS attacks, and the absolute horror of a database that randomly deletes its own files. He revisits how a 25-year-old master's thesis on the Semantic Web evolved into a massive spreadsheet-driven database builder. It's the one better Airtable alternative. Rather than forcing non-technical users into complex two-layer SQL architectures, Ragic utilizes a highly flexible, graph-based data model. Achieving this performance meant abandoning traditional ORMs to build a custom graph indexing engine on top of Berkeley DB, a key-value store. This custom implementation came with brutal growing pains, including a terrifying bug that would randomly delete the wrong data files. To survive, Ragic's team shares with us just exactly how they had to hijack the internal implementation to avoid these sorts of problems. When we get down to it, we review how they dealt with critical DDoS against their cloud providers, how they performed a cloud migration in just one weekend, and how they manage thousands of tenants on shared infrastructure. 💡 Notable Links: Berkeley DB ✨ Episode: Differences between single and multi-tenant architectures 🎯 Picks: Warren - DevOps Days conferences Jeff - Taroko National Park Taiwan

June 12, 2026Episode 27649 min

What If Tools Are Not Expensive To Build

Share Episode Developers spend more than 50% of their time reading code, making it the single largest expense in software engineering. Despite this massive cost, the industry rarely discusses or optimizes how we read code. So we've brought in Tudor Girba, CEO at Feenk to help us rethink, just how software engineering should be done. Instead of relying on manual reading and generic text editors, teams must shift toward building deterministic, contextual tools to directly extract information and answer questions about their systems. The suggested solution? Contextual and composable micro-tools writen by everyone focused on exposing just the right information at the right time. This creates the opportunity for structural interrogation of your solution. And how many tools should we? We'll if one example of tool is testing, and 50% or more of your code can be tests, imagine what percentage of your software should be actually production related! Most importantly, generic tools fall short, but where can we find how to build the right tools, listen in to find out.... 💡 Notable Links: ✨ Episode: IDE & Copilot & Critical Thinking Book: Moldable software development Wardley Map Guest Request: Formal Verification 🎯 Picks: Warren - The real stuff: Underwood Ranches Sriracha Tudor - The beaches of Normandy

June 5, 2026Episode 2751 hr 5 min

DR: Staying resilient in the cloud

Share Episode Welcome back to another hopefully, relief from architectural existential dread. This week, we've pulled in Seth Eliot from Arpio , (Ar-Pi-O, RPO, get it?), to dive headfirst into the beautiful, deeply expensive illusion that migrating your legacy infrastructure to a major hyperscaler magically grants it instant immortality. It doesn't . We break down the shared responsibility model for resilience, which was conveniently cribbed straight from the security model, and analyze how the foundational promise of automated fault isolation boundaries routinely crumbles. From cloud providers sticking multiple "independent" availability zones inside the exact same physical building, to multi-AZ cascading anomalies, to regional power grid failures, it's clear your provider's abstractions aren't nearly as resilient as their marketing slides suggest. Discussed within is the "Thundering Herd" phenomenon, that can't be ignored even when the failover clusters are designed correctly. From cross-organization KMS re-encryption loops to the horror of fragmented application logs across CloudFront edge regions, at the end of the day, true resilience isn't achieved by forcing your engineering team to implement features, it's about architecting your baseline, confidentiality for the inevitability of production burning to the ground. 💡 Notable Links: ✨ Episode: Eat your security vegetables ✨ Episode: Matt vibecodes ✨ Episode: on DNS and isolation 🎯 Picks: Warren - Book: Moldable software development Seth - Lockpick set

May 29, 2026Episode 27459 min

Eat your security vegetables

Share Episode This week's adventure tackles the absolute absurdity of modern enterprise infrastructure, where a single company can easily find itself running multiple different CI/CD platforms due to unchecked mergers and acquisitions. We've brought in Chris Farris , AWS Security Hero and consults with companies via Securosis . And dig deep to find the security cracks and philosophize about the real world impacts of tech debt in the AI age. Management rarely prioritizes standardization, leaving security teams to defend a chaotic swamp of mixed cloud providers, GitHub repositories, and nostalgic on-prem Bitbucket instances. We define this accumulated technical debt not as some abstract concept, but as literal potholes on the infrastructure Autobahn—annoying speed bumps that permanently damage velocity and set organizations up for an inevitable disaster. We contrast this with the evolution from old-school sysadmins cutting their fingers on rack screws to modern engineers spinning up entire architectures with a few lines of code, noting that the ease of deployment has far outpaced our willingness to clean up our own mess. The crisis is only accelerating now that the cost of writing code (but not having to maintain it) is rapidly approaching zero. While letting an AI agent autonomously build a website or manipulate an AWS sandbox over a single Saturday afternoon sounds magical, it creates a terrifying volume of unreviewed, context-devoid software. Compounding this systemic frailty, massive cloud provider layoffs mean the crucial institutional memory and human operational experience required to survive are walking right out the door. We expose the fundamental flaw of modern agentic tooling: they completely lack fine-grained access control, operating on a dangerous all-or-nothing identity model. Until autonomous agents are engineered with actual conscience, consequence, and common sense, security teams will continue fighting a losing battle against a digital supply chain. 💡 Notable Links: Chris' Article on AI Tech Debt Breaking Open Source: Malus - Article Vercel Security Incident ✨ Episode: 🎯 Picks: Warren - Rick & Morty S02 + S03 Chris - Risky Business: The latest actually good cybersecurity news

May 15, 2026Episode 27340 min

Automatic Data Pipelining: One More Turtle Ahead

Share Episode We grabbed Donald Nguyen, co-founder and CTO at Corvic, to discuss the absurd complexities of enterprise data and multimodal inference. We explore how organizations habitually hoard mountains of useless, "dead" data just out of the sheer fantascy that someone might ask for it later. We highlight the fundamental disconnect where data collectors using tools like Airbyte and Kafka speak a completely different language than the business consumers analyzing it in Excel. True scale isn't just about managing petabytes; it's the absolute nightmare of extracting subjective business meaning from flat PDFs and invoices. In the deep-end of vector embeddings, we're challenging translating data into a different semantic universe requires imposing a heavy business bias. Auditors and artists will view the exact same invoice completely differently, meaning your embedding model selection is incredibly subjective to the business context. The industry's desperate search for actual AI success stories beyond basic workflow automation is still ongoing as we laugh—and cry—at the reality that companies are likely budgeting 50% of an engineer's salary for LLM token usage, effectively enabling product managers to burn cash on infinite loops to generate prototype code. Reasonable or unreasonable? And lastly, we tackle the existential dread of securing autonomous AI agents. Because fine-grained access control for agent actions is basically an unsolved fantasy, we must treat their execution environments as entirely untrusted, relying on rigid sandboxes like AWS Firecracker VMs. Prompt injection attacks are an inevitable flaw of the transformer architecture, and the industry's best defense mechanism seems to be wrapping models inside of other models to validate the outputs. It is quite literally turtles all the way down, and the winner of enterprise security is simply the organization that manages to put one more turtle ahead of the attackers. 💡 Notable Links: Kuuk Thaayorre Aboriginal Tribe - Cardinal Directions ✨ Episode: Generating automatic integrations at scale 🎯 Picks: Warren - Dr. NEMO: Clockwise circle pit Donald - Book: InvestiGators

Is this your show?

Claim this listing to keep it up to date, reach guests who want to pitch you, and manage bookings with Guestify.

Claim this listing

More Technology podcasts

Earth Notes

Earth Notes / Damon Hart-Davis

Technology

The Civil Engineering Podcast

Anthony Fasano, PE, AEC PM, F. ASCE and James Taylor, JR., P.E.

BusinessTechnologyInterviews guests