<![CDATA[Prompt to Prod]]>Prompt to Prod is a podcast about building with AI. Hosted by Aaron Mog, each episode features practitioners from security and engineering who are shipping real things with AI: the tools they build, the workflows that stuck, and the lessons learned along the way.

Expect honest conversations about build vs. buy, guardrails, data and model choices, and what breaks on the way from a prompt to production. No hype and no vendor pitches, just people who build talking about what actually works.

]]>https://riverside.comRiverside.fm (https://riverside.com)Thu, 15 Oct 2026 02:51:38 GMTMon, 21 Sep 2026 15:42:55 GMT60Prompt to Prod<p>Prompt to Prod is a podcast about building with AI. Hosted by Aaron Mog, each episode features practitioners from security and engineering who are shipping real things with AI: the tools they build, the workflows that stuck, and the lessons learned along the way.</p><p></p><p>Expect honest conversations about build vs. buy, guardrails, data and model choices, and what breaks on the way from a prompt to production. No hype and no vendor pitches, just people who build talking about what actually works.</p>episodicPrompt to Prodaaron@yellowteam.aino<![CDATA[Treat Your LLM Like a Threat Actor | Prompt to Prod - with Assaf Kipnis]]>Assaf Kipnis spent 13 years hunting adversaries at LinkedIn, Facebook, Google and ElevenLabs. Now he builds with AI the same way he hunted them — by assuming the model is a threat actor and writing code for everywhere it might go that you didn't ask for.

Assaf and Aaron get into what it actually takes to build something you'd hand a client: why hallucinations are the wrong fear and agent decisions are the real one, why most of what he produces is code even though AI writes all of it, and the test he runs before reaching for a model at all — could I have done this 10 years ago? He walks through the "Kippy system" he built inside Claude Code to run his projects and his OSINT investigations, the document that blew up in his face when he let an LLM do it directly, and how he strips structure away from the model so it has less room to invent. Plus: why "we'll fight bad AI with good AI" is bananas, why a defender will never be faster than an attacker, why block lists never worked, and why he thinks billing by the hour is dead.

TIMESTAMPS

0:00 Cold Open

0:36 Meet Assaf Kipnis

2:24 What He's Building: The Kippy System

6:47 The Investigation System That Runs OSINT for Him

8:17 Why Personal AI Assistants Still Don't Work

12:09 Hallucinations Aren't the Problem. Decisions Are.

14:59 False Positives and Why He Writes Code Instead

17:04 The Document That Blew Up in His Face

19:27 Taking Structure Away From the LLM

20:59 Using AI to Write the Thing That Does the Job

23:00 AI Is Infrastructure, Marketed to the Wrong People

26:30 Educating Clients Who Think It Just Works

29:45 Treating the LLM as a Threat Actor

31:28 "Automagically" Became "AI"

36:00 Could I Have Done This 10 Years Ago?

38:14 Consulting After AI: Just Give Them the Answer

42:25 Billing by the Hour Is Dead

45:12 What AI Hands the Threat Actors

46:06 The Soapbox: Block Lists Never Worked

49:29 A Defender Will Never Be Faster Than the Attacker

50:19 What to Do Instead: Understand Threat Actors

54:00 How Do You Prove Something That Didn't Happen?

57:42 We Can't Control Content Anymore. Go to Intent.

61:53 Pattern Recognition and the Gut-Feeling Problem

63:15 Where to Find Assaf

ABOUT ASSAF

Assaf Kipnis has spent 13 years across information security, trust and safety, and integrity, mostly doing social media and large-scale investigations. He was an investigator at LinkedIn, then spent about five years at Facebook where he managed and investigated on the e-crime team covering financially motivated actors. He did scaled investigations across Google's products and voice investigations at ElevenLabs. He now consults on AI, threat intelligence, fraud and security, helping companies build deterministic, trustworthy workflows. LinkedIn: https://www.linkedin.com/in/assafkipnis/ X: [CONFIRM HANDLE] Site: [CONFIRM URL]

ABOUT PROMPT TO PROD

Prompt to Prod is a show about building with AI — what actually works, what breaks, and what the best and worst of it looks like in practice. Host Aaron Mog talks to the engineers shipping real systems, not the people talking about them.

#AISecurity #ThreatIntelligence #AgenticAI #AIEngineering #Cybersecurity #TrustAndSafety #BuildingWithAI

]]>ef4981cb-5007-4f73-9f4e-ebd7c97ba10fWed, 14 Oct 2026 18:07:18 GMT<p>Assaf Kipnis spent 13 years hunting adversaries at LinkedIn, Facebook, Google and ElevenLabs. Now he builds with AI the same way he hunted them — by assuming the model is a threat actor and writing code for everywhere it might go that you didn't ask for.</p><p>Assaf and Aaron get into what it actually takes to build something you'd hand a client: why hallucinations are the wrong fear and agent decisions are the real one, why most of what he produces is code even though AI writes all of it, and the test he runs before reaching for a model at all — could I have done this 10 years ago? He walks through the "Kippy system" he built inside Claude Code to run his projects and his OSINT investigations, the document that blew up in his face when he let an LLM do it directly, and how he strips structure away from the model so it has less room to invent. Plus: why "we'll fight bad AI with good AI" is bananas, why a defender will never be faster than an attacker, why block lists never worked, and why he thinks billing by the hour is dead.</p><p>TIMESTAMPS </p><p>0:00 Cold Open </p><p>0:36 Meet Assaf Kipnis </p><p>2:24 What He's Building: The Kippy System </p><p>6:47 The Investigation System That Runs OSINT for Him </p><p>8:17 Why Personal AI Assistants Still Don't Work </p><p>12:09 Hallucinations Aren't the Problem. Decisions Are. </p><p>14:59 False Positives and Why He Writes Code Instead </p><p>17:04 The Document That Blew Up in His Face </p><p>19:27 Taking Structure Away From the LLM </p><p>20:59 Using AI to Write the Thing That Does the Job </p><p>23:00 AI Is Infrastructure, Marketed to the Wrong People </p><p>26:30 Educating Clients Who Think It Just Works </p><p>29:45 Treating the LLM as a Threat Actor </p><p>31:28 "Automagically" Became "AI" </p><p>36:00 Could I Have Done This 10 Years Ago? </p><p>38:14 Consulting After AI: Just Give Them the Answer </p><p>42:25 Billing by the Hour Is Dead </p><p>45:12 What AI Hands the Threat Actors </p><p>46:06 The Soapbox: Block Lists Never Worked </p><p>49:29 A Defender Will Never Be Faster Than the Attacker </p><p>50:19 What to Do Instead: Understand Threat Actors </p><p>54:00 How Do You Prove Something That Didn't Happen? </p><p>57:42 We Can't Control Content Anymore. Go to Intent. </p><p>61:53 Pattern Recognition and the Gut-Feeling Problem </p><p>63:15 Where to Find Assaf</p><p>ABOUT ASSAF </p><p>Assaf Kipnis has spent 13 years across information security, trust and safety, and integrity, mostly doing social media and large-scale investigations. He was an investigator at LinkedIn, then spent about five years at Facebook where he managed and investigated on the e-crime team covering financially motivated actors. He did scaled investigations across Google's products and voice investigations at ElevenLabs. He now consults on AI, threat intelligence, fraud and security, helping companies build deterministic, trustworthy workflows. LinkedIn: <a href="https://www.linkedin.com/in/assafkipnis/" rel="noopener noreferrer nofollow">https://www.linkedin.com/in/assafkipnis/</a> X: [CONFIRM HANDLE] Site: [CONFIRM URL]</p><p>ABOUT PROMPT TO PROD </p><p>Prompt to Prod is a show about building with AI — what actually works, what breaks, and what the best and worst of it looks like in practice. Host Aaron Mog talks to the engineers shipping real systems, not the people talking about them.</p><p>#AISecurity #ThreatIntelligence #AgenticAI #AIEngineering #Cybersecurity #TrustAndSafety #BuildingWithAI</p>no01:03:5217Treat Your LLM Like a Threat Actor | Prompt to Prod - with Assaf Kipnisfull<![CDATA[Don't Be a Meat Proxy for AI | Prompt to Prod - with Alex Beaver]]>Alex Beaver's biggest problem with AI? It runs on GPUs. A security engineer who thinks about the SOC like a factory floor explains why the best AI projects end with less AI running in production, not more.

Alex and Aaron get into what AI actually looks like inside a SOC: using an agentic SOC for fast coverage, then instrumenting it and converting its repeated tool calls back into cheap, deterministic workflows. Alex walks through his build process — writing the types and schemas himself, then letting Claude Code and Codex fill in the implementation — why models still get third-party schemas wrong, inventing a detection schema to do gap analysis, and why build vs. buy comes down to having someone to call at 3 AM. Plus: how worried defenders should be about offensive AI, where to start if you're new to this, and why "meat proxies" between Jira and an AI won't last.

TIMESTAMPS

0:00 Cold open

0:35 Meet Alex Beaver

1:57 What AI Actually Looks Like in the SOC

3:04 Security Operations Through an Industrial Engineering Lens

4:23 Why Deterministic Matters (and the GPU Problem)

7:41 The Spectrum from People to Hot-Path Automation

10:33 Building a SOC from Scratch and the MSP Shake-Up

12:46 From Agentic SOC to Deterministic Workflows

15:18 Why Models Get Your Schemas Wrong

17:51 Write the Types First, Let AI Fill the Blanks

20:38 Rust, TypeScript and Typed Code for AI

22:55 AI for Threat Intel and Detection Engineering

26:20 Inventing a Detection Schema

29:38 Build vs. Buy: The 3 AM Phone Number

31:53 Real Wins: Shipping SOC Capabilities Faster

34:51 How Worried Should Defenders Be About Offensive AI?

39:03 Supply Chain Risk

40:53 What AI Is Bad At

43:16 Where to Start with AI

44:53 Cutting Through the Hype

47:18 Careers, Compression and Meat Proxies

50:28 Fundamentals Keep Coming Up

ABOUT ALEX Alex Beaver is a security engineer at Paramount Skydance, focused mainly on SOAR and detection and response, plus internal tooling. Before that he built an automation platform at a mobile network, and he's long been interested in network orchestration and optimizing incident response. He ran education for one of the largest university cybersecurity clubs in the country right as ChatGPT arrived. He writes about AI in security on his blog, Secure at Scale: [LINK]

ABOUT PROMPT TO PROD Prompt to Prod is a show about building with AI — what actually works, what breaks, and what the best and worst of it looks like in practice. Host Aaron Mog talks to the engineers shipping real systems, not the people talking about them.

#AISecurity #SecurityOperations #AISOC #DetectionEngineering #AIEngineering #Cybersecurity #AgenticAI

]]>2c07d4e8-b69f-489f-a349-3578728cb0efWed, 07 Oct 2026 20:50:25 GMT<p>Alex Beaver's biggest problem with AI? It runs on GPUs. A security engineer who thinks about the SOC like a factory floor explains why the best AI projects end with less AI running in production, not more.</p><p>Alex and Aaron get into what AI actually looks like inside a SOC: using an agentic SOC for fast coverage, then instrumenting it and converting its repeated tool calls back into cheap, deterministic workflows. Alex walks through his build process — writing the types and schemas himself, then letting Claude Code and Codex fill in the implementation — why models still get third-party schemas wrong, inventing a detection schema to do gap analysis, and why build vs. buy comes down to having someone to call at 3 AM. Plus: how worried defenders should be about offensive AI, where to start if you're new to this, and why "meat proxies" between Jira and an AI won't last.</p><p>TIMESTAMPS </p><p>0:00 Cold open </p><p>0:35 Meet Alex Beaver </p><p>1:57 What AI Actually Looks Like in the SOC </p><p>3:04 Security Operations Through an Industrial Engineering Lens </p><p>4:23 Why Deterministic Matters (and the GPU Problem) </p><p>7:41 The Spectrum from People to Hot-Path Automation </p><p>10:33 Building a SOC from Scratch and the MSP Shake-Up </p><p>12:46 From Agentic SOC to Deterministic Workflows </p><p>15:18 Why Models Get Your Schemas Wrong </p><p>17:51 Write the Types First, Let AI Fill the Blanks </p><p>20:38 Rust, TypeScript and Typed Code for AI </p><p>22:55 AI for Threat Intel and Detection Engineering </p><p>26:20 Inventing a Detection Schema </p><p>29:38 Build vs. Buy: The 3 AM Phone Number </p><p>31:53 Real Wins: Shipping SOC Capabilities Faster </p><p>34:51 How Worried Should Defenders Be About Offensive AI? </p><p>39:03 Supply Chain Risk </p><p>40:53 What AI Is Bad At </p><p>43:16 Where to Start with AI </p><p>44:53 Cutting Through the Hype </p><p>47:18 Careers, Compression and Meat Proxies </p><p>50:28 Fundamentals Keep Coming Up</p><p>ABOUT ALEX Alex Beaver is a security engineer at Paramount Skydance, focused mainly on SOAR and detection and response, plus internal tooling. Before that he built an automation platform at a mobile network, and he's long been interested in network orchestration and optimizing incident response. He ran education for one of the largest university cybersecurity clubs in the country right as ChatGPT arrived. He writes about AI in security on his blog, Secure at Scale: [LINK]</p><p>ABOUT PROMPT TO PROD Prompt to Prod is a show about building with AI — what actually works, what breaks, and what the best and worst of it looks like in practice. Host Aaron Mog talks to the engineers shipping real systems, not the people talking about them.</p><p>#AISecurity #SecurityOperations #AISOC #DetectionEngineering #AIEngineering #Cybersecurity #AgenticAI</p>no00:51:3916Don't Be a Meat Proxy for AI | Prompt to Prod - with Alex Beaverfull<![CDATA[The AI Security Control That Never Failed | Prompt to Prod - with Vince Ovando]]>Vince Ovando thinks prompt injection isn't a real vulnerability — and he has millions of inference calls to back up why the industry's favourite defences aren't the ones that work.

In this episode Vince breaks down harness engineering: why the model you pick matters far less than the code you wrap around it, why most problems should be solved with deterministic code before an LLM ever gets involved, and how grammar constrained decoding made every one of millions of test calls safe. We get into positive vs. negative security models, why he tests AI without running the model at all, agent permissions enforced at the token level, and knowledge graphs as a memory layer that lets a local model outperform a frontier one.

TIMESTAMPS

0:00 Cold open

0:09 Introduction to AI Security and Vince's Background

3:10 Harness Engineering: The Key to AI Efficiency

6:09 The Role of Deterministic Code in AI Solutions

9:01 Contrarian Views on AI Security and Traditional Models

12:07 Positive Security Models vs. Traditional Security

15:08 AI Security vs. AI Safety: Understanding the Difference

18:13 The Importance of Not Trusting AI Models

21:02 Implementing Security in AI Architectures

24:02 Grammar Constrained Decoding: A New Approach to Security

27:14 User Permissions and AI: Building Secure Interactions

29:00 Harness Engineering in AI Security

31:04 Dynamic Tool Management for AI Agents

33:40 The Role of Grammars in AI Compliance

36:37 Building Custom AI Harnesses

39:53 Optimizing AI Performance with Knowledge Graphs

46:23 Designing Efficient Knowledge Graphs

49:22 Starting Out with AI: A Practical Playbook

ABOUT VINCE

Vince Ovando came up through security operations, moved into application security and development, and has spent the last two years deep in AI security. His work covers agentic security, red teaming, and code analysis. He built Tantalus (tantalus.io), a free two-round indirect prompt injection arena, and used it to run the millions of inference calls behind his white paper on grammar constrained decoding.

ABOUT PROMPT TO PROD

Prompt to Prod is a show about building with AI — what actually works, what breaks, and what the best and worst of it looks like in practice. Host Aaron Mog talks to the engineers shipping real systems, not the people talking about them.

#AISecurity #PromptInjection #LLM #AIEngineering #Cybersecurity #AgenticAI #KnowledgeGraphs

]]>c7bfc2e7-b8a4-4575-a133-1a1a9a6117a6Wed, 30 Sep 2026 17:57:29 GMT<p>Vince Ovando thinks prompt injection isn't a real vulnerability — and he has millions of inference calls to back up why the industry's favourite defences aren't the ones that work.</p><p>In this episode Vince breaks down harness engineering: why the model you pick matters far less than the code you wrap around it, why most problems should be solved with deterministic code before an LLM ever gets involved, and how grammar constrained decoding made every one of millions of test calls safe. We get into positive vs. negative security models, why he tests AI without running the model at all, agent permissions enforced at the token level, and knowledge graphs as a memory layer that lets a local model outperform a frontier one.</p><p>TIMESTAMPS </p><p>0:00 Cold open </p><p>0:09 Introduction to AI Security and Vince's Background </p><p>3:10 Harness Engineering: The Key to AI Efficiency </p><p>6:09 The Role of Deterministic Code in AI Solutions </p><p>9:01 Contrarian Views on AI Security and Traditional Models </p><p>12:07 Positive Security Models vs. Traditional Security </p><p>15:08 AI Security vs. AI Safety: Understanding the Difference </p><p>18:13 The Importance of Not Trusting AI Models </p><p>21:02 Implementing Security in AI Architectures </p><p>24:02 Grammar Constrained Decoding: A New Approach to Security </p><p>27:14 User Permissions and AI: Building Secure Interactions </p><p>29:00 Harness Engineering in AI Security </p><p>31:04 Dynamic Tool Management for AI Agents </p><p>33:40 The Role of Grammars in AI Compliance </p><p>36:37 Building Custom AI Harnesses </p><p>39:53 Optimizing AI Performance with Knowledge Graphs </p><p>46:23 Designing Efficient Knowledge Graphs </p><p>49:22 Starting Out with AI: A Practical Playbook</p><p>ABOUT VINCE </p><p>Vince Ovando came up through security operations, moved into application security and development, and has spent the last two years deep in AI security. His work covers agentic security, red teaming, and code analysis. He built Tantalus (tantalus.io), a free two-round indirect prompt injection arena, and used it to run the millions of inference calls behind his white paper on grammar constrained decoding.</p><p>ABOUT PROMPT TO PROD </p><p>Prompt to Prod is a show about building with AI — what actually works, what breaks, and what the best and worst of it looks like in practice. Host Aaron Mog talks to the engineers shipping real systems, not the people talking about them.</p><p>#AISecurity #PromptInjection #LLM #AIEngineering #Cybersecurity #AgenticAI #KnowledgeGraphs</p>no00:55:0515The AI Security Control That Never Failed | Prompt to Prod - with Vince Ovandofull<![CDATA[AI Used 17,600 Events. APTs Use 50. | Prompt to Prod - with Justin Tiplitsky]]>Prompt to Prod: an AI agent completed its goal in 17,600 events. The average advanced attacker, by Justin Tiplitsky's count, uses about 50. Justin, director of red team at Adobe, joins Aaron to talk about where AI actually helps a red team, where it's still loud and sloppy, and why handmade tradecraft is what will make AI useful on offense.

We cover what changed when models started keeping context, why Justin doesn't let AI touch his code directly, how AI lowers the bar for zero days, using AI on huge recon datasets, turning red team output into detections defenders can use, the difference between AI red teaming and red teaming with AI, why you can give AI knowledge but not wisdom, and his advice for the next generation of red teamers.

TIMESTAMPS

0:00 Cold Open

0:33 Welcome and meet Justin

2:01 How a red team leader feels about AI right now

3:43 What changed: models that keep context

6:13 Don't AI yourself into a corner: handmade tradecraft

8:52 AI lowers the bar for zero days

14:06 When to use AI on a red team

16:30 Recon and large datasets

19:19 Turning red team output into better detections

22:19 Are AI red team products any good?

25:05 AI red teaming vs. red teaming with AI

30:41 What Justin wants to build next

34:51 Knowledge vs. wisdom

38:41 Advice for the next generation of red teamers

42:00 Where to find Justin

ABOUT JUSTIN

Justin Tiplitsky is director of red team at Adobe, where he built the company's current red team. He previously worked on red teams at Microsoft and Intuit, and writes about strategic red teaming at redteambrief.com.

ABOUT PROMPT TO PROD

Prompt to Prod is a podcast about building with AI, hosted by Aaron Mog. Each episode, practitioners from security and engineering share what they're actually building, what breaks on the way to production, and what they'd do differently.

#PromptToProd #RedTeam #AISecurity #OffensiveSecurity #Cybersecurity #BuildingWithAI #AIAgents

]]>2a4d7430-81f1-4b6e-a39f-930ff5cb5539Tue, 22 Sep 2026 01:21:06 GMT<p>Prompt to Prod: an AI agent completed its goal in 17,600 events. The average advanced attacker, by Justin Tiplitsky's count, uses about 50. Justin, director of red team at Adobe, joins Aaron to talk about where AI actually helps a red team, where it's still loud and sloppy, and why handmade tradecraft is what will make AI useful on offense.</p><p></p><p>We cover what changed when models started keeping context, why Justin doesn't let AI touch his code directly, how AI lowers the bar for zero days, using AI on huge recon datasets, turning red team output into detections defenders can use, the difference between AI red teaming and red teaming with AI, why you can give AI knowledge but not wisdom, and his advice for the next generation of red teamers.</p><p></p><p>TIMESTAMPS</p><p>0:00 Cold Open</p><p>0:33 Welcome and meet Justin</p><p>2:01 How a red team leader feels about AI right now</p><p>3:43 What changed: models that keep context</p><p>6:13 Don't AI yourself into a corner: handmade tradecraft</p><p>8:52 AI lowers the bar for zero days</p><p>14:06 When to use AI on a red team</p><p>16:30 Recon and large datasets</p><p>19:19 Turning red team output into better detections</p><p>22:19 Are AI red team products any good?</p><p>25:05 AI red teaming vs. red teaming with AI</p><p>30:41 What Justin wants to build next</p><p>34:51 Knowledge vs. wisdom</p><p>38:41 Advice for the next generation of red teamers</p><p>42:00 Where to find Justin</p><p></p><p>ABOUT JUSTIN</p><p>Justin Tiplitsky is director of red team at Adobe, where he built the company's current red team. He previously worked on red teams at Microsoft and Intuit, and writes about strategic red teaming at <a rel="noopener noreferrer nofollow" href="http://redteambrief.com" target="_blank">redteambrief.com</a>.</p><p></p><p>ABOUT PROMPT TO PROD</p><p>Prompt to Prod is a podcast about building with AI, hosted by Aaron Mog. Each episode, practitioners from security and engineering share what they're actually building, what breaks on the way to production, and what they'd do differently.</p><p></p><p>#PromptToProd #RedTeam #AISecurity #OffensiveSecurity #Cybersecurity #BuildingWithAI #AIAgents</p>no00:42:5314AI Used 17,600 Events. APTs Use 50. | Prompt to Prod - with Justin Tiplitskyfull<![CDATA[He Hasn't Hand-Written Code in 18 Months | Prompt to Prod - with Wyatt Roersma]]>Prompt to Prod: Wyatt Roersma hasn't hand-written code in over 18 months. His day is code review: AI builds in a sandbox, tests its own work, and attaches a GIF of the finished UI to every pull request. Wyatt and Aaron get into how a longtime security practitioner builds with AI now, and where it still falls short.

We cover build vs. buy for security teams, why Wyatt thinks everyone should build with AI, how he turns infrastructure into code so AI stops guessing, running Codex and Claude side by side with parallel agents, MCP, fine-tuning small models for YARA and MITRE ATT&CK mapping, why private data and subsidized tokens push toward private models, where models fall short on intent and access, and what a blue-team model should look like.

TIMESTAMPS

0:00 Cold Open

0:45 Welcome and meet Wyatt

1:17 Wyatt's background: memory forensics to Rust

3:05 What he's building: a security coverage platform

8:06 How AI fits in: no hand-written code in 18 months

9:24 PRs that come with videos

12:59 Build vs. buy for security teams

14:24 Why everybody should build with AI

19:37 Where AI struggles: infrastructure

21:04 Wyatt's stack: Codex, Claude, and parallel agents

24:19 Infrastructure as code so AI stops guessing

29:05 MCP and threat intelligence services

30:44 Private models and fine-tuning small models

37:45 Private data, subsidized tokens, and the price cliff

39:40 Where models fall short: intent and access

46:37 What Wyatt wants to build next: a blue-team model

51:38 Model vs. harness: what security AI is missing

55:43 Where to find Wyatt

ABOUT WYATT

Wyatt Roersma got his start in memory forensics, contributing to Volatility around 2012. He worked at Target after the breach, building a custom Cuckoo sandbox for the threat detection operations team, later worked as a cyber range developer, and has run the GrrCON digital forensics and incident response challenge since 2013. He now works at Countermeasure Security, building a coverage platform, and publishes open-source projects, many in Rust, under Threatflux.

Catch Wyatt's DFIR challenge at GrrCON in Grand Rapids, September 24–25.

ABOUT PROMPT TO PROD

Prompt to Prod is a podcast about building with AI, hosted by Aaron Mog. Each episode, practitioners from security and engineering share what they're actually building, what breaks on the way to production, and what they'd do differently.

#PromptToProd #AISecurity #BuildingWithAI #DetectionEngineering #Rust #ClaudeCode #AIAgents

]]>73f3b418-714a-45f7-8eb3-8ca7aa8cf29eMon, 21 Sep 2026 22:53:18 GMT<p>Prompt to Prod: Wyatt Roersma hasn't hand-written code in over 18 months. His day is code review: AI builds in a sandbox, tests its own work, and attaches a GIF of the finished UI to every pull request. Wyatt and Aaron get into how a longtime security practitioner builds with AI now, and where it still falls short.</p><p></p><p>We cover build vs. buy for security teams, why Wyatt thinks everyone should build with AI, how he turns infrastructure into code so AI stops guessing, running Codex and Claude side by side with parallel agents, MCP, fine-tuning small models for YARA and MITRE ATT&amp;CK mapping, why private data and subsidized tokens push toward private models, where models fall short on intent and access, and what a blue-team model should look like.</p><p></p><p>TIMESTAMPS</p><p>0:00 Cold Open</p><p>0:45 Welcome and meet Wyatt</p><p>1:17 Wyatt's background: memory forensics to Rust</p><p>3:05 What he's building: a security coverage platform</p><p>8:06 How AI fits in: no hand-written code in 18 months</p><p>9:24 PRs that come with videos</p><p>12:59 Build vs. buy for security teams</p><p>14:24 Why everybody should build with AI</p><p>19:37 Where AI struggles: infrastructure</p><p>21:04 Wyatt's stack: Codex, Claude, and parallel agents</p><p>24:19 Infrastructure as code so AI stops guessing</p><p>29:05 MCP and threat intelligence services</p><p>30:44 Private models and fine-tuning small models</p><p>37:45 Private data, subsidized tokens, and the price cliff</p><p>39:40 Where models fall short: intent and access</p><p>46:37 What Wyatt wants to build next: a blue-team model</p><p>51:38 Model vs. harness: what security AI is missing</p><p>55:43 Where to find Wyatt</p><p></p><p>ABOUT WYATT</p><p>Wyatt Roersma got his start in memory forensics, contributing to Volatility around 2012. He worked at Target after the breach, building a custom Cuckoo sandbox for the threat detection operations team, later worked as a cyber range developer, and has run the GrrCON digital forensics and incident response challenge since 2013. He now works at Countermeasure Security, building a coverage platform, and publishes open-source projects, many in Rust, under Threatflux.</p><p>Catch Wyatt's DFIR challenge at GrrCON in Grand Rapids, September 24–25.</p><p></p><p>ABOUT PROMPT TO PROD</p><p>Prompt to Prod is a podcast about building with AI, hosted by Aaron Mog. Each episode, practitioners from security and engineering share what they're actually building, what breaks on the way to production, and what they'd do differently.</p><p></p><p>#PromptToProd #AISecurity #BuildingWithAI #DetectionEngineering #Rust #ClaudeCode #AIAgents</p>no00:57:2513He Hasn't Hand-Written Code in 18 Months | Prompt to Prod - with Wyatt Roersmafull<![CDATA[AI Said Everything Was on Fire | Prompt to Prod - with Pablo Vidal]]>Prompt to Prod: when Pablo's team first pointed an AI analyst at their Okta logs, it treated a Monday holiday like an incident. Everything was on fire. Pablo and Aaron get into what it really takes to build with AI as a security engineer: giving models context, making sure your team can own what you build, and putting humans only where the risk demands it.

We cover why everyone can build now and what guardrails that requires, which data you should never send to a model, build vs. buy and why you shouldn't build in a cave, what to do when AI generates 5,000 PRs a day, risk-classifying PRs so humans review only what matters, alert fatigue and the holiday-calendar fix, baselining and timelines in detection, cost and cheaper models as token prices catch up, why he isn't more worried about the AI threat wave, and why AI is still bad at writing.

TIMESTAMPS

0:00 Cold Open

0:42 Welcome and meet Pablo

1:11 Pablo's background: from Cisco certs to AI security

2:35 How much of his day is AI

4:31 Enablement: everyone can build now

7:54 Guardrails, not gatekeeping

10:03 Sending sensitive data to models

15:17 Build vs. buy: don't build in a cave

19:12 Velocity vs. quality vs. security

22:57 Guardrails agents can read: risk-classifying PRs

26:46 Alert fatigue: how much can AI handle

28:20 Give AI context: the holiday-calendar false alarms

30:59 Baselining, timelines, and non-deterministic AI

36:05 Cost, local models, and deterministic scripts

41:02 The AI threat wave

45:22 What AI is bad at: writing

49:23 What Pablo is building next

50:59 Community and where to find Pablo

ABOUT PABLO

Pablo has spent about 11 years in security. He started with Cisco certifications, moved into security generalist work and cloud security on AWS, worked on cloud security and security operations at Rippling, then spent a year at Grow Therapy in an individual-contributor role where he went deep on AI. He recently joined Life360 as a senior staff engineer focused on AI security.

ABOUT PROMPT TO PROD

Prompt to Prod is a podcast about building with AI, hosted by Aaron Mog. Each episode, practitioners from security and engineering share what they're actually building, what breaks on the way to production, and what they'd do differently.

#PromptToProd #AISecurity #BuildingWithAI #SecurityEngineering #DetectionEngineering #ClaudeCode #AIAgents

]]>92e59a29-f26b-41fc-9fcc-22cb1c185346Mon, 21 Sep 2026 22:12:41 GMT<p>Prompt to Prod: when Pablo's team first pointed an AI analyst at their Okta logs, it treated a Monday holiday like an incident. Everything was on fire. Pablo and Aaron get into what it really takes to build with AI as a security engineer: giving models context, making sure your team can own what you build, and putting humans only where the risk demands it.</p><p></p><p>We cover why everyone can build now and what guardrails that requires, which data you should never send to a model, build vs. buy and why you shouldn't build in a cave, what to do when AI generates 5,000 PRs a day, risk-classifying PRs so humans review only what matters, alert fatigue and the holiday-calendar fix, baselining and timelines in detection, cost and cheaper models as token prices catch up, why he isn't more worried about the AI threat wave, and why AI is still bad at writing.</p><p></p><p>TIMESTAMPS</p><p>0:00 Cold Open</p><p>0:42 Welcome and meet Pablo</p><p>1:11 Pablo's background: from Cisco certs to AI security</p><p>2:35 How much of his day is AI</p><p>4:31 Enablement: everyone can build now</p><p>7:54 Guardrails, not gatekeeping</p><p>10:03 Sending sensitive data to models</p><p>15:17 Build vs. buy: don't build in a cave</p><p>19:12 Velocity vs. quality vs. security</p><p>22:57 Guardrails agents can read: risk-classifying PRs</p><p>26:46 Alert fatigue: how much can AI handle</p><p>28:20 Give AI context: the holiday-calendar false alarms</p><p>30:59 Baselining, timelines, and non-deterministic AI</p><p>36:05 Cost, local models, and deterministic scripts</p><p>41:02 The AI threat wave</p><p>45:22 What AI is bad at: writing</p><p>49:23 What Pablo is building next</p><p>50:59 Community and where to find Pablo</p><p></p><p>ABOUT PABLO</p><p>Pablo has spent about 11 years in security. He started with Cisco certifications, moved into security generalist work and cloud security on AWS, worked on cloud security and security operations at Rippling, then spent a year at Grow Therapy in an individual-contributor role where he went deep on AI. He recently joined Life360 as a senior staff engineer focused on AI security.</p><p></p><p>ABOUT PROMPT TO PROD</p><p>Prompt to Prod is a podcast about building with AI, hosted by Aaron Mog. Each episode, practitioners from security and engineering share what they're actually building, what breaks on the way to production, and what they'd do differently.</p><p></p><p>#PromptToProd #AISecurity #BuildingWithAI #SecurityEngineering #DetectionEngineering #ClaudeCode #AIAgents</p>no00:53:1112AI Said Everything Was on Fire | Prompt to Prod - with Pablo Vidalfull<![CDATA[ One AI Fix Found Four More Issues | Prompt to Prod - with Drew Pilarski ]]>One fix, four new issues.

Security engineer Drew Pilarski went from nervous about AI, to excited, to a little tired, and he's honest about all three. He and Aaron get into what it's really like to build with AI every day: what it makes possible, the tooling you now have to keep alive, and why one AI fix can turn into four new issues.

We cover build vs. buy when you're not a traditional developer, the boring corporate work AI does best, a LangGraph port that spiraled into "endless work," Aaron's loop of building with one model and validating with another, running parallel agents, frontier vs. private models as token costs catch up, where AI is already paying off in detection engineering, and what happens when attackers have models with no guardrails.

TIMESTAMPS

0:00 Cold Open

0:34 Welcome and meet Drew

2:41 From nervous to excited to tired

7:00 Build vs. buy: keeping the tooling alive

7:43 The boring work AI does best

10:03 Recall Radar and the joy of building

13:18 Where AI let him down: LangGraph and the Ouroboros loop

15:22 Fable builds, Astra validates

18:22 Claude Code, Herdr, and the 10x engineer question

21:36 Models over harnesses: frontier vs. private models

32:33 Where AI wins in detection engineering

37:40 What Drew wants to build next

40:02 AI asymmetry: attackers with no guardrails

45:01 Ransomware chatbots and follow the incentives

47:57 Where to find Drew

ABOUT DREW

Drew Pilarski has spent 11 years in information security. He started as a self-described "tool monkey" doing generalist work, and for the last five or six years has focused on threat detection, including cloud threat detection.

ABOUT PROMPT TO PROD

Prompt to Prod is a podcast about building with AI, hosted by Aaron Mog. Each episode, practitioners from security and engineering share what they're actually building, what breaks on the way to production, and what they'd do differently.

#PromptToProd #BuildingWithAI #AIAgents #ClaudeCode #Cybersecurity #DetectionEngineering #SecurityEngineering

]]>69ff5916-35ca-4dcf-844e-86f8b7a04af9Mon, 21 Sep 2026 16:35:32 GMT<p>One fix, four new issues. </p><p></p><p>Security engineer Drew Pilarski went from nervous about AI, to excited, to a little tired, and he's honest about all three. He and Aaron get into what it's really like to build with AI every day: what it makes possible, the tooling you now have to keep alive, and why one AI fix can turn into four new issues. </p><p></p><p>We cover build vs. buy when you're not a traditional developer, the boring corporate work AI does best, a LangGraph port that spiraled into "endless work," Aaron's loop of building with one model and validating with another, running parallel agents, frontier vs. private models as token costs catch up, where AI is already paying off in detection engineering, and what happens when attackers have models with no guardrails. </p><p></p><p>TIMESTAMPS </p><p>0:00 Cold Open </p><p>0:34 Welcome and meet Drew </p><p>2:41 From nervous to excited to tired </p><p>7:00 Build vs. buy: keeping the tooling alive </p><p>7:43 The boring work AI does best </p><p>10:03 Recall Radar and the joy of building </p><p>13:18 Where AI let him down: LangGraph and the Ouroboros loop </p><p>15:22 Fable builds, Astra validates </p><p>18:22 Claude Code, Herdr, and the 10x engineer question </p><p>21:36 Models over harnesses: frontier vs. private models </p><p>32:33 Where AI wins in detection engineering </p><p>37:40 What Drew wants to build next </p><p>40:02 AI asymmetry: attackers with no guardrails </p><p>45:01 Ransomware chatbots and follow the incentives </p><p>47:57 Where to find Drew </p><p></p><p>ABOUT DREW </p><p>Drew Pilarski has spent 11 years in information security. He started as a self-described "tool monkey" doing generalist work, and for the last five or six years has focused on threat detection, including cloud threat detection. </p><p></p><p>ABOUT PROMPT TO PROD </p><p>Prompt to Prod is a podcast about building with AI, hosted by Aaron Mog. Each episode, practitioners from security and engineering share what they're actually building, what breaks on the way to production, and what they'd do differently. </p><p></p><p>#PromptToProd #BuildingWithAI #AIAgents #ClaudeCode #Cybersecurity #DetectionEngineering #SecurityEngineering</p>no00:48:5511 One AI Fix Found Four More Issues | Prompt to Prod - with Drew Pilarski full