About this episode
?GUEST?Allie Mellen, Principal Analyst, Forrester | On LinkedIn: https://www.linkedin.com/in/hackerxbella/?HOST?Host: Sean Martin, Co-Founder at ITSPmagazine and Host of Redefining CyberSecurity Podcast | On ITSPmagazine: https://www.itspmagazine.com/sean-martin?EPISODE NOTES?In this episode, Allie Mellen, Principal Analyst on the Security and Risk Team at Forrester, joins Sean Martin to discuss the latest results from the MITRE ATT&CK Ingenuity Evaluations and what they reveal about detection and response technologies.The Role of MITRE ATT&CK EvaluationsMITRE ATT&CK is a widely adopted framework that maps out the tactics, techniques, and procedures (TTPs) used by threat actors. Security vendors use it to improve detection capabilities, and organizations rely on it to assess their security posture. The MITRE Ingenuity Evaluations test how different security tools detect and respond to simulated attacks, helping organizations understand their strengths and gaps.Mellen emphasizes that MITRE’s evaluations do not assign scores or rank vendors, which allows security leaders to focus on analyzing performance rather than chasing a “winner.” Instead, organizations must assess raw data to determine how well a tool aligns with their needs.Alert Volume and the Cost of Security DataOne key insight from this year’s evaluation is the significant variation in alert volume among vendors. Some solutions generate thousands of alerts for a single attack scenario, while others consolidate related activity into just a handful of actionable incidents. Mellen notes that excessive alerting contributes to analyst burnout and operational inefficiencies, making alert volume a critical metric to assess.Forrester’s analysis includes a cost calculator that estimates the financial impact of alert ingestion into a SIEM. The results highlight how certain vendors create a massive data burden, leading to increased costs for organizations trying to balance security effectiveness with budget constraints.The Shift Toward Detection and Response EngineeringMellen stresses the importance of detection engineering, where security teams take a structured approach to developing and maintaining high-quality detection rules. Instead of passively consuming vendor-generated alerts, teams must actively refine and tune detections to align with real threats while minimizing noise.Detection and response should also be tightly integrated. Forrester’s research advocates linking every detection to a corresponding response playbook. By automating these processes through security orchestration, automation, and response (SOAR) solutions, teams can accelerate investigations and reduce manual workl
