Building a Daily Threat Simulation Tool with Todd Beebe

Last updated at Sun, 31 Dec 2023 17:39:56 GMT

In the latest episode of our podcast, Security Nation, we sat down to talk with Todd Beebe about the automated threat simulation system that he built for his current employer. Todd is the information security officer for an oil and gas company in Texas. He’s been working in information security since the early ‘90s, and cut his teeth doing penetration tests on companies through dial-up modems. He was also part of the team that built the first web application firewall to successfully defend servers from attacks like Code Red and Nimda.

Needless to say, he has a lot of experience in vulnerability management and security. Here is our recap of the podcast:

Using the MITRE ATT&CK framework to build a new tool

Todd was initially brought onto his company to build a security program. His first initiative was to determine where he had control over his assets and what attacks were (or weren’t) being detected. Todd and his new team leveraged the MITRE ATT&CK framework to build a daily threat simulation tool.

The MITRE ATT&CK framework breaks an attack down into categories or phases. For instance, the attack may start with initial access and move to gaining access, privilege escalation, lateral movement, and then data exfiltration. This provides a much wider view into the similarities in what actions attackers use to break in, regardless of who they are or where they come from. With this kind of visibility, Todd and his team were able to mimic what attackers actually do and see if their defenses protected against them.

In order to build the simulation tool, Todd and his team used existing tools that were built by the security community as much as possible. Many different security groups were making tools on GitHub that did at least a subset of MITRE activity, and the team could plug those into their environment. These tools helped fill in gaps that off-the-shelf solutions couldn’t address. The resulting process is a daily threat detection tool that the team can use specifically within their org.

Now, they use these custom tools to validate that both their detection and preventive controls are working in the manner they are supposed to, every day. This daily testing shows the gaps in what their solution is detecting and blocking, allowing them to close those gaps with better detection rules in their security system. They repeat this gap-finding test with as many tools as possible until they can close as many gaps as they are able to find. The system also allows them to determine what attackers are focusing on, such as web attacks, email attacks, or living-off-the-land command attacks, using commands that are already in the system instead of malware.

Finding the time to maintain the tools

The biggest challenge to this approach is maintenance. By supplementing existing off-the-shelf tools with brand-new tools from GitHub, his team was basically producing new software. This added a level of maintenance to the tool that takes even more time than it might otherwise, especially if they want the tool to be current with threat activity as it changes.

Luckily, the senior management board at Todd’s company takes some of the time-sucking activities off of the security team’s shoulders, such as compliance and governance. This allows the security team to focus on criminal threats or catastrophic activity, and gives them more time to focus on their own tools to detect and block threats.

On top of the senior team’s willingness to prioritize security, Todd credits his pentesting background as one reason why his company’s security program is so focused on threats. Because he built the security program himself, he was able to bake threat detection into the program from the very beginning. While he is concerned with some governance and compliance issues, he is still focused on threats and keeping up with the changing methods attackers use to get into systems.
Risk and compliance vs. threat detection: Why organizations need both
Todd compares the security program team to police officers, and says there should be two departments. The risk and compliance team would be comparable to a police department that works with legislators to write laws and how to enforce them. The threat team would be like beat cops on the street, investigating crimes or even preventing them from taking place. In a big enough city (or organization), these teams are separate, working on their own priorities. But most businesses have to prioritize one over the other, and often choose compliance over threat detection.

This is not the best approach to security, according to Todd, because focusing solely on compliance and risk doesn’t help prevent attacks. For instance, he notes that none of the major companies that have had big data breaches in the past few years have failed audits before the breaches happened. Attackers will find new ways to get into systems that aren’t covered by current compliance audits. A security program that truly protects data and assets must include both types of security, hopefully performed by two separate teams who can prioritize properly.