Last updated at Sun, 31 Dec 2023 17:33:26 GMT

The Security Operations Center (SOC) has become a familiar term across many services and organizations, with many attempting to either stand up an in-house team, or outsource security operations to a managed security services provider.

While a security-in-depth approach to implementing technology designed to detect, slow, and/or halt adversaries is great, no technology or combination thereof can absolutely guarantee an organization will not experience a breach.

That’s why it requires highly skilled security analysts, maintaining persistent 24x7 eyes-on-glass monitoring to augment the technology and effectively provide ongoing coverage for analysis, detection, and response against known and unknown threats.

The challenge for most organizations who choose to build an in-house SOC is that it’s incredibly expensive to set up. Between implementing technologies and hiring analysts to maintain 24x7 coverage, it can be well over a $750,000 budget spent in the first year alone!

What those SOC costs won’t show is the ongoing expense to ensure detection effectiveness as attackers change tactics, techniques, and procedures (TTPs). Doing this successfully for the long term requires a team of threat researchers and detection engineers.

Rapid7’s Managed Detection and Response (MDR) team acts as a replacement for—or an extension of—an organization’s internal security team. They perform continuous monitoring of your networks and endpoints and ensure you’re one step ahead of attackers as they evolve.

With globally dispersed, state-of-the-art SOC facilities, equipped with the technology and security to enable our MDR analysts to monitor each customer’s environment around-the-clock, this “Follow-the-Sun” model allows Rapid7 MDR analysts from around the world to perform real-time log analysis, threat hunting, and alert validation at scale across all MDR customers.

How does the Rapid7 MDR team work?

We segmented our MDR security operations into multiple teams to ensure optimal visibility and coverage, while also ensuring our team has the best detections and tooling required to protect our customers. The MDR service delivery team is composed of:

  • A Threat Intelligence team
  • An Engineering team
  • SOC Pods (includes Threat Analysts and Cus'tomer Advisors)

MDR Threat Intelligence

Our MDR Analyst Pods operate using multiple technologies and tools, starting with the Insight Agent, which MDR analysts use to collect various data types from endpoint devices. Endpoint agent metadata and other log sources are fed into InsightIDR, Rapid7’s Gartner-recognized “Leader” for SIEM, and is combined with our internal, purpose-built threat intelligence platform (leveraged by our Threat Intelligence team) to detect any real-time threats.

The job of the Threat Intelligence team is to ensure that all identified indicators of compromise (IoCs) are used for detections to counter any threats for customers. These IoCs are informed by over 115+ Billion daily security events!

As attackers evolve and new threats are discovered, our Threat Intelligence team develops signature and analytic detections for existing and emerging threats to ensure coverage for various TTPs that malicious actors use in the wild.

These detections allow our analysts to incorporate all vulnerabilities and attacks based on the MITRE ATT&CK framework, including, but not limited to, User Behavior Analytics (UBA) and Attacker Behavior Analytics (ABA), into our detection methodologies. These detections get better over time as our analysts inform the threat intelligence team of rule suppressions to add granularity, reduce noise, and avoid recurrency.

MDR Engineering

With thousands of alerts triggered per day, it is a significant task for the analysts to ensure all of our customers have 24/7 coverage. Using the right technology, including InsightIDR and our internal tools, analysts around the globe follow a systematic approach to address all of the alerts and issues.

The Managed Services engineering team (the “R&D” behind MDR) provides continuous support and services to streamline, automate, and reduce the analysts’ manual workload. This includes support for developing all internal tools that the analysts use on a day-to-day basis, as well as incorporating security orchestration and automation (SOAR) into the analysts’ workflows.

While the efforts of the MDR Engineering team are not directly seen or experienced by customers first-hand, the MDR Engineering team is imperative to ensure efficiency and scale for our analyst pods to deliver the best MDR service possible to our customers.

SOC Pods (Threat Analysts and Customer Advisors)

Our teams operate in “Pods,” with multiple analysts and Customer Advisors aligned to each customer. This ensures a deep understanding of the customer environment by every analyst, no single points-of-failure, and improved scalability to ensure our teams can deliver efficient service for every customer.

When an alert is triggered, analysts acknowledge the triggered alert and perform a full analysis using a combination of data from InsightIDR and our internally developed tools to perform deep forensic investigations of the incident. The SOC uses an array of tools to quickly triage and investigate evidence to determine whether the event was malicious in nature.

For example, if a user account on an asset runs an obfuscated PowerShell script, Rapid7’s rules ensure that a “high-priority” alert is triggered. Once triggered, an analyst investigates the alert. If the script is determined to be malicious in nature, a Customer Advisor contacts the customer through email and/or by phone call. Rapid7 also provides recommendations for containing the spread of malware. While the Customer Advisor is contacting the customer, the analysts continue their investigation.

Via the Insight Agent functionality, analysts can acquire necessary logs or files by performing forensic data acquisitions on the associated asset if it is online at the time of analysis. Some of the files that an analyst can acquire include browser history artifacts, event logs, files from directories, and registry hives.

After a thorough investigation of the evidence, analysts then write a formal Findings Report, which describes the overall impact of the incident. Within the Findings Report, analysts aim to include a high-level overview of the incident, along with a detailed technical description of the evidence and overall findings.

The Findings Report also includes extensive guidelines for remediation, as well as mitigation strategies to help customers prevent the incident from happening again. After the report is complete, the analyst informs the Customer Advisor, who then sends the report to the customer. The timeline below details an average day for an MDR analyst.

Just a day in the life for Rapid7 MDR

05:00:00 UTC

MDR analysts on night shift in the United States scroll through the alerts in InsightIDR and triage any “high-priority” alerts.

05:04:30 UTC

MDR analyst ‘X’ discovers a “medium-priority” alert that states, “Encoded PowerShell in Command Line.” Analyst ‘X’ then acquires the suspected-malicious document from the host via the Rapid7 Insight Agent, as well as endpoint and network logs, to investigate whether the second-stage payload from the malicious obfuscated PowerShell was executed.

05:16:22 UTC

After investigating the logs and validating the incident, Analyst ‘X’ confirms that the payload executed and is infecting other hosts. A Customer Advisor contacts the customer to immediately remediate the infected host by quarantining it and resetting the associated user account’s credentials.

05:18:19 UTC

The customer starts remediating the infected systems, while the analyst gathers information regarding the initial infection and begins drafting a Findings Report.

05:30:15 UTC

After detonating the malicious document in a sandbox, Analyst ‘X’ identifies that the document contained malicious macros, which resulted in the execution of a PowerShell command.

05:45:45 UTC

Analyst ‘X’ then acquires the second-stage payload sample and detonates it in the sandbox to further examine the malware behavior.

05:50:18 UTC

While preparing the Findings Report, another “high-priority” alert for the same customer, but a different document, appears on multiple hosts.

06:00:20 UTC

Analyst ‘X’ prepares a snippet of information necessary for the customer to act on. The Customer Advisor then ensures that communications are sent out to the customer. The customer receives steps for remediation from Analyst ‘X’.

06:30:00 UTC

Analyst ‘X’ has all of the files and necessary logs to begin a detailed investigation.

07:40:00 UTC

Since it is almost time for Analyst ‘X’ to wrap up their shift, they gather all of the acquired information to begin the handover process to the next shift with the analysts in EMEA.

08:05:32 UTC

After the handoff from Analyst ‘X’, Analyst ‘Y’ now has sufficient information about the  investigation. Analyst ‘Y’ begins to draft the Findings Report using the discovered evidence, such as affected hostnames, usernames, process logs, network logs, and detailed analysis of malware behavior.

08:55:10 UTC

After Analyst ‘Y’ completes the Findings Report, the analyst uploads the report to the customer portal. A Customer Advisor then conducts a final review of the report before sending it out to the customer.

But wait, there’s more!

Remote Incident (Emergency Breach) Response

Not to be confused with a typical Incident Findings Report, Rapid7 MDR also performs Remote Incident Response in addition to the day-to-day Incident Response investigations outlined above.

A Remote Incident Response (also commonly referred to as Emergency Breach Response) is a remote service that involves a more intensive analysis effort performed by a team of analysts from the MDR SOC.

Customers may choose to use the Remote Incident Response service if there is any indication of an active or particularly dangerous threat detected in their environment, such as an active hands-on-keyboard attack, a ransomware attack, or the compromise of critical infrastructure.

During a Remote Incident Response, Rapid7 provides customers with a team of analysts to hunt for indicators of compromise, and analyze any threats detected within the environment. Analysts will continue to search for new evidence of compromise until the threat has been contained and remediated.

Threat hunting

Along with daily triage and Remote Incident Response, analysts also perform threat hunting for the customers on a monthly basis. Threat hunting is monitoring of threats and suspicious activities in the customer’s environment.

The analysts identify any threats that may have been left undetected for the month and convey it to the customers through monthly hunt reports.

These reports also include extensive information about the activity on the customer’s infrastructure, including the health status (active/inactive) of log data sources, such as network, Active Directory, and endpoint agent. Additionally, the monthly hunt reports also provide the total number of alerts and Findings Reports that were written for the month.

In conclusion

This was just an example of one of the many investigations SOC performs 24/7. While this may not seem complex, monitoring and analyzing each alert triggered for hundreds or more customers can be intimidating. However, that does not stop Rapid7 MDR from providing the most efficient services to its customers and working continuously to ensure that their environments are more secure.