Last updated at Tue, 09 Jan 2024 15:35:15 GMT

When I was in grade school, I played football. I was scrawny and afraid to go up against anyone bigger than I was (essentially everyone). I always hated Oklahoma drills and scrimmages with my team. For quite some time, I avoided “the tunnel” hoping to evade facing the bigger linemen. My coach sat me down and explained “why” we did these drills.

“We are building your muscle memory to increase your awareness and reflexes in a real game, when it really matters.” I was a snotty nosed kid back then, and I didn’t realize that this would be applicable throughout my career.

What scrimmages were to football, tabletop exercises (TTX) are to incident response, business continuity, disaster recovery, vulnerability management, and other critical components of your organization’s security program. TTXs build the essential muscle memory at all levels of the organization.

Three Levels of an Organization

Organizations can be divided into three core levels:

  • Strategic: Long term visionaries. Executive leaders that guide the organization forward and place a focus on larger strategic positioning.
  • Operational: Day to day management. Front line managers and the communication arm of the strategic level. This level takes strategic vision and turns it into tactical application.
  • Tactical: Where strategic planning and operational processes are put into action. These are the day-to-day tasks that are actioned on, such as monitoring and detection, deployment, onboarding, etc.

Delineating these levels is important when building a TTX. You need to know who you are facilitating a TTX for, to ensure that you’re incorporating the relevant data for them to understand, respond to, and discuss. In other words, you need to provide the technical artifacts to the Tactical level participants, operational processes to the Operational level participants, and strategic business impact events to Strategic level participants.

There are times in which you might deliver a TTX specific to just one level within the organization. While this may be better for building consistent muscle memory with those specific teams, scrimmaging with the entire business is also essential.

The Three TTX Methodologies

There are three methodologies that I discuss with our customers. Each of these methods have benefits for all organizational levels, but are ideally suited to specific levels as outlined below.

  • Break-The-Glass: This TTX method is great for the strategic level because it allows the teams to work backwards and forwards with an incident at the same time. A break-the-glass TTX drops the main incident event right at the beginning of the exercise, so the incident is known right out of the gate. This is great for testing the overall response to an incident and allows you to go as technical as you would like, while still enabling the strategic level to participate and add value. The downside is that nuanced operational processes may be missed and go untested, and those processes may be very important in a real-world scenario.
  • Escalatory Method: This method takes a more granular look at each process and response by beginning with lower-level events and escalating the severity of the incident as the scenario develops. This is great for the operational level as it really focuses on the operational processes, procedures, and response plans that have been developed. A lot of findings can come out of this from a procedural perspective that will help further develop your IR plan and playbooks. The only real downside is that it plays out more slowly than a real incident given we take a close look at each step within the response.
  • Choose Your Own Adventure: This method is the most robust and difficult to build out and facilitate. Nonetheless, if done well, you can reveal significant growth opportunities in your response plans. This method starts with a single artifact and allows participants to ask questions and go through their plans. Once a participant hits on a key point or says a key word, the scenario will begin to unfold. There are ways for the incident progress to be stifled should the participants go down a different path as well. This is great for the tactical level to practice their playbooks, technical analysis, and critical thinking skills. The downside is that this type of exercise requires a significant amount of time to fully develop while attempting to predict where things may go. You may also mistakenly create a “one right answer” issue which is a common mistake in TTXs.

The "One Right Answer” Issue

When I discuss a TTX with customers, there are times where they want to practice one specific thing to prove that there is an issue in the program or point out problems in other teams. This is never a good idea. If you build a scenario around one specific risk, you might miss other risks or and blind spots not currently known to the business. Additionally, it can create animosity and widen the divide, mistrust, and other sentiments toward the sponsoring team. One way to avoid this is to allow for freeform discussion and allow it to naturally be discussed. The scenario story should be broad enough to allow the discussion to go where it needs to go without feeling forced or coerced.  

The Goal

There are many goals that you may want to achieve when delivering your exercises. One of those goals should be bringing the organization together and practicing the plans and processes to ensure that the muscle memory is there when you need it most—gametime. Don’t be afraid to go for the big risks and talk about them. It’s better to find out you don’t have a developed process for something in the exercise than to find that out in the middle of a real-world incident that impacts the business. If your resources allow, you should run scenarios more than once per year. A good routine is as follows:

  • Tactical level: Once per quarter
  • Operational level: Twice a year
  • Strategic level: Once a year

You can combine levels so there aren’t as many exercises running per year or run additional exercises just to stay sharp. Additionally, your scenarios don’t all need to be hours long, building muscle memory is all about repetition. Keep your teams practicing, iterating, and building your organization’s muscle memory so that you can respond effectively and with precision to any incident that occurs.

If you need help with TTX scenario development and facilitation, Rapid7 Advisory Services provides a compact offering to develop and deliver a TTX as well as provide a detailed report of findings and recommendations to increase the effectiveness of your incident response program.