Last updated at Thu, 22 Aug 2024 18:44:41 GMT

Kubernetes use is rising rapidly—according to a 2019 Cloud Native Computing Foundation (CNCF) survey, 78% of respondents say they use Kubernetes today (58% more respondents than the previous year). With numbers like those, it looks like everyone is headed toward the cloud.

But as with any journey, you want to make sure your ride starts smoothly and stays smooth, which means you need to plan, prepare, and regularly monitor.

Think about how an airplane journey really operates. During security checks, you verify that whoever is asking to enter the more secured areas is allowed to, and you check their baggage and carry-ons to make sure only approved items enter the secure zones. Once afforded entry, you continue to track and monitor with cameras, tags, special authorizations and the like—only staff are allowed in the most sensitive areas, while some VIP passengers have special services available only to them, and passengers can only move around in the public areas.

In short, different groups are allowed to do different things. During this time, the ground staff is making sure the airplane is in order and flight staff in the tower is making sure monitoring and controls are working. Once in flight, you’re pretty confident that everything will go smoothly, but the control tower monitors and communicates with the pilot and crew for the duration of the flight to stay on top of things and act fast in case of emergency.

Kubernetes is really the same, all in all. But for Kubernetes, sometimes it can be hard to fulfill these requirements. Misconfigurations and security are still a primary concern for Kubernetes deployments. Compliance with different regulations and industries is a huge challenge. Additionally, K8s experts are scarce, and gaining enough visibility and control over a big deployment without that expertise can be nearly impossible.

Implementing some best practices and important security guardrails as early as possible will ensure that you’re at least one step closer to a calmer deployment experience.

To understand better, we can break things down into these parts:

  • Check-in and preliminary security screening: Planning and configuring your deployment
  • Pre-flight: Monitoring security areas prior to takeoff
  • In-flight: Runtime monitoring

Let’s take a closer look at each one of these parts in more detail.

Check-in and preliminary security screening

At any airport, you will always be welcomed by staff who know what they’re doing. This is because they’ve been trained, and they’ve been involved in the preparation process. Similarly, once you’ve planned your microservices architecture, it’s time to get your DevOps involved.

It’s important to educate these teams on security issues. Document and explain how they should participate in security efforts, and what they can do while working in order to keep your organization safe from data breaches. Know the risks related to Kubernetes deployments and be prepared before going live. Make sure everyone involved is aware of your security policy, accounting for new zero-day vulnerabilities and exploits, compliance breaches, and more.

Remember, whether a company finds and fixes the mistake themselves or leaves it for a hacker to find and exploit can mean the difference between a company’s success and failure.

Managing the configuration and deployment strategy of your Kubernetes services is like the security checks performed before allowing airplane crew and passengers to pass the gate to highly secured areas and to the boarding area. This is when you check luggage and personal items, interview passengers about where they’ve come from, check their tickets, and the like.

The Kubernetes vulnerability scanner helps perform checks early on in the development process by covering rich Kubernetes and Istio security best practices as well compliance checks such as (but certainly not limited to!):

  • Kubernetes vulnerability scanning
  • Hunting misplaced secrets
  • Excessive secret access
  • Workload hardening from pod security to network policies
  • Istio configuration and best practices
  • Ingress controllers for best practices
  • Kubernetes API server access privileges
  • Kubernetes operator best practices

You should push planning and security as far left as possible. By getting started early, you can make sure risks are flagged and removed before deployment, and that only an authorized deployment configuration is allowed into the production environment. This will save money in time invested, and more importantly, better protect your data and intellectual property.

Monitoring restricted areas

After the preliminary security checks, passengers and crew are allowed into more restricted areas. Just because your passengers have been allowed to enter the boarding area in general though, does not mean they’ve been authorized to enter the entire airport. To make sure there are no breaches, everyone must still be monitored and areas must be kept secure. Some are only allowed in the general public areas and to board the flights on which they’re scheduled; others can access VIP services, while some employees are allowed in different parts of the secure areas—different offices versus the runway, for example, are restricted to different employees.

In the same way, you should ensure Kubernetes administrative traffic through your deployments (from your control plane) is authorized and that you can easily identify anomalous behaviors and suspicious activity patterns.

K8s audit logs are your source of truth for coping with malicious activity. But dealing with audit logs can feel like looking for a needle in a haystack. Real-time forensics and analysis automation of Kubernetes audit logs enable early detection and reduce a lot of noise in the battle of audit logs.

Rapid7’s security solutions analyze your Kubernetes logs, identifying abnormal API and administrative activity and compromised k8s resources. Results are then displayed clearly from the advanced dashboard. This enables deep data-based investigation, helping you identify problems and take action immediately without having to sift through raw logs on your own.

Schedule and monitor throughout the flight

No less important than the configurations and preparations before takeoff are the ongoing monitoring and scheduling while in-flight. One of the best defenses against attacks and unexpected activity is to continue watching the screens for odd objects in the skies, maintaining contact with the control tower, and continuing to separate individuals on the flight based on access rights (only pilots and the head attendant can access the cockpit, only staff and authorized travellers can access first-class areas, etc.).

Implementing guardrails during development is critical, but never replaces ongoing monitoring at runtime. During production, there are a lot of things you need to be on the lookout for. As a “gatekeeper,” your admission controllers validate and prevent risky and vulnerable deployments.