Posts by Rapid7

5 min IT Ops

Unleash the power of node.js for Shell Scripting (Part 1)

Setting Up Running a Node Script We are going to talk about creating shell scripts with node.js. The first thing that you need to do is install node.js. You can get the installers from https://nodejs.org/en/download/, or use your favorite package manager. Let’s go straight in and write our first script. The first thing to do is to create a file called script.js with the following code: console.log('hello world') We can now execute this script by running the following command in our shell:

3 min IT Ops

Logging from Tableau for Successful DataOps

Lately, we’ve been seeing a growing number of customers using Logentries in support of DataOps – the practice of collecting, normalizing and redistributing data throughout an organization so teams can make smarter business decisions. With teams ranging from engineers to support to marketing & sales relying on data for every day decision making, a critical requirement of those within a DataOps role is to actively monitor their organization’s data platforms. I recently had the pleasure of chatt

4 min IT Ops

Exploring Lambdas and Streams in Java 8

Java 8 introduced a host of new features, including lambda functions and streams. In this article I will focus on these two features as they are the most impactful features that were added in this new version of Java. Lambda Functions If you ever had to write a GUI for an application with an OO language you’ll understand the pain of writing event handlers, such as a mouse click event. So much boilerplate needed because you had to define a disgusting inline class with a single method. With the

6 min Log Management

10 Best Practices for Log Management and Analytics

Introduction Today’s Log Management and Analytics Challenges Within the last decade, the advancement of distributed systems has introduced new complexities in managing log data. Today’s systems can include thousands of server instances or micro- service containers, each generating its own log data. With the rapid emergence and dominance of cloud-based systems, we have witnessed explosive growth in machine-generated log data. As a result, log management [https://www.rapid7.com/fundamentals/what

3 min IT Ops

Revealing Hidden Insights with Docker & timeslice()

Over the last few months, you’ve probably noticed that we’ve been talking about Docker quite a lot – we think this lightweight and powerful way of managing environments is only going to increase in popularity. Docker has been evolving their logging capabilities with each release, so of course we’re interested in how we can help you get log data from Docker into Logentries for quick and powerful analysis. In August, we announced a new way to collect and analyze Docker logs for free with our Doc

1 min IT Ops

Introducing LEQL: SORT()

If you’re familiar with a query language like SQL, you’re likely used to being able to sort your query results. When querying log data, sorting your results can come in handy when you want to analyze things like which Docker containers are using the most memory [/2015/07/an-all-inclusive-log-monitoring-container-for-docker/], or which URLs are being request most frequently from your CDN. Today, we’re announcing SORT as the latest function introduced into Logentries’ query language, LEQL. Much

7 min IT Ops

Do you need an Architect in a Software Company?

This may be a dangerous question to ask for someone whose role is that of an Architect, but I think it is a valid question for an Architect to ask. This is particularly true in the software industry where the role is interpreted in many different ways. In some cases, an Architect may work in an established enterprise company and hand down instructions on technology stacks to the developers. At the other extreme an Agile development team may work without the involvement of an Architect. Neither

7 min IT Ops

Log Analysis for Containers

Introduction The IT and DevOps world has come a long way with infrastructure. Virtualization revolutionized our ability to quickly deploy an application and scale up services when needed, paying only for the computing power used. Over the last few years, agile methodologies and continuous delivery have pushed VMs to their limits. Many teams still repeatedly use a single VM for releases and testing. Production VMs rarely change unless something goes seriously wrong. At the pace software develop

4 min IT Ops

Common Angular Routing Challenges

When it comes to frameworks, no one is perfect. As we migrate the Logentries application from legacy code to Angular, we’ve encountered a few interesting challenges along the way that we’ve enjoyed investigating and resolving. While specific challenges often depend on your project and migration strategy, the aim of this post is to share our solutions to problems one may encounter when migrating an app to Angular. In particular, I’ll focus on how Angular handles routing and some issues we’ve en

1 min IT Ops

Best Practices for Container Log Analysis: Part 2

This 3-part series explores the challenges presented by containers and the advantages of using an end-to-end container log monitoring solution for complete container environment visibility. [http://info.logentries.com/log-analysis-containers] When working with containers, setting up a local image repository like Docker’s Registry can enable a team to iterate quickly, easily storing image versions in a central location to be used as needed. Yet as multiple team members update images, start, st

1 min IT Ops

Best Practices for Container Log Analysis: Part 1

This 3-part series explores the challenges presented by containers and the advantages of using an end-to-end container log monitoring solution for complete container environment visibility. [http://info.logentries.com/log-analysis-containers] As container environments become mainstream, it’s important to consider the most common challenges of migrating a monolithic application into containerized microservices and how to overcome them. While containers introduce new levels of flexibility from

3 min IT Ops

What is Elastic Logging?

We’re all familiar with the concept of “Elasticity” – the way cloud infrastructures can automatically react to their required workloads, scaling resources up or down as needed. While elastic environments provide us with much needed flexibility, they have also historically presented challenges when trying to monitor activity from their ephemeral components. Automatically accounting for new nodes can be tricky while scaling up. And when scaling down, data associated with these nodes is potentially

14 min IT Ops

State of Log Management for AWS

Introduction The Log Management [https://www.rapid7.com/fundamentals/what-is-log-management/] industry was traditionally driven by regulatory compliance and security concerns resulting in a multi-billion dollar market focused on security and information event management (SIEM) solutions. However, log management has evolved into a market that is focused on both the management and analytics of log data. Log management technologies are becoming more powerful and dynamic, allowing for data to be

3 min IT Ops

Using Log Aggregation Across Dev & Ops: The Pricing Advantage

Rob Thatcher is Co-founder and Principal Consultant at Skelton Thatcher Consulting [http://skeltonthatcher.com/]. Summary: the pricing of tools or licenses for log aggregation can have a significant effect on organizational culture and the collaboration between Dev and Ops teams. Modern tools for log aggregation (of which Logentries [https://logentries.com]is one example) can be hugely enabling for DevOps approaches to building and operating business-critical software systems. However, the pri

3 min IT Ops

How to Add a GPS Time Source to ntpd

USB GPS dongles have come down significantly in price in recent years and I picked one up to play with recently. Apart from using a GPS module to report your latitude, longitude, altitude and time for mapping applications, it’s also possible to feed the time information to ntpd as a back-up time source or as a highly accurate time source depending on the GPS module you end up getting. [/content/images/le-img/2015/07/adding-gps-time-source-to-ntpd.png] The module that I use in this blog post