Last updated at Fri, 03 Nov 2017 13:30:55 GMT

As co-founder of Logentries I am often asked – “Why Logs?”  And I have to admit, upon first impression, ‘log management and analytics’ does not seem like the sexiest space 🙂 However at Logentries we are here to redefine that space, to provide a solution to access, manage and understand your log data that is easy to use, cost effective and intelligent (i.e. it does the hard work so you don’t have to).  But that being said it still begs the question, “Why logs?”

Logs are the most fine-grained data source for understanding today’s system. Unlike traditional monitoring and analytics tools which provide an aggregate view of what is happening in your system (such as server monitoring, application performance monitoring, web analytics etc.), logs capture every single event so that you can understand not only the general trends, but EXACTLY what happened, in what order, and by whom. Logs allow you to view this level of detail in real-time or to review it in a post-mortem fashion. At the same time, they can be rolled up into dashboards to give you a high level view of what is happening across your system. So in effect they can provide the best of both worlds: the low level detail of exactly what has happened as well as the high level trends across your systems.

However the biggest issue with many logging solutions today  is:

  • Too expensive: Keeping all that log data around for more that 30 days has been prohibitively expensive, so deep historical system understanding has been difficult to achieve with logs. People have instead turned to the traditional monitoring tools that give summary views that can span back indefinitely due to the ability to store this data in a much more cost effective manner vs. raw logs
  • Too difficult to use: Logging providers expect you to learn their query language, requiring deep technical skills and a lot of time on your hands to get value from them.
  • Too difficult to maintain: In particular open source or in house solutions are  difficult and costly to maintain and organizations quickly get frustrated with their in house logging solution.

At Logentries we address (and continue to address) each of the above points. We want you to send us all your data, and to make this available  in an easy-to-use, accessible and cost effective manner.

And sending us all of your data has just become even easier with our new Shinken plug-in for Nagios® and Diamond integrations:

  • Nagios® Plug-in via Shinken: Shinken is an open source monitoring framework, that is compatible with your Nagios® systems, but improves some of the traditional issues associated with the Nagios framework (e.g. scalability). With this integration, you can send results of your Nagios® or Shinken health checks to Logentries such that you can get a real time view of the health of your infrastructure, correlated with your traditional log data. You can also easily maintain a history of your health checks which has always been difficult with tools like Nagios®, so it’s easier now to look back historically at any major issues and to identify and recurring themes.

  • Diamond: Diamond is a python daemon for collecting metrics. It also has a bunch of collectors that provide the ability to collect detailed performance metrics from your OS as well as from common components like Hadoop, Mongo, Kafka, MySQL, NetApp, RabbitMQ, Redis, AWS S3… The new Logentries Diamond handler allows you to stream all of these metrics into your Logentries account in real time so you can easily visualize them in dashboards and again correlate with any traditional logs from your systems or apps.

Check out these new IT and Dev Ops plug-in designed to continue to provide the deepest, most fine-grained view of your system-wide operational data.