Last updated at Thu, 26 Oct 2017 20:10:07 GMT

Over my time at Logentries, we’ve had users contact us about where to find their logs while they were setting up Logentries. As a result, we recently released a feature for Amazon Web Services called the AWS Connector, which automatically discovers your log files across your Linux EC2 instances, no matter how many instances you have. Finding your linux logs however may only be a first step in the process as AWS logs can be all over the map… so to speak…. So where are they located? Here’s where you can start to find some of these.

Where Are the Operating System Logs?

Obviously OS logs are not exclusive to AWS but you are going to need these if you are running there and want to analyze your systems, so we’ll start with these.

For Linux, log files are located under the /var/log directory and its subdirectories. Within this directory there are several log files with different names and which record different types of info. Some examples include, but are not limited to:

  • /var/log/message - Contains global system messages, including the messages that are logged during system startup. Includes mail, cron, daemon, kern, auth, etc.
  • /var/log/auth.log - Authenication logs
  • /var/log/kern.log - Kernel logs
  • /var/log/cron.log - Crond logs

AWS Services Logs

Last week at re:Invent Andy Jassy took the attendees through the range of AWS services since their inception in 2006. They now boast 42 products and services!

AWS Log Management

Here’s where you find the logs of some of the most popular services such as RDS (database), S3, Beanstalk (platform as a service on Amazon), and CloudFront.

Amazon RDS

According to AWS documentation, you used to have to query the database to get your Amazon EDS logs. Now, however, you can view them using the AWS Management Console, or get them via the API, whether you’re using a MySQL, Orace or SQL Server database engine. You can either view your logs at a point in time, watch them live or download them to work with offline. The types of logs available include:

  • MySQL - MySQL Error Log
  • Slow Query Log
  • General Log
  • Oracle - Alert Log
  • Trace Files
  • SQL Server - Error Log
  • Agent Log
  • Trace Files

Amazon S3

Based on this write-up by Software Engineer Geoff Greer, your best bet for getting your S3 logs is to go through the S3 Management Console and set up the logs to be sent to a bucket.

Elastic Beanstalk

Based on this Amazon Documentation, your best bet for accessing your server logs from Elastic Beanstalk is through the AWS Management Console. In the Environment Details section you can view a snapshot of your logs and any time or you can set up your logs to be sent to Amazon S3 for storage and analysis.

CloudFront

This service provides log files with information about user requests. You can enable CloudFront to send access logs so that you can store and analyze the logs. This is enabled by an API call to the CloudFront API and the logs are then sent to your Amazon S3 bucket for easy access.

For the most part, many services’ logs are available through either the Amazon Management Console or the service’s specific console. Users can either consult recent log events in the Amazon console (i.e. on the AWS website; for S3, use the S3 Management Console) or get these log events sent to an S3 bucket. It’s not the default behavior, though, and users have to set up a bucket themselves and configure their AWS service so that their log events go there.

With 42 products and services, this should be a good overview to get you started on where you AWS logs are, and we’ll look to do a follow-up post soon as this is a first step in finding where some of your logs are.

Like we said, though, if you use our AWS Connector, though, it will automatically discover your log files across your Linux EC2 instances, no matter how many instances you have.

Ready to start getting insights from your applications? Sign up for a Logentries free trial today.