Last updated at Thu, 26 Oct 2017 19:00:55 GMT
Let’s say that you, like many of your colleagues, are hosting your application on AWS’s EC2 cloud infrastructure. You’re chugging along at a steady rate of growth when BAM! One day you get a spike of traffic and have to scale up quickly. “Good job,” you think as you pat yourself on the back in your mind, “this choice to host in the cloud means we can easily handle this load spike without a problem. We’ve set it to auto-scale, so we’ll have all the instances we need.” But is everything all good? How are you going to get logs off of those new instances? Even if you’re under the impression that you only need logs when things go wrong, do apps tend to misbehave less or more under heavy load?
It used to be that using a 3rd party logging tool on AWS’s EC2 cloud was tough. You’d have to manually discover your log files and set each one up to go into your solution of choice. If you cloned an instance or spun up new instances to scale up, you’d then have to go and do the process again for the new ones. One of the main reasons to us Amazon Web Services has always been the ability to scale effortlessly…so why does such a critical piece of your app such as logging have to be so hard?
Well today we’re happy to tell you that it doesn’t have to be that hard anymore! With the Logentries AWS connector you can near-instantly detect all your AWS EC2 logs and configure them to send data to Logentries. All your logs in one place. Easily. With alerts, search, tags, graphs and more.
What does it do?
Le_aws is a command line tool that makes it possible to automatically generate RSyslog configuration files and deploy them onto your Linux EC2 instances. Install it from our script, enter your AWS IAM credentials and Logentries account key. That’s it! Then simply run aws_setup to automatically forward your AWS EC2 instance logs to Logentries.
What do I need?
A computer running Linux (apt-get or yum required) or Mac OS X (Xcode and command line tools required). Python >= 2.6 is also required as well as python development headers and tools (for Linux users this can be installed from our installation script if it is not already available on your system). Finally, the AWS key pairs associated to your EC2 instances need to be available on your computer.
From there, our installation script installs le-aws commands in a directory called le_aws (created in the folder where the installation runs). Nothing is installed globally in your system. So if you wish to un-install it, simply remove the le_aws directory!
How does it work?
Our AWS connector uses your AWS credentials in order to retrieve basic information about your EC2 instances (whether it is running or not, the name of the associated key pair, etc). For each running instance le-aws looks for the corresponding key pair on your machine (in ~/.ssh by default but a list of directories can be provided instead) and creates a SSH config like configuration file.
Le-aws uses this file to connect to your AWS EC2 instances and retrieve paths to your log files (under /var/log by default but different paths can be provided). From there, an RSyslog configuration file is automatically generated for you and deployed to your EC2 instances. In the meantime, corresponding hosts and logs are created in Logentries and you simply need to log into your account to start seeing data coming in!
Where do I go from there?
By default, the connector assumes that your AWS EC2 key pairs are in your ~/.ssh directory and configures RSyslog on all the EC2 instances so that all the log files located under /var/log are forwarded to Logentries. You can keep these default settings at first, look at the output in Logentries and decide on how to refine them then. Update the le-aws configuration file, run le-aws again and the changes will be automatically picked up!
So what are you waiting for? Get your AWS EC2 instances set up for logging on Logentries and make life a little bit easier.
