Last updated at Mon, 30 Oct 2017 19:10:35 GMT
Staying on the subject of devops, specifically server automation and monitoring, I’m going to show you how you can automatically send logs to Logentries using Chef and Vagrant. If you are unfamiliar with either of these technologies I suggest you have a look through my previous posts to bring you up to speed on things.
We’re going to cover how to install the logentries agent with Chef on a clean Ubuntu server running in Amazon. Firstly if you’re using librarian-chef you will need to add the following line to your Cheffile.
cookbook 'logentries', :git => 'https://github.com/logentries/le_chef.git'
Running librarian-chef update will install the le_chef cookbook to your cookbooks directory.
Once we have this in place, we need to override attributes for the following:
default[:le][:userkey] = ""
default[:le][:hostname] = "Default"
default[:le][:logs_to_follow] = ["/var/log/syslog"]
I suggest we do this override in a role or environment, or possibly a mix of both.
By setting the userkey in an environment, lets say “dev”, any cookbook running with the dev environment will automatically inherit this attribute, regardless of whether it is used or not.
With Chef there is a tool called Ohai which is used to detect attributes on a node at runtime, one such attribute is the nodename, this is what we use by default to name your hosts when you log into Logentries. For more information on node attributes that are exposed look at the chef documentation about Ohai.
In an environment, the userkey and hostname overrides would look like this:
{
"name": "dev",
"description": "Random Development Environment",
"json_class": "Chef::Environment",
"chef_type": "environment",
"override_attributes": {
"le": {
"userkey": "ABCDEF123456"
}
}
This is fine for setting a new host in Logentries and naming it, however there are no logs being logged yet. To start sending logs to logentries we define an array of logfile locations as either a cookbook or role attribute override. We will override at a cookbook level as this makes most sense in custom cookbooks or wrapper cookbooks. Lets take a look at the following scenario. We will use two cookbooks which we want to log to logentries, wrapper_apache2 and wrapper_mysql. These have custom logs as specified below which we want to log to logentries.
In the attributes file for these cookbooks we can add the following lines which will basically tell it which logs to send to logentries.
wrapper_apache2
override[:le][:logs_to_follow] = ["/var/log/apache2/custom_access.log","/var/log/apache2/custom_error.log" ]
wrapper_mysql
override[:le][:logs_to_follow] = ["/var/log/mysql/custom_mysql.log" ]
We will also need to include the logentries cookbook in the default cookbook and add a dependency on logentries to the metadata file.
default.rb:
include_recipe 'logentries'
metadata.rb:
depends 'logentries'
With this in place, we can use vagrant to create a VM in Amazon EC2 that will automatically log to logentries.
config.vm.provider :aws do |aws, override|
aws.access_key_id = 'ACCESS_KEY'
aws.secret_access_key = 'SECRET_KEY'
aws.keypair_name = 'KEYPAIR'
aws.ami = 'ami-d1f308a6' #http://cloud-images.ubuntu.com/releases/12.04.2/release/
aws.region = 'eu-west-1'
aws.instance_type = 't1.micro'
aws.security_groups = ['default']
override.ssh.username = 'ubuntu'
override.ssh.private_key_path = '~/logentries/aws/devops.pem'
aws.tags = {
'Name' => 'logentries_test'
}
end
config.vm.provision :chef_client do |chef|
chef.node_name = 'logentries_test'
chef.chef_server_url = 'https://api.opscode.com/organizations/logentries'
chef.validation_key_path = '~/.chef/logentries-validator.pem'
chef.validation_client_name = 'logentries-validator'
chef.log_level = 'info'
chef.add_recipe 'wrapper_apache2'
chef.add_recipe 'wrapper_mysql'
chef.environment = 'dev'
end
Now if you log into your account you should see your new node and the files automatically being followed.
