Last updated at Wed, 13 Dec 2017 16:05:57 GMT
Synopsis:
This short article will demonstrate how to setup a minimal Bro cluster for testing. Because of its minimal nature, this article will exclude discussion of load balancing traffic across multiple bro workers (processes), security conscious permissions, and other bro related tuning and features such as sending e-mail. Its purpose is to get a Bro cluster up and running as quickly as possible so you can begin familiarizing yourself with cluster tasks and management.
Setup:
I’m using 3 Ubuntu Server VM’s on the same network segment. We will have:
- 3 Bro Nodes, Bro’s terminology for physical machines
- 2 Bro Workers, Bro’s terminology for processes analyzing traffic
- 1 Bro Manager, which handles events and manages the Bro processes
- 1 Bro proxy, which keeps state and shares information between the Bro workers
The manager and proxy processes will be running on the same Bro node, a common occurrence.
A Vagrant configuration is available to replicate the Bro cluster using virtual machines.
Procedure:
To install Bro follow the Install Bro official guide; install Bro and its dependencies on the manager and only Bro’s dependencies on the other 2 nodes.
The Bro manager manages Bro workers via SSH. The manager will log in to the workers to perform tasks such as starting, stopping, and monitoring the workers, pushing out new configurations, and printing statistics.
To allow Bro to perform these tasks we need to create a password-less SSH public-key on the manager and copy it to each Bro worker. We will use the root account for this article to keep things simple.
bro-manager# ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa): [ Press Enter ]
Enter passphrase (empty for no passphrase): [ Press Enter ]
Enter same passphrase again: [ Press Enter ]
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
...
If you’re using Ubuntu Server be sure to install the OpenSSH Server package on each system.
$ apt-get install openssh-server
Also, be sure that you can log in to the worker systems as the root user.
$ grep Root /etc/ssh/sshd_config
PermitRootLogin yes
Copy the public-key to the worker machines so that SSH negotiates with public-key authentication.
bro-manager# scp ~/.ssh/id_rsa.pub root@192.168.1.10:~/.ssh/authorized_keys2
bro-manager# scp ~/.ssh/id_rsa.pub root@192.168.1.20:~/.ssh/authorized_keys2
Now, we need to tell the manager which machines are the workers. We do this by editing the node configuration file, node.cfg. It’s located at $BROPATH/etc/node.cfg. The default install prefix is /usr/local/bro:
Be aware that the node names e.g. [….] are special and cannot be chosen like hostnames.
bro-manager# vi /usr/local/bro/etc/node.cfg
Comment out the standalone configuration at the top and add the cluster section.
# This is a complete standalone configuration. Most likely you will
# only need to change the interface.
[manager]
type=manager
host=192.168.1.5
[proxy-1]
type=proxy
host=192.168.1.5
[worker-1]
type=worker
host=192.168.1.10
interface=eth0
[worker-2]
type=worker
host=192.168.1.20
interface=eth0
On the manager, launch the broctl tool in interactive shell mode. Issue the install command which will copy the configuration to the bro workers.
bro-manager# /usr/local/bro/bin/broctl
Welcome to BroControl 1.1
Type "help" for help.
[BroControl] > install
creating policy directories ... done.
installing site policies ... done.
generating cluster-layout.bro ... done.
generating local-networks.bro ... done.
generating broctl-config.bro ... done.
updating nodes ... done.
Let’s make sure the configuration doesn’t contain any errors. When making future configuration changes you use “check” and then “install” if you don’t encounter any errors.
[BroControl] > check
manager is ok.
proxy-1 is ok.
worker-1 is ok.
worker-2 is ok.
Now, let’s start the cluster. If you don’t want to use broctl’s interactive mode, you can issue the commands in the following way.
bro-manager# /usr/local/bro/bin/broctl start
starting manager ...
starting proxy-1 ...
starting worker-1 ...
starting worker-2 ...
Note: A newer command is available in BroControl 1.4 which will check, install, and start Bro with a single command. It’s called deploy.
/usr/local/bro/bin/broctl deploy
Moving along, let’s check the status of all the Bro processes. If you see “crashed” under the Status column, view the logs on the machine that had the crashed process. For example, should worker-1’s Bro process crash, try viewing its logs to find out more information.
worker-1$ cat /usr/local/bro/spool/worker-1/stderr.log
Though, everything is configured properly for us.
[BroControl] > status
Name Type Host Status Pid Peers Started
manager manager 10.1.1.128 running 5334 3 01 Sep 20:12:41
proxy-1 proxy 10.1.1.128 running 5370 3 01 Sep 20:12:43
worker-1 worker 10.1.1.129 running 1731 2 01 Sep 20:12:48
worker-2 worker 10.1.1.131 running 3675 2 01 Sep 20:18:36
If the machines are not setup to receive live traffic an alternative is to generate some with PCAP files. Here, we generate traffic on one of the workers which will analyze the traffic and send events to the manager where they will be written to disk as logs.
worker-1# tcpreplay -i eth0 -M 1 milw0rm_exploits/*.pcap
Check the stats on the manager to make sure the packets have been received.
[BroControl] > netstats
worker-1: 1378093257.398824 recvd=103096 dropped=109 link=103207
worker-2: 1378093257.601587 recvd=101966 dropped=40 link=102006
Bro writes logs to folders by date. The symlink file named current, points to the directory where logs are stored until they’re rotated. Rotated logs are compressed with gzip and can be read with decompression tools such as zcat, zgrep, etc.
bro-manager# ls -l /usr/local/bro/logs/
total 12
drwxr-xr-x 2 root root 4096 Aug 31 21:58 2013-08-31
drwxr-xr-x 2 root root 4096 Sep 1 20:20 2013-09-01
lrwxrwxrwx 1 root root 28 Sep 1 19:51 current -> /usr/local/bro/spool/manager
drwxr-xr-x 2 root root 4096 Aug 31 21:56 stats
As Bro sees a specific traffic type it will write out log files that match its protocol analyzers. It does so in an easily parse-able tab-separated (column) format in ASCII. These log files can then be hacked at with the standard unix toolset ( grep, awk, sort, uniq, etc.) to mine for useful information.
bro-manager# cd /usr/local/bro/logs/current && ls
communication.log known_hosts.log notice_policy.log ssh.log
conn.log known_services.log packet_filter.log stderr.log
dns.log loaded_scripts.log reporter.log stdout.log
http.log notice.log software.log weird.log
Let’s examine a sample of the ASCII logs churned out by Bro. Here, we use bro-cut, a program packaged with Bro, to read in two connection flow logs so that we can convert the unix epoch timestamp to the more human-readable local time format with the -d option.
bro-manager# bro-cut -d < conn.log | tail -2
2013-09-01T20:54:17-0700 xFW8kGEp129 10.1.1.128 35564 91.189.91.14 80 tcp http 3.375464 385 480029 SF T 0 ShADadFf 173 7325 336 493473 (empty)
2013-09-01T20:54:17-0700 8rw1OMYRtZk 10.1.1.128 35564 91.189.91.14 80 tcp http 3.375237 385 480029 SF T 0 ShADadFf 173 7325 336 493473 (empty)
Anytime you modify your bro configuration or add bro scripts use the following procedure to put it in effect.
bro-manager# broctl deploy