Last updated at Tue, 16 Jan 2024 00:48:52 GMT

For HaXmas last December, I wrote about the introduction of Python modules to Metasploit Framework. As our module count keeps on growing, we thought that it would be a good time to update the community on where we are at. You should read the blog post to get the full idea, but here is the TL;DR for those who have not. We want modules to run in separate processes for two reasons. First, we aim to have module runs be more isolated from each other and msfconsole, which frees us to improve performance by taking advantage of multiple processor cores and reducing the code each module has to deal with. Second, we want people to be able to use the right languages and libraries for each module without having to port it to Ruby or worry about compatibility with all the other things in Metasploit (though if you want other people to use your module, you still have to worry about portability at least a little bit).

New tools

In order to test our design for external module architecture, we have used it to add capabilities to Framework that would have required porting or reimplementing a large amount of code before. So far, this has focused on using existing Python libraries, but we have also started work on running modules by themselves and extending our Ruby support.

Impacket

PSEXEC is fun, but sometimes using WMI or DCOM instead of vanilla SMB gets more shells (and getting more shells is always more fun). Until recently, the best options available were the dcomexec.py and wmiexec.py scripts from Impacket. Thanks to zeroSteiner, those scripts have been added as Metasploit modules. These modules still have a dependency on the development version of Impacket, so some environment assembly is still required. We are working toward getting Python and major libraries added to our Omnibus builds, which should make these modules widely available without any extra work.

Weird dependencies

The world of Enterprise(tm) software can get a little weird at times. Whether you find a debug server or an uncommon RDBMS, getting to the objective of a pen test in a dev shop can require interfacing with libraries you have to have the patience of a developer to install and use. ODBC drivers are no exception. Our own Ted Raffle persevered when confronted with a Teradata server and then wrote some modules to make your life easier. You still need to register on a website to download the drivers, and the setup can be a little weird, but if that is where the crown jewels are for your pen test, at least we have a client written for you.

Look, Ma, no msfconsole!

Since external modules are external to run in separate processes from the Metasploit console process, it is possible to run them from the command line without the Metasploit console process. Python modules can run themselves, and any kind of external module can be run with the tools/modules/solo.rb script, including those we haven't thought of yet. Both implementations use their respective languages' native argument parser and behave as idiomatically as they can while taking the same options. Logging messages are always sent over stderr and reports are always sent over stdout. Until we integrate this with the in-progress external data service for Metasploit 5, reports are formatted as JSON with a small message (which I envision keeping around as an option). Future improvements will include output customization (e.g., verbosity, report formatting); integration to the remote data service; and may even include taking argument values from files or standard input (like curl(1)).

It's not just for Python

Denial-of-service (DoS) attacks like slowloris are simple in theory: You open connections to a server (cheap for you) that convince the server to allocate a large amount of resources (like memory or processes) and you then hold that connection open for as long as you can. A successful run is a little trickier; you need to configure your box to allow a large number of outbound connections from your program, enough to consume so many server resources that you crash it. If you don't, you can easily crash your own program when the operating system denies your requests for more resources. This isn't too bad when running a one-off script or a proof of concept, but when the DoS is being run from inside something larger, the whole application might crash or at best become useless. Our original SMBloris module had checks to prevent this from happening, but that also made it less effective at denying service. The isolation of external processes is a huge advantage here, since it prevents process-wide problems like file descriptor exhaustion from affecting the control process that you interact with.

Anatomy of an external module

The overview and gritty details of how the RPC calls work are documented on our GitHub wiki, but what does the process of actually building a module that uses that look like? Our own jrobles-r7 has written a comprehensive guide for Python modules. The basic structure is the same for Ruby and Python external modules and is similar to how the old style of modules works. Below is an example from the Python guide, annotated:

#!/usr/bin/env python3
# -*- coding: utf-8 -*-

First, the shebang and encoding. Redirecting through /usr/bin/env ensures we work in a wide variety of environments, and the UTF-8 encoding makes sure any weird characters are handled as sanely as possible. If possible, we encourage being compatible with both Python 2 and Python 3 and using #!/usr/bin/env python. Otherwise, use python2.7 if you need Python 2 (most notably, macOS does not ship with a python2 executable).

# standard modules
import logging

# extra modules
dependencies_missing = False
try:
	import requests
except ImportError:
	dependencies_missing = True

from metasploit import module

The import section is pretty standard. Two important things to note: You are welcome to use libraries outside the standard distribution if you need them, but make sure they are in a try: ... except ImportError: block so that your script can live to report an error to the user when run from environments without that library; the metasploit.module module has all the magic that makes your program work, so it always needs to be imported.

metadata = {
	'name': 'Python Module Example',
	'description': '''
    	Python communication with msfconsole.
	''',
	'authors': [
    	'Jacob Robles'
	],
	'date': '2018-03-22',
	'license': 'MSF_LICENSE',
	'references': [
    	{'type': 'url', 'ref': 'https://blog.rapid7.com/2017/12/28/regifting-python-in-metasploit/'},
    	{'type': 'aka', 'ref': 'Coldstone'}
	],
	'type': 'single_scanner',
	'options': {
    	'targeturi': {'type': 'string', 'description': 'The base path', 'required': True, 'default': '/'},
    	'rhost': {'type': 'address', 'description': 'Target address', 'required': True, 'default': None}
	}
}

The metadata looks a lot like the classic Metasploit layout, but there are some key differences. The biggest difference is the new way options are registered. Instead of being defined programmatically, options are now declared in the same place as the rest of the metadata and all the attributes are clearly labeled. The types are all the same types that are used by Metasploit, and when run inside msfconsole, they will be validated by the same code. Since all the options are passed to the module via JSON, if you have an option that might have non-UTF8 data (like a binary payload), you will need to make sure it is base64 encoded before it leaves Framework (see the remote exploit template for an example).

Other differences are more subtle. There is now a type entry that is used by Metasploit to determine how it should expect the module to behave. These type entries map to templates that Metasploit uses to generate the shim layer that proxies between the external module interface and the current module system. For now, this is mostly used to include certain mixins for the shim so it acts more like a standard module, but as we continue to develop this interface, we hope to use these types natively for option normalization or even replace them with a full-on capabilities system.

Module metadata in Metasploit has been overloaded with a lot of functionality needed for a wide variety of cases, limited mostly by how comfortable the original author was adding it. Because of this, there is a lot of extra metadata functionality that we don't implement yet either because we thought implementing it would greatly complicate the design and we found a way to do without or because it is something we have not needed yet. If you need something that is not there, open a ticket to tell us what you need and what you are planning to do with it, and we will work with you to add a solution.

def run(args):
	module.LogHandler.setup(msg_prefix='{} - '.format(args['rhost']))
	if dependencies_missing:
    	logging.error('Module dependency (requests) is missing, cannot continue')
    	return

This is the first part of the callback we will pass to metasploit.module.run. It is mostly boilerplate, but since your code is always in control of how your module runs, we need to specify it here until we think of a better way. If you opt to use metasploit.module.log, you can skip the LogHandler setup, and if you don't have external dependencies, you can skip the rest.

	# Your code here
	try:
    	r = requests.get('https://{}/{}'.format(args['rhost'], args['targeturi']), verify=False)
	except requests.exceptions.RequestException as e:
    	logging.error('{}'.format(e))
    	return

	logging.info('{}...'.format(r.text[0:50]))

And at last we get to your code. Make sure to catch exceptions and log frequently so that the user knows what is going on!

if __name__ == '__main__':
	module.run(metadata, run)

Finally, we have the bit where the RPC code is started. Here we pass the metadata and callback we defined above to the library so it can use them to generate the required RPC messages or command line parser.

Thinking about contributing to Metasploit? Questions, comments, observations, or rebuttals? Reach out to us and the community on Slack or Github.