12 Days of HaXmas: Buffer Overflows Come and Go, Bad Passwords are Forever

Last updated at Tue, 16 Jan 2024 16:02:07 GMT

This post is the fourth in a series, 12 Days of HaXmas, where we take a look at some of more notable advancements and events in the Metasploit Framework over the course of 2014.

This summer, the Metasploit team began the large undertaking of reworking credentials throughout the project. Metasploit, as you already know, began as a collection of traditional exploits. Over the years it has grown into much more than that. Credentials were first introduced into Metasploit in the form of Auxiliary Scanner modules meant for bruteforcing. The data model that was created to support this was designed with only this use case in mind. The result was that our concept of creds was as follows:

• A Username
• A Password
• The Service it was valid on

This was an unfortunately limited and limiting approach. As Metasploit continued to grow and develop, Credentials kept cropping back up. We found new ways to find, validate, and exploit credentials. As we continued to grow we found ourselves chaffing under those constraints. There were a number of issues with this concept of creds:

• You couldn't have a Credential without knowing exactly what service it was for
• The same username and password on a different service was a different credential. This means you couldn't see all the services a given credential pair was valid on, easily.
• Several forms of authentication required additional components to be valid. For example an AD Domain Name, a PostgreSQL Database Name, or an Oracle SID.
• There was no way to store incomplete credentials. You couldn't get a list of valid usernames from AD, then use those to try and bruteforce their specific passwords.
• There was no good way to store Password hashes that were not replayable like SMB NTLM Hashes were. This made it difficult to properly store those hashes for offline cracking
• There was no normalization of credential data in the database, making it extremely inefficient to store and retrieve data about Credentials from the Database.

These were just some of the problems we encountered. We had to find dirty hacks to try and make things fit every time we wanted to do something new with Credentials. On top of that, the code responsible for dealing with creds, reporting them, bruteforcing for them, etc was showing its age. The code was diversified, with several different implementations in place, hacks in some places holding it all together. It was time for a change

A Fresh Approach

We began the work of redoing Credentials from the ground up. We had to redefine what a Credential meant. We identified three basic parts of a Credential

Public: A Public is the part of a Credential that is publicly known; typically a username. Currently in Metasploit, there are two type of Public, a Username and a BlankUsername. The Metasploit::Credential::Username class holds a known username (duh?). There should only ever be one instance of BlankUsername at a time. It represents a case where the username is not really needed for the credential, such as the case with VNC.

Private: A Private is the part of a Credential that is secret. We currently have several types of Privates in Metasploit:

• Password – a plaintext password
• BlankPassword – a single instance object used whenever we say that the password is Blank or doesn't matter
• SSHKey – an SSH private key
• NTLMHash – an NTLM hash is special because it is a password hash that can be replayed against SMB as a valid authentication mechanism. This is the basis of the “Pass the Hash” attack
• NonreplayableHash – This is all the other kinds of password hash. These are not usable for things like bruteforce or exploits like tomcat_mgr_deploy, but can be used by our offline cracking modules to try and derive plaintext passwords.

Realm: Realm is one of the trickier, and more confusing parts of our new credential concept. We mentioned earlier that some scenarios require some additional context for authentication. Let's take the example of an AD Domain. If you have a user  Administrator and one contosso\Administrator, those are NOT the same account. The first one is probably a local admin on a box, while the second one may be the Domain Admin. Chances are, hopefully, that they also will not have the same password. It is therefore important to clearly distinguish that these are not, in fact, the same creds.

Credential Core: We tie all of these components together into what we call a Credential Core. A credential core associates the components together to form a single credential, as the same individual components could belong to many different credentials

With those core pieces in place, it is time to extend the capabilities of the raw data model, beyond just recording that a Credential exists. The next step can be thought of as the Look Ahead and Look Behind operations of a Credential. That is to say “where did this Credential come from?” and “Where does this Credential work?”

Origin: A Credential Core will always have an Origin. The Origin represents how we first learned about this specific credential core in this workspace. There are currently several types of credential Origins

• Service: A Service Origin means that the Credential Core was obtained through remote interaction with a network service. This will usually be from either an Auxiliary Module or an authenticated exploit that succeeded. The Origin will contain the full name of the module that was used, as well as an association to the service it was run against to get that credential.
• Session: A Session Origin means we got the credential from post-exploitation methods on an open Session. It will contain the full name of the post module that was run, as well as an association to the Mdm::Session record, for the session it was obtained from.
• CrackedPassword: A CrackedPassword origin indicates that the Credential Core was obtained by using Offline Cracking against either an NTLMHash or a NonreplayableHash Credential. It will contain an association back to the original credential Core that it was cracked from. Because the Original Core will also have an Origin, this means we preserve an entire chain of origin on the Credential. For example, if you have an NTLMHash that was obtained from a Meterpreter Session then crack it. The Plaintext version of the Core will have a CrackedPassword Origin pointing back to the Hashed version of the Core. The hashed version of the Core will have a Session Origin that will tell you it came from the hashdump module run against a given session. The Session object will tell you the details of how that session was opened. The entire history of how you got from running a given exploit to obtaining that plaintext credential is now recorded.
• Import: An Import Origin indicates that the Credential Core was imported from an external file. Metasploit supports importing data from a number of different sources, and also has the ability to export Credentials in it's own format and reimport them.  The Import Origin also contains the name of the file that was imported.
• Manual: A Manual Origin indicates that a user manually entered that Credential in. In Metasploit Pro, Express, and Community the Manual Origin will also contain an association to the User who entered it.

Login: A Login is the association between a Credential Core and a Service it is believed to be valid on.  A Login will also contain two other useful pieces of metadata. It will tell you the last time it was attempted, and it will contain a status. There are several different accepted statuses for Logins currently:

• DENIED_ACCESS: This status is set on the Login when the credential was valid but had insufficient permission to login or perform the requested action
• DISABLED: this status is set when the credential was valid but the account has been disabled.
• INCORRECT: this status is set when the credential was not actually valid for the service.
• LOCKED_OUT: this status will currently only ever be set on SMB Logins as it is the only service that will give us this level of detail. It is set when the account is locked out.
• NO_AUTH_REQUIRED: This status will be set when the service does not actually appear to require auth. Indicates that the Login was a false positive due to the service accepting any authentication.
• SUCCESSFUL: pretty self-explanatory
• UNABLE_TO_CONNECT: This status is set when the last attempt to use the credential against this service returned some sort of a connection error.
• UNTRIED: This status is set when we think the Credential is valid on the service, but haven't actually attempted it yet. This will occur when we get credentials from something like post exploitation where we know where the cred is supposed to be valid on.

This may all seem very complicated, but it is extremely powerful. On a conceptual level, Credentials are a very difficult thing to capture accurately. This is especially true because of the conflicting authentication standards between widely differing services. This approach gives us the best flexibility to handle various different scenarios. It also allows us to do some pretty cool things with the data. Let's take a look at a visual example for a second.

In this example we compromise a machine with the ms08_067 exploit, and get a Meterpreter Session. That Session has an Mdm::session record saved to the database. That Session record includes what exploit was used to open it, and the datastore options that were selected for that particular run. Once we have the session we run the Hashdump post module and get some NTLM Hashes. We get one for the local Administrator account. We create the Public with the Username ‘Administrator', we create an NTLMHash private as well, and then create a Credential Core tying them together. When we create the Core, we also create a session Origin that says this core came from running ‘post/windows/gather/hashdump' against Mdm::Session 1. Now we run ‘auxiliary/analyze/jtr_crack_fast' which feeds those NTLM hashes into John the Ripper and successfully cracks the Administrator hash. We create a new Password object(‘Password1') and create Credential Core 2 which has that Password and uses the same Public object as Credential Core 1. Core 2 has a CrackedPaassword Origin which points back at Core1. We then take Core 2 and use it against an SMB Service on a Second host and successfully login, so we create Login1 which connects that service to Core 2 with a Successful status and a last_attempted_at timestamp of the exact time we successfully logged in.

Wheew, that was a mouthful. Let's see what that means looking at it in reverse now. You're getting ready to write up some findings, and you take a look at your data. You see that you have a successful Login on host2. It was a Login on the SMB Service, and you know exactly when it was that you Logged in. You see that you Logged in using Administrator:Password1. How did you get that credential? Well you cracked it from this NTLM Hash. Where did you get the NTLM Hash?  You got it by running Hashdump on the Meterpreter Session on Host1. You know exactly when you did that to because the Origins have timestamps. How did you get that session? Well you got it by running ms08_067 against that host, at this specific time, with these specific settings. See how powerful that is? You can show an entire chain of events, and analyze exactly what they mean for the environment you're testing.

LoginScanner: Bruteforcing Evolved

With our Credential modeling now completely redone, bruteforcing needed to level up as well. The old Auxiliary Scanner modules for creds had become a tangled mess. There were different approaches in each one, and we need a sane way of rationalizing exactly how we do bruteforce. Additionally we wanted clean, performant code that was covered by automated tests. This is the crucible in which we formed the LoginScanner classes. I won't bore you with as exhaustive a detailing of the inner workings of the LoginScanner classes. The important parts are that all LoginScanners derive from a common base which defines the standard behavior for a LoginScanner. This include things like Bruteforce Speed, Timeouts, the way it takes credentials etc. Each module then  instantiates the correct  LoginScanner class, passes in the correct datastore options, and then calls the scan! Method, and passing it a block. The scanners all take a block for each login attempt result, so that any module that uses them can decide what those results mean. This makes the LoginScanners reusable and extremely flexible.

You might ask what this means for you as a user. It means predominantly two things: The reliability of the LoginScanner modules is greatly increased. Each one should behave in the same general way. If you know how to use one, you know how to use all of them. The second is that the LoginScanners are now significantly faster and more efficient. In fact, it actually caused me some problems while making sure they were Timeout safe. I wrapped some scanners in a 1 second timeout just to make sure they cleaned up properly. To my surprise the Postgres LoginScanner completed before the timeout triggered. It turned out that the Postgres LoginScanner was now capable of making approximately 12 login requests in a one second period. Your mileage will vary based on network conditions and the protocol you are using, but there is no doubt that these new LoginScanners are capable of some serious speed compared to their predecessors.

Looking toward the New Year

This work is far from finished. This undertaking is the single largest rework Metasploit has seen since the conversion from Perl to Ruby. We bit off quite a chunk of work, and we are far from finished. There are still modules out there that use the old credentials. This will be one of our main focuses in the near future. We will also be looking at leveling up msfconsole's capabilities in terms of looking at the creds you have saved. Things like offline cracking will probably get additional attention in the future as well. The fact is that Credential based attacks are one of the most prevalent breach vectors. You can patch every system in your environment. You can hook up IPS and AV at every art of your network One weak credential can still bring your entire Organization to its knees. Buffer overflows come and go, bad passwords are forever. Merry Haxmas!