12 Days of HaXmas: Diving Into Git for Current and Future Metasploit Devs

Last updated at Fri, 12 Jan 2024 15:44:15 GMT

This post is the eleventh in a series, 12 Days of HaXmas, where we take a look at some of more notable advancements in the Metasploit Framework over the course of 2013.

Make no mistake -- the initial learning curve for git and GitHub can be pretty hairy. Way back in 2011, we made the initial move to GitHub for our source code hosting, but it took us until 2013 to remove the last vestiges of our old SVN infrastructure. In the meantime, we've picked up a fair amount of git and GitHub smarts. For HaXmas, I'd like to share with you some of the nuggets I've incorporated in how Metasploit uses git and GitHub. I know the guys over at nmap are considering the move, so maybe they can get a little ahead of the game with this post.

SSH Aliasing github.com

I heart strong crypto, so for talking to GitHub, I use public key authentication by creating some aliases in my .ssh/config. I also use different keys for different accounts, so if I manage to get owned on one, I don't automatically sacrifice the other. The relevant bit of my .ssh/config is here, and so when ever I need to create new mappings to remotes to GitHub, I just use "github-r7" for a username and password and it's all taken care of behind the scenes. For example, if I want to start tracking Juan's Metasploit fork, I'd just use:

git remote add -f jvazquez-r7 github-r7:jvazquez-r7/metasploit-framework.git

If Juan was kind enough to give me committer access to his fork, I could then go and push things to his fork directly. If he didn't, no big deal, I just get read only -- and as you'll see later, I wouldn't often (ever?) want to write to Juan's fork anyway -- I'll just send him Pull Requests. (In GitHub parlance, a pull request is a suggested change, feature, bug fix, documentation change, etc. -- we'll see more about those in a second.)

Origin vs. Upstream

In git, the names of remotes (places where copies of your source code live) are not supposed to be reserved or special. However, "origin" is usually treated specially -- it's normally where your default remote repo lives. This is great if you're the only one who works on your code, and you only occasionally get pull requests from others.

For Metasploit, we have a few folks with commit rights to the main repository, but because so many people fork and pull from it throughout the day, we avoid editing it directly too much. Plus, most people who contribute Metasploit code don't have direct committer rights.

Metasploit developers always treat "origin" as their own fork of Metasploit (where they can write), and "upstream" as the main rapid7 fork. This is true for bothprivileged committers and unprivileged pull requestors (aka, random hackers from the Internet).  That way, we all can use the same terminology when we're talking about pushing code to GitHub, and everyone communicates changes to Metasploit in the form of pull requests.

In my .git/config file for metasploit-framework, the two repos look like this:

[remote "origin"]
  fetch = +refs/heads/*:refs/remotes/origin/*
  fetch = +refs/pull/*/head:refs/remotes/origin/pr/*
  url = github-r7:todb-r7/metasploit-framework
[remote "upstream"]
  fetch = +refs/heads/*:refs/remotes/upstream/*
  fetch = +refs/pull/*/head:refs/remotes/upstream/pr/*
  url = github-r7:rapid7/metasploit-framework

This is not a default config; you need to edit it up yourself if you want to play along. Notice how I'm using "github-r7" as a user@server string -- as stated above, that's defined in my .ssh/config.  Also notice the magic pr refs line in the middle of each remote definition. That lets me do things like 'git checkout upstream/pr/1234' to get a copy of PR #1234's code locally. Super useful and that really speeds up development quite a bit, since I don't usually have to track everybody's remote seperately.

Signed commits

One of the greatest features of git, as far as security goes, is also one of the least used: the ability to PGP/GPG sign each commit that lands on any branch. Setting up git signing is pretty straightforward, and potentially offers a ton of integrety to the codebase. For Metasploit, I maintain a a registry of committer keys so anyone who cares to can verify that merges to Rapid7's master branch are really and truly from the people they're claiming to be, and not from someone who has managed to guess one of our passwords (or from someone simply pretending to be someone else via "git commit --author=HD Moore" or something).

Unfortunately, I say "potentially" for a reason -- while signing is supported today, it's surprisingly difficult to have a rule like, "Reject all unsigned merges" for your local checkout -- you basically have to use your eyeballs, or hack up some script that will parse the text output. At least, I haven't found a good way to do it yet -- if you know, tell me!

Useful Git Aliases

So let's pull all this together with some useful git aliases. These live in either your ~/.gitconfig or in your path-to-metasploit-framework/.git/config files (the latter will supersede the former in the case of conflicts).

It's often said around the Rapid7 office that some day, a really great versioning system will be built on top of git. In the meantime, we get by with a selection of aliases that pretty much anyone can start using right away.  If you want all the details of what I use for day-to-day development on Metasploit, you can peep the details of my ~/.gitconfig and my metasploit-framework specific .git/config files. Here are the highlights, though:

  m = merge -S --no-ff --edit
  c = commit -S --edit

With these, I never type "git commit" or "git merge" anymore; just "git c" and "git m". Under the covers, I'm always signing all of my commits with my GPG key, and I'm always getting the opportunity to edit my commit message (instead of using git's sometimes useless defaults).

  publish = push upstream upstream-master:master

Because we don't ever edit rapid7/metasploit-framework master branch directly, we never treat the rapid7 repository as "origin." This makes pushing to rapid7's master branch a little tricky, though, so this alias saves me the trouble of having to remember the syntax for how to push my merges once they're ready to go.

  tag-sprint=!"git tag -s $1 -m \"Tagging as $1\" && git push --tags upstream upstream-master:master && echo Tagged as"

Pushing tags, which could vaguely be thought of as version markers, is always a little weird in git's syntax, and when we're operating in our own particular upstream-instead-of-origin paradigm, it's even harder.  This alias automates that up pretty well, though. Now, I just type "git tag-sprint sprint-D02" and it does what I expect.

  pr-url =!"xdg-open https://github.com/$(git config github.user)/$(basename $(git rev-parse --show-toplevel))/pull/new/$1:$2...$(git branch-current) #"
 

This is the single most useful alias I have; with this, I can issue a pull request direct from my command line against anyone's branch of Metasploit Framework, or any other git repo I have a fork and local clone of. So, say there's a pull request in for some Metasploit module, from Kernelsmith, but I want to fix up the print_status statements; instead of complaining about it, I can fix it myself and send him the results to incorporate in his branch. Assuming Kernelsmith's branch is called "foo-module" and is part of pull request #1234, the procedure for this would be something like:

git checkout -b fix-grammar-foo-module --track upstream/pr/1234
gvim modules/exploits/windows/foo.rb # and save of course
git add -p
git c
git push origin
git pr-url kernelsmith foo-module

Once he reviews my pull request to his branch and lands it, the pull request over on Rapid7's repo will be updated automatically. This is exactly the kind of thing that makes git crazy powerful as a collaborative engine. Anyone, not just people with commit rights, can make suggestions for anyone's code. I can PR to you, you can PR to me, we can work things out totally outside of the rapid7 repo, and when we're done, one of us can fire off a pull request that retains all our respective commit histories.

Again, there's bunches more of these aliases in in my junk drawer, but hopefully these are enough to get you comfortable with dreaming up git aliases.

GitHub Account Security

We use signed commits so we can prove that code changes really come from us, and we also use seperate accounts for committing to Rapid7-authored source code -- if you pay any attention at all to our logs, you'll see that I'm "todb-r7" and not just "todb" when I commit to the Framework.

The reason for this is that when you associate your GitHub username to an e-mail address, any of those e-mail addresses can complete a password reset if you (or an attacker) forgot your password. This is kind of a huge bummer, because if any of your associated e-mail addresses gets compromised, it's a straight shot to a password reset. Therefore, all *-r7 accounts are associated with one well-controlled and well-monitored mail server (different accounts, of course), and we don't have freemail/webmail accounts associated with them at all. We've asked GitHub to at least enable some kind of flag that says "Only allow password resets from this e-mail address," but it's not been implemented yet. Boo.

That said, the risk of actually getting owned this way is a lot smaller today than it was two years ago, though, thanks to GitHub's most excellent two-factor authentication (2FA). It works pretty much the same as Google's 2FA, where you either get a SMS text or generate a code locally. Note that if you're a T-Mobile customer, or if you need to whitelist SMS sources via shortcode, GitHub's shortcode is "448-482" (GIT-HUB). One super nice feature of GitHub's is that if you're an organization owner, you can check to see which members of your organization have 2FA enabled simply by checking your organization's members page.

Getting Started Yet?

Using git and GitHub effectively for code sharing is kind of hard, especially if you have some habits from SVN and earlier source code control systems. But, I hope this post demystifies some of the plumbing a little bit. It took us probably 3-6 months to really get into the swing of things, and we still find some of the darker corners of git somewhat terrifying. So, if you're a n00b, you're definitely not alone. But, if you're willing to learn how to use the world's most popular collaborative open source development platform, we have some documentation at http://r-7.co/MSF-DEV that hopes to help make setting up an environment as straightforward as we can. We're also in the midst of putting together a solid Chef recipe that might make things a little easier.