Last updated at Mon, 01 May 2023 23:26:52 GMT

Rapid7 is proud to announce that Nexpose's 5.1 web application scanning capabilities can now detect all types of vulnerabilities in OWASP's Top10! We've completed this task with the addition of two new vulnerability checks, A5: Cross-Site Request Forgery (CSRF) and A8: Failure to Restrict URL Access . The next paragraphs will describe what we've done to detect each vulnerability and why we think these are the best techniques you'll find in any tool in the vulnerability management market.

Most of you have never faced the problem of trying to detect Cross-Site Request Forgeries nor Failure to Restrict URL Access in an automated manner and with low false positives/negative rates. Believe me when I tell you: it is not an easy task.

There are many techniques one can use to identify these vulnerabilities. The first and easiest one that comes to one's mind is: “We could fingerprint the target Web application, have a database of applications which are vulnerable to A5/A8 and report a vulnerability when there is a match”. While this solution does work, it will report the vulnerability only if:

  • The scanner knows how to fingerprint the Web application
  • The scanner knows that the fingerprinted application is vulnerable

In other words, this solution is only useful for a very small set of the Web applications the user is going to scan. We looked at this approach and concluded that we can do better than that for our customers.

After some serious thinking we got our requirements straight. We wanted our A5 and A8 analyzers to be able to identify these vulnerabilities in any Web application, not just those that have been fingerprinted. Sounds easy, but the implementation was complex!

A8 - Failure to Restrict URL Access


Understanding this vulnerability is fairly simple: most Web applications have a section that can be accessed only after the user has logged in using his credentials. The vulnerability appears when a non-authenticated user is able to browse to one of the “authenticated URLs” and access private information or perform actions that are only meant to be done by an authenticated user. This usually occurs because of a lack of authorization controls: a validation that should be performed in the application's code and was forgotten by the developers.

The algorithm we designed follows these main steps:

  • Crawl the web application without credentials and store URLs
  • Login to the web application with the user provided credentials
  • Crawl the web application keeping session state
  • For each new URL that's found, verify if it was previously found during the non-credentialed scan. If it wasn't then it is one of the URLs that should only be accessible after logging in
  • For that URL perform a request without any credentials/cookies and compare both responses
  • If the responses for the non-authenticated and the authenticated HTTP requests are very similar, then we've identified a failure to restrict URL access

We found that special attention was needed for the handling of images, redirects, javascript, directory indexing and CSS files. As expected, this vulnerability check required various development - QA cycles to refine it before achieving a final version.

A5 - Cross-Site Request Forgery (CSRF)


Cross-Site Request Forgery vulnerabilities, perfectly explained here and here, are tricky to identify if we want to keep a low false positive rate. Keep in mind that technically Google's search form is vulnerable to CSRF; using that as a starting point Rapid7's Web application security team had to be particularly careful in deciding which forms and links were going to be flagged as vulnerable.

In order to identify the vulnerability, Nexpose is going to follow these main steps:

  • Crawl the web application without credentials, store non-authenticated forms
  • Login to the web application with the user provided credentials
  • Crawl the web application keeping session state and store authenticated forms
  • Clear session cookies
  • Login to the web application with the user provided credentials
  • Crawl the web application keeping session state and store authenticated forms
  • For each form that can only be accessed after authenticating, analyze if it has an anti-CSRF token. The basic way of doing this is to check if it has a hidden parameter that considerably changed its value (using the two form instances gathered during steps 3 and 6)
  • Report forms that don't have anti-CSRF tokens as vulnerable to CSRF

Another approach includes performing HTTP requests to verify that the anti-CSRF token is properly verified by the application, but extensive testing showed that the accuracy of that heuristic was slightly lower than the one of the implemented algorithm and speed was reduced due to the need of more HTTP requests.

As you already noted, in order to detect these vulnerabilities in a generic way, the web application security scanner needs to have the capability to login to the Web application using user configured credentials, re-use session cookies and acquire a new session if for some reason the old one expires. Luckily Nexpose already supported all that before we started to work on detecting them, so the path to success was much simpler.

To sum up, Nexpose is now able to identify Cross-Site Request Forgeries and Failure to Restrict URL access in any Web application after the user provides login credentials, which gives us OWASP Top10 coverage!