Connected QA: Selenium + Log Analysis

Last updated at Fri, 03 Nov 2017 20:45:59 GMT

Does the idea of adding Quality Assurance (QA) test runs to your log analysis platform sound strange? It’s actually not! The value in adding QA test runs goes beyond helping the QA team; it helps the entire development shop as they move to a DevOps framework. 

I talk a lot about the integrated development environment and going beyond ad-hoc tool selection to a more deliberate and holistic approach. This also means bridging the gap between teams—Dev, Ops, QA and Delivery—because each team has their own powerful set of tools, and unless these tools are fully integrated between all teams, the delivery pipeline will remain broken.

QA’s Role in the DevOps Framework

DevOps leads us to look not only at individual releases, but the release of releases and the overall delivery pipeline. Over time, we find ways to enhance the entire process.

A critical element of the DevOps framework is ensuring software quality, something that has become everyone’s responsibility. The result of this shift means the role of the QA group must evolve to include facilitating test strategy and automation. They are in the critical role of touching all aspects of the application before it hits production and quickly addressing issues should they make it to production. The other element of their responsibility is communication: they must communicate with front-end development, back-end development and IT.

While the development and ops team have hopefully empowered themselves with a robust logging platform to provide a real-time view into the production and pre-production application releases, QA is often not given access to this platform. If they do receive access to the logging platform, it’s for the purpose of investigating issues that development and operations have stored in the logs. But, ultimately, there is no reason not to bring QA into the broader activity.

How Log Analysis Improves QA Processes

Log analysis is not only a way to report on QA activities, it also offers a way to improve QA processes, catch issues faster, identify potential issues before they grow and improve communication with the rest of the group.

Selenium is the number one tool used for QA automation and is the point where we should integrate the powerful analytics platform with QA automation. Selenium is a combination of libraries and web drivers that emulate user interactions and allow QA teams to create functional testing scripts in the language of their choice.

The benefit of tying log analysis into these test scripts is:

1. Observing how QA teams improve over time;
2. Using metrics for test coverage;
3. Catching issues in test runs more quickly;
4. Spotting potential issues before they grow;
5. Identifying portions of your test suite that are slow or old;
6. Creating a time series of bugs caught over time;
7. Correlating tests with application infrastructure and application logs; and
8. Integrating testing data with delivery tools and integration environments.

The method used to accomplish this is taking the API of the log analysis platform and making calls to it at the beginning of every large test block in your Selenium code. Ideally, each call identifies the test name, system date and time, release version, test start and stop times. For most applications, though, there are several additional pieces of metadata that should be included. Also in your scripts try{} catch{} blocks; it’s preferable to make calls with the entire stack trace or exception message.

When this is done, you’ll be able to access a time series in the log platform of test runs, their results (good or bad), and details of their performance.

require_once dirname(__FILE__) . '/le_php-master/logentries.php';

<?php 
class WebTest
{    
    public function testTitle()
    {
        $log->“Title Test : Run On ” . time() . “ABC App : v1.05”;
        $this->setBrowser('firefox');
        $this->setBrowserUrl('http://www.example.com/');
        $this->assertEquals('Example WWW Page', $this->title());
        $log->“Title Test : Test Finish ” . time() . “ABC App : v1.05”;
    }
}
?>

That’s it! Just a few lines of code, and you have empowered your entire QA/QE team as well as increased visibility across the entire development organization.

This integrated environment is a must for any organization moving to the modern development shop model and considering a DevOps framework where tools across teams tie into the one foundational platform for log analysis. Because a huge focus of the DevOps framework is quality, tying analytics platforms to QA activity and performance is as important as system and application logs. You’ll have a better picture of how your test cases are running, enhance QA communication and have an overall view of how your team has addressed application quality, the performance of testing infrastructure, scripts and processes over time. And the best part? It’s easy!