Digital Disruption - How to Get Ahead and Maintain Software Quality

Companies and organisations have found that they are able to disrupt well established markets and grow rapidly at the expense of others by better exploiting digital markets and channels. Essentially these organisations are not constrained by the architecture of legacy systems and related technical debt nor by outdated processes and siloed IT departments. Typically, they are embracing processes such as Continuous Integration (CI) and even Continuous Delivery (CD) that enable rapid solution delivery, establishing a presence quickly across multiple channels and then subsequently reacting to the market direction with frequent releases. 

The rest of the business world initially looked on with a mixture of envy and trepidation and are only now working to try and adapt to make their IT capability more proactive. In many cases the response has included making the cultural changes associated with DevOps, breaking down traditional silos between IT functions and implementing those same CI/CD processes. 

As with all seismic changes in the IT industry the implications of this new world, for those of us focussed on Quality Engineering, are far reaching. Just how do you adapt your testing processes to survive and indeed thrive in this new world? 

Continuous Testing

The answer to this challenge is to implement processes supporting Continuous Testing, ‘shifting left’ (testing earlier) whenever possible to make Quality a full lifecycle activity. 

Making testing a full lifecycle activity requires greater collaboration and a change to the mindset. We need to move from the old concept of ‘segregated validation’ where solutions are delivered into separate testing phases beyond development, to tight feedback loops with our delivery and support colleagues in parallel. Quality can no longer be incorporated either into subsequent sprints in a manner sometimes referred to as ‘agile-fall’, which is arguably just a ‘broken’ agile approach. 

BDD Ensuring Traceability

More than ever we need to make sure our tests are clearly driven by and linked to requirements by embracing techniques such as Behaviour Driven Development (BDD). There should be no room for Quality Engineering to be compromised by poor requirements in modern software delivery and traceability should be a given. Using the structure language (‘Given…Then…When’) to specify detailed requirements is a powerful technique and a great aid to us in testing – giving us the sound test basis we crave. 

Ruthless Automation

A key requirement to enable the speedy software delivery whilst maintaining quality, is that each of the disciplines across the SDLC must automate whenever possible. Gartner refer to this approach as ‘ruthless automation’. The Agile methodology emphasised the need for efficient test automation with the frequent regression required, but this is of paramount importance in making possible Continuous Delivery. Our automated test packs now need to be executed in very tight windows to maintain the speed of delivery. This almost certainly requires focus on the low level and integration layer scripts with fewer scripts and a lighter touch through the GUI. The reliability of the scripts also needs to be beyond question as any ‘false failure’ almost inevitably means a delayed release. The BDD approach described above should be inextricably linked, acting as an enabler and providing acceleration and transparency to our automation efforts. Our scripts must also be capable of execution against multiple platforms (browsers, laptops, tablets and mobile devices) to avoid duplicating effort. 

Continuous Performance Testing

Performance Testing equally needs to become a full lifecycle activity, with checks to make sure individual components are performant undertaken as soon as they are built. An up-front assessment of the risks and potential bottle-necks to direct our testing efforts takes on even greater significance. Our load tests must be automated to a greater degree than was previously expected so that these can also be invoked whenever necessary. 

Environment and Data Management

Providing the required mixture of persistent and temporary (those that can be routinely provisioned, configured, used to test and then torn down) realistic test environments will likely require a mix of cloud, on-premise and hybrid solutions and perhaps making use of container technology. 

In order to be able to support Continuous Testing our test data needs to be realistic and aligned to our test cases, deployed flexibly to many environments and in order to maintain velocity, available in an instant. At the same time, we must comply with all regulations (including the imminent GDPR) by ensuring sensitive data is anonymised/obfuscated when necessary and access/storage is strictly controlled. A new generation of Test Data Management (TDM) tools are emerging to help support this activity that we need to exploit. 

Opportunities in this New World

This new world of modern software delivery, ushered in by the impact of digital disruption, offers tremendous opportunities both to improve solution quality at the desired velocity and develop new capabilities as well as presenting the Quality Engineering world with the challenges described above. The opportunities can be roughly broken down into the categories of; people, process and technology and each of these will be considered in more detail in subsequent Roq articles which will be available soon. In our next instalment, we will discuss the ‘people’ aspect and consider whether we need to effectively make a move from being mere Testers to full blown Quality Engineers. 

Steve Mellor – Test Architect, Roq