The selection and development of test scenarios is one of the most critical components of any software testing strategy. Test scenarios describe what components, functions, interfaces or other pieces of the application a team will test. In addition, they will generate the foundation for the documentation that surrounds those testing procedures.
In this article, we'll examine the technical details behind test scenarios, including their direct connection with test cases, how to develop effective test scenarios that accelerate QA processes and -- perhaps most importantly -- the enormous role they play in determining which tests are sensible targets for automation.
We'll also review which team members should be involved in these types of decisions, as well as a technique known as the test automation pyramid.
While test scenarios and test cases are often discussed in conjunction with one another, and sometimes thought of as synonymous, it's important to remember that they each play a distinct role in both manual and automated testing efforts.
While the test cases form the coding structure QA teams will use to create automated test scripts, the test scenarios will help those teams determine which specific tests to automate.
Test scenarios are the high-level documentation of what QA professionals will test. They are developed according to business and systems requirements, and describe the functionality and how the user will execute the function.
Test cases, meanwhile, are specifications that provide a detailed, step-by-step documentation of application testing procedures, including a list of expected test results. QA professionals can execute test cases manually or develop them into automated scripts.
The selection and development of test scenarios is a critical component of any software testing strategy, and even more important in automated testing. While the test cases form the coding structure QA teams will use to create automated test scripts, the test scenarios will help those teams determine which specific tests to automate.
Typically, teams will assemble test scenarios using a combination of user stories and user acceptance criteria. However, test scenarios can also stem from personas, which act as an overarching profile of expected user behaviors and anticipated needs. Well-developed personas can help teams create test scenarios that focus on the features they think will be most important to the vast majority of users.
Behavior-driven development (BDD) can be another productive approach. BDD loops the business side into the development process, but still provides focus on anticipated user behavior. For instance, the Gherkin syntax implements the Given-When-Then formula using a plain, easy-to-read language that both software-side and business-side personnel can understand. With the support of the Cucumber BDD testing framework, Gherkin then facilitates the development of automated test scripts directly from the test scenarios it contains.
Once a team identifies their test scenarios, it's time to create an actual testing strategy. This involves determining the specific scenarios the QA team will test manually versus ones they will slate for automated testing. To make effective decisions, this work needs to be done early in the development process.
During this step, it's critical to identify the domain under which the testing will take place. For example, certain regulatory requirements can determine the types of test scenarios the team selects and executes, including the level of detail those testing procedures and corresponding documentation encapsulate. Another relevant factor might be the methodologies currently in use, as well as the general maturity of those methodologies.
While automation has increasingly become a critical part of today's test strategies, not every test scenario will be an appropriate target for automation. Even when merging an automated continuous testing process into a CI/CD pipeline, QA teams should still execute things such as UX-focused GUI tests manually.
However, those teams should choose manual test scenarios carefully. To help clarify this, here are some examples of test scenarios that are appropriate for manual testing:
Alternatively, test scenarios merit automation when critical features require repetitive testing and are simply too hard for teams to handle manually. For instance, tests that involve large volumes of data or complex data entry procedures often fit the bill for automation. And while test cases that run over long periods of time can be complex to automate, it's a labor investment that will significantly accelerate QA processes down the line.
Choosing test scenarios for automation should be a team effort that includes test leads, automation specialists, SDETs (software development engineers in test), developers and product owners. The product owners are especially important to loop in here, as they are uniquely qualified to point out scenarios that most accurately simulate the real-world user workflow. Developers also carry a lot of weight here, as they can highlight the scenarios that require testing on complex collections of code.
One technique that can be extremely helpful in the manual vs. automated testing decision is the test automation pyramid.
To start, the test automation pyramid dictates that automation should primarily reside at the unit and component test level, as these tests are typically the least complex and easiest to automate.
In addition, a high level of unit test coverage can help teams detect defects and implement remedial fixes in the early stages of the development process.
Meanwhile, GUI-level tests reside at the top of the test automation pyramid. Tests performed on functional components like a GUI are usually the most difficult to maintain; therefore, it's best to keep automated GUI tests to a minimum, provided that those targets for automation aren't particularly complex components.
Read more:
The crucial role of test scenarios, especially in automation - TechTarget
- Automation Personnel Services - Temporary Staffing ... [Last Updated On: March 25th, 2016] [Originally Added On: March 25th, 2016]
- Automation | Define Automation at Dictionary.com [Last Updated On: March 25th, 2016] [Originally Added On: March 25th, 2016]
- Automation | Definition of automation by Merriam-Webster [Last Updated On: March 25th, 2016] [Originally Added On: March 25th, 2016]
- Automation | The Car Company Tycoon Game [Last Updated On: March 25th, 2016] [Originally Added On: March 25th, 2016]
- Automation - Wikipedia, the free encyclopedia [Last Updated On: March 25th, 2016] [Originally Added On: March 25th, 2016]
- Automation - Cloud process & workflow automation | Microsoft ... [Last Updated On: June 29th, 2016] [Originally Added On: June 29th, 2016]
- Riverside Automation - Machine Controls [Last Updated On: July 3rd, 2016] [Originally Added On: July 3rd, 2016]
- Automation: The Car Company Tycoon Game Windows - Mod DB [Last Updated On: July 3rd, 2016] [Originally Added On: July 3rd, 2016]
- System Integration | Industrial Automation [Last Updated On: July 3rd, 2016] [Originally Added On: July 3rd, 2016]
- WinAutomation - Smart Macro Recorder, Web Automation ... [Last Updated On: July 3rd, 2016] [Originally Added On: July 3rd, 2016]
- Automation Solutions - Home [Last Updated On: July 3rd, 2016] [Originally Added On: July 3rd, 2016]
- The Automation Conference [Last Updated On: July 3rd, 2016] [Originally Added On: July 3rd, 2016]
- Rohtek Automation [Last Updated On: July 3rd, 2016] [Originally Added On: July 3rd, 2016]
- JL Automation, LLC | Home Automation, A/V Automation [Last Updated On: July 3rd, 2016] [Originally Added On: July 3rd, 2016]
- Four fundamentals of workplace automation | McKinsey & Company [Last Updated On: August 27th, 2016] [Originally Added On: August 27th, 2016]
- Leviton Security & Home Automation [Last Updated On: August 27th, 2016] [Originally Added On: August 27th, 2016]
- EVA Automation [Last Updated On: September 6th, 2016] [Originally Added On: September 6th, 2016]
- News | Automation | The Car Company Tycoon Game [Last Updated On: September 6th, 2016] [Originally Added On: September 6th, 2016]
- Automation - The Car Company Tycoon Game on Steam [Last Updated On: September 6th, 2016] [Originally Added On: September 6th, 2016]
- Test automation - Wikipedia, the free encyclopedia [Last Updated On: September 6th, 2016] [Originally Added On: September 6th, 2016]
- Job Seekers - Automation Personnel Services [Last Updated On: October 8th, 2016] [Originally Added On: October 8th, 2016]
- Custom Automation & Machine Design | Automation GT [Last Updated On: October 31st, 2016] [Originally Added On: October 31st, 2016]
- iAutomation [Last Updated On: October 31st, 2016] [Originally Added On: October 31st, 2016]
- Test automation - Wikipedia [Last Updated On: November 16th, 2016] [Originally Added On: November 16th, 2016]
- Automation - Official Site [Last Updated On: November 19th, 2016] [Originally Added On: November 19th, 2016]
- Beckhoff Automation - Wikipedia [Last Updated On: November 21st, 2016] [Originally Added On: November 21st, 2016]
- Automation - Security Hyperstore [Last Updated On: November 21st, 2016] [Originally Added On: November 21st, 2016]
- IT Automation - BMC [Last Updated On: November 29th, 2016] [Originally Added On: November 29th, 2016]
- ID Automation [Last Updated On: November 29th, 2016] [Originally Added On: November 29th, 2016]
- The Best Home Automation Systems of 2016 | Top Ten Reviews [Last Updated On: December 24th, 2016] [Originally Added On: December 24th, 2016]
- What is Home Automation? | Home Automation Systems [Last Updated On: December 24th, 2016] [Originally Added On: December 24th, 2016]
- Beyond Automation - hbr.org [Last Updated On: December 25th, 2016] [Originally Added On: December 25th, 2016]
- Build automation - Wikipedia [Last Updated On: December 26th, 2016] [Originally Added On: December 26th, 2016]
- Home automation - Wikipedia [Last Updated On: January 10th, 2017] [Originally Added On: January 10th, 2017]
- Automation | Food Engineering [Last Updated On: January 13th, 2017] [Originally Added On: January 13th, 2017]
- Home Automation - Enerwave Home Automation [Last Updated On: January 14th, 2017] [Originally Added On: January 14th, 2017]
- Automation - DESHAZO [Last Updated On: January 14th, 2017] [Originally Added On: January 14th, 2017]
- Robots, Automation, EOAT, Grippers, Conveyors, Guarding [Last Updated On: January 26th, 2017] [Originally Added On: January 26th, 2017]
- Werner Electric | Automation [Last Updated On: January 28th, 2017] [Originally Added On: January 28th, 2017]
- Automationtechies | Automation Engineering Recruiting [Last Updated On: January 28th, 2017] [Originally Added On: January 28th, 2017]
- Automation - Mazak Corporation [Last Updated On: January 28th, 2017] [Originally Added On: January 28th, 2017]
- Automation | Technologies | Systems | Integrator ... [Last Updated On: January 28th, 2017] [Originally Added On: January 28th, 2017]
- Test Automation Services for Development of Regression ... [Last Updated On: January 28th, 2017] [Originally Added On: January 28th, 2017]
- Carlo Gavazzi Automation Components [Last Updated On: January 30th, 2017] [Originally Added On: January 30th, 2017]
- UI Automation Overview - msdn.microsoft.com [Last Updated On: February 5th, 2017] [Originally Added On: February 5th, 2017]
- New telecom transformation goals require service automation - TechTarget [Last Updated On: February 6th, 2017] [Originally Added On: February 6th, 2017]
- Global Hazardous Waste Handling Automation Market: By Products ... - Business Wire (press release) [Last Updated On: February 6th, 2017] [Originally Added On: February 6th, 2017]
- 2M Automation wins IoT support from Schneider - Electronics EETimes (registration) [Last Updated On: February 6th, 2017] [Originally Added On: February 6th, 2017]
- Futures Shaped by Automation and Catastrophe: Peter Frase on Capitalism's Endgame - Truth-Out [Last Updated On: February 6th, 2017] [Originally Added On: February 6th, 2017]
- Automation expected to displace insurance underwriters, real estate brokers - CIO Dive [Last Updated On: February 6th, 2017] [Originally Added On: February 6th, 2017]
- Automation, robots could replace 250000 public sector workers in the next 15 years - Computer Business Review [Last Updated On: February 6th, 2017] [Originally Added On: February 6th, 2017]
- Design Automation Conference - Business Wire (press release) [Last Updated On: February 6th, 2017] [Originally Added On: February 6th, 2017]
- The Perks Of Automation And The Risks: Why To Think Twice About Getting Into That Driverless Uber - Forbes [Last Updated On: February 6th, 2017] [Originally Added On: February 6th, 2017]
- Lib Dems Should Embrace Automation of the Workforce - Liberal Democrat Voice [Last Updated On: February 7th, 2017] [Originally Added On: February 7th, 2017]
- Voices Reinventing enterprise finance by overhauling AP automation - Accounting Today [Last Updated On: February 7th, 2017] [Originally Added On: February 7th, 2017]
- How Accountants Can Use Automation Their Advantage - Accountingweb.com (blog) [Last Updated On: February 7th, 2017] [Originally Added On: February 7th, 2017]
- DFLabs Launches the First Security Automation and Orchestration Platform based Upon Supervised Active Intelligence - Business Wire (press release) [Last Updated On: February 7th, 2017] [Originally Added On: February 7th, 2017]
- QAD Automation Solutions is Honda Approved - Yahoo Finance [Last Updated On: February 7th, 2017] [Originally Added On: February 7th, 2017]
- VIDEO: Going Big on Automation in a Small Footprint Facility - ENGINEERING.com [Last Updated On: February 7th, 2017] [Originally Added On: February 7th, 2017]
- Building a better model of human-automation interaction - Phys.Org [Last Updated On: February 7th, 2017] [Originally Added On: February 7th, 2017]
- AlixPartners examines automation in manufacturing and logistics management - Logistics Management [Last Updated On: February 7th, 2017] [Originally Added On: February 7th, 2017]
- Report: Test automation is increasing - SD Times - SDTimes.com [Last Updated On: February 9th, 2017] [Originally Added On: February 9th, 2017]
- Automation is the unavoidable future of the economy - The Daily Cougar [Last Updated On: February 9th, 2017] [Originally Added On: February 9th, 2017]
- GM's Cruise Automation Is Testing An App to Order Self-Driving ... - Fortune [Last Updated On: February 9th, 2017] [Originally Added On: February 9th, 2017]
- Speeders beware: Legislation would allow automation crackdown ... - SFGate [Last Updated On: February 9th, 2017] [Originally Added On: February 9th, 2017]
- Orbita Ingenieria: New Age Terminal Automation - Port Technology International [Last Updated On: February 10th, 2017] [Originally Added On: February 10th, 2017]
- A Sharper Focus on the Edge - Automation World [Last Updated On: February 10th, 2017] [Originally Added On: February 10th, 2017]
- Rockwell Automation Surged 10% in January as Growth Picked Up Steam - Motley Fool [Last Updated On: February 10th, 2017] [Originally Added On: February 10th, 2017]
- Most people are optimistic about workplace automation, social data suggests - ZDNet [Last Updated On: February 10th, 2017] [Originally Added On: February 10th, 2017]
- Improving Behavior Through Automation of Vehicle Systems - School Transportation News (blog) [Last Updated On: February 11th, 2017] [Originally Added On: February 11th, 2017]
- 'We employ insane levels of automation' Kris Canekeratne - Times of India [Last Updated On: February 11th, 2017] [Originally Added On: February 11th, 2017]
- Why Don't We See More Automation in Federal Networks? - Nextgov [Last Updated On: February 11th, 2017] [Originally Added On: February 11th, 2017]
- Technobabble: Automation and the modern worker - CIO Dive [Last Updated On: February 11th, 2017] [Originally Added On: February 11th, 2017]
- Readers Write (Feb. 12): The moose population; jobs, start-ups and automation; diversity in the funny pages - Minneapolis Star Tribune [Last Updated On: February 12th, 2017] [Originally Added On: February 12th, 2017]
- Automation Nightmare: Philosopher Warns We Are Creating a World Without Consciousness - Big Think [Last Updated On: February 12th, 2017] [Originally Added On: February 12th, 2017]
- Automation can replace bureaucrats and save taxpayers money - Hot Air [Last Updated On: February 12th, 2017] [Originally Added On: February 12th, 2017]
- Automation can revitalize the US workforce - Fox News [Last Updated On: February 12th, 2017] [Originally Added On: February 12th, 2017]
- TigerStop hopes to ride automation to new heights - The Columbian [Last Updated On: February 13th, 2017] [Originally Added On: February 13th, 2017]
- Hexadite Unveils Custom Playbooks Following One Millionth Automated Cybersecurity Investigation - Yahoo Finance [Last Updated On: February 13th, 2017] [Originally Added On: February 13th, 2017]
- NEC updates postal automation system for Hongkong Post - ETCIO.com [Last Updated On: February 13th, 2017] [Originally Added On: February 13th, 2017]