Pradeep Kumar Govindasamy
Director - Mobile Testing, Cognizant
Pradeep heads the Automation and Mobile CoE at Cognizant. He is a technology executive with 14 ye... more>>
(This article was co- authored by Pradeep Kumar Govindasamy, Director - Mobile Testing, Cognizant and Ramakrishnan Venkatasubramanian, Associate Director - Mobile Testing, Cognizant)
The mobile phone has transitioned from its traditional role as a mere medium of communication to a multipurpose personal device. The technological advancements across the device models that drove this shift (Apple iPhone, Android, Windows Mobile) have introduced product development and rollout challenges for device manufacturers and application developers, alike.
Quality testing of applications across all of these operating systems, device platforms and networks is a daunting task. Therefore, non functional testing --including, usability, security and adaptability --becomes as critical as functional testing. Additionally, effective testing enables device makers and application developers to collect appropriate metrics that help improve product quality. Most applications designed for mobile platforms require the ability to run on a multitude of network modes, as well.
This article explores industry best practices on mobile testing and demonstrates effective ways of managing mobile application quality through test automation. Despite the pressure of short cycles in mobile development, thorough testing is vital for ensuring long-term success in a highly competitive market.
As with conventional applications, there is an increased need to automate mobile applications testing, which is the buzzword of the hour. A well-orchestrated automation strategy provides higher savings and shortens time-to-market.
The Rise of Mobile
The growth of mobile technology in the IT industry is being driven by a multitude of factors, including the following:
• Globally, the user base of location-based services has grown from 96 million in 2009 to more than 526 million in 2012.
• Worldwide mobile advertising and content revenue is forecast to hit $67 billion in 2012.
• IDC predicts that the volume of smart phones is estimated to reach over 500 million shipments by 2014.
• According to Forrester, by 2016, smart phones and tablets will put power in the pockets of a billion global consumers.
• Mobile device makers are developing sophisticated platforms to increase their share of the customers wallet.
Two Key Challenges in Mobile Application Test Automation
The various challenges of mobile application test automation fall into two main categories: mobile platform technology and test execution.
• Mobile platform technology coverage: The primary factor that determines an automation tool’s success is the ability to work across platforms and technology stacks. The following challenges influence automation success:
1. Device Diversity
a) Multiple platforms and multiple browsers
b) Rendering differences
c) Mobile devices with varied application run times
2. Network Challenges
a) Multiple types of networks (GSM/GPRS/Wi-Fi/Wi-Max)
b) Different speeds of connectivity across geographies
c) Multiple network operators with customized network features
3. Hardware Challenges
a) Limitations in processing speed
b) Limitations of memory size of mobile
c) Differences in communication protocols of device’s WAP/HTTP
• Test execution: Test execution across different devices and browsers is a huge effort that is difficult to deploy and run. For example, consider a test case that needs to be executed on three different platforms, such as IOS, Android and Blackberry. The test case is expected to be executed on 13 different devices and in 10 different combinations, which leads to 130 rounds. It is challenging and time-consuming to design such a test execution matrix.
Sample Test Execution Combination in a Real-Time Scenario for a Smartphone-Based App
Indicative List of Different Smartphone Platforms and Browser Combinations
Experts on QA