With ever-increasing ubiquity of mobile devices, the marketplaces have seen an exponential increase in number of applications in last few years. No wonder the word “app’’, which was referred to only in very technical contexts few years ago, has found its place in official English dictionaries now.
While a tester needs to apply the same analytical approach to test a mobile application, at the same time it is a bit more challenging as compared to testing a web application.
For a web application, testing needs to be accomplished on different browsers. On the other hand, when it comes to mobile application, there is lot more complexity added due to combination of multiple platforms each having numerous devices and various OS versions.
Some of the challenges in testing mobile applications are as follows:
1. The biggest challenge when it comes to mobile application testing is the plethora of devices spread across different platforms. Obviously, it is not feasible to test your app on each and every available device which means you have to strategically choose a few physical devices.
One needs to remember that testing on one device never assures it would work on any other device, irrespective of whether it is of same make, same OS Version or using the same platform! Not testing on a physical device always runs a risk of potential failure on that device, specially when the target audience for the application is widespread, like for a game.
One common way to overcome the issues of testing on numerous physical devices is to use emulated devices. This approach will be less expensive but the emulated devices will not present the quirks present in real devices. It’s quite possible that you might be testing your application in conditions not close enough to the real life ones.
Testing demands different physical devices to cover the following :
a) Varying screen sizes.
b) Different Form factors.
c) Different pixel density and resolution.
d) Different input methods like QWERTY, touch etc.
2. Different platform testing: In case of native app, it goes without saying that it will need dedicated testing effort on all platforms for which it is developed. It gets a bit tricky in case of HTML5 based hybrid applications. While the code remains same, lot of factors come into play on different platforms.
3. Testing on different OS versions of the same platform: So, you did test your application on all major platforms aka Android, iOS, Windows etc but each one of them have several OS versions floating in market. An obvious choice is to test on the most recent versions of all the platforms but this would not do justice for Android application. The latest version of Android is Jellybean introduced quite a while ago, still there are lot of devices which have not yet received OS updates ( and possibly will never be updated). Its interesting to note a big difference in Google’s and Apples’s approach in handling the OS updates. While the former relies on device manufacturers to update the respective devices, Apple handles the updates itself resulting into mass updating of all Apple devices as soon as a new OS version is released. Whatever is the OS version on a device, user can still install your application and use it, which calls for testing different OS versions.
4. Testing on various networks and network vendors: Most of the mobile applications require network connectivity sometime or the other. If the app talks to a server for flow of information to and fro, testing on various (atleast all major ones) networks is important. Mobile networks use different technologies like CDMA and GSM with their 2G, 3G and 4G versions. The network infrastructure used by network operators may affect data communication between app and the backend. Apart from the different operators, an application needs to be tested on Wi-Fi network as well.
5. Mobile environment: It poses another unique challenge to the tester. Mobile environment is very dynamic and has constraints like limited computing resources or available memory and battery life.
An application’s behaviour depends on factors like other applications running in background, battery state of the device, switching between networks, Adding to the complexity is lot of other sensors and features like accelerometer, gyroscope, GPS, memory cards, camera etc present in the device. An application should integrate with these features gracefully.
6. THE Mobile user: Mobile application audience comprises of various people ranging from non-tech savvy people to skilled users, from children to middle age users. Each of these have their way of using the application and have their own expectation from it. A middle age user will be much more patient than someone in twenties when it comes to the response time. Generally speaking, mobile users have incredibly high expectations from the applications! Tester have to wear different hats while testing the application and make sure that it provides a good overall response to all users and does not cause grief to a particular set of users.