Today, most web-based applications have a mobile version. Both being a different device, have different requirements to run an application. Consequently, the performance of an application differs on different devices and platform. In addition, there are various other factors like network capability that affects the user experience. There are many differences while testing performance of a mobile and web application. You must figure out those factors and plan accordingly; however, you must avoid migrating your existing test plans while building a performance test strategy for your mobile application.
There are three key factors that must be applied along with specialized testing strategies to ensure an excellent user experience irrespective of the device being used.
The Device
Web servers often send the content based on the kind of device user has. With the range of devices in the market today, it becomes extremely important to execute performance testing of applications based on web and mobile with different devices. The performance is extremely affected by the way in which the content is translated by different devices.
For mobile based applications, factors like display size, processing power, platform, bandwidth, etc. must be considered to build a clear performance testing plan. You can strategically include mobile emulators within the plan to manage this diversification. Emulators help in providing a real environment; however, they cannot replace real devices. For example, emulators would fail to capture the application behavior to battery consumption and CPU usage under load.
- Test a range of devices with different displays for both performance and functional differences
- Test different levels of processing capabilities, platform, and parallel connections of a set of physical devices
- Include much variety by incorporating real devices and emulators
The Network
Server accessed by mobile devices is slower than those used by PCs. The performance of an application hugely depends upon the network conditions and directly impacts the user experience. Factors such as latency, bandwidth, packet loss, etc. affect the server load time as well as client’s response time. For example, a low bandwidth will increase the time to download a resource; hence you will require more time to load a page. Emulating different conditions in a lab environment allows you to discover network connectivity issues in the development cycle.
In case of a mobile device, the page load time largely depends on the latency. As there are various sub-requests involved in fetching a web page, the latency increases with each request. Since, mobile devices have high latency and limited bandwidth; latency can be a bigger bottleneck.
- Test varying signal strengths with different connection speeds
- Observer application behavior by incorporating simulated latency and packet loss
- For ideal result, include geographic based testing
User Expectations
According to a recent survey, a mobile application is most likely to be abandoned by a user in case of slow performance. While performing load test, with user experience being the top priority, you need to consider new scenarios of failure.
You should take into account the device responsiveness, as how much delay can be tolerated by the user. Are the users happy about the responsiveness?
- You must consider the users’ understanding about the difference in performance on mobile device and PC’s by conducting usability tests
- Include quicker user expectations on mobile devices and adjust the acceptance criteria accordingly
- Compare the responsiveness of mobile site.
