Exploring Android apps with the T-PAIN Heuristic
A methodology for testing the communication between the system and applications.
Introduction
When we talk about programmes and applications, a few things come to mind: a famous mobile app, a computer programme used for work, perhaps even a website. Despite the similarities, the distinctions between these are relevant and should be noted.
Android apps have well-defined characteristics to support the various limitations of the mobile devices. The system has to constantly monitor active applications, the state they’re in and when they should be removed from memory — this is called App Lifecycle. It contrasts with computer systems, which keep applications on memory until the user manually closes them.
Each programme and app is developed differently, so testing needs to be more thorough. A properly written application has to communicate smoothly with the system.
The Heuristic Structure
I have been working with Android development for the last two years. In that time, I realised that only a handful of test heuristics addressed the application and operational system. Learning more about architecture and using the scarce material available, I mapped a few aspects that laid the foundation for this methodology, and enhanced my exploratory tests.
The technique focuses on the various ways the OS exerts control over the apps, such as removing them from the CPU because of a scheduling routine. It can be applied to any app developed for the Android API 23 (6.0) or above. Therefore, it’s compatible with more than 85% of the Android devices worldwide (statcounter, Mar/21).
It is composed of 5 pillars: Rotation, Permissions, Airplane Mode, Interruptions and Connections. For it directly involves the OS, exercising it will be especially efficient in early development stages of the app, commonly unveiling architectural problems. Below, I expand more on how to explore these characteristics:
Ro[T]ation
Fill the information on-screen, such as text fields, and rotate the device:
- Does the information persist?
- Is the layout properly rearranged?
Initiate a flow and rotate the device at any given moment:
- What happens if you rotate in the middle of a service request?
- Does the application lose its references after rotating?
[P]ermissions
- How does the app behave when only a portion of the permissions are enabled?
- What happens to the application when some of the permissions are disabled mid-use?
- Do the requested permissions make sense to the business?
[A]irplane Mode
- How much does the app depend on wireless connections?
- How does the app handle being offline?
[I]nterruptions
Third-party:
- How does the app behave when the system prioritises another application over it?
- Or when the device receives a phone call with the app in the foreground?
- Or when a notification is displayed?
System-calls:
- What happens when the OS kills the app processes in the background?
- How does the app handle being force-stopped?
- Is it possible to re-open the app after removing it from memory?
- How dependent is the app on its cached information?
Co[N]nections
- What happens to the requests when the network ping is high?
- How does the app manage its processes when connected to 2G or 3G?
- How are timeout errors being handled?
Proof of Concept
My mentor and friend, Júlio de Lima, gave me the opportunity to apply my heuristic to his Lojinha app — the same one used in his course “Programa de Testes e Qualidade de Software”. The goal was to demonstrate how it would perform in a real-life situation and, with luck, uncover inconsistencies.
Rotation
To rotate an application, the Android OS needs to destroy its current screen and recreate it on the new orientation. This simple movement can reveal architectural problems. On the GIF beside this text, the app doesn’t preserve the data inserted into the text fields.
Here, we crossed paths with a UX problem. Consider this: if this were a huge form, and the user accidentally rotated the device, they would’ve lost all the information and would have to input it again.
Permissions
From the API 23 onwards, apps have to list all the permissions they require beforehand, and request them only when needed. This gives the users more control over what the applications can do on their phone.
The Lojinha app was smartly developed to be simple, and no permissions were requested. Thus, this aspect couldn’t be tested.
Airplane Mode
Turning on the airplane mode shuts down every wireless sensor responsible for radio frequencies of mid and long range, such as bluetooth, mobile data and wifi. This forcibly interrupts any connection and data transfers.
On apps highly dependent on these sensors, this could pose a problem; what happens when the app cannot perform any request?
On the application under testing, the error was handled, but revealed something that could be improved: the error message contains irrelevant information to the user.
Interruptions
With limited hardware resources, the OS has to optimise how the apps consume it. To that end, it closely watches applications and processes, delegating and changing priorities whenever needed.
When another process requests permission to go from back to foreground, or to overlay another application, the Android OS has to interrupt the running app to give room for the new one with higher priority. Notice that the app is still open, giving users a seamless experience (they don’t need to know the complexities of the system).
In the case of the Lojinha app, we came across two interruptions:
- When the notification is displayed.
- When the device receives a phone call.
Having its priorities suddenly changed could make a poorly configured application to lose track of its lifecycle, often leading to a crash.
The app I used in my example responded properly to the interruptions, maintaining its use before and after the events.
Connections
One of the main characteristics that define a smartphone is its ability to connect to different network technologies, i.e. from 2G to 5G, from 2.4 GHz to 5 GHz. As such, the applications also have to adapt to these scenarios.
The more an application is used, the more it will be exposed to different situations. Not all of the users will have access to a stable WiFi connection.
Based on that, this last aspect attests to how the application adjusts to slow and/or unstable networks.
The Lojinha app displayed slowness when requesting my item list using a 2G connection, but managed to register a new one with no problems.
Conclusion
This heuristic was developed with the aggregated experience I’ve had with Android development and testing. Feel free to modify it to your use, bend it to your will, and revitalise it to engulf iOS apps as well. Just like me and you, this heuristic is in constant growth.
If you understood the reference in the name of this methodology, I’ll forever guard you in my heart. Hint — Google it.
To those that decide to apply the heuristic, I kindly ask you to respond to my research by clicking here. I kept it very short, and it will help to understand the issues it has found in different environments and how I could improve it.
I sincerely appreciate: my mentor, Júlio de Lima, for opening my mind and unleashing my potential. My Android development mentor, Jefferson Rodrigues, who is a splendid teacher and professional. My exceedingly talented brothers, Wesley Oliveira and Tiago Pereira, who have supported me with insights and companionship. My sister, Christianah, for coming through with a wonderful review.
