Mobile UI Automation Using Python uiautomator Module

Let's see how we can automate mobile UI using python uiautomator module. It works on Android 4.1 + .

Install python 2.7 on your system.First you need to install uiautomator package

1. How to Automate call functionality using uiautomator.

This is how you identify the element from uiautomator. You have to pick the unique properties like className,packageName,Text,resourceId etc. by using uiautomator.

from uiautomator import device as d
from uiautomator import Device
from time import sleep
import os

def VoLTE_call():
  print("Inside Call Method.")
  call = d(className='android.widget.TextView',
                     packageName='com.cyanogenmod.trebuchet',
                     text ='Phone').click()
  sleep(2)
  dialer = d(className='android.widget.ImageButton',
           packageName='com.android.dialer',
           resourceId='com.android.dialer:id/
                       floating_action_button').click()
  sleep(1)
  dialling_page = d(className='android.widget.EditText',
                    packageName='com.android.dialer',
                    resourceId='com.android.dialer:id/digits')
                                    .set_text(number)
  sleep(1)
  press_call_button = d(className='android.widget.ImageButton',
                    packageName='com.android.dialer',
                    resourceId='com.android.dialer:id/
                                dialpad_floating_action_button').click()
  sleep(5)
  #hold call
  hold = d(className='android.widget.ImageButton',
                    packageName='com.android.dialer',
                    resourceId='com.android.dialer:id/holdButton')
                             .click()
  sleep(5)
  #unhold Call
  unhold = d(className='android.widget.ImageButton',
                    packageName='com.android.dialer',
                    resourceId='com.android.dialer:id/holdButton')
                             .click()
  sleep(5)
  # mute Call
  mute= d(className='android.widget.ImageButton',
             packageName='com.android.dialer',
             resourceId='com.android.dialer:id/muteButton').click()
  sleep(5)

  #unmute call  
  unmute= d(className='android.widget.ImageButton',
             packageName='com.android.dialer',
             resourceId='com.android.dialer:id/muteButton').click()
  sleep(5)
  #speaker on  
  speaker_on= d(className='android.widget.ImageButton',
           packageName='com.android.dialer',
           resourceId='com.android.dialer:id/audioButton').click()
  sleep(5)
  # speaker off  
  speaker_off = d(className='android.widget.ImageButton',
             packageName='com.android.dialer',
             resourceId='com.android.dialer:id/audioButton').click()
  sleep(5)
  press_endcall_button = d(className='android.widget.ImageButton',
                        packageName='com.android.dialer',
                        resourceId='com.android.dialer:id/
                              floating_end_call_action_button').click()
  d.press.home()
#Call The main Method
VoLTE_call()
print(""Your test is completed...")

How to Capture screenshot and video using adb commands

There are multiple situations where you want to capture the screenshots or video and store in a folder all programmatically. This is simple and one line adb command that can be used to capture the screenshot and video.

Capture Screenshot:

Please find the below steps to capture the screenshot of an android device

1. Connect device to laptop. To confirm whether device is connected or not, run below command

 2. Screencap is the adb command which is used for capturing the screenshots

3. Once the screenshot is captured, its time to pull out from device to laptop.

The image is pulled on your system.

Capture Video:

Please find the below steps to capture the video of an android device. This is generally required when you want to give steps for reproducing certain issues.

1. Connect device to laptop. To confirm whether device is connected or not, run below command

2. Screenrecord is the adb command which is used for capturing the screenshots

3. Once the screen activities are recorded press ctrl+C to stop the recording.

The video which was captured is stored on sdcard. To pull out the video from sdcard please run below command

Here is simple program that depicts how programatically you can use above two commands

Same way we can also write script for video recording.

Locating UI Element Using uiautomator

How to use uiautomator for inspecting UI elements on mobile?

So there are multiple ways available by which you can identify the element on mobile. some of them as below. So here I am taking an example of Hotstar app for illustrating the various ways of element identification. Also I am using python as my scripting language.

1. By ID

This is the most efficient way to locate an element as IDs are unique. This is the most prefered way to identify the elements of android app. 

So for this the code will be as below.

app_opened=d(packageName='in.startv.hotstar', resourceId='in.startv.hotstar:id/iv_ad_background').wait.exists(timeout=20000)  

You can see that I have highlighted the portion where it depicts that element is identified by ID.

2. By className:

A class contains many elements. So you have to little careful while using element identification using class name because if an element has many classes then it will match against each of them. 

So for this the code will be as below.

home_page_click=d(packageName='in.startv.hotstar', className='android.support.v4.view.ViewPager',resourceId='in.startv.hotstar:id/pager').click()

3. By Name:

It is possible to locate the element by name but they should be unique.

So for this the code will be as below.

dream11=d(packageName='in.startv.hotstar',resourceId='in.startv.hotstar:id/tv_header',  text='Dream11').click()  

4. By Xpath:

The very last option to be used for identifying the elements. You end up in many situations that you don't have option but to use xpath for identifying the elements.There are two types of xpath, relative xpath and absolute xpath. It is suggested to use relative xpath because absolute xpath uses indexes and if there is any change in the code, it will result in the failure of test scripts which is trying to identify the UI element.

So for this the code will be as below.

dream11 = d(By.xpath("//*[@text = 'Dream11']"))

Different actions performed using uiautomator or Appium

Perform Actions

Once the element is identified, you want to perform different actions just like a normal user does on mobile.

So different actions which you can be performed are:

1. click():
     Clicks the center of the visible UI element

2. dragTo():
    Drags this object to arbitrary coordinates.

3.setText():
   Allows you to set a text in editable text field.

4. ClearTextField():
    This method is used to clear the text field if any text is already available.

5. swipeUp():
    Performs the swipe up action on the UI element.

6. swipeLeft():
    Perform the swipe left action on the UI element.

7. swipeRight():
    Perform the swipe right action on the UI element.

8. swipeDown():
    Perform the swipe down action on the UI element.

Ways to identify mobile app elements

How to identify the elements on mobile UI

You can identify element using either uiautomator viewer or using appium. Let me show you how you can use the uiautomator for identifying the UI elements and later on I will explain element identification using appium

Element identification using uiautomator viewer

It is convenient GUI Tool to scan and identify the UI elements of the app which is currently displayed on the device under test. uiautomator viewer is located at <android-sdk>/tools/bin

Double click on the batch file and you will be able see the uiautomatorviewer. Once you have clicked on the uiautomatorviewer, it will display a window which would be like as below

To check the elements on UI , you need to click on third element from element and then below screen will appear on uiautomatorviewer (Make sure you have connected a mobile device to laptop)

Why Mobile Automation Required?

             To compete in this very competitive world of smartphones we have to deliver a quality product and need more revenue in small tenure then it is necessary for an organisation to go for Mobile automation testing. Mobile automation will provide the various solutions, which helps you to verify and validate the overall functionality of the Mobile apps(native as well hybrid apps). Automation can be done to achieve better coverage in less duration with better test results.

Types of Apps available for Smartphones:

There are 3 types of apps available in smartphones. Below are the types

1. Native App
2.  Hybrid App
3. Web Apps

How to Automate??

There are so many tools available for automation testing. Some are paid and some are open source.Depending on the requirement of the project one should decide the automation tool for automation.

Here are some examples of tools:

1. Appium
2. Robotium
3. Selendroid(for Android)
4. UI Automator
5. Ranorex
6. MonkeyRunner
7. SeeTest