Updated guide to Android Unit Testing

In this post we are going to get an overview of unit testing in Android with a few practical examples. Unfortunately, testing in Android has some pitfalls, which we will try to uncover in this tutorial. A lot of the confusion stems from the fact that Android testing has undergone major development changes in the recent past, and therefore there are various – and sometimes – conflicting resources online, which may be depreciated in the mean time. Before we delve deeper, let’s create a new Android project. We are using preview version 4 of Android Studio 2.0, which can be downloaded from here.

Start by creating a new project “Say Hello” with a Blank Activity.

Before we change anything here, navigate to the app > java folder in the Project Window. There you will see two different folders*: one directory holds your Android code; and the other directory suffixed “(androidTest)” holds the sources for our Android Instrumentation tests, and currently contains one single Java file called ApplicationTest. (* Starting with Android Studio 2.0 (Preview 5) you should see three folders in total. One directory for the Android application source code, and two testing folders denoted ‘androidTest’ and ‘test’.)

Also, by navigating to the application’s source folder under ~/AndroidStudioProjects/SayHello/app/src in a file browser, you will notice that there are three folders “androidTest”, “main” and “test”.

If you take a look into this “androidTest” directory, you will find the mentioned ApplicationTest.java in there. So, obviously “androidTest” in the file system is linked to the annotated “androidTest” directory in the project window. But what about the “test” folder on the file system? This directory contains the sources for your “bare bone” unit tests (as opposed to Android Instrumentation tests)…

Android Instrumentation Testing vs. Unit Testing

In essence, there are two realms for testing. One is Android Instrumentation testing (a.k.a Device or Emulator tests) which is used when you test applications with tight Android dependencies, and on the other hand there is Unit Testing (a.k.a JVM Tests or Local Tests) which can be used for local testing on the host machine. Note that the borders between the two are not that strict, for instance you can inject certain Android dependencies into your Unit Tests with frameworks like mockito et al. Also you can run standard JUnit4 syntax tests inside Android Instrumentation, this is a big improvement that comes with the new Android Testing Support Library. The earlier testing library was based on JUnit3. (Generally, you can differentiate between the new Testing Support Library and the soon the be depreciated testing library by checking their package. The Testing Support Library is inside android.support, whereas the older testing library is part of android.testing).

Let us quickly change to the Unit Testing context. In Android Studio, open the “Build Variants” panel in the lower left corner, and then select “Unit Tests” from the “Test Artifact” drop down menu.

Notice how the java sources in the Project Window change from “(androidTest)” to “(test)”? Also, this folder now contains ExampleUnitTest.java, which is found inside ~/AndroidStudioProjects/SayHello/app/src/test on the file system. Be aware that it is only possible to have either Android Instrumentation Tests or Unit Tests active at any time. To continue in our tutorial, switch back to “Android Instrumentation Tests” for now.

Setting up the Test Support Library

Right click the androidTest folder in the project window and create a new Java Class “MainActivityTest” with the following contents:

package ch.serverbox.sayhello;

import org.junit.runner.RunWith;
import android.support.test.runner.AndroidJUnit4;

@RunWith(AndroidJUnit4.class)
public class MainActivityTest {

}

Both of the above includes will fail initially, and therefore we need to adapt our app/build.gradle file from the project window. There are two important additions:

Define the default Test Instrumentation Runner inside defaultConfig:
testInstrumentationRunner “android.support.test.runner.AndroidJUnitRunner”
AndroidJUnitRunner from the support library is an Instrumentation which runs JUnit3 and JUnit4, and replaces the earlier InstrumentationTestRunner.
Include the following libraries inside the dependencies scope (we will make use of the “rules” library later on):
androidTestCompile ‘com.android.support.test:runner:0.4.1’
androidTestCompile ‘com.android.support.test:rules:0.4.1’

After making these changes, start the Gradle project sync. This might fail, it does for me at least, with “Warning:Conflict with dependency ‘com.android.support:support-annotations’. Resolved versions for app (23.1.1) and test app (23.0.1) differ.” You can run the following command inside the project’s root folder to inspect the dependencies:

./gradlew -q app:dependencies

The above problem is that the support library has a dependency on support-annotations (23.0.1), but the default appcompat library has a dependency on version 23.1.1. One workaround is to explicitly load version 23.0.1 of the appcompat library. Here is the complete app/build.gradle file with the appropriate changes:

apply plugin: 'com.android.application'
 
android {
    compileSdkVersion 23
    buildToolsVersion "23.0.2"
 
    defaultConfig {
        applicationId "ch.serverbox.sayhello"
        minSdkVersion 15
        targetSdkVersion 23
        versionCode 1
        versionName "1.0"
        testInstrumentationRunner "android.support.test.runner.AndroidJUnitRunner"
    }
    buildTypes {
        release {
            minifyEnabled false
            proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.pro'
        }
    }
}
 
dependencies {
    compile fileTree(dir: 'libs', include: ['*.jar'])
    testCompile 'junit:junit:4.12'
    compile 'com.android.support:appcompat-v7:23.0.1'
    compile 'com.android.support:design:23.0.1'
    androidTestCompile 'com.android.support.test:runner:0.4.1'
    androidTestCompile 'com.android.support.test:rules:0.4.1'
}

Start the Gradle sync process once again, and head back to your MainActivityTest class. For testing, we will either need a real Android device connected via USB or an Emulator. Use whichever you prefer. You might argue that this test will not do anything. Nevertheless, start the test by right clicking “MainActivityTest” and selecting “Run MainActivityTest”. The first run will take some time, but afterwards it should be much quicker. If everything goes well, you should see a red message “1 test failed” in the Run tab within Android Studio. The error reads “java.lang.Exception: No runnable methods”, which is no surprise, after all we have not implemented any tests yet!

The Android application

I thought quite a while about what test cases would be a good fit for this tutorial. Ideally it should be something simple that has a text “input field” and some “output field”. Also, I came across this excellent tutorial from Evgenii, which was written when the new testing support library was not yet available (and is therefore based on JUnit3). For our tutorial, we will create a very similar application, while making use of the new features available.

What should our application do? We want a text field, where a name (e.g. Bob) can be entered. After that name is inserted, a button is clicked to confirm it. Then, a text view should show us a hello message in the form “Hello, Bob!”. Therefore we need:

An EditText widget to enter text
A Button widget to confirm the entered text
A TextView widget to show the hello message

The initial content of the hello message TextView before anything is entered into the EditText field shall be “Hello, stranger!”. And thus, the Android application would look something like this:

Test Driven Development (TDD)

Normally – and especially for an Android application this simple – you would just go ahead and create the mentioned widgets, implement their logic, and then – if at all – write your tests. For this tutorial however, we are using a test driven development (TDD) approach. The idea here is that we start by writing tests for UI or program logic that does not even exist yet. We then run our tests expecting to see multiple errors, and piece by piece start implementing the components needed until all errors are gone, or all tests have passed respectively. As mentioned, the initial content of our TextView should be “Hello, stranger!”. And this is exactly what our first test will make sure of. Open MainActivityTest.java and adapt it with the following contents:

package ch.serverbox.sayhello;
 
import org.junit.Assert;
import org.junit.Rule;
import org.junit.Test;
import org.junit.runner.RunWith;
 
import android.support.test.rule.ActivityTestRule;
import android.support.test.runner.AndroidJUnit4;
import android.widget.TextView;
 
@RunWith(AndroidJUnit4.class)
public class MainActivityTest {
 
    final String EXPECTED_INITIAL_MESSAGE = "Hello, stranger!";
     
    @Rule
    public ActivityTestRule<MainActivity> mActivityRule = new ActivityTestRule<MainActivity>(MainActivity.class);
 
    @Test
    public void checkInitialText(){
        MainActivity mainActivity = mActivityRule.getActivity();
        TextView textView = (TextView) mainActivity.findViewById(R.id.textview_message);
        String message = (String) textView.getText();
        Assert.assertEquals(EXPECTED_INITIAL_MESSAGE, message);
    }
 
}

If you run MainActivityTest again, it will fail because it can not find the TextView widget with id “textview_message” that we are trying to reference. So let’s go ahead and add our widgets in res/layout/content_main.xml:

<?xml version="1.0" encoding="utf-8"?>
<RelativeLayout ...>
    <EditText
        android:layout_width="match_parent"
        android:layout_height="wrap_content"
        android:id="@+id/edittext_name"/>
    <Button
        android:layout_width="match_parent"
        android:layout_height="wrap_content"
        android:id="@+id/button_ok"
        android:layout_below="@id/edittext_name"
        android:text="OK"/>
    <TextView
        android:layout_width="match_parent"
        android:layout_height="wrap_content"
        android:id="@+id/textview_message"
        android:layout_below="@id/button_ok"
        android:text="Hello"
        android:textSize="36sp"/>
</RelativeLayout>

Allright, allright, we cheated a little bit here.. The current test only needed the TextView, and we already added our EditText and Button as well. So, our approach it is not a 100% TDD conform, but that is OK. Running the test again will fail once more, because the initial text is “Hello” instead of “Hello, stranger!”. Therefore, adjust android:text in the above TextView. Afterwards, the test should return successfully!

More UI testing with Espresso

Let’s face it, until now our test is pretty dull. But we can do better! In the following, we will create a test that automatically writes something into the EditText, presses the Button and then checks the content of the TextView. These automated UI tests can be quite tricky, and there have been several approaches in the past, none of them really excel. Luckily, the Android support library contains an official UI testing framework called Espresso. Espresso is suitable for UI testing within an app. (There is also another UI testing framework called UI Automator, which is used for cross-app functional UI testing across system and installed apps).

To make use of Espresso, the following line needs to be inserted into the dependencies scope in app/build.gradle:

androidTestCompile 'com.android.support.test.espresso:espresso-core:2.2.1'

Adapt MainActivityTest with the following contents:


package ch.serverbox.sayhello;
 
import org.junit.Assert;
import org.junit.Rule;
import org.junit.Test;
import org.junit.runner.RunWith;
 
import android.support.test.rule.ActivityTestRule;
import android.support.test.runner.AndroidJUnit4;
import android.widget.TextView;
 
import static android.support.test.espresso.Espresso.onView;
import static android.support.test.espresso.action.ViewActions.click;
import static android.support.test.espresso.action.ViewActions.closeSoftKeyboard;
import static android.support.test.espresso.action.ViewActions.typeText;
import static android.support.test.espresso.assertion.ViewAssertions.matches;
import static android.support.test.espresso.matcher.ViewMatchers.withId;
import static android.support.test.espresso.matcher.ViewMatchers.withText;
 
@RunWith(AndroidJUnit4.class)
public class MainActivityTest {
 
    final String EXPECTED_INITIAL_MESSAGE = "Hello, stranger!";
    final String TEST_NAME = "Bob";
    final String EXPECTED_MESSAGE = "Hello, Bob!";
 
    @Rule
    public ActivityTestRule<MainActivity> mActivityRule = new ActivityTestRule<MainActivity>(MainActivity.class);
 
    @Test
    public void checkInitialText(){
        MainActivity mainActivity = mActivityRule.getActivity();
        TextView textView = (TextView) mainActivity.findViewById(R.id.textview_message);
        String message = (String) textView.getText();
        Assert.assertEquals(EXPECTED_INITIAL_MESSAGE, message);
    }
 
    @Test
    public void insertNameAndCheckMessage(){
        onView(withId(R.id.edittext_name))
                .perform(typeText(TEST_NAME), closeSoftKeyboard());
        onView(withId(R.id.button_ok))
                .perform(click());
        onView(withId(R.id.textview_message))
                .check(matches(withText(EXPECTED_MESSAGE)));
    }
 
}

Granted, writing Espresso tests is somewhat strange, especially if you don’t know the specific functions and imports needed. But to their credit, it is surprisingly easy to read the above test, wouldn’t you agree? The basic idea behind Espresso is:

Find a view
Perform an action
Check some state

Or in pseudo code (thanks to Stephan Linzner for providing his Droidcon Italy 2015 slides):

Also check out the official Espresso cheat sheet for more information.

If you haven’t already, run MainActivityTest once more. When looking at the emulator or Android device while the test is running, you will see how the text “Bob” is entered and the button is pressed in real-time. Unsurprisingly, this test will eventually fail, since there is no application logic which takes the input and converts it to the expected message. However, this should not be too hard of an exercise 🙂 . See here for a possible solution.

Local JUnit tests

Until now, we have only covered Android Instrumentation testing which needs to run on an emulator or real device, and therefore takes quite some time to complete. However, it is also possible to run Unit Tests on the JVM of your local machine.

As a quick tutorial, we will create a Java class “Converter”, which has functions for converting miles per hour to kilometer per hours and vice versa. This does not make for the most sophisticated test cases, but it will do alright for our purposes. Note that this should be created inside the Android sources (next to MainActivity.java):

package ch.serverbox.sayhello;

public class Converter {

    public static double mphToKmh(double mph){
        return 1.60934*mph;
    }

    public static double kmhToMph(double kmh){
        return 0.621371*kmh;
    }

}

Open the Build variants panel in the lower left and switch from “Android Instrumentation Tests” to “Unit Tests”. The source folder visible in the Project window changes from “androidTest” to “test” (and shows the contents of app/src/test). In there, create a new test class “ConverterTest” with the following contents:

package ch.serverbox.sayhello;

import org.junit.Test;

import static org.junit.Assert.*;

public class ConverterTest {

    final double MPH = 60;
    final double KMH = 96.5606;

    final double DELTA = 0.1;

    @Test
    public void kmhToMph_isCorrect(){
        double convertedMph = Converter.kmhToMph(KMH);
        assertEquals(MPH, convertedMph, DELTA);
    }

    @Test
    public void mphToKmh_isCorrect(){
        double convertedKmh = Converter.mphToKmh(MPH);
        assertEquals(KMH, convertedKmh, DELTA);
    }
}

Right click on ConverterTest in the project window and select “Run ConverterTest”, which should pass successfully. Note that we did not have to explicitly add a JUnit4 dependency in the app/build.gradle file, because by default, the following line is already included:

dependencies {
    ...
    testCompile 'junit:junit:4.12'
    ...
}

Likewise, if you need additional libraries for your local unit tests, you should declare them with the “testCompile” descriptor (as opposed to “androidTestCompile” which is used for Android Instrumentation tests).