Android Unit Tests

1

I have a unit test suite for an android app written in Kotlin. We had the test suite running with Robolectric on Android Studio 1.0 and Kotlin M10. In order to get this to work I had to write a custom build step to copy the compiled test .class files to the output location that JUnit expected them to be in.

After updating to M11 and Android Studio 1.2 we cannot get the tests to run. As far as I can tell there does not seem to be a gradle task to compile the unit tests under the new Android Studio 1.2 unit testing setup.
There is a compileDebugAndroidTestKotlin task that compiles the the automation tests in the androidTest folder, but no compileDebugUnitTestKotlin task for compiling the unit tests in the /test folder.

What is the status of unit testing support for android studio 1.2?

Thanks

Bob

2

Hi, Junit is currently supported only in kotlin-plugin 0.1-SNAPSHOT from sonatype repository. We are planing to push a new version to maven cental during the next week.

3

Hi Natalia,

We didn’t have any luck with the snapshot. Just checking, you meant kotlin-gradle-plugin, not kotlin-plugin, correct? Does it add a new test task, or should it work with the existing test task (e.g. “testDebug”)?

Thanks,
Robert

4

Hi, Sorry for misprinting, I ment kotlin-gradle-plugin. It adds a new task for compiling kotlin source files in test source roots ( ex. compileDebugUnitTestKotlin) and the tests should be executed during test task (ex. testDebug)

If you still have problems with running junit tests written in Kotlin, could you check that you’ve configured source sets correctly? The default source sets for kotlin test files are src/test/kotlin and src/test/java.

Also there is an instruction how to configure junit with android in Android Studio.

5

Hi Natalia,

Any plans to support TestNG?

6

Hi Cedric,

This is Android Feature -  running Junit tests with Android. Kotlin only provides a compile task for Kotlin source files in test folders.

7

Hi Natalia,

We have our source sets configured correctly, and the kotlin and java test folders are both green in Android Studio as a result. However, I get no compileDebugUnitTestKotlin task when using the snapshot (there is a compileDebugAndroidTestKotlin task, but I believe this is  for testing directly on a device, i.e. instrumentation testing.). Running ./gradlew :app:testDebug results in none of the Kotlin tests being compiled or executed (the Java test does get executed, however - we only have it there to confirm that we have the source set configuration right).

I even made sure that I have the most up-to-date version of the snapshot plugin by using:

rm -rf $HOME/.gradle/caches/


The next time I ran the testDebug task, it redownloaded the plugin:


Download https://oss.sonatype.org/content/repositories/snapshots/org/jetbrains/kotlin/kotlin-stdlib/0.1-SNAPSHOT/kotlin-stdlib-0.1-20150525.163934-1208.pom
Download https://oss.sonatype.org/content/repositories/snapshots/org/jetbrains/kotlin/kotlin-gradle-plugin-api/0.1-SNAPSHOT/kotlin-gradle-plugin-api-0.1-20150525.164530-222.pom
Download https://oss.sonatype.org/content/repositories/snapshots/org/jetbrains/kotlin/kotlin-runtime/0.1-SNAPSHOT/kotlin-runtime-0.1-20150526.023136-1211.pom
Download https://oss.sonatype.org/content/repositories/snapshots/org/jetbrains/kotlin/kotlin-gradle-plugin/0.1-SNAPSHOT/kotlin-gradle-plugin-0.1-20150525.164855-1178.jar

Do you have any more ideas?

Thanks!
Robert

8

You are right: compileDebugAndroidTestKotlin is for tests on device, so compileDebugUnitTestKotlin should be present... First of all, could you run 'gradlew --debug test' and attach the log here?

Some more questions:
1. Do you have custom output directories in your project?
2. Could you check that class files for Kotlin tests are absent in output directory after running ‘gradlew clean test’? (Default path is {module directory}/build/tmp/kotlin-classes and then they should be copied to build/intermediates/classes/test)
3. There is a link to a simple kotlin project with junit test and Android, could you check that there is one Kotlin test failed in it?

9

Hi Natalia,

I’ve played with the configuration a bit, and am currently using the following dependencies:

dependencies {
  …
  compile “org.jetbrains.kotlin:kotlin-runtime:0.11.91.4”
  compile “org.jetbrains.kotlin:kotlin-reflect:0.11.91.4”
  …
}

buildscript {
  repositories {
  mavenCentral()
  maven { url ‘https://oss.sonatype.org/content/repositories/snapshots/’ }
  maven { url ‘http://download.crashlytics.com/maven’ }
  }

  dependencies {
  classpath “org.jetbrains.kotlin:kotlin-gradle-plugin:0.1-SNAPSHOT”
  classpath ‘com.crashlytics.tools.gradle:crashlytics-gradle:1.16.0’
  }
}

Does it matter that the plugin and the runtime/reflect libraries don’t match? There was no 1.0-SNAPSHOT available for kotlin-reflect, which is why I stuck with 0.11.91.4.

And using the following command:
./gradlew --stacktrace clean :app:testDebug

I ran into the following problem:

FAILURE: Build failed with an exception.

* What went wrong:
A problem occurred configuring project ‘:app’.
> Failed to notify project evaluation listener.
  > javaCompileTask

* Try:
Run with --info or --debug option to get more log output.

* Exception is:
org.gradle.api.ProjectConfigurationException: A problem occurred configuring project ‘:app’.
  at org.gradle.configuration.project.LifecycleProjectEvaluator.addConfigurationFailure(LifecycleProjectEvaluator.java:91)
  at org.gradle.configuration.project.LifecycleProjectEvaluator.notifyAfterEvaluate(LifecycleProjectEvaluator.java:86)
  at org.gradle.configuration.project.LifecycleProjectEvaluator.evaluate(LifecycleProjectEvaluator.java:65)
  at org.gradle.api.internal.project.AbstractProject.evaluate(AbstractProject.java:504)
  at org.gradle.api.internal.project.AbstractProject.evaluate(AbstractProject.java:83)
  at org.gradle.execution.TaskPathProjectEvaluator.configureHierarchy(TaskPathProjectEvaluator.java:47)
  at org.gradle.configuration.DefaultBuildConfigurer.configure(DefaultBuildConfigurer.java:35)
  at org.gradle.initialization.DefaultGradleLauncher.doBuildStages(DefaultGradleLauncher.java:129)
  at org.gradle.initialization.DefaultGradleLauncher.doBuild(DefaultGradleLauncher.java:106)
  at org.gradle.initialization.DefaultGradleLauncher.run(DefaultGradleLauncher.java:86)
  at org.gradle.launcher.exec.InProcessBuildActionExecuter$DefaultBuildController.run(InProcessBuildActionExecuter.java:80)
  at org.gradle.launcher.cli.ExecuteBuildAction.run(ExecuteBuildAction.java:33)
  at org.gradle.launcher.cli.ExecuteBuildAction.run(ExecuteBuildAction.java:24)
  at org.gradle.launcher.exec.InProcessBuildActionExecuter.execute(InProcessBuildActionExecuter.java:36)
  at org.gradle.launcher.exec.InProcessBuildActionExecuter.execute(InProcessBuildActionExecuter.java:26)
  at org.gradle.launcher.cli.RunBuildAction.run(RunBuildAction.java:51)
  at org.gradle.internal.Actions$RunnableActionAdapter.execute(Actions.java:171)
  at org.gradle.launcher.cli.CommandLineActionFactory$ParseAndBuildAction.execute(CommandLineActionFactory.java:237)
  at org.gradle.launcher.cli.CommandLineActionFactory$ParseAndBuildAction.execute(CommandLineActionFactory.java:210)
  at org.gradle.launcher.cli.JavaRuntimeValidationAction.execute(JavaRuntimeValidationAction.java:35)
  at org.gradle.launcher.cli.JavaRuntimeValidationAction.execute(JavaRuntimeValidationAction.java:24)
  at org.gradle.launcher.cli.CommandLineActionFactory$WithLogging.execute(CommandLineActionFactory.java:206)
  at org.gradle.launcher.cli.CommandLineActionFactory$WithLogging.execute(CommandLineActionFactory.java:169)
  at org.gradle.launcher.cli.ExceptionReportingAction.execute(ExceptionReportingAction.java:33)
  at org.gradle.launcher.cli.ExceptionReportingAction.execute(ExceptionReportingAction.java:22)
  at org.gradle.launcher.Main.doAction(Main.java:33)
  at org.gradle.launcher.bootstrap.EntryPoint.run(EntryPoint.java:45)
  at org.gradle.launcher.bootstrap.ProcessBootstrap.runNoExit(ProcessBootstrap.java:54)
  at org.gradle.launcher.bootstrap.ProcessBootstrap.run(ProcessBootstrap.java:35)
  at org.gradle.launcher.GradleMain.main(GradleMain.java:23)
  at org.gradle.wrapper.BootstrapMainStarter.start(BootstrapMainStarter.java:33)
  at org.gradle.wrapper.WrapperExecutor.execute(WrapperExecutor.java:130)
  at org.gradle.wrapper.GradleWrapperMain.main(GradleWrapperMain.java:48)
Caused by: org.gradle.listener.ListenerNotificationException: Failed to notify project evaluation listener.
  at org.gradle.listener.BroadcastDispatch.dispatch(BroadcastDispatch.java:94)
  at org.gradle.listener.BroadcastDispatch.dispatch(BroadcastDispatch.java:31)
  at org.gradle.messaging.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:93)
  at com.sun.proxy.$Proxy13.afterEvaluate(Unknown Source)
  at org.gradle.configuration.project.LifecycleProjectEvaluator.notifyAfterEvaluate(LifecycleProjectEvaluator.java:79)
  … 31 more
Caused by: java.lang.NoSuchFieldError: javaCompileTask
  at org.jetbrains.kotlin.gradle.plugin.KotlinAndroidPlugin.processVariantData(KotlinPlugin.kt:368)
  at org.jetbrains.kotlin.gradle.plugin.KotlinAndroidPlugin.access$processVariantData$0(KotlinPlugin.kt:291)
  at org.jetbrains.kotlin.gradle.plugin.KotlinAndroidPlugin$apply$2.execute(KotlinPlugin.kt:343)
  at org.jetbrains.kotlin.gradle.plugin.KotlinAndroidPlugin$apply$2.execute(KotlinPlugin.kt:291)
  at org.gradle.listener.BroadcastDispatch$ActionInvocationHandler.dispatch(BroadcastDispatch.java:109)
  at org.gradle.listener.BroadcastDispatch$ActionInvocationHandler.dispatch(BroadcastDispatch.java:98)
  at org.gradle.listener.BroadcastDispatch.dispatch(BroadcastDispatch.java:83)
  … 35 more

To answer your questions:

  1. We don’t have custom output directories
  2. I wasn’t able to get far enough that the tests would have been copied.
  3. I downloaded your project and ran the test task. There is one failing test, as you predicted. Oddly I didn’t encounter the above exception, so the plugin obviously works under certain circumstances.

Any ideas? If you need any other information about our configuration, just let me know.

Thanks,
Robert

10

What about the exception Failed to notify project evaluation listener.   > javaCompileTask

This a known issue: for now kotlin-android plugin works only with com.android.tools.build:gradle:1.1.+ (do not use 1.2.+).

It’s better to use the same version of runtime and plugin but it shouldn’t cause the issue. By the way, kotlin-reflect 0.1-SNAPSHOT is present in ‘https://oss.sonatype.org/content/repositories/snapshots/’ repository so something should be wrong with your dependencies configuration.

To give some more ideas I’m waiting for the build log from you: ./gradlew –debug clean :app:testDebug. It should be long, so it will be better to attach it as a file.

11

Hi Natalia,

You were correct about kotlin-reflect. I had to add the snapshots repo to the normal repositories section as I only had it in the buildscript repositories section.

Also, I set the android gradle tools to com.android.tools.build:gradle:1.1.+ as you suggested.

When running ./gradlew --debug clean :app:testDebug I encountered a number of compilation errors that I don’t normally receive when building the app. I’m not sure if this is caused by the build tools or the kotlin plugin. I’ve attached the debug log.

Thanks,
Robert



snapshot.zip (530 KB)

12

I've also run the same command using the head of our develop branch in case you're interested in comparing (see attached).



[develop.zip|attachment](upload://zQ9ObEd7cfERb6K0qGxp7PNjUpF.zip) (694 KB)
13

Hi Natalia,

I migrated our project to M12 today and the tests now run with the updated plugin! Thanks for your help!

Cheers,
Robert