ParameterizedRobolectricTestRunner doesn't seem to work with Android classes in real projects

[jmartinesp]

Description

As I was trying to get some screenshot testing with Roborazzi working and passing some data as parameters, I realised I was running into errors saying some Android classes couldn't be found. I found #3805 and the test added to verify this worked fine with Kotlin so I tried creating a new project and copying that single test, and I found out it didn't pass and had the same error messages:

Method parse in android.net.Uri not mocked. See https://developer.android.com/r/studio-ui/build/not-mocked for details.
java.lang.RuntimeException: Method parse in android.net.Uri not mocked. See https://developer.android.com/r/studio-ui/build/not-mocked for details.
	at android.net.Uri.parse(Uri.java)
	at io.element.android.robolectricparameterizedtest.ParameterizedRobolectricTestRunnerTest$Companion.getTestData(ParameterizedRobolectricTestRunnerTest.kt:39)
	at io.element.android.robolectricparameterizedtest.ParameterizedRobolectricTestRunnerTest.getTestData(ParameterizedRobolectricTestRunnerTest.kt)
	...

I was quite surprised because the same test obviously passes inside this project, but I couldn't find what's causing it to fail in external projects.

Steps to Reproduce

1. Create an Android project using Kotlin.
2. Add Robolectric support.
3. Create a test using ParameterizedRobolectricTestRunner.
4. Use some Android API when generating the parameters.

Robolectric & Android Version

Robolectric version: 4.11.1 (latest).
Android version: API 34.

Link to a public git repo demonstrating the problem:

https://github.com/jmartinesp/RobolectricParameterizedTest

[jmartinesp]

I just tried returning a lambda so the parameters are only evaluated after all the Robolectric test runner, shadowing and classloader setup is done and now it's working properly. Could it be that it's working in the test project because it somehow has access to the Android classes in some way that's not available in other projects?

[hoisie]

@jmartinesp I would recommend avoiding Android objects as parameters using ParameterizedRobolectricTestRunner. ParameterizedRobolectricTestRunner is particularly finicky for these types of issues. For instance, instead of Uris, you can have parameters be strings and create the Uris in the test methods.

I have not studied the bytecode generated for the test class, but if you are seeing errors such as Method parse in android.net.Uri not mocked, it typically means that you are trying to invoke Android methods outside of a Robolectric class loader (e.g. the Uri constructor). You are probably trying to invoke methods in the stubs jar. This happens primarily when you are have a static initializer that interacts with Android classes. I would not be surprised if this what is occurring in the Kotlin companion object.

[jmartinesp]

Thanks for your reply: we might be able to work around this issue in some instances by doing what you suggest, but I don't think it'll work for some of our use cases. We'll have to study how much effort we'd need to do to refactor our parameters.

That said, should this test be deleted then from the project? It was added here as proof that using Android APIs in the parameters like Uri worked in Kotlin, and that's only true inside this project's tests, for some reason I couldn't really understand.