Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move LocationFactory to plugins to allow overriding it #3063

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

DerFrZocker
Copy link
Contributor

As part of a rework on a project I'm contributing to, Mockito was added to replace the use of InvocationHandler.
One particular test class validates the behavior of around 1000 objects, with each other, resulting in a particular set of code running over 1 million times, using the mock a few times in the process. This results in the mock object being used over 15 millions times.

Oversimplified current use:

public class TestClass {

    // Takes 1 minute 2 seconds with normal Location logic
    // Takes 17 seconds with dummy location factory extensions
    @Test
    public void test() {
        Object object = mock();
        when(object.toString()).thenReturn("Hello");

        for (int i = 0; i < 15000000; i++) {
            assertEquals("Hello", object.toString());
        }
    }
}

Resulting in the execution time of this particular test class to rise from a few seconds to over 2 minutes. After some profiling, we found that a huge part is caused by Mocktio's Location feature.

This PR moves the LocationFactory over to plugins, to allow others to override it. Which in our case we would use to return a dummy empty Location. Using a dummy Location reduced the execution time of our test class to under 1 minute. And after some other changes in our code base, not related to this PR, to around 20 seconds (which will it is not a few seconds as before, is much better than 2 minutes).

Will we understand and are thankful that Mockito wants to provide helpful error messages, in our particular use case we would like the faster testing time, so that this test can also be easily run during normal development. If the need for Location information arise, we could still just remove the extension and run it slower for a particular time.

Looking at some GitHub issues and PR, such as #2723 which also wanted to add a system property to use a dummy Location, it seems we are not the only one which would benefit from such an extension option.

Checklist

  • Read the contributing guide
  • PR should be motivated, i.e. what does it fix, why, and if relevant how
  • If possible / relevant include an example in the description, that could help all readers
    including project members to get a better picture of the change
  • Avoid other runtime dependencies
  • Meaningful commit history ; intention is important please rebase your commit history so that each
    commit is meaningful and help the people that will explore a change in 2 years
  • The pull request follows coding style
  • Mention Fixes #<issue number> in the description if relevant
  • At least one commit should mention Fixes #<issue number> if relevant

@codecov-commenter
Copy link

codecov-commenter commented Jul 15, 2023

Codecov Report

Patch coverage: 100.00% and project coverage change: +0.49 🎉

Comparison is base (a7bc931) 84.97% compared to head (dba1f01) 85.47%.

Additional details and impacted files
@@             Coverage Diff              @@
##               main    #3063      +/-   ##
============================================
+ Coverage     84.97%   85.47%   +0.49%     
- Complexity     2874     2890      +16     
============================================
  Files           329      329              
  Lines          8801     8811      +10     
  Branches       1093     1093              
============================================
+ Hits           7479     7531      +52     
+ Misses         1045      992      -53     
- Partials        277      288      +11     
Impacted Files Coverage Δ
...a/org/mockito/internal/MockedConstructionImpl.java 72.22% <100.00%> (ø)
...in/java/org/mockito/internal/MockedStaticImpl.java 80.51% <100.00%> (ø)
...l/configuration/plugins/DefaultMockitoPlugins.java 92.85% <100.00%> (+0.35%) ⬆️
...internal/configuration/plugins/PluginRegistry.java 100.00% <100.00%> (ø)
...ockito/internal/configuration/plugins/Plugins.java 100.00% <100.00%> (ø)
.../internal/creation/bytebuddy/MockMethodAdvice.java 81.66% <100.00%> (ø)
...rnal/creation/bytebuddy/MockMethodInterceptor.java 78.57% <100.00%> (+0.79%) ⬆️
...ockito/internal/creation/proxy/ProxyMockMaker.java 78.57% <100.00%> (ø)
...ito/internal/debugging/DefaultLocationFactory.java 100.00% <100.00%> (ø)
...java/org/mockito/internal/debugging/Localized.java 100.00% <100.00%> (ø)
... and 5 more

... and 7 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

@TimvdLippe
Copy link
Contributor

Yes this is an area where we have not yet figured out how we should solve this. One question I have before we analyze the various options: are you sure that on the slow path you are using the Stackwalker? We have seen significant improvements when that's used, to the extent that the requests for improved performance mostly were gone.

@DerFrZocker
Copy link
Contributor Author

Yes, StackWalker is used on the slow path, I also just tested it with the Java 8 Location implementation and there it is even worse with 4 minutes and 44 seconds.

StackWalker: 1 min 58 sec
Java 8 / Throwable: 4 min 44 sec
Empty Location: 40 sec

@md-5
Copy link

md-5 commented Oct 6, 2023

Bumping this. Without some way to disable this, mockito is an order of magnitude more expensive than a simple Java proxy

@TimvdLippe
Copy link
Contributor

I do think this is a legitimate issue to be fixed, but I am unsure whether this is the best way to solve it. Ideally, I would like to explore how to best integrate this with CI/IDE tooling, so that they can take advantage of this. E.g. in case a test passes, we don't need all of this information. But when a test fails, we do need this information. I can imagine a workflow where a test fails, on rerun we want more explicit information. And on CI I can imagine that we want the full stacktrace, to ease debugging. By making it overridable, we do have the building blocks, but the way to use it is not straightforward. Similar to what we have done with mockito-inline, we need a low entry barrier here so that we can make Mockito perform the best in each scenario, while still providing sufficient information for debuggability.

Given this feature, in which scenarios do you think it is worthwhile to drop the location information, keeping in mind we have to balance performance and debuggability in for example CI environments?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants