Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Testlib: Add test that putIfAbsent replaces a null value #6217

Open
ben-manes opened this issue Oct 16, 2022 · 20 comments
Open

Testlib: Add test that putIfAbsent replaces a null value #6217

ben-manes opened this issue Oct 16, 2022 · 20 comments

Comments

@ben-manes
Copy link
Contributor

ben-manes commented Oct 16, 2022

This was shown in a recent Java Collections Puzzlers. The JavaDoc for Map.putIfAbsent states the following.

If the specified key is not already associated with a value (or is mapped to {@code null}) associates it with the given value and returns {@code null}, else returns the current value.

@Test
public void putIfAbsent_nullExistingValue() {
  var map = new HashMap<Object, Object>();
  map.put("a", null);
  map.putIfAbsent("a", "b");
  assertThat(map).containsEntry("a", "b");
}

I found that multiple custom maps which permit null values do not honor this peculiarity (and similar for computeIfAbsent). For example fastutils' adopted the testlib and Object2ObjectOpenHashMap passes its suite, but it mistakenly does not replace the value as shown above.

@kevinb9n
Copy link
Contributor

Thanks Ben! Looks worth adding. If you or anyone else feels like making the PR we can merge it. If not, I hope we'd still get around to it sooner or later.

@alvonellos
Copy link
Contributor

@kevinb9n what would be the best way for me to get started contributing? This looks like a good first issue.

@ben-manes
Copy link
Contributor Author

Take a look at MapPutIfAbsentTester. These cases would be to allow null values so follow the pattern elsewhere for defining test matching constraints, e.g.

@MapFeature.Require({SUPPORTS_PUT, ALLOWS_NULL_VALUES})
public void testPut_nullValueSupported() {
assertNull("put(key, null) should return null", put(nullValueEntry));
expectAdded(nullValueEntry);
}

It would also probably be good to review the default methods promoted from ConcurrentMap (e.g. replace) and the newer ones like computes and merge to see if the ALLOWS_NULL_VALUES cases are being covered. The tests should be run by OpenJdk6MapTests which includes JDK maps that support or disallow null values, so verifying that yours works as expected means to see that your new tests intentionally fail and then go green when implemented. Otherwise, simply try to stay within the coding style of the surrounding code.

@alvonellos
Copy link
Contributor

Hey ben, I made the code change but I'm running into some issues getting guava to build on my system. I get an error: package sun.misc does not exist

any tips?

@ben-manes
Copy link
Contributor Author

I believe it should work, but I haven't run Guava's build for a long time. I know that Eclipse defaults to that error for restricted apis, if that is what you mean? If so, you can disable it under,

Screen Shot 2022-10-22 at 5 31 07 PM

@ben-manes
Copy link
Contributor Author

I gave it a build and another likely reason is if you are running on recent JDK which requires access modifiers. It worked for me on Java 11, and I believe the library targets Java 8.

@alvonellos
Copy link
Contributor

Take a look at my PR: #6225

@ben-manes
Copy link
Contributor Author

Looks like a good first iteration. Of course that doesn't cover this scenario.

I don't have commit access to the repository, so you'll need to have a champion on the Guava team marshall it through.

@alvonellos
Copy link
Contributor

I'm using IntelliJ. I'm having a hard time getting it to build so I can test that it all works. I signed the CLA. This will be my first contribution to a google project

@ben-manes
Copy link
Contributor Author

You can use command line, e.g. mvn test -Dtest=OpenJdk6MapTests -pl guava-testlib.

@alvonellos
Copy link
Contributor

Thanks, I did that and now I get this message:

alexanderalvonellos@Alexanders-MacBook-Air guava % mvn test -Dtest=OpenJdk6MapTests -pl guava-testlib [INFO] Scanning for projects... [INFO] [INFO] -------------------< com.google.guava:guava-testlib >------------------- [INFO] Building Guava Testing Library HEAD-jre-SNAPSHOT [INFO] --------------------------------[ jar ]--------------------------------- [INFO] [INFO] --- maven-enforcer-plugin:3.0.0-M3:enforce (enforce-versions) @ guava-testlib --- [INFO] [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ guava-testlib --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] Copying 0 resource [INFO] [INFO] --- maven-compiler-plugin:3.8.1:compile (default-compile) @ guava-testlib --- [INFO] Nothing to compile - all classes are up to date [INFO] [INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ guava-testlib --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] Copying 0 resource [INFO] [INFO] --- maven-compiler-plugin:3.8.1:testCompile (default-testCompile) @ guava-testlib --- [INFO] Changes detected - recompiling the module! [INFO] Compiling 37 source files to /Users/alexanderalvonellos/Documents/GitHub/guava/guava-testlib/target/test-classes An exception has occurred in the compiler (18.0.1.1). Please file a bug against the Java compiler via the Java bug reporting page (http://bugreport.java.com) after checking the Bug Database (http://bugs.java.com) for duplicates. Include your program, the following diagnostic, and the parameters passed to the Java compiler in your report. Thank you. java.lang.IllegalAccessError: class com.google.errorprone.BaseErrorProneJavaCompiler (in unnamed module @0x3d24420b) cannot access class com.sun.tools.javac.api.BasicJavacTask (in module jdk.compiler) because module jdk.compiler does not export com.sun.tools.javac.api to unnamed module @0x3d24420b at com.google.errorprone.BaseErrorProneJavaCompiler.addTaskListener(BaseErrorProneJavaCompiler.java:94) at com.google.errorprone.ErrorProneJavacPlugin.init(ErrorProneJavacPlugin.java:34) at jdk.compiler/com.sun.tools.javac.api.BasicJavacTask.initPlugin(BasicJavacTask.java:255) at jdk.compiler/com.sun.tools.javac.api.BasicJavacTask.initPlugins(BasicJavacTask.java:229) at jdk.compiler/com.sun.tools.javac.api.JavacTaskImpl.prepareCompiler(JavacTaskImpl.java:204) at jdk.compiler/com.sun.tools.javac.api.JavacTaskImpl.lambda$doCall$0(JavacTaskImpl.java:101) at jdk.compiler/com.sun.tools.javac.api.JavacTaskImpl.invocationHelper(JavacTaskImpl.java:152) at jdk.compiler/com.sun.tools.javac.api.JavacTaskImpl.doCall(JavacTaskImpl.java:100) at jdk.compiler/com.sun.tools.javac.api.JavacTaskImpl.call(JavacTaskImpl.java:94) at org.codehaus.plexus.compiler.javac.JavaxToolsCompiler.compileInProcess(JavaxToolsCompiler.java:126) at org.codehaus.plexus.compiler.javac.JavacCompiler.performCompile(JavacCompiler.java:174) at org.apache.maven.plugin.compiler.AbstractCompilerMojo.execute(AbstractCompilerMojo.java:1134) at org.apache.maven.plugin.compiler.TestCompilerMojo.execute(TestCompilerMojo.java:180) at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:137) at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2(MojoExecutor.java:370) at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute(MojoExecutor.java:351) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:215) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:171) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:163) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:117) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:81) at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:56) at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128) at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:294) at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:192) at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:105) at org.apache.maven.cli.MavenCli.execute(MavenCli.java:960) at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:293) at org.apache.maven.cli.MavenCli.main(MavenCli.java:196) at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104) at java.base/java.lang.reflect.Method.invoke(Method.java:577) at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:282) at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:225) at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:406) at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:347) [INFO] ------------------------------------------------------------- [ERROR] COMPILATION ERROR : [INFO] ------------------------------------------------------------- [ERROR] An unknown compilation problem occurred [INFO] 1 error [INFO] ------------------------------------------------------------- [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 2.009 s [INFO] Finished at: 2022-10-22T23:03:45-04:00 [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.8.1:testCompile (default-testCompile) on project guava-testlib: Compilation failure [ERROR] An unknown compilation problem occurred [ERROR] [ERROR] -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException

@ben-manes
Copy link
Contributor Author

You are probably running on a newer JDK. It works fine for me on Java 11, but I get this on 17 due reflective access controls being restricted in later versions.

copybara-service bot pushed a commit that referenced this issue Oct 27, 2022
…nd make builds work with newer JDKs.

This should fix the error reported in #6217 (comment). I'm not sure if the Error Prone update was necessary for that or if only the `pom.xml` changes were. Still, it seems inevitable that we'll be forced to upgrade Error Prone eventually, and it's rarely a bad idea to update a plugin.

This change is progress toward building and testing under Java 17 (#5801)...

...which we apparently regressed at when we enabled Error Prone (#2484).

I've set this change up in a way that lets builds continue to work under JDK8 (which is potentially useful for #3990 or for anyone building Guava manually with an old JDK), albeit with Error Prone disabled.

RELNOTES=n/a
PiperOrigin-RevId: 484299394
@cpovirk
Copy link
Member

cpovirk commented Oct 27, 2022

Thanks for taking a look at this, and sorry for the build problems. I am making an attempt at fixing them in #6230.

@ben-manes
Copy link
Contributor Author

I think you can use Maven toolchains for handling this, but I am rusty with that build tool.

copybara-service bot pushed a commit that referenced this issue Nov 3, 2022
This has some effects on us as we develop Guava, but it doesn't affect end users.

Specifically:

- When we build Guava with JDK8, we now have to disable Error Prone. (Again, users can still use Guava with JDK8, including with Error Prone. The effect is limited to people who are developing Guava.)
- We can now successfully build Guava with Error Prone under more recent JDKs. That is, this change should fix the error reported in #6217 (comment). (I'm not sure if the Error Prone update was necessary for that or if only the other `pom.xml` changes were. Still, it seems inevitable that we'll be forced to upgrade Error Prone eventually, and it's rarely a bad idea to update a plugin.)

This change is progress toward building and testing under Java 17 (#5801)...

...which we apparently regressed at when we enabled Error Prone (#2484).

Oddly, it seems that part of our existing Error Prone setup is _required_ to continue building Guava under JDK8. (Such builds are potentially useful for #3990 or for anyone building Guava manually with an old JDK.) That's the case even though we're now disabling Error Prone for those builds.

Again, all these changes affect only people who are developing Guava, not end users.

RELNOTES=n/a
PiperOrigin-RevId: 484299394
copybara-service bot pushed a commit that referenced this issue Nov 3, 2022
This has some effects on us as we develop Guava, but it doesn't affect end users.

Specifically:

- When we build Guava with JDK8, we now have to disable Error Prone. (Again, users can still use Guava with JDK8, including with Error Prone. The effect is limited to people who are developing Guava.)
- We can now successfully build Guava with Error Prone under more recent JDKs. That is, this change should fix the error reported in #6217 (comment). (I'm not sure if the Error Prone update was necessary for that or if only the other `pom.xml` changes were. Still, it seems inevitable that we'll be forced to upgrade Error Prone eventually, and it's rarely a bad idea to update a plugin.)

This change is progress toward building and testing under Java 17 (#5801)...

...which we apparently regressed at when we enabled Error Prone (#2484).

Oddly, it seems that part of our existing Error Prone setup is _required_ to continue building Guava under JDK8. (Such builds are potentially useful for #3990 or for anyone building Guava manually with an old JDK.) That's the case even though we're now disabling Error Prone for those builds.

Again, all these changes affect only people who are developing Guava, not end users.

RELNOTES=n/a
PiperOrigin-RevId: 485770768
@Speiger
Copy link

Speiger commented Nov 13, 2022

@ben-manes does this also affect my lib?
(Didn't have time to check this yet)

@ben-manes
Copy link
Contributor Author

@Speiger yep, your Object2ObjectOpenHashMap fails the test.

@ben-manes
Copy link
Contributor Author

It is also probably worth noting that Map.putIfAbsent and ConcurrentMap.putIfAbsent are defined differently wrt null values. The behavior of ConcurrentMap is insert-only (via a contains guard) whereas Map is insert-or-update (a get guard). That makes them inconsistent likely to allow for a faster default method when promoted, but it seems more like a bug that they decided not to fix. Of course null values in a ConcurrentMap is strongly discouraged, but unfortunately some do this with the usual mayhem. A test would need to decide which strategy to assert based on the type, or else allow either and document that the behavior is assumed to be implementation specific.

@Speiger
Copy link

Speiger commented Nov 13, 2022

Ty @ben-manes
I will address this bug most likely next week.
Also adding unit tests for it.

Yeah I allow null values in my ConcurrentMaps, since they are guava styled its actually rather easy to support null.
Since its just a fixed spot in the Map its not even causing issues.

@ben-manes
Copy link
Contributor Author

The segment style in Guava's is forked from Java 5's ConcurrentHashMap. There are definite gotchas with atomicity and null values, and I think allowing null values at all was a mistake for Map. It is best replaced by a sentinel value, e.g. Optional, which has the same cpu and memory footprint (as no one actually makes a valueless entry object for this case in their maps). My general rule is to follow Doug Lea's decisions, as if he found a feature too hard and confusing to support then everyone else will get it wrong (either the implementor or user, as too much of a footgun for a consistent understanding).

@Speiger
Copy link

Speiger commented Dec 16, 2022

@ben-manes while i agree to 99% of it, the one thing I disagree, I just realised so that's why the comment comes just now.
And that is if a element isn't present it should return some form of null/default value instead like C# crash.
Because all that cases is that you will implement a "safe" returning implementation that deals with the crash and returns a defaultValue/null instead (or in c#s case use the tryGet method for everything).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

6 participants