JsonUnmarshallerContext dependency issue with fasterxml jackson

[majestique]

I'm using aws sdk dynamo version 1.12.351 and jackson fasterxml version 2.12.1. It looks like a few other people have the same issue, but it's not resolved.

Other related issues:
(old in 2018) localstack/localstack#1031
(more recent, unresolved) #2909

I believe there are some dependency issues with dynamo & jackson, that's why I'm getting the error below. I've provided the error, pom & dependency tree. Help is much appreciated.

I've also tried aws sdk version 1.12.410 & jackson fasterxml 2.14.2, but the same error exists.

2023-02-20 02:46:09,805 - myapp - [main] ERROR org.apache.zookeeper.server.NIOServerCnxnFactory - Thread Thread[main,5,main] died
java.lang.NoSuchMethodError: com.amazonaws.transform.JsonUnmarshallerContext.getCurrentToken()Lshaded/com/fasterxml/jackson/core/JsonToken;
	at com.amazonaws.services.dynamodbv2.model.transform.CreateTableResultJsonUnmarshaller.unmarshall(CreateTableResultJsonUnmarshaller.java:39)
	at com.amazonaws.services.dynamodbv2.model.transform.CreateTableResultJsonUnmarshaller.unmarshall(CreateTableResultJsonUnmarshaller.java:29)
	at com.amazonaws.http.JsonResponseHandler.handle(JsonResponseHandler.java:118)
	at com.amazonaws.http.JsonResponseHandler.handle(JsonResponseHandler.java:43)
	at com.amazonaws.http.response.AwsResponseHandlerAdapter.handle(AwsResponseHandlerAdapter.java:69)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleResponse(AmazonHttpClient.java:1794)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleSuccessResponse(AmazonHttpClient.java:1477)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1384)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1157)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:814)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:781)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:755)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:715)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:697)
	at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:561)
	at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:541)
	at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.doInvoke(AmazonDynamoDBClient.java:6470)
	at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.invoke(AmazonDynamoDBClient.java:6437)
	at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.executeCreateTable(AmazonDynamoDBClient.java:1198)
	at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.createTable(AmazonDynamoDBClient.java:1162)
	at com.mycomp.mlp.myapp.spark.structured.local.LocalEnvironment$.setupLocalDynamo(LocalEnvironment.scala:413)
	at com.mycomp.mlp.myapp.spark.structured.local.LocalEnvironment$.initLocalEnv(LocalEnvironment.scala:88)
	at com.mycomp.mlp.myapp.spark.myapp.init(myapp.scala:104)
	at com.mycomp.mlp.myapp.spark.myapp$.mianExec(myapp.scala:182)
	at com.mycomp.mlp.myapp.spark.myapp$.main(myapp.scala:179)
	at com.mycomp.pegasus.ps.myapp.srf.money.myappApplication$.main(myappApplication.scala:14)
	at com.mycomp.pegasus.ps.myapp.srf.money.myappApplication.main(myappApplication.scala)

Here's my pom:

Click for pom details

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>com.mycompany.sbg.ml.spark</groupId>
    <artifactId>myapp-runner</artifactId>
    <version>1.4-SNAPSHOT</version>

    <parent>
        <groupId>com.mycompany.sbg.ml.spark</groupId>
        <artifactId>myapp-aggregator</artifactId>
        <version>1.4-SNAPSHOT</version>
        <relativePath>../pom.xml</relativePath>
    </parent>

    <properties>
        <jersey.version>2.26</jersey.version>
        <tiger.graph.data.mgr.version>3.1.7</tiger.graph.data.mgr.version>
        <uber.jar.ext>jar-with-dependencies</uber.jar.ext>
        <scoverage.plugin.version>1.4.0</scoverage.plugin.version>
        <aws.version>1.12.351</aws.version>
        <jackson.version>2.12.1</jackson.version>
    </properties>

    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>com.amazonaws</groupId>
                <artifactId>aws-java-sdk-dynamodb</artifactId>
                <version>${aws.version}</version>
            </dependency>
            <dependency>
                <groupId>com.amazonaws</groupId>
                <artifactId>aws-java-sdk-core</artifactId>
                <version>${aws.version}</version>
            </dependency>
            <dependency>
                <groupId>com.amazonaws</groupId>
                <artifactId>aws-java-sdk</artifactId>
                <version>${aws.version}</version>
            </dependency>
            <dependency>
                <groupId>com.amazonaws</groupId>
                <artifactId>aws-java-sdk-sts</artifactId>
                <version>${aws.version}</version>
            </dependency>

        </dependencies>
    </dependencyManagement>

    <dependencies>
        <dependency>
            <groupId>com.fasterxml.jackson.core</groupId>
            <artifactId>jackson-annotations</artifactId>
            <version>${jackson.version}</version>
        </dependency>
        <dependency>
            <groupId>com.fasterxml.jackson.core</groupId>
            <artifactId>jackson-databind</artifactId>
            <version>${jackson.version}</version>
        </dependency>
        <dependency>
            <groupId>com.fasterxml.jackson.core</groupId>
            <artifactId>jackson-core</artifactId>
            <version>${jackson.version}</version>
        </dependency>
        <dependency>
            <groupId>com.fasterxml.jackson.dataformat</groupId>
            <artifactId>jackson-dataformat-cbor</artifactId>
            <version>${jackson.version}</version>
        </dependency>
        <dependency>
            <groupId>com.fasterxml.jackson.module</groupId>
            <artifactId>jackson-module-scala_2.12</artifactId>
            <version>${jackson.version}</version>
        </dependency>
        <dependency>
            <groupId>com.fasterxml.jackson.dataformat</groupId>
            <artifactId>jackson-dataformat-csv</artifactId>
            <version>${jackson.version}</version>
        </dependency>
        <dependency>
            <groupId>com.fasterxml.jackson.datatype</groupId>
            <artifactId>jackson-datatype-jdk8</artifactId>
            <version>${jackson.version}</version>
        </dependency>
        <dependency>
            <groupId>com.fasterxml.jackson.dataformat</groupId>
            <artifactId>jackson-dataformat-yaml</artifactId>
            <version>${jackson.version}</version>
        </dependency>
        <dependency>
            <groupId>com.fasterxml.jackson.jaxrs</groupId>
            <artifactId>jackson-jaxrs-json-provider</artifactId>
            <version>${jackson.version}</version>
        </dependency>
        <dependency>
            <groupId>org.apache.kafka</groupId>
            <artifactId>kafka_2.12</artifactId>
            <version>2.6.0</version>
        </dependency>
        <dependency>
            <groupId>com.mycompany.idps</groupId>
            <artifactId>idps-sdk</artifactId>
            <version>3.90.0</version>
            <exclusions>
                <exclusion>
                    <groupId>javax.ws.rs</groupId>
                    <artifactId>jsr311-api</artifactId>
                </exclusion>
                <exclusion>
                    <groupId>com.sun.jersey</groupId>
                    <artifactId>jersey-client</artifactId>
                </exclusion>
                <exclusion>
                    <groupId>com.sun.jersey</groupId>
                    <artifactId>jersey-core</artifactId>
                </exclusion>
                <exclusion>
                    <groupId>com.sun.jersey.contribs</groupId>
                    <artifactId>jersey-multipart</artifactId>
                </exclusion>

            </exclusions>
        </dependency>
        <dependency>
            <groupId>com.mycompany.sbg.ml.spark</groupId>
            <artifactId>myapp-handler</artifactId>
            <version>${project.version}</version>
        </dependency>
        <dependency>
            <groupId>org.scala-lang</groupId>
            <artifactId>scala-library</artifactId>
            <version>2.12.10</version>
        </dependency>
        <dependency>
            <groupId>org.scala-lang</groupId>
            <artifactId>scala-reflect</artifactId>
            <version>2.12.10</version>
        </dependency>
        <dependency>
            <groupId>org.glassfish.jersey.media</groupId>
            <artifactId>jersey-media-json-jackson</artifactId>
            <version>${jersey.version}</version>
            <exclusions>
                <exclusion>
                    <groupId>com.fasterxml.jackson.core</groupId>
                    <artifactId>jackson-annotations</artifactId>
                </exclusion>
                <exclusion>
                    <groupId>com.fasterxml.jackson.core</groupId>
                    <artifactId>jackson-databind</artifactId>
                </exclusion>
            </exclusions>
        </dependency>

        <dependency>
            <groupId>org.glassfish.jersey.ext</groupId>
            <artifactId>jersey-entity-filtering</artifactId>
            <version>${jersey.version}</version>
        </dependency>
        <dependency>
            <groupId>org.glassfish.jersey.core</groupId>
            <artifactId>jersey-server</artifactId>
            <version>${jersey.version}</version>
        </dependency>
        <dependency>
            <groupId>org.glassfish.jersey.containers</groupId>
            <artifactId>jersey-container-servlet</artifactId>
            <version>${jersey.version}</version>
        </dependency>
        <dependency>
            <groupId>org.glassfish.jersey.containers</groupId>
            <artifactId>jersey-container-servlet-core</artifactId>
            <version>${jersey.version}</version>

        </dependency>
        <dependency>
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-core_2.12</artifactId>
            <version>3.3.0</version>
        </dependency>
        <dependency>
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-sql_2.12</artifactId>
            <version>3.3.0</version>
        </dependency>
        <dependency>
            <groupId>org.glassfish.jersey.media</groupId>
            <artifactId>jersey-media-json-jackson</artifactId>
            <version>${jersey.version}</version>
        </dependency>
        <dependency>
            <groupId>org.glassfish.jersey.ext</groupId>
            <artifactId>jersey-entity-filtering</artifactId>
            <version>${jersey.version}</version>
        </dependency>
        <dependency>
            <groupId>com.mycompany.css.voyager</groupId>
            <artifactId>tg-writer</artifactId>
            <version>${tiger.graph.data.mgr.version}</version>
            <exclusions>
                <exclusion>
                    <groupId>org.apache.spark</groupId>
                    <artifactId>spark-core_2.11</artifactId>
                </exclusion>
                <exclusion>
                    <groupId>org.apache.spark</groupId>
                    <artifactId>spark-sql_2.11</artifactId>
                </exclusion>
                <exclusion>
                    <groupId>org.apache.spark</groupId>
                    <artifactId>spark-hive_2.12</artifactId>
                </exclusion>
                <exclusion>
                    <groupId>org.scala-lang</groupId>
                    <artifactId>*</artifactId>
                </exclusion>
                <exclusion>
                    <groupId>com.mycompany.css.voyager.schema</groupId>
                    <artifactId>tg-salesforce-schema</artifactId>
                </exclusion>
                <exclusion>
                    <groupId>com.amazonaws</groupId>
                    <artifactId>aws-java-sdk-core</artifactId>
                </exclusion>
                <exclusion>
                    <groupId>com.tigergraph</groupId>
                    <artifactId>tg-jdbc-driver</artifactId>
                </exclusion>
            </exclusions>
        </dependency>
        <dependency>
            <groupId>com.amazon.emr.fork</groupId>
            <artifactId>emr-dynamodb-connector</artifactId>
            <version>4.15.1</version>
        </dependency>
    </dependencies>

    <build>
        <finalName>myapp</finalName>
        <plugins>
            <plugin>
                <artifactId>maven-shade-plugin</artifactId>
                <version>3.2.2</version>
                <configuration>
                    <finalName>${project.artifactId}-${uber.jar.ext}</finalName>
                    <shadedArtifactAttached>true</shadedArtifactAttached>
                    <shadedClassifierName>${uber.jar.ext}</shadedClassifierName>
                </configuration>
                <executions>
                    <execution>
                        <phase>package</phase>
                        <goals>
                            <goal>shade</goal>
                        </goals>
                        <configuration>
                            <finalName>${project.artifactId}-${uber.jar.ext}</finalName>
                            <artifactSet>
                                <excludes>
                                    <exclude>com.almworks.sqlite4java:*</exclude>
                                    <exclude>io.github.ganadist.sqlite4java:*</exclude>
                                    <exclude>org.bouncycastle:*:*:*</exclude>
                                </excludes>
                            </artifactSet>
                            <transformers>
                                <!--
                                    This AppendingTransformer is required for uber jars when dealin with akka packages.
                                     It expects a reference.conf to be in the JAR.
                                 -->
                                <transformer
                                        implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer">
                                    <resource>reference.conf</resource>
                                </transformer>

                                <transformer
                                        implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
                                    <mainClass>com.mycompany.mlp.myapp.spark.Application</mainClass>
                                </transformer>

                                <transformer
                                        implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer"/>

                            </transformers>
                            <filters>
                                <filter>
                                    <artifact>*:*</artifact>
                                    <excludes>
                                        <exclude>META-INF/*.SF</exclude>
                                        <exclude>META-INF/*.DSA</exclude>
                                        <exclude>META-INF/*.RSA</exclude>
                                    </excludes>
                                </filter>
                            </filters>
                            <relocations>
                                <relocation>
                                    <pattern>org.apache.hadoop.dynamodb</pattern>
                                    <shadedPattern>shaded.org.apache.hadoop.dynamodb</shadedPattern>
                                </relocation>
                                <relocation>
                                    <pattern>net.razorvine</pattern>
                                    <shadedPattern>shaded.net.razorvine</shadedPattern>
                                </relocation>
                                <relocation>
                                    <pattern>com.squareup.okhttp</pattern>
                                    <shadedPattern>shaded.com.squareup.okhttp</shadedPattern>
                                </relocation>
                                <relocation>
                                    <pattern>com.fasterxml.jackson</pattern>
                                    <shadedPattern>shaded.com.fasterxml.jackson</shadedPattern>
                                </relocation>
                                <relocation>
                                    <pattern>com.google.protobuf</pattern>
                                    <shadedPattern>shaded.com.google.protobuf</shadedPattern>
                                </relocation>
                            </relocations>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <groupId>net.alchim31.maven</groupId>
                <artifactId>scala-maven-plugin</artifactId>
                <version>3.2.2</version>
                <configuration>
                    <recompileMode>incremental</recompileMode>
                </configuration>
                <executions>
                    <execution>
                        <id>scala-compile-first</id>
                        <phase>process-resources</phase>
                        <goals>
                            <goal>add-source</goal>
                            <goal>compile</goal>
                        </goals>
                    </execution>
                    <execution>
                        <id>scala-test-compile</id>
                        <phase>process-test-resources</phase>
                        <goals>
                            <goal>testCompile</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>

            <plugin>
                <groupId>org.scoverage</groupId>
                <artifactId>scoverage-maven-plugin</artifactId>
                <version>${scoverage.plugin.version}</version>
                <configuration>
                    <excludedPackages>com.mycompany.mlp.myapp.spark.structured.reader.proto</excludedPackages>
                    <scalacPluginVersion>1.4.0</scalacPluginVersion>
                    <minimumCoverage>0</minimumCoverage>
                    <failOnMinimumCoverage>true</failOnMinimumCoverage>
                </configuration>
                <executions>
                    <execution>
                        <goals>
                            <goal>check</goal> <!-- or integration-check -->
                        </goals>
                    </execution>
                </executions>
            </plugin>

            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-dependency-plugin</artifactId>
                <version>2.10</version>
                <executions>
                    <execution>
                        <id>copy</id>
                        <phase>test-compile</phase>
                        <goals>
                            <goal>copy-dependencies</goal>
                        </goals>
                        <configuration>
                            <includeScope>test</includeScope>
                            <includeTypes>so,dll,dylib</includeTypes>
                            <outputDirectory>native-libs</outputDirectory>
                        </configuration>
                    </execution>
                </executions>
            </plugin>

            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-enforcer-plugin</artifactId>
                <version>1.0</version>
                <executions>
                    <execution>
                        <goals>
                            <goal>enforce</goal>
                        </goals>
                        <configuration>
                            <rules>
                                <bannedDependencies>
                                    <excludes>
                                        <exclude>org.slf4j:log4j-over-slf4j</exclude>
                                    </excludes>
                                </bannedDependencies>
                            </rules>
                        </configuration>
                    </execution>
                </executions>
            </plugin>

            <plugin>
                <groupId>com.mycompany</groupId>
                <artifactId>idps-maven-plugin</artifactId>
                <version>0.0.2</version>
            </plugin>

            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-jar-plugin</artifactId>
                <executions>
                    <execution>
                        <goals>
                            <goal>test-jar</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>

            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>2.0.2</version>
                <configuration>
                    <source>1.8</source>
                    <target>1.8</target>
                    <encoding>ISO-8859-1</encoding>
                </configuration>
                <executions>
                    <execution>
                        <phase>compile</phase>
                        <goals>
                            <goal>compile</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>

            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>2.17</version>
                <configuration>
                    <skipTests>false</skipTests>
                    <reuseForks>true</reuseForks>
                    <argLine>-Xmx1024m -XX:MaxPermSize=256m</argLine>
                </configuration>
            </plugin>

        </plugins>

    </build>

</project>

My dependency tree:
dependencytree.txt

[debora-ito]

Hi @majestique I edited your description and put the pom.xml behind a collapsible section, so it's easier to scroll down.

NoSuchMethodError usually indicates that the method was present at compile time but it was not found at runtime. My guess it's because you are shading com.fasterxml.jackson in your project, or the version you are using does not match with the version expected by the SDK, but I'm not sure. This is not a Java SDK bug, it's an issue with configuring dependencies.

A side note: the aws-java-sdk artifact is adding all the SDK modules as dependencies, and this is not recommended. We recommend to only add the service modules you are going to use.

            <dependency>
                <groupId>com.amazonaws</groupId>
                <artifactId>aws-java-sdk</artifactId>
                <version>${aws.version}</version>
            </dependency>

[majestique]

Thanks @debora-ito for the tip, I'll continue trying different dependency configurations.