Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] Duplicated shaded-grpc jar cause task fail #1548

Open
3 tasks done
lifeSo opened this issue Feb 28, 2024 · 3 comments
Open
3 tasks done

[Bug] Duplicated shaded-grpc jar cause task fail #1548

lifeSo opened this issue Feb 28, 2024 · 3 comments
Assignees

Comments

@lifeSo
Copy link
Collaborator

lifeSo commented Feb 28, 2024

Code of Conduct

Search before asking

  • I have searched in the issues and found no similar issues.

Describe the bug

There is shaded-grpc jar in client-tez,
When there is also shaded-grpc jar in user UDF,
then, there will be sometimes task fail.

Uploading image.png…

Affects Version(s)

0.7.0

Uniffle Server Log Output

No response

Uniffle Engine Log Output

###|||2024-02-23 11:07:41.996|||INFO|||-|||-|||main|||-|||RssDAGAppMaster--->Registering coordinators 172.19.193.73:19999,172.19.193.182:19999
###|||2024-02-23 11:07:41.996|||INFO|||-|||-|||main|||-|||CoordinatorClientFactory--->Start to create coordinator clients from 172.19.193.73:19999,172.19.193.182:19999
###|||2024-02-23 11:07:42.013|||ERROR|||-|||-|||main|||-|||RssDAGAppMaster--->Error starting RssDAGAppMaster
java.lang.NoSuchFieldError: EPOLL_DOMAIN_CLIENT_CHANNEL_TYPE
        at io.grpc.netty.shaded.io.grpc.netty.UdsNettyChannelProvider.isAvailable(UdsNettyChannelProvider.java:34)
        at io.grpc.ManagedChannelRegistry$ManagedChannelPriorityAccessor.isAvailable(ManagedChannelRegistry.java:172)
        at io.grpc.ManagedChannelRegistry$ManagedChannelPriorityAccessor.isAvailable(ManagedChannelRegistry.java:168)
        at io.grpc.ServiceProviders.loadAll(ServiceProviders.java:68)
        at io.grpc.ManagedChannelRegistry.getDefaultRegistry(ManagedChannelRegistry.java:96)
        at io.grpc.ManagedChannelProvider.provider(ManagedChannelProvider.java:41)
        at io.grpc.ManagedChannelBuilder.forAddress(ManagedChannelBuilder.java:39)
        at org.apache.uniffle.client.impl.grpc.GrpcClient.<init>(GrpcClient.java:43)
        at org.apache.uniffle.client.impl.grpc.CoordinatorGrpcClient.<init>(CoordinatorGrpcClient.java:91)
        at org.apache.uniffle.client.impl.grpc.CoordinatorGrpcClient.<init>(CoordinatorGrpcClient.java:87)
        at org.apache.uniffle.client.impl.grpc.CoordinatorGrpcClient.<init>(CoordinatorGrpcClient.java:83)
        at org.apache.uniffle.client.factory.CoordinatorClientFactory.createCoordinatorClient(CoordinatorClientFactory.java:43)
        at org.apache.uniffle.client.factory.CoordinatorClientFactory.createCoordinatorClient(CoordinatorClientFactory.java:69)
        at org.apache.uniffle.client.impl.ShuffleWriteClientImpl.registerCoordinators(ShuffleWriteClientImpl.java:431)
        at org.apache.tez.dag.app.RssDAGAppMaster.initAndStartRSSClient(RssDAGAppMaster.java:144)
        at org.apache.tez.dag.app.RssDAGAppMaster.serviceInit(RssDAGAppMaster.java:107)
        at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
        at org.apache.tez.dag.app.DAGAppMaster$9.run(DAGAppMaster.java:2662)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
        at org.apache.tez.dag.app.DAGAppMaster.initAndStartAppMaster(DAGAppMaster.java:2659)
        at org.apache.tez.dag.app.RssDAGAppMaster.main(RssDAGAppMaster.java:348)

Uniffle Server Configurations

No response

Uniffle Engine Configurations

No response

Additional context

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!
@lifeSo
Copy link
Collaborator Author

lifeSo commented Feb 28, 2024

In tez-client:
image

There is no "EPOLL_DOMAIN_CLIENT_CHANNEL_TYPE" field in user shaded-grpc jar:
image

@zuston
Copy link
Member

zuston commented Mar 14, 2024

Could you help fix this? @lifeSo

@lifeSo
Copy link
Collaborator Author

lifeSo commented Mar 18, 2024

Could you help fix this? @lifeSo

OK, I could try, remove or rename the shaded jar name to other, to not conflict user's jar.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants