Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Bug] Fix UnknownHostException: host.docker.internal in docker demo #429

Closed
daragu opened this issue Apr 30, 2024 · 1 comment · Fixed by #430
Closed

[Bug] Fix UnknownHostException: host.docker.internal in docker demo #429

daragu opened this issue Apr 30, 2024 · 1 comment · Fixed by #430
Labels
question Further information is requested

Comments

@daragu
Copy link
Contributor

daragu commented Apr 30, 2024 •
edited

Throw UnknownHostException when try the demo in docker under Linux OS.

MetaException(message:Could not connect to meta store using any of the URIs provided. Most recent failure: org.apache.thrift.transport.TTransportException: java.net.UnknownHostException: host.docker.internal
	at org.apache.thrift.transport.TSocket.open(TSocket.java:226)
	at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:478)
	at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:245)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1740)
	at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:83)
	at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:133)
	at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
	at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:97)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.iceberg.common.DynMethods$UnboundMethod.invokeChecked(DynMeth

图片
@daragu daragu mentioned this issue Apr 30, 2024
Merged
@vinishjail97
Copy link
Contributor

I'm not able to re-produce the issue and the demo works for me in local.

namespace: String = "demo"
namespaceArray: Array[String] = Array("demo")
catalogOptions: HashMap[String, String] = {uri=thrift://host.docker.internal:9083}
res8_3: String = null
icebergCatalogConfig: IcebergCatalogConfig = IcebergCatalogConfig(catalogImpl=org.apache.iceberg.hive.HiveCatalog, catalogName=iceberg, catalogOptions={uri=thrift://host.docker.internal:9083})
perTableConfig: PerTableConfigImpl = PerTableConfigImpl(tableBasePath=file:///home/data/db/hudi_dimCustomer, tableDataPath=file:/home/data/db/hudi_dimCustomer, tableName=hudi_dimCustomer, namespace=[demo], hudiSourceConfig=HudiSourceConfigImpl(partitionSpecExtractorClass=org.apache.xtable.hudi.ConfigurationBasedPartitionSpecExtractor, partitionFieldSpecs=[]), targetTableFormats=[ICEBERG], icebergCatalogConfig=IcebergCatalogConfig(catalogImpl=org.apache.iceberg.hive.HiveCatalog, catalogName=iceberg, catalogOptions={uri=thrift://host.docker.internal:9083}), syncMode=INCREMENTAL, targetMetadataRetentionInHours=168)
hudiConversionSourceProvider: HudiConversionSourceProvider = org.apache.xtable.hudi.HudiConversionSourceProvider@76964a4
conversionController: ConversionController = org.apache.xtable.conversion.ConversionController@7841cb4e
res8_9: Map[String, SyncResult] = {ICEBERG=SyncResult(mode=FULL, lastInstantSynced=2023-11-05T17:21:36.406Z, syncStartTime=2024-05-11T17:48:09.633Z, syncDuration=PT2.756S, status=SyncResult.SyncStatus(statusCode=SUCCESS, errorMessage=null, errorDescription=null, canRetryOnFailure=false), recommendedSyncMode=null)}

I don't have any special configuration in /etc/hosts as well for mapping host.docker.internal. Can you upgrade the docker version you are using and see if it resolves the issue ? This is the version I'm using.

image

@vinishjail97 vinishjail97 added the question Further information is requested label May 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
question Further information is requested
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[Demo] Fix UnknownHostException: host.docker.internal in docker demo daragu/incubator-xtable
2 participants
@vinishjail97 @daragu