Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

it cause an exception when query http plugin data with fullName #2851

Open
shfshihuafeng opened this issue Nov 27, 2023 · 7 comments
Open

it cause an exception when query http plugin data with fullName #2851

shfshihuafeng opened this issue Nov 27, 2023 · 7 comments
Labels

Comments

@shfshihuafeng
Copy link
Contributor

shfshihuafeng commented Nov 27, 2023

Before submitting a bug report, please verify that you are using the most current version of Drill.

Describe the bug
i create a http plugin named "http" and then use "use http" command to switch schema.
i query http Data with "select * from http.sunrise",It report except

To Reproduce
Steps to reproduce the behavior:

1. create plugin with name "http"
2. use http
3. select * from http.sunrise;

Expected behavior

apache drill> select * from http.sunrise;
+----------------------------------------------------------------------------------+--------+
|                                     results                                      | status |
+----------------------------------------------------------------------------------+--------+
| {"sunrise":"5:42:47 AM","sunset":"5:52:15 PM","solar_noon":"11:47:31 AM","day_length":"12:09:28","civil_twilight_begin":"5:21:47 AM","civil_twilight_end":"6:13:15 PM","nautical_twilight_begin":"4:56:00 AM","nautical_twilight_end":"6:39:02 PM","astronomical_twilight_begin":"4:30:08 AM","astronomical_twilight_end":"7:04:54 PM"} | OK     |
+----------------------------------------------------------------------------------+--------+
1 row selected (2.967 seconds)

Error detail, log output or screenshots
Error: CONNECTION ERROR: API 'http' does not exist in HTTP storage plugin 'http'

Drill version
1.22.0

Additional context

http plugin config
{
  "type": "http",
  "connections": {
    "sunrise": {
      "url": "https://api.sunrise-sunset.org/json",
      "requireTail": false,
      "method": "GET",
      "params": [
        "lat",
        "lng",
        "date"
      ],
      "authType": "none",
      "inputType": "json",
      "xmlDataLevel": 1,
      "postParameterLocation": "QUERY_STRING",
      "verifySSLCert": true
    }
  },
  "timeout": 5,
  "retryDelay": 1000,
  "proxyType": "direct",
  "authMode": "SHARED_USER",
  "enabled": true
}
@shfshihuafeng
Copy link
Contributor Author

shfshihuafeng commented Nov 27, 2023

we should determines whether the parameter name equals the plug-in name ?

--- a/contrib/storage-http/src/main/java/org/apache/drill/exec/store/http/HttpSchemaFactory.java
+++ b/contrib/storage-http/src/main/java/org/apache/drill/exec/store/http/HttpSchemaFactory.java
@@ -18,8 +18,11 @@
 package org.apache.drill.exec.store.http;

 import java.util.Collections;
+import java.util.Locale;
 import java.util.Map;
 import java.util.Map.Entry;
+import java.util.Set;
+import java.util.stream.Collectors;

 import org.apache.calcite.schema.SchemaPlus;
 import org.apache.calcite.schema.Table;
@@ -85,7 +88,7 @@ public class HttpSchemaFactory extends AbstractSchemaFactory {
       HttpAPIConnectionSchema subSchema = subSchemas.get(name);
       if (subSchema != null) {
         return subSchema;
-      } else if (tables.containsKey(name)) {
+      } else if (tables.containsKey(name) || isRegistryPluginName(name.toLowerCase(Locale.ROOT))) {
         return null;
       } else {
         throw UserException
@@ -95,6 +98,12 @@ public class HttpSchemaFactory extends AbstractSchemaFactory {
       }
     }

+    private boolean isRegistryPluginName(String name) {
+      Set<String> pluginNames = plugin.getRegistry().availablePlugins();
+      Set<String> pluginNamesToLower = pluginNames.stream().map(String::toLowerCase).collect(Collectors.toSet());
+      return pluginNamesToLower.contains(name);
+    }

@cgivre
Copy link
Contributor

cgivre commented Nov 27, 2023

@shfshihuafeng I don't think this is a bug. You are first running the query USE http which sets the root at http. Then you're running a SELECT ... FROM http.sunrise. So at that point, the plugin is looking for a path: http.http.sunrise which does not exist.

I'd bet that if you either skipped the USE query or ran a SELECT ... FROM sunrise it would work.

@Javelin2007
Copy link

Javelin2007 commented Nov 28, 2023

@cgivre You are right, but when use http, the default schema is http, when we use another data source schema, will report error.
image
It think this is not right.

@shfshihuafeng
Copy link
Contributor Author

@cgivre
image

@shfshihuafeng
Copy link
Contributor Author

shfshihuafeng commented Nov 28, 2023

@cgivre
when i use "use my.test" command to switch schema, It is correct to query data with full schema Name from other data sources
0880c4c80710fc8ae113

@cgivre
Copy link
Contributor

cgivre commented Dec 6, 2023

@shfshihuafeng
So are you saying the behavior with USE is inconsistent? I want to make sure I understand the issue.

@shfshihuafeng
Copy link
Contributor Author

@cgivre
There are two problems with this

  1. behavior with USE is inconsistent
  2. when i enter the scehma with 'use',data can not be queried
    image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants