Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Integration]: DuckDuckGo Zero Click Info API Integration #5642

Open
martyna-mindsdb opened this issue Apr 15, 2023 · 8 comments · May be fixed by #8903
Open

[Integration]: DuckDuckGo Zero Click Info API Integration #5642

martyna-mindsdb opened this issue Apr 15, 2023 · 8 comments · May be fixed by #8903
Assignees
@Sekhar-Kumar-Dash
Labels
app-integration New idea for the supported applications integrations help wanted Extra attention is needed
Projects
Community Contributors

Comments

@martyna-mindsdb
Copy link
Collaborator

martyna-mindsdb commented Apr 15, 2023 •
edited

The goal of this issue is to build a new handler that will utilize the DuckDuckGo Zero Click Info API to retrieve search results and other information directly from DuckDuckGo without having to click on any search results. This integration will enhance the search capabilities of our application by providing quick and relevant information to users without leaving the application.

Resources 💡

🏁 To get started, please refer to the following resources:

  • 📚 Building Application Handler docs: This documentation will guide you through the process of building a custom handler for our application. It contains essential information on the handler structure, methods, and best practices.

  • 📖 DuckDuckGo Zero Click Info API docs: This documentation contains all the endpoints and parameters that are available through the DuckDuckGo Zero Click Info API.

  • 👉 Twitter Handler: This existing handler connects MindsDB with the Twitter API, allowing users to automate tasks related to posting and analyzing tweets. You can use the Twitter Handler as a reference for building the Shopify API integration.

  • 👉 Binance API Handler: Another example of a successful MindsDB integration to the Binance API to make trade data available to use for model training and predictions.

Next Steps 🕵️‍♂️ 🕵️‍♀️

  • Familiarize yourself with the DuckDuckGo Zero Click Info API.
  • Create a new handler that connects our application to the DuckDuckGo Zero Click Info API.
  • Implement the core features mentioned above, such as retrieving search results and other information.
  • Thoroughly test the integration to ensure proper functionality.
  • Submit a pull request with the new DuckDuckGo Zero Click Info API Integration handler.

Thank you for contributing to MindsDB 🐻 !
@martyna-mindsdb martyna-mindsdb added help wanted Extra attention is needed app-integration New idea for the supported applications integrations hashnode-hackhathon App integrations issues for HashnodeHackhathon labels Apr 15, 2023
@martyna-mindsdb martyna-mindsdb removed the hashnode-hackhathon App integrations issues for HashnodeHackhathon label May 4, 2023
@parthi-siva
Copy link

I'm working on this @martyna-mindsdb

@martyna-mindsdb martyna-mindsdb moved this from To do to In progress in Community Contributors May 8, 2023
@parthi-siva
Copy link

Hi @martyna-mindsdb ..

the following is the schema of response of duckduckgo api

{
  "type": "object",
  "properties": {
    "Abstract": {
      "type": "string"
    },
    "AbstractSource": {
      "type": "string"
    },
    "AbstractText": {
      "type": "string"
    },
    "AbstractURL": {
      "type": "string",
      "format": "uri"
    },
    "Answer": {
      "type": "integer"
    },
    "AnswerType": {
      "type": "string"
    },
    "Definition": {
      "type": "string",
      "format": "style"
    },
    "DefinitionSource": {
      "type": "string"
    },
    "DefinitionURL": {
      "type": "string",
      "format": "uri"
    },
    "Heading": {
      "type": "string"
    },
    "Image": {
      "type": "string",
      "format": "uri"
    },
    "Redirect": {
      "type": "string",
      "format": "uri"
    },
    "RelatedTopics": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "FirstURL": {
            "type": "string",
            "format": "uri"
          },
          "Icon": {
            "type": "object",
            "properties": {
              "Height": {
                "type": "integer"
              },
              "URL": {
                "type": "string",
                "format": "uri"
              },
              "Width": {
                "type": "integer"
              }
            }
          },
          "Result": {
            "type": "string",
            "format": "style"
          },
          "Text": {
            "type": "string"
          }
        }
      }
    },
    "Results": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "FirstURL": {
            "type": "string",
            "format": "uri"
          },
          "Icon": {
            "type": "object",
            "properties": {
              "Height": {
                "type": "integer"
              },
              "URL": {
                "type": "string",
                "format": "uri"
              },
              "Width": {
                "type": "integer"
              }
            }
          },
          "Result": {
            "type": "string",
            "format": "style"
          },
          "Text": {
            "type": "string"
          }
        }
      }
    },
    "Type": {
      "type": "string"
    }
  }
}

So I should be creating a APITable class with all the fields above fields right ?

Because a typical response looks like this


{
	"Abstract": "Google LLC is an American multinational technology company focusing on online advertising, search engine technology, cloud computing, computer software, quantum computing, e-commerce, artificial intelligence, and consumer electronics. It has been referred to as \"the most powerful company in the world\" and one of the world's most valuable brands due to its market dominance, data collection, and technological advantages in the area of artificial intelligence. Its parent company Alphabet is considered one of the Big Five American information technology companies, alongside Amazon, Apple, Meta, and Microsoft. Google was founded on September 4, 1998, by computer scientists Larry Page and Sergey Brin while they were PhD students at Stanford University in California. Together they own about 14% of its publicly listed shares and control 56% of its stockholder voting power through super-voting stock. The company went public via an initial public offering in 2004.",
	"AbstractSource": "Wikipedia",
	"AbstractText": "Google LLC is an American multinational technology company focusing on online advertising, search engine technology, cloud computing, computer software, quantum computing, e-commerce, artificial intelligence, and consumer electronics. It has been referred to as \"the most powerful company in the world\" and one of the world's most valuable brands due to its market dominance, data collection, and technological advantages in the area of artificial intelligence. Its parent company Alphabet is considered one of the Big Five American information technology companies, alongside Amazon, Apple, Meta, and Microsoft. Google was founded on September 4, 1998, by computer scientists Larry Page and Sergey Brin while they were PhD students at Stanford University in California. Together they own about 14% of its publicly listed shares and control 56% of its stockholder voting power through super-voting stock. The company went public via an initial public offering in 2004.",
	"AbstractURL": "https://en.wikipedia.org/wiki/Google",
	"Answer": "",
	"AnswerType": "",
	"Definition": "",
	"DefinitionSource": "",
	"DefinitionURL": "",
	"Entity": "company",
	"Heading": "Google",
	"Image": "/i/0227507d.png",
	"ImageHeight": 200,
	"ImageIsLogo": 1,
	"ImageWidth": 608,
	"Infobox": {
		"content": [{
			"data_type": "string",
			"label": "Formerly called",
			"sort_order": "1000",
			"value": "Google Inc. (1998\u20132017)",
			"wiki_order": 0
		}, {
			"data_type": "string",
			"label": "Type",
			"sort_order": "1001",
			"value": "Subsidiary (LLC)",
			"wiki_order": 1
		}],
		"meta": [{
			"data_type": "string",
			"label": "article_title",
			"value": "Google"
		}, {
			"data_type": "string",
			"label": "template_name",
			"value": "infobox company"
		}, {
			"data_type": "string",
			"label": "caption",
			"value": "The Google logo used since 2015"
		}, {
			"data_type": "string",
			"label": "below",
			"value": "Footnotesreferences"
		}, {
			"data_type": "string",
			"label": "formatting_rules",
			"value": "company"
		}]
	},
	"Redirect": "",
	"RelatedTopics": [{
		"FirstURL": "https://duckduckgo.com/c/Google",
		"Icon": {
			"Height": "",
			"URL": "",
			"Width": ""
		},
		"Result": "<a href=\"https://duckduckgo.com/c/Google\">Google Category</a>",
		"Text": "Google Category"
	}, {
		"FirstURL": "https://duckduckgo.com/Outline_of_Google",
		"Icon": {
			"Height": "",
			"URL": "",
			"Width": ""
		},
		"Result": "<a href=\"https://duckduckgo.com/Outline_of_Google\">Outline of Google</a> - The following outline is provided as an overview of and topical guide to Google: Google is an American multinational technology company specializing in internet-related services and products that include online advertising technologies, search, cloud computing, software, and hardware.",
		"Text": "Outline of Google - The following outline is provided as an overview of and topical guide to Google: Google is an American multinational technology company specializing in internet-related services and products that include online advertising technologies, search, cloud computing, software, and hardware."
	}],
	"Results": [],
	"Type": "A",
	"meta": {
		"attribution": null,
		"blockgroup": null,
		"created_date": null,
		"description": "Wikipedia",
		"designer": null,
		"dev_date": null,
		"dev_milestone": "live",
		"developer": [{
			"name": "DDG Team",
			"type": "ddg",
			"url": "http://www.duckduckhack.com"
		}],
		"example_query": "nikola tesla",
		"id": "wikipedia_fathead",
		"is_stackexchange": null,
		"js_callback_name": "wikipedia",
		"live_date": null,
		"maintainer": {
			"github": "duckduckgo"
		},
		"name": "Wikipedia",
		"perl_module": "DDG::Fathead::Wikipedia",
		"producer": null,
		"production_state": "online",
		"repo": "fathead",
		"signal_from": "wikipedia_fathead",
		"src_domain": "en.wikipedia.org",
		"src_id": 1,
		"src_name": "Wikipedia",
		"src_options": {
			"directory": "",
			"is_fanon": 0,
			"is_mediawiki": 1,
			"is_wikipedia": 1,
			"language": "en",
			"min_abstract_length": "20",
			"skip_abstract": 0,
			"skip_abstract_paren": 0,
			"skip_end": "0",
			"skip_icon": 0,
			"skip_image_name": 0,
			"skip_qr": "",
			"source_skip": "",
			"src_info": ""
		},
		"src_url": null,
		"status": "live",
		"tab": "About",
		"topic": ["productivity"],
		"unsafe": 0
	}
}

Some fields are complex fields with nested data do we have to normalize it?
Should we store all the fields. Because to me few fields have valuable information like

Abstract
AbstractURL
Entity
Heading
Infobox

@martyna-mindsdb
Copy link
Collaborator Author

Hi @parthi-siva
I believe you can use just the relevant fields.

@ZoranPandovski Could you please have a look?

@ZoranPandovski
Copy link
Member

As an initial MVP just choose few, then we can increase the support

@parthi-siva
Copy link

Thanks @ZoranPandovski and @martyna-mindsdb

@Sekhar-Kumar-Dash
Copy link
Contributor

Sekhar-Kumar-Dash commented Mar 8, 2024 •
edited

hey @martyna-mindsdb can i get assigned for this issue i have already integrated this with mindsdb

@martyna-mindsdb
Copy link
Collaborator Author

Hey @Sekhar-Kumar-Dash
Please link your PR to this issue.

@Sekhar-Kumar-Dash Sekhar-Kumar-Dash linked a pull request Mar 11, 2024 that will close this issue
Open
11 tasks
@Sekhar-Kumar-Dash
Copy link
Contributor

Sekhar-Kumar-Dash commented Mar 11, 2024 •
edited

Hey @Sekhar-Kumar-Dash Please link your PR to this issue.

hey @martyna-mindsdb i just Linked the PR .Thank You for assigning me

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@Sekhar-Kumar-Dash Sekhar-Kumar-Dash

Labels
app-integration New idea for the supported applications integrations help wanted Extra attention is needed
Projects
Community Contributors
In progress
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Integrate DuckDuckgo Zero click with MindsDB Sekhar-Kumar-Dash/mindsdb
4 participants
@ZoranPandovski @parthi-siva @martyna-mindsdb @Sekhar-Kumar-Dash