Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Re-Implement Sites using the official APIs #306

Open
StegSchreck opened this issue Apr 28, 2024 · 1 comment
Open

Re-Implement Sites using the official APIs #306

StegSchreck opened this issue Apr 28, 2024 · 1 comment

Comments

@StegSchreck
Copy link
Owner

Scraping / Inserting data via Selenium was originally done to make the usage of this synchronisation script easier. Especially since some sites didn't offer an API, it was not free to use, or it was lacking data (e.g. for private profiles).

Since some sites have introduced some quite effective anti-bot measures (e.g. captchas on IMDb), this reduces the usefulness of this script/application.

With this issue, re-implement the most-important APIs to replace the Selenium-based scraping&inserting. The sites where a simple CSV upload/download is performed are out-of scope for the first phase of this.

The user might then have to generate private API credentials (e.g. client ID + secret) - this would need to be documented as a step-by-step how-to guide.

For each of the sites, this should be a feature toggle to let the user chose which way to use.

@StegSchreck
Copy link
Owner Author

StegSchreck commented Apr 28, 2024

Official API documentations

Potential 3rd party libraries

to be evaluated for being uptodate, feature-complete, etc. and possible alternatives

No known API available

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
RatS
  
To do
Development

No branches or pull requests

1 participant