Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Drop BigQuery and just provide a data dump or paginate API #375

Open
gitgitwhat opened this issue Apr 14, 2023 · 1 comment
Open

Drop BigQuery and just provide a data dump or paginate API #375

gitgitwhat opened this issue Apr 14, 2023 · 1 comment
Labels
enhancement New feature or request

Comments

@gitgitwhat
Copy link

Is your feature request related to a problem? Please describe.
BigQuery is a terrible solution for hosting data from a user perspective.

Describe the solution you'd like
A weekly or even monthly updated data dump would be much more useful. Or implement a pagination mechanism in the API so the full data set can be downloaded efficiently. I recognize it's a lot of data so even just a subset of the relevant details (score, date, hash) would be helpful at this point. Amplifying information could then be queried later if needed.

Describe alternatives you've considered
Single API calls of every library I'm interested in but the latency is too significant for any sufficiently large project. It would be easier to simply have this data in my local database to do queries that way.

Additional context
No

@gitgitwhat gitgitwhat added the enhancement New feature or request label Apr 14, 2023
@naveensrinivasan
Copy link
Member

Thanks for the suggestion on providing paginated API.

@naveensrinivasan naveensrinivasan transferred this issue from ossf/scorecard Apr 20, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants