Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Download issues with db_mgmt_json.py #1058

Open
policchio opened this issue Feb 15, 2024 · 5 comments
Open

Download issues with db_mgmt_json.py #1058

policchio opened this issue Feb 15, 2024 · 5 comments

Comments

@policchio
Copy link

I'm encountering connection issues when running ./sbin/db_mgmt_json.py -p.
I'm using the latest version (5.0.1), which allows setting parameters for the download.
It starts the download, but shortly after errors begin to occur. I've already tried what was mentioned in the two issues that had similar problems (#1039 #1040). I'm reporting what I get when I try with these download parameters (I've tried others as well) DOWNLOAD_SEM_FACTOR=30 and DOWNLOAD_BATCH_RANGE=5:

immagine

immagine

update_populate.log

@B1ank-H
Copy link

B1ank-H commented Feb 15, 2024

I tend to think that the quality of your network is not good enough to support accessing 5 api's at the same time.
You can try lowering DOWNLOAD_BATCH_RANG from 5 to 1 and check again.

@policchio
Copy link
Author

I tried setting DOWNLOAD_BATCH_RANGE=1, but it still doesn't seem to solve the problem (however, my connection is FTTH):

immagine

update_populate.log

@B1ank-H
Copy link

B1ank-H commented Feb 15, 2024

I tried setting DOWNLOAD_BATCH_RANGE=1, but it still doesn't seem to solve the problem (however, my connection is FTTH):

immagine

update_populate.log

It is strange, according to the code logic, when you set DOWNLOAD_BATCH_RANGE=1, it's pretty much the same as when you visit https://services.nvd.nist.gov/rest/json/cves/2.0/?noRejected&startIndex=6000(that number will change) line by line in a browser or somethink like that.
And you update_populate.log did not show you have set DOWNLOAD_BATCH_RANGE=1, it is the same as above, still five visits, can you re-upload it again?

@policchio
Copy link
Author

Yes, sorry, I evidently uploaded the wrong file.
Here is the correct file:

update_populate.log

@P-T-I
Copy link
Member

P-T-I commented Apr 14, 2024

@policchio Could you verify this issue still exists with the latest release of Cvexplore?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants