Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How can i get all the records using scan ... now i am getting only 780 items .. I need to scan all 50000 items #215

Open
Ashvin09 opened this issue Jun 23, 2017 · 4 comments

Comments

@Ashvin09
Copy link

No description provided.

@Ashvin09 Ashvin09 changed the title How can i get all the records using scan ... now i am getting only 780 items .. I want scan all 50000 items How can i get all the records using scan ... now i am getting only 780 items .. I need to scan all 50000 items Jun 23, 2017
@clarkie
Copy link

clarkie commented Jun 24, 2017

It's useful if you can post code when raising issues but I have a feeling that you may be hitting the scan limits set by AWS:

The result set from a Scan is limited to 1 MB per call. You can use the LastEvaluatedKey from the scan response to retrieve more results.

You can use parallelScan(n) to increase the throughput.

@Ashvin09
Copy link
Author

UserModel.scan()
.where('email').equals(email.toLowerCase()).exec((err, data) => {

@Ashvin09
Copy link
Author

how can i use LastEvaluatedKey ....??? or parallelScan ..??
what is totalSegments in parallelScan and how to get it for records like 1lac

@avtaniket
Copy link

@Ashvin09 Please use Global secondary index on email field, no need of scan operation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants