Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Memory Leak in query.entriesPaged #5798

Open
4 of 10 tasks
justraman opened this issue Feb 26, 2024 · 0 comments
Open
4 of 10 tasks

Memory Leak in query.entriesPaged #5798

justraman opened this issue Feb 26, 2024 · 0 comments

Comments

@justraman
Copy link

  • I'm submitting a ...
  • Bug report
  • Feature request
  • Support request
  • Other
  • What is the current behavior and expected behavior?

I've encountered an issue related to memory consumption while using the entriesPaged method. As I attempt to retrieve entries from the API in batches within a loop, I've noticed a steady increase in memory usage over time. This gradual buildup eventually leads to high memory consumption, crashes with
FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory

Here is the code which lead to memory leak

   const BATCH_SIZE = 500;
   let lastKey = '';

   while (true) {
   	const query = await api.query.multiTokens.tokens.entriesPaged({
       	args: [],
       	pageSize: BATCH_SIZE,
       	startKey: lastKey,
   	});
   	
   	if (query.length === 0) {
       	break;
   	}
   	for (const [key, value] of query) {
       	lastKey = key.toHex();
   	}
   }

please run it against rpc: wss://enjin-matrix-rpc-1.dwellir.com
or you can also quickly reproduce this by running this on polkadot.js web.

  • Please tell us about your environment:

    • Version: 10.11.2

    • Environment: Node, Browser

      • Node.js
      • Browser
      • Other (limited support for other environments)
    • Language:

      • JavaScript
      • TypeScript (include tsc --version)
      • Other
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Development

No branches or pull requests

2 participants