eth_getLogs hidden limit on returned data

[molecular]

I'm experiencing some undesirable behaviour in the rpc interface, specifically the eth_getLogs call. There seems to be some kind of hidden limit on how much data it will deliver. Given a block range, depending on wether I query that range in one go or in multiple separate non-overlapping ranges, I get different number of items returned

There's no error thrown afaict. Neither through response nor on stdout of smartbchd.

It's easy to reproduce:

#!/bin/bash
SBCH_ENDPOINT="https://smartbch.fountainhead.cash/mainnet:8545"

result1=`curl -s -X POST ${SBCH_ENDPOINT} -H "Content-Type: application/json" \
-d '{"jsonrpc": "2.0", "method": "eth_getLogs", "params": [{"address": "0x7b2B3C5308ab5b2a1d9a94d20D35CCDf61e05b72", "fromBlock": "0xf1b30", "toBlock": "0x10a1cf", "topics": ["0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef"]}], "id": 1}' | jq '.result | length'`

echo "--- requested blocks 990000 to 1089999, got ${result1} items"

result2a=`curl -s -X POST ${SBCH_ENDPOINT} -H "Content-Type: application/json" \
-d '{"jsonrpc": "2.0", "method": "eth_getLogs", "params": [{"address": "0x7b2B3C5308ab5b2a1d9a94d20D35CCDf61e05b72", "fromBlock": "0xf1b30", "toBlock": "0xfde7f", "topics": ["0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef"]}], "id": 1}' | jq '.result | length'`
echo "--- requested blocks 990000 to 1039999, got ${result2a} items"
result2b=`curl -s -X POST ${SBCH_ENDPOINT} -H "Content-Type: application/json" \
-d '{"jsonrpc": "2.0", "method": "eth_getLogs", "params": [{"address": "0x7b2B3C5308ab5b2a1d9a94d20D35CCDf61e05b72", "fromBlock": "0xfde80", "toBlock": "0x10a1cf", "topics": ["0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef"]}], "id": 1}' | jq '.result | length'`
echo "--- requested blocks 1040000 to 1089999, got ${result2b} items"

echo "$result2a + $result2b = $[ $result2a + $result2b ], should be $result1"

above script will give this output for me:

--- requested blocks 990000 to 1089999, got 6704 items
--- requested blocks 990000 to 1039999, got 6314 items
--- requested blocks 1040000 to 1089999, got 2893 items
6314 + 2893 = 9207, should be 6704

obviously querying block 990000 to 1089999 should give the same number of results as querying 990000 to 1039999 and 1040000 to 1089999 (probably 9207 results)

[zxh0]

OK, got it, we will investigate this problem. Thank you.

[wangkui0508]

Thank you very much, molecular. It is a bug.
There is a limit for the returned event count, and its default value is 10000. In the case provided by you, the count is below this limit. So there is a bug that limits the returned count to 6704.
I have fixed this bug in moeingdb smartbch/moeingdb@c43ac5c .
We still need some time to deploy this bug-fix version to the RPC servers. If this bug bothers you, you can bypass it by shortening the block range.