Regarding the issue of ClickHouse encountering an out-of-memory problem when querying array fields.

[asdas123123-svg]

Hello, I encountered a very bad situation while using ClickHouse and I hope to resolve it.

Specifically:
Initially, the queries were running normally, but last Saturday, there was a sudden spike in memory usage within the same query, causing the data not to be retrieved. Upon investigating the logs, it was found that the memory usage within the group was too large.

It's obvious that this is caused by insufficient memory. However, the data in my table is only 400,000 records, occupying less than 1GB of disk space. Why does querying the array field consume so much memory? And how can I solve this problem?

[UnamedRus]

How big are arrays?

What if set max_block_size = 1000

[davenger]

Yes it's hard to figure out what consumed memory without at least knowing the schema of the table and the sizes of arrays in individual rows, because if one (or several) rows have huge array they will have to be fully read and uncompressed into memory when processing the query.
Also you can notice that progress says: 359.48 thousand rows, 17.11 GB

[asdas123123-svg]

I increased the memory and then performed a length-based sorting query, which I think is because the array length is too long. Is there an appropriate range for the length of this array? Does this mean

record{
   id               Int64 comment 'ID',
   org_log_ids      Array(String) default [],
   datasource       String
  }

When my statement is: "SELECT id, org_log_ids, datasource FROM record LIMIT 10;", does the array field (org_log_ids) undergo aggregation, leading to excessive memory usage?