Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add Keyspace hits metrics for set #2579

Open
wants to merge 5 commits into
base: unstable
Choose a base branch
from

Conversation

chenbt-hz
Copy link
Collaborator

@chenbt-hz chenbt-hz commented Apr 3, 2024

修改内容
1.新增keyspace_hits、keyspace_misses指标
2.修改了set相关命令的返回结果,使得keyspace_hits、keyspace_misses可以正确统计
3.新增了go测试

需要审查的点

  • 当前修改新增了kNoExists是否合理?
  • 当前返回是否符合预期,尤其是异常列表中的情况能否接受?

说明

这次改动主要为了在pika支持keyspace hit相关参数。 关联的issue: #2423
参考redis的实现: redis在执行操作前会统一在内存查找key是否存在并统计
但在pika这一层,需要在各个命令执行阶段实现统计。

  • 本次的改动默认s_ = db_->storage()返回的状态对象能够符合预期的返回s_.IsNotFound()==true。
    即,当key不存在时s_.IsNotFound()==true。
  • 管理命令、有部分的write命令无关key是否存在,因此这些命令将不会纳入keyspace_hit相关的统计。
  • 对于执行报错的命令,不会纳入keyspace_hit相关的统计。
  • 对于read相关命令以及关于某个key的统计命令,应根据实际含义添加keyspace_hit相关统计。

当前只修改set,后续还需要将其他的数据类型命令全部修改。

异常返回统计表

当前版本由于storage层部分s_.IsNotFound()接口返回结果不适配,导致部分命令的返回结果会超出预期,这里是相关列表。


异常描述 命令列表
执行不存在的key命令返回keyspace_hits SInter,SInterstore,SDiffstore
执行存在的key命令返回keyspace_misses  
对存在的key,操作不存在的元素命令返回keyspace_misses Sismember,SMoveCmd
对存在的key,操作存在的元素命令返回keyspace_misses  
操作多个key时,有key存在且有key不存在时,返回keyspace_misses  
操作多个key时,所有key不存在时,返回keyspace_hits Sdiff,SUnionstore,SUnion
   

@chenbt-hz chenbt-hz changed the title Keyspace hits Add Keyspace hits metrics Apr 3, 2024
@chenbt-hz chenbt-hz changed the title Add Keyspace hits metrics feat: Add Keyspace hits metrics Apr 3, 2024
@chenbt-hz
Copy link
Collaborator Author

基本的验证

$redis-cli -p 9221 info stats |grep key
keyspace_hits:19
keyspace_misses:0
is_scaning_keyspace:No

集成测试
image

image

@github-actions github-actions bot added the ✏️ Feature New feature or request label Apr 3, 2024
@AlexStocks AlexStocks changed the title feat: Add Keyspace hits metrics feat: Add Keyspace hits metrics for set Apr 19, 2024
@AlexStocks
Copy link
Collaborator

@chenbt-hz please fix the confliction

chenbt-hz and others added 4 commits June 3, 2024 10:42
add   workflow_dispatch
add workflow_dispatch
2.upgrade Do() functions in pika_set.cc
@chenbt-hz
Copy link
Collaborator Author

@chenbt-hz please fix the confliction

done

@AlexStocks
Copy link
Collaborator

@CodiumAI-Agent /review

@CodiumAI-Agent
Copy link

PR Review 🔍

⏱️ Estimated effort to review [1-5]

4, because the PR involves a significant amount of changes across multiple files, including core functionality and tests. The changes impact the command execution flow and statistics calculation, which are critical areas in the application. Understanding and verifying the correctness of these changes requires a deep understanding of the existing architecture and the intended new functionality.

🧪 Relevant tests

Yes

⚡ Possible issues

Possible Bug: The handling of kNoExists in CmdRes::ToString() might not return a proper error message. The current implementation just returns message_, which might not be informative if not set elsewhere.

Performance Concern: The frequent atomic operations for incrementing keyspace hits and misses might impact performance, especially under high load. This should be benchmarked to ensure that the performance impact is minimal.

🔒 Security concerns

No

Code feedback:
relevant filesrc/pika_command.cc
suggestion      

Consider adding more specific error handling or messages for kNoExists in CmdRes::ToString(). This will improve debugging and logging capabilities. [important]

relevant linereturn message_;

relevant filesrc/pika_set.cc
suggestion      

Ensure consistency in response format for commands when s_.IsNotFound() is true. Currently, some commands append array lengths or other data which might not be expected in an error scenario. [important]

relevant lineres_.AppendArrayLenUint64(members.size());

relevant filesrc/pika_server.cc
suggestion      

Implement a mechanism to limit the rate of increment operations on keyspace hits and misses to mitigate potential performance impacts. This could involve batching increments or only updating the counters every few operations. [medium]

relevant linevoid PikaServer::incr_server_keyspace_hits() { ++(statistic_.server_stat.keyspace_hits); }

relevant filetests/integration/server_test.go
suggestion      

Add more comprehensive tests for edge cases and error conditions in keyspace hit/miss statistics to ensure robustness. [medium]

relevant lineExpect(client.SMembers(ctx, "keyspace_hits").Err()).NotTo(HaveOccurred());

@AlexStocks
Copy link
Collaborator

@CodiumAI-Agent /improve

@CodiumAI-Agent
Copy link

PR Code Suggestions ✨

CategorySuggestion                                                                                                                                    Score
Thread safety
Ensure thread safety by using atomic increment operations

Use atomic increment operations for thread safety when updating keyspace hits and misses.

src/pika_server.cc [957-958]

-void PikaServer::incr_server_keyspace_hits() { ++(statistic_.server_stat.keyspace_hits); }
-void PikaServer::incr_server_keyspace_misses() { ++(statistic_.server_stat.keyspace_misses); }
+void PikaServer::incr_server_keyspace_hits() { statistic_.server_stat.keyspace_hits.fetch_add(1, std::memory_order_relaxed); }
+void PikaServer::incr_server_keyspace_misses() { statistic_.server_stat.keyspace_misses.fetch_add(1, std::memory_order_relaxed); }
 
Suggestion importance[1-10]: 10

Why: This suggestion is crucial for ensuring thread safety when updating keyspace hits and misses, which is essential in a multi-threaded environment.

10
Possible bug
Separate the checks for 'noexist' and 'ok' to ensure accurate keyspace hit/miss tracking

Consider checking the result of res().noexist() and res().ok() separately to avoid
potential logical errors where a command might be marked as a miss incorrectly if
res().ok() returns false.

src/pika_command.cc [916-920]

-if (!IsAdmin() && res().ok()) {
+if (!IsAdmin()) {
   if (res().noexist()) {
     g_pika_server->incr_server_keyspace_misses();
-  } else {
+  } else if (res().ok()) {
     g_pika_server->incr_server_keyspace_hits();
   }
 }
 
Suggestion importance[1-10]: 9

Why: This suggestion addresses a potential logical error by ensuring that keyspace hits are only incremented when the result is explicitly OK, improving the accuracy of the keyspace hit/miss tracking.

9
Best practice
Initialize atomic variables to prevent undefined behavior

Consider initializing the newly added atomic variables keyspace_hits and keyspace_misses
to ensure they start with a defined state. This can prevent any undefined behavior or
incorrect statistics due to uninitialized values.

include/pika_statistic.h [40-41]

-std::atomic<long long> keyspace_hits;
-std::atomic<long long> keyspace_misses;
+std::atomic<long long> keyspace_hits{0};
+std::atomic<long long> keyspace_misses{0};
 
Suggestion importance[1-10]: 9

Why: Initializing atomic variables is a best practice to ensure they start with a defined state, preventing any potential undefined behavior or incorrect statistics due to uninitialized values.

9
Refactor the 'ok' method to strictly check for 'kOk' status

Refactor the ok() method to explicitly check for kOk status to avoid unintended behavior
where kNoExists is treated as a successful status.

include/pika_command.h [342]

-bool ok() const { return ret_ == kOk || ret_ == kNone || ret_ == kNoExists; }
+bool ok() const { return ret_ == kOk; }
 
Suggestion importance[1-10]: 8

Why: This suggestion improves the clarity and correctness of the ok() method by ensuring it only returns true for the kOk status, avoiding unintended behavior.

8
Enhancement
Add a space after the colon for consistent formatting in the output

Ensure consistent formatting by adding a space after the colon in the output for better
readability.

src/pika_admin.cc [972-973]

-tmp_stream << "keyspace_hits:" << g_pika_server->ServerKeyspaceHits() << "\r\n";
-tmp_stream << "keyspace_misses:" << g_pika_server->ServerKeyspaceMisses() << "\r\n";
+tmp_stream << "keyspace_hits: " << g_pika_server->ServerKeyspaceHits() << "\r\n";
+tmp_stream << "keyspace_misses: " << g_pika_server->ServerKeyspaceMisses() << "\r\n";
 
Suggestion importance[1-10]: 5

Why: This suggestion improves the readability of the output by ensuring consistent formatting, but it is a minor enhancement.

5

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
✏️ Feature New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants