[Agent] Add K8s event data #6054

TomatoMr · 2024-04-09T07:41:12Z

This PR is for:

Agent
Server

Add K8s event data

Send the k8s event to ingester

Checklist

Added unit test.

Backport to branches

sharang · 2024-04-09T07:49:22Z

server/controller/model/agent_group_config_example.yaml

@@ -727,6 +727,12 @@ vtap_group_id: g-xxxxxx
  ##   FlowAggregator/SessionAggregator.
  #flow-sender-queue-count: 1



上面还有一段要改改，体现出来 event（而且要有控制效果）

## K8s api resources ## Note: Specify kubernetes resources to watch. # The schematics of entries in list is: # { # name: string # group: string # version: string # disabled: bool # } # # Agent will watch the following resources by default: # - namespaces # - nodes # - pods # - replicationcontrollers # - services # - daemonsets # - deployments # - replicasets # - statefulsets # - ingresses # # To disable a resource, add an entry to the list with `disabled: true`: # # kubernetes-resources: # - name: services # disabled: true # # To enable a resource, add an entry of this resource to the list. Be advised that # this setting overrides the default of the same resource. For example, to enable # `statefulsets` in both group `apps` (the default) and `apps.kruise.io` will require # two entries: # # kubernetes-resources: # - name: statefulsets # group: apps # - name: statefulsets # group: apps.kruise.io # version: v1beta1 # # The old `ingress-flavour` setting is deprecated. Watching `routes` in openshift will # use these settings: # # kubernetes-resources: # - name: ingresses # disabled: true # - name: routes # #kubernetes-resources: []

目前是否采集 k8s event 受上面的配置项控制吗？ @TomatoMr

rvql · 2024-04-09T07:56:28Z

agent/src/platform/kubernetes/resource_watcher.rs

@@ -885,6 +948,9 @@ where

 pub trait Trimmable: 'static + Send {
    fn trim(self) -> Self;
+    fn trim_to_event(&self) -> BoxedKubernetesEvent {


这个只有 event 需要，不应该加在这里

rvql · 2024-04-09T07:59:02Z

server/controller/model/agent_group_config.go

@@ -56,6 +56,7 @@ type StaticConfig struct {
 	ToaLruCacheSize                    *int                        `yaml:"toa-lru-cache-size,omitempty"`
 	FlowSenderQueueSize                *int                        `yaml:"flow-sender-queue-size,omitempty"`
 	FlowSenderQueueCount               *int                        `yaml:"flow-sender-queue-count,omitempty"`
+	K8sEventSenderQueueSize            *int                        `yaml:"k8s-event-sender-queue-size,omitempty"`


之前所有的配置都写的 kubernetes

rvql · 2024-04-09T08:05:44Z

agent/src/trident.rs

+                exception_handler.clone(),
+                true,
+            );
+            api_watcher.set_k8s_events_sender(k8s_events_sender);


这部分创建代码考虑放到 api_watcher 创建前，避免 sender 加锁

rvql · 2024-04-09T08:12:02Z

agent/src/platform/kubernetes/resource_watcher.rs

            WatchEvent::Added(object) | WatchEvent::Modified(object) => {
+                if ctx.kind.name == "events" {


用 match guard

rvql · 2024-04-09T08:16:13Z

agent/src/platform/kubernetes/resource_watcher.rs

+        Event::default()
+    }
+
+    fn trim_to_event(&self) -> BoxedKubernetesEvent {


参数为啥不用self？

rvql · 2024-04-09T08:18:05Z

agent/src/platform/kubernetes/resource_watcher.rs

@@ -456,6 +480,7 @@ pub struct WatcherConfig {
 pub struct ResourceWatcher<K> {
    api: Api<K>,
    entries: Arc<Mutex<HashMap<String, Vec<u8>>>>,
+    events: Arc<Mutex<Vec<BoxedKubernetesEvent>>>,


这个字段不应该出现，因为只有 event 类型需要

[Agent] Add K8s event data

57fdb9c

sharang reviewed Apr 9, 2024

View reviewed changes

rvql reviewed Apr 9, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Agent] Add K8s event data #6054

[Agent] Add K8s event data #6054

TomatoMr commented Apr 9, 2024

sharang Apr 9, 2024

sharang Apr 9, 2024

rvql Apr 9, 2024

rvql Apr 9, 2024

rvql Apr 9, 2024

rvql Apr 9, 2024

rvql Apr 9, 2024

rvql Apr 9, 2024

		@@ -727,6 +727,12 @@ vtap_group_id: g-xxxxxx
		## FlowAggregator/SessionAggregator.
		#flow-sender-queue-count: 1

		WatchEvent::Added(object) \| WatchEvent::Modified(object) => {
		if ctx.kind.name == "events" {

[Agent] Add K8s event data #6054

Are you sure you want to change the base?

[Agent] Add K8s event data #6054

Conversation

TomatoMr commented Apr 9, 2024

This PR is for:

Add K8s event data

Checklist

Backport to branches

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment