Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

KSQL_KSQL_HEARTBEAT_ENABLE settings inquiry #10191

Open
sdYoo opened this issue Jan 17, 2024 · 2 comments
Open

KSQL_KSQL_HEARTBEAT_ENABLE settings inquiry #10191

sdYoo opened this issue Jan 17, 2024 · 2 comments

Comments

@sdYoo
Copy link

sdYoo commented Jan 17, 2024

Hi!

I am running ksqldb in 2 pods.
I enable ksql's "KSQL_KSQL_HEARTBEAT_ENABLE" setting I get excessive logs. Changing the KSQL_KSQL_HEARTBEAT_ENABLE setting to false will reduce the log.
However, I would like to know if there are any issues when changing the setting to false when two devices are in operation.
so ksqldb settings are also shared.

  1. What happens if I disable the "KSQL_KSQL_HEARTBEAT_ENABLE" setting?
  2. Is there a way to reduce the communication log with the "KSQL KSQL HEARTBEAT ENABLE" setting enabled?

**ref. ksqldb deployment yaml
spec:
progressDeadlineSeconds: 600
replicas: 2
revisionHistoryLimit: 10
..............
..............

  • env:
    - name: KSQL_HEAP_OPTS
    value: '-Xms4G -Xmx4G'
    - name: KSQL_LISTENERS
    value: 'http://0.0.0.0:8088'
    - name: KSQL_KSQL_HEARTBEAT_ENABLE
    value: 'true'
    - name: KSQL_KSQL_INTERNAL_TOPIC_MIN_INSYNC_REPLICAS
    value: '2'
    - name: KSQL_KSQL_INTERNAL_TOPIC_REPLICAS
    value: '3'
    - name: KSQL_KSQL_LAG_REPORTING_ENABLE
    value: 'true'
    - name: KSQL_KSQL_QUERY_PULL_ENABLE_STANDBY_READS
    value: 'true'
    - name: KSQL_KSQL_QUERY_PULL_TABLE_SCAN_ENABLE
    value: 'true'
    - name: KSQL_KSQL_SINK_REPLICAS
    value: '3'
    - name: KSQL_KSQL_STREAMS_NUM_STANDBY_REPLICAS
    value: '2'
    - name: KSQL_KSQL_STREAMS_PRODUCER_ACKS
    value: all
    - name: KSQL_KSQL_STREAMS_REPLICATION_FACTOR
    value: '3'
    - name: KSQL_KSQL_STREAMS_TOPIC_MIN_INSYNC_REPLICAS
    value: '2'
@sdYoo
Copy link
Author

sdYoo commented Jan 19, 2024

@ijuma @sullis @codyaray @ewencp
Does anyone know what happens when this "KSQL_KSQL_HEARTBEAT_ENABLE" setting is changed to false?

@juanluhidalgo
Copy link

Hi, if the problem are the logs you can disable just with this https://docs.ksqldb.io/en/latest/reference/server-configuration/#ksqlendpointloggingignoredpathsregex

My recommendation would be to keep the heartbeat enabled and adjust the interval of the heartbeats which is pretty aggressive.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants