v2.11.3

Broker

• [fix][broker] Fix typo in the config key #21690
• [fix] [broker] network package lost if enable haProxyProtocolEnabled #21684
• [fix][broker] Fix memory leak during topic compaction #21647
• [fix][broker] Fix incorrect unack count when using shared subscription on non-persistent topic #21592
• [fix][broker] Duplicate LedgerOffloader creation when namespace/topic… #21591
• [improve][broker][PIP-318] Support not retaining null-key message during topic compaction #21578
• [fix][broker] Correct schema deletion for partitioned topic #21574
• [fix][broker] Fix namespace bundle stuck in unloading status (#21445) #21566
• [fix][broker] Fix create topic with different auto creation strategies causes race condition #21545
• [fix] [ml] Fix orphan scheduled task for ledger create timeout check #21542
• [fix] [broker] Fix thousands orphan PersistentTopic caused OOM #21540
• [fix][ml] Fix unfinished callback when deleting managed ledger #21530
• [fix][broker] Fix setReplicatedSubscriptionStatus incorrect behavior #21510
• [fix][txn] OpRequestSend reuse problem cause tbClient commitTxnOnTopic timeout unexpectedly #21505
• [fix][broker] Avoid pass null role in MultiRolesTokenAuthorizationProvider #21486
• [fix][broker] Fix the deadlock when using BookieRackAffinityMapping with rackaware policy #21481
• [fix][broker] Fix issue with consumer read uncommitted messages from compacted topic #21465
• [fix][broker] Fix resource_quota_zpath #21461
• [fix][broker] Fix namespace bundle stuck in unloading status #21445
• [fix][broker] Fix MultiRoles token provider NPE when using anonymous clients #21429
• [fix][ci] Fix docker image building by releasing more disk space before building #21365
• [fix][broker] Fix heartbeat namespace create event topic and cannot delete heartbeat topic #21360
• [fix][broker] Fix heartbeat namespace create transaction internal topic #21348
• [fix] [bk-client] Fix bk client MinNumRacksPerWriteQuorum and EnforceMinNumRacksPerWriteQuorum not work problem. #21327
• [fix] [auto-recovery] [branch-2.11] Fix pulsar ledger auditor dead lock problem. #21319
• [fix] [auto-recovery] [branch-2.11] Fix PulsarLedgerUnderreplicationManager notify problem. #21318
• [fix][auto-recovery] [branch-2.11] Fix metadata store deadlock due to BookkeeperInternalCallbacks.Processor #21315
• [fix][ml] Fix thread safe issue with RangeCache.put and RangeCache.clear #21302
• [fix][txn] Ack all message ids when ack chunk messages with transaction. #21268
• [fix] [ml] fix wrong msg backlog of non-durable cursor after trim ledgers #21250
• [fix] [ml] Reader can set read-pos to a deleted ledger #21248
• [fix][broker]Fixed produce and consume when anonymousUserRole enabled #21237
• [fix][broker] Fix inconsistent topic policy #21231
• [fix][broker] Fixed reset for AggregatedNamespaceStats #21225
• [improve] [broker] Let the producer request success at the first time if the previous one is inactive #21220
• [improve] [proxy] Not close the socket if lookup failed caused by too many requests #21216
• [improve] [broker] Not close the socket if lookup failed caused by bundle unloading or metadata ex #21211
• [fix][broker] replicator leak when removeReplicator in NonPersistentTopic #21205
• [improve] [broker] Print warn log if ssl handshake error & print ledger id when switch ledger #21201
• [fix] [broker] Make the new exclusive consumer instead the inactive one faster #21183
• [fix] [auto-recovery] Fix pulsar ledger auditor dead lock problem. #21181
• [fix][broker] Backport fix UniformLoadShedder selecet wrong overloadbroker and underloadbroker #21178
• [improve] [broker] improve read entry error log for troubleshooting #21169
• [branch-2.11] Fix compatibility issues #21163
• [fix] [auto-recovery] Fix PulsarLedgerUnderreplicationManager notify problem. #21161
• [fix][auto-recovery] Fix metadata store deadlock due to BookkeeperInternalCallbacks.Processor #21159
• [fix] [bookie] Fix RocksDB configuration #21157
• [fix] [broker] Make specified producer could override the previous one #21155
• [improve] [broker] Improve logs for troubleshooting #21141
• [fix][broker] Fix unack count when mixing non batch index and batch index acks #21126
• [fix] [broker] Fix isolated group not work problem. #21096
• [fix][broker] Fix write duplicate entries into the compacted ledger after RawReader reconnects #21081
• [fix] [broker] remove bundle-data in local metadata store. #21078
• [fix][proxy] Fix Proxy 502 gateway error when it is configured with Keystore TLS and admin API is called #21077
• [fix][broker] Make sure all inflight writes have finished before completion of compaction #21067
• [improve][broker] Make read compacted entries support maxReadSizeBytes limitation #21065
• [improve] [broker] Improve cache handling for partitioned topic metadata when doing lookup #21063
• [fix][broker] Use MessageDigest.isEqual when comparing digests #21061
• [fix] [auto-recovery] Improve to the ReplicaitonWorker performance by deleting invalid underreplication nodes #21059
• [fix][broker] Fix potential case cause retention policy not working on topic level #21041
• [fix][fn] Fix ProducerConfig cannot update error #21037
• [fix] [bk] Correctct the bookie info after ZK client is reconnected #21035
• [fix][broker] fix UniformLoadShedder selecet wrong overloadbroker and underloadbroker #21025
• [fix][fn] Fix the --batch-builder not working error for functions #21023
• [fix][meta] Fix deadlock in AutoRecovery. #21010
• [fix][broker] Fix incorrect unack msk count when dup ack a message #20990
• [fix][broker] Fix can't stop phase-two of compaction even though messageId read reaches lastReadId #20988
• [fix][broker] Fix compaction subscription delete by inactive subscription check. #20983
• [fix][broker] Fix message loss during topic compaction #20980
• [fix][broker] Fix incorrect number of read compacted entries #20978
• [fix] [admin] Fix get topic stats fail if a subscription catch up concurrently #20971
• [branch-2.11] [fix] [broker] fix multiple versions of bouncy-castle #20960
• [fix] [broker] Producer is blocked on creation because backlog exceeded on topic, when dedup is enabled and no producer is there #20951
• [fix][broker]Fix chunked messages will be filtered by duplicating #20948
• [fix][broker] rackaware policy is ineffective when delete zk rack info after bkclient initialize #20944
• [fix][broker]Check that the super user role is in the MultiRolesTokenAuthorizationProvider plugin #20939
• [fix][broker] fix MessageDeduplication throw NPE when enable broker dedup and set namespace disable deduplication. #20905
• [fix] [ml] fix discontinuous ledger deletion #20898
• [fix][broker] In replication scenario, remote consumer could not be registered if there has no message was sent #20888
• [improve][txn]Add log when can't send TcClientConnectRequest to broker. #20880
• [fix][broker] Fix inconsensus namespace policies by getPoliciesIfCached #20855
• [fix][broker] Inconsistent behaviour for topic auto_creation #20843
• [improve] [ws] add cryptoKeyReaderFactoryClassName into the file websocket.conf #20840
• [fix][build] Upgrade PyYaml version to 6.0.1 #20831
• [fix][broker] Avoid infinite bundle unloading #20822
• [improve][offload] Extend the offload policies to allow specifying more conf #20804
• [improve][offload] Create offload resources lazily #20775
• [fix] [broker] Can not receive any messages after switch to standby cluster #20767
• [fix][broker] Fix get topic policies as null during clean cache #20763
• [fix][broker] Avoid throwing RestException in BrokerService #20761
• [improve][offload] Support to configure more offload driver #20736
• [fix][broker] Gracefully shutdown does not work with admin cli in standalone #20709
• [fix][broker] Fix ModularLoadManagerImpl always delete active bundle-data. sec ver. #20620
• [improve] [broker] Add consumer-id into the log when doing subscribe. #20568
• [fix][broker] Allow Access to System Topic Metadata for Reader Creation Post-Namespace Deletion #20304
• [improve][broker] Replace ScheduledExecutorService to ExecutorService in ModularLoadManagerImpl #19656
• [fix][broker] Fix namespace deletion if __change_events topic has not been created yet #18804
• [improve] Introduce the sync() API to ensure consistency on reads during critical metadata operation paths #18518
• [fix][broker] Fix namespace not found will cause request timeout #18512
• [fix][broker] fix bug caused by optimistic locking #18390
• [fix][broker] fix broker unackmessages become a negative number #17003

Clients

• [fix][client] Fix print error log 'Auto getting partitions failed' when expend partition. #21485
• [fix] [client] fix reader.hasMessageAvailable return false when incoming queue is not empty #21259
• [improve] [client] Merge lookup requests for the same topic #21232
• [fix] [client] fix same producer/consumer use more than one connection per broker #21144
• [fix][client] Fix repeat consume when using n-ack and batched messages #21116
• [fix][client] Fix logging problem in pulsar client #21094
• [fix][client] Fix subscribing pattern topics through Proxy not working #20739
• [fix][client] Fix producer could send timeout when enable batching #19191
• [fix][client] Fix perf-producer get OOM with high publish latency #20914
• [fix][client] Fix RawReader hasMessageAvailable returns true when no messages #21032
• [fix][client] Fix cannot retry chunk messages and send to DLQ #21048
• [fix][client] Fix consumer can't consume resent chunked messages #21070

Pulsar IO and Pulsar Functions

• [fix] [io] elastic-search sink connector not support JSON.String schema. #20741
• [improve][sql] Fix the wrong format of the logs #20907
• [fix][io] Fix --retain[-key]-ordering not working error for sink #21060
• [fix][io] Update test certs for Elasticsearch #21001
• [fix][io][branch-2.11] Not restart instance when kafka source pol...

v3.1.2

PIPs

• PIP-318: Support not retaining null-key message during topic compaction by @coderzc in #21578

Broker

• Fixed ServiceUnitStateChannel monitor to tombstone only inactive bundle states by @heesung-sn in #21721
• Fix typo in the config key by @coderzc in #21690
• Record GeoPersistentReplicator.msgOut before producer#sendAsync by @nodece in #21673
• Print recoverBucketSnapshot log if cursorProperties are empty by @coderzc in #21651
• Support not retaining null-key message during topic compaction by @coderzc in #21601
• Delete topic timeout due to NPE by @poorbarcode in #21595
• Duplicate LedgerOffloader creation when namespace/topic policies are updated by @shibd in #21591
• Fix issue with consumer read uncommitted messages from compacted topic by @coderzc in #21571
• Fix create topic with different auto creation strategies causes race condition by @mattisonchao in #21545
• Fix orphan scheduled task for ledger create timeout check by @poorbarcode in #21542
• Fix thousands orphan PersistentTopic caused OOM by @poorbarcode in #21540
• Fix unfinished callback when deleting managed ledger by @mattisonchao in #21530
• Fix setReplicatedSubscriptionStatus incorrect behavior by @liudezhi2098 in #21510
• Fix failure while creating non-durable cursor with inactive managed-ledger by @rdhabalia in #21508
• Fix PulsarService/BrokerService shutdown when brokerShutdownTimeoutMs=0 by @lhotari in #21496
• Do not write replicated snapshot marker when the topic which is not enable replication by @liangyepianzhou in #21495
• Avoid pass null role in MultiRolesTokenAuthorizationProvider by @mattisonchao in #21486
• Fix the deadlock when using BookieRackAffinityMapping with rackaware policy by @erobot in #21481
• Fix issue with consumer read uncommitted messages from compacted topic by @coderzc in #21465
• Fix resource_quota_zpath by @AnonHxy in #21461
• Fix namespace bundle stuck in unloading status by @mattisonchao in #21445
• Ignore individual acknowledgment for CompactorSubscription when an entry has been filtered. by @coderzc in #21434
• Fix MultiRoles token provider NPE when using anonymous clients by @Technoboy- in #21429
• Fix heartbeat namespace create event topic and cannot delete heartbeat topic by @TakaHiR07 in #21360
• Fix heartbeat namespace create transaction internal topic by @TakaHiR07 in #21348
• Fix unload operation stuck when use ExtensibleLoadManager by @Demogorgon314 in #21332
• Fix lookup heartbeat and sla namespace bundle when using extensible load manager(#21213) by @Demogorgon314 in #21314
• Fix thread safe issue with RangeCache.put and RangeCache.clear by @lhotari in #21302
• use ConcurrentHashMap in ServiceUnitStateChannel and avoid recursive update error by @heesung-sn in #21282
• Fix inconsistent topic policy by @mattisonchao in #21255
• Let the producer request success at the first time if the previous one is inactive by @poorbarcode in #21220
• Fix lookup heartbeat and sla namespace bundle when using extensible load manager by @Demogorgon314 in #21213
• Avoid record inactiveproducers when deduplication is disable by @lifepuzzlefun in #21193
• Make specified producer could override the previous one by @poorbarcode in #21155
• Rackaware policy is ineffective when delete zk rack info after bkclient initialize by @TakaHiR07 in #20944
• Check that the super user role is in the MultiRolesTokenAuthorizationProvider plugin by @tuteng in #20939

Client

• Fix inconsistent API annotations of getTopicName by @liangyepianzhou in #21620
• Fix print error log 'Auto getting partitions failed' when expend partition by @hanmz in #21485
• Avert extensive time consumption during table view construction by @liangyepianzhou in #21270

Transaction

• OpRequestSend reuse problem cause tbClient commitTxnOnTopic timeout unexpectedly by @TakaHiR07 in #21505
• Ack all message ids when ack chunk messages with transaction by @liangyepianzhou in #21268

CI and Test

• Fix LocalBookkeeperEnsemble resource leak in tests by @lhotari in #21407
• Add new CI unit test group "Broker Group 4" with cluster migration tests by @lhotari in #21391
• Fix resource leaks with Pulsar Functions tests by @lhotari in #21378
• Fix some resource leaks in compaction tests by @lhotari in #21374
• Fix a resource leak in ClusterMigrationTest by @lhotari in #21366
• Fix docker image building by releasing more disk space before building by @lhotari in #21365
• Fix flaky CompactionTest.testDispatcherMaxReadSizeBytes by @lhotari in #21329
• Fix flaky test NarUnpackerTest by @lhotari in #21328
• Fix zookeeper related flacky test by @horizonzy in #21310
• fix flaky test PatternTopicsConsumerImplTest by @poorbarcode in #21222

Security

• Upgrade rabbitmq client to address CVE-2023-46120 by @liangyepianzhou in #21619
• Upgrade Zookeeper to 3.8.3 to address CVE-2023-44981 by @lhotari in #21398
• Upgrade Netty to 4.1.100 to address CVE-2023-44487 by @lhotari in #21397
• Upgrade Jetty to 9.4.53 to address CVE-2023-44487 by @lhotari in #21395
• Fix MultiRoles token provider when using anonymous clients by @merlimat in #21338
• Add OWASP Dependency Check suppressions by @lhotari in #21281
• Upgrade snappy-java to 1.1.10.5 by @lhotari in #21280

Build

• Fix Stage Docker images fail on M1 Mac by @AnonHxy in #21659
• Fix apt download issue in building the docker image by @lhotari in #21489
• Rename schema_example.conf to schema_example.json by @poorbarcode in #21447
• Upgrade Lombok to 1.18.30 to support compiling with JDK21 by @lhotari in #21278

Others

• [offload] Don't cleanup data when offload met MetaStore exception by @zymap in #21686
• [proxy] Move status endpoint out of auth coverage by @mattisonchao in #21428
• [auto-recovery] Migrate the replication testing from BookKeeper to Pulsar by @horizonzy in #21340
• [bk-client] Fix bk client MinNumRacksPerWriteQuorum and EnforceMinNumRacksPerWriteQuorum not work problem by @horizonzy in #21327
• [sql] Support UUID for json and avro by @liangyepianzhou in #21267
• [log] fix the vague response if topic not found by @poorbarcode in #20932

v3.0.2

Broker

• [fix][broker] Fix namespace bundle stuck in unloading status (21445) #21565
• [fix][broker] Fix create topic with different auto creation strategies causes race condition #21545
• [fix][broker] Fix thousands orphan PersistentTopic caused OOM #21540
• [fix][broker] Fix failure while creating non-durable cursor with inactive managed-ledger #21508
• [fix][broker] Fix PulsarService/BrokerService shutdown when brokerShutdownTimeoutMs=0 #21496
• [fix][broker] Avoid pass null role in MultiRolesTokenAuthorizationProvider #21486
• [fix][broker] Fix the deadlock when using BookieRackAffinityMapping with rackaware policy #21481
• [fix][broker] Fix namespace bundle stuck in unloading status #21445
• [fix][broker] Ignore individual acknowledgment for CompactorSubscription when an entry has been filtered. #21434
• [fix][broker] Fix heartbeat namespace create event topic and cannot delete heartbeat topic #21360
• [fix][broker] Fix heartbeat namespace create transaction internal topic #21348
• [fix][broker] Fix unload operation stuck when use ExtensibleLoadManager #21332
• [fix][broker][branch-3.0] Fix lookup heartbeat and sla namespace bundle when using extensible load manager (21213) #21313
• [improve][broker] use ConcurrentHashMap in ServiceUnitStateChannel and avoid recursive update error #21282
• [fix][broker][branch-3.0] Fix inconsistent topic policy #21256
• [fix][broker]Fixed produce and consume when anonymousUserRole enabled #21237
• [fix][broker] Fix inconsistent topic policy #21231
• [fix][broker] Fixed reset for AggregatedNamespaceStats #21225
• [fix][broker] Fix lookup heartbeat and sla namespace bundle when using extensible load manager #21213
• [improve][broker] Not close the socket if lookup failed caused by bundle unloading or metadata ex #21211
• [fix][broker] replicator leak when removeReplicator in NonPersistentTopic #21205
• [improve] [broker] Print warn log if ssl handshake error & print ledger id when switch ledger #21201
• [fix][broker] Backport fix UniformLoadShedder selecet wrong overloadbroker and underloadbroker #21179
• [improve] [broker] improve read entry error log for troubleshooting #21169
• [fix][broker][branch-3.0] Fix write duplicate entries into the compacted ledger after RawReader reconnects (21081) #21165
• [improve][broker][branch-3.0] Make read compacted entries support maxReadSizeBytes limitation (21065) #21164
• [fix][broker] Avoid splitting one batch message into two entries in StrategicTwoPhaseCompactor #21156
• [fix][broker] Make specified producer could override the previous one #21155
• [improve][broker] Upgrade bookkeeper to 4.16.3 #21146
• [improve][broker] Improve logs for troubleshooting #21141
• [fix][broker] Fix web tls url null cause NPE #21137
• [fix][broker] Fix unack count when mixing non batch index and batch index acks #21126
• [fix][broker] Fix unsubscribe non-durable subscription error #21099
• [fix][broker] Fix isolated group not work problem. #21096
• [fix][broker] Fix deleting topic not delete the related topic policy and schema. #21093
• [fix][broker] Avoid splitting one batch message into two entries in StrategicTwoPhaseCompactor #21091
• [fix][broker] revert remove duplicate topics name when deleteNamespace #21087
• [fix][broker] Cleanup correctly heartbeat bundle ownership when handling broker deletion event #21083
• [fix][broker] Fix write duplicate entries into the compacted ledger after RawReader reconnects #21081
• [fix][broker] remove bundle-data in local metadata store. #21078
• [fix][broker] Make sure all inflight writes have finished before completion of compaction #21067
• [improve][broker] Make read compacted entries support maxReadSizeBytes limitation #21065
• [improve][broker] Improve cache handling for partitioned topic metadata when doing lookup #21063
• [fix][broker] Use MessageDigest.isEqual when comparing digests #21061
• [fix][broker] Fix potential case cause retention policy not working on topic level #21041
• [fix][broker] fix UniformLoadShedder selecet wrong overloadbroker and underloadbroker #21025
• [fix][broker] Fix incorrect unack msk count when dup ack a message #20990
• [fix][broker] Fix can't stop phase-two of compaction even though messageId read reaches lastReadId #20988
• [fix][broker] Fix compaction subscription delete by inactive subscription check. #20983
• [fix][broker] Fix message loss during topic compaction #20980
• [fix][broker] Fix incorrect number of read compacted entries #20978
• [fix][broker] Producer is blocked on creation because backlog exceeded on topic, when dedup is enabled and no producer is there #20951
• [fix][broker]Fix chunked messages will be filtered by duplicating #20948
• [fix][broker] rackaware policy is ineffective when delete zk rack info after bkclient initialize #20944
• [fix][broker]Check that the super user role is in the MultiRolesTokenAuthorizationProvider plugin #20939
• [fix][broker] fix MessageDeduplication throw NPE when enable broker dedup and set namespace disable deduplication. #20905
• [fix][broker] In replication scenario, remote consumer could not be registered if there has no message was sent #20888
• [fix][broker] Fix inconsensus namespace policies by getPoliciesIfCached #20855
• [improve][broker] Avoid print redirect exception log when get list from bundle #20846
• [fix][broker] Inconsistent behaviour for topic auto_creation #20843
• [fix][broker] Can not receive any messages after switch to standby cluster #20767
• [fix][broker] Fix get topic policies as null during clean cache #20763
• [fix][broker] Avoid throwing RestException in BrokerService #20761
• [fix][broker] Gracefully shutdown does not work with admin cli in standalone #20709
• [fix][broker] Fix ModularLoadManagerImpl always delete active bundle-data. sec ver. #20620
• [improve][broker] Add consumer-id into the log when doing subscribe. #20568
• [fix][broker] consider iowait as idle. #19110
• [fix][broker] fix bug caused by optimistic locking #18390
• [fix][ml] Fix orphan scheduled task for ledger create timeout check #21542
• [fix][ml] Fix unfinished callback when deleting managed ledger #21530
• [fix][ml] fix discontinuous ledger deletion #20898
• [fix][ml] Fix thread safe issue with RangeCache.put and RangeCache.clear #21302
• [fix][ml] fix wrong msg backlog of non-durable cursor after trim ledgers #21250
• [fix][ml] Reader can set read-pos to a deleted ledger #21248
• [improve][ml] Persist mark deleted ops to Z...

v3.1.1

Broker

• Fix incorrect number of read compacted entries #20978
• Fix message loss during topic compaction #20980
• Fix get outdated compactedTopicContext after compactionHorizon has been updated #20984
• Fix compaction subscription delete by inactive subscription check #20983
• Fix incorrect unack msk count when dup ack a message #20990
• Fix ModularLoadManagerImpl always delete active bundle-data #20620
• Use MessageDigest.isEqual when comparing digests #21061
• Fix can't stop phase-two of compaction even though messageId read reaches lastReadId #20988
• Make sure all inflight writes have finished before completion of compaction #21067
• Fix chunked messages will be filtered by duplicating #20948
• Fix potential case cause retention policy not working on topic level #21041
• Make read compacted entries support maxReadSizeBytes limitation #21065
• Remove bundle-data in local metadata store #21078
• Fix write duplicate entries into the compacted ledger after RawReader reconnects #21081
• Cleanup correctly heartbeat bundle ownership when handling broker deletion event #21083
• Avoid splitting one batch message into two entries in StrategicTwoPhaseCompactor #21091
• Fix unsubscribe non-durable subscription error #21099
• Fix isolated group not work problem #21096
• Consider iowait as idle #19110
• Fix deleting topic not delete the related topic policy and schema #21093
• Revert remove duplicate topics name when deleteNamespace #21087
• Improve logs for troubleshooting #21141
• Fix web tls url null cause NPE #21137
• Fix unack count when mixing non batch index and batch index acks #21126
• Backport fix UniformLoadShedder selecet wrong overloadbroker and underloadbroker #21180
• Fix bug caused by optimistic locking #18390
• Disable balancing based on DirectMemory #21168
• Fix replicator leak when removeReplicator in NonPersistentTopic #21205
• Fixed reset for AggregatedNamespaceStats #21225
• Fixed produce and consume when anonymousUserRole enabled #21237

Pulsar IO and Functions

• [FN] Fix ProducerConfig cannot update error #21037
• [FN] Fix the --batch-builder not working error for functions #21023
• [FN] Fix functions_log4j2.xml delete strategy config #21215
• [IO] Fix --retain[-key]-ordering not working error for sink #21060

Client

• Fix perf-producer get OOM with high publish latency #20914
• Fix RawReader hasMessageAvailable returns true when no messages #21032
• Fix consumer can't consume resent chunked messages #21070
• Fix cannot retry chunk messages and send to DLQ #21048
• Fix logging problem in pulsar client #21094
• Avoid ack hole for chunk message #21101
• Fix repeat consume when using n-ack and batched messages #21116
• Fix same producer/consumer use more than one connection per broker #21144
• Merge lookup requests for the same topic #21232

CLI

• [Admin] Fix get topic stats fail if a subscription catch up concurrently #20971

Others

• [META] Fix deadlock in AutoRecovery #21010
• [META] Improve fault tolerance of blocking calls by supporting timeout #21028
• [META] Improve to the ReplicaitonWorker performance by deleting invalid underreplication nodes #21059
• [META] Fix metadata store deadlock due to BookkeeperInternalCallbacks.Processor #21159
• [META] Fix PulsarLedgerUnderreplicationManager notify problem #21161
• [META] Fix pulsar ledger auditor dead lock problem #21181
• [ML] Persist mark deleted ops to ZK if create cursor ledger was failed #20935
• [BK] Improve getIsolationGroup by avoid creating arrayList #20952
• [BK] Fix RocksDB configuration #21157
• [Proxy] Support disabling metrics endpoint #21031
• [Proxy] Fix Proxy 502 gateway error when it is configured with Keystore TLS and admin API is called #21077
• [TXN] Fix the consumer stuck due to deduplicated messages in pending ack state #21177

Library updates

• Bump GRPC version to 1.55.3 to fix CVE #21057
• Bump broker okio version to 3.4.0 #21064
• Upgrade bookkeeper to 4.16.3 #21146

v3.1.0

Important notice

• Fix the reason label of authentication metrics #20030

PIPs

• PIP-275: Introduce topicOrderedExecutorThreadNum to deprecate numWorkerThreadsForNonPersistentTopic #20504
• PIP-255: Make the partition assignment strategy pluggable #20537
• PIP-278: Support pluggable topic compaction service #20624

Broker

• Fix MessageDeduplication throw NPE when enabling broker deduplication and disabling namespace deduplication #20905
• In replication scenarios, remote consumers could not be registered if there were no message sent #20888
• Fix IndexOutOfBoundsException in the CompactedTopicUtils #20887
• Add annotation for topic compaction strategy #20858
• Fix inconsensus namespace policies by getPoliciesIfCached #20855
• Pass bytesToRead when reading compacted entries #20850
• Fix the built-in admin failed to delete a topic with a custom authz provider #20848
• Avoid print redirect exception log when getting a list from bundle #20846
• Inconsistent behaviour for topic auto_creation #20843
• Follow up 4196 use PulsarByteBufAllocator handle OOM #20837
• Add broker filter sync method back to guarantee the API compatibility #20826
• Fix direct memory leak by delayed index OutOfDirectMemory #20823
• Avoid infinite bundle unloading #20822
• Add the MessageExpirer interface to make code clear #20800
• Broker failed to load v1 namespace resources cache #20783
• Cannot receive any messages after switching to a standby cluster #20767
• Fix get topic policies as null during clean cache #20763
• Fix typo in ServerCnx.java #20762
• Avoid throwing RestException in BrokerService #20761
• Call ServerCnxcloseProducer from correct thread #20747
• Ignore openIDTokenIssuerTrustCertsFilePath conf when blank #20745
• Support pluggable topic compaction service - part2 #20718
• Revert "Skip loading broker interceptor when disableBrokerInterceptors is true 20422" #20710
• Gracefully shutdown does not work with admin cli in standalone #20709
• Improve CompactedTopicImpl lock #20697
• Added the skipped message handler for ServiceUnitStateChannel #20677
• Make ExtensibleLoadManagerImpl's broker filter pure async #20666
• Add logs for topics or subscriptions when enabling rate limit #20660
• Make get list from bundle Admin API async #20652
• Fix the publish latency spike from the contention of MessageDeduplication #20647
• Support pluggable topic compaction service - part1 #20645
• Fix get owned service units NPE #20625
• Topic policy cannot work well if the replay policy message has any exceptions #20613
• Fix the publish latency spike issue with a large number of producers #20607
• AuthenticationFilter.doFilter #20598
• Fix NPE when resetting Replicator's cursor by position. #20597
• Add consumer-id into the log when doing subscribe #20568
• Release orphan replicator after topic closed #20567
• New load balancer system topic should not be auto-created now #20566
• Remove unused cache executor in PulsarService #20563
• Handle get owned namespaces admin API in ExtensibleLoadManager #20552
• Handle heartbeat namespace in ExtensibleLoadManager #20551
• Validate authz earlier in delete subscription logic #20549
• Should not throw NotFoundException when metadata already exists #20539
• REST Client Producer fails with TLS only #20535
• Fix redirect loop when using ExtensibleLoadManager and list in bundle admin API #20528
• Restore solution for certain topic unloading race conditions #20527
• Emit the namespace bundle listener event on extensible load manager #20525
• Choose random thread for consumerFlow in PersistentDispatcherSingleActiveConsumer #20522
• Disable EntryFilters for system topics #20514
• Support get/remove permissions for AuthorizationProvider #20496
• Improve the efficiency of checking message deletion #20490
• Reformat property in generateResponseWithEntry #20481
• Add metric prefix for topic_load_times #20472
• Fix return the earliest position when query position by timestamp. #20457
• Support revoking permission for AuthorizationProvider #20456
• Replace String.intern() with Guava Interner #20432
• Remove ineffective solution for reducing GC pressure #20428
• Skip loading broker interceptor when disableBrokerInterceptors is true #20422
• Avoid PersistentSubscription.expireMessages logic check backlog twice #20416
• Change limitStatsLogging config default value to true #20409
• PIP-255 Part-1: Add listener interface for namespace service #20406
• Use immutable base cursor properties #20400
• Fix partitioned __change_events topic is policy topic #20392
• Do not expose bucketDelayedIndexStats #20383
• Fix flaky test - testClusterMigrationWithReplica… #20379
• Change some static fields referencing mutable objects to ordinary instance fields in BrokerService #20372
• Fix pre-create non-partitioned system topics for load balance extension #20370
• Fix memory leak when rebatchMessage #20369
• Invalidate metadata children cache after key deleted #20363
• Fix managedLedger.getConfig().getProperties().putAll(properties) NPE #20361
• Update rest endpoint method names to avoid swagger conflict #20359
• Fix broker load manager class filter NPE #20350
• Copy BrokerEntryMetadata when rebatchMessage #20337
• In Key_Shared mode: remove unnecessary mechanisms of message skip to avoid unnecessary consumption stuck #20335
• When creating a namespace anti-affinity group with the default local policies, use the bundle size supplied by the user. #20327
• Fix skip message API when hole messages exist #20326
• Gracefully shut down load balancer extension #20315
• Fix class name typo PrecisPublishLimiter to "Precise" #20310
• Allow Access to System Topic Metadata for Reader Creation Post-Namespace Deletion #20304
• Fix NPE cause by topic publish rate limiter. #20302
• Remove Streaming Dispatcher Code #20279
• Fix UnsupportedOperationException when updating topic...

v3.0.1

Important notice

• Fix the reason label of authentication metrics in #20030

Broker

• [Broker] Add annotation for topic compaction strategy by @Demogorgon314 in #20858
• [Broker] Add broker filter sync method back to guarantee the API compatibility by @Demogorgon314 in #20826
• [Broker] Avoid infinite bundle unloading by @Demogorgon314 in #20822
• [Broker] Call ServerCnx#closeProducer from correct thread by @michaeljmarshall in #20747
• [Broker] Added the skipped message handler for ServiceUnitStateChannel by @heesung-sn in #20677
• [Broker] Make ExtensibleLoadManagerImpl's broker filter pure async by @Demogorgon314 in #20666
• [Broker] Fix the publish latency spike from the contention of MessageDeduplication by @codelipenghui in #20647
• [Broker] Fix get owned service units NPE by @Demogorgon314 in #20625
• [Broker] Topic policy can not work well if replay policy message has any exception. by @zhanghaou in #20613
• [Broker] Fix the publish latency spike issue with a large number of producers by @codelipenghui in #20607
• [Broker] New load balancer system topic should not be auto-created now by @heesung-sn in #20566
• [Broker] Handle get owned namespaces admin API in ExtensibleLoadManager by @Demogorgon314 in #20552
• [Broker] Handle heartbeat namespace in ExtensibleLoadManager by @Demogorgon314 in #20551
• [Broker] Validate authz earlier in delete subscription logic by @michaeljmarshall in #20549
• [Broker] REST Client Producer fails with TLS only by @maanders-tibco in #20535
• [Broker] Fix redirect loop when using ExtensibleLoadManager and list in bundle admin API by @Demogorgon314 in #20528
• [Broker] Restore solution for certain topic unloading race conditions by @lhotari in #20527
• [Broker] Emit the namespace bundle listener event on extensible load manager by @Demogorgon314 in #20525
• [Broker] Disable EntryFilters for system topics by @lhotari in #20514
• [Broker] Replace test call to Auth0 with call to WireMock by @michaeljmarshall in #20465
• [Broker] Fix return the earliest position when query position by timestamp. by @hanmz in #20457
• [Broker] Avoid PersistentSubscription.expireMessages logic check backlog twice. by @lifepuzzlefun in #20416
• [Broker] PIP-255 Part-1: Add listener interface for namespace service by @hleecs in #20406
• [Broker] Partitioned __change_events topic is policy topic by @michaeljmarshall in #20392
• [Broker] Do not expose bucketDelayedIndexStats by @coderzc in #20383
• [Broker] Pre-create non-partitioned system topics for load balance extension by @heesung-sn in #20370
• [Broker] ManagedLedger.getConfig().getProperties().putAll(properties) NPE by @heesung-sn in #20361
• [Broker] Fix broker load manager class filter NPE by @Demogorgon314 in #20350
• [Broker] In Key_Shared mode: remove unnecessary mechanisms of message skip to avoid unnecessary consumption stuck by @poorbarcode in #20335
• [Broker] When creating a namespace anti-affinity group with the default local policies, use the bundle size supplied by the user. by @syk-coder in #20327
• [Broker] Fix skip message API when hole messages exist by @crossoverJie in #20326
• [Broker] Gracefully shut download balancer extension by @heesung-sn in #20315
• [Broker] Fix class name typo PrecisPublishLimiter to "Precise" by @JooHyukKim in #20310
• [Broker] Allow Access to System Topic Metadata for Reader Creation Post-Namespace Deletion by @liangyepianzhou in #20304
• [Broker] Fix NPE caused by topic publish rate limiter. by @dragonls in #20302
• [Broker] Fix UnsupportedOperationException when updating topic properties. by @liangyepianzhou in #20261
• [Broker] Fix ledger cachemiss size metric by @AnonHxy in #20257
• [Broker] Fix default bundle size used while setting bookie affinity by @syk-coder in #20250
• [Broker] Fix the behavior of delayed message in Key_Shared mode by @codelipenghui in #20233
• [Broker] Fix Return value of getPartitionedStats doesn't contain subscription type by @1Jack2 in #20210
• [Broker] Skip split bundle if only one broker by @poorbarcode in #20190
• [Broker] Fix RoaringBitmap.contains can't check value 65535 by @coderzc in #20176
• [Broker] Release EntryBuffer after parsing proto object by @coderzc in #20170
• [Broker] Remove useless field in the DelayedMessageIndexBucketSegment.proto by @coderzc in #20166
• [Broker] Optimization protobuf code in the bucket delayed tracker by @coderzc in #20158
• [Broker] Move bitmap from lastMutableBucket to ImmutableBucket by @coderzc in #20156
• [Broker] Fix tenant admin authorization bug. (#20068)" by @michaeljmarshall in #20143
• [Broker] Producer/Consumer should call allowTopicOperationAsync by @michaeljmarshall in #20142
• [Broker] Fix entry filter feature for the non-persistent topic by @gaoran10 in #20141
• [Broker] Optimize delayed metadata index bitmap by @mattisonchao in #20136
• [Broker] Fast fix infinite HTTP call getSubscriptions caused by wrong topicName by @poorbarcode in #20131
• [Broker] Make timer execute immediately after load index by @coderzc in #20126
• [Broker] Cache LedgerHandle in BookkeeperBucketSnapshotStorage by @coderzc in #20117
• [Broker] Close temporary open ledger in BookkeeperBucketSnapshotStorage by @lifepuzzlefun in #20111
• [Broker] Fix use error TimeUnit to record publish latency by @TakaHiR07 in #20074
• [Broker] Fix tenant admin authorization bug. by @dragonls in #20068
• [Broker] Fix getPartitionedStats miss subscription's messageAckRate by @wangjialing218 in #19870
• [Broker] Do not filter system topic while shedding. by @thetumbled in #18949
• [Broker] There are two same-named managed ledgers in the one broker by @poorbarcode in #18688
• [Broker] If ledger lost, cursor mark delete position can not forward by @poorbarcode in #18620
• [Broker] Support cgroup v2 by using jdk.internal.platform.Metrics in Pulsar Loadbalancer by @coderzc in #16832

Clients

• [Client] Fix the deadlock issue of consumers while using multiple IO threads by @codelipenghui in #20669
• [Client] Make the whole grabCnx() progress atomic by @BewareMyPower in #20595
• [Client] Thread-safe seek by @tisonkun in #20321
• [Client] Fix where the function getMsgNumInReceiverQueue always returns 0 when using message listener by @crossoverJie in #20245
• [Client] Java Client's Seek Logic Not Threadsafe #1 by @JooHyukKim in #20242

Pulsar IO and Pulsar Functions

• [Functions] Exit JVM when the main thread throws an exception by @michaeljmarshall in #20689
• [Functions] TLS args admin download command use zero arity by @michaeljmarshall in #20513
• [Functions] Support customizing TLS config for function download command by @michaeljmarshall in [#20482](https://github.com/apache/pulsa...

v2.10.5

Broker

• [fix][broker] In replication scenario, remote consumer could not be registered if there has no message was sent (#20888)
• [improve] [broker] Print warn log if compaction failure (#19405)
• [branch-2.10][fix][broker] Fix inconsensus namespace policies by getPoliciesIfCached (#20873)
• [branch-2.10][fix][broker] Inconsistent behaviour for topic auto_creation (#20872)
• [fix] [broker] Can not receive any messages after switching to standby cluster (#20767)
• [improve] [broker] Add consumer-id into the log when doing subscribe. (#20568)
• [fix][broker][branch-2.10] Fix NPE when resetting Replicator's cursor by position. (#20597) (#20781)
• [fix][broker] Fix namespace deletion if __change_events topic has not been created yet (#18804)
• [fix][schema] Only handle exception when there has (#20730)
• [fix][broker] Topic policy can not work well if replay policy message has any exception. (#20613)
• [fix][broker] Fix return the earliest position when query position by timestamp. (#20457)
• [fix][broker] Return if AbstractDispatcherSingleActiveConsumer closed (#19934)
• [broker] clean inactive bundle from bundleData in loadData and bundlesCache (#13974)
• [fix][branch-2.10] Fix compilation issue introduced by fixing Repeated messages of shared dispatcher (#16812)
• Issue 16802: fix Repeated messages of shared dispatcher (#16812)
• [fix][branch-2.10]Fix compilation issue introduced by Save createIfMissing in TopicLoadingContext (#19993)
• [improve][broker] Save createIfMissing in TopicLoadingContext (#19993)
• [fix][broker] Invalidate metadata children cache after key deleted (#20363)
• [improve] [broker] Avoid PersistentSubscription.expireMessages logic check backlog twice. (#20416)
• [fix][broker]fix the publish latency spike issue with large number of producers (#20607)
• [fix][branch-2.10] Fix duplicated deleting topics (#20685)
• [improve][broker][branch-2.10] Backport Linux metrics changes from master branch (#20659)
• [cleanup][broker] Validate authz earlier in delete subscription logic (#20549)
• [fix][broker] release orphan replicator after topic closed (#20567)
• [fix][broker] REST Client Producer fails with TLS only (#20535)
• [fix][broker] Restore solution for certain topic unloading race conditions (#20527)
• [fix][ml] There are two same-named managed ledgers in the one broker (#18688)
• [fix][broker] Fix skip message API when hole messages exists (#20326)
• [fix][broker] If ledger lost, cursor mark delete position can not forward (#18620)
• [fix][broker] partitioned __change_events topic is policy topic (#20392)
• [fix][ml] Fix ledger left in OPEN state when enabling inactiveLedgerRollOverTimeMs (#20276)
• [fix][broker] Fix default bundle size used while setting bookie affinity (#20250)
• [fix][broker] Fix the behavior of delayed message in Key_Shared mode (#20233)
• [improve][broker] Get lowest PositionImpl from NavigableSet (#18278)
• [fix] [broker] error TimeUnit to record publish latency (#20074)
• [fix] [broker] In Key_Shared mode: remove unnecessary mechanisms of message skip to avoid unnecessary consumption stuck (#20335)
• [fix][broker]Fix deadlock of metadata store (#20189)
• [fix] [broker] [branch-2.10] Upgrade rocksDB version to 6.16.4 to keep sync with BookKeeper 4.14.7 (#20312)
• [improve] [broker] Skip split bundle if only one broker (#20190)
• [fix][monitor] topic with double quote breaks the prometheus format (#20230)
• [fix][broker] Fix RoaringBitmap.contains can't check value 65535 (#20176)
• [fix][broker] Fix the reason label of authentication metrics (#20030)
• [fix] [broker] Fix infinite ack of Replicator after topic is closed (#20232)
• [fix] [broker] Producer created by replicator is not displayed in topic stats (#20229)
• [fix] [broker] delete topic failed if disabled system topic (#19735)
• [fix] [ml] make the result of delete cursor is success if cursor is deleted (#19825)
• [fix] [broker] Fast fix infinite HTTP call getSubscriptions caused by wrong topicName (#20131)
• [fix][broker] Fix issue where msgRateExpired may not refresh forever (#19759)
• [fix][broker] Fix can't send ErrorCommand when message is null value (#19899)
• [fix][broker] Fix Return value of getPartitionedStats doesn't contain subscription type (#20210)
• [branch-2.10][fix][build] Upgrade swagger version to fix CVE-2022-1471 (#20172)
• [fix][broker] Fix getPartitionedStats miss subscription's messageAckRate (#19870)
• [fix] Use scheduled executor in BinaryProtoLookupService (#20043)
• [improve] [broker] Fix broker restart logic (#20113)

Transaction

• [fix] [txn] fix consumer can receive aborted txn message when readType is replay (#19815)

Pulsar IO and Pulsar Functions

• [fix][io][branch-2.10] Not restart instance when kafka source poll exception. (#20816)
• [fix][offload] Filesystem offloader class not found hadoop-hdfs-client (#20365)
• [fix][fn] Make KubernetesRuntime translate characters in function tenant, namespace, and name during function removal to avoid label errors (#19584)
• [fix][fn] Fix JavaInstanceStarter inferring type class name error (#19896)
• [fix][io] Close the kafka source connector got stuck (#20698)
• [fix][fn] Exit JVM when main thread throws exception (#20689)
• Optimize conusmer pause (#14566)
• [fix][broker] Fix the publish latency spike from the contention of MessageDeduplication (#20647)
• [fix][fn] Configure pulsar admin for TLS (#20533)
• [fix][fn] Go functions must retrieve consumers by non-particioned topic ID (#20413)
• [fix][io] Close the kafka source connector if there is uncaught exception (#20479)
• [fix][fn] Go functions need to use static grpcPort in k8s runtime (#20404)
• [fix][fn] Make pulsar-admin support update py/go with package url (#19897)
• [fix][txn] Fix transaction is not aborted when send or ACK failed (#20240)

CLI

• [fix] [cli] the variable producerName of BatchMsgContainer is null (#20819)
• [fix][ci] Update nar maven plugin version to fix excessive downloads (#20410)

Admin

• [improve][admin] Return BAD_REQUEST on cluster data is null for createCluster (#20346)

Security

• [fix][sec] Upgrade snappy-java to address multiple CVEs (#20604)
• [fix][sec] Upgrade Guava to 32.0.0 to address CVE-2023-2976 (#20459)
• [improve][misc] Upgrade Netty to 4.1.93.Final (#20423)
• [improve][misc] Upgrade Netty to 4.1.89.Final (#19649)
• [fix][sec] Upgrade sqlite-jdbc to resolve CVE-2023-32697 (#20411)

CI & Test

• [fix][test] Replace test call to Auth0 with call to WireMock (#20465)
• [fix][test] Fix flaky testCreateTopicWithZombieReplicatorCursor (#20037)
  *[fix][flaky-test]NamespaceServiceTest.flaky/testModularLoadManagerRemoveBundleAndLoad (#17487)
• [improve][test][branch-2.10] Backport disabling disk usage threshold for Elastic Testcontainers (#20676)

Others

• [fix][build]Fix compatibility issue caused by #20819 (#20834)
• [fix][build] Mongo is fixed for 2.10.5 (#20810)
• [fix][meta] Bookie Info lost by notification race condition. (#20642)
• [fix] [meta]Switch to the metadata store thread after zk operation (#20303)
• [fix][ws] Remove unnecessary ping/pong implementation (#20733)
• fix: bundle-data metadata leak because of bundlestats was not clean (#17095)
• [fix] [Perf] PerformanceProducer do not produce expected number of messages. (#19775)
• [fix][meta] Adding the missed bookie id in the registration manager. (#20641)
• [fix][build] Don't publish docker image with "latest" tag to docker repository
• [fix][build][branch-2.10] Fix ci-license check (#20505)
• [improve][monitor] Add JVM start time metric (#20381)
• [fix][build] update the zookeeper version to 3.6.4 (#20333)
• [fix][monitor] Fix the partitioned publisher topic stat aggregation bug (#18807)
• [fix][meta] deadlock of zkSessionWatcher when zkConnection loss (#20122)
• [improve][build] Upgrade dependencies to reduce CVE. (#20162)
• [branch-2.10][improve][build] Upgrade snakeyaml version to 2.0 (#20118)

Client

• [fix] [client] Messages lost when consumer reconnect (#20695)
• [fix][client] Make the whole grabCnx() progress atomic (#20595)
• [fix][client] Fix race condition that leads to caching failed CompletableFutures in ConnectionPool (#19661)
• [fix][client] Fix deadlock issue of consumer while using multiple IO threads (#20669)
• [fix][client] Cache empty schema version in ProducerImpl schemaCache. (#19929)
• [fix][fn] Reset idle timer correctly (#20450)
• [fix][client] Release the orphan producers after the primary consumer is closed (#19858)
• [fix][client] Fix DeadLetterProducer creation callback blocking client io thread. (#19930)

v2.11.2

Broker

• Fix broker restart logic #20113
• Fast fix infinite HTTP call getSubscriptions caused by wrong topicName #20131
• Fix getPartitionedStats miss subscription's messageAckRate #19870
• Fix deadlock of zkSessionWatcher when zkConnection loss #20122
• Fix Return value of getPartitionedStats doesn't contain subscription type #20210
• Producer created by replicator is not displayed in topic stats #20229
• Fix infinite ack of Replicator after topic is closed #20232
• Skip split bundle if only one broker #20190
• Fix RoaringBitmap.contains can't check value 65535 #20176
• Get the lowest PositionImpl from NavigableSet #18278
• Fix the behavior of delayed message in Key_Shared mode #20233
• Fix use error TimeUnit to record publish latency #20074
• Fix the thread safety issue of BrokerData#getTimeAverageData access #19889
• Fix can't send ErrorCommand when a message's value is null #19899
• Make LedgerOffloaderFactory can load the old nar #19913
• Ignore and remove the replicator cursor when the remote cluster is absent #19972
• Fix the reason label of authentication metrics #20030
• Optimize the ack/send future in TransactionImpl #20271
• Fix NPE caused by topic publish rate limiter #20302
• Fix default bundle size used while setting bookie affinity #20250
• Fix ledger left in OPEN state when inactiveLedgerRollOverTimeMs is enabled #20276
• Fix deadlock of metadata store #20189
• Fix class name typo PrecisPublishLimiter to "Precise" #20310
• Fix consume stuck of shared streaming dispatcher #18315
• Fix entry filter feature for the non-persistent topic #20141
• Fix partitioned __change_events topic is policy topic #20392
• If a ledger is lost, the cursor mark delete position cannot forward #18620
• Invalidate metadata children cache after key deleted #20363
• Avoid PersistentSubscription.expireMessages logic check backlog twice #20416
• Switch to the metadata store thread after zk operation #20303
• Fix skip message API when hole messages exist #20326
• Do not filter system topic while shedding #18949
• Restore solution for certain topic unloading race conditions #20527
• There are two same-named managed ledgers in the one broker #18688
• REST Client Producer fails with TLS only #20535
• Save createIfMissing in TopicLoadingContext #19993
• Unwrap the completion exception #20396
• Release orphan replicator after topic closed #20582
• Fix the publish latency spike issue with a large number of producers #20607
• Support cgroup v2 by using jdk.internal.platform.Metrics in Pulsar Loadbalancer #16832
• Validate authz earlier in delete subscription logic #20549
• Fix the publish latency spike from the contention of MessageDeduplication #20647
• Adding the missed bookie id in the registration manager #20641
• Fix NPE when resetting Replicator's cursor by position #20597
• Topic policy can not be work well if replay policy message has any exception #20613
• Bookie Info lost by notification race condition #20642
• Update new bundle-range to policies after bundle split #17797
• Fix consumer can receive aborted txn message when readType is replay #19815
• Fix return the earliest position when query position by timestamp #20457

Clients

• [Java] Use scheduled executor in BinaryProtoLookupService #20043
• [Java] Release the orphan producers after the primary consumer is closed #19858
• [Java] Fix NPE when acknowledging multiple messages #19874
• [Java] Cache empty schema version in ProducerImpl schemaCache #19929
• [Java] Fix where the function getMsgNumInReceiverQueue always returns 0 when using message listener #20245
• [Java] Fix deadlock issue of consumer while using multiple IO threads #20669
• [Java] Make the whole grabCnx() progress atomic #20595
• [Java] Messages lost when consumer reconnect #20695

Pulsar IO and Pulsar Functions

• [Functions] Use functions classloader in TopicSchema.newSchemaInstance() to fix ClassNotFoundException when using custom SerDe classes #20115
• [Functions] Make pulsar-admin support update py/go with package url #19897
• [Functions] Fix JavaInstanceStarter inferring type class name error #20426
• [Functions] Go functions must retrieve consumers by non-partitioned topic ID #20413
• [Functions] Fix function update error #19895
• [Functions] Reset idle timer correctly #20450 #20450
• [Functions] Go functions need to use static grpcPort in k8s runtime #20404
• [Functions] Configure pulsar admin for TLS #20533
• [Functions] Make KubernetesRuntime translate characters in function tenant, namespace, and name during function removal to avoid label errors #19584
• [Functions] Exit JVM when the main thread throws an exception #20689
• [IO Connector] Close the Kafka source connector if there is uncaught exception #20480
• [IO Connector] Close the Kafka source connector got stuck #20698

Observability

• [Broker] Fix the partitioned publisher topic stat aggregation bug #18807
• [Broker] Fix topic with double quote breaks the Prometheus format #20230
• [Broker] Add JVM start time metric #20381
• [Offload] Fix offload metrics error #20366

CLI

• [CLI] Allow pulser-client consumer to create a replicated subscription #20316
• [CLI] Update nar maven plugin version to fix excessive downloads #20410
• [CLI] Speed up OWASP dependency check in Pulsar CI workflow #20412
• [CLI] Fulfill add-opens to function-localrunner also #20417
• [Admin] Return BAD_REQUEST on cluster data is null for createCluster #20346
• [Admin] Report earliest msg in partitioned backlog #19465

Others

• [Build] Duplicate entry when merging services #17659
• [Offload] Use filesystem offloader with class not found error #20365
• [Perf] PerformanceProducer does not produce expected number of messages #19775
• [SQL] Remove useless configuration for Pulsar SQL #20605
• [WS] Remove unnecessary ping/pong implementation #20605
• [Schema] Only handle exception when there has #20730

Library updates

• Upgra...

v3.0.0

Important notice

• Upgrade the RocksDB version to 7.9.2 to keep sync with BookKeeper's RocksDB dependency in #20072
• Introduce consistent hash ring to distribute the load of multiple topics subscription in #19502
• Remove default 30s ackTimeout when setting DLQ policy on java consumer in #19486
• Require authRole is proxyRole to set originalPrincipal in #19455
• Update Elasticsearch sink idle cnx timeout to 30s in #19377
• Move checked exception into builder when newTransaction. in #19356
• TokenAuthenticationState: authenticate token only once in #19314
• OneStageAuth State: move authn out of constructor in #19295
• Update AuthenticationProvider to simplify HTTP Authn in #19197
• Remove AuthorizationProvider methods deprecated in 2.7 and 2.9 in #19182
• Deprecate blocking AuthorizationService, AuthorizationProvider methods in #19180
• Reject create non existent persistent partitions. in #19086
• Omit making a copy of CommandAck when there are no broker interceptors in #18997
• Use correct file path for tls trust certificates in #18712
• Enable certificate refresh for Quorum and Netty Servers in #18097
• Remove timestamp from Prometheus metrics in #17419
• Revert 5895: fix redeliveryCount in #17060
• Fix producer/consume permission can’t get v1/schema in #16018
• Autorecovery default reppDnsResolverClass to ZkBookieRackAffinityMapping in #15640
• Allow to build and push multi-arch Docker images in #19432

PIPs

• PIP-160 Metrics stats of Transaction buffered writer #15370
• PIP-167 Make it Configurable to Require Subscription Permission #15576
• PIP-169 Support split bundle by flow or qps #16782
• PIP-188: Cluster migration or Blue-Green cluster deployment support in Pulsar #16551
• PIP-192: New Pulsar Broker Load Balancer #16691
• PIP-214: Add broker level metrics statistics and expose to prometheus #18056
• PIP-218: Consumer batchReceive() single partition every receive #18182
• PIP-229: Add a common interface to get fields of the MessageIdData #18950(#18950)
• PIP-250: Add proxyVersion to CommandConnect #19623(#19623)
• PIP-257: Add Open ID Connect Support to Server Components #19771

Broker

• [Broker] incomplete PIP-97: Asynchronous Authentication Provider #12105
  • Implement for ServerCnx by @michaeljmarshall in #19409
  • Implement for ProxyConnection by @michaeljmarshall in #19292
• [Broker] incomplete PIP-180: Shadow Topic, an alternative way to support readonly topic ownership. #16153
  • Add ShadowManagedLedgerImpl by @Jason918 in #18265
  • Support shadow topic creation. by @Jason918 in #17711
  • Add Shadow Replicator by @Jason918 in #17371
• [Broker] PIP-192: New Pulsar Broker Load Balancer #16691
  • Fix getLastMessageId for compressed payload(And add compression and maxBatchSize for the load balance system topic) by @heesung-sn in #20087
  • Improve TransferSheder for overload outlier for large clusters by @heesung-sn in #20059
  • Update the lookup data path to support deployment and rollback by @Demogorgon314 in #19999
  • Filter the transfer dest broker by @Demogorgon314 in #19958
  • Updated metrics and cleanup broker selector by @heesung-sn in #19945
  • Support delete namespace bundle admin API by @Demogorgon314 in #19851
  • Improved Auto Unload Logic by @heesung-sn in #19813
  • Support broker isolation policy by @Demogorgon314 in #19592
  • Add large topic count filter by @Demogorgon314 in #19613
  • Added VersionId in ServiceUnitStateData by @heesung-sn in #19620
  • Added Deleted and Init states in ServiceUnitState by @heesung-sn in #19546
  • Added broker and top-bundles load reporters by @heesung-sn in #19471
  • Added TransferShedder by @heesung-sn in #18865
  • Added ServiceUnitStateCompactionStrategy by @heesung-sn in #19045
  • Added TopicCompactionStrategy for StrategicTwoPhaseCompactor and TableView. by @heesung-sn in #18195
  • Implement load data store by @Demogorgon314 in #18777
  • Implement broker registry for new load manager by @Demogorgon314 in #18810
  • Added ServiceUnitStateChannelImpl by @heesung-sn in #18489
• [Broker] Pip-257: Add AuthenticationProviderOpenID by @michaeljmarshall in #19849
• [Broker] Make LeastResourceUsageWithWeight thread safe by @lifepuzzlefun in #20159
• [Broker] Support disabling delayed bucket merging. by @mattisonchao in #20155
• [Broker] Supply download auth params when provided for k8s runtime by @michaeljmarshall in #20144
• [Broker] AuthenticationProviderOpenID k8s error logs by @michaeljmarshall in #20135
• [Broker] Implement authenticateAsync for AuthenticationProviderList by @michaeljmarshall in #20132
• [Broker] Fix broker restart logic by @wolfstudy in #20113
• [Broker] Ensure previous delayed index be removed from snapshotSegmentLastIndexTable & Make load operate asynchronous by @coderzc in #20086
• [Broker] Fix avoid future of clear delayed message can't complete by @coderzc in #20075
• [Broker] Refresh auth data if ProxyLookupRequests by @michaeljmarshall in #20067
• [Broker] Fix flaky testCreateTopicWithZombieReplicatorCursor by @BewareMyPower in #20037
• [Broker] Prevent range conflicts with Key Shared sticky consumers when TCP/IP connections get orphaned by @lhotari in #20026
• [Broker] Fix leader broker log by @nicoloboschi in #19987
• [Broker] Make LedgerOffloaderFactory can load the old nar. by @horizonzy in #19913
• [Broker] Ignore and remove the replicator cursor when the remote cluster is absent by @BewareMyPower in #19972
• [Broker] Fix DeadLetterProducer creation callback blocking client io thread. by @lifepuzzlefun in #19930
• [Broker] Merge multiple buckets at once by @coderzc in #19927
• [Broker] Fix the loss of bundle stats data reported to zookeeper, when the updateStats method is executed by @lordcheng10 in #19887
• [Broker] Clear delayed message when unsubscribe & Make clear operation asynchronous by @coderzc in #19901
• [Broker] Fix can't send ErrorCommand when message is null value by @coderzc in #19899
• [Broker] PIP-240: new public method unloadSubscription in PersistentTopic by @poorbarcode in #19737
• [Broker] PIP-250: Add proxyVersion to CommandConnect by @michaeljmarshall in #19618
• [Broker] Make bucket merge operation asynchronous by @coderzc in #19873
• [Broker] Suppress err...

v2.9.5

What's Changed

• [fix] [tx] [branch-2.9] Transaction buffer recover blocked by readNext by @poorbarcode in #18970
• [branch-2.9] Fix compile error caused by cherry-pick #19167. by @mattisonchao in #19241
• [fix][broker] Support deleting partitioned topics with the keyword -partition- (#19230) by @mattisonchao in #19240
• [improve][websocket][branch-2.9] Add ping support by @nodece in #19245
• [fix][websocket][branch-2.9] Fix webSocketPingDurationSeconds config by @nodece in #19293
• [branch-2.9][cherry-pick] Close TransactionBuffer when MessageDeduplication#checkStatus failed by @tjiuming in #19287
• [fix][Branch-2.9] Fix bookie JVM param BOOKIE_MEM, BOOKIE_GC miss problem. by @horizonzy in #19441
• [fix][broker][branch-2.9] Fix geo-replication admin by @nodece in #19614
• [Branch-2.9] [Build] Fix branch-2.9 build failed by @hangc0276 in #19645
• [Branch-2.9] [build] fix build failed by @hangc0276 in #19669
• [cherry-pick][branch-2.9] Add UncaughtExceptionHandler for every thread pool (#18211) by @coderzc in #19677
• [fix][branch-2.9] Support zookeeper read-only config by @horizonzy in #19693
• [cherry-pick][branch-2.9] Fix validatePersistencePolicies that Namespace/Topic persistent policies cannot set to < 0 by @dao-jun in #19687
• [cherry-pick][branch-2.9] fix txn coordinator recover handle committing and aborting txn race condition. #19201 by @thetumbled in #19699
• [cherry-pick][branch-2.9] Fix race condition while updating partition number (#19199) by @coderzc in #19691
• [improve][branch-2.9]upgrade the bookkeeper version to 4.14.7 by @horizonzy in #19178
• [fix][cli][branch-2.9] Fix release tool by @liangyepianzhou in #19712
• [Branch-2.9] [build] fix build failed by @coderzc in #19743
• [cherry-pick][branch-2.9] Upgrade Netty to 4.1.87.Final (#19417) by @coderzc in #19718
• [fix][license][brach-2.9] Fix license by @coderzc in #19935
• [branch-2.9][fix][broker] Fix index generator is not rollback after entries are failed added by @gaozhangmin in #19978

New Contributors

• @dao-jun made their first contribution in #19687

Full Changelog: v2.9.4...v2.9.5