v2.10.1

Important notice

• [fix][broker] Fix broker LoadBalance uneffective 15314
• [fix][admin] Fix producer/consume permission can’t get schema 15956

Broker

• [fix][broker] Fix race condition in getManagedLedgerInternalStats when includeLedgerMetadata=true 15918
• [improve][broker] Avoid contended synchronized block on topic load 15883
• [fix][broker] Fix NPE in MessageDeduplication 15820
• [improve][broker] Add timeout to closing CoordinationServiceImpl 15777
• [improve][broker] Store Subscription properties 15757
• [improve][broker] Support for updating the Subscription properties 15751
• [improve][broker] Disable memory limit controller for broker client and replication clients 15723
• [fix][broker] Fix NPE when put value to RangeCache. 15707
• [fix][broker] Fast return if ack cumulative illegal 15695
• [fix][broker] Fix creating producer failure when set backlog quota. 15663
• [fix][broker] Expose configurationMetadataStore and localMetadataStore 15661
• [fix][broker] Fix NPE when set AutoTopicCreationOverride 15653
• [fix][broker] Fix MetadataStoreException$NotFoundException while doing topic lookup 15633
• [fix][broker] Fix calculation in getNumberOfEntriesInStorage 15627
• [fix][broker] Use dispatchRateLimiterLock to update dispatchRateLimiter 15601
• [fix][broker] Sync topicPublishRateLimiter update 15599
• [fix][broker] Fix deadlock in broker after race condition in topic creation failure 15570
• [cleanup][broker] Override close method to avoid caching exception. 15529
• [fix][broker] Close publishLimiter when disable it 15520
• [fix][broker] Fix to avoid TopicStatsImpl NPE even if producerName is null 15502
• [fix][broker] Fix key-shared delivery of messages with interleaved delays 15409
• [fix][Broker] Fix bug in contructor of RocksdbMetadataStore 15405
• [feature][broker] EntryFilter (PIP-105) - support per-Consumer filtering 15391
• [fix][broker/client] Close connection if a ping or pong message cannot be sent 15382
• [improve][broker] Support shrink for ConcurrentSortedLongPairSet 15354
• [improve][broker] Support properties on NonDurable subscriptions (PIP-105) 15345
• [improve][broker] Use shrink map for message redelivery. 15342
• [fix][Broker] Fix error on recycled SubscriptionPropertiesList 15335
• [fix][broker] Fix problem at RateLimiter#tryAcquire 15306
• [fix][broker] Fix totalEntries calculation problem in AbstractBaseDispatcher#filterEntriesForConsumere 15298
• [fix][broker] Fix resource group does not report usage 15292
• [fix][Broker] Fix race condition between timeout and completion in OpAddEntry 15233
• [fix][broker] Fix MessageDeduplication#inactiveProducers may not be persistence correctly 15206
• [fix][broker] Cancel fencedTopicMonitoringTask when topic closed normally 15202
• [fix][broker] Fix parameter saslJaasBrokerSectionName in broker.conf 15110
• [cleanup][broker] Remove useless code to avoid confusion in OpReadEntry#checkReadCompletion 15104
• [fix][broker] Ensure NamespaceEphemeralData has equals() operator 15092
• [fix][broker] Fix potential to add duplicated consumer 15051
• [fix][broker] Fix rewind failed when redeliverUnacknowledgedMessages 15046
• [fix][broker]Fix race condition in updating lastMarkDeleteEntry field 15031
• [improve][broker] Avoid using blocking calls for the async method checkTopicOwnership 15023
• [fix][broker] Avoid heartbeat topic to offload. 15008
• [fix][broker] Return if reset in progress 14978
• [fix][broker] Fix topic policy reader close bug 14897
• [fix][broker] Fix getPendingAckInternalStats redirect issue 14876
• [fix][broker] Fix wrong state for non-durable cursor 14869
• [improve][broker] Add log when update namespace policies with error 14850
• [feature][broker] Support advertised listeners for HTTP and HTTPS services 14839
• [fix][broker] Filter the virtual NIC with relative path 14829
• [fix][broker] Fixed duplicated delayed messages when all consumers disconnect 14740
• [fix][broker] Fix cannot delete namespace with system topic 14730
• [fix][broker] Fixed 404 error msg not being returned correctly using http lookup 14677
• [fix][broker] Fix normal topic named ends with healthcheck becomes system topic issue 14671
• [improve][broker] Support shrink for map or set 14663
• [improve][broker] Changing the topic creation flow and optimize heartbeat topic not trigger compaction 14643
• [fix][broker] Fix wrong prompt exception when getting the non-persistent topic list without GET_BUDNLE permission 14638
• [fix][broker] Fix inconsistent prompt message when schema version is empty using AVRO. 14626
• [fix][broker] Fix update replication cluster but not update replicator 14570
• [improve][broker] Reduce unnecessary expansions for ConcurrentLong map and set 14562
• [improve][broker] Support ManagedCursorInfo compression 14542
• [improve][broker] Optimize memory usage: support to shrink for pendingAcks map 14515
• [improve][broker] Support shrink in ConcurrentLongHashMap 14497
• [improve][broker] Optimize RawReader#create when using Compactor 14447
• [fix][broker] Fix NPE when subscription is already removed 14363
• [improve][broker] Optimize load manager find nics process 14340
• [improve][broker] Make revokePermissionsOnTopic method async 14149
• [fix][broker] Fix when nextValidLedger is null caused NPE 13975

Clients

• [fix][Java] Fixes NPE when TableView handles null value message 15951
• [fix][Java] Fix conversion of TimestampMillisConversion has no effect when Jsr310Conversion enabled 15863
• [fix][Java] Fix messages sent by producers without schema cannot be decoded 15622
• [improve][Java] improve logic when ACK grouping tracker checks duplicated message id 15465
• [improve][Java] Add pending messages information while printing the producer stats 15440
• [fix][Java] Fix negative ack not redelivery 15312
• [improve][admin/client] AsyncHttpConnector doesn't use the system properties configured 15307
• [fix][Java] TableView should cache created readers 15178
• [fix][Java] Fix performance regression with message listener 15162
• [fix][Java] Fix internal receive used wrong timeout type 15014
• [fix][Java] ConsumerBuilderImpl can not set null to deadLetterPolicy. [14980](https:...

v2.10.0

Important notice

• Remove -XX:-ResizePLAB JVM option which degrades performance on JDK11 #12940
• Enable TCP keepAlive flag on the sockets #12982
• Reduce the time it takes for namespace bundle unloading to time out #12995
• Align configurations defaults between default file and Java object (broker.conf, proxy.conf, websocket.conf) #13272
• [PIP 118] Do not restart brokers when ZooKeeper session expires as default #13341
• [PIP 119] Enable consistent hashing by default on KeyShared subscriptions dispatcher #13352
• [PIP 120] Enable client memory limit controller by default #13344
• Make Docker images non-root, by default, and OpenShift compliant #13376
• [PIP 122] Change loadBalancer default loadSheddingStrategy to ThresholdShedder #13733
• Fix netcat returning early for probe #14088

PIPs

• [PIP 79] Add lazy-loading feature to PartitionedProducer #10279
• [PIP 84] Pulsar client: Redeliver command add epoch #10478
• [PIP 86] Pulsar Functions: Preload and release external resources #13205
• [PIP 92] Topic policy across multiple clusters #12517
• [PIP 104] Add new consumer type: TableView #12838
• [PIP-105] Support pluggable entry filter in Dispatcher #12869 #12970 #12979
• [PIP 106] Broker extensions to provide operators of enterprise-wide clusters better control and flexibility #12536
• [PIP 107] Introduce chunk message ID #12403
• [PIP 110] Support Topic metadata - PART-1 create topic with properties #12818
• [PIP 121] Pulsar cluster level auto failover on client side #13316
• [PIP-124] Create init subscription before sending message to DLQ #13355
• [PIP-130] Apply redelivery backoff policy for ack timeout #13707
• [PIP 131] Resolve produce chunk messages failed when topic level maxMessageSize is set #13599
• [PIP 135] Include MetadataStore backend for Etcd #13225

Broker

• [PIP 45] Pluggable metadata interface
  • Add BookKeeper metadata adapter based on MetadataStore #12770
  • Add Rocksdb metadata store #12776
  • Convert BookieRackAffinityMapping to use MetadataStore #12841
  • Allow to configure metadata store URL in broker.conf #13077
  • Remove old ZK caches implementations #13075
  • Allow to start bookie with Pulsar metadata store backend #13296
  • Remove ZooKeeperClientFactory #13303
  • Use reference counting in RocksDB metadata store #13309
  • Allow configuring metadata store URL in proxy.conf #13777
  • Allow configuring metadata store URL in functions_worker.yml #13782
  • Add configuration metadata store url setting in WebSocket #13786
  • Allow configuring configurationMetadataStore when initializing cluster metadata #13889
  • Using the consistent metadata store scheme name #13937
  • Allow configuring metadataStoreUrl in pulsar-perf managed-ledger #14145
  • Deprecate zookeeper settings #14147
  • Fix metadata cache inconsistency on doing refresh #14283
• Support broker level dispatch rate limiter #11325
• Support setting geo-replication clusters on topic level #12136
• Add Multi-tiered storage key to broker.conf #12173
• Support configuration to rate-limit dispatching on batch message #12294
• Support splitting the largest bundle of the namespace #12361
• Support creating a subscription by specifying the earliest or latest position #12872
• Support roll-over ledgers for inactive topics #13073
• Support graceful shutdown for Broker #14114
• Transparent batching of ZK operations #13043
• Add uniform load shedder strategy to distribute traffic uniformly across brokers #12902
• Provide option to split bundle based on load #12378
• Use AuthorizationService#grantPermissionAsync to grant topic permission #12515
• Only store authentication data after authentication is complete #12077
• Allow to have different instances LocalMemoryMetadataStore that share the same state #12390
• Allow GetTopicsOfNamespace op with consume permission #12600
• Add local filesystem backend for package manager #12708
• Add stop replicator producer logic when start replicator cluster failed #12724
• Apply PolicyHierarchyValue to inactiveTopicPolicies #12687
• Fix can not get leader broker in follower brokers #11353
• Fix broker gc log options #11285
• Fix NPE of ZkBookieRackAffinityMapping #11947
• Fix prefix setting in JWT authn and avoid multi calls for the getProperty #12132
• Fix missed check exit code of stop before calling start #12368
• Fix getting children of parent nodes in LocalMemoryMetadataStore #12491
• Fix collection get bug in ResourceGroupService #12499
• Fix deadlock in metadata-store callback thread #12753
• Improve exceptions thrown when handling the schema resource #12155
• Trim topic name #12453
• Avoid unnecessary recalculation of maxSubscriptionsPerTopic in AbstractTopic #12658
• Optimize isValidPath check in MetadataStore #12663
• Close connection after receiving unexpected SendCommand #12780
• Ensure cache is refreshed (and not just invalidated) after a store write #12788
• Optimize topicMaxMessageSize with topic local cache. #12830
• Optimize blocking backlogQuotaCheck to non-blocking in ServerCnx#handleProducer #12874
• Only refresh metadata if path is already in cache after write. #12896
• Optimize put and get methods in AbstractMetadataStore #12916
• Fix zk-node leak of admin path #12972
• Optimize TopicPolicy#deduplicationEnabled with HierarchyTopicPolicies #13000
• Fix race condition in FaultInjectionMetadataStore#programmedFailure #13007
• Fix usage of PULSAR_EXTRA_OPTS/BOOKIE_EXTRA_OPTS in startup scripts #13025
• Consider topics in pulsar/system namespace as system topics #13050
• Fix wrong result for looking up a non-exist topic by rest api #13055
• Key_Shared dispatcher with no connected consumers should be recreated if allowOutOfOrderDelivery changes #13063
• Make load-balancer config dynamic for the runtime tuning #13074
• Optimize TopicPolicy#maxProducersPerTopic with HierarchyTopicPolicies [#13082](https...

v2.9.2

ƒ Security

• Get rid of CVEs in Solr connector #13822
• Get rid of CVEs in InfluxDB connector #13821
• Get rid of CVEs in batch-data-generator #13820
• Get rid of CVEs brought in with aerospike #13819
• Suppress false positive Avro CVE-2021-43045 #13764
• Upgrade protobuf to 3.16.1 to address CVE-2021-22569 #13695
• Upgrade Jackson to 2.12.6 #13694
• Upgrade Log4j to 2.17.1 to address CVE-2021-44832 #13552
• Upgrade to Log4J 2.17.0 to mitigate CVE-2021-45105 #13392
• Cipher params not work in KeyStoreSSLContext #13322
• Use JDK default security provider when Conscrypt isn't available #12938
• Return if namespace policies are read only #12514
• Upgrade Postgre driver to 42.2.25 to get rid of CVE-2022-21724 #14119
• Bump slf4j from 1.7.25 to 1.7.32 #13595
• Upgrade Netty to 4.1.72 - CVE-2021-43797 #13328

Broker

• Fix lost message issue due to ledger rollover. #14664
• Fix thread leak in MLTransactionMetadataStore #14524
• Fix npe in ManagedLedgerImpl #14481
• Fix Future.join() causing deadlock. #14469
• Fix Broker HealthCheck Endpoint Exposes Race Conditions #14367
• Validate rack name #14336
• Fix avg-messagePerEntry metrics for consumer #14330
• Fix the wrong parameter in the log. #14309
• Fix batch ack count is a negative issue. #14288
• Bug fix: IllegalArgumentException: Invalid period 0.0 to calculate rate #14280
• Clean up individually deleted messages before the mark-delete position #14261
• If mark-delete operation fails, mark the cursor as "dirty" #14256
• Fix detecting number of NICs in EC2 #14252
• Remove log unacked msg. #14246
• Change broker producer fence log level #14196
• Avoid creating any topics in NamespaceService#checkTopicExists during topic lookup. #14134
• Fix NPE of cumulative ack mode and incorrect unack message count #14021
• Fix NPE - ReplicatedSubscriptionsController send marker message when enable deduplicated. #14017
• KeyShared stickyHashRange subscription: prevent stuck subscription in case of consumer restart #14014
• Add null check to workaround NPE in unit tests with Mockito/PowerMock #14006
• Fix handling of consumers with equal names on on key shared selector with consistent hashing #13991
• Trim configuration value string which contains blank prefix or suffix #13984
• WaitingCursors potential heap memory leak #13939
• Fix read schema compatibility strategy priority #13938
• Fix SystemTopicBasedTopicPoliciesService NPE issue. #13840
• NPE when get isAllowAutoUploadSchema #13831
• Fix topic produced through REST not support Authorization #13771
• Fix call sync method in async rest API for internalGetSubscriptionsForNonPartitionedTopic #13745
• Fix the deadlock while using zookeeper thread to create ledger #13744
• Fix inefficient forEach loop #13742
• Fix non persistent topic subscription error. #13685
• Fix invalid rack name cause bookie join rack failed #13683
• Release old bundle from ownership cache when operator split bundle #13678
• Avoid call sync method in async rest API for force delete subscription #13668
• Avoid call sync method in async rest API for delete subscription #13666
• Fix getInternalStats occasional lack of LeaderInfo #13656
• Fix internal topic effect by InactiveTopicPolicy. #13611
• Fix bug :Infinity value for CPU or Bandwidth usage #13609
• Change ContextClassLoader to NarClassLoader in BrokerInterceptor #13589
• Fix NPE when unloading namespace bundle #13571
• Allow both limit and limitsize be null #13557
• The log prints NamespaceService#isServiceUnitActive exception stack information. #13553
• Update log content #13540
• HealthCheck v2 failed #13525
• Change ContextClassLoader to NarClassLoader in AdditionalServlet #13501
• Optimize the debug log that affects performance, and unify the style #13498
• Fix multi roles authz cannot handle empty roles case #13477
• Fix the NPE in system topics policies service #13469
• Fix race conditions in closing producers and consumers #13428
• Fix deadlock in metadata-store callback thread for branch 2.9 #13426
• Remove check resource when delete failure domain #13421
• Fix create the dynamic configuration resource if not exist #13420
• Fix batch message ack does not decrease the unacked-msg count. #13383
• Fix dead loop in BacklogQuotaManager.dropBacklogForTimeLimit #13249
• Modify return result of NamespacesBase#internalGetPublishRate #13237
• Optimize ManagedLedger Ledger Ownership Check #13222
• Fix shedding heartbeat ns #13208
• Fix when deleting topic with NotFoundException, do not return to client. #13203
• Clean up active consumer on already closed connection #13196
• Update cursor last active timestamp when reseting cursor #13166
• Use current resourceUsage value as historyUsage when leader change in ThresholdShedder #13136
• Don't attempt to delete pending ack store unless transactions are enabled #13041
• Fix race condition in ResourceLockImpl#revalidate #13006
• Fix NPE in PersistentTopic.checkSubscriptionTypesEnable #12961
• Fix wrong isEmpty method of ConcurrentOpenLongPairRangeSet #12953
• Correct param of delete method for v1 topic #12936
• Change create topic return error to Status.BAD_REQUEST #12919
• Fix topic policy listener deleted by mistake. #12904
• Remove readerCaches and close reader when exception occurs in SystemTopicBasedTopicPoliciesService #12873
• Fix deleting tenants with active namespaces with 500. #12848
• Bug Fix: topic policy is not properly init if namespace is loaded first. #12833
• Add broker config isAllowAutoUpdateSchema #12786
• Fix TopicPoliciesCacheNotInitException issue. #12773
• The problem of two exception handling #12744
• Handle lookup redirect for V1-topics with different cluster #12743
• ...

v2.8.3

Important Notices

• Fix detecting number of NICs in EC2 #14252. In the event that Pulsar cannot determine the NIC speed from the host, please set loadBalancerOverrideBrokerNicSpeedGbps.
• Bump BookKeeper 4.14.3 12906
• Add broker config isAllowAutoUpdateSchema 12786

Security

• Upgrade Postgres driver to 42.2.25 to get rid of CVE-2022-21724 14119
• Get rid of CVEs in Solr connector 13822
• Get rid of CVEs in InfluxDB connector 13821
• Get rid of CVEs in batch-data-generator 13820
• Get rid of CVEs brought in with aerospike 13819
• [owasp] suppress false positive Avro CVE-2021-43045 13764
• Upgrade protobuf to 3.16.1 to address CVE-2021-22569 13695
• Upgrade Jackson to 2.12.6 13694
• Upgrade Log4j to 2.17.1 to address CVE-2021-44832 13552
• Cipher params not work in KeyStoreSSLContext 13322
• [Broker] Remove tenant permission verification when list partitioned-topic 13138
• Use JDK default security provider when Conscrypt isn't available 12938
• [Authorization] Return if namespace policies are read only 12514

Pulsar Admin

• Make sure policies.is_allow_auto_update_schema not null 14409
• pulsar admin exposes secret for source and sink 13059
• Fix deleting tenants with active namespaces with 500. 13020
• [function] pulsar admin exposes secrets for function 12950

Bookkeeper

• Upgrade BK to 4.14.4 and Grpc to 1.42.1 13714
• Bump BookKeeper 4.14.3 12906

Broker

• Fix the wrong parameter in the log. 14309
• Fix batch ack count is negative issue. 14288
• bug fix: IllegalArgumentException: Invalid period 0.0 to calculate rate 14280
• Clean up individually deleted messages before the mark-delete position 14261
• If mark-delete operation fails, mark the cursor as "dirty" 14256
• Fixed detecting number of NICs in EC2 14252
• Remove log unacked msg. 14246
• Change broker producer fence log level 14196
• Fix NPE of cumulative ack mode and incorrect unack message count 14021
• KeyShared stickyHashRange subscription: prevent stuck subscription in case of consumer restart 14014
• Trim configuration value string which contains blank prefix or suffix 13984
• waitingCursors potential heap memory leak 13939
• Fix read schema compatibility strategy priority 13938
• NPE when get isAllowAutoUploadSchema 13831
• Fix call sync method in async rest API for internalGetSubscriptionsForNonPartitionedTopic 13745
• Fix the deadlock while using zookeeper thread to create ledger 13744
• Fix inefficient forEach loop 13742
• [Issue 13640] Fix non persistent topic subscription error. 13685
• Fix invalid rack name cause bookie join rack failed 13683
• Avoid call sync method in async rest API for force delete subscription 13668
• Avoid call sync method in async rest API for delete subscription 13666
• Fix getInternalStats occasional lack of LeaderInfo 13656
• Fix reader skipped remaining compacted data during the topic unloading. 13629
• [Issue 13479] Fixed internal topic effect by InactiveTopicPolicy. 13611
• Fix bug :Infinity value for CPU or Bandwidth usage 13609
• Change ContextClassLoader to NarClassLoader in BrokerInterceptor 13589
• Fix NPE when unloading namespace bundle 13571
• update log content 13540
• Return message ID from compacted ledger while the compaction cursor reach the end of the topic 13533
• Change ContextClassLoader to NarClassLoader in AdditionalServlet 13501
• fix(Auth): Fix multi roles authz cannot handle empty roles case 13477
• Fix getting the last message-id from an empty compact ledger 13476
• Fixes the NPE in system topics policies service 13469
• Fix race conditions in closing producers and consumers 13428
• Fix batch message ack does not decrease the unacked-msg count. 13383
• [Issue 13194][pulsar-broker] Fix dead loop in BacklogQuotaManager.dropBacklogForTimeLimit 13249
• Modify return result of NamespacesBase#internalGetPublishRate 13237
• Optimize ManagedLedger Ledger Ownership Check 13222
• Close old compacted ledger when open new. 13210
• fix shedding heartbeat ns 13208
• Fix when deleting topic with NotFoundException, do not return to client. 13203
• Update cursor last active timestamp when reseting cursor 13166
• Remove tenant permission verification when list partitioned-topic 13138
• Use current resourceUsage value as historyUsage when leader change in ThresholdShedder 13136
• Don't attempt to delete pending ack store unless transactions are enabled 13041
• Fix NPE in PersistentTopic.checkSubscriptionTypesEnable 12961
• Fix wrong isEmpty method of ConcurrentOpenLongPairRangeSet 12953
• Correct param of delete method for v1 topic 12936
• Clean up the metadata of the non-persistent partitioned topics. 12910
• Fix topic policy listener deleted by mistake. 12904
• Fix deleting tenants with active namespaces with 500. 12848
• [Issue 12757] add broker config isAllowAutoUpdateSchema 12786
• Remove unused listeners if it have no listeners. 12654
• Clean up the metadata of the non-persistent partitioned topics. 12550
• [managedledger] NPE on OpAddEntry while ManagedLedger is closing 12364
• fix issues 11964, deadlock bug when use key_shared mode 11965

Build

• [C++] Fix GCC compilation failure caused by warning macro 14402
• [C++] Fix Version.h not found when CMake binary directory is customized 13324
• [Issue 9888] add python3.9 on manylinux2014 build support 10954

C++ Client

• Fix GCC compilation failure caused by warning macro 14402
• Fix pulsar client cpp build fail in gcc-4.8.5 14053
• Fix hasMessageAvailable returns wrong value for last message 13883
• Fix Version.h not found when CMake binary directory is customized 13324
• Fix in macOS CMake might find error boost-python libs path 13193

CI

• Upgrade Windows runner os to windows-2022 and generator 14368
• Replace deprecated "adopt" OpenJDK distribution with "temurin" in GitHub Actions ...

v2.8.2

Security

• Upgrade to Log4J 2.17.0 to mitigate CVE-2021-45105 #13392
• Upgrade Netty to 4.1.72 - CVE-2021-43797 #13328
• Bump log4j to 2.15.0 #13226
• Revert new AuthorizationProvider method #13133
• Support CLEAR_BACKLOG namespace op after enable auth #12963
• Upgrade netty to 4.1.68.Final #12218
• Support disabling non-TLS service ports #11681
• Upgrade Jetty to 9.4.43.v20210629 #11660

Broker

• Fix and improve topic ownership assignment #13069
• Fix LeaderElectionService.getCurrentLeader and add support for empheralOwner in MockZooKeeper #13066
• Do not reuse the Failed OpAddEntry object which leads to the bundle unloading timeout. #12993
• Remove readerCaches and close reader when exception occurs in SystemTopicBasedTopicPoliciesService #12873
• Fix TopicPoliciesCacheNotInitException issue. #12773
• Support UNSUBSCRIBE namespace op after enabling auth #12742
• Fix race condition in PersistentTopic#addReplicationCluster #12729
• Even if always compatible is set, consumers cannot be created #12721
• Fix the incorrect total size when BrokerEntryMetadata is enabled #12714
• Fix lost compaction data due to compaction properties missed during reset-cursor #12698
• Fix TestRunMain test #12675
• Support GET_METADATA topic op after enabling auth #12656
• Fix false positive ownership check in OwnershipCache#checkOwnership #12650
• Optimize exception information for schemas #12647
• Add @test annotation to test methods #12640
• Support retry when creating reader of Topic Policies #12622
• Fix String should use equals but not ==. #12619
• Fix 12614, waitingForPingResponse needs to be modified with volatile for concurrent sence #12615
• Cleanup ProxyPublishConsumeTest #12607
• Avoid passing OpAddEntry across a thread boundary in asyncAddEntry #12606
• Do not move the non-durable cursor position when trimming ledgers while topic with compaction #12602
• Allow GetTopicsOfNamespace op with consume permission #12600
• Allow configuring schema compatibility policy for system topics #12598
• Cleanup already deleted namespace topics. #12597
• Fix additional servlets NAR might extract to null directory #12585
• Fix log typo in NamespaceService#checkHeartbeatNamespace #12582
• Add OpAddEntry to pendingAddEntries after the state check #12570
• Cancel scheduled tasks when deleting ManagedLedgerImpl #12565
• Add git branch information for PulsarVersion #12541
• Websocket should pass the encryption context to consumers #12539
• The count of topics on the bundle is less than 2，skip split #12527
• Fix the reader skips compacted data which original ledger been removed #12522
• Fix messageDedup delete inactive producer name #12493
• Optimize the code: remove extra spaces #12470
• Future completed twice in the method of impl.MLPendingAckStore#closeAsync #12362
• Fix the race of delete subscription and delete topic #12240
• Disable stats recorder for built-in PulsarClient #12217
• Fix delete authentication policies when deleting topics. #12215
• Optimize the memory usage of Cache Eviction #12045
• Avoid adding duplicated BrokerEntryMetadata #12018
• Fix update ledger list to znode version mismatch failed, ledger not delete #12015
• Fix messages in TopicPolicies will never be cleaned up #11928
• Fix returned wrong hash ranges for the consumer with the same consumer name #12212
• Add Key_Shared metadata to topic stats #11839
• Fix build from submodules (broker, transaction coordinator) #11795
• Add method to clear up transaction buffer snapshot #11934
• Increase the test stability of transactionTest #11541
• Add maven.restlet.org repository #13248
• Fix and improve topic ownership assignment (#13069) #13117
• Evaluate the current protocol version once #13045
• Revert "Set default root log level to debug" and make PULSAR_LOG_ROOT_LEVEL default to PULSAR_LOG_LEVEL #12941
• Catch exceptions in scheduled tasks to prevent unintended cancellation #12853
• Fix producer getting incorrectly removed from topic's producers map #12846
• Synchronize updates to the inactiveProducers map in MessageD… #12820
• Close connection after receiving unexpected SendCommand #12780
• Fix namespace policy override ignored when creating subscription #12699
• Update lombok to 1.18.22 #12466
• Fix skips compacted data for reader/consumer #12464
• Remove data race in MultiTopicsConsumerImpl to ensure correct message order #12456
• Fix the retry topic's REAL_TOPIC & ORIGIN_MESSAGE_ID property #12451
• Change the producer fence error log to debug level #12447
• Reduce the readFailureBackoff time #12444
• Fix wrong property name in NamespaceIsolationDataImpl#secondary #12433
• Optimize SecurityUtility code flow #12431
• Fix compactor skips data from last compacted Ledger #12429
• Remove redundant code #12424
• Fix some tests not enabled in integration tests #12417
• Use weak ref to ClientConnection for timeout task #12409
• Fix cherry-pick issue #12397
• Fix the null point caused by deleting the system topic policy #12367
• Update delete inactive topic configuration documentation #12350
• Add log to print exception stack. #12345
• Avoid potentially blocking calls to metadata on critical threads #12339
• Fix NPE when removing cursor #12297
• Print log when configuration is failed to load #12280
• Fix incorrect returned last message ID while the lastConfirmedEntry with negative entry ID #12277
• Fix TTL expiry does not take effect #12266
• The loadbalancer should avoid offload the heartbeat namespace #12252
• Fix typo of the returned last message ID when the last message ID is from compacted ledger #12237
• Add support for splitting topics and partition labels in Prometheus #12225
• Fix lost message issues 12221 #12223
• Allow to config pulsar client allo...

v2.7.4

Security

• Upgrade log4j to 2.17.0 - CVE-2021-45105 #13392
• Upgrade commons-compress to 1.21 - CVE-2021-35515, CVE-2021-35516, CVE-2021-35517 and CVE-2021-36090 #11345

Broker

• Fix DispatchRateLimiter does not take effect #8611
• Fix set-publish-rate when using preciseTopicPublishRateLimiterEnable=true #10384
• Fix retention policy in topic policy not work #11021
• Fix GetTopicsOfNamespace with binary lookup service not check auth #11172
• Fix retention of keys in compaction #11287
• Fix using partitioned topic name to get Policy #11294
• Fix NPE when unloading persistent partitioned topic #11310
• Remove RateLimiter constructors and replace with builder usage #11599
• Add backoff for setting for getting topic policies #11574
• Fix decode compression managedLedgerInfo data #11569
• Fix some topic policy operation without backoff #11560
• Fix data lost when using earliest position to subscribe to a topic #11547
• Fix the schema deletion when delete topic with delete schema #11501
• Do not create system topic for heartbeat namespace #11499
• Compress managed ledger info #11490
• Add backoff for setting for getting topic policies #11487
• Solving precise rate limiting does not take effect #11446
• Fix replicated subscriptions direct memory leak #11396
• Close the replicator and replication client when deleting cluster #11390
• Invalidate the read handle after all cursors are consumed #11389
• Fix NPE when unloading persistent partitioned topic #11386
• Fix retention of keys in compaction #11384
• Disable replicate system topic across clusters #11376
• Parallel Precise Publish Rate Limiting Fix #11372
• Pulsar Admin List Subscription lists only subscriptions created for Partition-0 when partition specific subscriptions are created #11355
• Fix inconsistent behavior for Namespace bundles cache #11346
• Fix update ledger list to znode version mismatch failed, ledger not delete #12015
• Print position info when can't find next valid position #11969
• Forbid to read other topic's data in managedLedger layer #11913
• Fix using partitioned topic name to get topic policies #11897
• Refine topic level backlog quota policies warning log #11863
• Producer getting producer busy is removing existing producer from list #11804
• Handle NPE and memory leak when full key range isn't covered with active consumers #11749
• Call .release() when discarding entry to prevent direct memory leak #11748
• Fix the topic in fenced state and can not recover #11737
• Support disabling non-TLS service ports #11724
• Fix the bug, can not update topic when the update topicName is contained by an existed topic as a part #11686
• Fix Pulsar didn't respond error messages when throw InterceptException #11650
• Avoid redundant calls for getting the offload policies from the offloader #11629
• Optimize ManagedLedger Ledger Ownership Check #13222
• Do not reuse the Failed OpAddEntry object which lead bundle unloading timeout #12993
• Fix producer getting incorrectly removed from topic's producers map #12846
• Allow to configure schema compatibility policy for system topics #12598
• Cleanup already deleted namespace topics #12583
• Add OpAddEntry to pendingAddEntries after the state check #12570
• Cancel scheduled tasks when deleting ManagedLedgerImpl #12565
• NPE on OpAddEntry while ManagedLedger is closing #12364
• Avoid potentially blocking calls to metadata on critical threads #12340
• Fix used after recycle issue in OpAddEntry #12103
• Optimize the memory usage of Cache Eviction #12045
• Fix wrong key-hash selector used for new consumers after all the previous consumers disconnected #12035
• Only auth_errors should log at error level #9325

Dependency upgrade

• Remove Boost::System runtime dependency #9498
• Upgrade Jetty to 9.4.43.v20210629 #11660
• Add maven.restlet.org repository #13248
• Upgrade ZooKeeper version to 3.5.9 #12981
• Remove pulsar-dashboard from the publish process of images #12534
• Roll back to using Java 8 for docker images #12357
• Upgrade TestContainers to 1.15.1 #9120

Client

• [Java] Refactor seek to reuse common logic #9670
• [Java] Process partitioned-topic messages on different listener-threads #10017
• [Java] Fix Consumer listener does not respect receiver queue size #11455
• [Java] Add a optional params scope for pulsar oauth2 client #11931
• [Java] Fix seek at batchIndex level receive duplicated messages #11826
• [Java] Fix race condition on multi-topic consumer #11764
• [Java] Remove consumer reference from PulsarClient on subscription failure #11758
• [Java] Fix block forever bug in Consumer.batchReceive #11691
• [Java] Send CloseProducer on timeout #13161
• [Java] Remove invalid call to Thread.currentThread().interrupt() #12652
• [Python] Fix fields that are ignoring the required key argument #11508
• [C++] Remove usages of boost::regex #9533
• [C++] Use sendRequestWithId to add timeout to hasMessageAvailable #11600
• [Pulsar perf] Fix log level config for pulsar-admin, pulsar-client and pulsar-perf #12915

Functions and Pulsar IO

• Fix source stats exposing empty exceptions list #11478
• Fix function API can not use authdata to check superuser #11418
• Support setting KEY_BASED batch builder for Pulsar Sinks #11710
• Detect .nar files and prevent spammy logs on functions boot #12665
• Remove the deprecated api usage in hdfs #12080

Tiered Storage

• Fix the potential race condition in the BlobStore readhandler #12123

Build

• Force CMake to find Python2 #9690
• Use ubuntu 20.04 as docker image base #12017
• Fix docker image install python3.7-dev problem #11942

v2.8.1

Broker

• Fix the issue of all web threads get stuck when deleting a namespace #11596
• Improve error logs in BacklogQuotaManager #11469
• Solve the issue of precise rate limiting does not take effect #11446
• Fix replicated subscriptions direct memory leak #11396
• Change ContextClassLoader to NarClassLoader in ProtocolHandler #11276
• Fix the issue of ledger rollover scheduled tasks were executed before reaching the ledger maximum rollover time #11116
• Fix publish_time not set error when broker entry metadata enable without AppendBrokerTimestampMetadataInterceptor #11014
• Fix parseMessageMetadata error cause by not skip broker entry metadata #10968
• Fix issue where Key_Shared consumers could get stuck #10920
• Fix throwable exception not thrown #10863
• Handle multiple topic creation for the same topic-name in broker #10847
• Add REST API to enable or disable replicated subscriptions #10790
• Fix issue that message ordering could be broken when redelivering messages on Key_Shared subscription #10762
• Fix set-publish-rate when using preciseTopicPublishRateLimiterEnable=true #10384
• Upgrade BookKeeper version to resolve the BouncyCastle issue #11759
• Fix getPreviousPosition NPE #11621
• Remove duplicated configuration #11283
• Source tarball: apply executable file permissions to shell scripts #11858
• Fix java_test_functions build failed #11829
• Fix generate javadoc for kafka-connect-adaptor failed #11807
• Fix unnecessary user interactions when building pulsar-standalone image #11623
• Do not expose meaningless stats for publisher #11454
• Add metrics storageLogicalSize for the TopicStats and NamespaceStats #11430
• Compress managed ledger info #11490
• Print message metadata when getting message by id #11092
• Query parameter "negativeAckRedeliveryDelay" should be effective even if DLQ is disabled #11495
• Fix websocket TLS bug #11243
• Fix the Pulsar Proxy flaky test (Collector already registered that provides name: jvm_memory_direct_bytes_used) #11900
• Fix flaky test testReacquireLocksAfterSessionLost #11815
• Fix flaky test testUpdateDynamicLocalConfiguration #11115
• Fix flaky test testBrokerRanking #11114
• Fix flaky test in AdminApiOffloadTest #11028
• Fix the flaky test in the ManagedLedgerTest #11016
• Make Metadata ZKSessionTest less Flaky #10955
• Make MetadataCacheTest reliable. #10877
• Fix pulsar_standalone docker image build failed #11862
• Producer getting producer busy is removing existing producer from list #11804
• Revert PR 11594 to avoid copy data to direct buffer #11792
• Upgrade aircompressor to 0.20 #11790
• Fix wrappedBuffer always using the same block of memory #11782
• Fix com.squareup.okhttp-okhttp-2.7.4.jar unaccounted for in LICENSE bug #11769
• Handle NPE and memory leak when full key range isn't covered with active consumers #11749
• Call .release() when discarding entry to prevent direct memory leak #11748
• Avoid duplicated disconnecting producers after fail to add entry#11741
• Expose compaction metrics to Prometheus #11739
• Fix the topic in a fenced state and can not recover#11737
• Remove subscription when closing Reader on non-persistent topics #11731
• Fix branch-2.8 cherry-pick issue. #11694
• KeyShared dispatcher on non-persistent topics was not respecting consumer flow-control #11692
• Fix the bug, can not update topic when the update topicName is contained by an existed topic as a part #11686
• If a topic has compaction policies configured, we must ensure the subscription is always pre-created #11672
• Fix testSetReplicatedSubscriptionStatus run failed #11661
• Fix Pulsar didn't respond error messages when throw InterceptException #11650
• Fix license mismatch #11645
• Remove unnecessary jar name in LICENSE files #11644
• Fix java.lang.NoSuchMethodError: java.nio.ByteBuffer.position(I)Ljava/nio/ByteBuffer when enabling topic metadata compression #11594
• Fix decode compression managedLedgerInfo data #11569
• Fix data lost when using earliest position to subscribe to a topic #11547
• Add test for auto-created partitioned system topic #11545
• Replace orElse with orElseGet to avoid calling too many times. #11542
• Fix the schema deletion when delete topic with delete schema #11501
• Add metrics for writing or reading size of cursor #11500
• Do not create system topic for heartbeat namespace #11499
• Add additional servlet support to broker #11498
• Add metrics [AddEntryWithReplicasBytesRate] for namespace #11472
• Deep copy the tenants to avoid concurrent sort exception #11463
• Reduce the probability of cache inconsistencies #11423
• Reduce integration test memory usage in CI #11414
• Swap getTopicReference(topic) with serviceUnit.includes to reduce calling getTopicReference #11405
• Invalidate the read handle after all cursors are consumed#11389
• Parallel Precise Publish Rate Limiting Fix #11372
• Fix concurrency issues in NarUnpacker #11343
• Close the replicator and replication client when delete cluster#11342
• Add multi roles support for authorization #11341
• Fix NPE when unloading persistent partitioned topic #11310
• Fixed retention of keys in compaction #11287
• Fix missing replicator metrics #11264
• Simplify managedLedger retention trim logic #11255
• Fix retention size policy delete too much ledgers #11242
• Fix init WebSocketService with ClusterData #11234
• Make the compaction phase one loop timeout configurable #11206
• Fixed using CommandSubscribe.getConsumerName() without checking #11199
• Fix some typos of the PersistentTopics #11187
• Fix failing auth test. #11186
• Fix the deadlock when using hasMessageAvailableAsync and readNextAsync #11183
• Fix compaction entry read exception #11175
• Set -Dio.netty.tryReflectionSetAccessible=true for pulsar processes #11138
• Fix broker dispatch byte rate limiter. [#11135](https://github.com/apache/...

v2.7.3

Broker

• Fix Delayed Messages. #11374
• Fix missing replicator metrics. #11264
• Fix branch-2.7 test. #11254
• Fix broker dispatch byte rate limiter #11249
• Fix websocket TLS bug #11243
• Fix init WebSocketService with ClusterData #11234
• Fix ledger roll over scheduler task #11226
• Remove unused listener to reduce creating executor pool #11215
• Make the compaction phase one loop timeout configurable #11206
• Fix failing auth test. #11186
• Fix the dead lock when using hasMessageAvailableAsync and readNextAsync #11183
• Fix compaction entry read exception #11175
• On multi-topic consumer, we shouldn't keep checking the partitioned metadata #11168
• Fix replay topic policy message not work #11136
• Fix broker dispatch byte rate limiter. #11135
• Change test group to broker for ReplicatorTest and fix the test #11134
• Fix ledger rollover scheduled task #11116
• Fix race condition of the SystemTopicBasedTopicPoliciesService #11097
• Print message metadata when getting message by id #11092
• Fix flaky test testEnableAndDisableTopicDelayedDelivery #11009
• Fix potential data lost on the system topic when topic compaction has not triggered yet #11003
• Fix direct memory leak in getLastMessageId #10977
• Fix the backlog issue with --precise-backlog=true #10966
• Fix NonRecoverableLedgerException when get last message ID by Reader #10957
• Fix compaction not working for system topic #10941
• Fix issue where Key_Shared consumers could get stuck #10920
• When the Replicator is enabled, no managedLedger is created when updating the number of partitions #10910
• Handle multiple topic creation for the same topic-name in broker #10847
• Release OpAddEntry.data when entry is copied and discarded #10773
• Fix issue that message ordering could be broken when redelivering messages on Key_Shared subscription #10762
• Fix solution for preventing race conditions between timeout and completion #10740
• Cancel scheduled tasks as the first step in closing #10739
• MINOR: Add error message to setMaxPendingMessagesAcrossPartitions #10709
• Make PrometheusMetricsTest. testAuthMetrics pass on CI #10699
• Fix issue where StackOverflowError occurs when trying to redeliver a large number of already acked messages #10696
• Revert "Creating a topic does not wait for creating cursor of replicators" #10674
• Use single instance of parser #10664
• Ensure all the ReadHandle gets properly closed on cache invalidation #10659
• Fix ConcurrentOpenLongPairRangeSet remove all ranges #10656
• TopicPoliciesTest.testMaxSubscriptionsFailFast fails #10640
• Add metrics for non-contiguous deleted messages range #10638
• Fixed missed ZK caching when fetching a list of namespaces for a tenant #10594
• Made OpAddEntry.toString() more robust to nulls to prevent NPEs #10548
• Fix partitioned system topic check bug #10529
• Make failPendingMessages called from within the ProducerImpl object mutex #10528
• Fix deadlock on Monitoring thread blocked by LeaderService.isLeader() #10512
• Fix: Topic loading fails without any error when replicator init fails #10432
• Fix hasMessageAvailable return true but can't read message #10414
• Added more unit tests to the JavaInstanceTest class #10369
• Fix authorization error if partition number of partitioned topic is updated. #10333
• Fix the inconsistency of AdvertisedAddress #10312
• Fix missing LoggerFactoryPtr type. #10164
• Ensure read-lock is not continuously held on a section while iterating over concurrent maps #9787
• Zookeeper connections monitor data #9778
• Change getWorkerService method to throw UnsupportedOperationException #9738
• Fix flaky unit test #9262
• Supply debug log for OpAddEntry #9239

Dependency upgrade

• Upgrade Jetty to 9.4.42.v20210604 #10907

Proxy

• Enable AutoTopicCreationType partitioned through proxy #8048

Pulsar Admin

• Fix create partitioned topic in replicated namespace #11140
• Add authoritative flag for topic policy to avoid redirect loop #11131
• Fix non-persistent topic get partitioned metadata error on discovery #10806
• Fix kinesis sink backoff class not found #10744

Docker

• K8s Function Name Length Check Allows Invalid StatefulSet #10531

Client

• [Java] Cleaned some code in GenericJsonRecord #10527
• [C++] Avoid sending flow requests with zero permits #10506

Functions and Pulsar IO

• Fix kinesis sink connector does not ack messages #10769
• Remove reference to ProducerSpec from Pulsar Functions GO #10635
• Process async results in the same Java runnable thread #10618

v2.8.0

PIPs

• [PIP 45] Pluggable metadata interface #9148 #9221 #9240 #9273 #9274 #9338 #9346 #9351 #9412 #9485 #9586 #9967 #9973 #10391 #10457 #10532 #10545 #10647
• [PIP 50] Package management service #8637 #8680 #8744 #8764 #8816 #8817 #8858 #8861 #8893 #8907
• [PIP 68] Exclusive producer #8685 #8992 #9554 #9600
• [PIP 70] Lightweight broker entry metadata #8618 #9067 #9088 #9091
• [PIP 71] Pulsar SQL migrate SchemaHandle to Presto decoder #8422
• [PIP 74] Client memory limits #8965 #9676
• [PIP 75] Perform serialization/deserialization with LightProto #9046
• [PIP 76] Streaming offloader #9096
• [PIP 82] Tenant and namespace level rate limiting #10008 #10201 #10204 #10218
• [PIP 83] Message consumption with pooled buffer #10184
• [PIP 85] Support get reader schema for a message in Java Client #10476

Transactions

• Support pending ack state persistence #8881
• Stable position and low watermark for the transaction buffer #9195
• Transaction timeout support #9229
• Transaction coordinator retry to complete the transaction #9236
• Fix race condition when appending transaction log #9238
• Transaction buffer snapshot #9490
• Add metrics for transaction coordinator #9706
• Clean the useless transaction individual acknowledgements based on low watermark #9722
• Fix memory leak when deleting transaction marker #9751
• Check the transaction state at the client side #9776
• Clean aborted transactions for the transaction buffer #9974
• Fix transaction coordinator retry to end transaction #10131
• Fix NPE when opening a new transaction #10139
• Fix transaction log failed to recover #10146
• Fix transaction coordinator recover timeout #10162
• Handling committing and aborting state when recovering transaction coordinator #10179
• Fix NPE in case of enableTransactionCoordinator=false #10182
• Fix transaction buffer client timeout #10206
• Fix recover max local id issue for the transaction coordinator #10224
• Support using transactions on standalone #10238
• Fix transaction buffer lookup issue #10257
• Fix transaction timeout issue at the client side #10323
• Fix transaction client reconnect issue after transaction coordinator unloaded #10327
• Fix transaction timeout not canceled after expired #10366
• Transaction log low watermark optimization #10422
• Fix the issue of transaction buffer client channel is inactive #10407
• Add Admin API for getting transaction coordinator stats #10639
• Add Admin API for getting transaction in transaction buffer stats #10642
• Add Admin API for getting transaction in pending ack stats #10648
• Add Admin API for getting transaction buffer stats and pending ack stats #10650
• Add Admin API for getting transaction coordinator internal stats #10653
• Setup transaction metadata with metadata store #10677
• Fix issue with acknowledge messages from multiple subscriptions of a topic #10689
• Admin API for getting transaction metadata #10690
• Admin API for getting slow transactions #10701
• Fix transaction log handle managed ledger WriteFail state #10711
• Admin API for getting pending ack internal stats #10725
• Fix transaction ack delete marker position when don't have transaction ack #10741
• Fix transaction pending ack generate managedLedgerStats fail exception #10797
• Use Zookeeper Prometheus metric provider to export Zookeeper metrics #10803
• Always allow system topic for transaction buffer snapshot auto create #10876

Security

• Optional auth method name header in HTTP authentication #6799
• Topics level authorization support #7523
• Authorization for setting topic/subscription auto-creation and subscription expire time on namespace #7900
• Allow serializable stream-provider field into AuthenticationTls #10020
• Add configuration for running OWASP Dependency Check for all modules #10288
• Update default TLS protocols to TLSv1.3 and TLSv1.2 for the broker and proxy #10598

Broker

• Share EventLoopGroup between broker and BookKeeper client #2603
• Dispatch batch messages according to consumer permits 7266
• Improve the max pending publish buffer mechanism 7406
• Allow disabling HTTP TRACE/TRACK verbs #7907
• Fix DispatchRateLimiter does not take effect #8611
• Validate namespace isolation policy regex before updating #8804
• Perform automatically cert refresh for pulsar-admin #8831
• Fix updating rack info dynamically #8844
• Fix deadlock when checking message expiration #8877
• Support limit max topics per namespace #8942
• Make ledger rollover check task internally #8946
• Clean up topic that failed to unload from the cache #8968
• Support get broker response for the message acknowledgement #8996
• Support message index for a topic #9039
• Supporting limit max topics per namespace by namespace policy #9042
• Streaming dipsatcher support #9056
• Make Netty acceptor threadPool size con...

v2.6.4

Official release notes:
http://pulsar.apache.org/release-notes/#2.6.4

Download area:
http://pulsar.apache.org/en/download/

Broker

• Disallow parsing of token with none signature in authenticateToken #9172
• Fix marking individual deletes as dirty #9732
• Issue 9082: Broker expires messages one at a time after topic unload #9083
• [logging] Upgrade Log4j2 version to 2.14.0, replace legacy log4j dependency with log4j-1.2-api #8880
• Upgrade Bouncy castle to newest version #8047
• Fixed logic for forceful topic deletion #7356
• Perform periodic flush of ManagedCursor mark-delete posistions #8634
• Fix the batch index ack persistent issue. #9504
• Fix the partition number not equals expected error #9446
• fix the closed ledger did not delete after expired #9136
• Fix testBrokerSelectionForAntiAffinityGroup by increasing OverloadedThreshold #9393

Tiered storage

• [tiered-storage] Allow AWS credentials to be refreshed #9387

Java client

• Compression must be applied during deferred schema preparation and enableBatching is enabled #9396

C++ client

• [C++] Remove namespace check for MultiTopicsConsumerImpl #9520