v0.9.11

Summary:

• Fixed:
  • 30cdf5fb New leader must flush blank log.

Detail:

Fixed:

• Fixed: 30cdf5fb New leader must flush blank log; by 张炎泼; 2024-05-14

  This commit addresses a critical issue where if a new leader does not
  flush the blank log to disk upon becoming established and then restarts
  immediately, there is a possibility that previously committed data
  becomes invisible to readers.

  Before the blank log is flushed, the leader (identified by vote v3)
  assumes it will be flushed and commits this log once (|quorum|-1)
  replication responses are received. If the blank log is lost and the
  server is restarted, data committed by a new leader (vote v2) may
  not be visible.

  This issue is addressed by utilizing LeaderHandler::leader_append_entries()
  instead of ReplicationHandler::append_blank_log(), where the former
  does not wait for the blank log to flush.

  Changes:

  • When assigning log IDs to log entries, the Leading.last_log_id,
    which represents the state of the log proposer (equivalent term in
    Paxos is Proposer), should be used instead of RaftState.last_log_id,
    which represents the state of the log receiver (equivalent term in
    Paxos is Acceptor).

  • Consequently, the method assign_log_ids() has been moved from
    RaftState to Leading.

  • Avoid manual implementation of duplicated logic:

    • During initialize(), reuse FollowingHandler::do_append_entries()
      to submit the very first log to storage.

    • In establish_leader(), reuse
      LeaderHandler::leader_append_entries() to submit log to storage
      and remove ReplicationHandler::append_blank_log().

    • Remove Command::AppendEntry.

v0.9.10

Summary:

• Improved:
  • 14f174e9 cancel passed to full_snapshot() should be static.

Detail:

Improved:

• Improved: 14f174e9 cancel passed to full_snapshot() should be static; by 张炎泼; 2024-05-06

v0.9.9

Summary:

• Fixed:
  • 8b62c797 Immediate response when snapshot installation is unnecessary.

Detail:

Fixed:

• Fixed: 8b62c797 Immediate response when snapshot installation is unnecessary; by 张炎泼; 2024-05-05

  When Engine::handle_install_full_snapshot() is called and the provided
  snapshot is not up-to-date, the snapshot should not be installed, and
  the response should be sent back immediately. Previously, the method
  might delay the response unnecessarily, waiting for an installation
  process that would not proceed.

  This commit adjusts the logic so that if the snapshot is recognized as
  outdated, it immediately returns a None Condition, ensuring the
  caller is informed straightaway that no installation will occur.

v0.9.8

Summary:

• Fixed:
  • fd5c657f Chunked should reset offset 0 when a SnapshotMismatch error is received.

Detail:

Fixed:

• Fixed: fd5c657f Chunked should reset offset 0 when a SnapshotMismatch error is received; by 张炎泼; 2024-05-02

  When a SnapshotMismatch is received, the sending end should re-send
  all snapshot data from the beginning.

v0.9.7

Summary:

• Fixed:
  • ff4cf8fe openraft-0.9.6 requires openraft-macros-0.9.6 or newer.

Detail:

Fixed:

• Fixed: ff4cf8fe openraft-0.9.6 requires openraft-macros-0.9.6 or newer; by 张炎泼; 2024-04-25

v0.9.6

Summary:

• Added:
  • 5776139d Add #[since(version="1.0")] to specify first version of a feature.
  • b172dc8e Add macro expand!() to expand a template.
• Improved:
  • 99596c75 declare_raft_types allows the types in any order.
• Fixed:
  • 14d42e4f Load mebmership since last_applied, not last_membership.index on startup.

Detail:

Added:

• Added: 5776139d Add #[since(version="1.0")] to specify first version of a feature; by 张炎泼; 2024-04-12

  #[since(version = "1.0.0")] adds a doc line /// Since: Version(1.0.0).

  Example:

  /// Foo function
  ///
  /// Does something.
  #[since(version = "1.0.0")]
  fn foo() {}

  The above code will be transformed into:

  /// Foo function
  ///
  /// Does something.
  ///
  /// Since: Version(1.0.0)
  fn foo() {}

• Added: b172dc8e Add macro expand!() to expand a template; by 张炎泼; 2024-04-13

  expand!() renders a template with arguments multiple times.

  Example:

  expand!(KEYED, // ignore duplicate by `K`
          (K, T, V) => {let K: T = V;},
          (a, u64, 1),
          (a, u32, 2), // duplicate `a` will be ignored
          (c, Vec<u8>, vec![1,2])
  );

  The above code will be transformed into:

  let a: u64 = 1;
  let c: Vec<u8> = vec![1, 2];

Improved:

• Improved: 99596c75 declare_raft_types allows the types in any order; by 张炎泼; 2024-04-13

  By rewriting the template expanding part with a #[proc_macro]
expand!() defined in openraft_macros, declare_raft_types
  does not require the types in fixed order:

  Example:

  declare_raft_types!(All:
          D = (),
          NodeId = u64,
          R = (),
          Node = (),
  

Fixed:

• Fixed: 14d42e4f Load mebmership since last_applied, not last_membership.index on startup; by 张炎泼; 2024-04-25

  Modify StorageHelper::last_membership_in_log() to scan the log
  starting from the last applied index rather than the index of the last
  applied membership log. This change reduces unnecessary I/O operations
  during startup, previously caused by scanning from an incorrect starting
  point.

v0.9.5

Summary:

• Added:
  • e4fed706 Add Raft::is_initialized().
  • 3b18517a Add RaftTypeConfig::Responder to customize returning client write response.
  • c508a354 Raft::client_write_ff() ff for fire-and-forget.

Detail:

Added:

• Added: e4fed706 Add Raft::is_initialized(); by 张炎泼; 2024-04-08

  Raft::is_initialized() returns true if this raft node is already
  initialized with Raft::initialize(), by checking if log is empty and
  vote is not written.

• Added: 3b18517a Add RaftTypeConfig::Responder to customize returning client write response; by 张炎泼; 2024-04-03

  This commit introduces the Responder trait that defines the mechanism
  by which RaftCore sends responses back to the client after processing
  write requests. Applications can now customize response handling by
  implementing their own version of the RaftTypeConfig::Responder trait.

  The Responder::from_app_data(RaftTypeConfig::D) method is invoked to
  create a new Responder instance when a client write request is
  received.
  Once the write operation is completed within RaftCore,
  Responder::send(WriteResult) is called to dispatch the result
  back to the client.

  By default, RaftTypeConfig::Responder retains the existing
  functionality using a oneshot channel, ensuring backward compatibility.

  This change is non-breaking, requiring no modifications to existing
  applications.

  • Fix: #1068

• Added: c508a354 Raft::client_write_ff() ff for fire-and-forget; by 张炎泼; 2024-04-08

  Raft<C>::client_write_ff() -> C::Responder::Receiver submit a client
  request to Raft to update the state machine, returns an application
  defined response receiver Responder::Receiver to receive the response.

  _ff means fire and forget.

  It is same as [Raft::client_write] but does not wait for the response.
  When using this method, it is the application's responsibility for
  defining mechanism building and consuming the Responder::Receiver.

  • Part of #1068

Full Changelog: v0.9.4...v0.9.5

v0.9.4

Summary:

• Changed:
  • 2b8bc842 Add default value to declare_raft_types.
• Added:
  • 93c0d9ae Implement TryAsRef<ForwardToLeader<..>> for RaftError.

Detail:

Changed:

• Changed: 2b8bc842 Add default value to declare_raft_types;

  Types used in declare_raft_types can be omitted, in which case the default type will be used.
  The default values for each type are:

  • D: String
  • R: String
  • NodeId: u64
  • Node: ::openraft::BasicNode
  • Entry: ::openraft::Entry<Self>
  • SnapshotData: Cursor<Vec<u8>>
  • AsyncRuntime: ::openraft::TokioRuntime

  Note that The types must be specified in the exact order:
  D, R, NodeId, Node, Entry, SnapshotData, AsyncRuntime.

  For example, to declare with only D, R and Entry types:

  openraft::declare_raft_types!(
     pub TypeConfig:
         D = ClientRequest,
         R = ClientResponse,
         Entry = MyEntry,
  );

  Type NodeId, Node, SnapshotData and AsyncRuntime will be filled
  with default values mentioned above.

  Or one can just use the default types for all:

  openraft::declare_raft_types!(pub TypeConfig);

  Upgrade tip:

  Ensures types declared in declare_raft_types are in the correct order

Added:

• Added: 93c0d9ae Implement TryAsRef<ForwardToLeader<..>> for RaftError;

Full Changelog: v0.9.3...v0.9.4

Contributors

• @HaHa421

v0.9.2

• Added:
  • 8b54a80f Raft::config() returns a ref to Config this raft node uses.
  • 7d5326ce add RaftMetrics::millis_since_quorum_ack.
• Improved:
  • d4a3053f Update example network err implementation.
  • d2d155db Use Unreachable error in examples/raft-kv-rocksdb.
• Fixed:
  • 17795e7f Ensure heartbeat results are returned to RaftCore.
  • 768dfdc9 Ensure RaftMetrics are sent after RaftDataMetrics and RaftServerMetrics.

v0.9.0

Upgrade guide from 0.8 to 0.9:
https://docs.rs/openraft/latest/openraft/docs/upgrade_guide/upgrade_08_09/index.html

What's Changed

• Feature: define custom Entry type for raft log by @drmingdrmer in #720
• Fix: trait RaftLogId should be public by @drmingdrmer in #726
• Change: StoreBuilder does not need to run a test, it only needs to build a store by @drmingdrmer in #730
• Feature: add methods to StorageIOError to simplify error creation by @drmingdrmer in #738
• Change: remove Clone from trait AppData by @drmingdrmer in #740
• Change: instead of a slice, RaftStorage::append_to_log() now accepts an IntoIterator by @drmingdrmer in #742
• Improve: send AppendEntries response before committing entries by @drmingdrmer in #751
• Change: remove unused trait RaftStorageDebug by @drmingdrmer in #759
• Change: remove defensive check utilities by @drmingdrmer in #761
• Change: move RaftStateMachine out of RaftStorage by @drmingdrmer in #763
• Improve: move state machine operations to another task by @drmingdrmer in #767
• Fix: if the application does not persist snapshot, build a snapshot when starting up by @drmingdrmer in #768
• Improve: getting a snapshot does not block RaftCore task by @drmingdrmer in #773
• Fix: debug level check by @drmingdrmer in #780
• Improve: reduce rate to flush metrics by @drmingdrmer in #782
• Change: RaftMetrics.replication type to BTreeMap<NodeId, Option<LogId>> by @drmingdrmer in #785
• Feature: add backoff strategy for unreachable nodes by @drmingdrmer in #779
• Change: move snapshot type definition from storage traits to RaftTypeConfig by @drmingdrmer in #799
• Feature: add RaftMetrics::purged to report the last purged log id by @drmingdrmer in #809
• Feature: add Wait::purged() to wait for purged to become the expected by @drmingdrmer in #810
• Fix: replication should be able to shutdown when replicating snapshot to unreachable node by @drmingdrmer in #811
• Feature: add RaftMetrics.vote, Wait::vote() by @drmingdrmer in #815
• Improve: build snapshot in anohter task by @drmingdrmer in #816
• Feature: new RaftNetwork API with argument RCPOption by @drmingdrmer in #825
• Feature: RaftNetwork::send_append_entries() can return PartialSuccess by @drmingdrmer in #831
• Change: remove unused error CommittedAdvanceTooMany by @drmingdrmer in #838
• Fix: add_learner() should block forever by @drmingdrmer in #847
• Feature: leader lease by @drmingdrmer in #848
• Feature: add SnapshotPolicy::Never by @drmingdrmer in #854
• Feature: add Raft::purge_log() by @drmingdrmer in #855
• Change: add AsyncRuntime type parameter to RaftTypeConfig by @wvwwvwwv in #869
• Feature: add ChangeMembers::SetNodes by @drmingdrmer in #876
• Feature: add 'singlethreaded' raft mode by @wvwwvwwv in #878
• Fix: restore replication progress when a leader starts up by @drmingdrmer in #884
• Change: move external command trigger to dedicated Trigger struct by @drmingdrmer in #888
• Change: move runtime config API to dedicated RuntimeConfigHandle by @drmingdrmer in #889
• Change: RaftLogReaderExt::get_log_id() should not return last-purged-id by @drmingdrmer in #892
• Change: move get_log_state() from RaftLogReader to RaftStorage by @drmingdrmer in #893
• Feature: save committed log id by @drmingdrmer in #897
• Feature: permit follower log to revert to earlier state with --features loosen-follower-log-revert by @drmingdrmer in #903
• Feature: add feature flag "tracing-log" to emit log record for tracing event by @drmingdrmer in #908
• Fix: Do not report snapshot.last_log_id to metrics until snapshot is finished building/installing by @drmingdrmer in #913
• Fix: AsyncReadExt::read_buf() only reads at most 2MB per call by @drmingdrmer in #925
• Fix: End tick_loop() when the receiver is gone. by @schreter in #930
• Fix: avoid panic with "loosen-follower-log-revert" enabled by @drmingdrmer in #937
• Fix: For single-threaded execution, there is no need for Send/Sync bounds by @schreter in #934
• Change: remove N, LS, SM from Raft<C, N, LS, SM> by @drmingdrmer in #941
• Feature: add PayloadTooLarge error by @drmingdrmer in #951
• Feature: add Raft::access_raft_state() to access RaftState with a function by @drmingdrmer in #957
• Feature: Add new methods to openraft::metrics::Wait by @drmingdrmer in #960
• Feature: add Wait::voter_ids() and deprecate Wait::members() by @drmingdrmer in #961
• Feature: add Raft::ensure_linearizable() to ensure linearizable read by @drmingdrmer in #964
• Feature: add Wait::eq() and ge() to await a metics by @drmingdrmer in #967
• Change: #959 ditch async_trait for the singlethreade feature by @wvwwvwwv in #982
• Feature: add Raft::data_metrics() and Raft::server_metrics() by @YangKian in #990
• Fix: Split serde bound for RaftError into serialize and deserialize by @tvsfx in #991
• Feature: Add random number generator to AsyncRuntime by @schreter in #995
• Feature: add Raft::get_snapshot() to get the last snapshot from state machine by @drmingdrmer in #1000
• Fix: Remove implementation of Instant for std::time::Instant by @schreter in #1001
• Feature: add Raft::install_complete_snapshot() to install a snapshot by @drmingdrmer in #1002
• Feature: Add Raft::begin_receiving_snapshot() by @drmingdrmer in #1008
• Change: remove AsyncRuntime::abort() by @drmingdrmer in #1012
• Feature: RaftNetwork::snapshot() to send a complete snapshot by @drmingdrmer in #1009
• Feature: feature flag generic-snapshot-data by @drmingdrmer in #1016
• Change: remove deprecated RaftNetwork methods without option argument by @drmingdrmer in #1028
• Change: rename Raft::install_complete_snapshot() to install_full_snapshot() by @drmingdrmer in #1030
• Feature: AsyncRuntime::oneshot by @Miaxos in #1026
• Change: remove openraft-0.7 compatibility support by @drmingdrmer in #1033
• Feature: add trait RaftLogStorageExt to provide additional raft-log methods by @drmingdrmer in #1034
• Change: Raft::begin_receiving_snapshot() does not need to check Vote by @drmingdrmer in #1037
• Change: Raft::begin_receiving_snapshot() return only Fatal error. by @drmingdrmer in #1038
• Fix: install_snapshot() should return local vote, not request vote by @drmingdrmer in #1042

New Contributors

• @HHoflittlefish777 made their first contribution in #842
• @leiysky made their first contribution in #849
• @wvwwvwwv made their first contribution in #869
• @caicancai made their first contribution in #932
• @pfwang80s ma...