Khepri 0.14.0

At this point, Khepri should be considered Beta and not production ready. The API will likely evolve and future releases will likely introduce breaking changes. The goal of this release is to make it easy for anyone to try it and possibly give feedback.

What's new in Khepri 0.14.0

Command deduplication

Khepri uses a retry mechanism that send a command again whenever there is an error such as noproc or nodedown. However, the state machine could receive the command twice depending the actual underlying error and the timing.

This new version adds two new commands to wrap other commands that act as a kind of handshake between the "client" and the state machine. This new mechanism is only enabled by default for R/W transactions.

The version of the state machine is bumped from 0 to 1. The code is backward-compatible with Ra cluster members which don't know about these new commands.

See #250.

Simplify {error, timeout} error type

The {error, {timeout, LeaderId}} error was simplified to the more common {error, timeout}, because the leader ID isn't interesting to the caller.

Here is an example of the change to make to your code:

• Up to Khepri 0.13.x:

  case khepri:exists(StoreId, Path) of
      {error, {timeout, _LeaderId}} ->
          handle_timeout()
  end

• Start from Khepri 0.14.0:

  case khepri:exists(StoreId, Path) of
      {error, timeout} ->
          handle_timeout()
  end

Seee #256.

Other changes

• Fixed a bug where a projection could be executed for a tree node that doesn't match the pattern if this tree is a grand-child of the pattern (#257).
• Ra was bumped from 2.9.1 to 2.10.1 (#258).

Download

Khepri 0.14.0 is available from Hex.pm: https://hex.pm/packages/khepri/0.14.0

Upgrade

Using Rebar:

1. Update your rebar.config:

   %% In rebar.config
   {deps, [{khepri, "0.14.0"}]}.

2. Run rebar3 upgrade khepri.

Using Erlang.mk:

1. Update your Makefile:

   %% In your Makefile
   dep_khepri = hex 0.14.0

2. Remove the deps/khepri directory. The new version will be fetched the next time you build your project.

Documentation

The documentation for Khepri 0.14.0 is available on Hex.pm.

Khepri 0.13.0

At this point, Khepri should be considered Beta and not production ready. The API will likely evolve and future releases will likely introduce breaking changes. The goal of this release is to make it easy for anyone to try it and possibly give feedback.

What's new in Khepri 0.13.0

Preliminary support for Erlang/OTP 27

With the upgrade of Horus from 0.2.4 to 0.2.5, Khepri gains preliminary support for Erlang/OTP 27. We say preliminary because Erlang/OTP 27 has not been released yet, so we can't guarantee at that point that it will be fully compatible with the final version.

See #254.

Other changes

• Fixed a bug where projections could be out-of-sync with the actual database tree content because of the way they were initialized on Ra server recovery (#255).
• Made the #khepri_machine{} record private and opaque (#252, #253).

Download

Khepri 0.13.0 is available from Hex.pm: https://hex.pm/packages/khepri/0.13.0

Upgrade

Using Rebar:

1. Update your rebar.config:

   %% In rebar.config
   {deps, [{khepri, "0.13.0"}]}.

2. Run rebar3 upgrade khepri.

Using Erlang.mk:

1. Update your Makefile:

   %% In your Makefile
   dep_khepri = hex 0.13.0

2. Remove the deps/khepri directory. The new version will be fetched the next time you build your project.

Documentation

The documentation for Khepri 0.13.0 is available on Hex.pm.

Khepri 0.12.1

At this point, Khepri should be considered Beta and not production ready. The API will likely evolve and future releases will likely introduce breaking changes. The goal of this release is to make it easy for anyone to try it and possibly give feedback.

What's new in Khepri 0.12.1

Handling of unknown commands

Now, if a Khepri state machine receives an unknown command, it will throw an exception. This will be useful in the future in case of an upgrade of the state machine code if new commands are introduced. If the new commands are submitted before the Ra cluster upgraded the state machine version, then the caller will be notified the command couldn't be applied.

See #251.

Download

Khepri 0.12.1 is available from Hex.pm: https://hex.pm/packages/khepri/0.12.1

Upgrade

Using Rebar:

1. Update your rebar.config:

   %% In rebar.config
   {deps, [{khepri, "0.12.1"}]}.

2. Run rebar3 upgrade khepri.

Using Erlang.mk:

1. Update your Makefile:

   %% In your Makefile
   dep_khepri = hex 0.12.1

2. Remove the deps/khepri directory. The new version will be fetched the next time you build your project.

Documentation

The documentation for Khepri 0.12.1 is available on Hex.pm.

Khepri 0.12.0

At this point, Khepri should be considered Beta and not production ready. The API will likely evolve and future releases will likely introduce breaking changes. The goal of this release is to make it easy for anyone to try it and possibly give feedback.

What's new in Khepri 0.12.0

Ra 2.9.1

This new release of Ra comes with an important bug fix to the consistency of the list of members/nodes returned by khepri_cluster:members/{0,1,2} and khepri_cluster:nodes/{0,1,2}. Before this change, it was possible that the returned list was out-of-date and stale.

See rabbitmq/ra#417 and #249.

Other changes

• Fixed unregistering of non-existent projections (#248).

Download

Khepri 0.12.0 is available from Hex.pm: https://hex.pm/packages/khepri/0.12.0

Upgrade

Using Rebar:

1. Update your rebar.config:

   %% In rebar.config
   {deps, [{khepri, "0.12.0"}]}.

2. Run rebar3 upgrade khepri.

Using Erlang.mk:

1. Update your Makefile:

   %% In your Makefile
   dep_khepri = hex 0.12.0

2. Remove the deps/khepri directory. The new version will be fetched the next time you build your project.

Documentation

The documentation for Khepri 0.12.0 is available on Hex.pm.

Khepri 0.11.0

At this point, Khepri should be considered Beta and not production ready. The API will likely evolve and future releases will likely introduce breaking changes. The goal of this release is to make it easy for anyone to try it and possibly give feedback.

What's new in Khepri 0.11.0

Return {ok, _} | {error, _} from functions returning cluster members

Always returning a list, hiding any error behind an empty list, proved to be a bad design, as it made it way too easy to ignore or miss errors and act upon them.

This kind of return value was also inconsistent with the rest of the API.

The members list is now wrapped into an {ok, Members} tuple. Likewise for nodes lists. This forces the caller to be aware of errors and handle them. The afftected APIs are:

• khepri_cluster:members/{1,2}
• khepri_cluster:locally_known_members/{1,2}
• khepri_cluster:nodes/{1,2}
• khepri_cluster:locally_known_nodes/{1,2}

Here is an example of the change to make to your code:

• Up-to Khepri 0.10.x:

  lists:foreach(
      fun(Node) ->
          do_something_with_node(Node)
      end,
      khepri_cluster:nodes(StoreId)).

• Starting from Khepri 0.11.0:

  case khepri_cluster:nodes(StoreId) of
      {ok, Nodes} ->
          lists:foreach(
              fun(Node) ->
                  do_something_with_node(Node)
              end,
              Nodes);
      {error, _Reason} = Error ->
          handle_error(Error)
  end.

While working on this:

• khepri_cluster:members/0, khepri_cluster:locally_known_members/0, khepri_cluster:nodes/0 and khepri_cluster:locally_known_nodes/0 were added to work on the default store. This is inline with all other API functions.
• Type specifications and documentation was added for all these functions, existing and new.

See #244.

Speed up members/nodes local query

Ra now provides a new ra_leaderboard:lookup_members/1 function to query members locally that is faster that a call to the local Ra server. Khepri started to use it and, if it returns an error, it will fall back to the previous Ra server query.

See #240.

Other changes

• Use monotonic times to mesaure a duration, not system time (#245).
• Do not enforce the favor option when querying the "keep while" conditions or the projections states (#246).
• Ra was bumped from 2.7.0 to 2.7.3 (#241).

Download

Khepri 0.11.0 is available from Hex.pm: https://hex.pm/packages/khepri/0.11.0

Upgrade

Using Rebar:

1. Update your rebar.config:

   %% In rebar.config
   {deps, [{khepri, "0.11.0"}]}.

2. Run rebar3 upgrade khepri.

Using Erlang.mk:

1. Update your Makefile:

   %% In your Makefile
   dep_khepri = hex 0.11.0

2. Remove the deps/khepri directory. The new version will be fetched the next time you build your project.

Documentation

The documentation for Khepri 0.11.0 is available on Hex.pm.

Khepri 0.10.1

At this point, Khepri should be considered Beta and not production ready. The API will likely evolve and future releases will likely introduce breaking changes. The goal of this release is to make it easy for anyone to try it and possibly give feedback.

What's new in Khepri 0.10.1

This release fixes the list of files published to Hex.pm: the LICENSE-* files were missing.

See #237.

Download

Khepri 0.10.1 is available from Hex.pm: https://hex.pm/packages/khepri/0.10.1

Upgrade

Using Rebar:

1. Update your rebar.config:

   %% In rebar.config
   {deps, [{khepri, "0.10.1"}]}.

2. Run rebar3 upgrade khepri.

Using Erlang.mk:

1. Update your Makefile:

   %% In your Makefile
   dep_khepri = hex 0.10.1

2. Remove the deps/khepri directory. The new version will be fetched the next time you build your project.

Documentation

The documentation for Khepri 0.10.1 is available on Hex.pm.

Khepri 0.10.0

At this point, Khepri should be considered Beta and not production ready. The API will likely evolve and future releases will likely introduce breaking changes. The goal of this release is to make it easy for anyone to try it and possibly give feedback.

What's new in Khepri 0.10.0

This release brings a single API change around the async operations.

Unify khepri:handle_async_ret/2 return values

The new khepri:handle_async_ret/2 introduced in Khepri 0.9.0 is improved in this new release: it converts the "raw" return values contained in the ra_event to a format similar to all other Khepri sync commands. This allows the function to handle specific cases like exceptions and aborted transactions.

Here is an example of the change to make to your code:

• Up-to Khepri 0.9.x:

  receive
      {ra_event, _,
       {applied, [{CorrelationId, {ok, _} = Ret}]}} = AsyncRet ->
          %% Call `handle_async_ret/2' for every entries;
          %% here only one:
          ok = khepri:handle_async_ret(StoreId, AsyncRet),
  
          %% Do something with each `Ret'.
          Ret
  end

• Starting from Khepri 0.10.0:

  receive
      {ra_event, _, _} = AsyncRet ->
          %% Call `handle_async_ret/2' once and get a list of correlation
          %% ID/return value pairs.
          [{CorrelationId, Ret}] = khepri:handle_async_ret(StoreId, AsyncRet),
  
          %% Do something with each `Ret'.
          Ret
  end

See #234.

Download

Khepri 0.10.0 is available from Hex.pm: https://hex.pm/packages/khepri/0.10.0

Upgrade

Using Rebar:

1. Update your rebar.config:

   %% In rebar.config
   {deps, [{khepri, "0.10.0"}]}.

2. Run rebar3 upgrade khepri.

Using Erlang.mk:

1. Update your Makefile:

   %% In your Makefile
   dep_khepri = hex 0.10.0

2. Remove the deps/khepri directory. The new version will be fetched the next time you build your project.

Documentation

The documentation for Khepri 0.10.0 is available on Hex.pm.

Khepri 0.9.0

At this point, Khepri should be considered Beta and not production ready. The API will likely evolve and future releases will likely introduce breaking changes. The goal of this release is to make it easy for anyone to try it and possibly give feedback.

What's new in Khepri 0.9.0

This release focuses on a couple improvements and bug fixes to async commands.

Improvements to async commands

• Correctly use the cached leader ID if it is present to send the command (#228).
• Replace khepri:wait_for_async_ret/{1,2} with khepri:handle_async_ret/{1,2} (#229). The previous API didn't match the underlying Ra API. It's better for the caller to receive the message(s) from the Ra server, then call khepri:handle_async_ret/{1,2} to let Khepri do internal things such as updating the cached leader ID.

Other changes

• Expose the ?IS_KHEPRI_STORE_ID/1 macro (#231).
• Ra was bumped from 2.6.3 to 2.7.0 (#232).

Download

Khepri 0.9.0 is available from Hex.pm: https://hex.pm/packages/khepri/0.9.0

Upgrade

Using Rebar:

1. Update your rebar.config:

   %% In rebar.config
   {deps, [{khepri, "0.9.0"}]}.

2. Run rebar3 upgrade khepri.

Using Erlang.mk:

1. Update your Makefile:

   %% In your Makefile
   dep_khepri = hex 0.9.0

2. Remove the deps/khepri directory. The new version will be fetched the next time you build your project.

Documentation

The documentation for Khepri 0.9.0 is available on Hex.pm.

Khepri 0.8.0

At this point, Khepri should be considered Beta and not production ready. The API will likely evolve and future releases will likely introduce breaking changes. The goal of this release is to make it easy for anyone to try it and possibly give feedback.

What's new in Khepri 0.8.0

This release focuses on several important bug fixes and improvements to projections.

Many improvements to projections

Projections are a new mechanism introduced in Khepri 0.6.0 to allow fast queries with a low latency.

Here are five pull requests related to projections:

• Add a khepri:unregister_projection() function to unregister projections (#220).
• Make "copy"-style projections more efficient and straightforward to use (#204).
• Handle errors from ETS gracefullly (#205).
• Allow to pass options to Horus (#209).

Other changes

• New khepri:is_empty/1 public function (#210).
• khepri_cluster:join/3 is now idempotent (#219).
• Fix some corner cases in khepri_cluster:join/3 (#225).
• Handle missing default applications when determining the list of modules to skip when extracting a transaction function (#215).
• Handle Ra server shutdown gracefully (#211, #223).
• Improve logging for member queries (#208, #216).
• Fix documentation of put() (#206).
• Ra was bumped from 2.5.1 to 2.6.3 (#224).
• Horus was bumped from 0.2.3 to 0.2.4 (#224).

Download

Khepri 0.8.0 is available from Hex.pm: https://hex.pm/packages/khepri/0.8.0

Upgrade

Using Rebar:

1. Update your rebar.config:

   %% In rebar.config
   {deps, [{khepri, "0.8.0"}]}.

2. Run rebar3 upgrade khepri.

Using Erlang.mk:

1. Update your Makefile:

   %% In your Makefile
   dep_khepri = hex 0.8.0

2. Remove the deps/khepri directory. The new version will be fetched the next time you build your project.

Documentation

The documentation for Khepri 0.8.0 is available on Hex.pm.

Khepri 0.7.0

At this point, Khepri should be considered Beta and not production ready. The API will likely evolve and future releases will likely introduce breaking changes. The goal of this release is to make it easy for anyone to try it and possibly give feedback.

What's new in Khepri 0.7.0

This release focuses on several important bug fixes and improvements to projections.

We promoted this release from Alpha to Beta as we believe the API is a lot more stable now. We are not ready to commit to this API yet though and may introduce more breaking changes if the need arises.

Many improvements to projections

Projections are a new mechanism introduced in Khepri 0.6.0 to allow fast queries with a low latency.

Here are five pull requests related to projects:

• The deletion of an entire subtree is reflected in projections (#178).
• Projections' ETS tables are correctly recreated when a Ra server is restarted and enters the recovered state (#182, #193)).
• Query speed improvements (#188).
• Reduce memory footprint (#190).

Creation of the Horus library

The previous khepri_fun module was extracted from Khepri and the Horus library was created out of it. It makes this code easier to maintain and test, and it can be reused outside of Khepri.

See #192.

Split of the khepri_machine code into several modules

The khepri_machine module was really large because it hosted the entire implementation of the internal tree data structure. This module was refactored and split into multiple modules to improve the readability of the code and its maintenance.

See #180, #181, #185, #186, #187, #189 and #194.

Other changes

• Documentation improvements (#173).
• Supervision tree fixes (#174).
• New khepri_cluster:is_store_running/1 helper (#175).
• Work around Rebar's use of cth_readable parse_transform (#176).
• Support for new Erlang/OTP 26 assembler instructions (#183).
• Ra was bumped from 2.4.0 to 2.5.1 (#191, #199).

Download

Khepri 0.7.0 is available from Hex.pm: https://hex.pm/packages/khepri/0.7.0

Upgrade

Using Rebar:

1. Update your rebar.config:

   %% In rebar.config
   {deps, [{khepri, "0.7.0"}]}.

2. Run rebar3 upgrade khepri.

Using Erlang.mk:

1. Update your Makefile:

   %% In your Makefile
   dep_khepri = hex 0.7.0

2. Remove the deps/khepri directory. The new version will be fetched the next time you build your project.

Documentation

The documentation for Khepri 0.7.0 is available on Hex.pm.