v0.6.1

Changelog

The original announcement is located here.

• [FEATURE] add metric to detect filesystems rules that don’t match any local dataset (thanks, @gmekicaxcient).
• [BUG] zrepl status: hide progress bar once all filesystems reach terminal state (thanks, @0x3333).
• [BUG] handling of tenative cursor presence if protection strategy doesn’t use it (issue #714).
• [DOCS] address setup with two or more external disks (thanks, @se-jaeger).
• [DOCS] document replication and conflict_resolution options (thanks, @InsanePrawn).
• [DOCS] docs: talks: add note on keep_bookmarks option (thanks, @skirmess).
• [MAINT] dist: add openrc service file (thanks, @gramosg).
• [MAINT] grafana: update dashboard to Grafana 9.3.6.
• [MAINT] run platform tests as part of CI.
• [MAINT] build: upgrade to Go 1.21 and update golangci-lint; minimum Go version for builds is now 1.20

Breaking Changes

No breaking changes. zrepl 0.6.0 is interoperable with zrepl 0.6.1.

New Users

We provide quick-start guides for different usage scenarios.
Also, we recommend studying the overview section of the configuration chapter to understand how zrepl replication works.

Testing & Upgrading

• Read the Changelog
• Download & run the platformtest binary on a test system.

  sudo ./platformtest-linux-amd64 \
    -imagepath /tmp/zreplplatformtest.img \
    -mountpoint /tmp/zreplplatformtest \
    -poolname zreplplatformtest
  

• Deploy the binary release / install the package.

Donations

zrepl is a spare-time project; express your support through a small dontation to keep maintenance and feature development going.
Thank you!

v0.6.0

Changelog

The original announcement is located here.

• [FEATURE] Schedule-based snapshotting using cron syntax instead of an interval.
• [FEATURE] Configurable initial replication policy. When a filesystem is first replicated to a receiver, this control whether just the newest snapshot will be replicated vs. all existing snapshots. Learn more in the docs.
• [FEATURE] Configurable timestamp format for snapshot names via timestamp_format (Thanks, @ydylla).
• [FEATURE] Add ZREPL_DESTROY_MAX_BATCH_SIZE env var (default 0=unlimited) (Thanks, @3nprob).
• [FEATURE] Add zrepl configcheck --skip-cert-check flag (Thanks, @cole-h).
• [BUG] Fix resuming from interrupted replications that use send.raw on unencrypted datasets.
  • The send options introduced in zrepl 0.4 allow users to specify additional zfs send flags for zrepl to use. Before this fix, when setting send.raw=true on a job that replicates unencrypted datasets, zrepl would not allow an interrupted replication to resume. The reason were overly cautious checks to support the send.encrypted option.
  • This bugfix removes these checks from the replication planner. This makes send.encrypted a sender-side-only concern, much like all other send.* flags.
  • However, this means that the zrepl status UI no longer indicates whether a replication step uses encrypted sends or not. The setting is still effective though.
• [BREAK] convert Prometheus metric zrepl_version_daemon to zrepl_start_time metric
  • The metric still reports the zrepl version in a label. But the metric value is now the Unix timestamp at the time the daemon was started. The Grafana dashboard in dist/grafana has been updated.
• [BUG] transient zrepl status error: Post "http://unix/status": EOF
• [BUG] don’t treat receive-side bookmarks as a replication conflict. This facilitates chaining of replication jobs. See issue #490.
• [BUG] workaround for Go/gRPC problem on Illumos where zrepl would crash when using the local transport type (issue #598).
• [BUG] fix active child tasks panic that cold occur during replication plannig (issue #193abbe)
• [BUG] zrepl status off-by-one error in display of completed step count (commit ce6701f)
• [BUG] Allow using day & week units for snapshotting.interval (commit ffb1d89)
• [DOCS] docs/overview improvements (Thanks, @jtagcat).
• [MAINT] Update to Go 1.19.

Breaking Changes

• The RPC protocol version was bumped; all zrepl daemons in a setup must be updated and restarted before replication can resume.

New Users

We provide quick-start guides for different usage scenarios.
Also, we recommend studying the overview section of the configuration chapter to understand how zrepl replication works.

Testing & Upgrading

• Read the Changelog
• Download & run the platformtest binary on a test system.

  sudo ./platformtest-linux-amd64 \
    -imagepath /tmp/zreplplatformtest.img \
    -mountpoint /tmp/zreplplatformtest \
    -poolname zreplplatformtest
  

• Deploy the binary release / install the package.

Donations

zrepl is a spare-time project but nonetheless requires significant development and maintenance effort.
Please consider donating to support future development. Thank you!

Binary Releases

The released binaries were produced with the following docker container:

sudo docker tag zrepl_release problame/zrepl_build:v0.6.0
sudo docker push problame/zrepl_build:v0.6.0
v0.6.0: digest: sha256:b14873f2bb754489cc54ae07b3fc1fcfaf9284f24269c2f92008cf81d39e17ea size: 4101

v0.5.0

Please check out the docs changelog for a full list of changes.

Highlights

• Bandwidth limiting (Thanks, Prominic.NET, Inc.)
• zrepl status: use a * to indicate which filesystem is currently replicating
• fix encrypt-on-receive + placeholders use case (issue #504)
  • Before this fix, plain sends to a receiver with an encrypted root_fs could be received unencrypted if zrepl needed to create placeholders on the receiver.
  • Existing zrepl users should read the docs and check zfs get -r encryption,zrepl:placeholder PATH_TO_ROOTFS on the receiver.
  • Thanks to @mologie and @razielgn for reporting and testing!

Breaking Changes

• The RPC protocol version was bumped; all zrepl daemons in a setup must be updated and restarted before replication can resume.
• Rename mis-spelled send option embbeded_data to embedded_data.

New Users

We provide quick-start guides for different usage scenarios.
Also, we recommend studying the overview section of the configuration chapter to understand how zrepl replication works.

Testing & Upgrading

• Read the Changelog
• Download & run the platformtest binary on a test system.

  sudo ./platformtest-linux-amd64 \
    -imagepath /tmp/zreplplatformtest.img \
    -mountpoint /tmp/zreplplatformtest \
    -poolname zreplplatformtest
  

• Deploy the binary release / install the package.

Platform Issues

The long-standing Go runtime bug on FreeBSD 12.2 has been fixed in FreeBSD 13, the patch will be part of 12.4, and there will be an ERRATA for 12.3.

Community

The chat room moved from IRC to Matrix.

Donations

zrepl is a spare-time project but nonetheless requires significant development and maintenance effort.
Please consider donating to support future development. Thank you!

Binary Releases

The released binaries were produced with the following docker container:

sudo docker tag zrepl_release problame/zrepl_build:v0.5.0
sudo docker push problame/zrepl_build:v0.5.0
v0.5.0: digest: sha256:9ffbb9b6a6a579c0c9d17779b6ba1e4f3960e3b3139761b29a6e89ad1ff0aa97 size: 4101

v0.5.0-rc1

Please check out the docs changelog for a full list of changes.

Highlights

• Bandwidth limiting (Thanks, Prominic.NET, Inc.)
• zrepl status: use a * to indicate which filesystem is currently replicating
• fix encrypt-on-receive + placeholders use case (issue #504)
  • Before this fix, plain sends to a receiver with an encrypted root_fs could be received unencrypted if zrepl needed to create placeholders on the receiver.
  • Existing zrepl users should read the docs and check zfs get -r encryption,zrepl:placeholder PATH_TO_ROOTFS on the receiver.
  • Thanks to @mologie and @razielgn for reporting and testing!

Breaking Changes

• Rename mis-spelled send option embbeded_data to embedded_data.

New Users

We provide two quick-start guides for different usage scenarios: server-to-server continuous backup, and workstation-to-external-disk backup.

We highly recommend studying the overview section of the configuration chapter to understand how zrepl replication works.

Testing & Upgrading

• Read the Changelog
• Download & run the platformtest binary on a test system.

  sudo ./platformtest-linux-amd64 \
    -imagepath /tmp/zreplplatformtest.img \
    -mountpoint /tmp/zreplplatformtest \
    -poolname zreplplatformtest
  

• Deploy the binary release / install the package.

Platform Issues

There is a Go runtime bug on FreeBSD 12.2 and newer that manifests in Go runtime crashes and/or lock-ups of the zrepl daemon.
zrepl has a tracking issue which contains the link to the Go upstream issue.
A workaround is available.

Donations

zrepl is a spare-time project but nonetheless requires significant development and maintenance effort.
Please consider donating to support future development. Thank you!

Binary Releases

Binaries & distroy packages were produced with the following docker container:

sudo docker tag zrepl_release problame/zrepl_build:v0.5.0-rc1
sudo docker push problame/zrepl_build:v0.5.0-rc1
v0.5.0-rc1: digest: sha256:7bc817cada324fd5cbaa12d51cf82ece380c5de81b09633f325de59deffdcb8c size: 4101

v0.4.0

Please check https://zrepl.github.io/v0.4.0/changelog.html for a full list of changes.

Highlights

• Support for zfs send / recv flags in the config (send: -wLcepbS , recv: -ox ). (docs).
• Parallel replication is now configurable (docs)
• New zrepl status UI:
  • Interactive job selection.
  • Interactively zrepl signal jobs.
  • Filter filesystems in the job view by name.
  • An approximation of the old UI is still included as –mode legacy but will be removed in a future release of zrepl.

Breaking Changes

This release does not contain breaking changes.
However, users who skipped the 0.3.1 update should make sure that their pruning grid config is correct.
A bugfix in 0.3.1 caused unexpected snapshot deletions for some users.

New Users

We provide two quick-start guides for different usage scenarios: server-to-server continuous backup, and workstation-to-external-disk backup.

We highly recommend studying the overview section of the configuration chapter to understand how zrepl replication works.

Testing & Upgrading

• Read the Changelog
• Download & run the platformtest binary on a test system.

  sudo ./platformtest-linux-amd64 \
    -imagepath /tmp/zreplplatformtest.img \
    -mountpoint /tmp/zreplplatformtest \
    -poolname zreplplatformtest
  

• Deploy the binary release / install the package.

Platform Issues

There is a Go runtime bug on FreeBSD 12.2 and newer that manifests in Go runtime crashes and/or lock-ups of the zrepl daemon. zrepl has a tracking issue which contains the link to the Go upstream issue. A workaround is available. Note that this issue also occurs with zrepl 0.3 and many Go versions - it was not introduced by the changes in this release.

Donations

zrepl is a spare-time project but nonetheless requires significant development and maintenance effort.
Please consider donating to support future development. Thank you!

Binary Releases

Binaries & distroy packages were produced with the following docker container:

sudo docker tag zrepl_release problame/zrepl_build:v0.4.0
sudo docker push problame/zrepl_build:v0.4.0
v0.4.0: digest: sha256:62ce339af333f30e00a47cfc4cda6325c3d13a345425d3ba127f684d56a8d0ad size: 4099

v0.4.0-rc2

Changes since the last release candidate:

git shortlog v0.4.0-rc1..v0.4.0-rc2
Christian Schwarz (1):
      pruning/keep_last_n: correctly handle the case where `count` > matching snaps

Cole Helbling (1):
      client/status: allow raw mode without a tty

InsanePrawn (2):
      client/signal: Revert "add signal 'snapshot', rename existing signal 'wakeup' to 'replication'"
      status/interactive: Revert to simple wakeup/reset signalling

Lukas Schauer (1):
      zfs: pipe size: default to value of /proc/sys/fs/pipe-max-siz

• The zrepl signal snapshot functionality was reverted because of the UX issues reported in #451. A more generic solution for activating and resetting the different activities within a job will be released in zrepl 0.5.
• The pipe size change to the zfs package is a work-around for the Linux kernel bug that causes zfs recv to hang.

Binary Releases

Binaries & distro packages were produced with the following docker container:

docker tag zrepl_release problame/zrepl_build:v0.4.0-rc2
docker push problame/zrepl_build:v0.4.0-rc2
v0.4.0-rc2: digest: sha256:8e57281d1e6dda28e91fbdc402ac08200fea8714b15de13d83ccb67cf139c71f size: 4100

Donations

zrepl is a spare-time project but nonetheless requires significant development and maintenance effort.
Please consider donating to support future development. Thank you!

v0.4.0-rc1

Please check https://zrepl.github.io/v0.4.0-rc1/changelog.html for a full list of changes.

Highlights

• Support for zfs send / recv flags in the config (send: -wLcepbS , recv: -ox ).
• Parallel replication is now configurable.
• New zrepl status UI:
  • Interactive job selection.
  • Interactively zrepl signal jobs.
  • Filter filesystems in the job view by name.
  • An approximation of the old UI is still included as –mode legacy but will be removed in a future release of zrepl.

Breaking Changes

• New syntax to trigger a job replication: renamed zrepl signal wakeup JOB to zrepl signal replication JOB
• For users who skipped the 0.3.1 update: please make sure your pruning grid config is correct.
  The grid bugfix caused unexpected snapshot deletions for some users.

New Users

• We provide two quick-start guides for different usage scenarios: server-to-server continuous backup, and workstation-to-external-disk backup.

We highly recommend studying the overview section of the configuration chapter to understand how zrepl replication works.

Testing & Upgrading

There are no protocol or backwards-incompatible config format changes.
The only breaking change is the rename of the wakeup signal to replication.
Recommendations for updating:

• Read the Changelog
• Download & run the platformtest binary published with this release.

  sudo ./platformtest-linux-amd64 \
    -imagepath /tmp/zreplplatformtest.img \
    -mountpoint /tmp/zreplplatformtest \
    -poolname zreplplatformtest
  

• Deploy the binary release / install the package.

Note 1: There is a bug in Linux 5.8 and newer that sometimes leaves zfs recv unresponsive. A workaround is being investigated in #424 . Note that this issue also occurs with zrepl 0.3 and many Go versions - it was not introduced by the changes included in this release.

Note 2: There is a Go runtime bug on FreeBSD 12.2 and newer that manifests in Go runtime crashes and/or lock-ups of the zrepl daemon. zrepl has a tracking issue which contains the link to the Go upstream issue. A workaround is available. Note that this issue also occurs with zrepl 0.3 and many Go versions - it was not introduced by the changes included in this release.

Donations

zrepl is a spare-time project but nonetheless requires significant development and maintenance effort.
Please consider donating to support future development. Thank you!

Binary Releases

Binaries & distroy packages were produced with the following docker container:

sudo docker tag zrepl_release problame/zrepl_build:v0.4.0-rc1
sudo docker push problame/zrepl_build:v0.4.0-rc1      
v0.4.0-rc1: digest: sha256:8e57281d1e6dda28e91fbdc402ac08200fea8714b15de13d83ccb67cf139c71f size: 4100

v0.3.1

Please check https://zrepl.github.io/stable/changelog.html for a full list of changes.

Mostly a bugfix release for zrepl 0.3.

• pruning: add optional regex field to last_n rule
• pruning: grid : improve documentation and add an example
• pruning: grid: add all snapshots that do not match the regex to the rule's destroy list.
  This brings the implementation in line with the docs.
• easyrsa script in docs
• platformtest: fix skipping encryption-only tests on systems that don't support encryption
• replication: report AttemptDone if no filesystems are replicated
• status + replication: warning if replication succeeeded without any filesystem being replicated
• update multi-job & multi-host setup section
• RPM Packaging
• CI infrastructure rework
• Continuous deployment of that new stable branch to zrepl.github.io.

Binaries and Packages

• All assets attached to this release were created by the zrepl CI system.
• DEB and RPM repositories will be updated shortly.

Donations

zrepl is a spare-time project but nonetheless requires significant development and maintenance effort.
Please consider donating to support future development. Thank you!

v0.3.0

Please check https://zrepl.github.io/stable/changelog.html for a full list of changes.

Highlights

• Resumable Send & Recv Support No knobs required, automatically used where supported.
• Encrypted Send & Recv Support for OpenZFS native encryption, configurable at the job level, i.e., for all filesystems a job is responsible for.
• Replication Guarantees Automatic use of ZFS holds and bookmarks to protect a replicated filesystem from losing synchronization between sender and receiver. By default, zrepl guarantees that incremental replication will always be possible and interrupted steps will always be resumable.

We highly recommend studying the overview section of the configuration chapter to understand how zrepl replication works.

Important Note: Go 1.15 changed the default TLS validation policy to require Subject Alternative Names (SAN) in certificates.
The self-signed certs generated with the openssl commands that we provided in prior versions of the zrepl docs will most likely not work properly.
If you encounter certificate validation errors regarding SAN and wish to continue to use your old certificates, start the zrepl daemon with env var GODEBUG=x509ignoreCN=0.
Alternatively, generate new certificates with SANs (see both options int the TLS transport docs ).

New Users

• We provide two quick-start guides for different usage scenarios: server-to-server continuous backup, and workstation-to-external-disk backup.

Upgrading

This release has been in development and testing for more than half a year now.
The code for the major features has been running on our developers' production systems as well as several test systems for at least two months.
In order to avoid surprises when updating your systems, we recommend the following steps:

• Check that your TLS certificates contain Subject Alternative Name fields or re-generate your TLS certs
• Read the Changelog
• Study the overview section of the configuration chapter to understand how zrepl replication works.
• Download & run the platformtest binary published with this release.

  sudo ./platformtest-linux-amd64 \
    -imagepath /tmp/zreplplatformtest.img \
    -mountpoint /tmp/zreplplatformtest \
    -poolname zreplplatformtest
  

  The platform tests make sure that zrepl's assumptions about your version of OpenZFS will hold at runtime.
  • Post-release, we identified in #358 that the integration tests for incremental replication are flaky if the zfs commands run very fast (e.g. if no other pool is imported). The failures do not affect the zrepl daemon's operation and can be ignored if they are the same as reported in #358.
• The config format was changed in a backward-compatible way except for more restrictions on job names.
  It shouldn't be necessary but if you want, download the 0.3 binary and runzrepl-v0.3.0 configcheck.
  Exit status 0 and no output is good news!
• Deploy the update to all of your systems at once and restart all daemons.
  (The RPC protocol version was bumped, zrepl 0.3 will not communicate with earlier versions)
• Let replication run for a few weeks.
• Then run the zrepl migration command mentioned in the changelog.

Testing

If you want to test zrepl 0.3 more thoroughly before deploying it to production, we recommend stressing it in the following ways:

• Try the replication cursor migration mentioned in the update..
• Play through the two quick-start guides.
• Play with encrypted replication, in particular initial replication with many datasets.
  • After initial replication of all filesystems succeeded, extend the filesystems filter to include another filesystem that is a child dataset of an alread-replicated filesystem. Kick-off replication again and see what happens and whether it meets your expected behavior.
• Try to induce errors during replication. zrepl guarantees that sender and receiver never get out-of-sync by network disruption, unplugging of the remote pool, etc. Out-of-sync means that incremental replication is no longer possible.
• Run the platformtests on your platform. Read the Makefile to learn how to do that.

Please report any unexpected behavior as an issue, and do not forget to mention that you are running the release-candidate.

Donations

zrepl is a spare-time project but nonetheless requires significant development and maintenance effort.
Please consider donating to support future development. Thank you!

Binary Releases

Binaries were produced with the following docker container:

sudo docker push problame/zrepl_build:v0.3.0
v0.3.0: digest: sha256:a42d3f02e895048ac176382538d8510af5edbc22a09dadbef1476f6a02a1ab75 size: 4098

Debian Packages

The Debian repositories will be updated shortly have been updated, using the same binaries attached to this release.

Changes since -rc2

• e239d6f build: make platformtest-* usable
• 0bbe2be docs: prune: add prune interval visualisation
• fa4e048 readme: fix typo
• 4f9f21f logger: fix go-1.15-discovered conversion from int to string
• 480176b rpc/dataconn: fix go1.15-discovered recursive Error() method impl
• 1190c0f docs: supporters: update
• 720a284 dist/grafana: fix endpoint abstractions cache metric panel
• 83fdffb replication: prometheus metric for number of failed replications in last attempt

v0.3.0-rc2

Please check https://zrepl.github.io/v0.3.0-rc2/changelog.html for a full list of changes.

Highlights

• Resumable Send & Recv Support No knobs required, automatically used where supported.
• Encrypted Send & Recv Support for OpenZFS native encryption, configurable at the job level, i.e., for all filesystems a job is responsible for.
• Replication Guarantees Automatic use of ZFS holds and bookmarks to protect a replicated filesystem from losing synchronization between sender and receiver. By default, zrepl guarantees that incremental replication will always be possible and interrupted steps will always be resumable.

We highly recommend studying the overview section of the configuration chapter to understand how zrepl replication works.

Changes since -rc1

• 0ee7a49 [#289] zfs: workaround for OpenZFS 0.7 dry send info with zero estimated size
• 02db599 [#345] fix broken identification of parent-fs for initial replication ordering
• 1d7a84e build: extract debian binary packaging workflow trigger into a reusable script
• a0e3dc7 [#348] replication: add platformtest to check behavior on recv fail while still sending
• 43495d7 [#348] replication: return errors if .Send rpc returns a nil SendRes
• fa586b4 [#348] fix crash on early-recv error
• 30cdc14 replication + endpoint: replication guarantees: guarantee_{resumability,incremental,nothing}
  • This is perhaps the most significant change, but as long as you didn't set the send.step_holds.disable_incremental option (which only existed in -rc1), this doesn't affect you at all.
• 27673a2 config: add test for fromdefaults behavior
• 95fc299 daemon/job: test that sample configs are buildable
• 4e702ee cmd: zfs-abstraction list --json: fix panic
• 8ff83f2 [#342] endpoint: always create unencrypted placeholder filesystems
• 4b8f0ad docs: supporters: update
• dbc8bbe docs: config: prune: example: keep manual snapshots on receiver
• b3e856f docs: changelog: 0.3: fix broken issue link

Testing This Release-Candidate

We believe that this release-candidate is already pretty solid as it has been deployed to our developers' private test (and some production) systems for several weeks now.
However, a fresh eye (or system) might help catch some issues that were overlooked.

If you want to help test this release-candidate, please read the 0.3 changelog and attempt the update.

• Try the replication cursor migration mentioned in the update.
• Config syntax should be comptabile except for some job names.
• Play through the two quick-start guides.
• Play with encrypted replication, in particular initial replication with many datasets.
  • After initial replication of all filesystems succeeded, extend the filesystems filter to include another filesystem that is a child dataset of an alread-replicated filesystem. Kick-off replication again and see what happens and whether it meets your expected behavior.
• Try to induce errors during replication. zrepl guarantees that sender and receiver never get out-of-sync by network disruption, unplugging of the remote pool, etc. Out-of-sync means that incremental replication is no longer possible.
• Run the platformtests on your platform. Read the Makefile to learn how to do that.

Please report any unexpected behavior as an issue, and do not forget to mention that you are running the release-candidate.

Donations

zrepl is a spare-time project but nonetheless requires significant development and maintenance effort.
Please consider donating to support future development. Thank you!

Binary Releases

Binaries were produced with the following docker container:

sudo docker push problame/zrepl_build:v0.3.0-rc2
v0.3.0-rc2: digest: sha256:db9daf176828d3e63ef88d47e93f08705ac5975a42811e7abf7a877f6f519902 size: 4099

Debian packages built by CI from CI binaries (no signatures): https://github.com/zrepl/debian-binary-packaging/runs/912209864?check_suite_focus=true