Skip to content

Retry randomly failing rspec example. This is an extension to the original gem, that will log flakey tests to an output file.

License

Notifications You must be signed in to change notification settings

DFE-Digital/rspec-retry

Repository files navigation

RSpec::Retry

RSpec::Retry adds a :retry option for intermittently failing rspec examples. If an example has the :retry option, rspec will retry the example the specified number of times until the example succeeds.

Compatibility

Rspec Version Rspec-Retry Version
> 3.8 0.6.2 but untested
> 3.3, <= 3.8 0.6.2            
3.2       0.4.6            
2.14.8       0.4.4            

Installation

Add this line to your application's Gemfile:

gem 'rspec-retry', group: :test # Unlike rspec, this doesn't need to be included in development group

And then execute:

$ bundle

Or install it yourself as:

$ gem install rspec-retry

require in spec_helper.rb

# spec/spec_helper.rb
require 'rspec/retry'
require 'rspec/core/formatters/base_text_formatter'

RSpec.configure do |config|
  # (Optional) show retry status in spec process
  config.verbose_retry = true
  # (Optional) show exception that triggers a retry if verbose_retry is set to true
  config.display_try_failure_messages = true
  # (Optional) configure a reporter & formatter for tests which fail then pass in the alloted number of retries (a.k.a. flakey tests).
  reporter = RSpec::Core::Reporter.new(config)
  formatter = RSpec::Core::Formatters::BaseTextFormatter.new(File.open('tmp/rspec-retry-flakey-specs.log', 'ab')
  reporter.register_listener(formatter), 'message')
  config.retry_reporter = reporter

  # run retry 3 times on all specs
  config.around do |ex|
    ex.run_with_retry retry: 3
  end

  # callback to be run between retries
  config.retry_callback = proc do |ex|
    # run some additional clean up task - can be filtered by example metadata
    if ex.metadata[:js]
      Capybara.reset!
    end
  end
end

Usage

it 'should randomly succeed', :retry => 3 do
  expect(rand(2)).to eq(1)
end

it 'should succeed after a while', :retry => 3, :retry_wait => 10 do
  expect(command('service myservice status')).to eq('started')
end
# run spec (following log is shown if verbose_retry options is true)
# RSpec::Retry: 2nd try ./spec/lib/random_spec.rb:49
# RSpec::Retry: 3rd try ./spec/lib/random_spec.rb:49

Calling run_with_retry programmatically

You can call ex.run_with_retry(opts) on an individual example.

Configuration

  • :verbose_retry(default: false) Print retry status
  • :display_try_failure_messages (default: false) If verbose retry is enabled, print what reason forced the retry
  • :default_retry_count(default: 1) If retry count is not set in an example, this value is used by default. Note: If this is changed from the default of 0, all examples will be retried.
  • :default_sleep_interval(default: 0) Seconds to wait between retries
  • :clear_lets_on_failure(default: true) Clear memoized values for lets before retrying
  • :exceptions_to_hard_fail(default: []) List of exceptions that will trigger an immediate test failure without retry. Takes precedence over :exceptions_to_retry
  • :exceptions_to_retry(default: []) List of exceptions that will trigger a retry (when empty, all exceptions will)
  • :retry_callback(default: nil) Callback function to be called between retries
  • :retry_reporter(default: nil) Reporter which logs comma delimited data for indeterminate specs in the format: [attempts, max_retries, path_and_line_number_of_failed_spec, error_messages]

Environment Variables

  • RSPEC_RETRY_RETRY_COUNT can override the retry counts even if a retry count is set in an example or default_retry_count is set in a configuration.

Contributing

  1. Fork it
  2. Create your feature branch (git checkout -b my-new-feature)
  3. Commit your changes (git commit -am 'Added some feature')
  4. Push to the branch (git push origin my-new-feature)
  5. Create a pull request

About

Retry randomly failing rspec example. This is an extension to the original gem, that will log flakey tests to an output file.

Topics

Resources

License

Code of conduct

Security policy

Stars

Watchers

Forks

Packages

No packages published

Languages