Failing RSpec tests on ~>7.0.0

[AnkeshThakur]

I am setting up Knapsack Pro with gem version 7.0.0 and 7.0.1. Tests are failing with both the versions which otherwise is passing.

I am mostly getting two types of error which relates to instance_double tests and materialize-view dependent tests. Following is the error description from the runs

  1) SalesforceAppConfiguration AASM when revoked when activate with active subscription with stored auth transitions
     Failure/Error:
       allow(app_configuration).to receive(:salesforce_authentication)
                                     .and_return(instance_double(Salesforce::Authentication,
                                                                 refresh_token: 'aBc'))

       the Salesforce::Authentication class does not implement the instance method: refresh_token
     # ./spec/models/salesforce_app_configuration_spec.rb:121:in `block (7 levels) in <top (required)>'
     # ./spec/spec_helper.rb:209:in `block (2 levels) in <main>'

  2) SalesforceAppConfiguration AASM when new when activate with active subscription with stored auth transitions
     Failure/Error:
       allow(app_configuration).to receive(:salesforce_authentication)
                                     .and_return(instance_double(Salesforce::Authentication,
                                                                 refresh_token: 'aBc'))

       the Salesforce::Authentication class does not implement the instance method: refresh_token
     # ./spec/models/salesforce_app_configuration_spec.rb:63:in `block (7 levels) in <top (required)>'
     # ./spec/spec_helper.rb:209:in `block (2 levels) in <main>'
.
.
.
Finished in 3 minutes 19.4 seconds (files took 1.04 seconds to load)
888 examples, 10 failures, 1 pending

Other error:

       PG::ObjectNotInPrerequisiteState: ERROR:  materialized view "best_product_medals" has not been populated
       HINT:  Use the REFRESH MATERIALIZED VIEW command.
     # ./app/domain/medals/list.rb:47:in `map'
     # ./app/domain/medals/list.rb:47:in `wrap_medals'
     # ./app/domain/medals/list.rb:21:in `block in medals'
     # ./app/domain/medals/list.rb:21:in `each'
     # ./app/domain/medals/list.rb:21:in `flat_map'
     # ./app/domain/medals/list.rb:21:in `medals'
     # ./app/domain/vendor_admin/medals/product_medals.rb:15:in `medal_class_and_ids'
     # ./app/domain/vendor_admin/medals/product_medals.rb:9:in `exclude?'
     # ./app/models/product_reference_page_medal.rb:54:in `medal_invalid?'
     # ./spec/models/product_reference_page_medal_spec.rb:24:in `block (4 levels) in <top (required)>'
     # ./spec/spec_helper.rb:207:in `block (2 levels) in <main>'
     # ------------------
     # --- Caused by: ---
     # PG::ObjectNotInPrerequisiteState:
     #   ERROR:  materialized view "best_product_medals" has not been populated
     #   HINT:  Use the REFRESH MATERIALIZED VIEW command.
     #   ./app/domain/medals/list.rb:47:in `map'

I am trying to set it up on Github Actions with VCR/WebMosck setup. All of these tests are passing when I try it with 6.0.4.

Following is the command that I'm using to invoke the test in GHA runner bundle exec rake "knapsack_pro:queue:rspec[--require packs/rails/rspec --require spec_helper --order rand --require fivemat --format Fivemat --format RSpec::Github::Formatter --format RspecJunitFormatter --out report/ruby-rspec-${{ matrix.node_index }}.xml]"

[GeoffTidey]

I'm seeing the same issue with instance_double intermittently with v7

e.g/

Failure/Error: meh = instance_double(MyClass, this_does_exist: 42)

the MyClass class does not implement the instance method: this_does_exist

Just in case it's not obvious, the instance method does exist :)

[ArturT]

Could you prepare an integration test like this one to help us reproduce an issue?

Or it could be a new Rails app that we could use to debug it?

Thank you.

[GeoffTidey]

Hi @ArturT do you mean something like https://github.com/KnapsackPro/knapsack_pro-ruby/pull/247/files ?

[ArturT]

@GeoffTidey Yes.

I've run this test and it's passing successful.

➜  knapsack_pro-ruby git:(pr/247) rspec spec/integration/runners/queue/rspec_runner_spec.rb:382
Resolving dependencies...
Run options: include {:locations=>{"./spec/integration/runners/queue/rspec_runner_spec.rb"=>[382]}}

Randomized with seed 65516

KnapsackPro::Runners::Queue::RSpecRunner - Integration tests
  when rails_helper file exists
    requires the rails_helper file when running RSpec and runs hooks defined within it

Finished in 0.2351 seconds (files took 0.42629 seconds to load)
1 example, 0 failures

Randomized with seed 65516

We need to prepare a test that would fail with the same error as you see in your project. Then we could prepare a fix for that.

Are you able to identify what is causing the error in your project?

[GeoffTidey]

Afraid not. It's intermittent.

[ArturT]

I use the following loop to run test as long until it fails but it is successful. I'm not able to reproduce the issue.

while bundle exec rspec "spec/integration/runners/queue/rspec_runner_spec.rb:382"; do :; done

I use ruby 3.2.3 (2024-01-18 revision 52bb2ac0a6) [arm64-darwin22].

I tried to reproduce it on an older RSpec version without success:

# source Gemfile.lock in the knasack_pro gem 
rspec (3.11.0)
  rspec-core (~> 3.11.0)
  rspec-expectations (~> 3.11.0)
  rspec-mocks (~> 3.11.0)
rspec-core (3.11.0)
  rspec-support (~> 3.11.0)
rspec-expectations (3.11.0)
  diff-lcs (>= 1.2.0, < 2.0)
  rspec-support (~> 3.11.0)
rspec-its (1.3.0)
  rspec-core (>= 3.0.0)
  rspec-expectations (>= 3.0.0)
rspec-mocks (3.11.1)
  diff-lcs (>= 1.2.0, < 2.0)
  rspec-support (~> 3.11.0)
rspec-support (3.11.0)
rspec_junit_formatter (0.6.0)
  rspec-core (>= 2, < 4, != 2.12.0)

I also tried to use the latest gems, but I'm not able to reproduce the issue:

rspec (3.13.0)
  rspec-core (~> 3.13.0)
  rspec-expectations (~> 3.13.0)
  rspec-mocks (~> 3.13.0)
rspec-core (3.13.0)
  rspec-support (~> 3.13.0)
rspec-expectations (3.13.0)
  diff-lcs (>= 1.2.0, < 2.0)
  rspec-support (~> 3.13.0)
rspec-its (1.3.0)
  rspec-core (>= 3.0.0)
  rspec-expectations (>= 3.0.0)
rspec-mocks (3.13.0)
  diff-lcs (>= 1.2.0, < 2.0)
  rspec-support (~> 3.13.0)
rspec-support (3.13.1)
rspec_junit_formatter (0.6.0)
  rspec-core (>= 2, < 4, != 2.12.0)

@GeoffTidey Could you provide your Ruby and dependency versions? It will help pinpoint what differs between our setups.

[GeoffTidey]

Hi @ArturT , sorry for the radio silence. It's not been happening at all since my last update. Two things have changed, the first is probably the issue:

1. I moved the KnapsackPro::Adapters::RSpecAdapter.bind to spec/spec_helper.rb (it was perviously in another helper that is only loaded for system tests),
2. the other is we're now using GitHub Actions for CI, rather than Heroku CI.