API 35932 monthly email show consumers in table #16774

rockwellwindsor-va · 2024-05-15T21:18:40Z

Summary

Some ACs were previously completed. This PR is cleaning up a missed point, AC #2 which stated to have a row in the tables for each consumer

Updates monthly report to have a row for each consumer in each table
Refactors to share shared methods and templates between the daily and monthly report
- Moves shared partials so they are in a central spot between the two templates
- Adds a parent base class for the reporting Jobs so they can access thier shared methods
  - They will still, as sidekiq jobs, also have access to the service_base class
Updates/adjusts tests to handle the new data goiing to monthly submissions email

Related issue(s)

API-35932

Testing done

New code is covered by unit tests
Preview for newly developed template dummy data) - http://localhost:3000/rails/mailers/claims_api_submission_report_mailer/build.html
Preview for refactored template (dummy data) http://localhost:3000/rails/mailers/claims_api_unsuccessful_report_mailer/build.html

Testing Notes

To manually test locally you need to set up a few things

In settings.yml make sure claims_api.report_enabled is set to true
Rails c
- Run the job - 'ClaimsApi::ReportMonthlySubmissions.new.perform`
  - The above command will print the email out to the console, the email will not actually send since the proxy it uses is not configured (port 25) and you will see an error related to Port 25 refusing the connection. That is because of local development environment, not an error with the template or anything.

Screenshots

Note: Optional

What areas of the site does it impact?

modified: modules/claims_api/app/mailers/claims_api/submission_report_mailer.rb
modified: modules/claims_api/app/sidekiq/claims_api/report_monthly_submissions.rb
modified: modules/claims_api/app/sidekiq/claims_api/report_unsuccessful_submissions.rb
new file: modules/claims_api/app/sidekiq/claims_api/reporting_base.rb
modified: modules/claims_api/app/views/claims_api/submission_report_mailer/submission_report.html.erb
deleted: modules/claims_api/app/views/claims_api/unsuccessful_report_mailer/_claims_status_table.html.erb
deleted: modules/claims_api/app/views/claims_api/unsuccessful_report_mailer/_counts_table.html.erb
modified: modules/claims_api/app/views/claims_api/unsuccessful_report_mailer/unsuccessful_report.html.erb
modified: modules/claims_api/spec/mailers/report_monthly_submissions_spec.rb
modified: modules/claims_api/spec/sidekiq/report_monthly_submissions_spec.rb
modified: spec/mailers/previews/claims_api_submission_report_mailer_preview.rb

(Refactor. Moved but not modified)

new file: modules/claims_api/app/views/claims_api/_claims_status_table.html.erb <-- Formatting was modified
renamed: modules/claims_api/app/views/claims_api/unsuccessful_report_mailer/_ews_errors_table.html.erb -> modules/claims_api/app/views/claims_api/_ews_errors_table.html.erb
renamed: modules/claims_api/app/views/claims_api/unsuccessful_report_mailer/_ews_status_table.html.erb -> modules/claims_api/app/views/claims_api/_ews_status_table.html.erb
renamed: modules/claims_api/app/views/claims_api/unsuccessful_report_mailer/_itf_status_table.html.erb -> modules/claims_api/app/views/claims_api/_itf_status_table.html.erb
new file: modules/claims_api/app/views/claims_api/_monthly_claims_status_table.html.erb
renamed: modules/claims_api/app/views/claims_api/unsuccessful_report_mailer/_poa_errors_table.html.erb -> modules/claims_api/app/views/claims_api/_poa_errors_table.html.erb
renamed: modules/claims_api/app/views/claims_api/unsuccessful_report_mailer/_poa_status_table.html.erb -> modules/claims_api/app/views/claims_api/_poa_status_table.html.erb
renamed: modules/claims_api/app/views/claims_api/unsuccessful_report_mailer/_submission_table.html.erb -> modules/claims_api/app/views/claims_api/_submission_table.html.erb

Acceptance criteria

I fixed|updated|added unit tests and integration tests for each feature (if applicable).
No error nor warning in the console.
Events are being sent to the appropriate logging solution
Documentation has been updated (link to documentation)
No sensitive information (i.e. PII/credentials/internal URLs/etc.) is captured in logging, hardcoded, or specs
Feature/bug has a monitor built into Datadog or Grafana (if applicable)
If app impacted requires authentication, did you login to a local build and verify all authenticated routes work as expected
I added a screenshot of the developed feature

Requested Feedback

(OPTIONAL)What should the reviewers know in addition to the above. Is there anything specific you wish the reviewer to assist with. Do you have any concerns with this PR, why?

* Updates monthly report to have a row for each consumer in each table * Refactors to share shared methods and templates between the daily and monthly report * Moves shared partials so they are in a central spot between the two templates * Adds a parent base class for the reporting Jobs so they can access thier shared methods * They will still, as sidekiq jobs, also have access to the service_base class * Updates/adjusts tests to handle the new data goiing to monthly submissions email modified: app/models/concerns/kms_encrypted_model_patch.rb modified: modules/claims_api/app/mailers/claims_api/submission_report_mailer.rb modified: modules/claims_api/app/sidekiq/claims_api/report_monthly_submissions.rb modified: modules/claims_api/app/sidekiq/claims_api/report_unsuccessful_submissions.rb new file: modules/claims_api/app/sidekiq/claims_api/reporting_base.rb new file: modules/claims_api/app/views/claims_api/_claims_status_table.html.erb renamed: modules/claims_api/app/views/claims_api/unsuccessful_report_mailer/_ews_errors_table.html.erb -> modules/claims_api/app/views/claims_api/_ews_errors_table.html.erb renamed: modules/claims_api/app/views/claims_api/unsuccessful_report_mailer/_ews_status_table.html.erb -> modules/claims_api/app/views/claims_api/_ews_status_table.html.erb renamed: modules/claims_api/app/views/claims_api/unsuccessful_report_mailer/_itf_status_table.html.erb -> modules/claims_api/app/views/claims_api/_itf_status_table.html.erb new file: modules/claims_api/app/views/claims_api/_monthly_claims_status_table.html.erb renamed: modules/claims_api/app/views/claims_api/unsuccessful_report_mailer/_poa_errors_table.html.erb -> modules/claims_api/app/views/claims_api/_poa_errors_table.html.erb renamed: modules/claims_api/app/views/claims_api/unsuccessful_report_mailer/_poa_status_table.html.erb -> modules/claims_api/app/views/claims_api/_poa_status_table.html.erb renamed: modules/claims_api/app/views/claims_api/unsuccessful_report_mailer/_submission_table.html.erb -> modules/claims_api/app/views/claims_api/_submission_table.html.erb modified: modules/claims_api/app/views/claims_api/submission_report_mailer/submission_report.html.erb deleted: modules/claims_api/app/views/claims_api/unsuccessful_report_mailer/_claims_status_table.html.erb deleted: modules/claims_api/app/views/claims_api/unsuccessful_report_mailer/_counts_table.html.erb modified: modules/claims_api/app/views/claims_api/unsuccessful_report_mailer/unsuccessful_report.html.erb modified: modules/claims_api/spec/mailers/report_monthly_submissions_spec.rb modified: modules/claims_api/spec/sidekiq/report_monthly_submissions_spec.rb modified: spec/mailers/previews/claims_api_submission_report_mailer_preview.rb

github-actions · 2024-05-15T21:19:19Z

	1 Error
🚫	This PR changes `980` LoC (not counting whitespace/newlines). In order to ensure each PR receives the proper attention it deserves, those exceeding `500` will not be reviewed, nor will they be allowed to merge. Please break this PR up into smaller ones. If you have reason to believe that this PR should be granted an exception, please see the Submitting pull requests for approval - FAQ. File Summary Files modules/claims_api/app/mailers/claims_api/submission_report_mailer.rb (+7/-67) modules/claims_api/app/sidekiq/claims_api/report_monthly_submissions.rb (+11/-13) modules/claims_api/app/sidekiq/claims_api/report_unsuccessful_submissions.rb (+1/-121) modules/claims_api/app/sidekiq/claims_api/reporting_base.rb (+140/-0) modules/claims_api/app/views/claims_api/_claims_status_table.html.erb (+26/-0) modules/claims_api/app/views/claims_api/_monthly_claims_status_table.html.erb (+22/-0) modules/claims_api/app/views/claims_api/submission_report_mailer/submission_report.html.erb (+25/-3) modules/claims_api/app/views/claims_api/unsuccessful_report_mailer/_claims_status_table.html.erb (+0/-26) modules/claims_api/app/views/claims_api/unsuccessful_report_mailer/_counts_table.html.erb (+0/-18) modules/claims_api/app/views/claims_api/unsuccessful_report_mailer/unsuccessful_report.html.erb (+7/-7) modules/claims_api/spec/mailers/report_monthly_submissions_spec.rb (+6/-57) modules/claims_api/spec/shared_reporting_helper.rb (+75/-0) modules/claims_api/spec/sidekiq/report_monthly_submissions_spec.rb (+19/-84) modules/claims_api/spec/sidekiq/report_unsuccessful_submissions_spec.rb (+4/-127) modules/claims_api/spec/sidekiq/shared_reporting_examples_spec.rb (+60/-0) spec/mailers/previews/claims_api_submission_report_mailer_preview.rb (+42/-12) Note: We exclude files matching the following when considering PR size: `.csv, .json, .tsv, .txt, Gemfile.lock, app/swagger, modules/mobile/docs, spec/fixtures/, spec/support/vcr_cassettes/, modules/mobile/spec/support/vcr_cassettes/, db/seeds, modules/vaos/app/docs, modules/meb_api/app/docs, modules/appeals_api/app/swagger/, *.bru` Big PRs are difficult to review, often become stale, and cause delays.

Generated by 🚫 Danger

rockwellwindsor-va · 2024-05-15T21:25:45Z

modules/claims_api/app/views/claims_api/_claims_status_table.html.erb

+      </tr>
+    <% end if claims_consumers %>
+  </tbody>
+</table>


The reason this partial shows up as new is the formatting on the old one was wonky, so when I fixed it and moved it (along with the rest of the shared partials) it identified it as new.

…line count down (planning on adding in a follow up)

rockwellwindsor-va · 2024-05-15T21:42:40Z

modules/claims_api/app/sidekiq/claims_api/report_unsuccessful_submissions.rb

-        created_at: @from..@to,
-        status: %w[errored]
-      ).order(:cid, :status)
-    end
  end


Quite a few of the line changes stem from this refactor which was in an effort to share the methods being used by both reporting jobs. -117 here and then +125 in the shared reporting_base file. The only reason I would not make this change would be to avoid the line count warning, however since these are shared methods I believe this change is worth it.

…entages are out for monthly email

rockwellwindsor-va · 2024-05-16T18:36:28Z

modules/claims_api/app/sidekiq/claims_api/reporting_base.rb

+      end
+    end
+
+    def monthly_claims_totals


monthly_claims_totals is the only new method added in here. The rest of the additions are a refactor because they are shared between the monthly and nightly job now (mentioned above)

…table

rockwellwindsor-va · 2024-05-17T15:49:08Z

modules/claims_api/spec/shared_reporting_helper.rb

+    errored_evidence_waiver_submissions.push(FactoryBot.create(:claims_api_evidence_waiver_submission,
+                                                               cid: '0oa9uf05lgXYk6ZXn297'))
+  end
+end


These values are just refactored from the nightly job test file to be shared for both, nothing new added.

rockwellwindsor-va · 2024-05-17T15:49:43Z

modules/claims_api/spec/sidekiq/report_unsuccessful_submissions_spec.rb

-                                             ))
-    errored_evidence_waiver_submissions.push(FactoryBot.create(:claims_api_evidence_waiver_submission,
-                                                               cid: '0oa9uf05lgXYk6ZXn297'))
-  end


Relates to above comment, moved out into a shared file to share between both tests now.

rockwellwindsor-va · 2024-05-17T15:50:21Z

modules/claims_api/spec/sidekiq/report_unsuccessful_submissions_spec.rb

-        expect(itf_totals[1]['VA Connect Pro'][:errored]).to eq(1)
-        expect(itf_totals[1]['VA Connect Pro'][:totals]).to eq(2)
-      end
-    end


Same with these. These examples are used in both files to validate the metrics, so moved out (unchanged) into a shared examples file.

rockwellwindsor-va · 2024-05-17T15:50:45Z

modules/claims_api/spec/sidekiq/shared_reporting_examples_spec.rb

+      expect(itf_totals[1]['VA Connect Pro'][:totals]).to eq(2)
+    end
+  end
+end


See above comment for shared examples. Nothing was changed for these examples other then they are shared now.

mchristiansonVA

Ran ClaimsApi::ReportMonthlySubmissions.new.perform locally in console without issues; reviewed output from preview links for new & refactored templates with dummy data. Looks good!

…table

rockwellwindsor-va added 2 commits May 15, 2024 16:16

Revert KMS change made for quicker dev locally

bc1cf9d

Adds newline in and fixes a removed space that was unintentional

ff3dac2

rockwellwindsor-va commented May 15, 2024

View reviewed changes

rockwellwindsor-va added 4 commits May 15, 2024 16:28

Removes some additional 'nice to have' tests in an effort to get the …

bc46a61

…line count down (planning on adding in a follow up)

Removes unnessecary spaces from html template file

2bc79d5

Puts back removed send test

81764e5

Adjusts formatting for some helper values

b8b9ee8

rockwellwindsor-va commented May 15, 2024

View reviewed changes

va-vfs-bot had a problem deploying to API-35932-monthly-email-show-consumers-in-table/main/main May 15, 2024 21:47 Error

rockwellwindsor-va added 2 commits May 15, 2024 16:54

Updating for broken test

6c0036d

One more Rubocop adjust

f59bc19

va-vfs-bot temporarily deployed to API-35932-monthly-email-show-consumers-in-table/main/main May 15, 2024 22:08 Inactive

One more Rubocop adjust

dcffaf4

va-vfs-bot temporarily deployed to API-35932-monthly-email-show-consumers-in-table/main/main May 15, 2024 22:24 Inactive

An update to a test, places in monthly claim totals method since perc…

0af1c07

…entages are out for monthly email

rockwellwindsor-va commented May 16, 2024

View reviewed changes

Rubocop fix

f045629

va-vfs-bot temporarily deployed to API-35932-monthly-email-show-consumers-in-table/main/main May 16, 2024 18:45 Inactive

Further refactors shared examples and methods in the rspec files

15c9313

va-vfs-bot temporarily deployed to API-35932-monthly-email-show-consumers-in-table/main/main May 17, 2024 15:14 Inactive

rockwellwindsor-va added 2 commits May 17, 2024 10:16

Merge branch 'master' into API-35932-monthly-email-show-consumers-in-…

ea4a9e7

…table

Merge branch 'master' into API-35932-monthly-email-show-consumers-in-…

4cf5222

…table

rockwellwindsor-va added the claimsApi modules/claims_api label May 17, 2024

rockwellwindsor-va commented May 17, 2024

View reviewed changes

rockwellwindsor-va marked this pull request as ready for review May 17, 2024 15:52

rockwellwindsor-va requested a review from a team as a code owner May 17, 2024 15:52

va-vfs-bot temporarily deployed to API-35932-monthly-email-show-consumers-in-table/main/main May 17, 2024 16:45 Inactive

mchristiansonVA approved these changes May 20, 2024

View reviewed changes

Merge branch 'master' into API-35932-monthly-email-show-consumers-in-…

08e6940

…table

va-vfs-bot deployed to API-35932-monthly-email-show-consumers-in-table/main/main May 20, 2024 15:11 View deployment

rockwellwindsor-va merged commit a67b199 into master May 20, 2024
18 of 19 checks passed

rockwellwindsor-va deleted the API-35932-monthly-email-show-consumers-in-table branch May 20, 2024 16:41

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

API 35932 monthly email show consumers in table #16774

API 35932 monthly email show consumers in table #16774

rockwellwindsor-va commented May 15, 2024 •

edited

github-actions bot commented May 15, 2024 •

edited

Files

rockwellwindsor-va May 15, 2024

rockwellwindsor-va May 15, 2024

rockwellwindsor-va May 16, 2024 •

edited

rockwellwindsor-va May 17, 2024

rockwellwindsor-va May 17, 2024

rockwellwindsor-va May 17, 2024 •

edited

rockwellwindsor-va May 17, 2024 •

edited

mchristiansonVA left a comment

API 35932 monthly email show consumers in table #16774

API 35932 monthly email show consumers in table #16774

Conversation

rockwellwindsor-va commented May 15, 2024 • edited

Summary

Related issue(s)

Testing done

Testing Notes

Screenshots

What areas of the site does it impact?

(Refactor. Moved but not modified)

Acceptance criteria

Requested Feedback

github-actions bot commented May 15, 2024 • edited

Files

rockwellwindsor-va May 15, 2024

Choose a reason for hiding this comment

rockwellwindsor-va May 15, 2024

Choose a reason for hiding this comment

rockwellwindsor-va May 16, 2024 • edited

Choose a reason for hiding this comment

rockwellwindsor-va May 17, 2024

Choose a reason for hiding this comment

rockwellwindsor-va May 17, 2024

Choose a reason for hiding this comment

rockwellwindsor-va May 17, 2024 • edited

Choose a reason for hiding this comment

rockwellwindsor-va May 17, 2024 • edited

Choose a reason for hiding this comment

mchristiansonVA left a comment

Choose a reason for hiding this comment

rockwellwindsor-va commented May 15, 2024 •

edited

github-actions bot commented May 15, 2024 •

edited

rockwellwindsor-va May 16, 2024 •

edited

rockwellwindsor-va May 17, 2024 •

edited

rockwellwindsor-va May 17, 2024 •

edited