Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor Mock and Test Setup #651

Open
wants to merge 11 commits into
base: master
Choose a base branch
from

Conversation

ronilan
Copy link
Contributor

@ronilan ronilan commented May 10, 2022

Overview

This pull request refactors the S3 mocking setup (see: #540) simplifying implementation, separates the S3 tests from the build tests, introduces test apps using alternative bucket specification (see: #576) and production/staging hosts (see: #533) and enables easy switch between mock bucket and real bucket (given status of: #613).

This pull request comes "on top" of (i.e. includes changes from) #648, #649, #650.

Change

Code

  • Refactored mock setup (e3c9772)
    • mock modules moved to lib/mock.
    • mock modules are self contained and have a signature similar to that of other modules in the package.
    • simplified http mock to remove no longer needed interface.
    • modified http mock to support path style bucket access.
    • set http mock to work with default npg-mock-bucket bucket.
  • Refactored test setup (9085b88)
    • separated s3 tests from build tests.
    • refactored fetch test and proxy-bcrypt test to work with refactored mock setup.
  • Refactored test apps and increased test coverage (4572c86)
    • set hosts of test apps to point to mock bucket.
    • added app1.1 - identical to app1 but using production and staging binary host option.
    • added app1.2 - identical to app1 but using explicit host, region, bucket options.
  • Added service script to switch test apps bucket from mock to S3 and vis versa. Removed previous one (c2c54c5)

Testing

  • "Out of the Box" (repo clone) tests will skip interaction with S3.
  • Setting the environment variable node_pre_gyp_mock_s3 to true will run them against the mock (see: repo CI).
  • To run against a real bucket:
    • unset environment variable node_pre_gyp_mock_s3.
    • set environment variables AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY.
    • run npm run bucket {name-of-your-us-east-1-bucket}
  • To switch back to mock:
    • set environment variable node_pre_gyp_mock_s3 to true.
    • run npm run bucket
  • Manually configuring regions other than us-east-1 also works but switch is more work.
  • Results of running npm run coverage differ depending on what bucket is used (real s3 or mock).

Tests

  • All tests pass in both real bucket and mock bucket setups (including node 18).

ronilan added 10 commits May 4, 2022 12:37
- Updated abi_crosswalk.json to include node 18 (current).
- Added test coverage for cases of unsupported/unknown targets.
- Modified function signature. Instead of augmenting an object (side effect) the function now gets an options object and returns a configuration object.
- Cleaned up of code auto used to extract bucket and region from s3 url.
- Added comments.
- Increased test coverage.
- All tests pass.
- Modified versioning to add bucket name to hosted_path when s3ForcePathStyle is true.
- Modified s3_setup to remove bucket name from prefix when s3ForcePathStyle is true.
- Added test coverage for s3ForcePathStyle cases.
- mocking setup can be more easily used by automated tests and/or manually by developer.
- mock modules moved to lib/mock.
- mock modules are self contained and have a signature similar to that of other modules in the package.
- simplified http mock to remove no longer needed interface.
- modified http mock to support path style bucket access.
- set http mock to work with default npg-mock-bucket bucket.
- separated s3 tests from build tests.
- refactored fetch test and proxy-bcrypt test to work with refactored mock setup.
- set hosts of test apps to point to mock bucket.
- added app1.1 - identical to app1 but using production and staging binary host option.
- added app1.2 - identical to app1 but using explicit host, region, bucket options.
(uri) => {
const bucket = 'npg-mock-bucket';
const mockDir = uri.indexOf(bucket) === -1 ? `${basePath}/${bucket}` : basePath;
const filepath = path.join(mockDir, uri.replace('%2B', '+'));

Check failure

Code scanning / CodeQL

Incomplete string escaping or encoding High

This replaces only the first occurrence of '%2B'.
function http_mock() {
log.warn('mocking http requests to s3');

const baseHostname = 's3.us-east-1.amazonaws.com';

Check failure

Code scanning / CodeQL

Incomplete regular expression for hostnames High

This string, which is used as a regular expression
here
, has an unescaped '.' before 'amazonaws.com', so it might match more hosts than expected.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant