Merge pull request #7 from shanejansen/develop

Develop

.github/workflows/publish.yml

@@ -0,0 +1,26 @@
+name: Publish
+
+on:
+  release:
+    types: [created]
+
+jobs:
+  deploy:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v2
+      - name: Set up Python 3.8
+        uses: actions/setup-python@v1
+        with:
+          python-version: 3.8
+      - name: Install dependencies
+        run: |
+          pip install .
+          pip install setuptools wheel twine
+      - name: Build and publish
+        env:
+          TWINE_USERNAME: ${{ secrets.PYPI_USERNAME }}
+          TWINE_PASSWORD: ${{ secrets.PYPI_PASSWORD }}
+        run: |
+          python setup.py sdist bdist_wheel
+          twine upload dist/*

Pipfile

@@ -12,6 +12,7 @@ pika = "~=1.1"
 pyyaml = "~=5.2"
 pymongo = "~=3.10"
 pymysql = "~=0.9.3"
+minio = "~=5.0.7"
 touchstone-testing = {editable = true,path = "."}
 
 [requires]

README.md

@@ -3,15 +3,15 @@ Touchstone
 ![Unit Tests](https://github.com/shane-jansen/touchstone/workflows/Unit%20Tests/badge.svg?branch=develop)
 ![Touchstone Tests](https://github.com/shane-jansen/touchstone/workflows/Touchstone%20Tests/badge.svg?branch=develop)
 
-Touchstone is a testing framework for your services that focuses on [end-to-end](https://www.martinfowler.com/bliki/BroadStackTest.html) and [exploratory](https://martinfowler.com/bliki/ExploratoryTesting.html) testing.  
+Touchstone is a testing framework for your services that focuses on [component](https://martinfowler.com/articles/microservice-testing/#testing-component-out-of-process-diagram), [end-to-end](https://martinfowler.com/articles/microservice-testing/#testing-end-to-end-introduction), and [exploratory](https://martinfowler.com/bliki/ExploratoryTesting.html) testing.  
 **Touchstone is currently in alpha and APIs may change without warning.**
 
 
 ## Introduction
 ![Testing Pyramid](./docs/images/testing-pyramid.png)  
 [Image Credit](https://martinfowler.com/articles/microservice-testing/#conclusion-test-pyramid)
 
-Touchstone aims to simplify these two pieces of the testing pyramid by providing mock implementations of common service dependencies and exposing them via an easy to use testing framework. Whether your app is written in Java, Python, Go, C#, [Fortran](https://www.fortran.io/), or any other language, Touchstone handles its dependencies while you focus on writing tests. Not a single line of end-to-end testing code needs to change should you decide to refactor or rewrite your service.
+Touchstone aims to simplify the top three pieces of the testing pyramid by providing mock implementations of common service dependencies and exposing them via an easy to use testing framework. Whether your app is written in Java, Python, Go, C#, [Fortran](https://www.fortran.io/), or any other language, Touchstone handles its dependencies while you focus on writing tests. Not a single line of component or end-to-end testing code needs to change should you decide to refactor or rewrite your service.
 
 
 ## Use Case
@@ -22,7 +22,7 @@ Let's say we are building a microservice that is responsible for managing users.
  * `DELETE /user/{id}` - A user is deleted from a relational database. A message is also published to a broker on the exchange: 'user.exchange' with a routing key of: 'user-deleted' and a payload containing the user's id.
  * The service is also listening for messages published to the exchange: 'order-placed.exchange'. When a message is received, the order payload is saved to a NoSQL database.
 
-With Touchstone, it is possible to write end-to-end tests for all of the above requirements independent of the language/framework used. For example, we can write an end-to-end test for the `DELETE /user/{id}` endpoint that will ensure the user record is removed from the database and a message is published to the correct exchange with the correct payload. When ran, Touchstone will monitor mock instances of the service's dependencies to ensure the requirements are met. Touchstone also makes it easy to perform exploratory testing locally during development by starting dependencies and populating them with data in a single command.
+With Touchstone, it is possible to write component and end-to-end tests for all of the above requirements independent of the language/framework used. For example, we can write a component test for the `DELETE /user/{id}` endpoint that will ensure the user record is removed from the database and a message is published to the correct exchange with the correct payload. When ran, Touchstone will monitor mock instances of the service's dependencies to ensure the requirements are met. Touchstone also makes it easy to perform exploratory testing locally during development by starting dependencies and populating them with data in a single command.
 
 An example of the above requirements is implemented in a Java/Spring service in this repo. Touchstone tests have been written to test the [user endpoint requirements](./examples/java-spring/touchstone/tests/test_user.py) and [order messaging requirements](./examples/java-spring/touchstone/tests/test_order.py).
 
@@ -39,16 +39,16 @@ Requirements:
 After installation, Touchstone will be available via `touchstone` in your terminal.  
 Touchstone has three basic commands:
  * `touchstone init` - Initialize Touchstone in the current directory. Used for new projects.
- * `touchstone run` - Run all Touchstone tests and exit. This is typically how you would run your end-to-end tests on a build server. Ports will be auto-discovered in this mode to avoid collisions in case multiple runs occur on the same host. See [mocks docs](#mocks) for more information on how to hook into auto-discovered ports.
- * `touchstone develop` - Start a development session of Touchstone. You would typically use this to develop/debug a service locally. This will keep service dependencies running while you make changes to your end-to-end tests or the services themselves. This will also provide a web interface to each dependency for additional debugging. 
+ * `touchstone run` - Run all Touchstone tests and exit. This is typically how you would run your Touchstone tests on a build server. Ports will be auto-discovered in this mode to avoid collisions in case multiple runs occur on the same host. See [mocks docs](#mocks) for more information on how to hook into auto-discovered ports.
+ * `touchstone develop` - Start a development session of Touchstone. You would typically use this to develop/debug a service locally. This will keep service dependencies running while you make changes to your Touchstone tests or the services themselves. This will also provide a web interface to each mock dependency for additional debugging. Mocked dependencies can be altered or reset on the fly to make exploratory testing easier.
 
 After running `touchstone init`, a new directory will be created with the following contents:
 
-### `touchstone.yml`
+### `/touchstone.yml`
 [Example](./examples/java-spring/touchstone/touchstone.yml)  
 Your services and their monitored dependencies are defined here. Default values should be enough in most cases.
  * `host:` - Default: localhost. The host where your services are running.
- * `services:` - Each service included in your end-to-end tests is defined here.
+ * `services:` - Each service included in your Touchstone tests is defined here.
    * `name:` - Default: unnamed-service. The name of the service.
    * `tests:` - Default: ./tests. The path to Touchstone tests for this service.
    * `host:` - Default: parent host. Fine-grained host control per service.
@@ -61,15 +61,15 @@ Your services and their monitored dependencies are defined here. Default values
    * This is how Touchstone determines which mocked dependencies should be started.
    * View each [mock's docs](#mocks) for values and additional configuration.
 
-### `defaults`
+### `/defaults`
 [Example](./examples/java-spring/touchstone/defaults/mysql.yml)  
 This directory contains YAML files where default values for mocked dependencies are defined. Defaults make it easy to test your service(s) locally by setting up your mock dependencies with sensible defaults. The name of each YAML file should match the name of a mock. For instance, with the MySQL mock, a `mysql.yml` file would contain default databases and tables to be created as well as statements to insert initial data. View each [mock's docs](#mocks) for allowable values.
 
 
-### `tests`
+### `/tests`
 [Example](./examples/java-spring/touchstone/tests)  
-This directory is the default location for your end-to-end tests. This can optionally be configured for each service in `touchstone.yml`.  
-Touchstone follows a _given_, _when_, _then_ testing pattern. Each test is declared in a Python file prefixed with `test_` containing classes that extend `TouchstoneTest`. By extending this class, you can access Touchstone mocked dependencies to setup and then verify your requirements. For example, we can insert a document into a Mongo DB collection and then verify it exists using the following code:
+This directory is the default location for your Touchstone tests. This can optionally be configured for each service in `touchstone.yml`.  
+Touchstone follows a _given_, _when_, _then_ testing pattern. Each test is declared in a Python file prefixed with `test_` containing classes that extend `TouchstoneTest`. By extending this class, you can access Touchstone mocked dependencies to setup and then verify your requirements. For example, we can insert a document into a Mongo DB collection and then verify it exists using the following APIs:
 ```python
 self.mocks.mongodb.setup.insert_document('my_db', 'my_collection', {'foo': 'bar'})
 result: bool = self.mocks.mongodb.verify.document_exists('my_db', 'my_collection', {'foo': 'bar'})
@@ -84,9 +84,12 @@ Important APIs:
  * [Mongo DB](./docs/mocks/mongodb.md)
  * [MySQL](./docs/mocks/mysql.md)
  * [Rabbit MQ](./docs/mocks/rabbitmq.md)
+ * [S3](./docs/mocks/s3.md)
  * [Add one!](./docs/under-construction.md)
 
 When running via `touchstone develop`, dev ports for each mock are used. When running touchstone via `touchstone run`, ports are automatically discovered and available to your service containers via the following environment variables:
  * `TS_{MOCK_NAME}_HOST` - Host where the mock is running.
  * `TS_{MOCK_NAME}_PORT` - Port where the mock is running.
  * `TS_{MOCK_NAME}_URL` - Complete URL where the mock is running.
+ * `TS_{MOCK_NAME}_USERNAME` - Username for authenticating with the mock.
+ * `TS_{MOCK_NAME}_PASSWORD` - Password for authenticating with the mock.

docs/mocks/s3.md

@@ -0,0 +1,42 @@
+S3
+======
+Used to mock a S3 object storage dependency.
+
+
+## Specs
+ * Name: s3
+ * Dev Port: 9000
+ * Username: admin123
+ * Password: admin123
+
+
+## Configuration
+N/A
+
+
+## Defaults Example
+```yaml
+---
+buckets:
+  - name: mybucket
+    objects:
+      - name: foo.csv
+        content-type: text/csv
+        path: ./s3-objects/foo.csv
+      - name: test/bar.png
+        content-type: image/png
+        path: ./s3-objects/bar.png
+```
+
+
+## Usage Example
+```python
+# Create a bucket
+self.mocks.s3.setup.create_bucket('bucket_name')
+
+# Put an object in a bucket
+self.mocks.s3.setup.put_object('bucket_name', 'object_name', data)
+
+# Verify an object exists in a bucket
+result: bool = self.mocks.s3.verify.object_exists('bucket_name', 'object_name')
+```

setup.py

@@ -1,11 +1,13 @@
 from setuptools import setup, find_packages
 
+from touchstone import __version__
+
 with open("README.md", "r") as fh:
     long_description = fh.read()
 
 setup(
     name='touchstone-testing',
-    version='0.1.2',
+    version=__version__,
     description='Touchstone is a testing framework for your services that focuses on end-to-end and exploratory testing.',
     long_description=long_description,
     long_description_content_type="text/markdown",
@@ -19,7 +21,8 @@
         'pika~=1.1',
         'pyyaml~=5.2',
         'pymongo~=3.10',
-        'pymysql~=0.9.3'
+        'pymysql~=0.9.3',
+        'minio~=5.0.7'
     ],
     entry_points={
         'console_scripts': [

tests/lib/test_docker_manager.py

@@ -20,7 +20,7 @@ def test_buildDockerfile_CommandReturnsNon0_imageNotCreated(self, mock_subproces
         result = self.docker_manager.build_dockerfile(dockerfile_path)
 
         # Then
-        self.assertEqual(None, result)
+        self.assertIsNone(result)
 
     @mock.patch('touchstone.lib.docker_manager.subprocess')
     def test_runImage_commandReturnsNon0_exceptionRaised(self, mock_subprocess: Mock):

tests/test_common.py

@@ -0,0 +1,99 @@
+import unittest
+from unittest import TestCase, mock
+
+from touchstone import common
+
+
+class TestCommon(TestCase):
+    @mock.patch('touchstone.common.os')
+    def test_sanityCheckPasses_requirementsMet_ReturnsTrue(self, mock_os):
+        # Given
+        mock_os.getcwd.return_value = 'temp'
+        mock_os.path.exists.return_value = True
+
+        # When
+        result = common.sanity_check_passes()
+
+        # Then
+        self.assertTrue(result)
+
+    @mock.patch('touchstone.common.os')
+    def test_sanityCheckPasses_requirementsNotMet_ReturnsFalse(self, mock_os):
+        # Given
+        mock_os.getcwd.return_value = 'temp'
+        mock_os.path.exists.return_value = False
+
+        # When
+        result = common.sanity_check_passes()
+
+        # Then
+        self.assertFalse(result)
+
+    def test_dictMerge_emptyOverride_ReturnsBase(self):
+        # Given
+        base = {'foo': 'bar'}
+        override = {}
+
+        # When
+        result = common.dict_merge(base, override)
+
+        # Then
+        self.assertDictEqual(result, base)
+
+    def test_dictMerge_noOverlap_ReturnsCombined(self):
+        # Given
+        base = {'foo': 'bar'}
+        override = {'bar': 'foo'}
+
+        # When
+        result = common.dict_merge(base, override)
+
+        # Then
+        expected = {'foo': 'bar', 'bar': 'foo'}
+        self.assertDictEqual(result, expected)
+
+    def test_dictMerge_withOverlap_OverrideTakesPrecedence(self):
+        # Given
+        base = {'foo': 'bar', 'bar': 'foo'}
+        override = {'bar': 'buzz'}
+
+        # When
+        result = common.dict_merge(base, override)
+
+        # Then
+        expoected = {'foo': 'bar', 'bar': 'buzz'}
+        self.assertDictEqual(result, expoected)
+
+    def test_toSnake_string_ReturnsSnake(self):
+        # Given
+        input = 'fooBar'
+
+        # When
+        result = common.to_snake(input)
+
+        # Then
+        self.assertEqual(result, 'foo_bar')
+
+    def test_toSnake_dict_ReturnsSnakeKeys(self):
+        # Given
+        input = {'fooKey': 'barValue'}
+
+        # When
+        result = common.to_snake(input)
+
+        # Then
+        self.assertDictEqual(result, {'foo_key': 'barValue'})
+
+    def test_toSnake_listOfDicts_ReturnsSnakeKeys(self):
+        # Given
+        input = [{'fooKey': 'barValue'}, {'barKey': 'fooValue'}]
+
+        # When
+        result = common.to_snake(input)
+
+        # Then
+        self.assertListEqual(result, [{'foo_key': 'barValue'}, {'bar_key': 'fooValue'}])
+
+
+if __name__ == '__main__':
+    unittest.main()

touchstone-tests/touchstone/defaults/s3-objects/foo.csv

@@ -0,0 +1,3 @@
+first name,last name
+John      ,Smith
+Jane      ,Brown

touchstone-tests/touchstone/defaults/s3.yml

@@ -0,0 +1,10 @@
+---
+buckets:
+  - name: mybucket
+    objects:
+      - name: foo.csv
+        content-type: text/csv
+        path: ./s3-objects/foo.csv
+      - name: test/bar.png
+        content-type: image/png
+        path: ./s3-objects/bar.png