Skip to content

Latest commit

 

History

History
392 lines (293 loc) · 13.9 KB

CONTRIBUTING.md

File metadata and controls

392 lines (293 loc) · 13.9 KB
Raw

Contributing to Dpl

Table of Contents

Dpl is a central component in Travis CI, and has been around for a long time.

This library always has been a community effort first. There probably is not a single person in the world who is very familiar with all deployment providers supported by Dpl.

Thank you all for this!

This document is for you if you are looking to contribute to dpl, be it by adding a new deployment provider, fixing a bug, or adding a new feature.

Dpl has a code of conduct, please follow it in all interactions with the project.

Dpl is written in Ruby, and we assume that you familiarize yourself with our documentation as much as needed.

Resources

Hopefully helpful resources are:

Navigating the Codebase

All provider specific classes live in dpl/providers. These represent the CLI commands that are executed when the command line executable dpl is run with a given provider name as the first argument.

Each provider is a subclass of Dpl::Provider, which is defined in dpl/provider.rb. The provider base class itself subclasses from Cl::Cmd, so it represents an executable sub command of the main command dpl.

For instance, the command dpl s3 --bucket bucket instantiates and runs the provider class S3.

The class Cl::Cmd contributes the command line options parser, and its class level DSL. Please see the cl README for this DSL, and the S3 provider for an example how dpl uses it.

The class Dpl::Provider adds, amongst other things, the order of stages (methods) that make up the deployment process:

  • init
  • install
  • login
  • setup
  • validate
  • prepare
  • deploy
  • finish

Implementors of concrete provider classes may or may not choose to implement any of these instance methods according to their needs, and semantics of their tooling and service providers. Please refer to Dpl::Provider for details.

The DSL that is used to declare features, dependencies, environment integration etc. on the concrete provider classes is defined in the module Dpl::Provider::DSL, in dpl/provider/dsl.

Also of interest is Dpl::Ctx::Bash, the Bash execution context, that runs shell commands, installs dependencies etc. (while the Test context class is used for testing in order to keep your development machine clean and safe when you run tests locally).

lib
└── dpl
    ├── assets                # Stores larger shell scripts
    ├── ctx
    │   ├── bash.rb           # Bash execution context
    │   └── test.rb           # Test execution context
    ├── provider.rb           # Base class for all providers
    ├── provider
    │   ├── dsl.rb            # DSL for defining providers
    │   └── example.rb        # Generating example commands for help output
    └── providers
        ├── anynines.rb       # Concrete providers
        ├── atlas.rb
        ├── azure_webapps.rb
        ├── bintray.rb
        ├── bitballoon.rb
        └── ⋮

Lifecycle of the Deployment Process

When a provider class is instantiated and run it will go through a number of stages that make up the deployment process.

These are documented in dpl/provider.rb. If you are adding a new deployment provider please familiarize yourself with this lifecycle.

Feel free to pick and interpret these stages according to the needs and semantics of the service provider you are adding. By no means do all of these stages have to be filled in or implmented. The Provider base class checks for these methods, and runs them, if present, so that implementors can choose semantically fitting names for their providers.

Deployment Tooling

If you are adding a new deployment provider please choose the tooling you are going to use carefully.

Dpl is a long lived library, and it has outlived many tools that once were supported, and no longer are. Thus tooling stability is a major concern for this project.

Ideally use official CLI tooling supported by the company who's service provider you are about to add. Often, such CLI tools can be installed via standard package managers, or manually downloaded using curl and installed with a few simple Bash commands.

Such CLI tooling is preferrable over Ruby gem runtime dependencies as they can be executed in a child process, and won't introduce any dependency resolution problems later on.

If no such CLI is available, or it does not look well supported, and your provider implementation needs to talk to an external HTTP API then please consider using Net::HTTP from Ruby's standard library.

If you absolutely have to rely on a runtime Ruby gem dependency, such as a provider client implementation, please only do so if the gem is supported by the respective company officially. We may choose to reject including runtime dependencies that do not look stable or widely supported.

Runtime Dependencies

Runtime dependencies can be declared on the provider class using the DSL.

In the case of APT, NPM, and Pip dependencies these will be installed via shell commands at the beginning of the deployment process.

Ruby gem dependencies will be installed using Bundler's inline API, at the beginning of the deployment process, so they are available in the same Ruby process from then on.

Unit Tests

Dpl uses RSpec for tests. The specs reside in spec, and each provider class has a corresponding file spec/dpl/providers/*_spec.rb to hold tests.

Provider tests should be implemented on an input/output acceptance level, as much as possible.

They use a Ctx::Test execution context in order to avoid running actual shell commands, or actually installing dependencies at test time. There are custom RSpec matchers in place that help with making assertions against this execution context.

If your provider has to talk to an external HTTP API then ideally use Webmock to stub external requests. If by any means possible try to avoid mocking or stubbing Ruby client classes (this is not always possible, but should be considered).

Running Unit Tests Locally

You can run the unit test suite locally as follows:

bundle install
bundle exec rspec

In order to execute tests only for a certain provider you can run:

bundle exec rspec spec/dpl/providers/[provider]_spec.rb

In order to execute a single test or group of tests add a line number like so:

bundle exec rspec spec/dpl/providers/[provider]_spec.rb:25

These tests can be run safely on any development machine, anywhere.

Runtime Dependency Installation Tests

We additionally run tests that exercise runtime dependency installation on Travis CI.

These live in .travis/test_install. It is not advisable to run these tests outside of an ephemeral VM or container that can be safely discarded, as they are going to leave various artifacts around.

Integration Tests

In order to ensure proper integration with the service providers supported we also periodically run a test suite that exercises actual deployments to these providers.

These tests live in .travis/providers, and the are triggered using the script trigger.

An integration test consists of:

  • A setup script that creates an application (or artifact) to deploy (or upload).
  • A YAML config snippet that configures and triggers the deployment as part of a build on Travis CI.
  • A test script that tests if the deployment was successful.

For example:

The tests can be run on Travis CI individually, or combined, by triggering a build via our API, using the script .travis/trigger. This takes a provider name as an argument, and requires a Travis CI API token.

For example, this triggers a build that executes the GitHub Pages test on Travis CI:

.travis/trigger github-pages --token [token]

The token can also be set as an environment variable:

export TRAVIS_API_TOKEN=[token]
.travis/trigger github-pages

The trigger script accepts multiple provider names as arguments. If no arguments are given then tests for all providers will be run.

Integration Test Configuration

In the build config YAML snippet make sure to use the branch of your fork for the deployment tooling, and allow the deployment to run on your branch:

deploy:
  - provider: [name]
    edge:
      source: [your-login]/dpl
      branch: [your-branch]
    on:
      branch: [your-branch]

Ideally use credentials for an isolated account on the service you are deploying to. This is generally good practice, and way you can hand things off to someone else.

In order to get things working encrypt the credentials against your fork, and add them to the build config YAML snippet. If you are in the root directory of your fork then this command should do the trick:

travis encrypt password=[password]

If you do not have the travis CLI installed you can install it using:

gem install travis

When you add encrypted credentials to the build config YAML snippet also add a comment that allows others to identify the account used. E.g:

deploy:
  - provider: pages
    github_token:
      # personal access token with repo scope on the account [name]
      secure: "[encrypted token]"

Open a pull request. In order for us to merge your test, and get it working on our repository you will need to re-encrypt the credentials against travis-ci/dpl, like so:

travis encrypt -r travis-ci/dpl password=[password]

Whatever minimal deployment you can get working is be a great contribution. Even if for some reason it proves hard to test the deployment in an automated fashion, but you have a successful deployment that can be verified manually, please still open a pull request, and talk to us. Any test is better than no test.

Testing Dpl Branches or Forks on Travis CI

It is possible to test a new deployment provider or new functionality of dpl on Travis CI. In order to do so, add proper configuration on the edge key to your .travis.yml like so:

deploy:
  provider: [name]
  edge:
    source: [github-handle]/dpl
    branch: [branch]
  on:
    branch: TEST_BRANCH # or all_branches: true
  ⋮ # rest of your provider's configuration

This builds the dpl gem on the Travis CI build environment from your repository, on the given branch. Then it installs the gem built from this code base, and uses it to run your deployment.

When submitting a pull request, please be sure to run at least one deployment with the new configuration, and provide a link to the build in your pull request.

Code Conventions

Dpl does not follow any strict code styleguide.

Please take a look around other providers, and try to follow a similar code style to what you find.

Try to use the DSL as much as possible. It keeps the code declarative and readable, so that people not familiar with Ruby or programming in general can still follow it, and make sense of it.

If you find yourself trying to achieve something that should be, but is not supported by the DSL please open an issue about it.

If you are rather unfamiliar with Ruby, and have trouble following our code style then please submit your pull request anyway, or get in touch, so we can help.

Naming Conventions

Dpl uses constant names following Ruby naming conventions. I.e. constant names use CamelCase, and they live in files named in snake_case.rb.

If you pick such names for a new provider please try to follow these conventions.

Real world service provider or company names do not always translate to such conventional Ruby names one-to-one. That is ok, they don't have to. These Ruby constant names are representations of real world service and company names in Ruby code.

Other Ruby libraries often (not always) follow a similar thinking. E.g. even though Amazon Web Services brand name is AWS the module name they chose in their aws-sdk is Aws, not AWS.

Updating the README

The README is generated from a template.

In order to update the README please edit the template, and run:

gem install ffi-icu
bin/readme > README.md