Merge pull request #730 from motdotla/env-vault

.env.vault

CHANGELOG.md

@@ -2,7 +2,15 @@
 
 All notable changes to this project will be documented in this file. See [standard-version](https://github.com/conventional-changelog/standard-version) for commit guidelines.
 
-## [Unreleased](https://github.com/motdotla/dotenv/compare/v16.0.3...master)
+## [Unreleased](https://github.com/motdotla/dotenv/compare/v16.1.0...master)
+
+## [16.1.0](https://github.com/motdotla/dotenv/compare/v16.0.3...v16.1.0) (2023-04-01)
+
+### Added
+
+- Add `.env.vault` support. 🎉 ([#730](https://github.com/motdotla/dotenv/pull/730))
+
+ℹ️ `.env.vault` extends the `.env` file format standard with a localized encrypted vault file. Package it securely with your production code deploys. It's cloud agnostic so that you can deploy your secrets anywhere – without [risky third-party integrations](https://techcrunch.com/2023/01/05/circleci-breach/).
 
 ## [16.0.3](https://github.com/motdotla/dotenv/compare/v16.0.2...v16.0.3) (2022-09-29)
 

README.md

@@ -53,18 +53,20 @@
 
 Dotenv is a zero-dependency module that loads environment variables from a `.env` file into [`process.env`](https://nodejs.org/docs/latest/api/process.html#process_process_env). Storing configuration in the environment separate from code is based on [The Twelve-Factor App](http://12factor.net/config) methodology.
 
-[![BuildStatus](https://img.shields.io/travis/motdotla/dotenv/master.svg?style=flat-square)](https://travis-ci.org/motdotla/dotenv)
-[![Build status](https://ci.appveyor.com/api/projects/status/github/motdotla/dotenv?svg=true)](https://ci.appveyor.com/project/motdotla/dotenv/branch/master)
 [![NPM version](https://img.shields.io/npm/v/dotenv.svg?style=flat-square)](https://www.npmjs.com/package/dotenv)
 [![js-standard-style](https://img.shields.io/badge/code%20style-standard-brightgreen.svg?style=flat-square)](https://github.com/feross/standard)
 [![Coverage Status](https://img.shields.io/coveralls/motdotla/dotenv/master.svg?style=flat-square)](https://coveralls.io/github/motdotla/dotenv?branch=coverall-intergration)
 [![LICENSE](https://img.shields.io/github/license/motdotla/dotenv.svg)](LICENSE)
-[![Conventional Commits](https://img.shields.io/badge/Conventional%20Commits-1.0.0-yellow.svg)](https://conventionalcommits.org)
-[![Featured on Openbase](https://badges.openbase.com/js/featured/dotenv.svg?token=eE0hWPkhC2JGSD4G9hwg5C54EBxjJAyvurGfQsYoKiQ=)](https://openbase.com/js/dotenv?utm_source=embedded&utm_medium=badge&utm_campaign=featured-badge&utm_term=js/dotenv)
-[![Limited Edition Tee Original](https://img.shields.io/badge/Limited%20Edition%20Tee%20%F0%9F%91%95-Original-yellow?labelColor=black&style=plastic)](https://dotenv.gumroad.com/l/original)
-[![Limited Edition Tee Redacted](https://img.shields.io/badge/Limited%20Edition%20Tee%20%F0%9F%91%95-Redacted-gray?labelColor=black&style=plastic)](https://dotenv.gumroad.com/l/redacted)
 
-## Install
+* [🌱 Install](#install)
+* [🏗️ Usage (.env)](#usage)
+* [🚀 Deploying (.env.vault) 🆕](#deploying)
+* [🌴 Examples](#examples)
+* [Docs](#documentation)
+* [FAQ](#faq)
+* [Changelog](./CHANGELOG.md)
+
+## 🌱 Install
 
 ```bash
 # install locally (recommended)
@@ -73,7 +75,7 @@ npm install dotenv --save
 
 Or installing with yarn? `yarn add dotenv`
 
-## Usage
+## 🏗️ Usage
 
 Create a `.env` file in the root of your project:
 
@@ -180,7 +182,68 @@ You need to add the value of another variable in one of your variables? Use [dot
 
 You need to keep `.env` files in sync between machines, environments, or team members? Use [dotenv-vault](https://github.com/dotenv-org/dotenv-vault).
 
-## Examples
+## 🚀 Deploying
+
+**Note: Unreleased. Coming soon! Releasing as dotenv@16.1.0.**
+
+Up until recently (year 2023), we did not have an opinion on deploying your secrets to production. Dotenv had been focused on solving development secrets only. However, with the increasing number of secrets breaches like the [CircleCI breach](https://techcrunch.com/2023/01/05/circleci-breach/) we have formed an opinion.
+
+Don't scatter your secrets across these platforms. Use a `.env.vault` file.
+
+The `.env.vault` file encrypts your secrets and decrypts them just-in-time on boot of your application. It uses a `DOTENV_KEY` environment variable that you set on your cloud platform or server. If there is a secrets breach, an attacker only gains access to your decryption key, not your secrets. They would additionally have to gain access to your codebase, find your .env.vault file, and decrypt it to get your secrets. This is much harder and more time consuming for an attacker.
+
+It works in 3 easy steps.
+
+### 1. Create .env.ENVIRONMENT files
+
+In addition to your `.env` (development) file, create a `.env.ci`, `.env.staging`, and `.env.production` file.
+
+(Have a custom environment? Just append it's name. For example, `.env.prod`.)
+
+Put your respective secrets in each of those files, just like you always have with your `.env` files. These files should NOT be committed to code.
+
+### 2. Generate .env.vault file
+
+Run the build command to generate your `.env.vault` file.
+
+```
+$ npx dotenv-vault local build
+```
+
+This command will read the contents of each of your `.env.*` files, encrypt them, and inject the encrypted versions into your `.env.vault` file. For example:
+
+```
+# .env.vault (generated with npx dotenv-vault local build)
+DOTENV_VAULT_DEVELOPMENT="X/GOMD7h/Fygjyq3+K2zbdyTBUBVA+mLivaSebqDMnLAencDGu9YvJji"
+DOTENV_VAULT_CI="SNnKvHTezcd0B8L+81lhcig+6GfkRxnlrgS1GG/2tJZ7KghOEJnM"
+DOTENV_VAULT_PRODUCTION="FudgivxdMrCKOKUeN+QieuCAoGiC2MstXL8JU6Pp4ILYu9wEwfqe4ne3e2jcVys="
+DOTENV_VAULT_STAGING="CZXrvrTusPLJlgm62uEppwCKZt6zEr4TGwlP8Z0McJd7I8KBF522JnhT9/8="
+```
+
+Commit your `.env.vault` file safely to code. It SHOULD be committed to code.
+
+### 3. Set DOTENV_KEY
+
+The build command also created a `.env.keys` file for you. This is where your `DOTENV_KEY` decryption keys live per environment.
+
+```
+# DOTENV_KEYs (generated with npx dotenv-vault local build)
+DOTENV_KEY_DEVELOPMENT="dotenv://:key_fc5c0d276e032a1e5ff295f59d7b63db75b0ae1a5a82ad411f4887c23dc78bd1@dotenv.local/vault/.env.vault?environment=development"
+DOTENV_KEY_CI="dotenv://:key_c6bc0b1269b53ee852b269c4ea6d82d82619081f2faddb1e05894fbe90c1ef46@dotenv.local/vault/.env.vault?environment=ci"
+DOTENV_KEY_STAGING="dotenv://:key_09ec9bfe7a4512b71b3b1ab12aa2f843f47b8c9dc7d0d954e206f37ca125da69@dotenv.local/vault/.env.vault?environment=staging"
+```
+
+Go to your web server or cloud platform and set the environment variable `DOTENV_KEY` with the production value. For example, in heroku I'd run the following command.
+
+```
+heroku config:set DOTENV_KEY=dotenv://:key_bfa00115ecacb678ba44376526b2f0b3131aa0060f18de357a63eda08af6a7fe@dotenv.local/vault/.env.vault?environment=production
+```
+
+Then deploy your code. On boot, the `dotenv` library (>= 16.1.0) will see that a `DOTENV_KEY` is set and use its value to decrypt the production contents of the `.env.vault` file and inject them into your process.
+
+No more scattered secrets across multiple platforms and tools.
+
+## 🌴 Examples
 
 See [examples](https://github.com/dotenv-org/examples) of using dotenv with various frameworks, languages, and configurations.
 
@@ -455,3 +518,7 @@ See [CHANGELOG.md](CHANGELOG.md)
 [These npm modules depend on it.](https://www.npmjs.com/browse/depended/dotenv)
 
 Projects that expand it often use the [keyword "dotenv" on npm](https://www.npmjs.com/search?q=keywords:dotenv).
+
+[![Limited Edition Tee Original](https://img.shields.io/badge/Limited%20Edition%20Tee%20%F0%9F%91%95-Original-yellow?labelColor=black&style=plastic)](https://dotenv.gumroad.com/l/original)
+[![Limited Edition Tee Redacted](https://img.shields.io/badge/Limited%20Edition%20Tee%20%F0%9F%91%95-Redacted-gray?labelColor=black&style=plastic)](https://dotenv.gumroad.com/l/redacted)
+

lib/main.js

@@ -1,6 +1,7 @@
 const fs = require('fs')
 const path = require('path')
 const os = require('os')
+const crypto = require('crypto')
 const packageJson = require('../package.json')
 
 const version = packageJson.version
@@ -46,16 +47,147 @@ function parse (src) {
   return obj
 }
 
+function _parseVault (options) {
+  const vaultPath = _vaultPath(options)
+
+  // Parse .env.vault
+  const result = DotenvModule._configDotenv({ path: vaultPath })
+  if (!result.parsed) {
+    throw new Error(`MISSING_DATA: Cannot parse ${vaultPath} for an unknown reason`)
+  }
+
+  // handle scenario for comma separated keys - for use with key rotation
+  // example: DOTENV_KEY="dotenv://:key_1234@dotenv.org/vault/.env.vault?environment=prod,dotenv://:key_7890@dotenv.org/vault/.env.vault?environment=prod"
+  const keys = _dotenvKey().split(',')
+  const length = keys.length
+
+  let decrypted
+  for (let i = 0; i < length; i++) {
+    try {
+      // Get full key
+      const key = keys[i].trim()
+
+      // Get instructions for decrypt
+      const attrs = _instructions(result, key)
+
+      // Decrypt
+      decrypted = DotenvModule.decrypt(attrs.ciphertext, attrs.key)
+
+      break
+    } catch (error) {
+      // last key
+      if (i + 1 >= length) {
+        throw error
+      }
+      // try next key
+    }
+  }
+
+  // Parse decrypted .env string
+  return DotenvModule.parse(decrypted)
+}
+
 function _log (message) {
+  console.log(`[dotenv@${version}][INFO] ${message}`)
+}
+
+function _warn (message) {
+  console.log(`[dotenv@${version}][WARN] ${message}`)
+}
+
+function _debug (message) {
   console.log(`[dotenv@${version}][DEBUG] ${message}`)
 }
 
+function _dotenvKey () {
+  if (process.env.DOTENV_KEY && process.env.DOTENV_KEY.length > 0) {
+    return process.env.DOTENV_KEY
+  }
+
+  return ''
+}
+
+function _instructions (result, dotenvKey) {
+  // Parse DOTENV_KEY. Format is a URI
+  let uri
+  try {
+    uri = new URL(dotenvKey)
+  } catch (error) {
+    if (error.code === 'ERR_INVALID_URL') {
+      throw new Error('INVALID_DOTENV_KEY: Wrong format. Must be in valid uri format like dotenv://:key_1234@dotenv.org/vault/.env.vault?environment=development')
+    }
+
+    throw error
+  }
+
+  // Get decrypt key
+  const key = uri.password
+  if (!key) {
+    throw new Error('INVALID_DOTENV_KEY: Missing key part')
+  }
+
+  // Get environment
+  const environment = uri.searchParams.get('environment')
+  if (!environment) {
+    throw new Error('INVALID_DOTENV_KEY: Missing environment part')
+  }
+
+  // Get ciphertext payload
+  const environmentKey = `DOTENV_VAULT_${environment.toUpperCase()}`
+  const ciphertext = result.parsed[environmentKey] // DOTENV_VAULT_PRODUCTION
+  if (!ciphertext) {
+    throw new Error(`NOT_FOUND_DOTENV_ENVIRONMENT: Cannot locate environment ${environmentKey} in your .env.vault file.`)
+  }
+
+  return { ciphertext, key }
+}
+
+function _vaultPath (options) {
+  let dotenvPath = path.resolve(process.cwd(), '.env')
+
+  if (options && options.path && options.path.length > 0) {
+    dotenvPath = options.path
+  }
+
+  // Locate .env.vault
+  return dotenvPath.endsWith('.vault') ? dotenvPath : `${dotenvPath}.vault`
+}
+
 function _resolveHome (envPath) {
   return envPath[0] === '~' ? path.join(os.homedir(), envPath.slice(1)) : envPath
 }
 
-// Populates process.env from .env file
-function config (options) {
+function _configVault (options) {
+  _log('Loading env from encrypted .env.vault')
+
+  const parsed = DotenvModule._parseVault(options)
+
+  const debug = Boolean(options && options.debug)
+  const override = Boolean(options && options.override)
+
+  // Set process.env
+  for (const key of Object.keys(parsed)) {
+    if (Object.prototype.hasOwnProperty.call(process.env, key)) {
+      if (override === true) {
+        process.env[key] = parsed[key]
+      }
+
+      if (debug) {
+        if (override === true) {
+          _debug(`"${key}" is already defined in \`process.env\` and WAS overwritten`)
+        } else {
+          _debug(`"${key}" is already defined in \`process.env\` and was NOT overwritten`)
+        }
+      }
+    } else {
+      process.env[key] = parsed[key]
+    }
+  }
+
+  return { parsed }
+}
+
+function _configDotenv (options) {
   let dotenvPath = path.resolve(process.cwd(), '.env')
   let encoding = 'utf8'
   const debug = Boolean(options && options.debug)
@@ -72,7 +204,7 @@ function config (options) {
 
   try {
     // Specifying an encoding returns a string instead of a buffer
-    const parsed = DotenvModule.parse(fs.readFileSync(dotenvPath, { encoding }))
+    const parsed = parse(fs.readFileSync(dotenvPath, { encoding }))
 
     Object.keys(parsed).forEach(function (key) {
       if (!Object.prototype.hasOwnProperty.call(process.env, key)) {
@@ -84,9 +216,9 @@ function config (options) {
 
         if (debug) {
           if (override === true) {
-            _log(`"${key}" is already defined in \`process.env\` and WAS overwritten`)
+            _debug(`"${key}" is already defined in \`process.env\` and WAS overwritten`)
           } else {
-            _log(`"${key}" is already defined in \`process.env\` and was NOT overwritten`)
+            _debug(`"${key}" is already defined in \`process.env\` and was NOT overwritten`)
           }
         }
       }
@@ -95,18 +227,77 @@ function config (options) {
     return { parsed }
   } catch (e) {
     if (debug) {
-      _log(`Failed to load ${dotenvPath} ${e.message}`)
+      _debug(`Failed to load ${dotenvPath} ${e.message}`)
     }
 
     return { error: e }
   }
 }
 
+// Populates process.env from .env file
+function config (options) {
+  const vaultPath = _vaultPath(options)
+
+  // fallback to original dotenv if DOTENV_KEY is not set
+  if (_dotenvKey().length === 0) {
+    return DotenvModule._configDotenv(options)
+  }
+
+  // dotenvKey exists but .env.vault file does not exist
+  if (!fs.existsSync(vaultPath)) {
+    _warn(`You set DOTENV_KEY but you are missing a .env.vault file at ${vaultPath}. Did you forget to build it?`)
+
+    return DotenvModule._configDotenv(options)
+  }
+
+  return DotenvModule._configVault(options)
+}
+
+function decrypt (encrypted, keyStr) {
+  const key = Buffer.from(keyStr.slice(-64), 'hex')
+  let ciphertext = Buffer.from(encrypted, 'base64')
+
+  const nonce = ciphertext.slice(0, 12)
+  const authTag = ciphertext.slice(-16)
+  ciphertext = ciphertext.slice(12, -16)
+
+  try {
+    const aesgcm = crypto.createDecipheriv('aes-256-gcm', key, nonce)
+    aesgcm.setAuthTag(authTag)
+    return `${aesgcm.update(ciphertext)}${aesgcm.final()}`
+  } catch (error) {
+    const isRange = error instanceof RangeError
+    const invalidKeyLength = error.message === 'Invalid key length'
+    const decryptionFailed = error.message === 'Unsupported state or unable to authenticate data'
+
+    if (isRange || invalidKeyLength) {
+      const msg = 'INVALID_DOTENV_KEY: It must be 64 characters long (or more)'
+      throw new Error(msg)
+    } else if (decryptionFailed) {
+      const msg = 'DECRYPTION_FAILED: Please check your DOTENV_KEY'
+      throw new Error(msg)
+    } else {
+      console.error('Error: ', error.code)
+      console.error('Error: ', error.message)
+      throw error
+    }
+  }
+}
+
 const DotenvModule = {
+  _configDotenv,
+  _configVault,
+  _parseVault,
   config,
+  decrypt,
   parse
 }
 
+module.exports._configDotenv = DotenvModule._configDotenv
+module.exports._configVault = DotenvModule._configVault
+module.exports._parseVault = DotenvModule._parseVault
 module.exports.config = DotenvModule.config
+module.exports.decrypt = DotenvModule.decrypt
 module.exports.parse = DotenvModule.parse
+
 module.exports = DotenvModule

tests/.env.vault

@@ -0,0 +1 @@
+DOTENV_VAULT_DEVELOPMENT="s7NYXa809k/bVSPwIAmJhPJmEGTtU0hG58hOZy7I0ix6y5HP8LsHBsZCYC/gw5DDFy5DgOcyd18R"