Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: firebase/firebase-tools
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: v10.1.5
Choose a base ref
...
head repository: firebase/firebase-tools
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: v10.2.0
Choose a head ref
  • 6 commits
  • 61 files changed
  • 5 contributors

Commits on Feb 8, 2022

  1. Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
    Copy the full SHA
    c139879 View commit details

Commits on Feb 9, 2022

  1. Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
    Copy the full SHA
    6bc00b4 View commit details

Commits on Feb 10, 2022

  1. Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
    Copy the full SHA
    f0dda1f View commit details
  2. Improves login experience when no localhost is available. (#4147)

    * Proof-of-concept for signing in remotely via auth proxy server.
    
    * Fix lint error
    
    * Update output of --remote
    
    * Replace --no-localhost with remote login.
    
    * Remove vestiges of --remote
    
    * CHANGELOG, fixes emu test.
    
    * Fix track import.
    
    * Wrap auth code error in better message.
    
    Co-authored-by: Bryan Kendall <bkend@google.com>
    mbleigh and bkendall authored Feb 10, 2022

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
    Copy the full SHA
    b03a51b View commit details
  3. Release CF3's support for environment variables and secrets (#4149)

    * Use runtime delegate to parse function triggers in the Functions Emulator (#4012)
    
    Today, Functions Emulator parses function trigger from source as follows:
    
    1) Spin up an instance of Functions runtime process
    2) Invoke a "function" which triggers a path that parses the trigger by calling out to `extractTriggers.js`
    3) Send parsed triggers via IPC from runtime to emulator. Emulator now knows about the triggers.
    
    This has the advantage of running the trigger parsing in the emulated runtime (which properly mocks out calls to the DB, applies network filtering, uses the same node version when possible, etc.) but has the disadvantage of complicating the runtime implementation as well as diverging from how the triggers are parsed in `firebase deploy`.
    
    Using runtime delegate, we have:
    
    1) Use runtime delegate to discover the delegate appropriate for function source (i.e. Node, but in the future can be some other runtime)
    2) Spin up a node subprocess to parse trigger. Emulator now knows about the triggers.
    
    i.e. the same procedure used during `firebase deploy`
    
    By using runtime delegate, we align the function deploy to production and to the emulated environment and simplify the runtime code a bit. This also puts us into a good position in the future when we make the function deploy process a little more complex, e.g. params and secrets support.
    
    * Slim down Functions Emulator Runtime (i.e. args sent over to emulated functions) (#4105)
    
    Now that we've simplified the Functions Emulator to separate out process for loading triggers from the one running the trigger, we can slim down [`FunctionsRuntimeBundle`](https://github.com/firebase/firebase-tools/blob/2e68803f994dbe4f72eb0965dd6a12e7a043b597/src/emulator/functionsEmulatorShared.ts#L53-L88) that is passed between the Functions Emulator and the Functions Runtime process.
    
    This change removes almost all payload attributes in the Functions Runtime Bundle except `proto`. This is nice - we are getting very close to the payload that's passed to a production function instance. We have to leave couple of things like socketpath and debug features - this will probably be removed when we move over to pure-http based protocol (socket) and SDK based debug feature enablement. This could be worked later when I have a little more time!
    
    One more change - we pass around the whole trigger definition in the Functions Emulator instead of pieces of it. This makes it easier to do something else I'm doing... (secret emulator) in the subsequent PR.
    
    * CF3 Secrets Support (#3959)
    
    Support deploying secret environment variables on a function.
    
    Prior to deploying functions with secret configuration, the CLI will run somewhat comprehensive validation to ensure that secret config will work when deployed, e.g.
    
    1. Secret version exists.
    2. Secret version is in ENABLED state.
    3. Secret version can be access by the runtime service account.
    
    We do this since the GCF doesn't do the same level of validation and instead repeatedly fail to spin up a new instance with an invalid secret config. This often results on super long deploys (probably until some master timeout is met for function instance deploy).
    
    I took the opportunity to refactor the code a little to group various "ensure" and "validate" used in function deploys in their own files.
    
    Emulator support for secrets will come in a separate PR.
    
    * Add new command (functions:secrets:set) for creating secrets to be used for CF3. (#4021)
    
    One of several family of commands to be implemented for managing secrets for CF3.
    
    `functions:secrets:set` command is used to create a new secret version in Secret Manager. If a secret doesn't exist, a secret will be created before adding a new version.
    
    To guide users to our recommended best practices, we will only allow users to create secrets in `UPPER_SNAKE_CASE` - this makes it more obvious how these secrets can be accessed at runtime (via environment variable of the same name).
    
    Usage:
    
    ```
    $ echo SHHHH > SECRET_FILE
    $ firebase functions:secrets:set MY_SECRET --data-file=SECRET_FILE
    ✔  Created a new secret version projects/my-project/secrets/MY_SECRET/versions/0
    i  Please deploy your functions for the change to take effect by running:
        firebase deploy --only functions
    
    // Calling set on existing secret name will create a new version.
    $ echo SHHHHHHHH > SECRET_FILEE
    $ firebase functions:secrets:set MY_SECRET --data-file=SECRET_FILE
    ✔  Created a new secret version projects/my-project/secrets/MY_SECRET/versions/1
    i  Please deploy your functions for the change to take effect by running
        firebase deploy --only functions
    
    // "-" as STDIN is supported but discouraged since it will leave the secret in shell history
    $ echo SHHHHHHHHHH | firebase functions:secrets:set --data-file=- MY_SECRET
    ✔  Created a new secret version projects/my-project/secrets/MY_SECRET/versions/2
    i  Please deploy your functions for the change to take effect by running
        firebase deploy --only functions
    
    
    // Without --data-file flag, begin interactive prompt to take user input
    $ firebase functions:secrets:set MY_SECRET
    ? Enter a value for MY_SECRET [input is hidden]:
    ✔  Created a new secret version projects/my-project/secrets/MY_SECRET/versions/3
    i  Please deploy your functions for the change to take effect by running:
        firebase deploy --only functions
    
    ```
    
    * Add functions:secrets:{access, destroy, get} commands. (#4026)
    
    Follow up #4021 to add other management commands for CF3 secrets.
    
    Note that `destroy` commands can be improved by making sure we don't accidentally delete secrets versions currently in use (which would immediately break the function!). I'll add these feature in a follow up PR when we finish reviewing the PR w/ `prune` command.
    
    * Add command to prune unused secrets (#4108)
    
    Each active secret version cost money. To help save cost on using Secret Manager, we add `functions:secrets:prune` command which:
    
    1) Looks up all secret versions from secrets marked with label "firebase-managed". All secrets created using the Firebase CLI will have this label.
    
    2) Look up all secret bindings for CF3 function instance.
    
    3) Figure out which secret version isn't currently being used.
    
    Since destroying a secret version is irrevocable and immediately breaking for clients that depend on it, we will always ask for a confirmation for the destroy operations (and not support -f flag).
    
    Note that we now query `v1` of Secret Manager since `v1beta` does not offer filtering by labels.
    
    * Add support for secrets in the Functions Emulator (#4106)
    
    Emulator will now recognize function triggers with secret environments and ensure that secret environment variables are populated in the emulated runtime.
    
    Secrets in Functions Emulator can come from 2 sources:
    
    1) From local override file (`.secret.local`).
    
    2) From Google Cloud Secret Manager. In this case, default application credentials (i.e. credentials used in Firebase CLI) will be used to fetch the secret from GCP.
     
    As suspected, (1) take precedence over (2). If accessing secret from GCP fails for any reason, the Emulator logs, but does not throw, the failed attempt and proceeds to execute the functions code.
    
    Some refactoring changes needed to be in the Emulator:
    
    * Some functions turned into async.
    * We pass around the whole trigger in more places.
    
    * Remove preview flag, add option to disable dotenv support. (#4022)
    
    Preparing for launching dotenv support for CF3.
    
    At launch, CF3 environment variables support will default to picking up dotenv file if any, without need for preview flag.
    
    * Add EXT_ as a reserved environment variable prefix (#4148)
    
    Firebase Extensions use environment variables with `EXT_` for many of their "first class" environment variable keys. We will add it to the reserved prefix list before releasing the dotenv support for all users.
    taeold authored Feb 10, 2022

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
    Copy the full SHA
    0c747c9 View commit details
  4. 10.2.0

    google-oss-bot committed Feb 10, 2022

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
    Copy the full SHA
    5f97f8e View commit details
Showing with 2,666 additions and 881 deletions.
  1. +2 −1 CHANGELOG.md
  2. +2 −2 npm-shrinkwrap.json
  3. +1 −1 package.json
  4. +1 −70 scripts/emulator-tests/fixtures.ts
  5. +87 −8 scripts/emulator-tests/functionsEmulator.spec.ts
  6. +53 −37 scripts/emulator-tests/functionsEmulatorRuntime.spec.ts
  7. +3 −0 scripts/firepit-builder/Dockerfile
  8. +1 −1 scripts/firepit-builder/pipeline.js
  9. +1 −0 src/api.js
  10. +82 −29 src/auth.ts
  11. +2 −1 src/commands/ext-configure.ts
  12. +2 −0 src/commands/ext-install.ts
  13. +2 −0 src/commands/ext-uninstall.ts
  14. +2 −0 src/commands/ext-update.ts
  15. +19 −0 src/commands/functions-secrets-access.ts
  16. +51 −0 src/commands/functions-secrets-destroy.ts
  17. +23 −0 src/commands/functions-secrets-get.ts
  18. +68 −0 src/commands/functions-secrets-prune.ts
  19. +56 −0 src/commands/functions-secrets-set.ts
  20. +7 −3 src/commands/index.js
  21. +1 −4 src/commands/login.ts
  22. +21 −0 src/deploy/functions/backend.ts
  23. +162 −0 src/deploy/functions/ensure.ts
  24. +0 −61 src/deploy/functions/ensureCloudBuildEnabled.ts
  25. +15 −31 src/deploy/functions/prepare.ts
  26. +1 −0 src/deploy/functions/runtimes/discovery/v1alpha1.ts
  27. +2 −1 src/deploy/functions/runtimes/index.ts
  28. +15 −0 src/deploy/functions/runtimes/node/parseTriggers.ts
  29. +88 −0 src/deploy/functions/validate.ts
  30. +3 −4 src/emulator/controller.ts
  31. +9 −1 src/emulator/emulatorLogger.ts
  32. +154 −150 src/emulator/functionsEmulator.ts
  33. +133 −116 src/emulator/functionsEmulatorRuntime.ts
  34. +84 −33 src/emulator/functionsEmulatorShared.ts
  35. +2 −12 src/emulator/functionsEmulatorShell.ts
  36. +1 −1 src/emulator/functionsRuntimeWorker.ts
  37. +1 −1 src/extensions/askUserForParam.ts
  38. +74 −0 src/extensions/diagnose.ts
  39. +9 −0 src/extensions/extensionsHelper.ts
  40. +1 −1 src/extensions/secretsUtils.ts
  41. +4 −8 src/functions/env.ts
  42. +170 −0 src/functions/secrets.ts
  43. +5 −1 src/gcp/cloudfunctions.ts
  44. +273 −62 src/gcp/secretManager.ts
  45. +0 −2 src/previews.ts
  46. +2 −2 src/serve/functions.ts
  47. +7 −0 src/test/deploy/functions/backend.spec.ts
  48. +267 −0 src/test/deploy/functions/ensure.spec.ts
  49. +0 −134 src/test/deploy/functions/ensureCloudBuildEnabled.ts
  50. +14 −14 src/test/deploy/functions/prepare.spec.ts
  51. +24 −0 src/test/deploy/functions/runtimes/node/parseTriggers.spec.ts
  52. +0 −1 src/test/deploy/functions/runtimes/node/validate.spec.ts
  53. +119 −0 src/test/deploy/functions/validate.spec.ts
  54. +1 −70 src/test/emulators/fixtures.ts
  55. +2 −6 src/test/emulators/functionsRuntimeWorker.spec.ts
  56. +96 −0 src/test/extensions/diagnose.spec.ts
  57. +2 −2 src/test/extensions/secretUtils.spec.ts
  58. +8 −10 src/test/functions/env.spec.ts
  59. +287 −0 src/test/functions/secrets.spec.ts
  60. +131 −0 src/test/gcp/secretManager.spec.ts
  61. +13 −0 src/utils.ts
3 changes: 2 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1 +1,2 @@
- Fixes bug where local extension installation and ext:publish failed to upload source to GCS bucket.
- Improves experience for `firebase login --no-localhost`.
- Add support for specifying environment variable of CF3 function using dotenv.
4 changes: 2 additions & 2 deletions npm-shrinkwrap.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "firebase-tools",
"version": "10.1.5",
"version": "10.2.0",
"description": "Command-Line Interface for Firebase",
"main": "./lib/index.js",
"bin": {
71 changes: 1 addition & 70 deletions scripts/emulator-tests/fixtures.ts
Original file line number Diff line number Diff line change
@@ -6,17 +6,6 @@ export const TIMEOUT_MED = 5000;
export const MODULE_ROOT = findModuleRoot("firebase-tools", __dirname);
export const FunctionRuntimeBundles: { [key: string]: FunctionsRuntimeBundle } = {
onCreate: {
adminSdkConfig: {
databaseURL: "https://fake-project-id-default-rtdb.firebaseio.com",
storageBucket: "fake-project-id.appspot.com",
},
emulators: {
firestore: {
host: "localhost",
port: 8080,
},
},
cwd: MODULE_ROOT,
proto: {
data: {
value: {
@@ -41,22 +30,8 @@ export const FunctionRuntimeBundles: { [key: string]: FunctionsRuntimeBundle } =
},
},
},
triggerId: "us-central1-function_id",
targetName: "function_id",
projectId: "fake-project-id",
},
onWrite: {
adminSdkConfig: {
databaseURL: "https://fake-project-id-default-rtdb.firebaseio.com",
storageBucket: "fake-project-id.appspot.com",
},
emulators: {
firestore: {
host: "localhost",
port: 8080,
},
},
cwd: MODULE_ROOT,
proto: {
data: {
value: {
@@ -81,22 +56,8 @@ export const FunctionRuntimeBundles: { [key: string]: FunctionsRuntimeBundle } =
},
},
},
triggerId: "us-central1-function_id",
targetName: "function_id",
projectId: "fake-project-id",
},
onDelete: {
adminSdkConfig: {
databaseURL: "https://fake-project-id-default-rtdb.firebaseio.com",
storageBucket: "fake-project-id.appspot.com",
},
emulators: {
firestore: {
host: "localhost",
port: 8080,
},
},
cwd: MODULE_ROOT,
proto: {
data: {
oldValue: {
@@ -121,22 +82,8 @@ export const FunctionRuntimeBundles: { [key: string]: FunctionsRuntimeBundle } =
},
},
},
triggerId: "us-central1-function_id",
targetName: "function_id",
projectId: "fake-project-id",
},
onUpdate: {
adminSdkConfig: {
databaseURL: "https://fake-project-id-default-rtdb.firebaseio.com",
storageBucket: "fake-project-id.appspot.com",
},
emulators: {
firestore: {
host: "localhost",
port: 8080,
},
},
cwd: MODULE_ROOT,
proto: {
data: {
oldValue: {
@@ -173,25 +120,9 @@ export const FunctionRuntimeBundles: { [key: string]: FunctionsRuntimeBundle } =
timestamp: "2019-05-15T16:21:15.148831Z",
},
},
triggerId: "us-central1-function_id",
targetName: "function_id",
projectId: "fake-project-id",
},
onRequest: {
adminSdkConfig: {
databaseURL: "https://fake-project-id-default-rtdb.firebaseio.com",
storageBucket: "fake-project-id.appspot.com",
},
emulators: {
firestore: {
host: "localhost",
port: 8080,
},
},
cwd: MODULE_ROOT,
triggerId: "us-central1-function_id",
targetName: "function_id",
projectId: "fake-project-id",
proto: {},
},
};

95 changes: 87 additions & 8 deletions scripts/emulator-tests/functionsEmulator.spec.ts
Original file line number Diff line number Diff line change
@@ -1,9 +1,13 @@
import * as fs from "fs";

import { expect } from "chai";
import * as express from "express";
import * as sinon from "sinon";
import * as supertest from "supertest";
import * as winston from "winston";
import * as logform from "logform";

import { SignatureType } from "../../src/emulator/functionsEmulatorShared";
import { EmulatedTriggerDefinition } from "../../src/emulator/functionsEmulatorShared";
import {
EmulatableBackend,
FunctionsEmulator,
@@ -14,8 +18,7 @@ import { RuntimeWorker } from "../../src/emulator/functionsRuntimeWorker";
import { TIMEOUT_LONG, TIMEOUT_MED, MODULE_ROOT } from "./fixtures";
import { logger } from "../../src/logger";
import * as registry from "../../src/emulator/registry";
import * as winston from "winston";
import * as logform from "logform";
import * as secretManager from "../../src/gcp/secretManager";

if ((process.env.DEBUG || "").toLowerCase().includes("spec")) {
const dropLogLevels = (info: logform.TransformableInfo) => info.message;
@@ -32,6 +35,7 @@ if ((process.env.DEBUG || "").toLowerCase().includes("spec")) {

const functionsEmulator = new FunctionsEmulator({
projectId: "fake-project-id",
projectDir: MODULE_ROOT,
emulatableBackends: [
{
functionsDir: MODULE_ROOT,
@@ -96,6 +100,23 @@ functionsEmulator.setTriggersForTesting(
httpsTrigger: {},
labels: {},
},
{
platform: "gcfv1",
name: "secrets_function_id",
id: "us-central1-secrets_function_id",
region: "us-central1",
entryPoint: "secrets_function_id",
secretEnvironmentVariables: [
{
projectId: "fake-project-id",
secret: "MY_SECRET",
key: "MY_SECRET",
version: "1",
},
],
httpsTrigger: {},
labels: {},
},
],
testBackend
);
@@ -108,13 +129,11 @@ function useFunctions(triggers: () => {}): void {
// eslint-disable-next-line @typescript-eslint/unbound-method
functionsEmulator.startFunctionRuntime = (
backend: EmulatableBackend,
triggerId: string,
targetName: string,
triggerType: SignatureType,
trigger: EmulatedTriggerDefinition,
proto?: any,
runtimeOpts?: InvokeRuntimeOpts
): RuntimeWorker => {
return startFunctionRuntime(testBackend, triggerId, targetName, triggerType, proto, {
): Promise<RuntimeWorker> => {
return startFunctionRuntime(testBackend, trigger, proto, {
nodeBinary: process.execPath,
serializedTriggers,
});
@@ -700,4 +719,64 @@ describe("FunctionsEmulator-Hub", () => {
});
}).timeout(TIMEOUT_MED);
});

describe("secrets", () => {
let readFileSyncStub: sinon.SinonStub;
let accessSecretVersionStub: sinon.SinonStub;

beforeEach(() => {
readFileSyncStub = sinon.stub(fs, "readFileSync").throws("Unexpected call");
accessSecretVersionStub = sinon
.stub(secretManager, "accessSecretVersion")
.rejects("Unexpected call");
});

afterEach(() => {
readFileSyncStub.restore();
accessSecretVersionStub.restore();
});

it("should load secret values from local secrets file if one exists", async () => {
readFileSyncStub.returns("MY_SECRET=local");

useFunctions(() => {
return {
secrets_function_id: require("firebase-functions").https.onRequest(
(req: express.Request, res: express.Response) => {
res.json({ secret: process.env.MY_SECRET });
}
),
};
});

await supertest(functionsEmulator.createHubServer())
.get("/fake-project-id/us-central1/secrets_function_id")
.expect(200)
.then((res) => {
expect(res.body.secret).to.equal("local");
});
}).timeout(TIMEOUT_LONG);

it("should try to access secret values from Secret Manager", async () => {
readFileSyncStub.throws({ code: "ENOENT" });
accessSecretVersionStub.resolves("secretManager");

useFunctions(() => {
return {
secrets_function_id: require("firebase-functions").https.onRequest(
(req: express.Request, res: express.Response) => {
res.json({ secret: process.env.MY_SECRET });
}
),
};
});

await supertest(functionsEmulator.createHubServer())
.get("/fake-project-id/us-central1/secrets_function_id")
.expect(200)
.then((res) => {
expect(res.body.secret).to.equal("secretManager");
});
}).timeout(TIMEOUT_LONG);
});
});
Loading