Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add blocking trigger type #4395

Merged
merged 36 commits into from
Apr 12, 2022
Merged
Show file tree
Hide file tree
Changes from 13 commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
25f3d62
tinkering with implementations
colerogers Jan 31, 2022
6fe07de
Merge remote-tracking branch 'origin/master' into colerogers.auth-blo…
colerogers Feb 17, 2022
f616332
adding in cache for discovery
colerogers Feb 21, 2022
8566842
adding in skeleton for service object and identity platform api calls
colerogers Feb 25, 2022
6a4b7b9
merging master
colerogers Feb 25, 2022
43b2316
fix conflicts
colerogers Mar 23, 2022
eb016f7
adding in all the service code
colerogers Mar 29, 2022
9ce55af
formatter
colerogers Mar 29, 2022
801cb5f
rename
colerogers Mar 30, 2022
0b22d42
Merge remote-tracking branch 'origin/master' into colerogers.auth-blo…
colerogers Apr 2, 2022
94c4db4
removing the lookup before endpointToFunction & adding v2 stuff
colerogers Apr 2, 2022
b1cbf95
add tests & changelog
colerogers Apr 3, 2022
bee1b93
adding tsdoc comments
colerogers Apr 3, 2022
ed416c8
addressing pr comments
colerogers Apr 4, 2022
a298aad
fixing conflicts
colerogers Apr 4, 2022
62999db
removing the empty object from gcp/identityPlatform
colerogers Apr 5, 2022
e35f6c0
Merge remote-tracking branch 'origin/master' into colerogers.auth-blo…
colerogers Apr 5, 2022
fc47475
added typing & fixing endpointFromFunction to set securityLevel and u…
colerogers Apr 5, 2022
43516bc
removing changelog entries
colerogers Apr 5, 2022
851c3a3
cleaning up & set default endpoint options to false
colerogers Apr 5, 2022
b674320
adding in inferBlockingDetails
colerogers Apr 5, 2022
bd9a0b9
checking if config changed before calling set blocking config
colerogers Apr 5, 2022
7f4fce2
formatter
colerogers Apr 5, 2022
7b69417
cleaning up
colerogers Apr 6, 2022
320d222
Merge remote-tracking branch 'origin/master' into colerogers.auth-blo…
colerogers Apr 6, 2022
fbd7e3e
adding event type if in fab & changing api to match prod
colerogers Apr 7, 2022
4fff5c0
adding serviceForEndpoint in fab setTrigger
colerogers Apr 7, 2022
153bffb
converting to options record & yanking auth blocking events from even…
colerogers Apr 7, 2022
c671094
removing old comments
colerogers Apr 7, 2022
3c95a35
fixing delete blocking function bug
colerogers Apr 8, 2022
9faf7a8
merging master
colerogers Apr 8, 2022
77de23e
adding a promise queue for trigger registration & deletion
colerogers Apr 8, 2022
e758d2b
merge master
colerogers Apr 8, 2022
bb56d25
making auth blocking service a class & moving trigger queue into it
colerogers Apr 12, 2022
a4bfd25
remove comment
colerogers Apr 12, 2022
a2f66cd
merging master
colerogers Apr 12, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,3 +5,4 @@
- Fixes issue with importing Storage Emulator data exported prior to v10.3.0 (#4358).
- Adds ergonomic improvements to CF3 secret commands to automatically redeploy functions and delete unused secrets (#4130).
- Fixes issue with alpha users setting timeouts (#4381)
- Adds a blocking trigger type (#4395).
33 changes: 32 additions & 1 deletion src/deploy/functions/backend.ts
Original file line number Diff line number Diff line change
Expand Up @@ -131,6 +131,17 @@ export interface TaskQueueTriggered {
taskQueueTrigger: TaskQueueTrigger;
}

export interface BlockingTrigger {
eventType: string;
colerogers marked this conversation as resolved.
Show resolved Hide resolved
accessToken?: boolean;
idToken?: boolean;
refreshToken?: boolean;
colerogers marked this conversation as resolved.
Show resolved Hide resolved
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have no suggestion on how to fix this, but I'm worried that this is not scalable. These should be blocking functions, not auth functions. Maybe we should have a map of features or something?

Copy link
Contributor Author

@colerogers colerogers Apr 7, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@inlined How about we go with a record structure for options and for auth blocking triggers we'd assume they'd use these three options. So the trigger will now be

export interface BlockingTrigger {
  eventType:
    | typeof events.v1.AUTH_BLOCKING_EVENTS[number]
    | typeof events.v2.AUTH_BLOCKING_EVENTS[number];
  options: Record<string, any>;
}

What do you think? The biggest thing is we'd lose out on is type completion in ts

}

export interface BlockingTriggered {
blockingTrigger: BlockingTrigger;
}

/** A user-friendly string for the kind of trigger of an endpoint. */
export function endpointTriggerType(endpoint: Endpoint): string {
if (isScheduleTriggered(endpoint)) {
Expand All @@ -143,6 +154,8 @@ export function endpointTriggerType(endpoint: Endpoint): string {
return endpoint.eventTrigger.eventType;
} else if (isTaskQueueTriggered(endpoint)) {
return "taskQueue";
} else if (isBlockingTriggered(endpoint)) {
return "blocking";
colerogers marked this conversation as resolved.
Show resolved Hide resolved
} else {
throw new Error("Unexpected trigger type for endpoint " + JSON.stringify(endpoint));
}
Expand Down Expand Up @@ -227,7 +240,8 @@ export type Triggered =
| CallableTriggered
| EventTriggered
| ScheduleTriggered
| TaskQueueTriggered;
| TaskQueueTriggered
| BlockingTriggered;

/** Whether something has an HttpsTrigger */
export function isHttpsTriggered(triggered: Triggered): triggered is HttpsTriggered {
Expand All @@ -254,6 +268,11 @@ export function isTaskQueueTriggered(triggered: Triggered): triggered is TaskQue
return {}.hasOwnProperty.call(triggered, "taskQueueTrigger");
}

/** Whether something has a BlockingTrigger */
export function isBlockingTriggered(triggered: Triggered): triggered is BlockingTriggered {
return {}.hasOwnProperty.call(triggered, "blockingTrigger");
}

/**
* An endpoint that serves traffic to a stack of services.
* For now, this is always a Cloud Function. Future iterations may use complex
Expand Down Expand Up @@ -285,6 +304,16 @@ export interface RequiredAPI {
api: string;
}

interface ResourceOptions {
identityPlatform?: {
// auth blocking trigger options at the resource level
// these are set as the OR of each auth blocking trigger
accessToken: boolean;
idToken: boolean;
refreshToken: boolean;
};
}

/** An API agnostic definition of an entire deployment a customer has or wants. */
export interface Backend {
/**
Expand All @@ -294,6 +323,7 @@ export interface Backend {
environmentVariables: EnvironmentVariables;
// region -> id -> Endpoint
endpoints: Record<string, Record<string, Endpoint>>;
resourceOptions: ResourceOptions;
}

/**
Expand All @@ -306,6 +336,7 @@ export function empty(): Backend {
requiredAPIs: [],
endpoints: {},
environmentVariables: {},
resourceOptions: {},
};
}

Expand Down
2 changes: 2 additions & 0 deletions src/deploy/functions/prepare.ts
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ import { ensureTriggerRegions } from "./triggerRegionHelper";
import { ensureServiceAgentRoles } from "./checkIam";
import { FirebaseError } from "../../error";
import { normalizeAndValidate } from "../../functions/projectConfig";
import { serviceForEndpoint } from "./services";

function hasUserConfig(config: Record<string, unknown>): boolean {
// "firebase" key is always going to exist in runtime config.
Expand Down Expand Up @@ -186,6 +187,7 @@ export function inferDetailsFromExisting(
usedDotenv: boolean
): void {
for (const wantE of backend.allEndpoints(want)) {
serviceForEndpoint(wantE).copyResourceOptionsToEndpoint(wantE as any, want);
colerogers marked this conversation as resolved.
Show resolved Hide resolved
colerogers marked this conversation as resolved.
Show resolved Hide resolved
const haveE = have.endpoints[wantE.region]?.[wantE.id];
if (!haveE) {
continue;
Expand Down
45 changes: 42 additions & 3 deletions src/deploy/functions/release/fabricator.ts
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ import * as reporter from "./reporter";
import * as run from "../../../gcp/run";
import * as scheduler from "../../../gcp/cloudscheduler";
import * as utils from "../../../utils";
import * as authBlocking from "../services/authBlocking";
colerogers marked this conversation as resolved.
Show resolved Hide resolved

// TODO: Tune this for better performance.
const gcfV1PollerOptions: Omit<poller.OperationPollerOptions, "operationResourceName"> = {
Expand Down Expand Up @@ -54,6 +55,7 @@ export interface FabricatorArgs {
const rethrowAs =
<T>(endpoint: backend.Endpoint, op: reporter.OperationType) =>
(err: unknown): T => {
logger.error((err as Error).message);
colerogers marked this conversation as resolved.
Show resolved Hide resolved
throw new reporter.DeploymentError(endpoint, op, err);
};

Expand Down Expand Up @@ -164,7 +166,7 @@ export class Fabricator {
assertExhaustive(endpoint.platform);
}

await this.setTrigger(endpoint);
await this.setTrigger(endpoint, false);
}

async updateEndpoint(update: planner.EndpointUpdate, scraper: SourceTokenScraper): Promise<void> {
Expand All @@ -183,7 +185,7 @@ export class Fabricator {
assertExhaustive(update.endpoint.platform);
}

await this.setTrigger(update.endpoint);
await this.setTrigger(update.endpoint, true);
}

async deleteEndpoint(endpoint: backend.Endpoint): Promise<void> {
Expand Down Expand Up @@ -248,6 +250,13 @@ export class Fabricator {
})
.catch(rethrowAs(endpoint, "set invoker"));
}
} else if (backend.isBlockingTriggered(endpoint)) {
// set invoker to all users
colerogers marked this conversation as resolved.
Show resolved Hide resolved
await this.executor
.run(async () => {
await gcf.setInvokerCreate(endpoint.project, backend.functionName(endpoint), ["public"]);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here and elsewhere, it's public because it's an auth blocking function, not because it's a blocking function. Please make the code forwards compatible to other blocking trigger types.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a check

if (backend.isBlockingTriggered(endpoint) && AUTH_BLOCKING_EVENTS.includes(endpoint.blockingTrigger.eventType)) { ... }

})
.catch(rethrowAs(endpoint, "set invoker"));
}
}

Expand Down Expand Up @@ -317,6 +326,11 @@ export class Fabricator {
})
.catch(rethrowAs(endpoint, "set invoker"));
}
} else if (backend.isBlockingTriggered(endpoint)) {
// set invoker to all users
colerogers marked this conversation as resolved.
Show resolved Hide resolved
await this.executor
.run(() => run.setInvokerCreate(endpoint.project, serviceName, ["public"]))
.catch(rethrowAs(endpoint, "set invoker"));
}

const mem = endpoint.availableMemoryMb || backend.DEFAULT_MEMORY;
Expand Down Expand Up @@ -354,6 +368,8 @@ export class Fabricator {
invoker = endpoint.httpsTrigger.invoker;
} else if (backend.isTaskQueueTriggered(endpoint)) {
invoker = endpoint.taskQueueTrigger.invoker;
} else if (backend.isBlockingTriggered(endpoint)) {
invoker = ["public"];
colerogers marked this conversation as resolved.
Show resolved Hide resolved
}
if (invoker) {
await this.executor
Expand Down Expand Up @@ -395,6 +411,8 @@ export class Fabricator {
invoker = endpoint.httpsTrigger.invoker;
} else if (backend.isTaskQueueTriggered(endpoint)) {
invoker = endpoint.taskQueueTrigger.invoker;
} else if (backend.isBlockingTriggered(endpoint)) {
invoker = ["public"];
}
if (invoker) {
await this.executor
Expand Down Expand Up @@ -460,7 +478,7 @@ export class Fabricator {

// Set/Delete trigger is responsible for wiring up a function with any trigger not owned
// by the GCF API. This includes schedules, task queues, and blocking function triggers.
async setTrigger(endpoint: backend.Endpoint): Promise<void> {
async setTrigger(endpoint: backend.Endpoint, update: boolean): Promise<void> {
if (backend.isScheduleTriggered(endpoint)) {
if (endpoint.platform === "gcfv1") {
await this.upsertScheduleV1(endpoint);
Expand All @@ -472,6 +490,8 @@ export class Fabricator {
assertExhaustive(endpoint.platform);
} else if (backend.isTaskQueueTriggered(endpoint)) {
await this.upsertTaskQueue(endpoint);
} else if (backend.isBlockingTriggered(endpoint)) {
await this.registerBlockingTrigger(endpoint, update);
}
}

Expand All @@ -487,6 +507,8 @@ export class Fabricator {
assertExhaustive(endpoint.platform);
} else if (backend.isTaskQueueTriggered(endpoint)) {
await this.disableTaskQueue(endpoint);
} else if (backend.isBlockingTriggered(endpoint)) {
await this.unregisterBlockingTrigger(endpoint);
}
}

Expand Down Expand Up @@ -519,6 +541,15 @@ export class Fabricator {
}
}

async registerBlockingTrigger(
endpoint: backend.Endpoint & backend.BlockingTriggered,
update: boolean
): Promise<void> {
await this.executor
.run(() => authBlocking.registerAuthBlockingTriggerToIdentityPlatform(endpoint, update))
.catch(rethrowAs(endpoint, "register blocking trigger"));
}

async deleteScheduleV1(endpoint: backend.Endpoint & backend.ScheduleTriggered): Promise<void> {
const job = scheduler.jobFromEndpoint(endpoint, this.appEngineLocation);
await this.executor
Expand Down Expand Up @@ -546,6 +577,14 @@ export class Fabricator {
.catch(rethrowAs(endpoint, "disable task queue"));
}

async unregisterBlockingTrigger(
endpoint: backend.Endpoint & backend.BlockingTriggered
): Promise<void> {
await this.executor
.run(() => authBlocking.unregisterAuthBlockingTriggerFromIdentityPlatform(endpoint))
.catch(rethrowAs(endpoint, "unregister blocking trigger"));
}

logOpStart(op: string, endpoint: backend.Endpoint): void {
const runtime = getHumanFriendlyRuntimeName(endpoint.runtime);
const label = helper.getFunctionLabel(endpoint);
Expand Down
2 changes: 2 additions & 0 deletions src/deploy/functions/release/planner.ts
Original file line number Diff line number Diff line change
Expand Up @@ -234,6 +234,8 @@ export function checkForIllegalUpdate(want: backend.Endpoint, have: backend.Endp
return "a scheduled";
} else if (backend.isTaskQueueTriggered(e)) {
return "a task queue";
} else if (backend.isBlockingTriggered(e)) {
return "a blocking";
colerogers marked this conversation as resolved.
Show resolved Hide resolved
}
// Unfortunately TypeScript isn't like Scala and I can't prove to it
// that all cases have been handled
Expand Down
8 changes: 7 additions & 1 deletion src/deploy/functions/release/reporter.ts
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,9 @@ export type OperationType =
| "create topic"
| "delete topic"
| "set invoker"
| "set concurrency";
| "set concurrency"
| "register blocking trigger"
| "unregister blocking trigger";

/** An error with a deployment phase. */
export class DeploymentError extends Error {
Expand Down Expand Up @@ -245,5 +247,9 @@ export function triggerTag(endpoint: backend.Endpoint): string {
return `${prefix}.https`;
}

if (backend.isBlockingTriggered(endpoint)) {
return `${prefix}.blocking`;
}

return endpoint.eventTrigger.eventType;
}
14 changes: 14 additions & 0 deletions src/deploy/functions/runtimes/discovery/v1alpha1.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ export type ManifestEndpoint = backend.ServiceConfiguration &
Partial<backend.CallableTriggered> &
Partial<backend.EventTriggered> &
Partial<backend.TaskQueueTriggered> &
Partial<backend.BlockingTriggered> &
Partial<backend.ScheduleTriggered> & {
region?: string[];
entryPoint: string;
Expand Down Expand Up @@ -93,6 +94,7 @@ function parseEndpoints(
eventTrigger: "object",
scheduleTrigger: "object",
taskQueueTrigger: "object",
blockingTrigger: "object",
});
let triggerCount = 0;
if (ep.httpsTrigger) {
Expand All @@ -110,6 +112,9 @@ function parseEndpoints(
if (ep.taskQueueTrigger) {
triggerCount++;
}
if (ep.blockingTrigger) {
triggerCount++;
}
if (!triggerCount) {
throw new FirebaseError("Expected trigger in endpoint " + id);
}
Expand Down Expand Up @@ -180,6 +185,15 @@ function parseEndpoints(
});
}
triggered = { taskQueueTrigger: ep.taskQueueTrigger };
} else if (backend.isBlockingTriggered(ep)) {
requireKeys(prefix + ".blockingTrigger", ep.blockingTrigger, "eventType");
assertKeyTypes(prefix + ".blockingTrigger", ep.blockingTrigger, {
eventType: "string",
accessToken: "boolean",
idToken: "boolean",
refreshToken: "boolean",
});
triggered = { blockingTrigger: ep.blockingTrigger };
} else {
throw new FirebaseError(
`Do not recognize trigger type for endpoint ${id}. Try upgrading ` +
Expand Down
24 changes: 23 additions & 1 deletion src/deploy/functions/runtimes/node/parseTriggers.ts
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,12 @@ export interface TriggerAnnotation {
};
invoker?: string[];
};
blockingTrigger?: {
eventType: string;
accessToken?: boolean;
idToken?: boolean;
refreshToken?: boolean;
};
failurePolicy?: {};
schedule?: ScheduleAnnotation;
timeZone?: string;
Expand Down Expand Up @@ -180,7 +186,10 @@ export function addResourcesToBackend(

// +!! is 1 for truthy values and 0 for falsy values
const triggerCount =
+!!annotation.httpsTrigger + +!!annotation.eventTrigger + +!!annotation.taskQueueTrigger;
+!!annotation.httpsTrigger +
+!!annotation.eventTrigger +
+!!annotation.taskQueueTrigger +
+!!annotation.blockingTrigger;
if (triggerCount !== 1) {
throw new FirebaseError(
"Unexpected annotation generated by the Firebase Functions SDK. This should never happen."
Expand Down Expand Up @@ -211,6 +220,19 @@ export function addResourcesToBackend(
reason: "Needed for scheduled functions.",
});
triggered = { scheduleTrigger: annotation.schedule };
} else if (annotation.blockingTrigger) {
want.requiredAPIs.push({
api: "identityplatform.googleapis.com",
reason: "Needed for auth blocking functions.",
});
triggered = {
blockingTrigger: {
eventType: annotation.blockingTrigger!.eventType,
accessToken: annotation.blockingTrigger.accessToken || false,
idToken: annotation.blockingTrigger.idToken || false,
refreshToken: annotation.blockingTrigger.refreshToken || false,
},
};
} else {
triggered = {
eventTrigger: {
Expand Down