Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Webhooks for when renderMediaOnLambda() is finished #1369

Merged
merged 27 commits into from
Oct 10, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
16e857a
initial draft for webhook support
paulkuhle Oct 4, 2022
ecd348e
fix test for timeout integration test
paulkuhle Oct 4, 2022
b7610c9
Merge branch 'main' into pr/1369
JonnyBurger Oct 5, 2022
b8b8bc3
Merge branch 'main' into pr/1369
JonnyBurger Oct 5, 2022
e7d6495
Update old-version.test.ts
JonnyBurger Oct 5, 2022
a90dd12
lower timeout, it's too fast to pass
JonnyBurger Oct 5, 2022
ffd7942
log errors if webhooks failed
JonnyBurger Oct 5, 2022
8f50dce
use http client to make request
JonnyBurger Oct 5, 2022
e179502
add a 5 second timeout
JonnyBurger Oct 5, 2022
4c1dfbf
integrate feedback and finish documentation
paulkuhle Oct 9, 2022
938a5d4
fix integration test with new x-remotion-mode header
paulkuhle Oct 9, 2022
1413d70
fix webhook example endpoint in docs
paulkuhle Oct 9, 2022
b222171
Merge branch 'main' into pr/1369
JonnyBurger Oct 10, 2022
a8f56dc
update webhook payload
JonnyBurger Oct 10, 2022
f1f2436
validateWebhookSignature() endpoint
JonnyBurger Oct 10, 2022
c0b8663
add more see also links
JonnyBurger Oct 10, 2022
9a87675
webhook shape
JonnyBurger Oct 10, 2022
4e3a915
update webhook shape
JonnyBurger Oct 10, 2022
c648dbd
CLI params
JonnyBurger Oct 10, 2022
840dd66
document CLI params
JonnyBurger Oct 10, 2022
c8e41db
fix payload not being sent
JonnyBurger Oct 10, 2022
eb81308
pass content-length when invoking webhook
JonnyBurger Oct 10, 2022
2287536
Update lambda.tsx
JonnyBurger Oct 10, 2022
25c6fe4
fix test
JonnyBurger Oct 10, 2022
5072485
fix next js types
JonnyBurger Oct 10, 2022
fcad0ca
finetuning
JonnyBurger Oct 10, 2022
748ef49
headers
JonnyBurger Oct 10, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
12 changes: 12 additions & 0 deletions packages/docs/components/Spinner.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
import React from "react";

const css: React.CSSProperties = {
border: "10px solid #f3f3f3",
borderTop: "10px solid var(--ifm-color-primary)",
borderRadius: "50%",
width: "80px",
height: "80px",
animation: "spin 1s linear infinite",
};

export const Spinner: React.FC = () => <div style={css} />;
4 changes: 4 additions & 0 deletions packages/docs/components/TableOfContents/lambda.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,10 @@ export const TableOfContents: React.FC = () => {
<strong>simulatePermissions()</strong>
<div>Ensure permissions are correctly set up</div>
</TOCItem>
<TOCItem link="/docs/lambda/validatewebhooksignature">
<strong>validateWebhookSignature()</strong>
<div>Validate an incoming webhook request is authentic</div>
</TOCItem>
</Grid>
</div>
);
Expand Down
11 changes: 8 additions & 3 deletions packages/docs/components/TextInput.tsx
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import type { DetailedHTMLProps} from "react";
import type { DetailedHTMLProps } from "react";
import React, { useCallback, useState } from "react";

const LIGHT_BLUE = "#42e9f5";
Expand Down Expand Up @@ -34,7 +34,7 @@ type Props = Omit<
"onFocus" | "onBlur"
>;

export const CoolInput: React.FC<Props> = (props) => {
export const CoolInput: React.FC<Props> = ({ style, ...props }) => {
const [focus, setFocused] = useState(false);

const onFocus = useCallback(() => {
Expand All @@ -47,7 +47,12 @@ export const CoolInput: React.FC<Props> = (props) => {

return (
<div style={backgroundStyle(focus)}>
<input style={inputStyle} {...props} onFocus={onFocus} onBlur={onBlur} />
<input
style={{ ...inputStyle, ...style }}
{...props}
onFocus={onFocus}
onBlur={onBlur}
/>
</div>
);
};
188 changes: 188 additions & 0 deletions packages/docs/components/lambda/webhook-test.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,188 @@
import React, { useCallback, useState } from "react";
import { BlueButton } from "../layout/Button";
import { Spinner } from "../Spinner";
import { CoolInput } from "../TextInput";

// This is a re-implementation of the calculateSignature function
// used in @remotion/lambda. This version uses the Crypto Web APIs
// instead of the NodeJS Crypto library and can therefore run in the browser.
async function calculateSignature(payload: string, secret?: string) {
if (!secret) {
return "NO_SECRET_PROVIDED";
}

const enc = new TextEncoder();
const algorithm = { name: "HMAC", hash: "SHA-512" };
const key = await crypto.subtle.importKey(
"raw",
enc.encode(secret),
algorithm,
false,
["sign", "verify"]
);
const signature = await crypto.subtle.sign(
algorithm.name,
key,
enc.encode(payload)
);
const digest = Array.from(new Uint8Array(signature))
.map((x) => x.toString(16).padStart(2, "0"))
.join("");
return `sha512=${digest}`;
}

const buttonFlex: React.CSSProperties = {
paddingBottom: 48,
paddingTop: 8,
display: "flex",
justifyContent: "flex-start",
gap: 16,
};

const inputFlex: React.CSSProperties = {
paddingBottom: 16,
paddingTop: 8,
display: "flex",
flexDirection: "column",
gap: 16,
};

export const WebhookTest: React.FC = () => {
const [request, setRequest] = useState({});
const [response, setResponse] = useState<Object | null>(null);
const [responseStatus, setResponseStatus] = useState<null | boolean>(null);
const [loading, setLoading] = useState(false);
const [data, setData] = useState({
url: "http://localhost:8080/webhook",
secret: "",
});
const handleWebhook = useCallback(
async (type: "success" | "error" | "timeout") => {
setLoading(true);
setRequest({});
setResponseStatus(null);
setResponse(null);
const payload = {
result: type,
renderId: 'demo-render-id',
bucketName: 'demo-bucket-name',
expectedBucketOwner: 'demo-bucket-owner',
...((type === 'success') && {
outputUrl: 'https://www.example.com',
outputFile: 'demo-output.mp4',
timeToFinish: 1500,
}),
lambdaErrors: [],
...((type === 'error') && {
errors: [{
message: "demo-error-message",
name: "demo-error-name",
stack: "demo-error-stack",
}],
})
};
const stringifiedPayload = JSON.stringify(payload);
const req: RequestInit = {
method: "POST",
mode: "cors",
headers: {
"Content-Type": "application/json",
"X-Remotion-Signature": await calculateSignature(
stringifiedPayload,
data.secret
),
"X-Remotion-Status": type,
"X-Remotion-Mode": 'demo',
},
body: stringifiedPayload,
};
fetch(data.url, req)
.then((res) => {
setResponseStatus(res.ok);
return res.json();
})
.then((res) => {
setResponse(res);
})
.catch((err) => {
setResponse(err);
})
.finally(() => {
setRequest({ ...req, body: payload });
setLoading(false);
});
},
[data.secret, data.url]
);
return (
<div>
<div style={inputFlex}>
<div style={{ display: "flex", flexDirection: "column" }}>
<span style={{ paddingBottom: 8 }}>Your webhook URL:</span>
<CoolInput
style={{ width: "100%" }}
placeholder="Your webhook endpoint"
value={data.url}
onChange={(e) =>
setData((prev) => ({ ...prev, url: e.target.value }))
}
/>
</div>
<div style={{ display: "flex", flexDirection: "column" }}>
<span style={{ paddingBottom: 8 }}>Your webhook secret:</span>
<CoolInput
style={{ width: "100%" }}
placeholder="Your webhook secret"
value={data.secret}
onChange={(e) =>
setData((prev) => ({ ...prev, secret: e.target.value }))
}
/>
</div>
</div>
<div style={buttonFlex}>
<BlueButton
size="sm"
fullWidth={false}
loading={loading}
onClick={() => handleWebhook("success")}
>
Send success
</BlueButton>
<BlueButton
size="sm"
fullWidth={false}
loading={loading}
onClick={() => handleWebhook("timeout")}
>
Send timeout
</BlueButton>
<BlueButton
size="sm"
fullWidth={false}
loading={loading}
onClick={() => handleWebhook("error")}
>
Send error
</BlueButton>
</div>
{/* results */}
{loading ? <div style={{ width: '100%', display: 'flex', justifyContent: 'center'}}><Spinner /></div> : null}
{response ? (
<>
<span style={{ paddingBottom: 8 }}>What we sent:</span>
<pre>{JSON.stringify(request, null, 2)}</pre>
<span style={{ paddingBottom: 8 }}>What we received:</span>
<pre>
{responseStatus ? (
<div style={{ color: "green" }}>response ok</div>
) : (
<div style={{ color: "red" }}>response not ok</div>
)}
{JSON.stringify(response, null, 2)}
</pre>
</>
) : null}
</div>
);
};
12 changes: 12 additions & 0 deletions packages/docs/docs/lambda/cli/render.md
Original file line number Diff line number Diff line change
Expand Up @@ -173,3 +173,15 @@ _available from v3.2.25_
If a custom out name is specified and a file already exists at this key in the S3 bucket, decide whether that file will be deleted before the render begins. Default `false`.

An existing file at the output S3 key will conflict with the render and must be deleted beforehand. If this setting is `false` and a conflict occurs, an error will be thrown.

### `--webhook`

_available from v3.2.30_

Sets a webhook to be called when the render finishes or fails. [`renderMediaOnLambda() -> webhook.url`](/docs/lambda/rendermediaonlambda#webhook). To be used together with `--webhook-secret`.

### `--webhook-secret`

_available from v3.2.30_

Sets a webhook secret for the webhook (see above). [`renderMediaOnLambda() -> webhook.secret`](/docs/lambda/rendermediaonlambda#webhook). To be used together with `--webhook`.
4 changes: 4 additions & 0 deletions packages/docs/docs/lambda/light-client.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,8 @@ import {
getFunctions,
AwsRegion,
RenderProgress,
validateWebhookSignature,
WebhookPayload
} from "@remotion/lambda/client";
```

Expand All @@ -21,3 +23,5 @@ These functions don't have any Node.JS dependencies and can be bundled with a bu
**We don't recommend calling these functions from the browser directly, as you will leak your AWS credentials.**

Instead, this light client is meant to reduce the bundle size and avoid problems if you are calling Remotion Lambda APIs from another Lambda function and therefore need to bundle your function code.

Commonly, Next.JS serverless endpoints or similar use AWS Lambda functions under the hood, for which `@remotion/lambda/client` can be used.
46 changes: 37 additions & 9 deletions packages/docs/docs/lambda/rendermediaonlambda.md
Original file line number Diff line number Diff line change
Expand Up @@ -209,6 +209,42 @@ Accepted values:
The default for Lambda is `swangle`, but `null` elsewhere.
:::

### `overwrite`

_available from v3.2.25_

If a custom out name is specified and a file already exists at this key in the S3 bucket, decide whether that file will be deleted before the render begins. Default `false`.

An existing file at the output S3 key will conflict with the render and must be deleted beforehand. If this setting is `false` and a conflict occurs, an error will be thrown.

### `webhook`

_optional, available from v3.2.30_

If specified, Remotion will send a POST request to the provided endpoint to notify your application when the Lambda rendering process finishes, errors out or times out.

```tsx twoslash
import { RenderMediaOnLambdaInput } from "@remotion/lambda";

const webhook: RenderMediaOnLambdaInput["webhook"] = {
url: "https://mapsnap.app/api/webhook",
secret: process.env.WEBHOOK_SECRET as string,
};
```

If you don't want to set up validation, you can set `secret` to null:

```tsx twoslash
import { RenderMediaOnLambdaInput } from "@remotion/lambda";

const webhook: RenderMediaOnLambdaInput["webhook"] = {
url: "https://mapsnap.app/api/webhook",
secret: null,
};
```

[See here for detailed instructions on how to set up your webhook](/docs/lambda/webhooks).

## Return value

Returns a promise resolving to an object containing two properties: `renderId`, `bucketName`, `cloudWatchLogs`. Those are useful for passing to `getRenderProgress()`
Expand All @@ -223,18 +259,10 @@ The S3 bucket name in which all files are being saved.

### `cloudWatchLogs`

_Available from v3.2.10_
_available from v3.2.10_

A link to CloudWatch (if you haven't disabled it) that you can visit to see the logs for the render.

### `overwrite`

_available from v3.2.25_

If a custom out name is specified and a file already exists at this key in the S3 bucket, decide whether that file will be deleted before the render begins. Default `false`.

An existing file at the output S3 key will conflict with the render and must be deleted beforehand. If this setting is `false` and a conflict occurs, an error will be thrown.

## See also

- [Source code for this function](https://github.com/remotion-dev/remotion/blob/main/packages/lambda/src/api/render-media-on-lambda.ts)
Expand Down