AWS Lambda support (Docker)

[vigneshgurusamy]

Hi,

From early 2021, AWS lambda supports docker based deployment which supports upto 10GB in container image size.

https://aws.amazon.com/blogs/aws/new-for-aws-lambda-container-image-support/

Do you have a plan to support lambda based deployment using docker in the future releases?

[gulien]

Hello @vigneshgurusamy,

Why not 👍 Last time I checked though, there were some limitations regarding the payload size for HTTP requests, which may be limiting for some features.

[gulien]

See https://pkg.go.dev/github.com/awslabs/aws-lambda-go-api-proxy/echo.

[gulien]

Also #770 (comment).

[goyal-aman]

@gulien I would like to work on this. I'm newbie when it comes to open source contribution.

[gulien]

Feel free to try @goyal-aman!

[f-dig]

Hi @goyal-aman,

I found something that could be helpful: aws-lambda-web-adapter.

I've built an image using the Dockerfile below, and deployed it to AWS Lambda (including a function URL):

FROM gotenberg/gotenberg:8.3.0

# install AWS lambda web adapter
COPY --from=public.ecr.aws/awsguru/aws-lambda-adapter:0.8.1 /lambda-adapter /opt/extensions/lambda-adapter
ENV AWS_LWA_PORT=3001
ENV AWS_LWA_READINESS_CHECK_PATH=/health
ENV AWS_LWA_INVOKE_MODE=response_stream

EXPOSE 3001

ENTRYPOINT ["gotenberg", "--api-port=3001", "--log-level=debug"]

I can now invoke the Lambda via its URL, for example (in python):

import requests

lambda_url = 'https://<url_assigned_by_aws>.lambda-url.<aws-region>.on.aws'
files = {'files': open('path/to/file.doc', 'rb')}

response = requests.post(f'{lambda_url}/forms/libreoffice/convert', files= files)

The aws-lambda-web-adapter works as expected, and the request is passed to the gotenberg API.
However, the process fails due to the read-only filesystem of AWS Lambda:

{
    "level": "debug",
    "ts": 1712161362.0916717,
    "logger": "libreoffice-api.libreoffice.usrliblibreofficeprogramsoffice.bin.stderr",
    "msg": "(process:21): dconf-CRITICAL **: 16:22:42.091: unable to create directory '/home/sbx_user1051/.cache/dconf': Read-only file system.  dconf will not work properly."
}

Any idea how to solve this issue?
AWS Lambda only allows you to write under the \tmp prefix.

[gulien]

Thanks @f-dig for your investigations!

Most I/O operations in Gotenberg are occurring within the /tmp folder. Did you mean that by \tmp prefix?
Regarding the issue, it looks like LibreOffice is trying to create files outside the user profile directory (under /tmp):

gotenberg/pkg/modules/libreoffice/api/libreoffice.go

Line 72 in c11a2d0

fmt.Sprintf("-env:UserInstallation=file://%s", userProfileDirPath),

You'll have to find what are those file, and if a flag exist to specify a directory for them.

[f-dig]

@gulien: it looks like LibreOffice uses the HOME env var to determine where to write files. I have now managed to solve the above issue by setting it to /tmp (in the definition of the Lambda function).

I also added the auto-start option to the Dockerfile:

FROM gotenberg/gotenberg:8.3.0

# install AWS lambda web adapter
COPY --from=public.ecr.aws/awsguru/aws-lambda-adapter:0.8.1 /lambda-adapter /opt/extensions/lambda-adapter
ENV AWS_LWA_PORT=3001
ENV AWS_LWA_READINESS_CHECK_PATH=/health
ENV AWS_LWA_INVOKE_MODE=response_stream
ENV AWS_LWA_ASYNC_INIT=true

EXPOSE 3001

ENTRYPOINT ["gotenberg", "--api-port=3001", "--log-level=debug", "--libreoffice-auto-start=true"]

Unfortunately, there is still an open issue. I try to convert a 25 KB .odt file, and the response content that I get back is empty:

{
'Date': 'Thu, 04 Apr 2024 08:18:49 GMT', 
'Content-Type': 'application/pdf', 
'Content-Length': '0', 
'Connection': 'keep-alive', 
'x-amzn-RequestId': '3c873107-8142-48bf-a72f-084f06723389', 
'last-modified': 'Thu, 04 Apr 2024 08:18:49 GMT', 
'content-disposition': 'attachment; filename="d905a9fe-037b-4302-9fa4-3c927d72be02.pdf"', 
'gotenberg-trace': 'b5c111a6-39b4-476b-8806-67cab15a1dad', 
'x-amzn-Remapped-content-length': '29319', 
'X-Amzn-Trace-Id': 'root=1-660e6267-484293270e19c32b782a5332;parent=06fcacb40fa6e53b;sampled=0;lineage=8e2324df:0', 'accept-ranges': 'bytes', 
'x-amzn-Remapped-date': 'Thu, 04 Apr 2024 08:18:49 GMT'
}

[gulien]

Maybe there is an intermediary service that removes the body for whatever reason? Does it works with the Chromium module?

[f-dig]

The Chrome module fails to start:

{
    "level": "debug",
    "ts": 1712224813.305541,
    "logger": "chromium.browser",
    "msg": "[0404/100013.302163:ERROR:bus.cc(407)] Failed to connect to the bus: Failed to connect to socket /run/dbus/system_bus_socket: No such file or directory\n"
}

Anyway, due to the AWS Lambda payload limit of 6 MB, it may not be the best service to use here.
I'll give AWS Fargate + API Gateway a try and keep you posted

[gulien]

I'm pretty sure it will work with AWS Fargate 👍 Also, a proven serverless solution for container is Google Cloud Run. Not sure it's worth the hassle with AWS Lambda.