Deploying PythonFunction fails due to rsync unable to copy files to /asset-input

[cgeudens]

I'm relatively new to AWS CDK, and I'm trying to deploy a Python Lambda function that contains external dependencies. My function's source code is located at root/backend/projects/my-project:

├── README.md
├── my_project
│   ├── __init__.py
│   ├── __pycache__
│   ├── exceptions.py
│   ├── lambda_function.py
│   ├── logger.py
|   ├── requirements.txt
│   └── settings.py

The Lambda handler is defined in the lambda_function.py module. My CDK code is located at root/backend/cdk. This is the relevant stack:

from aws_cdk import (
    Stack,
    Duration,
    aws_events as events,
    aws_events_targets as targets,
    aws_lambda as lambda_,
    aws_lambda_python_alpha as lambda_python,
)
from constructs import Construct

from settings import settings
from stacks.infrastructure.sqs import SQSStack


class MyProjectStack(Stack):
    def __init__(
        self,
        scope: Construct,
        id: str,
        sqs_stack: SQSStack,
        **kwargs
    ) -> None:
        super().__init__(scope, id, **kwargs)

        # Define Lambda function
        scraper_lambda = lambda_python.PythonFunction(
            self, "ScraperFunction",
            architecture=lambda_.Architecture.ARM_64,
            entry="../backend/projects/my-project/my_project",
            index="lambda_function.py",
            handler="lambda_handler",
            runtime=lambda_.Runtime.PYTHON_3_10,
            timeout=Duration.minutes(1),
            environment={
                "SQS_QUEUE_URL": sqs_stack.scraper_queue.queue_url,
            },
        )

Deploying MyProjectStack locally works fine, but it doesn't work inside my CICD pipeline. I'm using GitHub Actions for this. Here is the Actions workflow:

name: My-Project Lambda

on:
  push:
    # irrelevant

env:
  AWS_REGION: eu-west-1
  POETRY_VERSION: 1.5.1
  PYTHON_VERSION: 3.11

jobs:
  my_project_deployment:

    permissions:
      contents: 'read'
      id-token: 'write'

    runs-on: ubuntu-latest

    steps:
      - name: Checkout repository
        uses: actions/checkout@v3

      - name: Configure AWS credentials
        uses: aws-actions/configure-aws-credentials@v1
        with:
          aws-access-key-id: ${{ secrets.AWS_SANDBOX_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SANDBOX_SECRET_ACCESS_KEY }}
          aws-region: ${{ env.AWS_REGION }}

      - name: Install desired Python version
        uses: actions/setup-python@v5
        with:
          python-version: ${{ env.PYTHON_VERSION }}

      - name: Set up QEMU
        uses: docker/setup-qemu-action@v2

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v1

      - name: Run `cdk deploy`
        uses: youyo/aws-cdk-github-actions@v2
        env:
          AWS_ACCOUNT_ID: ${{ secrets.AWS_ACCOUNT_ID_SANDBOX}}
        with:
          cdk_subcommand: 'deploy'
          cdk_stack: 'MyProjectStack'
          cdk_version: 'latest'
          cdk_args: '--require-approval never'
          actions_comment: false
          working_dir: './cdk'

The cdk deploy action fails. Here are the logs:

added 1 package in 819ms
Install aws-cdk latest
Successful install aws-cdk latest
Install requirements.txt
Successful install requirements.txt
Run cdk deploy --require-approval never "MyProjectStack"
#1 [internal] load .dockerignore
#1 sha256:8eecea38bb4aa56de92d0e68262d833982834ba66ec9ab20995bdd15988a21cf
#1 transferring context: 2B done
#1 DONE 0.0s

#2 [internal] load build definition from Dockerfile
#2 sha256:7bf2cf2b1b7ddf14c0a6b063d7a0369f0ba751c5cae8c4b1ef75575462eb4cf0
#2 transferring dockerfile: 1.32kB done
#2 DONE 0.0s

#3 [internal] load metadata for public.ecr.aws/sam/build-python3.10:latest
#3 sha256:51150e044088b50a1a7302a5f1506a942ee7448efb0481d80d9cc82cfb0234d5
#3 DONE 0.8s

#5 [1/2] FROM public.ecr.aws/sam/build-python3.10@sha256:b2a33158feaa70e1dc79aeb4d784043153637160b79533dc68095550c6057913
...

#4 [2/2] RUN     python -m venv /usr/app/venv &&     mkdir /tmp/pip-cache &&     chmod -R 777 /tmp/pip-cache &&     pip install --upgrade pip &&     mkdir /tmp/poetry-cache &&     chmod -R 777 /tmp/poetry-cache &&     pip install pipenv==2022.4.8 poetry==1.5.1 &&     rm -rf /tmp/pip-cache/* /tmp/poetry-cache/*
...

#4 DONE 137.1s

#6 exporting to image
#6 sha256:0cee2deb735b0ed1681d81d5f9b69a2b5af40f051628406be3617510cc4ea553
#6 exporting layers
#6 exporting layers 4.4s done
#6 writing image sha256:b723631b844bbf2334922ad13abd637d75eaa8bddd9ced696f519ce0de0caa74 done
#6 naming to docker.io/library/cdk-4a98528be2863558ead19ab43b69a7076c3065f3624f51a0013bd8896c1fd4f9:latest done
#6 DONE 4.4s
Bundling asset BackEndService/MeetingsScraperStack/ScraperFunction/Code/Stage...
WARNING: The requested image's platform (linux/arm64) does not match the detected host platform (linux/amd64/v3) and no specific platform was requested
sending incremental file list

sent 38 bytes  received 12 bytes  100.00 bytes/sec
total size is 0  speedup is 0.00
ERROR: Could not open requirements file: [Errno 2] No such file or directory: 'requirements.txt'
jsii.errors.JavaScriptError: 
  Error: Failed to bundle asset BackEndService/MeetingsScraperStack/ScraperFunction/Code/Stage, bundle output is located at /github/workspace/cdk/cdk.out/asset.7abda78a3b0f40a4f07dbf3b8056b2cf02747f110a006400f4c13027adc7eac8-error: Error: docker exited with status 1
  --> Command: docker run --rm -u "0:0" -v "/github/workspace/backend/projects/my-project/my_project:/asset-input:delegated" -v "/github/workspace/cdk/cdk.out/asset.7abda78a3b0f40a4f07dbf3b8056b2cf02747f110a006400f4c13027adc7eac8:/asset-output:delegated" -w "/asset-input" cdk-4a98528be2863558ead19ab43b69a7076c3065f3624f51a0013bd8896c1fd4f9 bash -c "rsync -rLv /asset-input/ /asset-output && cd /asset-output && python -m pip install -r requirements.txt -t /asset-output"
      at AssetStaging.bundle (/tmp/jsii-kernel-FkpEcE/node_modules/aws-cdk-lib/core/lib/asset-staging.js:2:619)
      at AssetStaging.stageByBundling (/tmp/jsii-kernel-FkpEcE/node_modules/aws-cdk-lib/core/lib/asset-staging.js:1:5297)
      at stageThisAsset (/tmp/jsii-kernel-FkpEcE/node_modules/aws-cdk-lib/core/lib/asset-staging.js:1:2728)
      at Cache.obtain (/tmp/jsii-kernel-FkpEcE/node_modules/aws-cdk-lib/core/lib/private/cache.js:1:242)
      at new AssetStaging (/tmp/jsii-kernel-FkpEcE/node_modules/aws-cdk-lib/core/lib/asset-staging.js:1:3125)
      at new Asset (/tmp/jsii-kernel-FkpEcE/node_modules/aws-cdk-lib/aws-s3-assets/lib/asset.js:1:1080)
      at AssetCode.bind (/tmp/jsii-kernel-FkpEcE/node_modules/aws-cdk-lib/aws-lambda/lib/code.js:1:4881)
      at new Function (/tmp/jsii-kernel-FkpEcE/node_modules/aws-cdk-lib/aws-lambda/lib/function.js:1:9422)
      at new PythonFunction (/tmp/jsii-kernel-FkpEcE/node_modules/@aws-cdk/aws-lambda-python-alpha/lib/function.js:40:9)
      at Kernel._Kernel_create (/tmp/tmp42syzfa1/lib/program.js:10108:25)

As the logs show, Docker mounts "/github/workspace/backend/projects/my-project/my_project" to /asset-input, then sets /asset-input as the working directory, and synchronizes /asset-input/ with /asset-output. However, it fails to find any files, and hence the pip install command also fails (since there is no requirements.txt file present). Any idea why this is happening? As I mentioned, I'm able to deploy locally, so it cannot be due to file paths in my source code.