AWS ECS Task Definition Port Mapping Name ignored by pulumi

[chan-vince]

What happened?

I want to create a containerised service on AWS ECS Fargate, where I want a port mapping on a container in the Task Definition, which works.

However there is also an optional name argument (e.g. here in python awsx) which can be added to the port mapping but this does not appear to work - there is no error but it's like it is ignored. I don't think Pulumi is reading/using/applying it, as the diff doesn't pick up on changes either.

The ability to name a port mapping is probably an unusual use case - typically you'd only care about which ports/protocols you're mapping into the container - there's no obvious reason to want to name it. The reason why it is important to me is because a named port mapping is how ECS Service Connect works - without it, it cannot register a service instance in AWS Cloud Map. It is described here in the AWS docs that it exists for Service Connect.

Whether I use awsx.ecs.FargateService() or the aws.ecs.* resources directly, seems to make no difference. But to be more concise here, I'll use the awsx example. I am using python and I have only used python, so I am unable to determine whether this is an issue specific to the python runtime or not.

Example

1. Create a Fargate Service like this example. (I have omitted other required resources to reduce noise, like the VPC, ECS Cluster, ECR image etc but I can add them to give a standalone working example if necessary.)
   Note the port name of "this-should-be-present" is defined in the container port_mappings port, and is referenced by the service as port_name in service_connect_configuration. It is this link that enables ECS Service Connect to do service discovery on the containers.

service = awsx.ecs.FargateService(
    "fargate-service",
    cluster=ecs_cluster.arn,
    network_configuration=aws.ecs.ServiceNetworkConfigurationArgs(
        assign_public_ip=True,
        subnets=vpc.public_subnet_ids,
        security_groups=[fargate_security_group.id],
    ),
    service_connect_configuration=aws.ecs.ServiceServiceConnectConfigurationArgs(
        enabled=True,
        namespace=namespace.arn,
        services=[
            aws.ecs.ServiceServiceConnectConfigurationServiceArgs(
                port_name="this-should-be-present",
                client_alias=[
                    aws.ecs.ServiceServiceConnectConfigurationServiceClientAliasArgs(
                        port=80,
                    )
                ],
            )
        ],
    ),
    task_definition_args=awsx.ecs.FargateServiceTaskDefinitionArgs(
        container=awsx.ecs.TaskDefinitionContainerDefinitionArgs(
            name="my-task-definition",
            image=ecr_image.image_uri,
            cpu=512,
            memory=256,
            essential=True,
            port_mappings=[
                awsx.ecs.TaskDefinitionPortMappingArgs(
                    name="this-should-be-present",
                    container_port=80,
                    protocol="tcp",
                )
            ],
        ),
        runtime_platform=aws.ecs.TaskDefinitionRuntimePlatformArgs(
            cpu_architecture="ARM64",
            operating_system_family="LINUX",
        ),
    ),
)

3. Run pulumi up to create it. It will fail because the port name in the port mapping is not applied, and so the service can't find a port with that name.
   It fails with this AWS error in the diagnostics when it tries to apply:

InvalidParameterException: portName(this-should-be-present) does not refer to any named PortMapping in the container definitions.

So comment out the FargateService.service_connect_configuration to skip it for now, then it will work.

4. Go to the AWS console to inspect the Task Definition. The port mapping for port 80 will be there, but the name will be unset, which aligns with what the error above is saying. I expect it to be "this-should-be-present". Alternatively, use the awscli with jq to see the same issue:
   aws ecs describe-task-definition --task-definition task-definition-name-here | jq -r '.taskDefinition.containerDefinitions[].portMappings[]'

The output is:

{
  "containerPort": 80,
  "hostPort": 80,
  "protocol": "tcp"
}

but I expected it to be:

{
  "containerPort": 80,
  "hostPort": 80,
  "protocol": "tcp",
  "name": "this-should-be-present"
}

5. Modify the stack so that the port name is something else. Pulumi will not detect that any changes are necessary.

Output of pulumi about

CLI
Version 3.92.0
Go Version go1.21.3
Go Compiler gc

Plugins
NAME VERSION
python unknown

Host
OS darwin
Version 14.1
Arch arm64

This project is written in python: executable='/Users/vince/.pyenv/shims/python3' version='3.11.4'

Current Stack: organization/project/dev

TYPE URN
[omitted]

Found no pending operations associated with dev

Backend
Name Vinces-Macbook-Pro.local
URL s3://sandbox-vc-pulumi/pulumi?region=eu-west-1&awssdk=v2
User vince
Organizations
Token type personal

Dependencies:
NAME VERSION
black 23.10.1
pip 23.2.1
pylint 2.17.5
pytest 7.4.3
ruff 0.1.4
setuptools 65.5.0

Pulumi locates its logs in /var/folders/w5/nm4wxtn55sg2szxyrlxd6vv80000gn/T/ by default

Additional context

No response

Contributing

Vote on this issue by adding a 👍 reaction.
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

[chan-vince]

I only started using Pulumi last night and looking at the source today, so I'm not sure at all but looking at the code,
it seems like the issue (or part of it) is that the name attribute is missing from the interface definition here maybe?

pulumi-aws/sdk/nodejs/ecs/container.ts

Lines 189 to 193 in 4c3960c

	export interface PortMapping {
	containerPort?: pulumi.Input<number>;
	hostPort?: pulumi.Input<number>;
	protocol?: Protocol;
	}

[mikhailshilkov]

@chan-vince Thank you for filing this issue. I think your analysis is basically correct and we need to add missing properties to PortMapping and then plumb them through AWSX resources. If you are up for a contribution here, we are happy to take it and also happy to guide you through it.

[chan-vince]

@mikhailshilkov
Ok, thanks for your input :) I'm not really sure what is meant by 'plumbed through the resources' but I've got a draft PR up #3043 here with an updated interface for starters. Would appreciate some pointers as to what further work is required :)

[iwahbe]

Some background: Pulumi has pulumi-aws and pulumi-awsx. pulumi-awsx contains component resources. You can think of the resources in pulumi-awsx as mini pulumi programs that define a bunch of resources. pulumi-aws defines the resources that actually talk to the cloud (AWS) and most pulumi-awsx resources are bundles of pulumi-aws resources.

It would be extremely helpful to have a self-contained repro program that uses only aws.* resources. In the mean time, I believe that #3043 is correct. That file sits outside our code generation.

[robsoned]

Hello guys, I'm having the same problem here, trying to use serviceConnectConfiguration. To avoid this error, I started using the serviceRegistries.

But I think the error also applies when you try using the network mode host or bridge in your taskDefinition.

InvalidParameterException: When specifying 'host' or 'bridge' for networkMode, values for 'containerName' and 'containerPort' must be specified from the task definition.

[mjeffryes]

I believe this should be fixed now that #3043 is merged.