[program-gen] Emit Output-returning JSON serialization methods without rewriting applies

[Zaid-Ajaj]

Description

A while ago we started implementing specialized JSON serialization methods for Pulumi programs which can accept nested outputs without having to rewrite and combine applies.

• Output.SerializeJson in .NET
• pulumi.jsonStringify in nodejs
• pulumi.Output.json_dumps in Python

This PR extends program-gen for TypeScript, C# and Python to start emitting these JSON serialization functions (when necessary). The PR special-cases the toJSON PCL function when rewriting applies so that nested outputs aren't rewritted.

Example PCL program and generated results:

Also check out the downstream codegen tests to see improved generated examples

resource vpc "aws:ec2:Vpc" {
	cidrBlock = "10.100.0.0/16"
	instanceTenancy = "default"
}

resource policy "aws:iam/policy:Policy" {
	description = "test"
	policy = toJSON({
		"Version" = "2012-10-17"
		"Interpolated" = "arn:${vpc.arn}:value"
		"Value" = vpc.id
	})
}

Generated TypeScript Before

import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";

const vpc = new aws.ec2.Vpc("vpc", {
    cidrBlock: "10.100.0.0/16",
    instanceTenancy: "default",
});
const policy = new aws.iam.Policy("policy", {
    description: "test",
    policy: pulumi.all([vpc.arn, vpc.id]).apply(([arn, id]) => JSON.stringify({
        Version: "2012-10-17",
        Interpolated: `arn:${arn}:value`,
        Value: id,
    })),
});

Generated TypeScript After

import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";

const vpc = new aws.ec2.Vpc("vpc", {
    cidrBlock: "10.100.0.0/16",
    instanceTenancy: "default",
});
const policy = new aws.iam.Policy("policy", {
    description: "test",
    policy: pulumi.jsonStringify({
        Version: "2012-10-17",
        Interpolated: pulumi.interpolate`arn:${vpc.arn}:value`,
        Value: vpc.id,
    }),
});

Generated Python Before

import pulumi
import json
import pulumi_aws as aws

vpc = aws.ec2.Vpc("vpc",
    cidr_block="10.100.0.0/16",
    instance_tenancy="default")
policy = aws.iam.Policy("policy",
    description="test",
    policy=pulumi.Output.all(vpc.arn, vpc.id).apply(lambda arn, id: json.dumps({
        "Version": "2012-10-17",
        "Interpolated": f"arn:{arn}:value",
        "Value": id,
    })))

Generated Python After

import pulumi
import json
import pulumi_aws as aws

vpc = aws.ec2.Vpc("vpc",
    cidr_block="10.100.0.0/16",
    instance_tenancy="default")
policy = aws.iam.Policy("policy",
    description="test",
    policy=pulumi.Output.json_dumps({
        "Version": "2012-10-17",
        "Interpolated": vpc.arn.apply(lambda arn: f"arn:{arn}:value"),
        "Value": vpc.id,
    }))

Generated C# Before

using System.Collections.Generic;
using System.Linq;
using System.Text.Json;
using Pulumi;
using Aws = Pulumi.Aws;

return await Deployment.RunAsync(() => 
{
    var vpc = new Aws.Ec2.Vpc("vpc", new()
    {
        CidrBlock = "10.100.0.0/16",
        InstanceTenancy = "default",
    });

    var policy = new Aws.Iam.Policy("policy", new()
    {
        Description = "test",
        PolicyDocument = Output.Tuple(vpc.Arn, vpc.Id).Apply(values =>
        {
            var arn = values.Item1;
            var id = values.Item2;
            return JsonSerializer.Serialize(new Dictionary<string, object?>
            {
                ["Version"] = "2012-10-17",
                ["Interpolated"] = $"arn:{arn}:value",
                ["Value"] = id,
            });
        }),
    });

});

Generated C# After

using System.Collections.Generic;
using System.Linq;
using System.Text.Json;
using Pulumi;
using Aws = Pulumi.Aws;

return await Deployment.RunAsync(() => 
{
    var vpc = new Aws.Ec2.Vpc("vpc", new()
    {
        CidrBlock = "10.100.0.0/16",
        InstanceTenancy = "default",
    });

    var policy = new Aws.Iam.Policy("policy", new()
    {
        Description = "test",
        PolicyDocument = Output.JsonSerialize(Output.Create(new Dictionary<string, object?>
        {
            ["Version"] = "2012-10-17",
            ["Interpolated"] = vpc.Arn.Apply(arn => $"arn:{arn}:value"),
            ["Value"] = vpc.Id,
        })),
    });

});

Checklist

• I have run make tidy to update any new dependencies
• I have run make lint to verify my code passes the lint check
  • I have formatted my code using gofumpt

• I have added tests that prove my fix is effective or that my feature works

• I have run make changelog and committed the changelog/pending/<file> documenting my change

• Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version

[pulumi-bot]

Changelog

[uncommitted] (2024-02-04)

Features

• [programgen/{dotnet,nodejs,python}] Emit Output-returning JSON serialization methods without rewriting applies for top-level function expression
  #15371

[Zaid-Ajaj]

Added this so that program-gen can install and test generated programs against a specific version of .NET Pulumi SDK. In this case, latest version that contains the Output.JsonSerialize function. In case of pulumi convert, it already emits 3.* so latest is resolved there as well

[Zaid-Ajaj]

We need Output.JsonSerialize(Output.Create(...)) instead of Output.JsonSerialize(...) because the function only accepts Output<T>. We could simplify that in a subsequent PR after we generalize Output.JsonSerialize to accept any T