Waiting for custom condition to be met

[RaicuRobert]

Hello!

• Vote on this issue by adding a 👍 reaction
• If you want to implement this feature, comment to let us know (we'll work with you on design, scheduling, etc.)

Issue details

I have an existing Zone and I am trying to create a StaticSite, RecordSet, and then a StaticSiteCustomDomain.
The problem is that the time between adding a RecordSet with the CNAME for the StaticSite to the Zone and the time of execution for the StaticSiteCustomDomain is too short.

I often get the following error and I have to rerun "azure-native:web:StaticSiteCustomDomain StaticSite-CustomDomain creating (2s) error: Code="BadRequest" Message="CNAME Record is invalid. Please ensure the CNAME record has been created." Details=[{"Message":"CNAME Record is invalid. Please ensure the CNAME record has been created."},{"Code":"BadRequest"},{"ErrorEntity":{"Code":"BadRequest","ExtendedCode":"51021","Message":"CNAME Record is invalid. Please ensure the CNAME record has been created.","MessageTemplate":"{0} is invalid. {1}","Parameters":["CNAME Record","Please ensure the CNAME record has been created."]}}]"

Is there a way to inject custom code to wait until a certain condition is met?
I have a method "Task WaitForTxtEntryDnsPropagation(string domain, string expectedValue, TimeSpan checkFrequency, TimeSpan timeout, string[] nameServersToUse)" that I would like to await before creating the StaticSiteCustomDomain resource.

I could not find a straightforward way of doing it.

The only way I see it being done currently is by using the automation API and having two stacks.
Execute stack 1 (StaticSite and RecordSet)
run WaitForTxtEntryDnsPropagation
Execute stack 2 (StaticSiteCustomDomain)
Thus I have to split what could have been a ComponentResource into two parts.

Affected area/feature

[iwahbe]

Hi @RaicuRobert. You should be able to create a dependency between resources via:

a = new ResourceA("A", args);
b = new ResourceB("B", { a.output.apply(o => { waitOnA(); o}) });

This is typescript. You can do the same thing in C#, I'm just less familiar with the language. If you post the code for two resource you need a dependency between, I can probably figure out the correct Apply between them to create the dependency.

[RaicuRobert]

This would be the current code without unrelated things.

var name = "MySite";
var rgName = "MyResourceGroup";
var domain = "mydomain.com";
var subdomain = "mysubdomain";

var staticSite = new StaticSite($"{name}-StaticSite", new StaticSiteArgs
{
    ResourceGroupName = rgName,
    Location = "westeurope",
    Sku = new SkuDescriptionArgs
    {
        Name = "Free",
        Tier = "Free"
    },
    StagingEnvironmentPolicy = StagingEnvironmentPolicy.Enabled,
    AllowConfigFileUpdates = true,
    EnterpriseGradeCdnStatus = EnterpriseGradeCdnStatus.Disabled,
    PublicNetworkAccess = "Enabled",
    Name = name
},
new CustomResourceOptions
{
    Parent = this
});

var staticSiteCnameDelegation = new RecordSet($"{name}-StaticSite-CName-Delegation", new RecordSetArgs
{
    ZoneName = domain,
    ResourceGroupName = rgName,
    RelativeRecordSetName = subdomain,
    RecordType = "CNAME",
    Ttl = 3600,
    CnameRecord = new CnameRecordArgs
    {
        Cname = staticSite.DefaultHostname

    }
},
new CustomResourceOptions
{
    Parent = this
});

var staticSiteCustomDomain = new StaticSiteCustomDomain($"{name}-StaticSite-CustomDomain", new StaticSiteCustomDomainArgs()
{
    ResourceGroupName = rgName,
    Name = staticSite.Name,
    DomainName = $"{subdomain}.{domain}",
    ValidationMethod = "cname-delegation"
},
new CustomResourceOptions
{
    Parent = this,
    DependsOn = [staticSiteCnameDelegation]
});

[RaicuRobert]

The issue is not with pulumi not waiting for completion. The resources do get created in the right order. The problem is the async nature of the DNS propagation and lack of acknowledgment on either the Pulumi provider or microsofts' implementation.

I have even tried something more crazy like mixing apply with some async code. But with no luck

.......
var domainWithSubdomainOutput = domainWithSubdomain.Apply(dws => expectedValue.Apply(e => {
WaitForTxtEntryDnsPropagation(dws , e, TimeSpan.FromSeconds(10), TimeSpan.FromMinutes(10), ["someServer"]).Wait(); return dws }));

var staticSiteCustomDomain = new StaticSiteCustomDomain($"{name}-StaticSite-CustomDomain", new StaticSiteCustomDomainArgs()
{
    ResourceGroupName = rgName,
    Name = staticSite.Name,
    DomainName = domainWithSubdomainOutput,
    ValidationMethod = "cname-delegation"
},
.......

[RaicuRobert]

Managed to make it work this time tho.

Key was to call WaitForTxtEntryDnsPropagation with .GetAwaiter().GetResult()

[iwahbe]

I'm glad you found a solution that works!

[RaicuRobert]

Well, I was happy a bit too soon @iwahbe .

It seems that even if the code now waits for the dns check to succed, I am not using the same dns server as the StaticSiteCustomDomain provider does.

As such, my check succeded, then the StaticSiteCustomDomain executed and failed.

My initial run seems to have succeded by the chance of a race condition between dns servers propagating my info

[RaicuRobert]

Here is a very bad log of things that happened
Fyi, I have the snipped above as a ComponentResource and deploying 3 static sites in parallel.

@ Updating....
 +  azure-native:web:StaticSite Administration-StaticSite created (6s) 
 +  azure-native:web:StaticSite Business-StaticSite created (7s) 
Waiting for DNS propagation for Administration.my.domain with query type CNAME and expected value happy-flower-randomnum.4.azurestaticapps.net
 +  azure-native:network:RecordSet Administration-StaticSite-CName-Delegation creating (0s) 
Waiting for DNS propagation for Business.my.domain with query type CNAME and expected value polite-sky-randomnum.4.azurestaticapps.net
@ Updating....
 +  azure-native:network:RecordSet Business-StaticSite-CName-Delegation creating (0s) 
DNS propagation check failed. Retying...
DNS propagation Check for Administration.my.domain with query type CNAME and expected value happy-flower-randomnum.4.azurestaticapps.net succeeded
 +  azure-native:web:StaticSite Booking-StaticSite created (8s) 
Waiting for DNS propagation for Booking.my.domain with query type CNAME and expected value brave-forest-randomnum.4.azurestaticapps.net
@ Updating....
 +  azure-native:network:RecordSet Booking-StaticSite-CName-Delegation creating (0s) 
DNS propagation check failed. Retying...
 +  azure-native:network:RecordSet Administration-StaticSite-CName-Delegation created (2s) 
@ Updating....
 +  azure-native:network:RecordSet Business-StaticSite-CName-Delegation created (1s) 
 +  azure-native:network:Subnet ManagedEnvService-Subnet created (4s) 
 +  azure-native:web:StaticSiteCustomDomain Administration-StaticSite-CustomDomain creating (0s) 
 +  azure-native:network:RecordSet Booking-StaticSite-CName-Delegation created (1s) 
@ Updating....
 +  azure-native:app/v20230801preview:ManagedEnvironment ManagedEnv creating (0s) 
@ Updating......
 +  azure-native:web:StaticSiteCustomDomain Administration-StaticSite-CustomDomain creating (3s) error: Code="BadRequest" Message="CNAME Record is invalid.  Please ensure the CNAME record has been created." Details=[{"Message":"CNAME Record is invalid.  Please ensure the CNAME record has been created."},{"Code":"BadRequest"},{"ErrorEntity":{"Code":"BadRequest","ExtendedCode":"51021","Message":"CNAME Record is invalid.  Please ensure the CNAME record has been created.","MessageTemplate":"{0} is invalid.  {1}","Parameters":["CNAME Record","Please ensure the CNAME record has been created."]}}]
 +  azure-native:web:StaticSiteCustomDomain Administration-StaticSite-CustomDomain **creating failed** error: Code="BadRequest" Message="CNAME Record is invalid.  Please ensure the CNAME record has been created." Details=[{"Message":"CNAME Record is invalid.  Please ensure the CNAME record has been created."},{"Code":"BadRequest"},{"ErrorEntity":{"Code":"BadRequest","ExtendedCode":"51021","Message":"CNAME Record is invalid.  Please ensure the CNAME record has been created.","MessageTemplate":"{0} is invalid.  {1}","Parameters":["CNAME Record","Please ensure the CNAME record has been created."]}}]
DNS propagation Check for Business.my.domain with query type CNAME and expected value polite-sky-randomnum.4.azurestaticapps.net succeeded
DNS propagation check failed. Retying...
DNS propagation Check for Booking.my.domain with query type CNAME and expected value brave-forest-randomnum.4.azurestaticapps.net succeeded

[RaicuRobert]

Adding custom timeouts does not seem to help either as the error is returned and is considered final although I set
CustomTimeouts = new CustomTimeouts { Create = TimeSpan.FromMinutes(30), Update = TimeSpan.FromMinutes(30), }
for domain resources

At this point, I think something like this pulumi/pulumi#7932 would have probably solved my issue.
Maybe even if it was implemented like a callback function so I can choose the action to take and have control over the retries.

[iwahbe]

pulumi/pulumi#7932 would provide some support. I think you had the right idea vis a vis checking DNS propagation, you just need to do it against azure.

P.S. AWS has an equivalent problem, and it solved it with https://www.pulumi.com/registry/packages/aws/api-docs/acm/certificatevalidation/. The resource implements the same kind of DNS check that you implemented.

[RaicuRobert]

So I did not specify it, but I did use the exact Microsoft nameservers that the DNS Zone uses for my custom check and that went fine.

The issue is with the Certificate resource itself. After my check on the servers that I know should be good passes, the Certificate is deployed but it uses some other nameserver to check for the records and it fails to find them in time.
This is a Microsoft issue but maybe could be solved by a workaround in pulumi.

[vogonistic]

I have the same issue with asuid txt records that is required to make a web app name binding. There is no way for me to ensure that I check with the internal dns servers Azure uses when checking this, so I need to re-run the whole deployment until all the checks goes through. I've even set the creation of the dns records as something that happens before the web apps are created, but it still randomly fails.

  azure-native:web:WebAppHostNameBinding (foo-binding):
    error: autorest/azure: Service returned an error. Status=400 Code="BadRequest" Message="A TXT record pointing from asuid.foo to cafebabe was not found." Details=[{"Message":"A TXT record pointing from asuid.foo to cafebabe was not found."},{"Code":"BadRequest"},{"ErrorEntity":{"Code":"BadRequest","ExtendedCode":"04006","Message":"A TXT record pointing from asuid.foo to cafebabe was not found.","MessageTemplate":"A TXT record pointing from asuid.{0} to {1} was not found.","Parameters":["foo","cafebabe"]}}]

(Text record name changed to asuid.foo and ASUID value to cafebabe)

Edit: I've also added a wrapper that waits for the TXT record to be created, then waits additional time before creating the binding. It helps, but it's no guarantee.