You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Azure Storage, and tools using Azure Storage expect the bytes of an MD5 content hash to be base64 encoded. Terraform's native filemd5() function outputs the hash as a string of hexadecimal characters. Terraform does not have any native tools to support properly converting the byte value of filemd5() to base64.
Considering that setting content_md5 with the filemd5() function appears to be the only Terraform native means of tracking and planning changes to the source file for the azurerm_storage_share_file resource, content_md5 should either properly convert the output of filemd5() or an alternative argument such as content_md5_hex should be added that performs the appropriate conversion internally within the provider.
Actual Behaviour
azurerm_storage_share_file set the ContentMD5 property of the file to the literal string output from filemd5 which causes hash mismatch errors for other clients (e.g. Azure Storage Explorer, AzCopy) trying to download the file.
Terraform has no built in functions or tools that I've been able to find to support calculating an MD5 hash and outputting it as base64, nor any functions that would support converting the byte value of the hex output by filemd5 to base64. Terraform's native base64encode only converts the text value of the hex to base64, which also results in an inconsistent value.
Terraform filemd5() returns the hash encoded as hexadecimal:
On apply, the Azure provider sets the md5 hash using the literal string value of the hexadecimal:
The file fails an integrity check when downloaded via Azure Storage Explorer:
A file uploaded via Azure Storage Explorer has the same MD5 checksum, but the bytes have been encoded as base64. I have confirmed the match when doing a proper hex -> byte -> base64 conversion on the output of filemd5():
Steps to Reproduce
Create a terraform configuration with an azurerm_storage_share_file resource that sets content_md5 using the filemd5 function within terraform.
Apply the configuration.
Attempt to download the file using Azure Storage Explorer, and see the data integrity error in the logs
Important Factoids
No response
References
The most official documentation on MD5 encoding for Azure Storage that I was able to find was in the AzCopy github project, which states:
Note that, in Azure, the Content-MD5 blob property (where the hash is stored) is not just the raw bytes out of the MD5 algorithim. Instead, to produce a valid Azure Content-MD5, you must take the raw bytes returned by the MD5 algorithm, and base64 encode them. AzCopy does this automatically. This paragraph is just for users who want to do their own hash computations and compare to those produced by AzCopy
The text was updated successfully, but these errors were encountered:
Is there an existing issue for this?
Community Note
Terraform Version
1.7.1
AzureRM Provider Version
3.99.0
Affected Resource(s)/Data Source(s)
azurerm_storage_share_file
Terraform Configuration Files
Debug Output/Panic Output
See "Actual Behavior"
Expected Behaviour
Azure Storage, and tools using Azure Storage expect the bytes of an MD5 content hash to be base64 encoded. Terraform's native filemd5() function outputs the hash as a string of hexadecimal characters. Terraform does not have any native tools to support properly converting the byte value of filemd5() to base64.
Considering that setting content_md5 with the filemd5() function appears to be the only Terraform native means of tracking and planning changes to the source file for the azurerm_storage_share_file resource, content_md5 should either properly convert the output of filemd5() or an alternative argument such as content_md5_hex should be added that performs the appropriate conversion internally within the provider.
Actual Behaviour
azurerm_storage_share_file set the ContentMD5 property of the file to the literal string output from filemd5 which causes hash mismatch errors for other clients (e.g. Azure Storage Explorer, AzCopy) trying to download the file.
Terraform has no built in functions or tools that I've been able to find to support calculating an MD5 hash and outputting it as base64, nor any functions that would support converting the byte value of the hex output by filemd5 to base64. Terraform's native base64encode only converts the text value of the hex to base64, which also results in an inconsistent value.
Terraform filemd5() returns the hash encoded as hexadecimal:
On apply, the Azure provider sets the md5 hash using the literal string value of the hexadecimal:
The file fails an integrity check when downloaded via Azure Storage Explorer:
A file uploaded via Azure Storage Explorer has the same MD5 checksum, but the bytes have been encoded as base64. I have confirmed the match when doing a proper hex -> byte -> base64 conversion on the output of filemd5():
Steps to Reproduce
Important Factoids
No response
References
The most official documentation on MD5 encoding for Azure Storage that I was able to find was in the AzCopy github project, which states:
The text was updated successfully, but these errors were encountered: