Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Nullable Changefeed Retention and Public Access Enablement - avm/res/storage/storage-account #1508

Open
wants to merge 8 commits into
base: main
Choose a base branch
from

Conversation

fblix
Copy link
Contributor

@fblix fblix commented Apr 2, 2024

Description

Bugfixes to enable

  • Blob Services changefeed retention without a time limit
  • Ability to deploy a storage account with fully public access
  • Added test case for blob services changefeeds

Fixes #1346
Fixes #1385

Pipeline Reference

Pipeline
avm.res.storage.storage-account

Type of Change

  • Update to CI Environment or utlities (Non-module effecting changes)
  • Azure Verified Module updates:
    • Bugfix containing backwards compatible bug fixes, and I have NOT bumped the MAJOR or MINOR version in version.json:
      • Someone has opened a bug report issue, and I have included "Closes #{bug_report_issue_number}" in the PR description.
      • The bug was found by the module author, and no one has opened an issue to report it yet.
    • Feature update backwards compatible feature updates, and I have bumped the MINOR version in version.json.
    • Breaking changes and I have bumped the MAJOR version in version.json.
    • [] Update to documentation

Checklist

  • I'm sure there are no other open Pull Requests for the same update/change
  • I have run Set-AVMModule locally to generate the supporting module files.
  • My corresponding pipelines / checks run clean and green without any errors or warnings

@microsoft-github-policy-service microsoft-github-policy-service bot added the Needs: Triage 🔍 Maintainers need to triage still label Apr 2, 2024

Important

The "Needs: Triage 🔍" label must be removed once the triage process is complete!

Tip

For additional guidance on how to triage this issue/PR, see the BRM Issue Triage documentation.

Note

This label was added as per ITA06.

@microsoft-github-policy-service microsoft-github-policy-service bot added the Type: AVM 🅰️ ✌️ Ⓜ️ This is an AVM related issue label Apr 2, 2024

Note

The "Type: AVM 🅰️ ✌️ Ⓜ️" label was added as per ITA08BCP.

@fblix fblix changed the title feat: Nullable Changefeed and Public Access Enablement feat: Nullable Changefeed Retention and Public Access Enablement Apr 2, 2024
@fblix fblix marked this pull request as ready for review April 2, 2024 20:33
@fblix fblix requested review from a team as code owners April 2, 2024 20:33
@@ -78,7 +78,7 @@ resource blobServices 'Microsoft.Storage/storageAccounts/blobServices@2022-09-01
automaticSnapshotPolicyEnabled: automaticSnapshotPolicyEnabled
changeFeed: changeFeedEnabled ? {
enabled: true
retentionInDays: changeFeedRetentionInDays
retentionInDays: changeFeedRetentionInDays ?? null
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
retentionInDays: changeFeedRetentionInDays ?? null
retentionInDays: changeFeedRetentionInDays

Not required as ?? takes the first non-null value. Having null as an 'else' case means it's just ignored 😄

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The idea was to actually supply a null value. Having any value in changeFeedRetentionInDays means its not stored indefinetely.

ChangefeedRetention works as follows: values from 1 to 146000 sets a specific point in time when the data gets deleted. setting null (not 0!) causes the data to be retained indefinitely - which is the goal here. So setting ?? should make sense, right?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nono absolutely. But as the parameter is nullable, not setting it results in a null and a null would be passed to the resource.

With

  • retentionInDays: changeFeedRetentionInDays ?? null all you're getting after the evaluation is
  • retentionInDays: null ?? null

and Bicep will pick any null.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fblix, could we move forward on this? This PR has been open for a very very long time. If bad comes to worse I can also create the contribution, yet would need to you review and approve it.

bypass: 'AzureServices'
defaultAction: 'Deny'
}
param networkAcls networkAclsType
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why removing the default? Would it not make more sense to move it into the network ACLS like suggested here: #1472 (comment)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just wondering how we could make it so that a firewall would be enabled by default, but could be disabled if needed. I guess with the implementation in the comment it's also not exactly possible, right? 😄

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Somehow in my mind it doesn't check out...

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@eriqua / @ChrisSidebotham any ideas?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removing the default value was the only solution I was able to come up with. Everything else just fails the linting spectacularly, maybe you guys have a better idea on how to achieve this?

Copy link
Contributor

@eriqua eriqua Apr 4, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@eriqua / @ChrisSidebotham any ideas?

@AlexanderSehr bare with me, still need to look into this more closely.

The rule of thumb, if that helps, is to make it more difficult, rather than easier, to deviate from the value we want to enforce as default. That is only in case the intention of the default is to make the resource secure by default/WAF compliant/pass PSRule.

Regarding the implementation, please consider the condition for public network access, normally enabled or disabled by default also based on network ACL values.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just had some spare time again, and I still think it would be the easiest/cleanest solution to just have networkACLs set as mandatory. Default being the Action: Deny. If someone then wants to open the firewall again, it would be as easy as setting Action: Allow. What do you think? @AlexanderSehr @eriqua @ChrisSidebotham

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @fblix,
I took another stab.

How about this to avoid making it mandatory.

Template

@description('Required. Networks ACLs, this value contains IPs to whitelist and/or Subnet information. If in use, bypass needs to be supplied. For security reasons, it is recommended to set the DefaultAction Deny.')
param networkAcls networkAclsType // Removed default as nullable types cannot have defaults

(...)
    networkAcls: !empty(networkAcls)
      ? {
          resourceAccessRules: networkAcls.?resourceAccessRules
          bypass: networkAcls.?bypass
          defaultAction: networkAcls.?defaultAction ?? 'Deny'
          virtualNetworkRules: networkAcls.?virtualNetworkRules
          ipRules: networkAcls.?ipRules
        }
      : { // New default case that enables the firewall by default
          bypass: 'AzureServices'
          defaultAction: 'Deny'
        }

type networkAclsType = {
  (...)  
}? // Nullable => optional
(...)

Test case (defaults test)

module testDeployment '../../../main.bicep' = [
  for iteration in ['init', 'idem']: {
    scope: resourceGroup
    name: '${uniqueString(deployment().name, resourceLocation)}-test-${serviceShort}-${iteration}'
    params: {
      name: '${namePrefix}${serviceShort}001'
      allowBlobPublicAccess: false
      location: resourceLocation
      networkAcls: {
        defaultAction: 'Allow'
        bypass: 'None'
      }
    }
  }
]

In Azure, that showed up as
image

So the firewall was disabled.

When I remove the explicit specification / it falls back to the default it shows
image

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 on @AlexanderSehr proposed implementation.

Not sure I follow on the defaults test case though.
Shouldn't we just use required parameters there? I think I'm missing something.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're correct @eriqua. I just used the default test as a showcase to not muddy the waters.

Comment on lines +3 to +4
metadata name = 'Using only defaults'
metadata description = 'This instance deploys the module with the minimum set of required parameters.'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Must be updated to reflect the changefeed example

@matebarabas matebarabas changed the title feat: Nullable Changefeed Retention and Public Access Enablement feat: Nullable Changefeed Retention and Public Access Enablement - avm/res/storage/storage-account Apr 3, 2024
@matebarabas matebarabas added Class: Resource Module 📦 This is a resource module and removed Needs: Triage 🔍 Maintainers need to triage still labels Apr 3, 2024
@AlexanderSehr
Copy link
Contributor

Hey @fblix,
this PR has been open for a month now. How's it going? Do you need support?

@eriqua eriqua added the Needs: Author Feedback 👂 Awaiting feedback from the issue/PR author label May 11, 2024
@ChrisSidebotham ChrisSidebotham added the Status: Looking For Assistance 🦆 This item is looking for anyone to help develop the code and submit a PR for resolution label May 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Class: Resource Module 📦 This is a resource module Needs: Author Feedback 👂 Awaiting feedback from the issue/PR author Status: Looking For Assistance 🦆 This item is looking for anyone to help develop the code and submit a PR for resolution Type: AVM 🅰️ ✌️ Ⓜ️ This is an AVM related issue
Projects
None yet
6 participants