Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update tests to work with new test proxy #22869

Open
3 of 85 tasks
chlowell opened this issue May 8, 2024 · 0 comments
Open
3 of 85 tasks

Update tests to work with new test proxy #22869

chlowell opened this issue May 8, 2024 · 0 comments
Labels
Client This issue points to a problem in the data-plane of the library. Mgmt This issue is related to a management-plane library. test-proxy Anything relating to test-proxy requests or issues. test-reliability Issue that causes tests to be unreliable

Comments

@chlowell
Copy link
Contributor

chlowell commented May 8, 2024

New versions of the test proxy include a set of sanitizers enabled by default during recording and playback. These sanitizers can break tests by replacing values used to match recordings and values that tests require to match a known constant or format. In playback mode the affected tests are skipped with a message referencing this issue. They still run as before in live and recording modes. If skipped tests cause a pipeline to fail because the test suite no longer reaches the module's coverage goal, you can temporarily adjust that goal in eng/config.json.

Updating these tests means recording them with test proxy version 1.0.0-dev.20240508.1 or later and modifying them to work with newly sanitized values. If your tests call recording.StartTestProxy, you'll get the right proxy version automatically by basing your changes on main. If you need to replace one of the new sanitizers to record a different value or a test requires a real value and you're certain that value isn't a secret, you can call recording.RemoveRegisteredSanitizers to remove default sanitizers by their IDs e.g. "AZSDK3493". Sanitizers and their IDs are listed here. When uncertain whether something is secret, prefer the sanitized value. Note also that sdk/internal v1.6.0 and v1.7.0 add these registered sanitizers and don't allow removing them so if you must remove one, first upgrade to sdk/internal v1.8.0.

Modules having at least one affected test are listed below.

  • sdk/internal

Data plane:

  • ai/azopenai
  • ai/azopenaiassistants
  • containers/azcontainerregistry
  • data/azappconfig
  • data/aztables
  • messaging/azwebpubsub
  • messaging/eventgrid/azeventgrid
  • messaging/eventgrid/aznamespaces
  • security/keyvault/azadmin
  • security/keyvault/azcertificates
  • security/keyvault/azkeys
  • security/keyvault/azsecrets
  • storage/azblob
  • storage/azdatalake
  • storage/azfile

Management plane:

  • resourcemanager/advisor/armadvisor
  • resourcemanager/alertsmanagement/armalertsmanagement
  • resourcemanager/analysisservices/armanalysisservices
  • resourcemanager/apicenter/armapicenter
  • resourcemanager/apimanagement/armapimanagement
  • resourcemanager/appconfiguration/armappconfiguration
  • resourcemanager/appplatform/armappplatform
  • resourcemanager/attestation/armattestation
  • resourcemanager/batch/armbatch
  • resourcemanager/botservice/armbotservice
  • resourcemanager/cdn/armcdn
  • resourcemanager/changeanalysis/armchangeanalysis
  • resourcemanager/chaos/armchaos
  • resourcemanager/compute/armcompute
  • resourcemanager/containerregistry/armcontainerregistry
  • resourcemanager/containerservice/armcontainerservice
  • resourcemanager/containerservicefleet/armcontainerservicefleet
  • resourcemanager/cosmos/armcosmos
  • resourcemanager/cosmosforpostgresql/armcosmosforpostgresql
  • resourcemanager/databricks/armdatabricks
  • resourcemanager/datafactory/armdatafactory
  • resourcemanager/datalake-store/armdatalakestore
  • resourcemanager/devcenter/armdevcenter
  • resourcemanager/dns/armdns
  • resourcemanager/dnsresolver/armdnsresolver
  • resourcemanager/dynatrace/armdynatrace
  • resourcemanager/eventgrid/armeventgrid
  • resourcemanager/eventhub/armeventhub
  • resourcemanager/internal/testutil
  • resourcemanager/iotcentral/armiotcentral
  • resourcemanager/iothub/armiothub
  • resourcemanager/keyvault/armkeyvault
  • resourcemanager/kusto/armkusto
  • resourcemanager/labservices/armlabservices
  • resourcemanager/loadtesting/armloadtesting
  • resourcemanager/logic/armlogic
  • resourcemanager/managementgroups/armmanagementgroups
  • resourcemanager/maps/armmaps
  • resourcemanager/mariadb/armmariadb
  • resourcemanager/mediaservices/armmediaservices
  • resourcemanager/mobilenetwork/armmobilenetwork
  • resourcemanager/monitor/armmonitor
  • resourcemanager/msi/armmsi
  • resourcemanager/network/armnetwork
  • resourcemanager/notificationhubs/armnotificationhubs
  • resourcemanager/operationalinsights/armoperationalinsights
  • resourcemanager/postgresql/armpostgresql
  • resourcemanager/privatedns/armprivatedns
  • resourcemanager/quota/armquota
  • resourcemanager/recoveryservices/armrecoveryservices
  • resourcemanager/redis/armredis
  • resourcemanager/redisenterprise/armredisenterprise
  • resourcemanager/relay/armrelay
  • resourcemanager/resources/armdeploymentscripts
  • resourcemanager/resources/armfeatures
  • resourcemanager/resources/armpolicy
  • resourcemanager/resources/armresources
  • resourcemanager/resources/armsubscriptions
  • resourcemanager/resources/armtemplatespecs
  • resourcemanager/search/armsearch
  • resourcemanager/selfhelp/armselfhelp
  • resourcemanager/servicebus/armservicebus
  • resourcemanager/servicefabric/armservicefabric
  • resourcemanager/signalr/armsignalr
  • resourcemanager/storage/armstorage
  • resourcemanager/storagecache/armstoragecache
  • resourcemanager/storagemover/armstoragemover
  • resourcemanager/streamanalytics/armstreamanalytics
  • resourcemanager/webpubsub/armwebpubsub
@chlowell chlowell added Client This issue points to a problem in the data-plane of the library. Mgmt This issue is related to a management-plane library. test-reliability Issue that causes tests to be unreliable test-proxy Anything relating to test-proxy requests or issues. labels May 8, 2024
@tadelesh tadelesh assigned tadelesh and unassigned tadelesh May 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Client This issue points to a problem in the data-plane of the library. Mgmt This issue is related to a management-plane library. test-proxy Anything relating to test-proxy requests or issues. test-reliability Issue that causes tests to be unreliable
Projects
None yet
Development

No branches or pull requests

2 participants