Test failures with vault 1.9

[yan12125]

With vault 1.9, three tests always fails:

____________ TestOIDC.test_oidc_authorization_url_request_0_success ____________

a = (<tests.integration_tests.api.auth_methods.test_oidc.TestOIDC testMethod=tes
t_oidc_authorization_url_request_0_success>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

/usr/lib/python3.10/site-packages/parameterized/parameterized.py:533:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
tests/integration_tests/api/auth_methods/test_oidc.py:201: in test_oidc_authoriz
ation_url_request
    self.client.secrets.identity.create_or_update_role(
hvac/api/secrets_engines/identity.py:1467: in create_or_update_role
    return self._adapter.post(
hvac/adapters.py:126: in post
    return self.request("post", url, **kwargs)
hvac/adapters.py:364: in request
    response = super(JSONAdapter, self).request(*args, **kwargs)
hvac/adapters.py:330: in request
    utils.raise_for_error(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

method = 'post'
url = 'https://localhost:8200/v1/identity/oidc/role/hvac-oidc-test'
status_code = 400, message = None, errors = ['cannot find key "oidc-test-key"']

    def raise_for_error(method, url, status_code, message=None, errors=None):
        """Helper method to raise exceptions based on the status code of a respo
nse received back from Vault.

        :param method: HTTP method of a request to Vault.
        :type method: str
        :param url: URL of the endpoint requested in Vault.
        :type url: str
        :param status_code: Status code received in a response from Vault.
        :type status_code: int
        :param message: Optional message to include in a resulting exception.
        :type message: str
        :param errors: Optional errors to include in a resulting exception.
        :type errors: list | str

        :raises: hvac.exceptions.InvalidRequest | hvac.exceptions.Unauthorized |
 hvac.exceptions.Forbidden |
            hvac.exceptions.InvalidPath | hvac.exceptions.RateLimitExceeded | hv
ac.exceptions.InternalServerError |
            hvac.exceptions.VaultNotInitialized | hvac.exceptions.BadGateway | h
vac.exceptions.VaultDown |
            hvac.exceptions.UnexpectedError

        """
        if status_code == 400:
>           raise exceptions.InvalidRequest(message, errors=errors, method=metho
d, url=url)
E           hvac.exceptions.InvalidRequest: cannot find key "oidc-test-key", on
post https://localhost:8200/v1/identity/oidc/role/hvac-oidc-test

hvac/utils.py:37: InvalidRequest

This seems related to hashicorp/vault#12208, where role creation now requires key existence.

_________ TestAzure.test_delete_config_0_create_and_then_delete_config _________

a = (<tests.integration_tests.api.secrets_engines.test_azure.TestAzure testMetho
d=test_delete_config_0_create_and_then_delete_config>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

/usr/lib/python3.10/site-packages/parameterized/parameterized.py:533:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
tests/integration_tests/api/secrets_engines/test_azure.py:107: in test_delete_co
nfig
    self.assertEqual(
E   AssertionError: '' != 0

This seems related to hashicorp/vault-plugin-secrets-azure#67 and hashicorp/vault-plugin-secrets-azure#70, which introduce a boolean value use_microsoft_graph_api and several time fields, respectively.

_____________________ IntegrationTest.test_auth_kubernetes _____________________

self = <tests.integration_tests.v1.test_integration.IntegrationTest testMethod=t
est_auth_kubernetes>

    def test_auth_kubernetes(self):
        test_role_name = "test_role"
        test_host = "127.0.0.1:80"
        test_mount_point = "k8s"

        # Turn on the kubernetes backend with a custom mount_point path specifie
d.
        if (
            "{0}/".format(test_mount_point)
            in self.client.sys.list_auth_methods()["data"]
        ):
            self.client.sys.disable_auth_method(test_mount_point)
        self.client.sys.enable_auth_method("kubernetes", path=test_mount_point)
        with open(utils.get_config_file_path("client-cert.pem")) as fp:
            certificate = fp.read()
            self.client.create_kubernetes_configuration(
                kubernetes_host=test_host,
                pem_keys=[certificate],
                mount_point=test_mount_point,
            )

        self.client.create_kubernetes_role(
            name=test_role_name,
            bound_service_account_names="*",
            bound_service_account_namespaces="vault_test",
            mount_point=test_mount_point,
        )

        # Test that we can authenticate
        with open(utils.get_config_file_path("example.jwt")) as fp:
            test_jwt = fp.read()
            with self.assertRaises(
                exceptions.InternalServerError
            ) as assertRaisesContext:
                # we don't actually have a valid JWT to provide, so this method
will throw an exception
                self.client.auth_kubernetes(
                    role=test_role_name,
                    jwt=test_jwt,
                    mount_point=test_mount_point,
                )

        expected_exception_message = 'claim "iss" is invalid'
        actual_exception_message = str(assertRaisesContext.exception)
>       self.assertIn(expected_exception_message, actual_exception_message)
E       AssertionError: 'claim "iss" is invalid' not found in 'namespace not aut
horized, on post https://localhost:8200/v1/auth/k8s/login'

tests/integration_tests/v1/test_integration.py:785: AssertionError

This seems related to hashicorp/vault-plugin-auth-kubernetes#127, where token issuer validation is now disabled by default.

A complete build log is available at https://pkgbuild.com/~foutrelis/failed-py310-builds/python-hvac.log. There are some other failures related to unstable networking and I cannot reliably reproduce them. Will report them once I do.

[yan12125]

There are 29 extra test failures with vault 1.11 compared to 1.9. Here is a complete log: test-failures.txt

[yan12125]

Closing - test failures with Vault 1.9 are further tracked in split issues:

#860
#861
#864