List all secret paths a user has permission to view

[drewmullen]

I have a hashi ticket opened with this information: https://support.hashicorp.com/hc/en-us/requests/10584 I tried to adapt the ticket info here so please let me know if something is unclear

Is your feature request related to a problem? Please describe.
Currently, for our users to be able to navigate the vault GUI, we need to apply path "secret/*" { capabilities = ["list"] } to that user. Without it, they receive an error after they click into the secret engine (see pic). However, when we grant that permission, the user can then list ALL secrets. From a user stand point, this is confusing because they can list every secret in our vault but cannot access the actual secret.

Describe the solution you'd like
it would be great for our users to be able to list (view in the gui) only secret paths that they have access to and not see secret paths if they dont have access.

Describe alternatives you've considered
Cant think of any.. probably the current solution is the only other alternative: grant secret/* list

Explain any additional use-cases
We're using 0.11 enterprise preimum + hsm with a kv2 secret engine. The guide ive been following is: https://www.vaultproject.io/docs/secrets/kv/kv-v2.html#acl-rules and https://support.hashicorp.com/hc/en-us/articles/360000953148-Vault-UI-Secrets-Navigation

In the example below, ive also tried in the CLI with the same results:

1. the policy we've applied to the user
   Taf policy

path "secret/metadata/taf/*" { capabilities = ["list", "read"] }
path "secret/data/taf/*" { capabilities = ["list", "read"] }

2. a secret exists at the end of that path

$ vault kv get secret/taf/test 
====== Metadata ====== 
Key Value 
--- ----- 
created_time 2018-09-12T13:25:57.871481033Z 
deletion_time n/a 
destroyed false 
version 1

====== Data ====== 
Key Value 
--- ----- 
test_key value

3. can list the secret at the path the policy provides

$ vault kv list secret/taf/ 
Keys 
---- 
taf/ 
test

4. we can not list at the root secret level <-- as i understand, this is what the GUI does when you click through past the secret engine

$ vault kv list secret/ 
Error listing secret/metadata: Error making API request.

URL: GET https://<ip>:8200/v1/secret/metadata?list=true 
Code: 403. Errors:

* 1 error occurred:

* permission denied

[valentin-fischer]

+1

[SanghmitraJohri]

I am also facing the same issue. Any update on this?

[mfuxi]

I'm facing the same issue as well, a workaround for this issue is to supply manually the secret path in the URL, like this:
secret/list/foo

[royt-via]

Hi @catsby, do you know if there's an estimation to when (if at all) this issue will be resolved?

[rfc1459]

This issue seriously limits the usefulness of Vault usage by humans because they would have to remember the path to each and every secret they have access to or the organization should give blanket list permissions to every human user (which might be unacceptable in some contexts and would create needless confusion among users).

By reading other issues about similar requests it seems that the stated position regarding policies is that filtering API replies using them will not be implemented. Is that position still standing? And if so, how is this specific issue going to be fixed if replies cannot be filtered?

[avbru]

+1

[dhrapson]

+1

[matsgoran]

+1

[shanewow]

+1

This is holding us back as well. We are using role based generators. Without developing systems/documentation to show users what roles (paths) they have access to, users are required to click on each path to see which they can use with trial and error.

[kjvellajr]

+1

Some sort of filtering of list api based on access would be ideal for many resources

[abhikalluru]

+1
From a user's standpoint, it is difficult to hop to every path available and check if the token is accessible. Listing only those which can be accessed will help.

[dpolasi]

+1

[paka36394]

+1

[ArtemShevelyukhin]

+1
It is also very frustrating and annoying that the presence of this bug is NOT described ANYWHERE in the documentation. I spent about 8 hours in total trying to write appropriate policy and only now I found this ticket

[oprocopio]

+1

[BartDrown]

Same as above
There should be other way to allow access to the certain engine resources subdirectory.
Sadly for now, even 'deny' policy on path listing does not work (according to this issue).
If denying listing on path would be a thing, workaround would be pretty easy.

[skuffe]

+1

[Xhet]

+1

[LukeBalizet]

+1

[tfpereira]

+1

[lukpep]

+1

[timota]

+1
same here

[janmaghuyop]

+1

[japneet-sahni]

@hsimon-hashicorp : I have a similar issue where our end users have to generate OTP using SSH-OTP secret engine. I have created below policy based on alias metadata. The user doesn't see his specific SSH-OTP role unless I add the third path-capability as shown below. But once I add this, he is shown all the roles (although he can generate OTP through his role). Not a security issue but not a user-friendly behavior as well.

path "ssh/creds/{{identity.entity.aliases.auth_oidc_f080901a.metadata.oid}}" {
    capabilities = ["read", "update", "list"]
}

path "ssh/roles/{{identity.entity.aliases.auth_oidc_f080901a.metadata.oid}}" {
    capabilities = ["read", "list"]
}

path "ssh/roles" {
    capabilities = ["read", "list"]
}

[hellobontempo]

Hi @drewmullen - thanks for submitting your issue! Apologies for the delayed response. We've made some updates to this part of the UI so a user can type the path of the secret they want to read without list access to metadata. In the gif below I'm logged in as a user with the policy mentioned in the original issue:

path "secret/metadata/taf/*" { capabilities = ["list", "read"] }
path "secret/data/taf/*" { capabilities = ["list", "read"] }

Although I'm currently developing on 1.13.x this functionality was added by #12626 in 1.9

Does this resolve your issue?

[oilegor1029]

Hi @drewmullen - thanks for submitting your issue! Apologies for the delayed response. We've made some updates to this part of the UI so a user can type the path of the secret they want to read without list access to metadata. In the gif below I'm logged in as a user with the policy mentioned in the original issue:

path "secret/metadata/taf/*" { capabilities = ["list", "read"] }
path "secret/data/taf/*" { capabilities = ["list", "read"] }

Although I'm currently developing on 1.13.x this functionality was added by #12626 in 1.9

Does this resolve your issue?

Then with this feature users must know the list of secrets they can access so that they can access them through the search bar, did I understand correctly?

I think that this issue/feature request is about automatically show secrets that the users has access.
Currently (if I remember correctly, it's been a while since I last touched Vault) you either have access to the whole list of secrets or none, you can't filter the permissions so that it only shows a list of secrets you have access to.

[cbednarski]

Claire, thanks for the response. I'm afraid #12626 does not resolve the issue here.

This issue is primarily cause by a discrepancy in user expectations between vault's UI and API. Primarily, the UI treats paths like "folders" and users expect to browse those as they would Windows Explorer or Mac's Finder.

However, the policy information required to make those "folders" appear (so users know what they can see in Vault) is overly broad, so it shows too much.

Removing the policy that makes "folders" appear certainly solves the problem of too many folders appearing but it makes the situation worse because people then can't browse vault at all.

The changes you demonstrated in #12626 assume that the user already knows the paths they have access to, as if they were accessing the API path directly. This is not the case.

In larger deployments people gain access by group membership and vault spaces are managed across dozens or hundreds of projects at a time. Users do not have individual knowledge of the vault policy or specific, individual paths of secrets in the API and rely on browsing the "folders" to find their space and secrets.

[hellobontempo]

Thank you for the clarification! I think the issue name here confused me a bit, since the UI does show the secret if a user/token has access to that path. However, as you mentioned, it requires a user knows the full secret path they want to access, which I understand, is not the desired UX 😅. There's absolutely room for an enhancement here.

We are currently investigating permission-based listing which I believe would address this issue. If that's the case (and to confirm I'm understanding the ask here), may I suggest updating the name or description to something like: List all secret paths a user has permission to view

A possible solution the UI could implement in the meantime is providing an input that expects a folder path. For example, if a user inputted secret/taf (with the above policy) they would see a list view of all the secrets within that path and can browse from there. This is similar to running vault kv list secret/taf in the cli.

proposed LIST view if a user inputs secret/taf

I realize this still relies on some knowledge of the secret's path, so it may not be that helpful of an improvement.

[jbgcarnes]

Have the same issue. Must like cbednarski, my users primarily operate through the UI and expect to be able to traverse like normal folders. Because of my setup, I cannot have users having list permissions at the root level and seeing everyone else's secrets, even if its just that the secret exists. It would be helpful if a user has access (lets say list permissions to a path), that that path show up in the UI like a normal folder would.

[maxb]

At present, that simply isn't part of Vault's feature set, though.

Perhaps @hellobontempo can give us an update on whether HashiCorp is still looking in to:

We are currently investigating permission-based listing which I believe would address this issue.

The original requester probably isn't still around nearly 5 years after opening the issue, so the waiting-for-response label is likely not realistic.

[drewmullen]

I am still around :) @maxb

didnt realize they were waiting for me to respond. reviewing now

[maxb]

Oh! Well, give a response so HashiCorp take the waiting-for-response label off, then! :-)

[drewmullen]

hello @hellobontempo the convo bw you and @cbednarski is a better description of the true issue. i agree the request is not resolved yet

thank you for getting back to us though and considering this issue :D

[hellobontempo]

FWIW the waiting-for-response isn't just reserved for the OP but more of a waiting-for-anyone-out-there-to-respond 😅 which helps track where we've responded and which conversations are still active 😄

Thank you all for following up! If folks give me the thumbs up, I'll rename this issue to List all secret paths a user has permission to view to more accurately depict the request here. There is an internal initiative tracking this work, but it hasn't been prioritized yet. I've reminded product this feature has a lot of community sentiment behind it and am hopeful we can get something slated for the next couple of releases. Stay tuned 📺

[hellobontempo]

Hello following up 👋 I apologize for all of the back and forth but as we're working on this now I want to make we address this correctly ✔

With the policy above, are you able to update the URL manually and list the secrets you'd like to? The path would look something like ui/vault/secrets/secret/list/taf/ (screenshot below 📸 )

Surfacing the above view (instead of manually changing the URL 😓 ) at least aligns with the CLI where a user can view the LIST of secrets/directories inside a known path folder

[hellobontempo]

Partially addressed by # #22502

[lucasoares]

I would like to have this in the API, not just in the UI, so that my app can get all the secrets it has permission, rather than getting each path individually.

[marcportabellaclotet-mt]

This could also prove beneficial for other confidential secret engines such as databases. Currently, we are tasked with overseeing numerous database roles, which poses a significant challenge for users when sifting through an extensive list of roles, especially considering that they often lack access to most of them. The implementation of a list filtering policy at API level will be highly advantageous.

[UlascanY]

I would like to have this in the API, not just in the UI, so that my app can get all the secrets it has permission, rather than getting each path individually.

This worked for me.

# Grant permissions on user specific path
path "user-kv/data/{{identity.entity.name}}/*" {
    capabilities = [ "create", "update", "read", "delete", "list" ]
}

# For Web UI usage
path "user-kv/metadata/*" {
  capabilities = ["list", "read"]
}

[AleksArabadzhy]

+1
I support this feature request. This function is highly needed and will streamline the work.

[samuelrusso]

@UlascanY With those updated permissions, what did you run to see what paths you had access to?

[UlascanY]

@samuelrusso

path "user-kv/data/{{identity.entity.name}}/*"

1. create a Secrets-Engine in this case with the name "user-kv"
2. Then the Users should only see their secrets because of this {{identity.entity.name}}

As far as I remember :)

[tot19]

Hello @hellobontempo, is there any update on this? As far as I can tell, there is still no way to give users list access to only the paths they have access to and have them see only those subdirectories in the UI.

I can only seem to grant access to list all the subdirectories in KV. If I do not do that, the search prompt comes up when the user enters the KV engine.

Still think @cbednarski 's description of the issue is the best in this thread.

[hellobontempo]

Hi @tot19 - not yet. List-based permissions (that is, only listing items a user has access to) is still in the works.

[SharpEdgeMarshall]

Almost 6 years and an essential feature like this one is still missing, I don't understand how you expect to charge companies hundreds of thousands of $ a year without improving the product in any way.
I consider myself lucky for choosing to go with OSS instead of buying it, otherwise, I would have been very frustrated after discovering this kind of issue.