Configurable masking and hiding for the env endpoint #6711

timyates · 2022-01-05T09:28:26Z

Changes the env endpoint to be disabled by default (requires user to enable in config)
If enabled, all env endpoint values are masked with *****
Allow user to add a bean implementing EnvironmentEndpointFilter which:
- allows legacy operation to be allowed so it behaves as before
- can be set to mask or reveal all
- allows patterns for masking to be defined if reveal all is set
- allows patterns for plain-text values if mask all is set

The call to the filter also contains the Principal if there is one, so the user can choose
to alter visibility of items based on the currently logged in user

- Changes the env endpoint to be disabled by default (requires user to enable in config) - If enabled, all env endpoint values are masked with ***** - Allow user to add a bean implementing EnvironmentEndpointFilter which: - allows legacy operation to be allowed so it behaves as before - can be set to mask or reveal all - allows patterns for masking to be defined if reveal all is set - allows patterns for plain-text values if mask all is set The call to the filter also contains the Principal if there is one, so the user can choose to alter visibility of items based on the currently logged in user

CLAassistant · 2022-01-05T09:28:30Z

All committers have signed the CLA.

management/src/main/java/io/micronaut/management/endpoint/env/EnvironmentEndpoint.java

...ement/src/main/java/io/micronaut/management/endpoint/env/EnvironmentFilterSpecification.java

management/src/main/java/io/micronaut/management/endpoint/env/EnvironmentEndpoint.java

management/src/main/java/io/micronaut/management/endpoint/env/EnvironmentEndpointFilter.java

...ement/src/main/java/io/micronaut/management/endpoint/env/EnvironmentFilterSpecification.java

src/main/docs/guide/management/providedEndpoints/environmentEndpoint.adoc

management/src/main/java/io/micronaut/management/endpoint/env/EnvironmentEndpointFilter.java

For binary compatibility. And annotate new constructor with Inject so it is used

timyates · 2022-01-07T12:04:39Z

@graemerocher Hope I addressed all your feedback 🤞

graemerocher · 2022-01-07T13:58:18Z

@timyates from my side yeah, @jameskleeh needs to review next

jameskleeh

I think this is missing:

the ability to mask all properties except a set a patterns
the ability to mask all properties except a set of static keys
the ability to mask a set of static keys

jameskleeh · 2022-01-07T17:32:35Z

...ement/src/main/java/io/micronaut/management/endpoint/env/EnvironmentFilterSpecification.java

+     * @return itself for chaining calls.
+     */
+    public @NotNull EnvironmentFilterSpecification maskPatterns(Pattern... propertySourceNameMask) {
+        maskedPatterns.addAll(Arrays.asList(propertySourceNameMask));


This would have no effect without allMasked = false; so that should probably be set here

If allMasked = true, then this sets the patterns that are allowed to be in the clear

https://github.com/micronaut-projects/micronaut-core/pull/6711/files#diff-4e7e5d0b9696543f54847a8d042130fc33e81143e23d6c414504d106ec0fa4c7R15

maskPatterns is probably a rubbish name... But toggleMask was wrong too, as it's not a toggle, it's additive

I think maskPatterns should define the patterns that are masked, thus only making it useful if not all keys are masked

With a separate list for unmasked patterns, that only come into play if allMasked is true?

Or we could name it "excludedPatterns" which are excluded from whichever mode you're in... And change the method to exclude() ?

That sounds OK to me. Need to support static keys as well

Also the ability to supply predicates. Everything could be converted to predicates

Switch to using Predicate<String> for all exclusions

To avoid unchecked generic array creation for varargs parameter warnings in user code (due to the Predicate<String> varargs), introduced EnvironmentFilterNamePredicate as a typealias

Also added a factory method to generate a predicate from a pattern and optional flags

...ement/src/main/java/io/micronaut/management/endpoint/env/EnvironmentFilterSpecification.java

management/src/main/java/io/micronaut/management/endpoint/env/EnvironmentEndpoint.java

...ement/src/main/java/io/micronaut/management/endpoint/env/EnvironmentFilterSpecification.java

graemerocher · 2022-01-10T15:09:52Z

...ement/src/main/java/io/micronaut/management/endpoint/env/EnvironmentFilterSpecification.java

+    public static EnvironmentFilterNamePredicate regularExpressionPredicate(String pattern) {
+        return regularExpressionPredicate(pattern, 0);
+    }
+
+    public static EnvironmentFilterNamePredicate regularExpressionPredicate(String pattern, int flags) {
+        return s -> Pattern.compile(pattern, flags).matcher(s).matches();
+    }


These appear now to be part of the public API? if it is the case they need javadoc

Done. Would you say they were necessary? I added them to make it simpler to create the Predicates, but can see that they may add noise 🤔

timyates · 2022-01-10T15:48:07Z

Also, having the EnvironmentFilterNamePredicate type-alias as an inner-class makes the javadoc quite long 😐 Should it be made top-level? Or does someone know of a better way around the unchecked generic array creation for varargs parameter warning?

timyates · 2022-01-10T18:18:54Z

After a sync up with James, we have a better API... Incoming asap

jameskleeh · 2022-01-24T19:23:15Z

@graemerocher Can you please give this a final look?

* Configurable masking and hiding for the env endpoint - Changes the env endpoint to be disabled by default (requires user to enable in config) - If enabled, all env endpoint values are masked with ***** - Allow user to add a bean implementing EnvironmentEndpointFilter which: - allows legacy operation to be allowed so it behaves as before - can be set to mask or reveal all - allows patterns for masking to be defined if reveal all is set - allows patterns for plain-text values if mask all is set The call to the filter also contains the Principal if there is one, so the user can choose to alter visibility of items based on the currently logged in user * Address first round of feedback * Fix tests * Fix since tag * Add single arg constructor For binary compatibility. And annotate new constructor with Inject so it is used * Switch to Predicate<String> in place of Patterns * Checkstyle fixes * Move env endpoint into disabled section of doc * Add deprecated methods and JavaDoc * Remove principal and simplify API * Cleanup Co-authored-by: jameskleeh <james.kleeh@gmail.com>

timyates changed the title ~~New version is 3.3.0-SNAPSHOT~~ Configurable masking and hiding for the env endpoint Jan 5, 2022

timyates linked an issue Jan 5, 2022 that may be closed by this pull request

The Environment Endpoint security issue #6663

Closed

timyates commented Jan 5, 2022

View reviewed changes

management/src/main/java/io/micronaut/management/endpoint/env/EnvironmentEndpoint.java Show resolved Hide resolved

timyates commented Jan 5, 2022

View reviewed changes

...ement/src/main/java/io/micronaut/management/endpoint/env/EnvironmentFilterSpecification.java Outdated Show resolved Hide resolved

timyates requested a review from jameskleeh January 5, 2022 09:45

timyates self-assigned this Jan 5, 2022

timyates added this to the 3.3.0 milestone Jan 5, 2022

graemerocher requested changes Jan 5, 2022

View reviewed changes

ilopmar reviewed Jan 5, 2022

View reviewed changes

management/src/main/java/io/micronaut/management/endpoint/env/EnvironmentEndpointFilter.java Outdated Show resolved Hide resolved

timyates added 4 commits January 5, 2022 10:38

Address first round of feedback

f9ac54f

Fix tests

3ccae51

Fix since tag

0c9e307

Add single arg constructor

7bc7553

For binary compatibility. And annotate new constructor with Inject so it is used

jameskleeh suggested changes Jan 7, 2022

View reviewed changes

timyates added 3 commits January 10, 2022 10:27

Switch to Predicate<String> in place of Patterns

1a1891e

Checkstyle fixes

bfbea1a

Move env endpoint into disabled section of doc

1a76524

graemerocher requested changes Jan 10, 2022

View reviewed changes

Add deprecated methods and JavaDoc

85d2897

timyates and others added 3 commits January 10, 2022 21:09

Remove principal and simplify API

3d03c23

Cleanup

6adfd90

Merge branch '3.3.x' into timyates/environment-endpoint-filtering

4cc473c

jameskleeh approved these changes Jan 24, 2022

View reviewed changes

graemerocher approved these changes Jan 25, 2022

View reviewed changes

jameskleeh merged commit 7c6149c into 3.3.x Jan 25, 2022

jameskleeh deleted the timyates/environment-endpoint-filtering branch January 25, 2022 15:02

graemerocher mentioned this pull request Jan 26, 2022

Password is not masked in /env micronaut-projects/micronaut-mongodb#224

Closed

timyates mentioned this pull request Jan 26, 2022

The Environment Endpoint security issue #6663

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Configurable masking and hiding for the env endpoint #6711

Configurable masking and hiding for the env endpoint #6711

timyates commented Jan 5, 2022

CLAassistant commented Jan 5, 2022 •

edited

Loading

timyates commented Jan 7, 2022

graemerocher commented Jan 7, 2022

jameskleeh left a comment

jameskleeh Jan 7, 2022

timyates Jan 7, 2022 •

edited

Loading

jameskleeh Jan 7, 2022

timyates Jan 7, 2022

jameskleeh Jan 7, 2022

jameskleeh Jan 7, 2022

timyates Jan 10, 2022

graemerocher Jan 10, 2022

timyates Jan 10, 2022

timyates commented Jan 10, 2022

timyates commented Jan 10, 2022

jameskleeh commented Jan 24, 2022

Configurable masking and hiding for the env endpoint #6711

Configurable masking and hiding for the env endpoint #6711

Conversation

timyates commented Jan 5, 2022

CLAassistant commented Jan 5, 2022 • edited Loading

timyates commented Jan 7, 2022

graemerocher commented Jan 7, 2022

jameskleeh left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

timyates Jan 7, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

timyates commented Jan 10, 2022

timyates commented Jan 10, 2022

jameskleeh commented Jan 24, 2022

CLAassistant commented Jan 5, 2022 •

edited

Loading

timyates Jan 7, 2022 •

edited

Loading