Brief description
Envoy does not normalize HTTP URL paths in Envoy 1.9 and before. A remote attacker may craft a path with a relative path, e.g. something/../admin, to bypass access control, e.g. a block on /admin. A backend server could then interpret the unnormalized path and provide an attacker access beyond the scope provided for by the access control policy.
CVSS
CVSS:3.0/AV:N/AC:L/PR:N/UI:N/S:C/C:L/I:L/A:L (8.3, High)
Affected version(s)
Envoy 1.9.0 and before.
Affected component(s)
RBAC, HTTP router, external authorization, rate limiting service, likely others.
Attack vector(s)
Relative path URLs delivered by untrusted client.
Discover(s)/Credits
Erlend Oftedal
Example exploit or proof-of-concept
- Envoy’s RBAC filter is configured to only allow a prefix match of “/info/” to entity X.
- X sends “/info/../private”, the RBAC filter will allow access.
- The backend normalizes the path per RFC 3986 and provides access to “/private”.
Note that this is not limited to RBAC, anywhere that paths are handled in Envoy, the unnormalized variant is used, including route matching, ext_authz and the rate limiting service.
Details
Network proxies such as Envoy make decisions on access control and routing based on path component of a request URL. RFC 3986 describes URL normalization and comparison; if a proxy makes a decision based on a unnormalized view of a path, and the backend or operator intent at configuration is that this decision reflects the normalized view, it is possible for an attacker to bypass the intent of the access control or routing policy. For example, “/info/../private” will match a prefix match of “/info/” but implies “/private”. This is a classic path traversal attack.
The unnormalized view of the path is used for routing and all L7 filter execution, including RBAC, ext_authz and the rate limiting service. This allows bypass of access control and could also be used to circumvent DoS prevention systems such as rate limiting and authorization for a given backend server.
The fix in Envoy 1.9.1 allows optional URL normalization (via a port of the relevant parts of the Chromium URL library to Envoy) in the HTTP Connection Manager. If enabled, this applies shortly after request decoding and before any routing decision is made or filter executed. The normalized URL is used in the rest of the L7 data pipeline and presented to the backend as the intended path. The fix in Envoy applies percent-encoded normalization and path segment normalization, but not case normalization; it is unlikely that match rules will depend on the case of percent-encoded characters and the Chromium URL library does not support this.
In Envoy 1.9.1, this fix is opt-in (in the interest of not breaking existing users) and must be enabled via the HTTP Connection Manager configuration field HttpConnectionManager.normalize_path. There is also a runtime option http_connection_manager.normalize_path to enable normalization. Note: the path forwarded to the upstream will be also normalized when this setting is enabled.
Other proxies, e.g. Nginx, have a distinction between normalization that occurs for the purpose of matching and normalization that is applied as a transform to a path when forwarding to the backend. Future versions of Envoy may introduce such a distinction and default to RFC 3986 normalization for the purpose of matching.
Mitigations
One of the most direct potential exploits with path traversal is via prefix based path matching. Avoiding this both in Envoy and external servers such as ext_authz will avoid one possible common attack, but not all, since an exact match on headers with ext_authz in a “deny” rule can also be bypassed. Suffix match rules should be largely safe, although the backend remains unprotected from this invalid input. Regular expression matches will largely depend on the contents of the regular expression.
External data plane sidecall servers such as ext_authz that interact with Envoy supplied headers can be modified to detect and reject requests with relative path components or perform their own path normalization.
Detection
Envoy’s access logs (whether file-based or gRPC) will contain the unnormalized path, so it is possible to examine these logs to detect suspicious patterns and requests that are incongruous with the intended operator configuration intent. In addition, unnormalized paths are available at ext_authz, rate limiting and backend servers for log inspection.
References
CVE-2019-9901
Brief description
Envoy does not normalize HTTP URL paths in Envoy 1.9 and before. A remote attacker may craft a path with a relative path, e.g. something/../admin, to bypass access control, e.g. a block on /admin. A backend server could then interpret the unnormalized path and provide an attacker access beyond the scope provided for by the access control policy.
CVSS
CVSS:3.0/AV:N/AC:L/PR:N/UI:N/S:C/C:L/I:L/A:L (8.3, High)
Affected version(s)
Envoy 1.9.0 and before.
Affected component(s)
RBAC, HTTP router, external authorization, rate limiting service, likely others.
Attack vector(s)
Relative path URLs delivered by untrusted client.
Discover(s)/Credits
Erlend Oftedal
Example exploit or proof-of-concept
Note that this is not limited to RBAC, anywhere that paths are handled in Envoy, the unnormalized variant is used, including route matching, ext_authz and the rate limiting service.
Details
Network proxies such as Envoy make decisions on access control and routing based on path component of a request URL. RFC 3986 describes URL normalization and comparison; if a proxy makes a decision based on a unnormalized view of a path, and the backend or operator intent at configuration is that this decision reflects the normalized view, it is possible for an attacker to bypass the intent of the access control or routing policy. For example, “/info/../private” will match a prefix match of “/info/” but implies “/private”. This is a classic path traversal attack.
The unnormalized view of the path is used for routing and all L7 filter execution, including RBAC, ext_authz and the rate limiting service. This allows bypass of access control and could also be used to circumvent DoS prevention systems such as rate limiting and authorization for a given backend server.
The fix in Envoy 1.9.1 allows optional URL normalization (via a port of the relevant parts of the Chromium URL library to Envoy) in the HTTP Connection Manager. If enabled, this applies shortly after request decoding and before any routing decision is made or filter executed. The normalized URL is used in the rest of the L7 data pipeline and presented to the backend as the intended path. The fix in Envoy applies percent-encoded normalization and path segment normalization, but not case normalization; it is unlikely that match rules will depend on the case of percent-encoded characters and the Chromium URL library does not support this.
In Envoy 1.9.1, this fix is opt-in (in the interest of not breaking existing users) and must be enabled via the HTTP Connection Manager configuration field HttpConnectionManager.normalize_path. There is also a runtime option http_connection_manager.normalize_path to enable normalization. Note: the path forwarded to the upstream will be also normalized when this setting is enabled.
Other proxies, e.g. Nginx, have a distinction between normalization that occurs for the purpose of matching and normalization that is applied as a transform to a path when forwarding to the backend. Future versions of Envoy may introduce such a distinction and default to RFC 3986 normalization for the purpose of matching.
Mitigations
One of the most direct potential exploits with path traversal is via prefix based path matching. Avoiding this both in Envoy and external servers such as ext_authz will avoid one possible common attack, but not all, since an exact match on headers with ext_authz in a “deny” rule can also be bypassed. Suffix match rules should be largely safe, although the backend remains unprotected from this invalid input. Regular expression matches will largely depend on the contents of the regular expression.
External data plane sidecall servers such as ext_authz that interact with Envoy supplied headers can be modified to detect and reject requests with relative path components or perform their own path normalization.
Detection
Envoy’s access logs (whether file-based or gRPC) will contain the unnormalized path, so it is possible to examine these logs to detect suspicious patterns and requests that are incongruous with the intended operator configuration intent. In addition, unnormalized paths are available at ext_authz, rate limiting and backend servers for log inspection.
References