Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add knob for switching container port #3333

Merged
merged 1 commit into from May 20, 2024

Conversation

zvlb
Copy link
Contributor

@zvlb zvlb commented May 6, 2024

issue - #3226, #2310
Previos PR - #2405 (not my, not finished)

@arkodg I add new field to EnvoyProxy (not EnvoyGateway), bc with this EnvoyProxy We can use kubernetes Gateway CRD for functionality)

Local tests finished good

@zvlb zvlb requested a review from a team as a code owner May 6, 2024 13:44
// This allows the container to bind to the port without needing a CAP_NET_BIND_SERVICE capability.
//
// +optional
DisablePortShift *bool `json:"disablePortShift,omitempty"`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

any other alternate name suggestions ?DisablePortShift is too vague imo and doesnt account for the fact that this issue is tied to ContainerPort
cc @envoyproxy/gateway-maintainers @envoyproxy/gateway-reviewers

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To come up with variable names is harder than writing code.

Something like DisableShiftContainerPort?

Copy link
Member

@cnvergence cnvergence May 7, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could it be an other way around, like BindPort or StaticPort bool value?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BindPort and StaticPort looks like fields for port value. Not for bool

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I prefer UseListenerPortAsContainerPort from https://github.com/envoyproxy/gateway/pull/2405/files
@envoyproxy/gateway-maintainers / @envoyproxy/gateway-reviewers wdyt

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 for UseListenerPortAsContainerPort .

I can tell what UseListenerPortAsContainerPort does by its name, but can't infer what DisablePortShift without looking into the comment. :-)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No problem. I can use this variable. However, I see a small logical issue. If I plan to use a port higher than 1024 in the Listener (for example, 3000), there will be no 'port offsets'. That means the UseListenerPortAsContainerPort variable will be set to false (the default value), and at the same time, the listener port and the container port will be equal.

@Xunzhuo Xunzhuo changed the title Add feature for "don't switching container port" feat: add knob for switching container port May 14, 2024

// DisablePortShift disables the port shifting feature in the Envoy Proxy.
// When set to false (default value), if the service port is a privileged port (1-1023), add a constant to the value converting it into an ephemeral port.
// This allows the container to bind to the port without needing a CAP_NET_BIND_SERVICE capability.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this require user involvement, or does EG automatically add the CAP_NET_BIND_SERVICE capability to the Envoy container?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure about "automatically".
In my case I set this CAP in envoyProxy, like:

apiVersion: gateway.envoyproxy.io/v1alpha1
kind: EnvoyProxy
metadata:
  name: default
  namespace: envoy-gateway
spec:
  provider:
    kubernetes:
      disablePortShift: true.  # tmp name
      envoyDaemonSet:
        pod:
          nodeSelector:
            node-role.kubernetes.io/gateway: "true"
        container:
          image: envoyproxy/envoy:v1.29.2
          securityContext:
            allowPrivilegeEscalation: true
            capabilities:
              add:
              - NET_BIND_SERVICE
              drop:
              - ALL
            seccompProfile:
              type: RuntimeDefault
        patch:
          type: StrategicMerge
          value:
            spec:
              template:
                spec:
                  hostNetwork: true
                  dnsPolicy: ClusterFirstWithHostNet
    type: Kubernetes

And need to use NOT distroless images, bc distroless can't bind on 1-1024 ports

@zvlb
Copy link
Contributor Author

zvlb commented May 18, 2024

I rebase my branch from main and change field name from DisablePortShift to UseListenerPortAsContainerPort.

But I can't run e2e test localy. If something broken - I will test all and fix in monday. (I'm not sure about route.go... Many changes. Now this package don't collect container ports)

Copy link

codecov bot commented May 18, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 67.43%. Comparing base (8206e11) to head (65253e1).
Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3333      +/-   ##
==========================================
+ Coverage   67.41%   67.43%   +0.01%     
==========================================
  Files         166      166              
  Lines       19186    19197      +11     
==========================================
+ Hits        12934    12945      +11     
  Misses       5322     5322              
  Partials      930      930              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Signed-off-by: zvlb <vl.zemtsov@gmail.com>
@zvlb
Copy link
Contributor Author

zvlb commented May 20, 2024

Just rebase

@zvlb
Copy link
Contributor Author

zvlb commented May 20, 2024

Test error: Tokenless has reached GitHub rate limit

Copy link
Member

@AliceProxy AliceProxy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

branch needs rebase again but lgtm

Copy link
Contributor

@arkodg arkodg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM thanks !

@arkodg arkodg merged commit d49f984 into envoyproxy:main May 20, 2024
28 checks passed
@zvlb zvlb deleted the disable-switch-ports branch May 21, 2024 08:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants