Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

activator can't connect to autoscaler #478

Open
mensylisir opened this issue Aug 10, 2023 · 1 comment
Open

activator can't connect to autoscaler #478

mensylisir opened this issue Aug 10, 2023 · 1 comment
Labels
bug Something isn't working

Comments

@mensylisir
Copy link

Description
A clear and concise description of what the bug is.

I install openfunction by helm package

helm install openfunction openfunction/openfunction -n openfunction --set revisionController.enable=true

Then the activator can't be ready

dapr-system        dapr-dashboard-797b56749f-c24c5                            1/1     Running   1 (5h14m ago)    23h
dapr-system        dapr-operator-7df6df9787-bxc6s                             1/1     Running   7 (5h14m ago)    23h
dapr-system        dapr-placement-server-0                                    1/1     Running   17 (5h14m ago)   23h
dapr-system        dapr-sentry-6fd7776f6-h7qx6                                1/1     Running   1 (5h14m ago)    23h
dapr-system        dapr-sidecar-injector-59d9fb8755-5kdgs                     1/1     Running   1 (5h14m ago)    23h
keda               keda-operator-6d55cf7dd8-zdj9j                             1/1     Running   1 (5h14m ago)    23h
keda               keda-operator-metrics-apiserver-685bc8f78f-t5p4d           1/1     Running   1 (5h14m ago)    23h
knative-serving    activator-5fbdcfb5fd-6lpzc                                 0/1     Running   6 (55s ago)      18m
knative-serving    autoscaler-86bc55dc7d-wk2qg                                1/1     Running   0                19m
knative-serving    controller-64cc8b4f75-b5pcv                                1/1     Running   1 (5h14m ago)    23h
knative-serving    domain-mapping-5d49fd9ff7-sf9jc                            1/1     Running   1 (5h14m ago)    23h
knative-serving    domainmapping-webhook-f684c795d-6fwc5                      1/1     Running   1 (5h14m ago)    23h
knative-serving    net-contour-controller-857cf8c796-wwr9h                    1/1     Running   1 (5h14m ago)    23h
knative-serving    webhook-5876cb94cc-qd6c7                                   1/1     Running   1 (5h14m ago)    23h
openfunction       openfunction-controller-manager-755d89c654-8hzz2           2/2     Running   2 (5h14m ago)    23h
openfunction       openfunction-revision-controller-854fb97cd5-8rzjb          1/1     Running   1 (5h14m ago)    23h
projectcontour     contour-contour-8454bd9f6-rkkwv                            1/1     Running   1 (5h14m ago)    23h
projectcontour     contour-envoy-zznlc                                        2/2     Running   2 (5h14m ago)    23h
shipwright-build   shipwright-build-controller-5954dff84f-dxd6h               1/1     Running   1 (5h14m ago)    23h
tekton-pipelines   tekton-pipelines-controller-5848858869-qrg66               1/1     Running   1 (5h14m ago)    23h
tekton-pipelines   tekton-pipelines-webhook-665545b554-qrlg9                  1/1     Running   1 (5h14m ago)    23h

{"severity":"ERROR","timestamp":"2023-08-10T06:06:17.144801644Z","logger":"activator","caller":"websocket/connection.go:192","message":"Failed to send ping message to ws://autoscaler.knative-serving.svc.cluster.local:8080","commit":"ac29233","knative.dev/controller":"activator","knative.dev/pod":"activator-5fbdcfb5fd-6lpzc","error":"connection has not yet been established","stacktrace":"knative.dev/pkg/websocket.NewDurableConnection.func3\n\tknative.dev/pkg@v0.0.0-20220301181942-2fdd5f232e77/websocket/connection.go:192"}
{"severity":"ERROR","timestamp":"2023-08-10T06:06:18.488498248Z","logger":"activator","caller":"websocket/connection.go:145","message":"Websocket connection could not be established","commit":"ac29233","knative.dev/controller":"activator","knative.dev/pod":"activator-5fbdcfb5fd-6lpzc","error":"dial tcp: i/o timeout","stacktrace":"knative.dev/pkg/websocket.NewDurableConnection.func1\n\tknative.dev/pkg@v0.0.0-20220301181942-2fdd5f232e77/websocket/connection.go:145\nknative.dev/pkg/websocket.(*ManagedConnection).connect.func1\n\tknative.dev/pkg@v0.0.0-20220301181942-2fdd5f232e77/websocket/connection.go:226\nk8s.io/apimachinery/pkg/util/wait.ConditionFunc.WithContext.func1\n\tk8s.io/apimachinery@v0.23.4/pkg/util/wait/wait.go:217\nk8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtectionWithContext\n\tk8s.io/apimachinery@v0.23.4/pkg/util/wait/wait.go:230\nk8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtection\n\tk8s.io/apimachinery@v0.23.4/pkg/util/wait/wait.go:223\nk8s.io/apimachinery/pkg/util/wait.ExponentialBackoff\n\tk8s.io/apimachinery@v0.23.4/pkg/util/wait/wait.go:418\nknative.dev/pkg/websocket.(*ManagedConnection).connect\n\tknative.dev/pkg@v0.0.0-20220301181942-2fdd5f232e77/websocket/connection.go:223\nknative.dev/pkg/websocket.NewDurableConnection.func2\n\tknative.dev/pkg@v0.0.0-20220301181942-2fdd5f232e77/websocket/connection.go:163"}
{"severity":"ERROR","timestamp":"2023-08-10T06:06:18.488645875Z","logger":"activator","caller":"websocket/connection.go:164","message":"Failed connecting to ws://autoscaler.knative-serving.svc.cluster.local:8080","commit":"ac29233","knative.dev/controller":"activator","knative.dev/pod":"activator-5fbdcfb5fd-6lpzc","error":"timed out waiting for the condition","stacktrace":"knative.dev/pkg/websocket.NewDurableConnection.func2\n\tknative.dev/pkg@v0.0.0-20220301181942-2fdd5f232e77/websocket/connection.go:164"}
{"severity":"INFO","timestamp":"2023-08-10T06:06:18.488675842Z","logger":"activator","caller":"websocket/connection.go:162","message":"Connecting to ws://autoscaler.knative-serving.svc.cluster.local:8080","commit":"ac29233","knative.dev/controller":"activator","knative.dev/pod":"activator-5fbdcfb5fd-6lpzc"}
{"severity":"WARNING","timestamp":"2023-08-10T06:06:19.357531373Z","logger":"activator","caller":"handler/healthz_handler.go:36","message":"Healthcheck failed: connection has not yet been established","commit":"ac29233","knative.dev/controller":"activator","knative.dev/pod":"activator-5fbdcfb5fd-6lpzc"}
{"severity":"ERROR","timestamp":"2023-08-10T06:06:20.477660307Z","logger":"activator","caller":"websocket/connection.go:192","message":"Failed to send ping message to ws://autoscaler.knative-serving.svc.cluster.local:8080","commit":"ac29233","knative.dev/controller":"activator","knative.dev/pod":"activator-5fbdcfb5fd-6lpzc","error":"connection has not yet been established","stacktrace":"knative.dev/pkg/websocket.NewDurableConnection.func3\n\tknative.

Environmental
Tell us what your cluster environment is like, e.g.

  • Operating System Information Centos7.9
  • kubernetes version v1.24.9
  • OpenFunction version
NAME            NAMESPACE       REVISION        UPDATED                                 STATUS          CHART                   APP VERSION
openfunction    openfunction    1               2023-08-09 02:38:49.749790533 -0400 EDT deployed        openfunction-0.6.1      1.1.1
  • Versions of dependent components (e.g. dapr, keda, shipwright, knative)

Expected behavior
Tell us what should happen.

Actual behavior
Tell us what happens instead. Provide a log message if relevant.

To Reproduce
Steps to reproduce the behavior:

  1. '..'
  2. '...'
  3. '....'
  4. See error

Screenshots
If applicable, add screenshots to help explain your problem.

Additional context
Add any other context about the problem here.
@mensylisir mensylisir added the bug Something isn't working label Aug 10, 2023
@mensylisir
Copy link
Author

autoscaler logs

2023/08/10 08:43:06 Registering 4 clients
2023/08/10 08:43:06 Registering 5 informer factories
2023/08/10 08:43:06 Registering 5 informers
2023/08/10 08:43:06 Registering 1 filtered informers
2023/08/10 08:43:06 Registering 2 controllers
{"severity":"INFO","timestamp":"2023-08-10T08:43:06.390358743Z","caller":"logging/config.go:116","message":"Successfully created the logger."}
{"severity":"INFO","timestamp":"2023-08-10T08:43:06.390387512Z","caller":"logging/config.go:117","message":"Logging level set to: info"}
{"severity":"INFO","timestamp":"2023-08-10T08:43:06.39047542Z","logger":"autoscaler","caller":"profiling/server.go:64","message":"Profiling enabled: false","commit":"ac29233"}
{"severity":"INFO","timestamp":"2023-08-10T08:43:06.391665896Z","logger":"autoscaler","caller":"kpa/controller.go:75","message":"Setting up ConfigMap receivers","commit":"ac29233"}
{"severity":"INFO","timestamp":"2023-08-10T08:43:06.391778447Z","logger":"autoscaler","caller":"kpa/controller.go:89","message":"Setting up KPA-Class event handlers","commit":"ac29233"}
{"severity":"INFO","timestamp":"2023-08-10T08:43:06.394667498Z","logger":"autoscaler","caller":"metrics/metrics_worker.go:76","message":"Flushing the existing exporter before setting up the new exporter.","commit":"ac29233"}
{"severity":"INFO","timestamp":"2023-08-10T08:43:06.394761745Z","logger":"autoscaler","caller":"metrics/prometheus_exporter.go:51","message":"Created Prometheus exporter with config: &{knative.dev/serving autoscaler prometheus 5000000000 false 9090 0.0.0.0}. Start the server for Prometheus exporter.","commit":"ac29233"}
{"severity":"INFO","timestamp":"2023-08-10T08:43:06.394770012Z","logger":"autoscaler","caller":"metrics/metrics_worker.go:91","message":"Successfully updated the metrics exporter; old config: ; new config &{knative.dev/serving autoscaler prometheus 5000000000 false 9090 0.0.0.0}","commit":"ac29233"}
{"severity":"INFO","timestamp":"2023-08-10T08:43:06.595885233Z","logger":"autoscaler","caller":"autoscaler/main.go:178","message":"Running with Standard leader election","commit":"ac29233"}
{"severity":"INFO","timestamp":"2023-08-10T08:43:06.596197349Z","logger":"autoscaler","caller":"leaderelection/context.go:143","message":"autoscaler-bucket-00-of-01 will run in leader-elected mode with id "autoscaler-86bc55dc7d-vqtmf_10.233.90.68"","commit":"ac29233"}
{"severity":"INFO","timestamp":"2023-08-10T08:43:06.596282852Z","logger":"autoscaler","caller":"controller/controller.go:479","message":"Starting controller and workers","commit":"ac29233","knative.dev/controller":"knative.dev.serving.pkg.reconciler.metric.reconciler","knative.dev/kind":"autoscaling.internal.knative.dev.Metric"}
{"severity":"INFO","timestamp":"2023-08-10T08:43:06.596296321Z","logger":"autoscaler","caller":"controller/controller.go:489","message":"Started workers","commit":"ac29233","knative.dev/controller":"knative.dev.serving.pkg.reconciler.metric.reconciler","knative.dev/kind":"autoscaling.internal.knative.dev.Metric"}
{"severity":"INFO","timestamp":"2023-08-10T08:43:06.59634485Z","logger":"autoscaler.stats-websocket-server","caller":"statserver/server.go:96","message":"Starting","commit":"ac29233","address":":8080"}
{"severity":"INFO","timestamp":"2023-08-10T08:43:06.596392521Z","logger":"autoscaler","caller":"leaderelection/context.go:143","message":"autoscaler-bucket-00-of-01 will run in leader-elected mode with id "autoscaler-86bc55dc7d-vqtmf_10.233.90.68"","commit":"ac29233"}
{"severity":"INFO","timestamp":"2023-08-10T08:43:06.596410327Z","logger":"autoscaler","caller":"controller/controller.go:479","message":"Starting controller and workers","commit":"ac29233","knative.dev/controller":"knative.dev.serving.pkg.reconciler.autoscaling.kpa.Reconciler","knative.dev/kind":"autoscaling.internal.knative.dev.PodAutoscaler"}
{"severity":"INFO","timestamp":"2023-08-10T08:43:06.596414191Z","logger":"autoscaler","caller":"controller/controller.go:489","message":"Started workers","commit":"ac29233","knative.dev/controller":"knative.dev.serving.pkg.reconciler.autoscaling.kpa.Reconciler","knative.dev/kind":"autoscaling.internal.knative.dev.PodAutoscaler"}
I0810 08:43:06.596458 1 leaderelection.go:248] attempting to acquire leader lease knative-serving/autoscaler-bucket-00-of-01...
I0810 08:43:06.596602 1 leaderelection.go:248] attempting to acquire leader lease knative-serving/autoscaler-bucket-00-of-01...
I0810 08:43:06.599735 1 leaderelection.go:258] successfully acquired lease knative-serving/autoscaler-bucket-00-of-01
E0810 08:43:06.599998 1 leaderelection.go:367] Failed to update lock: Operation cannot be fulfilled on leases.coordination.k8s.io "autoscaler-bucket-00-of-01": the object has been modified; please apply your changes to the latest version and try again
{"severity":"INFO","timestamp":"2023-08-10T08:43:06.60001642Z","logger":"autoscaler","caller":"leaderelection/context.go:152","message":""autoscaler-86bc55dc7d-vqtmf_10.233.90.68" has started leading "autoscaler-bucket-00-of-01"","commit":"ac29233"}
{"severity":"INFO","timestamp":"2023-08-10T08:43:06.600290765Z","logger":"autoscaler","caller":"kpa/kpa.go:148","message":"SKS should be in Proxy mode: want = -1, ebc = -100, #act's = 3 PA Inactive? = true","commit":"ac29233","knative.dev/controller":"knative.dev.serving.pkg.reconciler.autoscaling.kpa.Reconciler","knative.dev/kind":"autoscaling.internal.knative.dev.PodAutoscaler","knative.dev/traceid":"e63acec2-ff22-4474-807e-2cf6aebc16b4","knative.dev/key":"default/serving-cc7q7-ksvc-vcrjq-v200"}
{"severity":"INFO","timestamp":"2023-08-10T08:43:06.600314114Z","logger":"autoscaler","caller":"kpa/kpa.go:168","message":"PA scale got=1, want=-1, desiredPods=-1 ebc=-100","commit":"ac29233","knative.dev/controller":"knative.dev.serving.pkg.reconciler.autoscaling.kpa.Reconciler","knative.dev/kind":"autoscaling.internal.knative.dev.PodAutoscaler","knative.dev/traceid":"e63acec2-ff22-4474-807e-2cf6aebc16b4","knative.dev/key":"default/serving-cc7q7-ksvc-vcrjq-v200"}
{"severity":"INFO","timestamp":"2023-08-10T08:43:06.600321005Z","logger":"autoscaler","caller":"kpa/kpa.go:178","message":"Observed pod counts=kpa.podCounts{want:-1, ready:1, notReady:0, pending:0, terminating:0}","commit":"ac29233","knative.dev/controller":"knative.dev.serving.pkg.reconciler.autoscaling.kpa.Reconciler","knative.dev/kind":"autoscaling.internal.knative.dev.PodAutoscaler","knative.dev/traceid":"e63acec2-ff22-4474-807e-2cf6aebc16b4","knative.dev/key":"default/serving-cc7q7-ksvc-vcrjq-v200"}
{"severity":"INFO","timestamp":"2023-08-10T08:43:06.604439097Z","logger":"autoscaler","caller":"statforwarder/leases.go:194","message":"Created Service for Lease knative-serving/autoscaler-bucket-00-of-01","commit":"ac29233"}
{"severity":"INFO","timestamp":"2023-08-10T08:43:06.606467915Z","logger":"autoscaler","caller":"statforwarder/leases.go:274","message":"Bucket Endpoints autoscaler-bucket-00-of-01 updated with IP 10.233.90.68","commit":"ac29233"}
{"severity":"INFO","timestamp":"2023-08-10T08:43:06.606500084Z","logger":"autoscaler","caller":"statforwarder/leases.go:200","message":"Created Endpoints for Lease knative-serving/autoscaler-bucket-00-of-01","commit":"ac29233"}
{"severity":"INFO","timestamp":"2023-08-10T08:43:06.608884768Z","logger":"autoscaler","caller":"controller/controller.go:543","message":"Reconcile succeeded","commit":"ac29233","knative.dev/controller":"knative.dev.serving.pkg.reconciler.autoscaling.kpa.Reconciler","knative.dev/kind":"autoscaling.internal.knative.dev.PodAutoscaler","knative.dev/traceid":"e63acec2-ff22-4474-807e-2cf6aebc16b4","knative.dev/key":"default/serving-cc7q7-ksvc-vcrjq-v200","duration":"8.828637ms"}
{"severity":"INFO","timestamp":"2023-08-10T08:43:06.608994985Z","logger":"autoscaler","caller":"kpa/kpa.go:148","message":"SKS should be in Proxy mode: want = -1, ebc = -100, #act's = 3 PA Inactive? = true","commit":"ac29233","knative.dev/controller":"knative.dev.serving.pkg.reconciler.autoscaling.kpa.Reconciler","knative.dev/kind":"autoscaling.internal.knative.dev.PodAutoscaler","knative.dev/traceid":"b072cc9c-54e4-44ef-9a79-69959270f178","knative.dev/key":"default/serving-cc7q7-ksvc-vcrjq-v200"}
{"severity":"INFO","timestamp":"2023-08-10T08:43:06.609013385Z","logger":"autoscaler","caller":"kpa/kpa.go:168","message":"PA scale got=1, want=-1, desiredPods=-1 ebc=-100","commit":"ac29233","knative.dev/controller":"knative.dev.serving.pkg.reconciler.autoscaling.kpa.Reconciler","knative.dev/kind":"autoscaling.internal.knative.dev.PodAutoscaler","knative.dev/traceid":"b072cc9c-54e4-44ef-9a79-69959270f178","knative.dev/key":"default/serving-cc7q7-ksvc-vcrjq-v200"}
{"severity":"INFO","timestamp":"2023-08-10T08:43:06.609020153Z","logger":"autoscaler","caller":"kpa/kpa.go:178","message":"Observed pod counts=kpa.podCounts{want:-1, ready:1, notReady:0, pending:0, terminating:0}","commit":"ac29233","knative.dev/controller":"knative.dev.serving.pkg.reconciler.autoscaling.kpa.Reconciler","knative.dev/kind":"autoscaling.internal.knative.dev.PodAutoscaler","knative.dev/traceid":"b072cc9c-54e4-44ef-9a79-69959270f178","knative.dev/key":"default/serving-cc7q7-ksvc-vcrjq-v200"}
{"severity":"INFO","timestamp":"2023-08-10T08:43:06.609044838Z","logger":"autoscaler","caller":"controller/controller.go:543","message":"Reconcile succeeded","commit":"ac29233","knative.dev/controller":"knative.dev.serving.pkg.reconciler.autoscaling.kpa.Reconciler","knative.dev/kind":"autoscaling.internal.knative.dev.PodAutoscaler","knative.dev/traceid":"b072cc9c-54e4-44ef-9a79-69959270f178","knative.dev/key":"default/serving-cc7q7-ksvc-vcrjq-v200","duration":"113.263µs"}
{"severity":"ERROR","timestamp":"2023-08-10T08:43:08.601248947Z","logger":"autoscaler","caller":"scaling/autoscaler.go:170","message":"Failed to obtain metrics","commit":"ac29233","knative.dev/key":"default/serving-cc7q7-ksvc-vcrjq-v200","error":"no metrics are being collected for the requested resource","stacktrace":"knative.dev/serving/pkg/autoscaler/scaling.(*autoscaler).Scale\n\tknative.dev/serving/pkg/autoscaler/scaling/autoscaler.go:170\nknative.dev/serving/pkg/autoscaler/scaling.(*MultiScaler).tickScaler\n\tknative.dev/serving/pkg/autoscaler/scaling/multiscaler.go:329\nknative.dev/serving/pkg/autoscaler/scaling.(*MultiScaler).runScalerTicker.func1\n\tknative.dev/serving/pkg/autoscaler/scaling/multiscaler.go:292"}
{"severity":"ERROR","timestamp":"2023-08-10T08:43:10.600282178Z","logger":"autoscaler","caller":"scaling/autoscaler.go:170","message":"Failed to obtain metrics","commit":"ac29233","knative.dev/key":"default/serving-cc7q7-ksvc-vcrjq-v200","error":"no metrics are being collected for the requested resource","stacktrace":"knative.dev/serving/pkg/autoscaler/scaling.(*autoscaler).Scale\n\tknative.dev/serving/pkg/autoscaler/scaling/autoscaler.go:170\nknative.dev/serving/pkg/autoscaler/scaling.(*MultiScaler).tickScaler\n\tknative.dev/serving/pkg/autoscaler/scaling/multiscaler.go:329\nknative.dev/serving/pkg/autoscaler/scaling.(*MultiScaler).runScalerTicker.func1\n\tknative.dev/serving/pkg/autoscaler/scaling/multiscaler.go:292"}
{"severity":"ERROR","timestamp":"2023-08-10T08:43:12.600651252Z","logger":"autoscaler","caller":"scaling/autoscaler.go:170","message":"Failed to obtain metrics","commit":"ac29233","knative.dev/key":"default/serving-cc7q7-ksvc-vcrjq-v200","error":"no metrics are being collected for the requested resource","stacktrace":"knative.dev/serving/pkg/autoscaler/scaling.(*autoscaler).Scale\n\tknative.dev/serving/pkg/autoscaler/scaling/autoscaler.go:170\nknative.dev/serving/pkg/autoscaler/scaling.(*MultiScaler).tickScaler\n\tknative.dev/serving/pkg/autoscaler/scaling/multiscaler.go:329\nknative.dev/serving/pkg/autoscaler/scaling.(*MultiScaler).runScalerTicker.func1\n\tknative.dev/serving/pkg/autoscaler/scaling/multiscaler.go:292"}
{"severity":"ERROR","timestamp":"2023-08-10T08:43:14.601342371Z","logger":"autoscaler","caller":"scaling/autoscaler.go:170","message":"Failed to obtain metrics","commit":"ac29233","knative.dev/key":"default/serving-cc7q7-ksvc-vcrjq-v200","error":"no metrics are being collected for the requested resource","stacktrace":"knative.dev/serving/pkg/autoscaler/scaling.(*autoscaler).Scale\n\tknative.dev/serving/pkg/autoscaler/scaling/autoscaler.go:170\nknative.dev/serving/pkg/autoscaler/scaling.(*MultiScaler).tickScaler\n\tknative.dev/serving/pkg/autoscaler/scaling/multiscaler.go:329\nknative.dev/serving/pkg/autoscaler/scaling.(*MultiScaler).runScalerTicker.func1\n\tknative.dev/serving/pkg/autoscaler/scaling/multiscaler.go:292"}
{"severity":"ERROR","timestamp":"2023-08-10T08:43:16.600963556Z","logger":"autoscaler","caller":"scaling/autoscaler.go:170","message":"Failed to obtain metrics","commit":"ac29233","knative.dev/key":"default/serving-cc7q7-ksvc-vcrjq-v200","error":"no metrics are being collected for the requested resource","stacktrace":"knative.dev/serving/pkg/autoscaler/scaling.(*autoscaler).Scale\n\tknative.dev/serving/pkg/autoscaler/scaling/autoscaler.go:170\nknative.dev/serving/pkg/autoscaler/scaling.(*MultiScaler).tickScaler\n\tknative.dev/serving/pkg/autoscaler/scaling/multiscaler.go:329\nknative.dev/serving/pkg/autoscaler/scaling.(*MultiScaler).runScalerTicker.func1\n\tknative.dev/serving/pkg/autoscaler/scaling/multiscaler.go:292"}
I0810 08:43:17.768004 1 leaderelection.go:258] successfully acquired lease knative-serving/autoscaler-bucket-00-of-01
{"severity":"INFO","timestamp":"2023-08-10T08:43:17.768144024Z","logger":"autoscaler","caller":"leaderelection/context.go:152","message":""autoscaler-86bc55dc7d-vqtmf_10.233.90.68" has started leading "autoscaler-bucket-00-of-01"","commit":"ac29233"}
{"severity":"INFO","timestamp":"2023-08-10T08:43:17.768378415Z","logger":"autoscaler","caller":"controller/controller.go:543","message":"Reconcile succeeded","commit":"ac29233","knative.dev/controller":"knative.dev.serving.pkg.reconciler.metric.reconciler","knative.dev/kind":"autoscaling.internal.knative.dev.Metric","knative.dev/traceid":"fe973963-0327-48ee-91a0-87876265b0ce","knative.dev/key":"default/serving-cc7q7-ksvc-vcrjq-v200","duration":"132.704µs"}
{"severity":"INFO","timestamp":"2023-08-10T08:43:40.889563338Z","logger":"autoscaler","caller":"kpa/scaler.go:233","message":"Probing activator = false, err = error roundtripping [http://serving-cc7q7-ksvc-vcrjq-v200.default.svc.cluster.local:80//healthz](http://serving-cc7q7-ksvc-vcrjq-v200.default.svc.cluster.local//healthz): dial tcp 10.233.11.181:80: connect: connection refused","commit":"ac29233","knative.dev/controller":"knative.dev.serving.pkg.reconciler.autoscaling.kpa.Reconciler","knative.dev/kind":"autoscaling.internal.knative.dev.PodAutoscaler","knative.dev/traceid":"1a1da538-d017-4503-994f-6aae809cee1b","knative.dev/key":"default/serving-cc7q7-ksvc-vcrjq-v200"}
{"severity":"INFO","timestamp":"2023-08-10T08:43:40.88962591Z","logger":"autoscaler","caller":"kpa/scaler.go:280","message":"PA is not yet backed by activator, cannot scale to zero","commit":"ac29233","knative.dev/controller":"knative.dev.serving.pkg.reconciler.autoscaling.kpa.Reconciler","knative.dev/kind":"autoscaling.internal.knative.dev.PodAutoscaler","knative.dev/traceid":"1a1da538-d017-4503-994f-6aae809cee1b","knative.dev/key":"default/serving-cc7q7-ksvc-vcrjq-v200"}
{"severity":"INFO","timestamp":"2023-08-10T08:43:40.889674453Z","logger":"autoscaler","caller":"kpa/kpa.go:148","message":"SKS should be in Proxy mode: want = 0, ebc = -100, #act's = 3 PA Inactive? = true","commit":"ac29233","knative.dev/controller":"knative.dev.serving.pkg.reconciler.autoscaling.kpa.Reconciler","knative.dev/kind":"autoscaling.internal.knative.dev.PodAutoscaler","knative.dev/traceid":"1a1da538-d017-4503-994f-6aae809cee1b","knative.dev/key":"default/serving-cc7q7-ksvc-vcrjq-v200"}
{"severity":"INFO","timestamp":"2023-08-10T08:43:40.889732924Z","logger":"autoscaler","caller":"kpa/kpa.go:168","message":"PA scale got=1, want=0, desiredPods=0 ebc=-100","commit":"ac29233","knative.dev/controller":"knative.dev.serving.pkg.reconciler.autoscaling.kpa.Reconciler","knative.dev/kind":"autoscaling.internal.knative.dev.PodAutoscaler","knative.dev/traceid":"1a1da538-d017-4503-994f-6aae809cee1b","knative.dev/key":"default/serving-cc7q7-ksvc-vcrjq-v200"}
{"severity":"INFO","timestamp":"2023-08-10T08:43:40.889753348Z","logger":"autoscaler","caller":"kpa/kpa.go:178","message":"Observed pod counts=kpa.podCounts{want:0, ready:1, notReady:0, pending:0, terminating:0}","commit":"ac29233","knative.dev/controller":"knative.dev.serving.pkg.reconciler.autoscaling.kpa.Reconciler","knative.dev/kind":"autoscaling.internal.knative.dev.PodAutoscaler","knative.dev/traceid":"1a1da538-d017-4503-994f-6aae809cee1b","knative.dev/key":"default/serving-cc7q7-ksvc-vcrjq-v200"}
{"severity":"INFO","timestamp":"2023-08-10T08:43:40.89752313Z","logger":"autoscaler","caller":"controller/controller.go:543","message":"Reconcile succeeded","commit":"ac29233","knative.dev/controller":"knative.dev.serving.pkg.reconciler.autoscaling.kpa.Reconciler","knative.dev/kind":"autoscaling.internal.knative.dev.PodAutoscaler","knative.dev/traceid":"1a1da538-d017-4503-994f-6aae809cee1b","knative.dev/key":"default/serving-cc7q7-ksvc-vcrjq-v200","duration":"20.296107726s"}
{"severity":"INFO","timestamp":"2023-08-10T08:43:50.157691333Z","logger":"autoscaler","caller":"kpa/scaler.go:233","message":"Probing activator = false, err = error roundtripping [http://serving-cc7q7-ksvc-vcrjq-v200.default.svc.cluster.local:80//healthz](http://serving-cc7q7-ksvc-vcrjq-v200.default.svc.cluster.local//healthz): dial tcp 10.233.11.181:80: connect: connection refused","commit":"ac29233","knative.dev/controller":"knative.dev.serving.pkg.reconciler.autoscaling.kpa.Reconciler","knative.dev/kind":"autoscaling.internal.knative.dev.PodAutoscaler","knative.dev/traceid":"75095934-035b-4cb2-819d-a5e8ccf98693","knative.dev/key":"default/serving-cc7q7-ksvc-vcrjq-v200"}
{"severity":"INFO","timestamp":"2023-08-10T08:43:50.157753392Z","logger":"autoscaler","caller":"kpa/scaler.go:280","message":"PA is not yet backed by activator, cannot scale to zero","commit":"ac29233","knative.dev/controller":"knative.dev.serving.pkg.reconciler.autoscaling.kpa.Reconciler","knative.dev/kind":"autoscaling.internal.knative.dev.PodAutoscaler","knative.dev/traceid":"75095934-035b-4cb2-819d-a5e8ccf98693","knative.dev/key":"default/serving-cc7q7-ksvc-vcrjq-v200"}
{"severity":"INFO","timestamp":"2023-08-10T08:43:50.157771472Z","logger":"autoscaler","caller":"kpa/scaler.go:282","message":"Probe for revision is already in flight","commit":"ac29233","knative.dev/controller":"knative.dev.serving.pkg.reconciler.autoscaling.kpa.Reconciler","knative.dev/kind":"autoscaling.internal.knative.dev.PodAutoscaler","knative.dev/traceid":"75095934-035b-4cb2-819d-a5e8ccf98693","knative.dev/key":"default/serving-cc7q7-ksvc-vcrjq-v200"}
{"severity":"INFO","timestamp":"2023-08-10T08:43:50.157805439Z","logger":"autoscaler","caller":"kpa/kpa.go:148","message":"SKS should be in Proxy mode: want = 0, ebc = -100, #act's = 3 PA Inactive? = true","commit":"ac29233","knative.dev/controller":"knative.dev.serving.pkg.reconciler.autoscaling.kpa.Reconciler","knative.dev/kind":"autoscaling.internal.knative.dev.PodAutoscaler","knative.dev/traceid":"75095934-035b-4cb2-819d-a5e8ccf98693","knative.dev/key":"default/serving-cc7q7-ksvc-vcrjq-v200"}
{"severity":"INFO","timestamp":"2023-08-10T08:43:50.157870938Z","logger":"autoscaler","caller":"kpa/kpa.go:168","message":"PA scale got=1, want=0, desiredPods=0 ebc=-100","commit":"ac29233","knative.dev/controller":"knative.dev.serving.pkg.reconciler.autoscaling.kpa.Reconciler","knative.dev/kind":"autoscaling.internal.knative.dev.PodAutoscaler","knative.dev/traceid":"75095934-035b-4cb2-819d-a5e8ccf98693","knative.dev/key":"default/serving-cc7q7-ksvc-vcrjq-v200"}
{"severity":"INFO","timestamp":"2023-08-10T08:43:50.157902444Z","logger":"autoscaler","caller":"kpa/kpa.go:178","message":"Observed pod counts=kpa.podCounts{want:0, ready:1, notReady:0, pending:0, terminating:0}","commit":"ac29233","knative.dev/controller":"knative.dev.serving.pkg.reconciler.autoscaling.kpa.Reconciler","knative.dev/kind":"autoscaling.internal.knative.dev.PodAutoscaler","knative.dev/traceid":"75095934-035b-4cb2-819d-a5e8ccf98693","knative.dev/key":"default/serving-cc7q7-ksvc-vcrjq-v200"}
{"severity":"INFO","timestamp":"2023-08-10T08:43:50.158029408Z","logger":"autoscaler","caller":"controller/controller.go:543","message":"Reconcile succeeded","commit":"ac29233","knative.dev/controller":"knative.dev.serving.pkg.reconciler.autoscaling.kpa.Reconciler","knative.dev/kind":"autoscaling.internal.knative.dev.PodAutoscaler","knative.dev/traceid":"75095934-035b-4cb2-819d-a5e8ccf98693","knative.dev/key":"default/serving-cc7q7-ksvc-vcrjq-v200","duration":"9.260479374s"}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@mensylisir