You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
*** Reading local file: /root/airflow/logs/ml-intermediate/get/2022-01-30T21:40:44+00:00/1.log
[2022-01-30, 21:40:47 UTC] {taskinstance.py:1035} INFO - Dependencies all met for <TaskInstance: ml-intermediate.get manual__2022-01-30T21:40:44+00:00 [queued]>
[2022-01-30, 21:40:47 UTC] {taskinstance.py:1035} INFO - Dependencies all met for <TaskInstance: ml-intermediate.get manual__2022-01-30T21:40:44+00:00 [queued]>
[2022-01-30, 21:40:47 UTC] {taskinstance.py:1241} INFO -
--------------------------------------------------------------------------------
[2022-01-30, 21:40:47 UTC] {taskinstance.py:1242} INFO - Starting attempt 1 of 1
[2022-01-30, 21:40:47 UTC] {taskinstance.py:1243} INFO -
--------------------------------------------------------------------------------
[2022-01-30, 21:40:47 UTC] {taskinstance.py:1262} INFO - Executing <Task(KubernetesPodOperator): get> on 2022-01-30 21:40:44+00:00
[2022-01-30, 21:40:47 UTC] {standard_task_runner.py:52} INFO - Started process 374 to run task
[2022-01-30, 21:40:47 UTC] {standard_task_runner.py:76} INFO - Running: ['airflow', 'tasks', 'run', 'ml-intermediate', 'get', 'manual__2022-01-30T21:40:44+00:00', '--job-id', '9', '--raw', '--subdir', 'DAGS_FOLDER/ml-intermediate.py', '--cfg-path', '/tmp/tmp9ongxm85', '--error-file', '/tmp/tmpuk0o_489']
[2022-01-30, 21:40:47 UTC] {standard_task_runner.py:77} INFO - Job 9: Subtask get
[2022-01-30, 21:40:47 UTC] {logging_mixin.py:109} INFO - Running <TaskInstance: ml-intermediate.get manual__2022-01-30T21:40:44+00:00 [running]> on host b732220cc430
[2022-01-30, 21:40:48 UTC] {logging_mixin.py:109} WARNING - /opt/conda/lib/python3.9/site-packages/airflow/models/xcom.py:323 SAWarning: TypeDecorator UtcDateTime(timezone=True) will not produce a cache key because the ``cache_ok`` attribute is not set to True. This can have significant performance implications including some performance degradations in comparison to prior SQLAlchemy versions. Set this attribute to True if this type object's state is safe to use in a cache key, or False to disable this warning. (Background on this error at: https://sqlalche.me/e/14/cprf)
[2022-01-30, 21:40:48 UTC] {logging_mixin.py:109} WARNING - /opt/conda/lib/python3.9/site-packages/airflow/models/renderedtifields.py:126 SAWarning: TypeDecorator UtcDateTime(timezone=True) will not produce a cache key because the ``cache_ok`` attribute is not set to True. This can have significant performance implications including some performance degradations in comparison to prior SQLAlchemy versions. Set this attribute to True if this type object's state is safe to use in a cache key, or False to disable this warning. (Background on this error at: https://sqlalche.me/e/14/cprf)
[2022-01-30, 21:40:48 UTC] {logging_mixin.py:109} WARNING - /opt/conda/lib/python3.9/site-packages/airflow/models/renderedtifields.py:162 SAWarning: Coercing Subquery object into a select() for use in IN(); please pass a select() construct explicitly
[2022-01-30, 21:40:48 UTC] {logging_mixin.py:109} WARNING - /opt/conda/lib/python3.9/site-packages/airflow/models/renderedtifields.py:159 SAWarning: TypeDecorator UtcDateTime(timezone=True) will not produce a cache key because the ``cache_ok`` attribute is not set to True. This can have significant performance implications including some performance degradations in comparison to prior SQLAlchemy versions. Set this attribute to True if this type object's state is safe to use in a cache key, or False to disable this warning. (Background on this error at: https://sqlalche.me/e/14/cprf)
[2022-01-30, 21:40:48 UTC] {taskinstance.py:1427} INFO - Exporting the following env vars:
AIRFLOW_CTX_DAG_OWNER=airflow
AIRFLOW_CTX_DAG_ID=ml-intermediate
AIRFLOW_CTX_TASK_ID=get
AIRFLOW_CTX_EXECUTION_DATE=2022-01-30T21:40:44+00:00
AIRFLOW_CTX_DAG_RUN_ID=manual__2022-01-30T21:40:44+00:00
[2022-01-30, 21:40:48 UTC] {taskinstance.py:1703} ERROR - Task failed with exception
Traceback (most recent call last):
File "/opt/conda/lib/python3.9/site-packages/airflow/models/taskinstance.py", line 1332, in _run_raw_taskself._execute_task_with_callbacks(context)
File "/opt/conda/lib/python3.9/site-packages/airflow/models/taskinstance.py", line 1458, in _execute_task_with_callbacks
result =self._execute_task(context, self.task)
File "/opt/conda/lib/python3.9/site-packages/airflow/models/taskinstance.py", line 1514, in _execute_task
result = execute_callable(context=context)
File "/opt/conda/lib/python3.9/site-packages/airflow/providers/cncf/kubernetes/operators/kubernetes_pod.py", line 323, in execute
pod_list = client.list_namespaced_pod(self.namespace, label_selector=label_selector)
File "/opt/conda/lib/python3.9/site-packages/kubernetes/client/api/core_v1_api.py", line 12803, in list_namespaced_pod
(data) =self.list_namespaced_pod_with_http_info(namespace, **kwargs) # noqa: E501
File "/opt/conda/lib/python3.9/site-packages/kubernetes/client/api/core_v1_api.py", line 12850, in list_namespaced_pod_with_http_inforaiseValueError("Missing the required parameter `namespace` when calling `list_namespaced_pod`") # noqa: E501ValueError: Missing the required parameter `namespace` when calling `list_namespaced_pod`
[2022-01-30, 21:40:48 UTC] {taskinstance.py:1270} INFO - Marking task as FAILED. dag_id=ml-intermediate, task_id=get, execution_date=20220130T214044, start_date=20220130T214047, end_date=20220130T214048
[2022-01-30, 21:40:48 UTC] {standard_task_runner.py:88} ERROR - Failed to execute job 9 for task get
Traceback (most recent call last):
File "/opt/conda/lib/python3.9/site-packages/airflow/task/task_runner/standard_task_runner.py", line 85, in _start_by_fork
args.func(args, dag=self.dag)
File "/opt/conda/lib/python3.9/site-packages/airflow/cli/cli_parser.py", line 48, in commandreturn func(*args, **kwargs)
File "/opt/conda/lib/python3.9/site-packages/airflow/utils/cli.py", line 92, in wrapperreturn f(*args, **kwargs)
File "/opt/conda/lib/python3.9/site-packages/airflow/cli/commands/task_command.py", line 292, in task_run
_run_task_by_selected_method(args, dag, ti)
File "/opt/conda/lib/python3.9/site-packages/airflow/cli/commands/task_command.py", line 107, in _run_task_by_selected_method
_run_raw_task(args, ti)
File "/opt/conda/lib/python3.9/site-packages/airflow/cli/commands/task_command.py", line 180, in _run_raw_task
ti._run_raw_task(
File "/opt/conda/lib/python3.9/site-packages/airflow/utils/session.py", line 70, in wrapperreturn func(*args, session=session, **kwargs)
File "/opt/conda/lib/python3.9/site-packages/airflow/models/taskinstance.py", line 1332, in _run_raw_taskself._execute_task_with_callbacks(context)
File "/opt/conda/lib/python3.9/site-packages/airflow/models/taskinstance.py", line 1458, in _execute_task_with_callbacks
result =self._execute_task(context, self.task)
File "/opt/conda/lib/python3.9/site-packages/airflow/models/taskinstance.py", line 1514, in _execute_task
result = execute_callable(context=context)
File "/opt/conda/lib/python3.9/site-packages/airflow/providers/cncf/kubernetes/operators/kubernetes_pod.py", line 323, in execute
pod_list = client.list_namespaced_pod(self.namespace, label_selector=label_selector)
File "/opt/conda/lib/python3.9/site-packages/kubernetes/client/api/core_v1_api.py", line 12803, in list_namespaced_pod
(data) =self.list_namespaced_pod_with_http_info(namespace, **kwargs) # noqa: E501
File "/opt/conda/lib/python3.9/site-packages/kubernetes/client/api/core_v1_api.py", line 12850, in list_namespaced_pod_with_http_inforaiseValueError("Missing the required parameter `namespace` when calling `list_namespaced_pod`") # noqa: E501ValueError: Missing the required parameter `namespace` when calling `list_namespaced_pod`
[2022-01-30, 21:40:48 UTC] {local_task_job.py:154} INFO - Task exited with return code 1
[2022-01-30, 21:40:48 UTC] {local_task_job.py:264} INFO - 0 downstream tasks scheduled from follow-on schedule check
[2022-01-30, 21:40:48 UTC] {logging_mixin.py:109} WARNING - /opt/conda/lib/python3.9/site-packages/airflow/models/dag.py:1076 SADeprecationWarning: Query.value() is deprecated and will be removed in a future release. Please use Query.with_entities() in combination with Query.scalar() (deprecated since: 1.4)
[2022-01-30, 21:40:48 UTC] {logging_mixin.py:109} WARNING - /opt/conda/lib/python3.9/site-packages/airflow/models/dag.py:1076 SADeprecationWarning: Query.values() is deprecated and will be removed in a future release. Please use Query.with_entities() (deprecated since: 1.4)
The text was updated successfully, but these errors were encountered:
I tried running the example (using KubernetesPodOperator) and it failed, something in Airflow broke.
I tried downgrading Airflow, the k8s provider package, and the k8s package, but didn't find the solution.
*** Reading local file: /root/airflow/logs/ml-intermediate/get/2022-01-30T21:40:44+00:00/1.log [2022-01-30, 21:40:47 UTC] {taskinstance.py:1035} INFO - Dependencies all met for <TaskInstance: ml-intermediate.get manual__2022-01-30T21:40:44+00:00 [queued]> [2022-01-30, 21:40:47 UTC] {taskinstance.py:1035} INFO - Dependencies all met for <TaskInstance: ml-intermediate.get manual__2022-01-30T21:40:44+00:00 [queued]> [2022-01-30, 21:40:47 UTC] {taskinstance.py:1241} INFO - -------------------------------------------------------------------------------- [2022-01-30, 21:40:47 UTC] {taskinstance.py:1242} INFO - Starting attempt 1 of 1 [2022-01-30, 21:40:47 UTC] {taskinstance.py:1243} INFO - -------------------------------------------------------------------------------- [2022-01-30, 21:40:47 UTC] {taskinstance.py:1262} INFO - Executing <Task(KubernetesPodOperator): get> on 2022-01-30 21:40:44+00:00 [2022-01-30, 21:40:47 UTC] {standard_task_runner.py:52} INFO - Started process 374 to run task [2022-01-30, 21:40:47 UTC] {standard_task_runner.py:76} INFO - Running: ['airflow', 'tasks', 'run', 'ml-intermediate', 'get', 'manual__2022-01-30T21:40:44+00:00', '--job-id', '9', '--raw', '--subdir', 'DAGS_FOLDER/ml-intermediate.py', '--cfg-path', '/tmp/tmp9ongxm85', '--error-file', '/tmp/tmpuk0o_489'] [2022-01-30, 21:40:47 UTC] {standard_task_runner.py:77} INFO - Job 9: Subtask get [2022-01-30, 21:40:47 UTC] {logging_mixin.py:109} INFO - Running <TaskInstance: ml-intermediate.get manual__2022-01-30T21:40:44+00:00 [running]> on host b732220cc430 [2022-01-30, 21:40:48 UTC] {logging_mixin.py:109} WARNING - /opt/conda/lib/python3.9/site-packages/airflow/models/xcom.py:323 SAWarning: TypeDecorator UtcDateTime(timezone=True) will not produce a cache key because the ``cache_ok`` attribute is not set to True. This can have significant performance implications including some performance degradations in comparison to prior SQLAlchemy versions. Set this attribute to True if this type object's state is safe to use in a cache key, or False to disable this warning. (Background on this error at: https://sqlalche.me/e/14/cprf) [2022-01-30, 21:40:48 UTC] {logging_mixin.py:109} WARNING - /opt/conda/lib/python3.9/site-packages/airflow/models/renderedtifields.py:126 SAWarning: TypeDecorator UtcDateTime(timezone=True) will not produce a cache key because the ``cache_ok`` attribute is not set to True. This can have significant performance implications including some performance degradations in comparison to prior SQLAlchemy versions. Set this attribute to True if this type object's state is safe to use in a cache key, or False to disable this warning. (Background on this error at: https://sqlalche.me/e/14/cprf) [2022-01-30, 21:40:48 UTC] {logging_mixin.py:109} WARNING - /opt/conda/lib/python3.9/site-packages/airflow/models/renderedtifields.py:162 SAWarning: Coercing Subquery object into a select() for use in IN(); please pass a select() construct explicitly [2022-01-30, 21:40:48 UTC] {logging_mixin.py:109} WARNING - /opt/conda/lib/python3.9/site-packages/airflow/models/renderedtifields.py:159 SAWarning: TypeDecorator UtcDateTime(timezone=True) will not produce a cache key because the ``cache_ok`` attribute is not set to True. This can have significant performance implications including some performance degradations in comparison to prior SQLAlchemy versions. Set this attribute to True if this type object's state is safe to use in a cache key, or False to disable this warning. (Background on this error at: https://sqlalche.me/e/14/cprf) [2022-01-30, 21:40:48 UTC] {taskinstance.py:1427} INFO - Exporting the following env vars: AIRFLOW_CTX_DAG_OWNER=airflow AIRFLOW_CTX_DAG_ID=ml-intermediate AIRFLOW_CTX_TASK_ID=get AIRFLOW_CTX_EXECUTION_DATE=2022-01-30T21:40:44+00:00 AIRFLOW_CTX_DAG_RUN_ID=manual__2022-01-30T21:40:44+00:00 [2022-01-30, 21:40:48 UTC] {taskinstance.py:1703} ERROR - Task failed with exception Traceback (most recent call last): File "/opt/conda/lib/python3.9/site-packages/airflow/models/taskinstance.py", line 1332, in _run_raw_task self._execute_task_with_callbacks(context) File "/opt/conda/lib/python3.9/site-packages/airflow/models/taskinstance.py", line 1458, in _execute_task_with_callbacks result = self._execute_task(context, self.task) File "/opt/conda/lib/python3.9/site-packages/airflow/models/taskinstance.py", line 1514, in _execute_task result = execute_callable(context=context) File "/opt/conda/lib/python3.9/site-packages/airflow/providers/cncf/kubernetes/operators/kubernetes_pod.py", line 323, in execute pod_list = client.list_namespaced_pod(self.namespace, label_selector=label_selector) File "/opt/conda/lib/python3.9/site-packages/kubernetes/client/api/core_v1_api.py", line 12803, in list_namespaced_pod (data) = self.list_namespaced_pod_with_http_info(namespace, **kwargs) # noqa: E501 File "/opt/conda/lib/python3.9/site-packages/kubernetes/client/api/core_v1_api.py", line 12850, in list_namespaced_pod_with_http_info raise ValueError("Missing the required parameter `namespace` when calling `list_namespaced_pod`") # noqa: E501 ValueError: Missing the required parameter `namespace` when calling `list_namespaced_pod` [2022-01-30, 21:40:48 UTC] {taskinstance.py:1270} INFO - Marking task as FAILED. dag_id=ml-intermediate, task_id=get, execution_date=20220130T214044, start_date=20220130T214047, end_date=20220130T214048 [2022-01-30, 21:40:48 UTC] {standard_task_runner.py:88} ERROR - Failed to execute job 9 for task get Traceback (most recent call last): File "/opt/conda/lib/python3.9/site-packages/airflow/task/task_runner/standard_task_runner.py", line 85, in _start_by_fork args.func(args, dag=self.dag) File "/opt/conda/lib/python3.9/site-packages/airflow/cli/cli_parser.py", line 48, in command return func(*args, **kwargs) File "/opt/conda/lib/python3.9/site-packages/airflow/utils/cli.py", line 92, in wrapper return f(*args, **kwargs) File "/opt/conda/lib/python3.9/site-packages/airflow/cli/commands/task_command.py", line 292, in task_run _run_task_by_selected_method(args, dag, ti) File "/opt/conda/lib/python3.9/site-packages/airflow/cli/commands/task_command.py", line 107, in _run_task_by_selected_method _run_raw_task(args, ti) File "/opt/conda/lib/python3.9/site-packages/airflow/cli/commands/task_command.py", line 180, in _run_raw_task ti._run_raw_task( File "/opt/conda/lib/python3.9/site-packages/airflow/utils/session.py", line 70, in wrapper return func(*args, session=session, **kwargs) File "/opt/conda/lib/python3.9/site-packages/airflow/models/taskinstance.py", line 1332, in _run_raw_task self._execute_task_with_callbacks(context) File "/opt/conda/lib/python3.9/site-packages/airflow/models/taskinstance.py", line 1458, in _execute_task_with_callbacks result = self._execute_task(context, self.task) File "/opt/conda/lib/python3.9/site-packages/airflow/models/taskinstance.py", line 1514, in _execute_task result = execute_callable(context=context) File "/opt/conda/lib/python3.9/site-packages/airflow/providers/cncf/kubernetes/operators/kubernetes_pod.py", line 323, in execute pod_list = client.list_namespaced_pod(self.namespace, label_selector=label_selector) File "/opt/conda/lib/python3.9/site-packages/kubernetes/client/api/core_v1_api.py", line 12803, in list_namespaced_pod (data) = self.list_namespaced_pod_with_http_info(namespace, **kwargs) # noqa: E501 File "/opt/conda/lib/python3.9/site-packages/kubernetes/client/api/core_v1_api.py", line 12850, in list_namespaced_pod_with_http_info raise ValueError("Missing the required parameter `namespace` when calling `list_namespaced_pod`") # noqa: E501 ValueError: Missing the required parameter `namespace` when calling `list_namespaced_pod` [2022-01-30, 21:40:48 UTC] {local_task_job.py:154} INFO - Task exited with return code 1 [2022-01-30, 21:40:48 UTC] {local_task_job.py:264} INFO - 0 downstream tasks scheduled from follow-on schedule check [2022-01-30, 21:40:48 UTC] {logging_mixin.py:109} WARNING - /opt/conda/lib/python3.9/site-packages/airflow/models/dag.py:1076 SADeprecationWarning: Query.value() is deprecated and will be removed in a future release. Please use Query.with_entities() in combination with Query.scalar() (deprecated since: 1.4) [2022-01-30, 21:40:48 UTC] {logging_mixin.py:109} WARNING - /opt/conda/lib/python3.9/site-packages/airflow/models/dag.py:1076 SADeprecationWarning: Query.values() is deprecated and will be removed in a future release. Please use Query.with_entities() (deprecated since: 1.4)The text was updated successfully, but these errors were encountered: