Kubernetes Async API Calls with Callbacks Support

[dirrao]

Kubernetes API calls do support async operations. As of now, we don't a have to provide the callbacks for async calls. It would be good idea to provide success and failure callbacks to avoid polling for the result.

Example: create_namespaced_pod(namespace, body, async_req=True, async_callback, async_error_callback)

[dirrao]

I would like to submit the PR if we align on this.

[roycaihw]

Upstream Kubernetes API doesn't support such callback mechanism AFAICT. Could you check with the upstream on this feature request?

[dirrao]

@roycaihw
Async calls are submitted through python thread pool. Python thread pool does support callbacks.

Check api_client.py for the same

        if not async_req:
            return self.__call_api(resource_path, method,
                                   path_params, query_params, header_params,
                                   body, post_params, files,
                                   response_type, auth_settings,
                                   _return_http_data_only, collection_formats,
                                   _preload_content, _request_timeout, _host)

        return self.pool.apply_async(self.__call_api, (resource_path,
                                                       method, path_params,
                                                       query_params,
                                                       header_params, body,
                                                       post_params, files,
                                                       response_type,
                                                       auth_settings,
                                                       _return_http_data_only,
                                                       collection_formats,
                                                       _preload_content,
                                                       _request_timeout,
                                                       _host))

[roycaihw]

CC @tomplus Could you take a look?

[tomplus]

apply_async returns AsyncResult so as you've written it doesn't support callbacks. These functions are auto-generated by OpenAPI Generator so it's not easy to introduce such change.

I can suggest using sync api calls but from a dedicated thread pool where you can add callbacks as you need. What do you think?

[dirrao]

@tomplus
We were avoiding the synchronous requests due to performance reasons. It would be great if we make the requests asyncronous with callbacks support.

[tomplus]

Correct me if I'm wrong but you want to have something like this:

create_namespaced_pod("pod-1", "ns-1", cb_success, cb_error)
create_namespaced_pod("pod-2", "ns-2", cb_success, cb_error)

so instead of using async_req i suggest you implement it in this way (draft):

def cb_helper(func_name, cb_success, cb_error, *args, **kwargs):
    try:
       result = func_name(*args, **kwargs)
    except Exception as ex:
       cb_error(ex)
    cb_success(result)

pool = with ThreadPool()
pool.apply_async(cb_helper, create_namespaced_pod, "pod-1", "ns-1", cb_success, cb_error)
pool.apply_async(cb_helper, create_namespaced_pod, "pod-2", "ns-2", cb_success, cb_error)    
....
....

so it uses synchronous requests but from threads so main thread is not blocked...