Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added logic to clean up executor interchange log #513

Closed
wants to merge 2 commits into from

Conversation

yongyanrao
Copy link
Contributor

As described in Issue #511, for endpoint, there is a log line being generated every second for executor interchange, even there is no outstanding task. It is good for debugging, but might not be the case for production running. This PR changes the log level from info to debug, when there is no outstanding task.

@yongyanrao yongyanrao linked an issue Jun 11, 2021 that may be closed by this pull request
@codecov
Copy link

codecov bot commented Jun 11, 2021

Codecov Report

Merging #513 (91c96ce) into main (fa66411) will decrease coverage by 2.38%.
The diff coverage is 0.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #513      +/-   ##
==========================================
- Coverage   38.52%   36.14%   -2.39%     
==========================================
  Files          27       27              
  Lines        2855     3511     +656     
==========================================
+ Hits         1100     1269     +169     
- Misses       1755     2242     +487     
Impacted Files Coverage Δ
funcx_endpoint/funcx_endpoint/strategies/simple.py 20.28% <0.00%> (-2.30%) ⬇️
...dpoint/funcx_endpoint/endpoint/endpoint_manager.py 55.85% <0.00%> (-3.59%) ⬇️
...ncx_endpoint/executors/high_throughput/executor.py 31.07% <0.00%> (-1.44%) ⬇️
...ncx_endpoint/executors/high_throughput/messages.py 58.06% <0.00%> (-1.26%) ⬇️
...utors/high_throughput/interchange_task_dispatch.py 8.48% <0.00%> (-1.09%) ⬇️
..._endpoint/executors/high_throughput/interchange.py 10.52% <0.00%> (-0.99%) ⬇️
...x_endpoint/executors/high_throughput/worker_map.py 47.44% <0.00%> (-0.91%) ⬇️
...nt/executors/high_throughput/test_funcx_manager.py 100.00% <0.00%> (ø)
...ndpoint/executors/high_throughput/funcx_manager.py 32.63% <0.00%> (+2.40%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update fa66411...91c96ce. Read the comment docs.

@@ -47,7 +47,13 @@ def strategize(self, *args, **kwargs):

def _strategize(self, *args, **kwargs):
task_breakdown = self.interchange.get_outstanding_breakdown()
logger.info(f"Task breakdown {task_breakdown}")
num_tasks = 0
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no need to sum here. We can log if there is any non-zero entry, which is a bit faster.

@sirosen
Copy link
Member

sirosen commented Jan 11, 2022

Although I don't know what that "breakdown" object is, this seems to me like an unnecessary change which potentially loses information.

@yadudoc, I'd like to close (not merge) this to cleanup our outstanding PRs. Do you agree?

@yadudoc
Copy link
Collaborator

yadudoc commented Jan 12, 2022

@sirosen I agree. Sorry, I did not see your comment here sooner.

@yadudoc yadudoc closed this Jan 12, 2022
@yadudoc yadudoc deleted the yrao2-issue-511 branch January 12, 2022 16:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Clean up executor interchange's log
4 participants