Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Umbrella] Enhance dashboard capabilities. #1401

Open
3 tasks done
yl09099 opened this issue Dec 28, 2023 · 13 comments · May be fixed by #1449 or #1430
Open
3 tasks done

[Umbrella] Enhance dashboard capabilities. #1401

yl09099 opened this issue Dec 28, 2023 · 13 comments · May be fixed by #1449 or #1430

Comments

@yl09099
Copy link
Contributor

yl09099 commented Dec 28, 2023

Code of Conduct

Search before asking

  • I have searched in the issues and found no similar issues.

Describe the proposal

You can configure one dashboard to manage multiple clusters, and add dashboard management functions for Coordinators and ShuffleServer.Reference documents for specific functions:
https://docs.google.com/document/d/1pR-pTn3aIZHpKSYDmFVdTijDwaT7INFVXJd5VG0WVpg/edit

Task list

  1. Added the ability for the dashboard to connect to multiple coordinators and shuffle servers.
  2. Add sorting and search functions to the dashboard list area for ease of use.
  3. Added the dashboard account password login function.
  4. Dashboard added functions for adding and deleting ShuffleServer nodes offline and blacklist.
  5. Added monitoring status indicators of the ShuffleServer node.

Are you willing to submit PR?

  • Yes I am willing to submit a PR!
@yl09099
Copy link
Contributor Author

yl09099 commented Dec 28, 2023

@zuston @jerqi @lifeSo Let's see if these functions make sense.

@lifeSo
Copy link
Collaborator

lifeSo commented Dec 28, 2023

@zuston @jerqi @lifeSo Let's see if these functions make sense.

OK, I am new to dashboard, let's me see see first and will try.

@yl09099
Copy link
Contributor Author

yl09099 commented Jan 2, 2024

Is the current dashboard functioning correctly? I see many 404 error requests on all dashboard pages. image image image @yl09099 @lifeSo @jerqi

Is the code up to date?

@yl09099
Copy link
Contributor Author

yl09099 commented Jan 2, 2024

It's up to date. I don't know why this is happening. Am I missing something? @yl09099

Is the Coordinator interface access address correctly configured? The error is that the interface cannot be accessed,Refer to configuration here:https://github.com/apache/incubator-uniffle/blob/master/docs/dashboard_guide.md.

@yl09099
Copy link
Contributor Author

yl09099 commented Jan 2, 2024

So I am just wondering, if the latest code works fine?

You should have no problem using the latest version of the code.

@lifeSo lifeSo linked a pull request Jan 10, 2024 that will close this issue
yl09099 added a commit to yl09099/incubator-uniffle that referenced this issue Jan 14, 2024
yl09099 added a commit to yl09099/incubator-uniffle that referenced this issue Jan 14, 2024
yl09099 added a commit to yl09099/incubator-uniffle that referenced this issue Jan 15, 2024
yl09099 added a commit to yl09099/incubator-uniffle that referenced this issue Jan 15, 2024
@rickyma
Copy link
Contributor

rickyma commented Jan 15, 2024

Can we display usedDirectMemory on the Dashboard's shuffle server page(which is very useful in my opinion)? Maybe add a column? If you think there are too many columns and it doesn't look nice, can we remove the first column Id? Displaying Id on the page doesn't serve any purpose, right? Id can be composed of the values in the following columns (IP + GrpcPort + NettyPort).

PTAL @yl09099

@lifeSo
Copy link
Collaborator

lifeSo commented Jan 16, 2024

I personly do not think add usedDirectMemory is necessary if your company have other monitor system like:

image

If I want to see the metric, I will use metric system instead of dashboard.

@lifeSo
Copy link
Collaborator

lifeSo commented Jan 16, 2024

Id can be composed of the values in the following columns (IP + GrpcPort + NettyPort).

@yl09099 What do you think ?

@rickyma
Copy link
Contributor

rickyma commented Jan 16, 2024

image
usedDirectMemory has already become a very important metric in the latest Uniffle(especially when enabling Netty).

I'm just making a suggestion for your reference. Since the Id is not very useful anymore(actually useless), you might as well conveniently replace it with usedDirectMemory.

@lifeSo

@lifeSo
Copy link
Collaborator

lifeSo commented Jan 16, 2024

It is my personal opinion, if you persist it.
I just think dashboard is the place to operate server, and there is monitor system to see metric.
@jerqi @yl09099 What's your idea?

yl09099 added a commit to yl09099/incubator-uniffle that referenced this issue Jan 21, 2024
@yl09099
Copy link
Contributor Author

yl09099 commented Jan 24, 2024

Can we display usedDirectMemory on the Dashboard's shuffle server page(which is very useful in my opinion)? Maybe add a column? If you think there are too many columns and it doesn't look nice, can we remove the first column Id? Displaying Id on the page doesn't serve any purpose, right? Id can be composed of the values in the following columns (IP + GrpcPort + NettyPort).

PTAL @yl09099

I don't think it's useful either. We can change it.

@yl09099
Copy link
Contributor Author

yl09099 commented Jan 24, 2024

I personly do not think add usedDirectMemory is necessary if your company have other monitor system like:

image

If I want to see the metric, I will use metric system instead of dashboard.

I also do not have what good idea, document change inside, https://docs.google.com/document/d/1pR-pTn3aIZHpKSYDmFVdTijDwaT7INFVXJd5VG0WVpg/edit

@yl09099
Copy link
Contributor Author

yl09099 commented Jan 24, 2024

Id can be composed of the values in the following columns (IP + GrpcPort + NettyPort).

@yl09099 What do you think ?

I think it's useless. We can change it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants