You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I had searched in the issues and found no similar feature requirement.
Description
We have these feature within our company and hope to contribute them. What feature communities require?
Worker and master grayscale function based on traffic
Optimization of worker and master load protection strategies (optimizing remaining physical memory and CPU algorithms on the machine) to address the issue of worker downtime
Force skip dependency detection during complement
Heartbeat reporting from master and worker to detect disconnection issues
Write the Yarn appid into the task_instance table during task execution to facilitate worker fault tolerance (solving the problem of worker machine downtime and inability to tolerate faults)
Workflow migration project interface
Recommend users to set the cron for the workflow (query the completion time of all upstream workflows in the last 7 days and then recommend the start time of this workflow)
After the task timeout fails, it is necessary to kill the Yarn task
Change workflow instances and task instances to single table queries to solve the problem of slow queries
Provide a batch recovery failure interface to solve the problem of slow recovery of workflow instances caused by cluster or MySQL issues
Add feature control tables to MySQL. All important functions require parameter control to turn on and off
SbloodyS
changed the title
[Improvement][all] We are ready to contribute multiple features
[Feature][all] We are ready to contribute multiple features
Apr 28, 2024
What causes the fourth problem?
And I have some suggest:
1.The fifth problem may cause task loss. A data processing task may contain more then one yarn job. If it fails between yarn tasks, this may cause the fault-tolerant task to succeed but miss some yarn task submissions.
2.Can the eighth one be configured? The easy failure of some long tasks may lead to a waste of complement resources.
Please Create DSIP to describe why you want to do this change, it's hard to know what you want to do by a simple description, and some of these should already been supported.
What causes the fourth problem? And I have some suggest: 1.The fifth problem may cause task loss. A data processing task may contain more then one yarn job. If it fails between yarn tasks, this may cause the fault-tolerant task to succeed but miss some yarn task submissions. 2.Can the eighth one be configured? The easy failure of some long tasks may lead to a waste of complement resources.
What causes the fourth problem?
Mysql network exception for a few minutes
A data processing task may contain more then one yarn job。
It will write multiple appids to the task_instance table
Search before asking
Description
We have these feature within our company and hope to contribute them. What feature communities require?
Are you willing to submit a PR?
Code of Conduct
The text was updated successfully, but these errors were encountered: