Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] [dolphinscheduler-master] The handling of task dependency logic is problematic. #15927

Open
3 tasks done
ILZZY opened this issue Apr 28, 2024 · 1 comment
Open
3 tasks done
Labels
bug Something isn't working Waiting for reply Waiting for reply

Comments

@ILZZY
Copy link

ILZZY commented Apr 28, 2024

Search before asking

  • I had searched in the issues and found no similar issues.

What happened

org.apache.dolphinscheduler.server.master.utils.DependentExecute#findLastProcessInterval

491714282989_ pic

Considering tasks A and B, where task B is contingent upon the completion of task A, and this dependency is established within the operational period of the current month. It is known that task B initiated its first successful run on the 28th day and task A is configured to be scheduled 2 minutes subsequent to B's scheduling time. The present scenario involves an evaluation logic that has segmented the month into a period ranging from April 1st to April 30th. Upon assessing April 1st – a date for which no instance of task B exists – the logic promptly returns a 'waiting' status without proceeding to evaluate the remaining days of the month.

What you expected to happen

Given that the dependency timeframe is set for the current month, and there indeed exists a successful execution instance of task B within this month, the expected outcome of the dependency check should logically yield a 'success' status. This, in turn, would permit task A to proceed with its scheduled execution.

How to reproduce

  1. Task A schedule is [0 30 10 28 4 ? *]
  2. TaskB schedule is [0 32 10 28 4 ? *]
  3. Task B relies on Task A, with the dependency anchored within the business cycle of the current month.

Anything else

No response

Version

dev

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

@ILZZY ILZZY added bug Something isn't working Waiting for reply Waiting for reply labels Apr 28, 2024
@CloudSen
Copy link

I think the logic behind the dependent node design should be that for it to be considered successful, every time slice needs successful results. For periods like this month or this week, it means starting from the first day right up to today, there has to be a successful dependent task each day for it to count as a success. If even one day is missing a task run, or the tasks on that day weren't successful, then the whole task is marked as a failure. It will be checked again during the next scheduled run. The product team should probably think that if a day was missed, someone needs to manually run the tasks for that day to make up for it. Otherwise, they might just 'force success'.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working Waiting for reply Waiting for reply
Projects
None yet
Development

No branches or pull requests

2 participants