Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug 1863187 - Add job description to the job info, and also add full job description tab. #7868

Draft
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

arai-a
Copy link
Contributor

@arai-a arai-a commented Nov 8, 2023

for https://bugzilla.mozilla.org/show_bug.cgi?id=1863187

Added 2 feature:

  • show "job description" field into job info pane, under "job name" field
    • this shows the first line of the job's description field in the source YAML file
  • add "Full Description" tab
    • this shows the job's description field in the source YAML file
    • the field can be multi-line, and I'm about to put more info there, such as "how to run locally", and possibly a URL of document

This modifies the database in the following way:

  • job_desc table is added
    • this is used for deduplicating the description field across multiple jobs
  • job_desc_id column is added to job table
    • this contains the job_desc table's id
    • this column defaults to an empty text in job_desc table

The job's description field is retrieved via the pulse's jobInfo.summary property, that seems to be the result of the following.

https://searchfox.org/mozilla-central/rev/02841791400cf7cf5760c0cfaf31f5d772624253/taskcluster/gecko_taskgraph/transforms/task.py#2004

"task-reference": "{description} ([Treeherder job]({th_job_link}))".format(

It contains the concatenation of the job's description field and the treeherder job link.
job_loader drops the treehreder job link part and uses it as description.
if there's better way to get the raw description field, let me know.

Concern:

  • job_desc table will store all description values, and whenever the description is modified (such as, adding more info "how to run locally", URL of documents, etc), a new row is added, and the old row is kept around

@arai-a
Copy link
Contributor Author

arai-a commented Nov 8, 2023

@arai-a
Copy link
Contributor Author

arai-a commented Nov 8, 2023

Another concern is that the description is shown also in the taskcluster task page, and the view doesn't seem to expect multi-line string.
https://firefox-ci-tc.services.mozilla.com/tasks/Zfq9gthfTYCFBoUSQPdC8A

@arai-a arai-a changed the title Bug 1863187 - Add job description to the job info, and also add fu\ ll job description tab. Bug 1863187 - Add job description to the job info, and also add full job description tab. Nov 8, 2023
@arai-a
Copy link
Contributor Author

arai-a commented Nov 8, 2023

it turns out there's extra field in the job/task schema, which is suitable for putting documentation like "how to run locally".
also using it won't break the taskcluster task page's title.
I'll look into it.

@arai-a arai-a marked this pull request as draft November 8, 2023 15:08
@@ -167,6 +169,7 @@ class TabsPanel extends React.Component {
) : (
<Tab disabled>Test Groups</Tab>
)}
<Tab>Full Description</Tab>
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: to save UI space, can we make this Description ?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have an 'Artifacts and Debugging' tab and this is about debugging. The artifacts list can get long but putting the instructions how to run the task locally above the artifacts if preferable. (If the instructions are below, the might be missed. The same applies to the artifacts, maybe there should be a first line which tells users to scroll down for artifacts?)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what's existing "Debugging Tools" ?
if there's no "debugging" right now, could we make the existing tab "Artifacts", and add "Debugging" as separate tab?

@@ -558,6 +571,9 @@ class Job(models.Model):
machine = models.ForeignKey(Machine, on_delete=models.CASCADE)
option_collection_hash = models.CharField(max_length=64)
job_type = models.ForeignKey(JobType, on_delete=models.CASCADE, related_name='jobs')
job_desc = models.ForeignKey(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The job description information is only needed when a job gets selected. The information about the job type is also used in that view.
The job description should be a column of the job_type table. This means we only store the data for a few thousand rows and not for many millions.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is the scenario of the description getting updated in one tree or e.g. try.

Recommendation:

  1. If there is no job description for the task, store it independent of tree.
  2. If there is a job description and the task is central or autoland, update it.
  3. No action for other cases.

This could also be tracked by a job_type.id + repository.id key.

@@ -69,6 +69,8 @@ def to_representation(self, job):
'job_type_id': job.job_type_id,
'job_type_name': job.job_type.name,
'job_type_symbol': job.job_type.symbol,
'job_desc_id': job.job_desc_id,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This data seems unused by the frontend.

@Archaeopteryx
Copy link
Collaborator

Concern:

* `job_desc` table will store all `description` values, and whenever the `description` is modified (such as, adding more info "how to run locally", URL of documents, etc), a new row is added, and the old row is kept around

When all jobs which use the description have been purged 120 days after their submission, the description will also be deleted because of the CASCADE rule.

@arai-a
Copy link
Contributor Author

arai-a commented Nov 9, 2023

thank you all for reviewing!
opened #7869 with those comments addressed, and also switched to using extra.local-run field of the YAML file, to avoid breaking the TaskCluster page.

@arai-a
Copy link
Contributor Author

arai-a commented Nov 9, 2023

For this PR, I'll rework it to handle single-line description, and show it only in the job info pane,
after #7869 .

@arai-a
Copy link
Contributor Author

arai-a commented Dec 26, 2023

So far prepared a patch to add job description in the job info, based on #7869

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants