Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

lib/uktest: Introduce FIX ME assertion in uktest #1401

Open
wants to merge 1 commit into
base: staging
Choose a base branch
from

Conversation

YuanSamuel
Copy link

@YuanSamuel YuanSamuel commented Apr 30, 2024

Prerequisite checklist

  • Read the contribution guidelines regarding submitting new changes to the project;
  • Tested your changes against relevant architectures and platforms;
  • Ran the checkpatch.uk on your commit series before opening this PR;
  • Updated relevant documentation.

Base target

  • Architecture(s): N/A
  • Platform(s): N/A
  • Application(s): N/A

Additional configuration

  • CONFIG_LIBUKTEST_STRICT_MODE=y

Description of changes

This PR attempts to address #1027 , which introduces a new FIX ME assertion to downgrade known failing tests. Tests marked as FIX_ME will show up as OMITTED in the testing logs, and will be counted toward the overall failure count only if the strict mode config is turned on.

An example output log is below:
image

GitHub-Closes: #1027

@YuanSamuel YuanSamuel requested a review from a team as a code owner April 30, 2024 20:57
@github-actions github-actions bot added area/lib Internal Unikraft Microlibrary lang/c Issues or PRs to do with C/C++ labels Apr 30, 2024
@YuanSamuel
Copy link
Author

Hey team!

As mentioned in #1027 , I've made some initial changes in order to support the FIX_ME test downgrades.

These are just some initial changes, please feel free to leave any feedback before I add support for more tests.

Thanks!

@nderjung nderjung changed the title Introduce FIX ME assertion in uktest lib/uktest: Introduce FIX ME assertion in uktest May 2, 2024
@YuanSamuel
Copy link
Author

Hey all, I've gone ahead and integrated the rest of appropriate FIX ME downgrades for tests. Please feel free to leave any more feedback!

Signed-off-by: syuan <syuan@node0.syuan-204435.wisr-pg0.utah.cloudlab.us>

lib/uktest: formatting changes

Signed-off-by: syuan <syuan@node0.syuan-204435.wisr-pg0.utah.cloudlab.us>

lib/uktest: formatting changes

Signed-off-by: syuan <syuan@node0.syuan-204435.wisr-pg0.utah.cloudlab.us>

lib/uktest: add rest of tests

Signed-off-by: syuan <syuan@node0.syuan-206086.wisr-pg0.utah.cloudlab.us>

lib/uktest: style changes

Signed-off-by: syuan <syuan@node0.syuan-206086.wisr-pg0.utah.cloudlab.us>

lib/uktest: style changes

Signed-off-by: syuan <syuan@node0.syuan-206086.wisr-pg0.utah.cloudlab.us>

Adds FIX ME test downgrade to uktest testing suite

lib/uktest: style changes
Copy link
Member

@nderjung nderjung left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks so much for tackling this and apologies for the delay in getting back to you!

On my side I still need to test this properly and will get back to you soon with the status of my tests.

I have one inline comment, but additionally, the FIXME macro should be ansi-coloured into yellow (similar to pass/fail).

@@ -29,4 +29,8 @@ config LIBUKTEST_TEST_MYSELF
bool "Run self-test to check sanity"
default n

config LIBUKTEST_STRICT_MODE
bool "Count failing FIX_ME tests"
Copy link
Member

@nderjung nderjung May 2, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To keep things consistent internally, lets ensure FIXME is a single word, i.e.:

- 	bool "Count failing FIX_ME tests"
+ 	bool "Count failing FIXME tests"

Please ensure all other instances of "FIX ME" and "FIX_ME" (including commits) are adjusted to "FIXME".

I think the only exception should be the visual message displayed in the console log.

@razvand razvand self-assigned this May 21, 2024
@razvand razvand added this to the v0.17.0 (Calypso) milestone May 21, 2024
@razvand
Copy link
Contributor

razvand commented May 21, 2024

@YuanSamuel , could you please update this PR to include the feedback from @nderjung ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/lib Internal Unikraft Microlibrary lang/c Issues or PRs to do with C/C++
Projects
Status: 🏗 In Progress
Development

Successfully merging this pull request may close these issues.

Introduce FIX ME assertion in uktest
3 participants