Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Behavior validation docs #408

Merged
merged 27 commits into from
May 23, 2024
Merged

Conversation

iamleeg
Copy link
Contributor

@iamleeg iamleeg commented May 9, 2024

Introduces additional documentation for expectations and confirmations.

Checklist:

  • Code and documentation should follow the style of the Style Guide.
  • If public symbols are renamed or modified, DocC references should be updated.

@iamleeg iamleeg added documentation Improvements or additions to documentation enhancement New feature or request labels May 9, 2024
@iamleeg iamleeg self-assigned this May 9, 2024
@iamleeg
Copy link
Contributor Author

iamleeg commented May 9, 2024

@swift-ci please test


## Overview

The testing library provides `#expect()` and `#require()` macros you use to
validate expected outcomes. To validate that an error is thrown, or _not_ thrown,
Use `#expect()` and `#require()` macros to validate expected outcomes.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please use double-backticks and link to the expect(_:_:) macro rather than writing #expect(). Ibid for #require(), but note the overloads are different. (This is an error in the existing documentation.)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On first use only, or every time?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Every time: DocC does not render macros with leading # or @ characters, so having them suddenly appear is likely to be confusing. If, later, DocC does add those characters, we'll get them everywhere "for free".

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

However, if your department already has guidelines for this sort of thing, I defer to you.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FWIW, what a developer types is #expect() or #require(), so I think it's more helpful to show them that; nonetheless, I'll change this to match the local style.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'd much prefer the DocC compiler would add the # for us in this case. There's an issue or radar or some such floating around out there about it.


### Validate that your code doesn't throw an error

Validate that the code under test doesn't throw an error by comparing the error to `Never`:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's more nuance here: a throwing test function is almost always sufficient:

@Test func f() throws {
  try g() // if this throws, the test fails
}

#expect(throws: Never.self) is useful when the potentially-throwing expression is part of a larger operation, but a thrown error should not interrupt execution. See the documentation for #expect(throws: Never.self) itself for some more discussion.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you think of this, is it sufficient or would an example help?


### Validate that your code doesn't throw an error

Validate that the code under test doesn't throw an error by comparing
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should emphasize here that a test function that throws will fail if an error is thrown, so usually you don't need to use #expect(throws: Never.self). You'd really only use it if you want to record an issue for a thrown error but not stop execution of the test function. It's pretty nuanced, to the point that I'm not sure we should even call it out in the documentation.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's some history here that we don't need to document, but for your own edification/amusement: XCTest in Objective-C has XCTAssertNoThrow() that asserts a given expression doesn't throw an Objective-C or C++ exception. This is different from a Swift error because if an exception is thrown through a test method's frame, it tends to wreak some havoc on the way out before being caught by XCTest, so you want to proactively catch them if you know they might occur. This was inherited as XCTAssertNoThrow() in Swift which checks if a function throws a Swift error rather than an exception.

case pineapple
}

struct PizzaToppings {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You have a Toppings type and a PizzaToppings type. But this type also describes the base. Perhaps this should just be Pizza and the other types should be nested Pizza.Topping and Pizza.Base?

Copy link
Contributor

@grynspan grynspan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See comments above.

Copy link
Contributor

@chuckdude chuckdude left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK from Editorial. Looks like there are still issues to sort out with the code. If those changes affect the text, please feel free to pass this by me for another look.

Sources/Testing/Testing.docc/testing-asynchronous-code.md Outdated Show resolved Hide resolved
Sources/Testing/Testing.docc/testing-asynchronous-code.md Outdated Show resolved Hide resolved
Sources/Testing/Testing.docc/Expectations.md Outdated Show resolved Hide resolved
Sources/Testing/Testing.docc/Expectations.md Outdated Show resolved Hide resolved
Sources/Testing/Testing.docc/Expectations.md Outdated Show resolved Hide resolved
Sources/Testing/Testing.docc/Expectations.md Outdated Show resolved Hide resolved
Copy link
Contributor

@chuckdude chuckdude left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All good here. Thanks.

@grynspan
Copy link
Contributor

Looks good. Documentation only, so skipping CI.

@grynspan grynspan merged commit a5b39aa into main May 23, 2024
@grynspan grynspan deleted the iamleeg/123052393-behavior-validation-docs branch May 23, 2024 16:04
stmontgomery added a commit that referenced this pull request May 24, 2024
Small fixup to the changes landed in #408 to fix the capitalization of
two DocC symbol references:

```
⚠️ Sources/Testing/Testing.docc/testing-for-errors-in-swift-code.md:29:21:
'expect(throws:_:sourcelocation:performing:)-1xr34' doesn't exist at '/Testing/testing-for-errors-in-swift-code'
```

### Checklist:

- [x] Code and documentation should follow the style of the [Style
Guide](https://github.com/apple/swift-testing/blob/main/Documentation/StyleGuide.md).
- [x] If public symbols are renamed or modified, DocC references should
be updated.
stmontgomery added a commit that referenced this pull request May 24, 2024
Small fixup to the changes landed in #408 to fix the capitalization of
two DocC symbol references:

```
⚠️ Sources/Testing/Testing.docc/testing-for-errors-in-swift-code.md:29:21:
'expect(throws:_:sourcelocation:performing:)-1xr34' doesn't exist at '/Testing/testing-for-errors-in-swift-code'
```

### Checklist:

- [x] Code and documentation should follow the style of the [Style
Guide](https://github.com/apple/swift-testing/blob/main/Documentation/StyleGuide.md).
- [x] If public symbols are renamed or modified, DocC references should
be updated.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants