Skip to content

Commit

Permalink
WIP
Browse files Browse the repository at this point in the history
  • Loading branch information
mjerabek authored and horenmar committed Oct 28, 2021
1 parent 71c97fd commit b1fcb70
Show file tree
Hide file tree
Showing 5 changed files with 28 additions and 32 deletions.
6 changes: 3 additions & 3 deletions docs/command-line.md
Expand Up @@ -178,7 +178,7 @@ If one or more test-specs have been supplied too then only the matching tests wi

<a id="sending-output-to-a-file"></a>
## Sending output to a file
<pre>-o, --out &lt;filename>
<pre>-o, --out &lt;filename&gt;
</pre>

Use this option to send all output to a file. By default output is sent to stdout (note that uses of stdout and stderr *from within test cases* are redirected and included in the report - so even stderr will effectively end up on stdout).
Expand Down Expand Up @@ -406,15 +406,15 @@ There are some limitations of this feature to be aware of:
- Code outside of sections being skipped will still be executed - e.g. any set-up code in the TEST_CASE before the
start of the first section.</br>
- At time of writing, wildcards are not supported in section names.
- If you specify a section without narrowing to a test case first then all test cases will be executed
- If you specify a section without narrowing to a test case first then all test cases will be executed
(but only matching sections within them).


<a id="filenames-as-tags"></a>
## Filenames as tags
<pre>-#, --filenames-as-tags</pre>

When this option is used then every test is given an additional tag which is formed of the unqualified
When this option is used then every test is given an additional tag which is formed of the unqualified
filename it is found in, with any extension stripped, prefixed with the `#` character.

So, for example, tests within the file `~\Dev\MyProject\Ferrets.cpp` would be tagged `[#Ferrets]`.
Expand Down
4 changes: 2 additions & 2 deletions docs/reporters.md
Expand Up @@ -19,13 +19,13 @@ There are four reporters built in to the single include:
* `console` writes as lines of text, formatted to a typical terminal width, with colours if a capable terminal is detected.
* `compact` similar to `console` but optimised for minimal output - each entry on one line
* `junit` writes xml that corresponds to Ant's [junitreport](http://help.catchsoftware.com/display/ET/JUnit+Format) target. Useful for build systems that understand Junit.
Because of the way the junit format is structured the run must complete before anything is written.
Because of the way the junit format is structured the run must complete before anything is written.
* `xml` writes an xml format tailored to Catch. Unlike `junit` this is a streaming format so results are delivered progressively.

There are a few additional reporters, for specific build systems, in the Catch repository (in `include\reporters`) which you can `#include` in your project if you would like to make use of them.
Do this in one source file - the same one you have `CATCH_CONFIG_MAIN` or `CATCH_CONFIG_RUNNER`.

* `teamcity` writes the native, streaming, format that [TeamCity](https://www.jetbrains.com/teamcity/) understands.
* `teamcity` writes the native, streaming, format that [TeamCity](https://www.jetbrains.com/teamcity/) understands.
Use this when building as part of a TeamCity build to see results as they happen ([code example](../examples/207-Rpt-TeamCityReporter.cpp)).
* `tap` writes in the TAP ([Test Anything Protocol](https://en.wikipedia.org/wiki/Test_Anything_Protocol)) format.
* `automake` writes in a format that correspond to [automake .trs](https://www.gnu.org/software/automake/manual/html_node/Log-files-generation-and-test-results-recording.html) files
Expand Down
20 changes: 12 additions & 8 deletions src/catch2/catch_config.cpp
Expand Up @@ -20,16 +20,18 @@ namespace Catch {
namespace {
class RDBufStream : public IStream {
mutable std::ostream m_os;

public:
// Note that the streambuf must live at least as long as this object.
RDBufStream(std::streambuf *sb) : m_os( sb ) {}
// Note that the streambuf must live at least as long as this
// object.
RDBufStream( std::streambuf* sb ): m_os( sb ) {}
~RDBufStream() override = default;

public: // IStream
std::ostream& stream() const override { return m_os; }
};
} // unnamed namespace
} // namespace Detail
} // namespace Detail

std::ostream& operator<<( std::ostream& os,
ConfigData::ReporterAndFile const& reporter ) {
Expand Down Expand Up @@ -63,12 +65,14 @@ namespace Catch {
}
m_testSpec = parser.testSpec();

m_reporterStreams.reserve(m_data.reporterSpecifications.size());
for (auto const& reporterAndFile : m_data.reporterSpecifications) {
if (reporterAndFile.outputFileName.none()) {
m_reporterStreams.emplace_back(new Detail::RDBufStream(m_defaultStream->stream().rdbuf()));
m_reporterStreams.reserve( m_data.reporterSpecifications.size() );
for ( auto const& reporterAndFile : m_data.reporterSpecifications ) {
if ( reporterAndFile.outputFileName.none() ) {
m_reporterStreams.emplace_back( new Detail::RDBufStream(
m_defaultStream->stream().rdbuf() ) );
} else {
m_reporterStreams.emplace_back(openStream(*reporterAndFile.outputFileName));
m_reporterStreams.emplace_back(
openStream( *reporterAndFile.outputFileName ) );
}
}
}
Expand Down
4 changes: 3 additions & 1 deletion src/catch2/catch_config.hpp
Expand Up @@ -73,7 +73,9 @@ namespace Catch {
#ifndef CATCH_CONFIG_DEFAULT_REPORTER
#define CATCH_CONFIG_DEFAULT_REPORTER "console"
#endif
std::vector<ReporterAndFile> reporterSpecifications = { {CATCH_CONFIG_DEFAULT_REPORTER, {}} };
std::vector<ReporterAndFile> reporterSpecifications = {
{CATCH_CONFIG_DEFAULT_REPORTER, {}}
};
// Internal: used as parser state
bool _nonDefaultReporterSpecifications = false;
#undef CATCH_CONFIG_DEFAULT_REPORTER
Expand Down
26 changes: 8 additions & 18 deletions tools/scripts/approvalTests.py
Expand Up @@ -182,8 +182,12 @@ def run_test(baseName, args):
f.close()


def check_outputs(baseName, baselinesPath, rawResultsPath, filteredResultsPath):
def check_outputs(baseName):
global overallResult
rawResultsPath = get_rawResultsPath(baseName)
baselinesPath = get_baselinesPath(baseName)
filteredResultsPath = get_filteredResultsPath(baseName)

rawFile = io.open(rawResultsPath, 'r', encoding='utf-8', errors='surrogateescape')
filteredFile = io.open(filteredResultsPath, 'w', encoding='utf-8', errors='surrogateescape')
for line in rawFile:
Expand Down Expand Up @@ -212,12 +216,8 @@ def check_outputs(baseName, baselinesPath, rawResultsPath, filteredResultsPath):


def approve(baseName, args):
baselinesPath = get_baselinesPath(baseName)
rawResultsPath = get_rawResultsPath(baseName)
filteredResultsPath = get_filteredResultsPath(baseName)

run_test(baseName, args)
check_outputs(baseName, baselinesPath, rawResultsPath, filteredResultsPath)
check_outputs(baseName)


print("Running approvals against executable:")
Expand Down Expand Up @@ -247,19 +247,9 @@ def approve(baseName, args):
reporter_args += ['-r', '{}:{}'.format(reporter, get_rawResultsPath(filename))]

run_test("default.sw.multi", common_args + reporter_args)
check_outputs(
"default.sw.multi",
get_baselinesPath("default.sw.multi"),
get_rawResultsPath("default.sw.multi"),
get_filteredResultsPath("default.sw.multi")
)
check_outputs("default.sw.multi")
for reporter, filename in zip(reporters, filenames):
check_outputs(
filename,
get_baselinesPath(filename),
get_rawResultsPath(filename),
get_filteredResultsPath(filename)
)
check_outputs(filename)


if overallResult != 0:
Expand Down

0 comments on commit b1fcb70

Please sign in to comment.