Two test failures with 4.5.4 release

[rmcgibbo]

• I have read the Filing Issues and subsequent “How to Get Help” sections of the documentation.
• I have searched the issues (including closed ones) and believe that this is not a duplicate.

• OS version and name: NixOS (Linux) 20.09
• Python version: 3.8.7
• Pelican version: 4.5.4

Issue

Hello Pelican team,

I'm seeing some two errors when I run the tests with the 4.5.4 release. They are

FAILED pelican/tests/test_pelican.py::TestPelican::test_basic_generation_works
FAILED pelican/tests/test_pelican.py::TestPelican::test_custom_generation_works

The failure log is here: https://gist.github.com/rmcgibbo/23a665a147f3106f4fb38d36aaf83c2d. Likely, this is some kind of user error but any help would be appreciated if it's not too much of a bother.

[justinmayer]

Hey Robert. I can't reproduce those failures on my local workstation, and tests are passing in the CI environment. Do you perhaps have another environment in which you can try running the test suite to see if you get the same result?

[rmcgibbo]

Understood. It’s likely user error or some corner case specific to my operating system, NixOS, which is a rather atypical Linux. I will investigate further but any ideas about what aspects of the environment might be able to trigger something like this would be appreciated.

[mcepl]

I have exactly this issue with even more tests failing:

[   16s] =================================== FAILURES ===================================
[   16s] ___________________ TestPelican.test_basic_generation_works ____________________
[   16s] [gw1] linux -- Python 3.6.13 /usr/bin/python3.6
[   16s]
[   16s] self = <pelican.tests.test_pelican.TestPelican testMethod=test_basic_generation_works>
[   16s]
[   16s]     def test_basic_generation_works(self):
[   16s]         # when running pelican without settings, it should pick up the default
[   16s]         # ones and generate correct output without raising any exception
[   16s]         settings = read_settings(path=None, override={
[   16s]             'PATH': INPUT_PATH,
[   16s]             'OUTPUT_PATH': self.temp_path,
[   16s]             'CACHE_PATH': self.temp_cache,
[   16s]             'LOCALE': locale.normalize('en_US'),
[   16s]         })
[   16s]         pelican = Pelican(settings=settings)
[   16s]         mute(True)(pelican.run)()
[   16s]         self.assertDirsEqual(
[   16s] >           self.temp_path, os.path.join(OUTPUT_PATH, 'basic'))
[   16s]
[   16s] pelican/tests/test_pelican.py:108:
[   16s] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
[   16s]
[   16s] self = <pelican.tests.test_pelican.TestPelican testMethod=test_basic_generation_works>
[   16s] left_path = '/tmp/pelicantests.y7s3y42v'
[   16s] right_path = '/home/abuild/rpmbuild/BUILD/pelican-4.6.0+git.1626161722.332be6e5/pelican/tests/output/basic'
[   16s]
[   16s]     def assertDirsEqual(self, left_path, right_path):
[   16s]         out, err = subprocess.Popen(
[   16s]             ['git', 'diff', '--no-ext-diff', '--exit-code',
[   16s]              '-w', left_path, right_path],
[   16s]             env={'PAGER': ''},
[   16s]             stdout=subprocess.PIPE,
[   16s]             stderr=subprocess.PIPE
[   16s]         ).communicate()
[   16s]
[   16s]         def ignorable_git_crlf_errors(line):
[   16s]             # Work around for running tests on Windows
[   16s]             for msg in [
[   16s]                     "LF will be replaced by CRLF",
[   16s]                     "CRLF will be replaced by LF",
[   16s]                     "The file will have its original line endings"]:
[   16s]                 if msg in line:
[   16s]                     return True
[   16s]             return False
[   16s]         if err:
[   16s]             err = '\n'.join([line for line in err.decode('utf8').splitlines()
[   16s]                              if not ignorable_git_crlf_errors(line)])
[   16s] >       assert not out, out
[   16s] E       AssertionError: b'diff --git a/tmp/pelicantests.y7s3y42v/category/misc.html b/home/abuild/rpmbuild/BUILD/pelican-4.6.0+git.1626161722....r></span></pre></div>
[   16s] E          <p>Lovely.</p>
[   16s] E          </div>
[   16s] E          <div class="section" id="testing-even-more-sourcecode-directives">
[   16s] E         '
[   16s] E       assert not b'diff --git a/tmp/pelicantests.y7s3y42v/category/misc.html b/home/abuild/rpmbuild/BUILD/pelican-4.6.0+git.1626161722....r></span></pre></div>\n <p>Lovely.</p>\n </div>\n <div class="section" id="testing-even-more-sourcecode-directives">\n'
[   16s]
[   16s] pelican/tests/test_pelican.py:78: AssertionError
[   16s] ------------------------------ Captured log call -------------------------------
[   16s] WARNING  pelican.settings:log.py:157 Feeds generated without SITEURL set properly may not be valid
[   16s] WARNING  pelican.settings:log.py:157 No timezone information specified in the settings. Assuming your timezone is UTC for feed generation. Check https://docs.getpelican.com/en/latest/settings.html#TIMEZONE for more information
[   16s] ERROR    pelican.contents:log.py:162 Skipping /home/abuild/rpmbuild/BUILD/pelican-4.6.0+git.1626161722.332be6e5/samples/content/pages/jinja2_template.html: could not find information about 'title'
[   16s] WARNING  pelican.contents:log.py:157 Unable to find '/images/Fat_Cat.jpg', skipping url replacement.
[   16s] ___________________ TestPelican.test_custom_generation_works ___________________
[   16s] [gw1] linux -- Python 3.6.13 /usr/bin/python3.6
[   16s]
[   16s] self = <pelican.tests.test_pelican.TestPelican testMethod=test_custom_generation_works>
[   16s]
[   16s]     def test_custom_generation_works(self):
[   16s]         # the same thing with a specified set of settings should work
[   16s]         settings = read_settings(path=SAMPLE_CONFIG, override={
[   16s]             'PATH': INPUT_PATH,
[   16s]             'OUTPUT_PATH': self.temp_path,
[   16s]             'CACHE_PATH': self.temp_cache,
[   16s]             'LOCALE': locale.normalize('en_US.UTF-8'),
[   16s]         })
[   16s]         pelican = Pelican(settings=settings)
[   16s]         mute(True)(pelican.run)()
[   16s]         self.assertDirsEqual(
[   16s] >           self.temp_path, os.path.join(OUTPUT_PATH, 'custom'))
[   16s]
[   16s] pelican/tests/test_pelican.py:125:
[   16s] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
[   16s]
[   16s] self = <pelican.tests.test_pelican.TestPelican testMethod=test_custom_generation_works>
[   16s] left_path = '/tmp/pelicantests.8g29_oz3'
[   16s] right_path = '/home/abuild/rpmbuild/BUILD/pelican-4.6.0+git.1626161722.332be6e5/pelican/tests/output/custom'
[   16s]
[   16s]     def assertDirsEqual(self, left_path, right_path):
[   16s]         out, err = subprocess.Popen(
[   16s]             ['git', 'diff', '--no-ext-diff', '--exit-code',
[   16s]              '-w', left_path, right_path],
[   16s]             env={'PAGER': ''},
[   16s]             stdout=subprocess.PIPE,
[   16s]             stderr=subprocess.PIPE
[   16s]         ).communicate()
[   16s]
[   16s]         def ignorable_git_crlf_errors(line):
[   16s]             # Work around for running tests on Windows
[   16s]             for msg in [
[   16s]                     "LF will be replaced by CRLF",
[   16s]                     "CRLF will be replaced by LF",
[   16s]                     "The file will have its original line endings"]:
[   16s]                 if msg in line:
[   16s]                     return True
[   16s]             return False
[   16s]         if err:
[   16s]             err = '\n'.join([line for line in err.decode('utf8').splitlines()
[   16s]                              if not ignorable_git_crlf_errors(line)])
[   16s] >       assert not out, out
[   16s] E       AssertionError: b'diff --git a/tmp/pelicantests.8g29_oz3/author/alexis-metaireau3.html b/home/abuild/rpmbuild/BUILD/pelican-4.6.0+git....r></span></pre></div>
[   16s] E          <p>Lovely.</p>
[   16s] E          </div>
[   16s] E          <div class="section" id="testing-even-more-sourcecode-directives">
[   16s] E         '
[   16s] E       assert not b'diff --git a/tmp/pelicantests.8g29_oz3/author/alexis-metaireau3.html b/home/abuild/rpmbuild/BUILD/pelican-4.6.0+git....r></span></pre></div>\n <p>Lovely.</p>\n </div>\n <div class="section" id="testing-even-more-sourcecode-directives">\n'
[   16s]
[   16s] pelican/tests/test_pelican.py:78: AssertionError
[   16s] _______________ TestPelican.test_custom_locale_generation_works ________________
[   16s] [gw1] linux -- Python 3.6.13 /usr/bin/python3.6
[   16s]
[   16s] self = <pelican.tests.test_pelican.TestPelican testMethod=test_custom_locale_generation_works>
[   16s]
[   16s]     @unittest.skipUnless(locale_available('fr_FR.UTF-8') or
[   16s]                          locale_available('French'), 'French locale needed')
[   16s]     def test_custom_locale_generation_works(self):
[   16s]         '''Test that generation with fr_FR.UTF-8 locale works'''
[   16s]         if sys.platform == 'win32':
[   16s]             our_locale = 'French'
[   16s]         else:
[   16s]             our_locale = 'fr_FR.UTF-8'
[   16s]
[   16s]         settings = read_settings(path=SAMPLE_FR_CONFIG, override={
[   16s]             'PATH': INPUT_PATH,
[   16s]             'OUTPUT_PATH': self.temp_path,
[   16s]             'CACHE_PATH': self.temp_cache,
[   16s]             'LOCALE': our_locale,
[   16s]         })
[   16s]         pelican = Pelican(settings=settings)
[   16s]         mute(True)(pelican.run)()
[   16s]         self.assertDirsEqual(
[   16s] >           self.temp_path, os.path.join(OUTPUT_PATH, 'custom_locale'))
[   16s]
[   16s] pelican/tests/test_pelican.py:145:
[   16s] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
[   16s]
[   16s] self = <pelican.tests.test_pelican.TestPelican testMethod=test_custom_locale_generation_works>
[   16s] left_path = '/tmp/pelicantests.wqln4ru1'
[   16s] right_path = '/home/abuild/rpmbuild/BUILD/pelican-4.6.0+git.1626161722.332be6e5/pelican/tests/output/custom_locale'
[   16s]
[   16s]     def assertDirsEqual(self, left_path, right_path):
[   16s]         out, err = subprocess.Popen(
[   16s]             ['git', 'diff', '--no-ext-diff', '--exit-code',
[   16s]              '-w', left_path, right_path],
[   16s]             env={'PAGER': ''},
[   16s]             stdout=subprocess.PIPE,
[   16s]             stderr=subprocess.PIPE
[   16s]         ).communicate()
[   16s]
[   16s]         def ignorable_git_crlf_errors(line):
[   16s]             # Work around for running tests on Windows
[   16s]             for msg in [
[   16s]                     "LF will be replaced by CRLF",
[   16s]                     "CRLF will be replaced by LF",
[   16s]                     "The file will have its original line endings"]:
[   16s]                 if msg in line:
[   16s]                     return True
[   16s]             return False
[   16s]         if err:
[   16s]             err = '\n'.join([line for line in err.decode('utf8').splitlines()
[   16s]                              if not ignorable_git_crlf_errors(line)])
[   16s] >       assert not out, out
[   16s] E       AssertionError: b'diff --git a/tmp/pelicantests.wqln4ru1/author/alexis-metaireau3.html b/home/abuild/rpmbuild/BUILD/pelican-4.6.0+git....r></span></pre></div>
[   16s] E          <p>Lovely.</p>
[   16s] E          </div>
[   16s] E          <div class="section" id="testing-even-more-sourcecode-directives">
[   16s] E         '
[   16s] E       assert not b'diff --git a/tmp/pelicantests.wqln4ru1/author/alexis-metaireau3.html b/home/abuild/rpmbuild/BUILD/pelican-4.6.0+git....r></span></pre></div>\n <p>Lovely.</p>\n </div>\n <div class="section" id="testing-even-more-sourcecode-directives">\n'
[   16s]
[   16s] pelican/tests/test_pelican.py:78: AssertionError
[   16s] ------------------------------ Captured log call -------------------------------
[   16s] ERROR    pelican.contents:log.py:162 Skipping /home/abuild/rpmbuild/BUILD/pelican-4.6.0+git.1626161722.332be6e5/samples/content/pages/jinja2_template.html: could not find information about 'title'
[   16s] =============================== warnings summary ===============================
[   16s] pelican/tests/test_readers.py::RstReaderTest::test_typogrify
[   16s] pelican/tests/test_readers.py::MdReaderTest::test_typogrify_dashes_config
[   16s]   /usr/lib/python3.6/site-packages/smartypants.py:276: DeprecationWarning: invalid escape sequence \S
[   16s]     if re.match("\S", prev_token_last_char):
[   16s]
[   16s] pelican/tests/test_readers.py::RstReaderTest::test_typogrify
[   16s] pelican/tests/test_readers.py::MdReaderTest::test_typogrify_dashes_config
[   16s]   /usr/lib/python3.6/site-packages/smartypants.py:270: DeprecationWarning: invalid escape sequence \S
[   16s]     if re.match("\S", prev_token_last_char):
[   16s]
[   16s] -- Docs: https://docs.pytest.org/en/stable/warnings.html
[   16s] =========================== short test summary info ============================
[   16s] FAILED pelican/tests/test_pelican.py::TestPelican::test_basic_generation_works
[   16s] FAILED pelican/tests/test_pelican.py::TestPelican::test_custom_generation_works
[   16s] FAILED pelican/tests/test_pelican.py::TestPelican::test_custom_locale_generation_works
[   16s] ================== 3 failed, 254 passed, 4 warnings in 4.87s ===================

This happens in the closed build system for openSUSE packages.

Complete build log with all details of packages used and steps done.

[justinmayer]

I'm seeing test failures related to the presence of .DS_Store files, so while we are at it, I'd like us to fix that, too.

[avaris]

@mcepl Seems like your build uses Pygments==2.7.4, but our test output is generated with Pygments==2.8.1. I think that's the difference.

[mcepl]

And apparently 2.9.0 is too new:

[   15s] =================================== FAILURES ===================================
[   15s] ___________________ TestPelican.test_basic_generation_works ____________________
[   15s] [gw0] linux -- Python 3.6.13 /usr/bin/python3.6
[   15s]
[   15s] self = <pelican.tests.test_pelican.TestPelican testMethod=test_basic_generation_works>
[   15s]
[   15s]     def test_basic_generation_works(self):
[   15s]         # when running pelican without settings, it should pick up the default
[   15s]         # ones and generate correct output without raising any exception
[   15s]         settings = read_settings(path=None, override={
[   15s]             'PATH': INPUT_PATH,
[   15s]             'OUTPUT_PATH': self.temp_path,
[   15s]             'CACHE_PATH': self.temp_cache,
[   15s]             'LOCALE': locale.normalize('en_US'),
[   15s]         })
[   15s]         pelican = Pelican(settings=settings)
[   15s]         mute(True)(pelican.run)()
[   15s]         self.assertDirsEqual(
[   15s] >           self.temp_path, os.path.join(OUTPUT_PATH, 'basic'))
[   15s]
[   15s] pelican/tests/test_pelican.py:108:
[   15s] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
[   15s]
[   15s] self = <pelican.tests.test_pelican.TestPelican testMethod=test_basic_generation_works>
[   15s] left_path = '/tmp/pelicantests.6oevkbc3'
[   15s] right_path = '/home/abuild/rpmbuild/BUILD/pelican-4.6.0+git.1626161722.332be6e5/pelican/tests/output/basic'
[   15s]
[   15s]     def assertDirsEqual(self, left_path, right_path):
[   15s]         out, err = subprocess.Popen(
[   15s]             ['git', 'diff', '--no-ext-diff', '--exit-code',
[   15s]              '-w', left_path, right_path],
[   15s]             env={'PAGER': ''},
[   15s]             stdout=subprocess.PIPE,
[   15s]             stderr=subprocess.PIPE
[   15s]         ).communicate()
[   15s]
[   15s]         def ignorable_git_crlf_errors(line):
[   15s]             # Work around for running tests on Windows
[   15s]             for msg in [
[   15s]                     "LF will be replaced by CRLF",
[   15s]                     "CRLF will be replaced by LF",
[   15s]                     "The file will have its original line endings"]:
[   15s]                 if msg in line:
[   15s]                     return True
[   15s]             return False
[   15s]         if err:
[   15s]             err = '\n'.join([line for line in err.decode('utf8').splitlines()
[   15s]                              if not ignorable_git_crlf_errors(line)])
[   15s] >       assert not out, out
[   15s] E       AssertionError: b'diff --git a/tmp/pelicantests.6oevkbc3/feeds/all-en.atom.xml b/home/abuild/rpmbuild/BUILD/pelican-4.6.0+git.16261617...r></span></pre></div>
[   15s] E          <p>Lovely.</p>
[   15s] E          </div>
[   15s] E          <div class="section" id="testing-even-more-sourcecode-directives">
[   15s] E         '
[   15s] E       assert not b'diff --git a/tmp/pelicantests.6oevkbc3/feeds/all-en.atom.xml b/home/abuild/rpmbuild/BUILD/pelican-4.6.0+git.16261617...r></span></pre></div>\n <p>Lovely.</p>\n </div>\n <div class="section" id="testing-even-more-sourcecode-directives">\n'
[   15s]
[   15s] pelican/tests/test_pelican.py:78: AssertionError
[   15s] ------------------------------ Captured log call -------------------------------
[   15s] WARNING  pelican.settings:log.py:157 Feeds generated without SITEURL set properly may not be valid
[   15s] WARNING  pelican.settings:log.py:157 No timezone information specified in the settings. Assuming your timezone is UTC for feed generation. Check https://docs.getpelican.com/en/latest/settings.html#TIMEZONE for more information
[   15s] ERROR    pelican.contents:log.py:162 Skipping /home/abuild/rpmbuild/BUILD/pelican-4.6.0+git.1626161722.332be6e5/samples/content/pages/jinja2_template.html: could not find information about 'title'
[   15s] WARNING  pelican.contents:log.py:157 Unable to find '/images/Fat_Cat.jpg', skipping url replacement.
[   15s] ___________________ TestPelican.test_custom_generation_works ___________________
[   15s] [gw0] linux -- Python 3.6.13 /usr/bin/python3.6
[   15s]
[   15s] self = <pelican.tests.test_pelican.TestPelican testMethod=test_custom_generation_works>
[   15s]
[   15s]     def test_custom_generation_works(self):
[   15s]         # the same thing with a specified set of settings should work
[   15s]         settings = read_settings(path=SAMPLE_CONFIG, override={
[   15s]             'PATH': INPUT_PATH,
[   15s]             'OUTPUT_PATH': self.temp_path,
[   15s]             'CACHE_PATH': self.temp_cache,
[   15s]             'LOCALE': locale.normalize('en_US.UTF-8'),
[   15s]         })
[   15s]         pelican = Pelican(settings=settings)
[   15s]         mute(True)(pelican.run)()
[   15s]         self.assertDirsEqual(
[   15s] >           self.temp_path, os.path.join(OUTPUT_PATH, 'custom'))
[   15s]
[   15s] pelican/tests/test_pelican.py:125:
[   15s] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
[   15s]
[   15s] self = <pelican.tests.test_pelican.TestPelican testMethod=test_custom_generation_works>
[   15s] left_path = '/tmp/pelicantests.71j54ju9'
[   15s] right_path = '/home/abuild/rpmbuild/BUILD/pelican-4.6.0+git.1626161722.332be6e5/pelican/tests/output/custom'
[   15s]
[   15s]     def assertDirsEqual(self, left_path, right_path):
[   15s]         out, err = subprocess.Popen(
[   15s]             ['git', 'diff', '--no-ext-diff', '--exit-code',
[   15s]              '-w', left_path, right_path],
[   15s]             env={'PAGER': ''},
[   15s]             stdout=subprocess.PIPE,
[   15s]             stderr=subprocess.PIPE
[   15s]         ).communicate()
[   15s]
[   15s]         def ignorable_git_crlf_errors(line):
[   15s]             # Work around for running tests on Windows
[   15s]             for msg in [
[   15s]                     "LF will be replaced by CRLF",
[   15s]                     "CRLF will be replaced by LF",
[   15s]                     "The file will have its original line endings"]:
[   15s]                 if msg in line:
[   15s]                     return True
[   15s]             return False
[   15s]         if err:
[   15s]             err = '\n'.join([line for line in err.decode('utf8').splitlines()
[   15s]                              if not ignorable_git_crlf_errors(line)])
[   15s] >       assert not out, out
[   15s] E       AssertionError: b'diff --git a/tmp/pelicantests.71j54ju9/feeds/alexis-metaireau.atom.xml b/home/abuild/rpmbuild/BUILD/pelican-4.6.0+gi...r></span></pre></div>
[   15s] E          <p>Lovely.</p>
[   15s] E          </div>
[   15s] E          <div class="section" id="testing-even-more-sourcecode-directives">
[   15s] E         '
[   15s] E       assert not b'diff --git a/tmp/pelicantests.71j54ju9/feeds/alexis-metaireau.atom.xml b/home/abuild/rpmbuild/BUILD/pelican-4.6.0+gi...r></span></pre></div>\n <p>Lovely.</p>\n </div>\n <div class="section" id="testing-even-more-sourcecode-directives">\n'
[   15s]
[   15s] pelican/tests/test_pelican.py:78: AssertionError
[   15s] _______________ TestPelican.test_custom_locale_generation_works ________________
[   15s] [gw0] linux -- Python 3.6.13 /usr/bin/python3.6
[   15s]
[   15s] self = <pelican.tests.test_pelican.TestPelican testMethod=test_custom_locale_generation_works>
[   15s]
[   15s]     @unittest.skipUnless(locale_available('fr_FR.UTF-8') or
[   15s]                          locale_available('French'), 'French locale needed')
[   15s]     def test_custom_locale_generation_works(self):
[   15s]         '''Test that generation with fr_FR.UTF-8 locale works'''
[   15s]         if sys.platform == 'win32':
[   15s]             our_locale = 'French'
[   15s]         else:
[   15s]             our_locale = 'fr_FR.UTF-8'
[   15s]
[   15s]         settings = read_settings(path=SAMPLE_FR_CONFIG, override={
[   15s]             'PATH': INPUT_PATH,
[   15s]             'OUTPUT_PATH': self.temp_path,
[   15s]             'CACHE_PATH': self.temp_cache,
[   15s]             'LOCALE': our_locale,
[   15s]         })
[   15s]         pelican = Pelican(settings=settings)
[   15s]         mute(True)(pelican.run)()
[   15s]         self.assertDirsEqual(
[   15s] >           self.temp_path, os.path.join(OUTPUT_PATH, 'custom_locale'))
[   15s]
[   15s] pelican/tests/test_pelican.py:145:
[   15s] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
[   15s]
[   15s] self = <pelican.tests.test_pelican.TestPelican testMethod=test_custom_locale_generation_works>
[   15s] left_path = '/tmp/pelicantests.p_6ic327'
[   15s] right_path = '/home/abuild/rpmbuild/BUILD/pelican-4.6.0+git.1626161722.332be6e5/pelican/tests/output/custom_locale'
[   15s]
[   15s]     def assertDirsEqual(self, left_path, right_path):
[   15s]         out, err = subprocess.Popen(
[   15s]             ['git', 'diff', '--no-ext-diff', '--exit-code',
[   15s]              '-w', left_path, right_path],
[   15s]             env={'PAGER': ''},
[   15s]             stdout=subprocess.PIPE,
[   15s]             stderr=subprocess.PIPE
[   15s]         ).communicate()
[   15s]
[   15s]         def ignorable_git_crlf_errors(line):
[   15s]             # Work around for running tests on Windows
[   15s]             for msg in [
[   15s]                     "LF will be replaced by CRLF",
[   15s]                     "CRLF will be replaced by LF",
[   15s]                     "The file will have its original line endings"]:
[   15s]                 if msg in line:
[   15s]                     return True
[   15s]             return False
[   15s]         if err:
[   15s]             err = '\n'.join([line for line in err.decode('utf8').splitlines()
[   15s]                              if not ignorable_git_crlf_errors(line)])
[   15s] >       assert not out, out
[   15s] E       AssertionError: b'diff --git a/tmp/pelicantests.p_6ic327/feeds/alexis-metaireau.atom.xml b/home/abuild/rpmbuild/BUILD/pelican-4.6.0+gi...r></span></pre></div>
[   15s] E          <p>Lovely.</p>
[   15s] E          </div>
[   15s] E          <div class="section" id="testing-even-more-sourcecode-directives">
[   15s] E         '
[   15s] E       assert not b'diff --git a/tmp/pelicantests.p_6ic327/feeds/alexis-metaireau.atom.xml b/home/abuild/rpmbuild/BUILD/pelican-4.6.0+gi...r></span></pre></div>\n <p>Lovely.</p>\n </div>\n <div class="section" id="testing-even-more-sourcecode-directives">\n'
[   15s]
[   15s] pelican/tests/test_pelican.py:78: AssertionError
[   15s] ------------------------------ Captured log call -------------------------------
[   15s] ERROR    pelican.contents:log.py:162 Skipping /home/abuild/rpmbuild/BUILD/pelican-4.6.0+git.1626161722.332be6e5/samples/content/pages/jinja2_template.html: could not find information about 'title'
[   15s] =============================== warnings summary ===============================
[   15s] pelican/tests/test_readers.py::RstReaderTest::test_typogrify
[   15s]   /usr/lib/python3.6/site-packages/smartypants.py:276: DeprecationWarning: invalid escape sequence \S
[   15s]     if re.match("\S", prev_token_last_char):
[   15s]
[   15s] pelican/tests/test_readers.py::RstReaderTest::test_typogrify
[   15s]   /usr/lib/python3.6/site-packages/smartypants.py:270: DeprecationWarning: invalid escape sequence \S
[   15s]     if re.match("\S", prev_token_last_char):
[   15s]
[   15s] -- Docs: https://docs.pytest.org/en/stable/warnings.html
[   15s] =========================== short test summary info ============================
[   15s] FAILED pelican/tests/test_pelican.py::TestPelican::test_basic_generation_works
[   15s] FAILED pelican/tests/test_pelican.py::TestPelican::test_custom_generation_works
[   15s] FAILED pelican/tests/test_pelican.py::TestPelican::test_custom_locale_generation_works
[   15s] ================== 3 failed, 254 passed, 2 warnings in 5.34s ===================

I would say that these three tests are darn too sensitive.

[avaris]

Tests are fine but Pygments keeps changing output every single release and I'm starting to get tired of it. As far as I can see, there is a single file that triggers Pygments and that's the cause for errors after every change. I would be normally against this but I'm more and more leaning towards removing that file and at least keep Pygments out of those overall generation tests.

We probably should still keep Pygments tests but as a separate unit test that's locked down to specific versions and skipped otherwise. Or at least a much more lenient test that doesn't rely on the HTML output being identical.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your participation and understanding.