Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

windows: fix some crashes, change handling of invalid paths; lex: add better error detection; accurate: fix out of bounds writes #1793

Open
wants to merge 32 commits into
base: master
Choose a base branch
from

Conversation

sebsura
Copy link
Contributor

@sebsura sebsura commented Apr 29, 2024

Thank you for contributing to the Bareos Project!

This PR contains the following changes:

  • accurate: fix out of bounds writes in rare circumstances
  • compression: remove use of thread_local, which may lead to a crash (delete after free) on windows
  • message: fix trying to call /usr/bin/sendmail on windows
  • windows: replace deprecated SHGetFolderPath by SHGetKnownFolderPath
  • windows: fix support for invalid paths

Please check

  • Short description and the purpose of this PR is present above this paragraph
  • Your name is present in the AUTHORS file (optional)

If you have any questions or problems, please give a comment in the PR.

Helpful documentation and best practices

Checklist for the reviewer of the PR (will be processed by the Bareos team)

Make sure you check/merge the PR using devtools/pr-tool to have some simple automated checks run and a proper changelog record added.

General
  • Is the PR title usable as CHANGELOG entry?
  • Purpose of the PR is understood
  • Commit descriptions are understandable and well formatted
  • Required backport PRs have been created
Source code quality
  • Source code changes are understandable
  • Variable and function names are meaningful
  • Code comments are correct (logically and spelling)
  • Required documentation changes are present and part of the PR
Tests
  • Decision taken that a test is required (if not, then remove this paragraph)
  • The choice of the type of test (unit test or systemtest) is reasonable
  • Testname matches exactly what is being tested
  • On a fail, output of the test leads quickly to the origin of the fault

Our win32-cross toolchain does not handle thread_local correctly.
This can load to crashes because it may free the storage before
destructing the object inside the storage.

See here for reference: https://sourceforge.net/p/mingw-w64/bugs/727/

This is extremely easy to reproduce if run in a debugger (gdb in my
case).  In the wild this does not happen 100%.

As such thread_local was replaced by a manually managed pthread_key.
In the trace of one customer we saw the fd try to add a single file
multiple times!  This is a work around for that.
@sebsura sebsura added this to the 24.0.0 milestone Apr 29, 2024
@sebsura sebsura self-assigned this Apr 29, 2024
@sebsura sebsura force-pushed the dev/ssura/master/fix-windows-crash branch from 15dd02e to 9708140 Compare April 29, 2024 06:28
@sebsura sebsura force-pushed the dev/ssura/master/fix-windows-crash branch from 9708140 to 5b40ea3 Compare April 29, 2024 08:13
@sebsura sebsura linked an issue Apr 29, 2024 that may be closed by this pull request
@sebsura sebsura force-pushed the dev/ssura/master/fix-windows-crash branch from 219655b to 9c03a19 Compare April 29, 2024 12:48
@arogge arogge requested a review from pstorz April 30, 2024 09:56
@bruno-at-bareos
Copy link
Contributor

This has fixed issues encountered in support ticket 75261.

@sebsura sebsura force-pushed the dev/ssura/master/fix-windows-crash branch 2 times, most recently from 5d68c76 to 0799099 Compare May 14, 2024 13:39
Copy link
Member

@pstorz pstorz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As spoken

Comment on lines +79 to +83
jobid=$(cat "$log_home/backup-jobid.out" | grep "jobid" | tail -n1 | cut -d':' -f2 | cut -d'"' -f2)

# make the director miscount the number of files
cat <<END_OF_DATA >"$tmp/bconcmds"
@$out $log_home/query.out
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add test for lmdb as well + add doc string for lmdb options.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Adds a systemtest that checks some parser behaviours.  This needs to
be a systemtest as some parser errors lead to aborts/exits and cannot
be run in unit tests as such.
sebsura added 11 commits May 24, 2024 12:43
Also print how many files are expected to be sent
Since we are supposed to send pktsiz many bytes, we should also print
that number in case we did not send everything, otherwise we may run
into situations where message_length == sent bytes != pktsiz (which
happened at a customer) and the message does not make sense anymore.
@sebsura sebsura force-pushed the dev/ssura/master/fix-windows-crash branch from 0799099 to 9859d90 Compare May 24, 2024 10:43
@sebsura sebsura requested a review from pstorz May 24, 2024 11:47
Copy link
Member

@pstorz pstorz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please see comments,

The accurate-lmdb tests still fail as already talked about.

@@ -0,0 +1,3 @@
.. note::
These worker threads are mostly used to compute checksums, and to compress & encrypt data.
If this is set to at least 1, then bareos will use a seperate thread for sending data.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
If this is set to at least 1, then bareos will use a seperate thread for sending data.
If this is set to at least 1, bareos will use a separate thread for sending data.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@sebsura sebsura requested a review from pstorz June 5, 2024 05:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

write log about accurate details when data is complete
3 participants