Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

weak_ptr: Make it possible to produce constant pointers #2226

Merged
merged 1 commit into from May 16, 2024

Conversation

xemul
Copy link
Contributor

@xemul xemul commented May 7, 2024

The patch introduces weak_ptr(weak_ptr&&) constructor that creates
weak_ptr out of convertible to T* Y's. This has two implications.

First, the main one, is that it's now possible to obtain a constant weak
pointer on an object.

Another, a nice side effect, is: given a base-class we can now create
weak_ptr out of weak_ptr.

@xemul xemul requested a review from tchaikov May 7, 2024 09:57
@xemul
Copy link
Contributor Author

xemul commented May 7, 2024

The ultimate intent is to apply

--- a/cql3/statements/prepared_statement.hh
+++ b/cql3/statements/prepared_statement.hh
@@ -33,7 +33,7 @@ struct invalidated_prepared_usage_attempt {
 
 class prepared_statement : public seastar::weakly_referencable<prepared_statement> {
 public:
-    typedef seastar::checked_ptr<seastar::weak_ptr<prepared_statement>> checked_weak_ptr;
+    typedef seastar::checked_ptr<seastar::weak_ptr<const prepared_statement>> checked_weak_ptr;
 
 public:
     const seastar::shared_ptr<cql_statement> statement;
@@ -51,7 +51,7 @@ class prepared_statement : public seastar::weakly_referencable<prepared_statemen
 
     prepared_statement(seastar::shared_ptr<cql_statement>&& statement_);
 
-    checked_weak_ptr checked_weak_from_this() {
+    checked_weak_ptr checked_weak_from_this() const {
         return checked_weak_ptr(this->weak_from_this());
     }
 };

to Scylla, since prepare_statements are in fact immutable throughout their lifetime

@xemul
Copy link
Contributor Author

xemul commented May 7, 2024

:( ./configure.py --mode=release --compiler=g++ --c++-standard=23 doesn't like it

FAILED: tests/unit/CMakeFiles/test_unit_weak_ptr.dir/weak_ptr_test.cc.o 
/usr/bin/g++ -DBOOST_ALL_DYN_LINK -DBOOST_NO_CXX98_FUNCTION_BASE -DFMT_SHARED -DSEASTAR_API_LEVEL=7 -DSEASTAR_DEFERRED_ACTION_REQUIRE_NOEXCEPT -DSEASTAR_HAS_MEMBARRIER -DSEASTAR_HAVE_ASAN_FIBER_SUPPORT -DSEASTAR_HAVE_HWLOC -DSEASTAR_HAVE_NUMA -DSEASTAR_HAVE_SYSTEMTAP_SDT -DSEASTAR_HAVE_URING -DSEASTAR_LOGGER_COMPILE_TIME_FMT -DSEASTAR_LOGGER_TYPE_STDOUT -DSEASTAR_SCHEDULING_GROUPS_COUNT=16 -DSEASTAR_SSTRING -DSEASTAR_TESTING_MAIN -DSEASTAR_TESTING_WITH_NETWORKING=1 -I/home/src/tests/unit -I/home/src/src -I/home/src/include -I/home/src/build/release/gen/include -I/home/src/build/release/gen/src -O2 -g -DNDEBUG -std=gnu++23 -UNDEBUG -Wall -Werror -Wimplicit-fallthrough -Wdeprecated -Wno-error=deprecated -Wno-error=stringop-overflow -Wno-error=array-bounds -Wdeprecated-declarations -Wno-error=deprecated-declarations -fvisibility=hidden -gz -U_FORTIFY_SOURCE -Wno-maybe-uninitialized -Wno-error=unused-result -MD -MT tests/unit/CMakeFiles/test_unit_weak_ptr.dir/weak_ptr_test.cc.o -MF tests/unit/CMakeFiles/test_unit_weak_ptr.dir/weak_ptr_test.cc.o.d -o tests/unit/CMakeFiles/test_unit_weak_ptr.dir/weak_ptr_test.cc.o -c /home/src/tests/unit/weak_ptr_test.cc
In file included from /home/src/tests/unit/weak_ptr_test.cc:26:
/home/src/include/seastar/core/weak_ptr.hh: In instantiation of 'seastar::weak_ptr<const T> seastar::weakly_referencable<T>::weak_from_this() const [with T = myclass]':
/home/src/tests/unit/weak_ptr_test.cc:54:91:   required from here
/home/src/include/seastar/core/weak_ptr.hh:145:60: error: dereferencing type-punned pointer will break strict-aliasing rules [-Werror=strict-aliasing]
  145 |         _ptr_list.push_back(reinterpret_cast<weak_ptr<T>&>(ptr));

tests/unit/weak_ptr_test.cc Show resolved Hide resolved

weak_ptr<const T> weak_from_this() const noexcept {
weak_ptr<const T> ptr(static_cast<const T*>(this));
_ptr_list.push_back(reinterpret_cast<weak_ptr<T>&>(ptr));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of this problematic construct (as you saw, the compiler didn't like it either), I wonder if you can make the _ptr_list definition use something like const T or std::remove_const<T> instead of T, which will maybe allow you to push the weak_ptr onto _ptr_list.
(I haven't thought this through so maybe you can come up with a better solution)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought about it too, but couldn't break the boost intrusive list requirement to explicitly specify the type of objects it keeps :( I pushed an updated version that made the compiler happy

@xemul xemul changed the title weak_ptr: Support weak_ptr<const T> weak_ptr: Make it possible to produce constand pointers May 7, 2024
@xemul xemul changed the title weak_ptr: Make it possible to produce constand pointers weak_ptr: Make it possible to produce constant pointers May 7, 2024
Copy link
Contributor

@nyh nyh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, the const_cast trick on the object itself (not all sorts of weak_ptr things) is a nice trick. It's lucky that the hook type does not depend on T so the final swap_nodes() work.

@nyh
Copy link
Contributor

nyh commented May 7, 2024

@xemul until we get #2050 can you please confirm that you ran this new version on a couple of compilers and it worked? Thanks.

@nyh
Copy link
Contributor

nyh commented May 7, 2024

@tchaikov I merged your PR for adding the in-github CI (#2050). Any idea how I can get the new workflow to run on this pre-existing PR?

@xemul
Copy link
Contributor Author

xemul commented May 7, 2024

@xemul until we get #2050 can you please confirm that you ran this new version on a couple of compilers and it worked? Thanks.

I checked compilation with all compiler-version-standard triplets that can be found in .circleci/config.yml

@tchaikov
Copy link
Contributor

tchaikov commented May 8, 2024

@tchaikov I merged your PR for adding the in-github CI (#2050). Any idea how I can get the new workflow to run on this pre-existing PR?

the workflow is configured to be triggered with push and pull_request, but i didn't add workflow_dispatch to it. to use it to test a pre-existing PR, we also need to configure it to accept an input parameter so we can specify the pull request number. but we haven't define such a parameter. so, no, you cannot.

weak_ptr<const T> ret(std::exchange(tmp._ptr, nullptr));
tmp._hook.swap_nodes(ret._hook);
return ret;
}
Copy link
Contributor

@tchaikov tchaikov May 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@xemul hi Pavel, i'd propose a different, but probably more standard compliant, and extensive solution to address this problem.

from the standard, weak_ptr<Y> should be constructible from another weak_ptr<T>. as long as Y is "compatible" with T. see https://eel.is/c++draft/util.sharedptr#util.smartptr.shared.general-6 .

so instead of marking _ptr_list mutable. we could just use something like:

namespace internal {

// quote from
// http://eel.is/c++draft/util.sharedptr#util.smartptr.shared.general-6,
//
// a pointer type Y* is said to be compatible with a pointer type T* when
// either Y* is convertible to T* or Y is U[N] and T is cv U[].
template <typename Y, typename T>
struct bounded_array_of : std::false_type {};

template <typename U, typename T, size_t N>
struct bounded_array_of<U[N], T> : std::is_same<std::remove_cv_t<T>, U[]> {};

template <typename Y, typename T>
concept pointer_type_compatible_with = (std::convertible_to<Y*, T*> ||
                                        bounded_array_of<Y, T>::value);
}
///...
template<typename T>
class weak_ptr {
    template<typename U>
    friend class weakly_referencable;
    template<typename U>
    friend class weak_ptr;
private:
///...
    template<internal::pointer_type_compatible_with<T> Y>
    weak_ptr(const weak_ptr<Y>& o) noexcept {
        if (o._ptr) {
            swap(o._ptr->weak_from_this());
        }
    }
///...
template<typename T>
class weakly_referencable {
///...
    weak_ptr<T> weak_from_this() const noexcept {
        return const_cast<weakly_referencable*>(this)->weak_from_this();
    }
};

for couple reasons:

  1. more standard compliant. so easier to learn from a regular C++ programmer's perspective
  2. we get a better weak_ptr, as casting to a compatible pointer type should be possible, and this behavior is more intuitive.
  3. less mutable. personally, i'd avoid it if possible. as it is an exception, which means.. surprise.

please note, instead of using std::is_bounded_array, i am creating yet another type trait here, as std::is_bounded_array does not provide convenient way to access the element type, and the proposed implementation is basically a straightforward translation of the standard. so i prefer (re)inventing it here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mutable is not required, I erroneously forgot to remove it. But I like the idea of introducing weak_ptr<const T>(weak_ptr<T>&&) constructor, I'll try it.

However at this point no more flexibility is requires from my perspective. In particular, your suggestion means that weak_ptr<T>(weak_ptr<const T>&&) will be introduced, since `std::is_convertible<const T, T>::value == true, but it's not correct, casting const pointer to non-const shouldn't happen automatically

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think you misread the line of

std::convertible_to<Y*, T*>

also, my original intention was not "flexibility", but more about standard compliance and correctness.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think you misread the line of

std::convertible_to<Y*, T*>

Yes, thank you, I misread the *'s :( However, I'm still concerned about the bounded_array. The

template <typename Y, typename T>
concept pointer_type_compatible_with = (std::convertible_to<Y*, T*> ||
                                        bounded_array_of<Y, T>::value);

means that we can create weak_ptr<T> from weak_ptr<Y> provided std::converible_to<Y, T>, but in case Y is an array, we cannot get weak_ptr<Y> itself, because array cannot inherit from weakly_referencable

Copy link
Contributor

@tchaikov tchaikov May 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sorry, i was appreciating the design of the standard library, and completely ignored the connection between seastar::weak_ptr and seastar::weakly_referencable . as in our implementation, we don't create a seastar::weak_ptr from an existing instance of seastar::shared_ptr or seastar::lw_shared_ptr, like the standard library does. so we have to to rely on the type of T instead of its smart pointer wrapper.

@xemul
Copy link
Contributor Author

xemul commented May 9, 2024

upd:

  • more standard compliance with constructing weak_ptr out of any weak_ptr with Y* being convertible to T*

Copy link
Contributor

@tchaikov tchaikov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm. we could also add a templated copy ctor. but that's probably not in the scope of this change.

@tchaikov
Copy link
Contributor

tchaikov commented May 9, 2024

@scylladb/seastar-maint hello maintainers, could you help merge this change?

@xemul
Copy link
Contributor Author

xemul commented May 13, 2024

upd:

  • updated comment to better reflect what the change does

The patch introduces weak_ptr<T>(weak_ptr<Y>&&) constructor that creates
weak_ptr<T> out of convertible to T* Y's. This has two implications.

First, the main one, is that it's now possible to obtain a constant weak
pointer on an object.

Another, a nice side effect, is: given a base-class we can now create
weak_ptr<base> out of weak_ptr<inherited>.

Signed-off-by: Pavel Emelyanov <xemul@scylladb.com>
@xemul
Copy link
Contributor Author

xemul commented May 13, 2024

Error: ../../../../../../dpdk/lib/net/net_crc_avx512.c:324:22: error: always_inline function '_mm512_broadcast_i32x4' requires target feature 'evex512', but would be inlined into function 'crc32_load_init_constants' that is compiled without support for 'evex512'
  324 |         crc32_eth.rk1_rk2 = _mm512_broadcast_i32x4(a);

environment issue? (@tchaikov )

@tchaikov
Copy link
Contributor


Error: ../../../../../../dpdk/lib/net/net_crc_avx512.c:324:22: error: always_inline function '_mm512_broadcast_i32x4' requires target feature 'evex512', but would be inlined into function 'crc32_load_init_constants' that is compiled without support for 'evex512'

  324 |         crc32_eth.rk1_rk2 = _mm512_broadcast_i32x4(a);

environment issue? (@tchaikov )

Will take a look.

public:
template <typename U>
requires std::convertible_to<U*, T*>
weak_ptr(weak_ptr<U>&& o)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cosmetic: it's nicer to specify this as weak_ptr(std::convertible_to<T&> auto&& o), as we don't introduce a variable U which is immediately forgotten (if it works).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't manage to make it work this-or-similar way.

In particular, this exact syntax fails like this

/home/xemul/src/seastar/tests/unit/weak_ptr_test.cc:66:24: note:
no known conversion for argument 1 from ‘weak_ptr<myiclass>’ to ‘weak_ptr<baseclass>’

I guess it's because std::convertible applies to pointer type, while o's type is about weak_ptr<> over the type itself, so compiler doesn't match one against each other

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought it would know to convert references. But it's just cosmetic anyway.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK. Is there anything else that prevents this patch from being merged? :)

@avikivity
Copy link
Member

Looks good.

@tchaikov
Copy link
Contributor

@scylladb/seastar-maint hello maintainers, could you help merge this change?

@avikivity avikivity merged commit 0ad9d82 into scylladb:master May 16, 2024
11 of 12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants