Add convenience byte offset/check align functions to pointers #95643

WaffleLapkin · 2022-04-04T11:23:50Z

This PR adds the following APIs:

impl *const T {
    // feature gates `pointer_byte_offsets` and `const_pointer_byte_offsets
    pub const unsafe fn byte_offset(self, count: isize) -> Self;
    pub const fn wrapping_byte_offset(self, count: isize) -> Self;
    pub const unsafe fn byte_offset_from(self, origin: *const T) -> isize;
    pub const unsafe fn byte_add(self, count: usize) -> Self;
    pub const unsafe fn byte_sub(self, count: usize) -> Self;
    pub const fn wrapping_byte_add(self, count: usize) -> Self;
    pub const fn wrapping_byte_sub(self, count: usize) -> Self;

    // feature gate `pointer_is_aligned`
    pub fn is_aligned(self) -> bool where T: Sized;
    pub fn is_aligned_to(self, align: usize) -> bool;
}
// ... and the same for` *mut T`

Note that all functions except is_aligned do not require T: Sized as their pointee-sized-offset counterparts.

cc @oli-obk (you may want to check that I've correctly placed consts)
cc @RalfJung

rust-highfive · 2022-04-04T11:23:53Z

r? @m-ou-se

(rust-highfive has picked a reviewer for you, use r? to override)

Gankra · 2022-04-04T13:33:59Z

I proposed and drafted the initial code for this PR, so I 100% endorse the concept.

For motivation we need look no farther than #95241, where I poked at most of the ptr<->int casts in the codebase. Patterns that showed up that I left in their "manual" state:

debug_asserting alignment
using the distance between addresses as byte-wise len/capacity
adjusting a pointer for alignment
working with *c_void, where yes we implement byte offsets (and this is a common C extension) but morally void* is !Sized (but Thin) so it's nice to be explicit about byte offsets. More generally for actual extern types this is nice to have (which is why it's actually unironically useful that these methods work on ?Sized).

library/core/src/ptr/const_ptr.rs

ChayimFriedman2 · 2022-04-05T22:57:16Z

Are you sure we need the check & panic in is_aligned_to()? It'll probably often get used with constants so it doesn't matter, but I fell uncomfortable to add a branch that isn't really needed (though if not this branch, we'll have a branch for the modulo 0 check, but it may be better optimized out thanks to NonZeroUsize and friends and also it uses less instructions).

Gankra · 2022-04-05T23:04:15Z

I was just mirroring the semantics of the more evil existing alignment-handling function, on the assumption that it was needed for... Reasons.

ChayimFriedman2 · 2022-04-06T00:15:47Z

I was just mirroring the semantics of the more evil existing alignment-handling function, on the assumption that it was needed for... Reasons.

Are you talking about align_offset()? Well it looks like it really needs that (TBH, I don't really understand this complicated piece of code, nor I tried to).

Gankra · 2022-04-06T01:18:07Z

Yeah, I figured it align_offset insisted it was important I should be consistent with it, and we can always relax the panic to "it's all fine"... right? I guess we would have to say "might panic" or "it's UB" or something? So that people aren't allowed to treat it as an assert they can depend on?

ChayimFriedman2 · 2022-04-06T01:49:11Z

I guess we would have to say "might panic" or "it's UB" or something? So that people aren't allowed to treat it as an assert they can depend on?

I don't think "it's UB" is a good idea (especially not when the method is safe and I don't think it should be unsafe just because of that), but "might panic" sounds good.

Gankra · 2022-04-17T01:15:10Z

@WaffleLapkin is this ready to land? are there still things to do?

WaffleLapkin · 2022-04-17T07:22:38Z

@Gankra I believe this is ready to land, yes.

Gankra · 2022-04-21T13:42:15Z

Filed

please add these to the unstable decls. insofar as my opinion carries weight anymore, I would be happy to r+ this

yaahc · 2022-04-21T22:31:31Z

r? rust-lang/libs-api @rustbot label +T-libs-api -T-libs

yaahc · 2022-04-29T22:52:28Z

I'm reassigning this PR because I'm taking a break from the review rotation for a little while. Thank you for your patience.

r? rust-lang/libs-api

The functions added: - {*const T,*mut T}::{wrapping_,}byte_{offset,add,sub} - {*const T,*mut T}::{byte_offset_from,is_aligned,is_aligned_to}

Since they work on byte pointers (by `.cast::<u8>()`ing them), there is no need to know the size of `T` and so there is no need for `T: Sized`. The `is_aligned_to` is similar, though it doesn't need the _alignment_ of `T`.

Apparently LLVM is unable to understand that if count_ones() == 1 then self != 0. Adding `assume(align != 0)` helps generating better asm: https://rust.godbolt.org/z/ja18YKq91

…ter_byte_offsets and pointer_is_aligned

WaffleLapkin · 2022-05-12T08:58:07Z

I wander if we also need byte_sub_ptr

joshtriplett · 2022-05-18T22:41:02Z

Looks reasonable to me!

We can always bikeshed names later, but I personally think these names look good.

@bors r+

bors · 2022-05-18T22:41:04Z

📌 Commit 03d4569 has been approved by joshtriplett

bors · 2022-05-18T23:18:06Z

⌛ Testing commit 03d4569 with merge d8a3fc4...

bors · 2022-05-19T01:41:01Z

☀️ Test successful - checks-actions
Approved by: joshtriplett
Pushing d8a3fc4 to master...

rust-timer · 2022-05-19T03:49:29Z

Finished benchmarking commit (d8a3fc4): comparison url.

Summary: This benchmark run did not return any relevant results.

If you disagree with this performance assessment, please file an issue in rust-lang/rustc-perf.

@rustbot label: -perf-regression

1. change `PtrMut::as_ptr(self)` and `OwnedPtr::as_ptr(self)` to take `&self`, otherwise printing the pointer will prevent doing anything else afterwards 2. make all `as_ptr` methods safe. There's nothing unsafe about obtaining a pointer, these kinds of methods are safe in std as well [str::as_ptr](https://doc.rust-lang.org/stable/std/primitive.str.html#method.as_ptr), [Rc::as_ptr](https://doc.rust-lang.org/stable/std/rc/struct.Rc.html#method.as_ptr) 3. rename `offset`/`add` to `byte_offset`/`byte_add`. The unprefixed methods in std add in increments of `std::mem::size_of::<T>`, not in bytes. There's a PR for rust to add these byte_ methods rust-lang/rust#95643 and at the call site it makes it much more clear that you need to do `.byte_add(i * layout_size)` instead of `.add(i)`

…o, r=workingjubilee Optimize `pointer::as_aligned_to` This PR replaces `addr % align` with `addr & align - 1`, which is correct due to `align` being a power of two. Here is a proof that this makes things better: [[godbolt]](https://godbolt.org/z/Wbq3hx6YG). This PR also removes `assume(align != 0)`, with the new impl it does not improve anything anymore ([[godbolt]](https://rust.godbolt.org/z/zcnrG4777), [[original concern]](rust-lang#95643 (comment))).

Add pointer masking convenience functions This PR adds the following public API: ```rust impl<T: ?Sized> *const T { fn mask(self, mask: usize) -> *const T; } impl<T: ?Sized> *mut T { fn mask(self, mask: usize) -> *const T; } // mod intrinsics fn mask<T>(ptr: *const T, mask: usize) -> *const T ``` This is equivalent to `ptr.map_addr(|a| a & mask)` but also uses a cool llvm intrinsic. Proposed in rust-lang#95643 (comment) cc `@Gankra` `@scottmcm` `@RalfJung` r? rust-lang/libs-api

Add pointer masking convenience functions This PR adds the following public API: ```rust impl<T: ?Sized> *const T { fn mask(self, mask: usize) -> *const T; } impl<T: ?Sized> *mut T { fn mask(self, mask: usize) -> *const T; } // mod intrinsics fn mask<T>(ptr: *const T, mask: usize) -> *const T ``` This is equivalent to `ptr.map_addr(|a| a & mask)` but also uses a cool llvm intrinsic. Proposed in rust-lang#95643 (comment) cc ``@Gankra`` ``@scottmcm`` ``@RalfJung`` r? rust-lang/libs-api

Add pointer masking convenience functions This PR adds the following public API: ```rust impl<T: ?Sized> *const T { fn mask(self, mask: usize) -> *const T; } impl<T: ?Sized> *mut T { fn mask(self, mask: usize) -> *const T; } // mod intrinsics fn mask<T>(ptr: *const T, mask: usize) -> *const T ``` This is equivalent to `ptr.map_addr(|a| a & mask)` but also uses a cool llvm intrinsic. Proposed in rust-lang#95643 (comment) cc `@Gankra` `@scottmcm` `@RalfJung` r? rust-lang/libs-api

Add pointer masking convenience functions This PR adds the following public API: ```rust impl<T: ?Sized> *const T { fn mask(self, mask: usize) -> *const T; } impl<T: ?Sized> *mut T { fn mask(self, mask: usize) -> *const T; } // mod intrinsics fn mask<T>(ptr: *const T, mask: usize) -> *const T ``` This is equivalent to `ptr.map_addr(|a| a & mask)` but also uses a cool llvm intrinsic. Proposed in rust-lang/rust#95643 (comment) cc `@Gankra` `@scottmcm` `@RalfJung` r? rust-lang/libs-api

…ingjubilee Optimize `pointer::as_aligned_to` This PR replaces `addr % align` with `addr & align - 1`, which is correct due to `align` being a power of two. Here is a proof that this makes things better: [[godbolt]](https://godbolt.org/z/Wbq3hx6YG). This PR also removes `assume(align != 0)`, with the new impl it does not improve anything anymore ([[godbolt]](https://rust.godbolt.org/z/zcnrG4777), [[original concern]](rust-lang/rust#95643 (comment))).

Add pointer masking convenience functions This PR adds the following public API: ```rust impl<T: ?Sized> *const T { fn mask(self, mask: usize) -> *const T; } impl<T: ?Sized> *mut T { fn mask(self, mask: usize) -> *const T; } // mod intrinsics fn mask<T>(ptr: *const T, mask: usize) -> *const T ``` This is equivalent to `ptr.map_addr(|a| a & mask)` but also uses a cool llvm intrinsic. Proposed in rust-lang/rust#95643 (comment) cc `@Gankra` `@scottmcm` `@RalfJung` r? rust-lang/libs-api

1. change `PtrMut::as_ptr(self)` and `OwnedPtr::as_ptr(self)` to take `&self`, otherwise printing the pointer will prevent doing anything else afterwards 2. make all `as_ptr` methods safe. There's nothing unsafe about obtaining a pointer, these kinds of methods are safe in std as well [str::as_ptr](https://doc.rust-lang.org/stable/std/primitive.str.html#method.as_ptr), [Rc::as_ptr](https://doc.rust-lang.org/stable/std/rc/struct.Rc.html#method.as_ptr) 3. rename `offset`/`add` to `byte_offset`/`byte_add`. The unprefixed methods in std add in increments of `std::mem::size_of::<T>`, not in bytes. There's a PR for rust to add these byte_ methods rust-lang/rust#95643 and at the call site it makes it much more clear that you need to do `.byte_add(i * layout_size)` instead of `.add(i)`

rust-highfive assigned m-ou-se Apr 4, 2022

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Apr 4, 2022

ChayimFriedman2 reviewed Apr 5, 2022

View reviewed changes

library/core/src/ptr/const_ptr.rs Outdated Show resolved Hide resolved

ChayimFriedman2 reviewed Apr 5, 2022

View reviewed changes

library/core/src/ptr/const_ptr.rs Show resolved Hide resolved

Gankra mentioned this pull request Apr 9, 2022

Add sub_ptr on pointers (the usize version of offset_from) #95837

Merged

rustbot added the T-libs Relevant to the library team, which will review and decide on the PR/issue. label Apr 14, 2022

This was referenced Apr 21, 2022

Tracking Issue for pointer_bytes_offsets #96283

Closed

Tracking Issue for pointer_is_aligned_to #96284

Open

rustbot added the T-libs-api Relevant to the library API team, which will review and decide on the PR/issue. label Apr 21, 2022

rust-highfive assigned joshtriplett and unassigned m-ou-se Apr 21, 2022

rustbot removed the T-libs Relevant to the library team, which will review and decide on the PR/issue. label Apr 21, 2022

yaahc assigned yaahc and unassigned joshtriplett Apr 21, 2022

rust-highfive assigned m-ou-se and unassigned yaahc Apr 29, 2022

oli-obk mentioned this pull request May 3, 2022

Avoid zero-sized allocs in ThinBox if T and H are both ZSTs. #96642

Merged

jakobhellermann mentioned this pull request May 5, 2022

[Merged by Bors] - some cleanup for bevy_ptr bevyengine/bevy#4668

Closed

WaffleLapkin added 5 commits May 12, 2022 12:54

Add convenience functions to pointers

c8c91f7

The functions added: - {*const T,*mut T}::{wrapping_,}byte_{offset,add,sub} - {*const T,*mut T}::{byte_offset_from,is_aligned,is_aligned_to}

Lift the Sized requirement from convenience ptr fns

a908eec

Since they work on byte pointers (by `.cast::<u8>()`ing them), there is no need to know the size of `T` and so there is no need for `T: Sized`. The `is_aligned_to` is similar, though it doesn't need the _alignment_ of `T`.

Implement ptr.is_aligned() in terms of .is_aligned_to()

6c1ebff

Optimize ptr.is_aligned_to()

5a5d62a

Apparently LLVM is unable to understand that if count_ones() == 1 then self != 0. Adding `assume(align != 0)` helps generating better asm: https://rust.godbolt.org/z/ja18YKq91

Fill-in tracking issues for features pointer_byte_offsets, const_poin…

03d4569

…ter_byte_offsets and pointer_is_aligned

WaffleLapkin force-pushed the ptr_convenience branch from ad372d3 to 03d4569 Compare May 12, 2022 08:55

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels May 18, 2022

bors added the merged-by-bors This PR was explicitly merged by bors. label May 19, 2022

bors merged commit d8a3fc4 into rust-lang:master May 19, 2022

rustbot added this to the 1.63.0 milestone May 19, 2022

WaffleLapkin deleted the ptr_convenience branch May 19, 2022 14:04

WaffleLapkin mentioned this pull request Aug 5, 2022

Optimize pointer::is_aligned_to #100169

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add convenience byte offset/check align functions to pointers #95643

Add convenience byte offset/check align functions to pointers #95643

WaffleLapkin commented Apr 4, 2022

rust-highfive commented Apr 4, 2022

Gankra commented Apr 4, 2022

ChayimFriedman2 commented Apr 5, 2022 •

edited

Gankra commented Apr 5, 2022

ChayimFriedman2 commented Apr 6, 2022 •

edited

Gankra commented Apr 6, 2022 •

edited

ChayimFriedman2 commented Apr 6, 2022 •

edited

Gankra commented Apr 17, 2022

WaffleLapkin commented Apr 17, 2022

Gankra commented Apr 21, 2022

yaahc commented Apr 21, 2022

yaahc commented Apr 29, 2022

WaffleLapkin commented May 12, 2022

joshtriplett commented May 18, 2022

bors commented May 18, 2022

bors commented May 18, 2022

bors commented May 19, 2022

rust-timer commented May 19, 2022

Add convenience byte offset/check align functions to pointers #95643

Add convenience byte offset/check align functions to pointers #95643

Conversation

WaffleLapkin commented Apr 4, 2022

rust-highfive commented Apr 4, 2022

Gankra commented Apr 4, 2022

ChayimFriedman2 commented Apr 5, 2022 • edited

Gankra commented Apr 5, 2022

ChayimFriedman2 commented Apr 6, 2022 • edited

Gankra commented Apr 6, 2022 • edited

ChayimFriedman2 commented Apr 6, 2022 • edited

Gankra commented Apr 17, 2022

WaffleLapkin commented Apr 17, 2022

Gankra commented Apr 21, 2022

yaahc commented Apr 21, 2022

yaahc commented Apr 29, 2022

WaffleLapkin commented May 12, 2022

joshtriplett commented May 18, 2022

bors commented May 18, 2022

bors commented May 18, 2022

bors commented May 19, 2022

rust-timer commented May 19, 2022

ChayimFriedman2 commented Apr 5, 2022 •

edited

ChayimFriedman2 commented Apr 6, 2022 •

edited

Gankra commented Apr 6, 2022 •

edited

ChayimFriedman2 commented Apr 6, 2022 •

edited