Add support for allocator-api2

[zakarumych]

This change allows using hashbrown with other crates that support allocator-api2.
It opens possibility for any allocator that supports allocator-api2 to be used with hashbrown.

allocator-api2 mimics types and functions of unstable allocator-api feature but works on stable.
hashbrown will be using allocator-api2 only when `"nightly" feature is not enabled.

Typical use-case for allocator-api2 is to use it both with and without "nightly" feature, however hashbrownis special-case since it is built as part ofstdandallocator-api2` is not ready for this.

If fitzgen/bumpalo#201 is accepted, explicit support for bumpalo would be obsolete.

caveat: Either "nightly", "allocator-api2" or both features should be enabled for hashbrown to compile.

[Amanieu]

Could the allocator-api2 crate have a nightly feature that causes it to simply re-export the Allocator trait from nightly rust? That way hashbrown can always use allocator_api2::Allocator and users can implement allocator_api2::Allocator without having to worry about whether the nightly feature is enabled. We would still need a special workaround for the rustc-dep-of-std case, but that can be handled easily.

[zakarumych]

Could the allocator-api2 crate have a nightly feature that causes it to simply re-export the Allocator trait from nightly rust?

It does. However it can't be used when building for std.
So hashbrown will have to use Allocator from core::alloc either way at least with rustc-dep-of-std feature enabled.
And in this case I think there's no much reason to not use it directly with "nightly".
Consumers still would be able to use allocator_api2::Allocator with hashbrown with or without "nightly"

[Amanieu]

This doesn't work well due to the way Cargo features works. Consider this case:

• Crate A uses hashbrown with the nightly feature enabled.
• Crate B uses hashbrown without the nightly feature, but with allocator-api2.

Crate B will try to implement the allocator traits using allocator-api2, but this will fail because hashbrown will expect the standard library traits.

I prefer making hashbrown uses std's Allocator trait only under the rustc-dep-of-std feature, and otherwise letting allocator-api2 decide which Allocator trait to use.

[zakarumych]

Yes, if nothing would enable "nightly" feature in allocator-api2 then it will be problematic.

Originally I tried to do what you suggests.

The problem with this is that "nightly" feature will enable "allocator-api2/nightly", enabling "allocator-api2" itself.
Since "rustc-dep-of-std" feature enables "nightly", it will add allocator-api2 as dependency. Which won't compile when building for std, I expect.
Removing "rustc-dep-of-std" -> "nightly" link is required then. Which I tried to avoid.

[Amanieu]

Could we just make nightly in hashbrown not imply nightly in allocator-api2? I think that would be the best way forward.

[zakarumych]

I found a problem with current approach.
If another crate depends on allocator-api2 and enables "allocator-api2/nightly", while no crates that depend on hashbrown enable "hashbrown/nightly", the hashbrown will use unstable API reexported from core and alloc without enabling #![feature(allocator_api)] and will fail to compile.

I tried few things to detect if "nightly" feature is enabled allocator-api2. It's easy to do, but I cant find a way that allows to put this in crate attribute. Crate attributes are inner attributes and proc-macro attributes are not allowed to be inner-attributes.
A few unstable features are required for this, but it must also work on stable, so not an option.

The possible workaround for this is to make allocator-api2 expose both stable and nightly APIs and allow each crate to choose which one to use.
But at this point there's no much reason in keeping unstable API in allocator-api2, a crate may simply use alloc crate.
So I can keep only stable part in allocator-api2 and make changes required to make it work in hashbrown.

[Amanieu]

Good point. In that case perhaps we should have 3 settings for hashbrown:

• allocator-api2 => uses Allocator trait from allocator-api2.
• allocator-nightly => uses unstable Allocator trait from std.
• If both features are enabled, compiler_error!.
• If neither feature is enabled, hide allocator support like we currently do.

This forces crates which need the allocator API to opt-in to an implementation, while catching the case that we can't support at compile time (2 crates needing different allocator APIs).

Since this is a breaking change, it will be part of hashbrown 0.14. I would also like the bumpalo support to be folded into this feature once fitzgen/bumpalo#201 is merged and published.

[zakarumych]

I would like to avoid compilation error when two features are enabled.
It is easier to deal with "enable at least one feature and it works", since each crate can fix the problem.
If two crates depend with on hashbrown different features and it doesn't compile... that's frustrating :)

In last iteration I removed re-export of unstable API from allocator-api2 keeping it with stable mirror, so crates may to choose which to use: alloc crate with unstable API or allocator-api2 crate with stable API, preferring alloc when nightly feature is enabled.
However this is still not ideal, since a crate that depends on bumpalo for example will not know what API it might use if that crate in particular doesn't not enables "bumpalo/nightly".

I think the solution should be simple.
The crate that uses hashbrown::HashMap should know that with or without "nightly" feature, it works with allocator_api2::Allocator (except when building for std)

[zakarumych]

Ok. I think I figured it out.

Typical crates would use allocator-api2 and enable usage of unstable allocator_api feature using "allocator-api2/nightly".
hashbrown is not typical case since it is built as part of std and thus does not depend on allocator-api2 when it is building as part of std.

Other crates would typically use hashbrown with "allocator-api2" enabled by default.
Optionally enabling nightly feature that will cause "allocator-ap2" to switch to re-exporting unstable allocator_api.

If default features are disabled and neither "allocator-api2" nor "nightly" feature enabled, custom non-public Allocator crate would be using, preventing users from using any non-default allocator parameter.

"nightly" enables "allocator-api2/nightly" and then never uses it. One may wonder why.
Because one crate may use hashbrown with allocator-api2, and other crate may enable "nightly" in hashbrown.
If it won't trigger "nightly" in allocator-api2 then it will cause hashbrown with "allocator-api2" to be incompatible with allocator-api2 crate which is silly :)

[zakarumych]

I removed custom bumpalo support so it would require release of bumpalo with allocator-api2 support before this can be merged.

[Amanieu]

Thanks for working on this, I like this new setup! However I would like the feature names to be renamed:

• nightly-base should just be nightly.
• The current nightly should be renamed to nightly-allocator, and used by rustc-dep-of-std.

This can be made to work by using a new feature of Cargo available since Rust 1.60: weak dependencies. Essentially the nightly line would become:

nightly-allocator = ["allocator-api2?/nightly", "nightly", "bumpalo?/allocator_api"]

This will disable the behavior of automatically enabling the optional dependency when the nightly feature is enabled, and only enable those features if the allocator-api2 or bumpalo crates are used by other crates.

[zakarumych]

Oh, great! I somehow missed the feature that I've being waiting for so long :)

[Amanieu]

Suggested change

	nightly = ["allocator-api2?/nightly", "bumpalo/allocator_api"]
	nightly = ["allocator-api2?/nightly", "bumpalo?/allocator_api"]

[zakarumych]

bumpalo is not optional dependency, it's dev-dependency.
So ? is not applicable and not necessary, nightly won't add bumpalo dependency except for examples, tests and benches.

[Amanieu]

@bors r+

[bors]

📌 Commit 326e80d has been approved by Amanieu

It is now in the queue for this repository.

[bors]

⌛ Testing commit 326e80d with merge 89184d4...

[bors]

☀️ Test successful - checks-actions
Approved by: Amanieu
Pushing 89184d4 to master...

[zakarumych]

bumpalo with allocator-api2 is released.
Maybe hashbrown with next patch version can be released too?