ExpandableAllocator

A low-level .NET allocator that grows lazily.

Usage

// Reserve 1 TB of memory.
use alloc = Allocator.Create(Protection.Read, 0x1_000_000_000_000L)

let addr = alloc.Address
let ptr = NativePtr.fromNativeInt addr

// Actually allocate 512 bytes.
alloc.ActualSize <- nativeint 512

// Make sure the pointer hasn't moved.
alloc.Address |> should equal addr

// Read something.
NativePtr.read ptr |> should equal 0

// This would throw an AccessViolationException:
// NativePtr.write ptr 42

// Change the region's protection.
alloc.Protection <- Protection.ReadWrite

// Now, actually write our value.
NativePtr.write ptr 42
NativePtr.read ptr |> should equal 42

// A safe wrapper is also provided.
use stream = new ExpandableStream(alloc)

stream.CanWrite |> should equal true
stream.CanRead  |> should equal true
stream.CanSeek  |> should equal true

More examples can be seen in the Tests directory.

Installation

Install-Package ExpandableAllocator

How does it work?

Internally, the Allocator reserves memory when it is created (it does not actually allocate it) using VirtualAlloc or mmap.
This means that large chunks of memory (ie greater than 1TB) can be reserved without any allocation.

Then, when more memory is actually needed (using TryReserve or ActualSize), the Allocator commits the memory using VirtualAlloc or mprotect.

Finally, when the Allocator is Disposed, VirtualFree or munmap is called on the allocated memory region.