Skip to content

A Kotlin library providing a simple, high-performance way to use off-heap native memory in JVM applications.

License

Notifications You must be signed in to change notification settings

target/native_memory_allocator

Repository files navigation

native_memory_allocator

A library which uses sun.misc.Unsafe to allocate off-heap native memory.

Motivation

The goal of this project is to provide a simple way to allow JVM applications to store many gigabytes of data in local memory without garbage collection pauses and overhead.

This project provides the following components:

  • NativeMemoryAllocator
    • Allocates a configurable-size block of native memory on creation
    • Breaks this block into pages of configurable size and maintains a list of free pages
    • Supports operations to allocate, resize, and free native memory buffers similar to the malloc, realloc, and free functions in C
    • Advantages of this design:
      • All allocate, resize, and free operations after startup are very fast because they do not do native-memory allocation or free operations
      • Fragmentation is not an issue because NativeMemoryBuffer has no requirement for memory pages to be contiguous
  • Buffer
    • Supports OnHeapMemoryBuffer as a normal on-heap byte array
    • Supports NativeMemoryBuffer as a list of native memory pages allocated by NativeMemoryAllocator
    • Supports operations to copy data between OnHeap and Native buffers efficiently
  • NativeMemoryMap
    • Simple, high-level Map interface supporting get and put operations
    • Manages native memory automatically as entries are added and deleted using NativeMemoryAllocator
    • Supports configurable map backends
  • Metrics
    • Metrics reporter classes exist for NativeMemoryAllocator and NativeMemoryMap using Micrometer

Benchmarks

See the benchmarks directory for JMH benchmarks.

Benchmark results:

Using as a maven or gradle dependency

Options:

  1. Use jitpack.io: https://jitpack.io/#target/native_memory_allocator

    jitpack allows anonymous downloads of maven/gradle artifacts.

    Add repository to maven/gradle build: https://jitpack.io

    Add dependency to maven/gradle build: com.github.target:native_memory_allocator:VERSION See releases page for current version.

OR

  1. Use github packages: https://github.com/target/native_memory_allocator/packages/1407699

    github packages requires using a github.com personal access token (PAT) in maven/gradle build: https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-apache-maven-registry

    Add repository to maven/gradle build: https://maven.pkg.github.com/target/native_memory_allocator

    Add dependency to maven/gradle build: com.target:native_memory_allocator:VERSION See releases page for current version.

Example Usage

See the examples directory for complete runnable example applications.

Basic steps to use NativeMemoryAllocator:

Define a map value object.

data class CacheObject(
    val s: String,
)

Define a NativeMemoryMapSerializer to convert the value object to and from byte buffers. This serializer can use any form of serialization (e.g. Google Flatbuffers, Google Protocol Buffers, etc). The serializer will be invoked on every get and put operation to convert between a byte buffer and a normal object.

private class CacheObjectSerializer : NativeMemoryMapSerializer<CacheObject> {

    override fun deserializeFromOnHeapMemoryBuffer(onHeapMemoryBuffer: OnHeapMemoryBuffer): CacheObject {
        return CacheObject(
            s = String(onHeapMemoryBuffer.toTrimmedArray()),
        )
    }

    override fun serializeToByteArray(value: CacheObject): ByteArray {
        return value.s.toByteArray()
    }

}

Define a NativeMemoryAllocator with a certain page size and native memory size. In this example we use a 20GB native memory size and a 4KB page size.

The NativeMemoryAllocator will do a single allocation of all native memory when it is constructed. This is intended to be used for the life of the application.

val nativeMemoryAllocator = NativeMemoryAllocatorBuilder(
    pageSizeBytes = 4_096,
    nativeMemorySizeBytes = (20L * 1_024L * 1_024L * 1_024L),
).build()

Define a NativeMemoryMap specifying the valueSerializer, nativeMemoryAllocator, and backend type to use.

Multiple NativeMemoryMaps may share a single NativeMemoryAllocator.

val nativeMemoryMap = NativeMemoryMapBuilder<Int, CacheObject>(
    valueSerializer = CacheObjectSerializer(),
    nativeMemoryAllocator = nativeMemoryAllocator,
    backend = NativeMemoryMapBackend.CONCURRENT_HASH_MAP,
).build()

Put a value into the map. The key will be stored as a normal object on-heap memory. The value will be serialized and stored in off-heap memory allocated by NativeMemoryAllocator.

nativeMemoryMap.put(
    key = 1,
    value = CacheObject(
        s = "test",
    ),
)

Get a value from the map. The value will be fetched from native memory, copied into an OnHeapBuffer, deserialized, and returned as a nullable value to indicate presence or absence in the map.

val value = nativeMemoryMap.get(key = 1)

Class Diagram

NMAComponents

Generating KDoc HTML documentation

Kotlin classes in this library are documented with KDoc style comments.

The Dokka plugin is present in build.gradle.kts.

To generate HTML documentation use the following command:

./gradlew -PdokkaEnabled=true dokkaHtml

Open build/dokka/html/index.html to view generated documentation.