Native Image Committer Community Meeting 2021-11-18

[christianwimmer]

List of all past and upcoming meetings: #3933

New and Noteworthy

• Feature freeze for GraalVM 22.0 is on November 26

• JDK 8 support is going to be deleted. Working through the whole GraalVM gate system to remove JDK 8 gates.
  e02fcc1

• JDK 17 improvements / fixes
  091e9c1
  2a20d63
  99c9d7e
  c05d660

• Module system support:
  [GR-31710] Support building js-launcher with image-builder on module-path. WIP #3748
  [GR-34108] Extend boot module layer to include required modules #3821

• Robustness:
  f191598

• Compatibility improvements:
  [GR-34403] Add missing exception edge for array allocations. #3883
  [GR-33683] Update the process of setting the default locale. #4001
  4397715
  Register String(byte[]) constructor for runtime JNI access #3680
  [GR-33955] Handle bundle instances in the heap in the localization optimized mode. #3956
  [GR-30205] Extend the agent to collect the locale and classname of resource bundles. #3557
  2ad587c

• JFR support:
  99b234f

• Image size reduction:
  213b71c
  9530581
  7b5f831

• Cleanups:
  [GR-29957] Remove DeclarativeSubstitutionProcessor. #3271

• Medium-term planning:
  New user-friendly build output: [GR-33602] [RFC] More User-friendly Output for Native Image #3955
  Continuations/coroutines (support for Project Loom). Initial support was merged a year ago, but several bigger pieces and optimizations are still missing.

Deep Dive: Native Image Serial GC policy and performance

Native Image memory management manual including tunables: https://www.graalvm.org/reference-manual/native-image/MemoryManagement/

Serial GC Overview

• Single-threaded generational copying GC
• The object heap consists of chunks (including the image heap)
  • Same-sized chunks store small to medium objects (default: 1 MB chunks)
    • allocated as needed and cached for reuse
  • Variably-sized chunks store a single large object (currently only arrays)
• A collection copies the reachable small to medium-sized objects to new chunks
  • For large objects, their chunk is simply reassigned to the target space
  • Image heap objects are never copied or reassigned
  • In the worst case, GC needs twice the heap size while copying
• Each chunk has a card table (remembered set)
  • Entries indicate whether a memory range contains a reference
    • from the old generation to the young generation
    • from the image heap to the runtime heap
  • Card table entries are updated when writing such a reference
  • Enables collections to scan only objects in those memory ranges
• Collection policy decides
  • When to collect

    graal/substratevm/src/com.oracle.svm.core.genscavenge/src/com/oracle/svm/core/genscavenge/CollectionPolicy.java

    Line 116 in 9d6d87d

    boolean shouldCollectOnAllocation();

  • Whether to do an young collection or a complete collection

    graal/substratevm/src/com.oracle.svm.core.genscavenge/src/com/oracle/svm/core/genscavenge/CollectionPolicy.java

    Line 126 in 9d6d87d

    boolean shouldCollectCompletely(boolean followingIncrementalCollection);

  • The (changing) sizes of the different spaces of the heap and other parameters

    graal/substratevm/src/com.oracle.svm.core.genscavenge/src/com/oracle/svm/core/genscavenge/CollectionPolicy.java

    Lines 146 to 147 in 9d6d87d

    	/** The maximum capacity of the young generation, comprising eden and survivor spaces. */
    	UnsignedWord getMaximumYoungGenerationSize();

Old GC policy: BySpaceAndTime

graal/substratevm/src/com.oracle.svm.core.genscavenge/src/com/oracle/svm/core/genscavenge/BasicCollectionPolicies.java

Line 232 in 9d6d87d

public static final class BySpaceAndTime extends BasicPolicy {

• The maximum heap size is 80% of the physical memory size
• The young generation size is 10% of the maximum heap size, but at most 256 MB
• The young generation consists only of newly created objects ("eden", "nursery") which are promoted to the old generation
  • There are no survivor spaces to delay promotion
• Collections are triggered when the young generation is full
• Complete collections are triggered
  

  graal/substratevm/src/com.oracle.svm.core.genscavenge/src/com/oracle/svm/core/genscavenge/BasicCollectionPolicies.java

  Lines 239 to 240 in 9d6d87d

  	return estimateUsedHeapAtNextIncrementalCollection().aboveThan(getMaximumHeapSize()) ||
  	GCImpl.getChunkBytes().aboveThan(getMinimumHeapSize()) && enoughTimeSpentOnIncrementalGCs();

  • If promoting the entire young generation would exceed the maximum heap size
  • or if more than 50% of GC time has been spent on incremental collections

New GC policy: Adaptive

graal/substratevm/src/com.oracle.svm.core.genscavenge/src/com/oracle/svm/core/genscavenge/AdaptiveCollectionPolicy.java

Line 45 in 9d6d87d

final class AdaptiveCollectionPolicy extends AbstractCollectionPolicy {

• The maximum heap size is still 80% of the physical memory size
• The young generation size is at most 33% of the maximum heap size
  • Eden space for newly created objects
  • Survivor spaces for young collection survivors (at most 33% of young generation)
• Collections are triggered when eden is full
• Complete collections are triggered
  

  graal/substratevm/src/com.oracle.svm.core.genscavenge/src/com/oracle/svm/core/genscavenge/AdaptiveCollectionPolicy.java

  Line 151 in 9d6d87d

  public boolean shouldCollectCompletely(boolean followingIncrementalCollection) { // should_{attempt_scavenge,full_GC}

  • only following a young collection: reduces peak memory usage because unreachable object are already freed
  • If the old generation is full after the young collection
  • or if the old generation is likely to become full after the next young collection
  • or if the time spent on young collections since the last complete collection is more than the time two complete collections would take (ensure old garbage does not linger for too long)
• Space sizes are continuously adjusted for throughput and footprint
  • Throughput: percentage of wall-clock time the application spends not in garbage collection (cost: percentage spent in GC)
  • Based on HotSpot's ParallelGC policy, but since our collector is single-threaded, we cannot expect to reach a fixed throughput goal such as 98% for multi-threaded applications
  • Spaces start at a fixed capacity: 64 MB each for the young and old generation
  • The policy keeps track of the cost of young collections and complete collections (and other statistics)
    

    graal/substratevm/src/com.oracle.svm.core.genscavenge/src/com/oracle/svm/core/genscavenge/AdaptiveCollectionPolicy.java

    Line 399 in 9d6d87d

    public void onCollectionEnd(boolean completeCollection, GCCause cause) { // {major,minor}_collection_end

  • Costs for a young/complete collection at a specific eden/old generation size are modeled using least-squares regression with exponential discounting of older data points
    

    graal/substratevm/src/com.oracle.svm.core.genscavenge/src/com/oracle/svm/core/genscavenge/ReciprocalLeastSquareFit.java

    Line 36 in 9d6d87d

    final class ReciprocalLeastSquareFit {

  • Eden space and old generation are expanded unless the models predict cost reductions below a certain threshold for doing so
    

    graal/substratevm/src/com.oracle.svm.core.genscavenge/src/com/oracle/svm/core/genscavenge/AdaptiveCollectionPolicy.java

    Line 240 in 9d6d87d

    private void computeEdenSpaceSize() {

    
    

    graal/substratevm/src/com.oracle.svm.core.genscavenge/src/com/oracle/svm/core/genscavenge/AdaptiveCollectionPolicy.java

    Line 452 in 9d6d87d

    private void computeOldGenSpaceSize(UnsignedWord oldLive) { // compute_old_gen_free_space

  • Survivor space size and tenuring threshold (number of young collections after which an object is promoted to the old generation) are adjusted to balance young and complete collection costs
    

    graal/substratevm/src/com.oracle.svm.core.genscavenge/src/com/oracle/svm/core/genscavenge/AdaptiveCollectionPolicy.java

    Line 203 in 9d6d87d

    private void computeSurvivorSpaceSizeAndThreshold(boolean isSurvivorOverflow, UnsignedWord survivorLimit) {

Future Improvements

Main issue: #2386

• Variably-sized TLABs: currently fixed-size chunks are used as TLABs. Many threads but low allocation rates result in significant fragmentation and premature collections
• Mark&Compact GC for the old generation to avoid memory overhead during copying
• Optimized object pinning
• Exact write barriers especially for large arrays
• Clustering of related image heap objects
• ...

Open Discussion

Possible deep dive topics for next meeting

Please send suggestions, or "upvote" a suggestion, by adding a comment to this discussion.

• Reflection configuration: "query only" vs. "allow invocation" configuration for methods
• Conditional reflection configuration
• Static analysis: planned changes to image heap snapshotting
• Module system implementation: state and implications

[zakkak]

Hi, I would be interested to learn more about the planned changes to image heap snapshoting in the static analysis. Thank you Christian!

[christianwimmer]

Sounds good, talking about heap snapshotting is confirmed for the next meeting.