Native Image Committer and Community Meeting 2023-06-29

[christianwimmer]

List of all past and upcoming meetings: #3933

New and Noteworthy

• Simulation of class initializer is now merged:
  [GR-39406] Simulate class initializer. #6710
  Now we are working to enable the new class initalization strategy by default.

• Starting JDK 22 work early this time:
  [GR-46613] Initial JDK 22 support. #6803
  [GR-46732] Revive UnsafeFieldAccessorFactory for Native Image #6879

• Reduce the module dependencies of the image generator (for cleanliness reasons):
  [GR-35914] Reduce module dependencies of the native image generator. #6807
  [GR-46756] Further reduce image builder module dependencies. #6901

• Small steps to make image builds more deterministic:
  [GR-27034] Deterministic compilations in helloworld and petclinic. #6900

• Performance
  [GR-45765] Optimize Throwable#fillInStackTrace for JIT compiled code #6765

• Compatibility:
  [GR-46507] Support instance main methods (JEP 445) #6835

• Monitoring / tools:
  [GR-46599] Integrate foreign debuginfo types PR #6625. #6774
  The old "enterprise" Dwarf information generation is now deleted from our code base: [GR-46169] Remove SubstrateOptions.UseOldDebugInfo. #6665
  [GR-46190] Support JFR event GCHeapSummary. #6705

Deep Dive: "Parse Once" for images with JIT compilation

[teshull]

Runtime Compilation Support

---

ParseOnce

• (CWimmer) Added ParseOnce feature in May 2021

  • Same graph used to create analysis flows also is used for compilation
  • Enables optimizations to be performed before analysis

• Allowed Spring to run on Native-Image

• Still not enabled for applications which perform runtime compilation (i.e., Truffle languages)

---

ParseOnceJIT

• Goal: Support -H:ParseOnce in Truffle languages

• Main Idea: At analysis time correctly generate and analyze all needed graphs

  • Change analysis to handle multiple graph implementations of a given Java method

• We want to unconditionally use ParseOnce

  • Currently we have many SubstrateOptions.parseOnce() hooks throughout the code
  • ParseOnce needed for Truffle language support in Spring

• I've been adding support throughout the last year

  • New flag option: -H:ParseOnceJIT

---

Native Image Runtime Compilation Support Overview

• Main idea: store encoded Graal graphs in the executable

  • At runtime the Partial Evaluator can use them to generate new code

• New runtime-generated code can deoptimize back to code generated at buildtime

  • Destination is called the Deoptimization Target

• Generate two versions of methods at buildtime

  • Deoptimization Target
  • Original
  • Do this because Deoptimization Target methods are slower
  • Not discussed: how to optimize Deopt Methods
    • Paper written about how it works

---

Terminology

• Runtime Method: graphs used for runtime compilation
• Deopt(imization) Method: method compiled at buildtime that serves as the deoptimization target for Runtime Methods
• Original/Normal/Regular Method: method compiled at buildtime which is not a deoptimization target
• AOT Method: either Original or Deopt Method
• Deopt Entry: place within the Deopt Method where execution is transferred to when a Runtime Method deoptimizes

---

Runtime Compilation Example

---

Runtime Deoptimization Example (1)

---

Runtime Deoptimization Example (2)

---

Deeper Look: Runtime Compiled Methods

• Compilation at runtime requires storing encoded versions of graphs at buildtime

• During native-image analysis we must determine which methods are candidates for runtime compilation

  • Find all methods reachable from "compilation roots" (methods registered by prepareMethodForRuntimeCompilation)
  • Disregard reachable methods which are not eligible for runtime compilation

• Truffle provides guidelines for methods ineligible for runtime compilation

  • @TruffleBoundary
  • built-in Blocklist
  • More critera in allowRuntimeCompilation

• Truffle implementations also can provide guidelines on the Max # of Runtime Compiled Methods

  • Helps to catch Truffle code regressions

• Takeaway: Runtime Compilation is very dependent on analysis to ensure runtime compilation space does not explode

---

Deeper Look: Deoptimization Targets & Deoptimization Entrypoints

• Deopt methods must serve as the entrypoint for all runtime deopts

• Constraint: Deoptimization can only be from a FrameState with the Runtime method

• A FrameState describes a specific bytecode index & state

  • Created during initial graph creation (in BytecodeParser)
  • Used in Graal-JIT mode to deoptimize back to the interpreter
  • Also used in native-image mode to track debug information (via FullInfoPointNodes)

• During runtime graph encoding we record all FrameStates present within the graph

  • Build Deopt Target graphs to allow Deopt Entries at all these points
  • Validate at image creation with VerifyDeoptimizationEntryPoints

---

(Quick) Analysis TypeFlow Overview

• During Analysis important nodes in Graal Graphs are represented by TypeFlows (within FlowGraphs)

• TypeFlows are linked to their uses (link from def → use)

• A TypeFlow can also observe a TypeFlow to receive updates (link from observer ← observee)

• Uses and observers form a directed graph of the reachable methods

• Each TypeFlow tracks its TypeState (i.e., types present at that TypeFlow node)

• When a TypeFlow node's TypeState changes, it notifies all of its listeners (uses and observers) of its new TypeState

• When a listener TypeFlow node is notified of a change, it can react to this change

  • Main callbacks: addState, onObservedUpdate
  • May update its own TypeState
    • When this happens it must also update its listeners

---

InvokeTypeFlow Linking Overview

• InvokeTypeFlows serve as the connection between different AnalysisMethod FlowGraphs

• On a TypeState update it performs the following tasks;

  1. Determine call targets for new types
  2. Links caller's invoke argument TypeFlows to callee's parameter TypeFlows
  3. Links callee's return TypeFlow to the invoke's TypeFlow uses

• Important: For each new receiver type, linking is performed once

  • Updates are only posted for new receiver types

• More info: see DefaultStaticInvokeTypeflow, DefaultSpecialInvokeTypeflow, DefaultVirtualInvokeTypeflow

---

ParseOnceJIT Main Tasks

• Introduce Multi-Method Concept
• Selective linking between method type flows
• Dynamically link new methods reachable from runtime compilation (and deoptimization)
• Support inline before analysis

---

Multi-Method Concept

• One ResolvedJavaMethod can correspond to multiple underlying implementations

  • Original, Deopt, and Runtime methods
  • All have different graph implementations

• Now AnalysisMethod and HostedMethod implement MultiMethod

  • Each MultiMethod has completely separate analysis and hosted information
  • A given MultiMethod variant is identified by its MultiMethodKey
    • Also allows for easy navigation between different variants

• Runtime Compilation uses the following keys:

  • ORIGINAL_METHOD
  • DEOPT_TARGET_METHOD
  • RUNTIME_COMPILED_METHOD

• ORIGINAL_METHOD is "more equal" than other MultiMethodKeys

  • Used more in universe building, image layout

---

Runtime Compilation Invoke Patterns

• A given Method can "call" (transfer to) a variety of MultiMethods

• → denotes can call MultiMethod type

• ← denotes can be returned to from MultiMethod type

• Runtime

  • ↔ Runtime
  • ↔ Original
  • → Deopt

• Original

  • ↔ Original
  • ↔ Runtime
  • ← Deopt

• Deopt

  • ↔ Original
  • ← Deopt
  • ← Runtime

---

MultiMethod Invoke Patterns: Runtime

• ↔ Runtime
• ↔ Original
• → Deopt

---

MultiMethod Invoke Patterns: Original

• ↔ Original
• ↔ Runtime
• ← Deopt

---

MultiMethod Invoke Patterns: Deopt

• ↔ Original
• ← Deopt
• ← Runtime

---

InvokeTypeFlow Linking

• Added logic to InvokeTypeFlows to allow linking to multiple methods
• Allow parameter and return TypeFlows to be selective linked between caller ↔ callee
• hooks in code: determineCallees, performReturnLinking, performParameterLinking

---

Stub flows

• Since linking is performed once, we must eagerly link to runtime and deopt target methods

• However, want to lazily create flows

• Solution: create STUB flows

  • Stub flows link to incoming parameters / going uses, but do not have internal flows
  • Created in MethodTypeFlowBuilder

• We generate full flow lazily when needed

  • Runtime Method: Once it is reachable from another runtime compiled method (or is a compilation root)
  • Deopt Method: Once it is seen within a Runtime method's FrameState

---

Updating Deopt Target TypeFlow

• We create deopt entries for every Runtime method's framestate

• Due to inlining, a given method many be present in many different runtime methods' framestates

• Requires us to recreate Deopt graphs and redo internal flow

  • Why recreate? Deopt Entrypoints are inserted during the initial Graal graph generation
    • Nutshell reason: Deopt Entrypoint exception handling logic needs to created at this point

• Added new logic: updateFlowsGraph

---

Runtime Method Inlining

• It is important to allow inlining to prevent the number of reachable runtime method from exploding

  • When an invoke saturates it can cause many methods to be reachable for runtime compilation
  • Especially problematic for generic container classes

• Legacy implementation uses TruffleInlineDuringParsing to guide inlining during parsing

  • Without this most truffle languages can't be built

• We created a new InlineBeforeAnalysis policy for Runtime methods

---

New Runtime Inlining Policy

• Classify methods as being trivial to inline

1. Inline methods up to InlineDuringParsingMaxDepth (default 10) via this approach
2. If step 1 fails, inline methods according to "regular" policy
   • Inline methods as long as the number of new nodes introduced into the graph is less than InlineBeforeAnalysisAllowedInvokes (default 1)

• Code is implemented in RuntimeCompilationAnalysisPolicy and InlineBeforeAnalysisPolicyUtils

• No inlining is performed within Deopt methods

  • Calls can serve as implicit Deopt entries
  • Want to exit Deopt methods as quickly as possible

---

Linking Deopt Methods During Inlining

• Inlining methods prevents typeflows from entering Deopt Targets

  • No longer method boundary where we can hook multiple targets

• Added hooks to insert InlinedInvokeArgumentsNode when inlining before analysis

  • Passes parameter values to the Deopt Target, but not other variants

• Does not link the return value

  • Unneeded due to Deopt method's flow being more conservative than Runtime method.

• We create "DeoptInvokeTypeFlows" which link only to the Deopt Method.

---

Generating Runtime Graphs

• Runtime methods perform different optimizations than AOT methods

  • No explicit exceptions
  • Allow assumptions
  • Never fold @CompilationFinal (at build-time)
  • additional GraphBuilderPlugins

• PE will run during runtime compilation

  • Too much build-time optimization can prevent PE optimizations
    • for example, @CompilationFinal array objects
  • Still beneficial for code size and validation

• Added hooks in analysis create Runtime method graphs correctly: ParseGraph

• Compile & encode runtime graphs in onCompileQueueCreation hook

• Much of the logic for creating the Runtime providers is in TruffleFeature

  • Very unintuitive
  • I will (eventually) clean this up

---

ParseOnceJIT Current Status

• Most functionality is in master
• Enable via adding -H:+ParseOnceJIT to build process
  • off by default
• Goal: have -H:+ParseOnceJIT as default for GraalVM 23.1
  • will enable soon (upcoming weeks) on master
• Improvements will continue for foreseeable future (beyond 23.1)

---

Code References

• All of these files will undergo major revisions soon

• ParseOnceRuntimeCompilationFeature

  • Logic for all ParseOnceJIT-specific functionality

• RuntimeCompilationFeature

  • Logic for shared functionality between old (Legacy) & new (ParseOnceJIT) implementations

• TruffleFeature

  • Creates runtime graph creation environment
  • Provides criteria for whether a method can be runtime-compiled