Premature initialization of SVM encodings

[sgammon]

Describe the issue

When building a Truffle-enabled native-image with a certain combination of Feature implementations on the classpath, initialization of registered encodings for TruffleString may occur before the full suite of Truffle languages has loaded; this causes an invalid state which causes TruffleRuby to crash later, at runtime.

Truffle decides whether to enable JCodings based on a simple condition:

graal/truffle/src/com.oracle.truffle.api.strings/src/com/oracle/truffle/api/strings/JCodings.java

Line 60 in 407ca89

if (TruffleOptions.AOT && !TStringAccessor.getNeedsAllEncodings()) {

TStringAccessor delegates to the engine:

graal/truffle/src/com.oracle.truffle.api.strings/src/com/oracle/truffle/api/strings/TStringAccessor.java

Lines 66 to 68 in 407ca89

	static boolean getNeedsAllEncodings() {
	return ENGINE.getNeedsAllEncodings();
	}

Which delegates to EngineAccessor:

graal/truffle/src/com.oracle.truffle.polyglot/src/com/oracle/truffle/polyglot/EngineAccessor.java

Lines 1982 to 1985 in 407ca89

	@Override
	public boolean getNeedsAllEncodings() {
	return LanguageCache.getNeedsAllEncodings();
	}

Which ultimately calls into the LanguageCache:

graal/truffle/src/com.oracle.truffle.polyglot/src/com/oracle/truffle/polyglot/LanguageCache.java

Lines 214 to 220 in 407ca89

	public static boolean getNeedsAllEncodings() {
	for (LanguageCache cache : languages().values()) {
	if (cache.isNeedsAllEncodings()) {
	return true;
	}
	}
	return false;

... which then queries the registered Truffle languages to determine if all codings are needed.

However, at this stage of execution, it is not guaranteed that Truffle languages have loaded yet via TruffleBaseFeature:

graal/substratevm/src/com.oracle.svm.truffle/src/com/oracle/svm/truffle/TruffleBaseFeature.java

Lines 247 to 255 in 407ca89

	invokeStaticMethod("com.oracle.truffle.api.impl.Accessor", "getTVMCI", Collections.emptyList());
	invokeStaticMethod("com.oracle.truffle.polyglot.InternalResourceCache", "initializeNativeImageState",
	Collections.singletonList(ClassLoader.class), imageClassLoader);
	invokeStaticMethod("com.oracle.truffle.polyglot.LanguageCache", "initializeNativeImageState",
	Collections.singletonList(ClassLoader.class), imageClassLoader);
	invokeStaticMethod("com.oracle.truffle.polyglot.InstrumentCache", "initializeNativeImageState",
	Collections.singletonList(ClassLoader.class), imageClassLoader);
	invokeStaticMethod("com.oracle.truffle.api.impl.TruffleLocator", "initializeNativeImageState",
	Collections.emptyList());

TruffleRuby at latest release happens to depend on all encodings. As a result, the native-image created under these conditions either breaks immediately at runtime, with:

Caused by: java.lang.IllegalArgumentException: unknown encoding: "ASCII-8BIT"
        at org.graalvm.truffle/com.oracle.truffle.api.strings.InternalErrors.illegalArgument(InternalErrors.java:76)
        at org.graalvm.truffle/com.oracle.truffle.api.strings.InternalErrors.unknownEncoding(InternalErrors.java:96)
        at org.graalvm.truffle/com.oracle.truffle.api.strings.TruffleString$Encoding.fromJCodingName(TruffleString.java:1094)
        at org.truffleruby.core.encoding.TStringUtils.jcodingToTEncoding(TStringUtils.java:43)
        at org.truffleruby.core.encoding.RubyEncoding.<init>(RubyEncoding.java:57)
        at org.truffleruby.core.encoding.Encodings.initializeRubyEncodings(Encodings.java:95)
        at org.truffleruby.core.encoding.Encodings.<clinit>(Encodings.java:43)

... or at build time with the same exception, if --initialize-at-build-time=org.truffleruby.core.encoding.Encodings is passed.

Steps to reproduce the issue

In our case (Elide), we are building against latest (24.0.1) native-image, with GraalPython, TruffleRuby, and GraalJs.

GraalPython triggers the early initialization of TruffleString encodings via a handful of calls in the BouncyCastleFeature and JNIFeature classes. In both cases, there are checks against PythonOptions static fields; for example here. It is the class initialization of PythonOptions which initializes encodings with TruffleString literals used at class-init time.

1. git clone git@github.com:elide-dev/elide.git -b main
2. cd elide && ./gradlew build -x test -x check
3. vim ./packages/cli/build.gradle.kts
4. Change enableRuby to true
5. ./gradlew :packages:cli:nativeCompile
6. ./packages/cli/build/native/nativeCompile/elide.debug shell --language=RUBY

Describe GraalVM and your environment:

• GraalVM version (latest snapshot builds can be found here), or commit id if built from source: 24.0.1
• JDK major version: 22.0.1
• OS: macOS Sonoma 14.3.1
• Architecture: M-series (aarch64)

[fernando-valdez]

Thanks for reporting this. I will take a look at it

[eregon]

This specific code changed on master, compare
24.0: https://github.com/oracle/graal/blob/release/graal-vm/24.0/truffle/src/com.oracle.truffle.api.strings/src/com/oracle/truffle/api/strings/TruffleString.java#L1049-L1051
master:

graal/truffle/src/com.oracle.truffle.api.strings/src/com/oracle/truffle/api/strings/JCodings.java

Line 143 in c0959ee

static TruffleString.Encoding fromJCodingsName(String jCodingsName) {

Could you try with master too?
There are dev builds at https://github.com/graalvm/graalvm-ce-dev-builds and https://github.com/graalvm/oracle-graalvm-ea-builds/releases.
I suspect no error or a different error on master.
The logic to detect if all encoding are needed also changed, which might have fixed this.
cc @woess