Extended nullabillity support for Kotlin and other null-safe languages

[xpathexception]

Hello!

I'm currently developing Kotlin language support in order to bring Kaitai Struct to the Kotlin Multiplatform world. For now I've implemented most of the things and the implementation is able to pass 231 out of 233 tests on JVM and MacOS ARM64 targets.

But there's an issue with 7 of them related to kotlin's Null Safety concept.

There are at least two main operators in Kotlin exist to provide null-safety support - Non-null Assertion Operator (!!) and Safe Calls Operator (?.). The main issue is with the safe call operator - in order to use it and in order to be able to perform safe call chaining, both compiler and translator should be aware of nullability of each type.

I've came up with two approaches which could help resolve this issue. Let's pick NavParentRecursive and it's test as an example.

The most interesting part of the test looks like this:

assertEquals(actual = r.next().value(), expected = 1)
assertEquals(actual = r.next().parentValue(), expected = 255)
assertNull(r.next().next())

We don't care about null

Using this approach we can simply make every attributeReader return non-nullable type by requiring attribute being not null explicitly. Nullability in instances can be handled in the same way.

More on this

Pros:

• easy to implement
• already works

Cons:

• on the call site it is unclear will reading fail or not
• nullability is basically unmanageable
• need to adjust tests by replacing assertNull with assertFails

    /* attributeDeclaration */
    /* isNullable: false */
    private var value: IntU1 by Delegates.notNull()  // avoiding default values for primitive types
    fun value(): IntU1 = value

    /* attributeDeclaration */
    /* isNullable: true */
    private var next: NavParentRecursive? = null
    fun next(): NavParentRecursive = requireNotNull(next)
    
    private fun _read() {
        this.value = this._io.readU1()
        if (value() == 255.toIntU1()) {
            this.next = NavParentRecursive(_io = this._io, _parent = this, _root = _root)
        }
    }

    /* instanceDeclaration */
    private var parentValue: IntU1? = null
    private var f_parentValue = false

    fun parentValue(): IntU1? {
        if (f_parentValue) { return this.parentValue }
        if (value() != 255.toIntU1()) {
            this.parentValue = ((_parent() as NavParentRecursive).value()).toIntU1()
        }

        f_parentValue = true
        return this.parentValue
    }

Giving that adjusting test as follows:

    assertFails { r.next().next() }

Nullable types should be supported at DataType level

Using this approach wi need to be able to propagate nullability to the resulting type of every expression.

More on this

Pros:

• explicitly nullable types would be easier to use on the call site
• looks much safer than the other way
• expected behaviour in Kotlin
• tests will receive support automatically

Cons:

• looks like it requires adjusting the whole DataType layer
• requires to develop a set of rules for cases like DataType? %op% DataType in order to resolve resulting nullability
• requires special handling for most of the operators
• requires adjusting compiler and translator contracts and implementations

Giving that if we declare attribute reader's type nullable like this:

    /* attributeDeclaration */
    /* isNullable: true */
    private var next: NavParentRecursive? = null
    fun next(): NavParentRecursive? = next

compiler should be able to generate something like this:

assertEquals(actual = r.next()?.value(), expected = 1)
assertEquals(actual = r.next()?.parentValue(), expected = 255)
assertNull(r.next()?.next())

In order to do so we need to be able to know if the lvalue nullable or not and propagate nullability to the right side. Moreover there's one more caveat: if expression is used as parameter, we need to now if target function accepts nullable parameters and perform non-null assertion in some cases.

As the bottom line, I believe it would be much better to extend nullability support, at least for the Kotlin implementation.

I'm not sure that approach with adjusting the whole DataType system with, for example isNullable flag is the proper way to implement null safety support. Therefore I'm looking for some help with this using that or alternative approach.

In short, there are very few things needs to be done:

• a way to resolve nullability of expression
• a way to resolve nullability of a combined type
• a way to provide nullability with the DataType itself
• ability to translate expressions depending on operand's type nullability