Add LUB computation for class types

[dcodeIO]

Addresses #2407. Previously, the LHS type was used to determine the resulting type of binary / ternary expressions. This is plausible for basic numeric types, but not so much for classes. Hence, computation of the common type now tries whether LHS and RHS map to the contextual type first, and if not (or there is none) computes the least upper bound of the two class types.

For example, in

class A {}
class B extends A {}
class C extends A {}

the least upper bound when mixing types A, B and or C is A, which is used if there isn't a more suitable contextual type. For example, in

class A {}
class B extends A implements I {}
class C extends A implements I {}

if contextual type is I, and B and C are mixed, the result remains I (both are assignable) instead of A (the LUB).

• I've read the contributing guidelines
• I've added my name and email to the NOTICE file

[dcodeIO]

Seems to be the same breakage as previously seen in #2575. Only happens on bootstrap:release and in CI.

[dcodeIO]

Last log is interesting in that its output looks a lot like memory corruption. Something is missing, though, given that the problem cannot be reproduced locally. Things I've seen in the past were differences in line endings leading to subtly different compilation and GH actions runners running out of memory and instead of exiting showing undefined behavior.

[dcodeIO]

Looking at the bootstrapped artifact (js to wasm, step 1), it's identical to what I have locally. Narrows it down to how the artifact is invoked / used.

[dcodeIO]

In fact becomes reproducible locally when replacing \r\n with \n in all parsed files. There is no difference in ordering, so the issue indeed appears to surface due to subtly different memory layout. Could be a stdlib component writing OOB.

[dcodeIO]

Slightly better stacktrace (preserving debug info even though optimizing):

     abort
     at src/flow/Flow#inheritAlternatives (wasm://wasm/0032745e:wasm-function[512]:0x8d8f)
     at src/compiler/Compiler#compileSwitchStatement (wasm://wasm/0032745e:wasm-function[741]:0x1c9fd)    
     at src/compiler/Compiler#compileStatement (wasm://wasm/0032745e:wasm-function[754]:0x1eb8a)
     at src/compiler/Compiler#compileStatements (wasm://wasm/0032745e:wasm-function[755]:0x1ef2d)
     at src/compiler/Compiler#doCompileDoStatement (wasm://wasm/0032745e:wasm-function[733]:0x1b42d)      
     at src/compiler/Compiler#compileStatement (wasm://wasm/0032745e:wasm-function[754]:0x1e960)
     at src/compiler/Compiler#compileStatements (wasm://wasm/0032745e:wasm-function[755]:0x1ef2d)
     at src/compiler/Compiler#compileFunctionBody (wasm://wasm/0032745e:wasm-function[816]:0x282e9) 

[CountBleck]

What if you revert #2586?

[dcodeIO]

Reverted locally to check, doesn't make a difference.

[dcodeIO]

Greetings from Rtrace

RTRACE: Error: OOB store32 at address 106533488
    at src/flow/Flow#set:flags (wasm://wasm/005ebf1a:wasm-function[1134]:0xdbef)
    at src/flow/Flow#inheritAlternatives (wasm://wasm/005ebf1a:wasm-function[1621]:0x11a8b)

RTRACE: Error: OOB store32 at address 106534128
    at ~lib/array/Array<i32>#__set (wasm://wasm/005ebf1a:wasm-function[1228]:0xe5e8)
    at src/flow/Flow#inheritAlternatives (wasm://wasm/005ebf1a:wasm-function[1621]:0x11bda)

[dcodeIO]

In a different setup (main branch), the first Rtrace error happens in Compiler#compileSwitchStatement, where breakingFlowAlternatives is assigned innerFlow and in subsequent loops breakingFlowAlternatives.inheritAlternatives(breakingFlowAlternatives, innerFlow) is called. Within that function, this#flags is assigned, which writes OOB, indicating that breakingFlowAlternatives points to invalid (say, already collected) memory.

Interestingly, temporarily assigning breakingFlowAlternatives to a global, just to keep it alive without relying on the shadow stack, mitigates the Rtrace errors, hinting towards a codegen / GC / shadow stack issue.

[CountBleck]

In other words, the initial compiler is bugged, not the bootstrapped one.

[dcodeIO]

Looks like it, with the issue only materializing in the Wasm variant when the stars align. The surprise "fix" I've employed just to work around the issue, in the particular setup above, is this change in src/compiler.ts

      // Combine all alternatives that merge again with outer flow
      if (possiblyBreaks || (isLast && possiblyFallsThrough)) {
        if (breakingFlowAlternatives) breakingFlowAlternatives.inheritAlternatives(breakingFlowAlternatives, innerFlow);
        else {
          breakingFlowAlternatives = innerFlow;
          Compiler.breakingFlowAlternatives = breakingFlowAlternatives; // HERE
        }

      // Otherwise just merge the effects of a non-merging branch
      } else if (!possiblyFallsThrough) {
        outerFlow.mergeSideEffects(innerFlow);
      }

with a static breakingFlowAlternatives: Flow | null = null added. Might well be that this unbreaks #2575 as well - in fact would be interesting to know if it does.

[dcodeIO]

Just tried with #2596, and indeed that mitigates the issue.