Cross compilation to Windows - link time evaluation

[avdv]

Hi.

I am trying to compile my project on Linux to Windows, arch is the same: x86_64.

I have set the target triplet to x86_64-windows-gnu.

I get linker error about scalanative__sc_nprocessors_onln, sysconf and sched_yield:

[error] lld-link: error: undefined symbol: scalanative__sc_nprocessors_onln
[error] >>> referenced by /home/claudio/code/scala-native-seed-project/target/scala-3.3.3/native/generated/cb163dbb.ll.o:(_SM17java.lang.RuntimeD19availableProcessorsiEO)
[error] lld-link: error: undefined symbol: sysconf
[error] >>> referenced by /home/claudio/code/scala-native-seed-project/target/scala-3.3.3/native/generated/cb163dbb.ll.o:(_SM17java.lang.RuntimeD19availableProcessorsiEO)
[error] lld-link: error: undefined symbol: sched_yield

(there are more, but I save these for another day ;-))

I am using scala-native 0.5.2-RC2 currently.

Looking at javalib/src/main/scala/java/lang/Runtime.scala:

def availableProcessors(): Int = {
  88   │     val available = if (isWindows) {
  89   │       val sysInfo = stackalloc[SystemInfo]()
  90   │       GetSystemInfo(sysInfo)
  91   │       sysInfo.numberOfProcessors.toInt
  92   │     } else sysconf(_SC_NPROCESSORS_ONLN).toInt
  93   │     // By contract returned value cannot be lower then 1
  94   │     available max 1
  95   │   }

my understanding is that it should take the first branch and call GetSystemInfo. Apparently it takes the second branch which leads to the errors.

Any help would be much appreciated.

[ekrich]

I am not sure we support cross compile well based on some of my knowledge about the tools.

In general, you need to install windows tools on Linux so your compiler can have all the windows headers and libraries available. When the Scala Native tools is using the correct tools it should take the correct branch above.

Here is something I grabbed that may help - https://swarminglogic.com/article/2014_11_crosscompile

[avdv]

Thank you for your quick response!

I am not sure we support cross compile well based on some of my knowledge about the tools.

In general, you need to install windows tools on Linux so your compiler can have all the windows headers and libraries available. When the Scala Native tools is using the correct tools it should take the correct branch above.

Yes, I do have a toolchain and the headers available that is able to target Windows (I am using zig cc).

And compilation is not the problem... linking the final executable is.

I thought that link time evaluation is carried out by scala-native, before producing LLVM assembly which is later compiled by clang to machine code. So it should only depend on the targetTriple setting and emit Windows specific code, right?

A small experiment:

import scala.scalanative.meta.LinktimeInfo.isWindows
import scalanative.unsafe.*

package foo {
  @extern
  object system:
    @name("foo_system_is_windows")
    def win: CInt = extern
    @name("foo_system_is_unix")
    def unix: CInt = extern
}

object Main {
  def main(args: Array[String]): Unit =
    if isWindows then
      println(foo.system.win)
    else
      println(foo.system.unix)
}

Trying to nativeLink that program fails with:

[error] lld-link: error: undefined symbol: foo_system_is_unix
[error] >>> referenced by /home/claudio/code/scala-native-seed-project/target/scala-3.3.3/native/generated/494ee56d.ll.o:(_SM5Main$D4mainLAL16java.lang.String_uEO)

I would expect it to fail with an undefined symbol foo_system_is_windows instead.

It seems that link time evaluation is somewhat broken here.

$ rg foo_system_is_
generated/494ee56d.ll
746:declare i32 @"foo_system_is_unix"()
760:  %_4000003 = call i32 @"foo_system_is_unix"()

As you can see, it took the second branch, not the first one.

[ekrich]

That is correct then I think. I don't know what your C helper functions look like.

Your C helper function foo_system_is_unix needs to compile on Windows too. There is no conditional compilation in Scala in regards to your extern. Typically, the C function called will be the same for both platforms and then use C conditional compilation to control which part gets referenced on which system. Your main function only calls the Win side but the linker still needs to resolve the symbols for the extern functions.

This is the glue code issue as far as I understand. See this and the next section - https://scala-native.org/en/latest/user/native.html#experimental-deployment-descriptor-for-passing-settings-to-the-compiler

[avdv]

Hah, i just found this comment:

This parser is a bit naive e.g. it will not work correctly for e.g. x86_64-linux-gnu where the vendor is omitted. However it seems to work fine for the discovered target triple emitted by LLVM. If the user configures it manually they should be careful to specify all the parts e.g. x86_64-unknown-linux-gnu.

I'll try with a four parts triplet when I'm at my computer again.

[ekrich]

I'd be curious to see your configuration once you get to that point. I think @lolgab tried zig in the past.

[avdv]

OK, using x86_64-unknown-windows-gnu makes it work better. After sbt compile:

$ rg foo_system_is_
generated/494ee56d.ll
772:declare i32 @"foo_system_is_windows"()
786:  %_4000003 = call i32 @"foo_system_is_windows"()

There is no mention of foo_system_is_unix in the .ll files anymore, instead a call to foo_system_is_windows is emitted. 🚀

The only complication is that zig cc does not support this sort of triplet... 😄 but I can work around that.

I'd be curious to see your configuration once you get to that point.

Currently it's quite a hack. I'll give an update when I have something to share. However, I do cross compile my project to arm64 Linux and x86_64 and arm64 Darwin; see https://github.com/avdv/scalals/blob/main/.github/crosscompile

[ekrich]

Since this is windows zig support, maybe adjust the title?