Ninja: Cross-compiling in subfolder fails

[mtribiere]

Bug description
When cross-compiling a static library in a subfolder, the generated ninja doesn't work and fail with this error:

CreateProcess failed: The system cannot find the file specified.

Tools

• SCons v4.5.2
• Cross-compilation with a GCC-like compiler
• Windows Environment
• Ninja 1.11.1

Setup/To redproduce

SConstruct:

import os
SetOption('experimental', 'ninja')

# Define cross-compilation variables
env_options = {
    "PLATFORM": "posix",
    "HOST_OS": "posix",
    "CC"    : "ccppc",
    "LD"    : "ldppc",
    "AR"    : "arppc",
    "CCCOM" : '$CC $CFLAGS $_CPPINCFLAGS -c -o $TARGET $SOURCES',
    "LINKCOM" : "$LD $LINKFLAGS -o $TARGET $SOURCES $_LIBDIRFLAGS $_LIBFLAGS",
    "CFLAGS": "-MP -MMD -MF ${TARGET}.d -fno-cprop-registers -fno-guess-branch-probability",
    "PROGSUFFIX" : "",
}

# Create an environmnet
env = Environment(**env_options)  
env.Tool('ninja')
env.Append(
            ENV = {'PATH' : os.environ['PATH']},
            CPPPATH = [
                '#/dependencies/a',
                '#/dependencies/b',
                '#/dependencies/c',
                '#/dependencies/d',

            ], 
            LINKFLAGS = [],
            #CCCOMSTR = "Compiling $TARGET",
            LINKCOMSTR = "Linking $TARGET"
        )

# Export the environement for build scripts
Export('env')

# Start the build
src_obj = SConscript('SConscript')

SConsript:

Import('env')

# Build the manual code lib
SConscript('subfolder-a\subfolder-b\SConscript')

subfolder-a\subfolder-b\SConscript:

Import('env')
LIB_MANUAL = env.Library(target = "LIB_MANUAL", 
            source = [
                       'src/filea.c',
                       'src/fileb.c',
                       'src/filec.c',
                       'src/filed.c'
           ])

[mwichmann]

in your list of sources in the subdir sconscript, the items are not properly comma-separated, which means string catenation is in effect. Is that just a transcription error, or does it change the behavior to fix that? (a little hard to reproduce here, since don't have ccppc, etc.)

[mtribiere]

It's a transcription error my bad, I'm going to correct that.

[mtribiere]

@mwichmann ccppc is sadly a private compiler, but if that helps, it's GCC 3.3.2 based compiled for windows (Yes an antique)

[mwichmann]

Maybe post a snip from the build.ninja file that shows the instruction gone wrong (and a bit more of the error message)? Commenting out the cross-compile bits it "looks okay to me" here (I'm not the ninja expert at all, though). e.g., produces stuff like:

build subA/subB/src/filea.o: CC subA/subB/src/filea.c | /usr/lib64/ccache/gcc subA/subB/src/filea.c
  CC = gcc
  env = 
  rspc = "-MP" "-MMD" "-MF" "subA/subB/src/filea.o.d" "-fno-cprop-registers" $
      "-fno-guess-branch-probability" "-Idependencies/a" "-Idependencies/b" "-Idependencies/c" $
      "-Idependencies/d" "-c" "-o" "subA/subB/src/filea.o" "subA/subB/src/filea.c"

[mtribiere]

Here is my output:

I will see what part of the Ninja file I can share

[bdbaddog]

Here is my output:

I will see what part of the Ninja file I can share

Please paste the text. Posting a screenshot is far less useful and harder to read/use.

[mwichmann]

Can't tell much from that, personally, but the line that starts with BUILD looks like the end is garbled?

[mwichmann]

BTW, when I asked can you post more, I was hoping the sample pasted into the issue here was a working (broken) repro, and you could paste info from that.

[mtribiere]

Here is one of the generated ninja rule:

build build\vaps_gen\QCG\main\fileA.o: CC vaps_gen\QCG\main\fileA.c | $
    vaps_gen\QCG\main\fileA.c
  CC = ccppc
  env = export PATHEXT='.COM;.EXE;.BAT;.CMD';
  rspc = "-MMD" "-MF" "build/vaps_gen/QCG/main/fileA.o.d" "-fno-cprop-registers" $
      "-fno-guess-branch-probability" "-fno-delayed-branch" "-fno-if-conversion2" $
      "-fno-if-conversion" "-fno-crossjumping" "-fno-loop-optimize" "-fno-thread-jumps" $
      "-fno-merge-constants" "-fno-defer-pop" "-fno-zero-initialized-in-bss" "-fsigned-char" $
      "-mcpu=603" "-mstrict-align" "-G" "0" "-g" "-O1" "-fvolatile" "-fno-builtin" "-Wall" $
      "-mvthreads" "-DCPU=PPC603" "-DTOOL=gnu" "-D__TARGET__" $
      "-Idependencies/CIG_PART_HMI/DEPA/" $
      "-Idependencies/MRCL_CIG_LIB/DEPB/" $
      "-Idependencies/MRCL_CIG_LIB/DEPC/" $
      "-Idependencies/MRCL_CIG_LIB/DEPD/" $
      "-Idependencies/MRCL_CIG_LIB/DEPE" "-c" "-o" $
      "build/vaps_gen/QCG/main/filaA.o" "vaps_gen/QCG/main/fileA.c"

And the text outut:

scons: Reading SConscript files ...
Initializing ninja tool... this feature is experimental. SCons internals and all environments will be affected.
SCons running in ninja mode. build.ninja will be generated.
scons: done reading SConscript files.
scons: Building targets ...
Generating: build.ninja
ninja will be run with command line targets:
Executing: ninja -f C:\Tools\DEV\HLX_HMI_CORE_DEV\build.ninja -j1

FAILED: build/vaps_gen/QCG/main/fileA.o←[2K
export PATHEXT='.COM;.EXE;.BAT;.CMD';ccppc "-MP" "-MMD" "-MF" "build/vaps_gen/QCG/main/fileA.o.d" "-fno-cprop-registers" "-fno-guess-branch-probability" "-fno-delayed-branch" "-fno-if-conversion2" "-fno-if-conversion" "-fno-crossjumping" "-fno-loop-optimize" "-fno-thread-jumps" "-fno-merge-constants" "-fno-defer-pop" "-fno-zero-initialized-in-bss" "-fsigned-char" "-mcpu=603" "-mstrict-align" "-G" "0" "-g" "-O1" "-fvolatile" "-fno-builtin" "-Wall" "-mvthreads" "-DCPU=PPC603" "-DTOOL=gnu" "-D__TARGET__" [... ALL MY DEPS... ] "-c" "-o" "build/vaps_gen/QCG/main/fileA.o" "vaps_gen/QCG/main/fileA.c"
CreateProcess failed: The system cannot find the file specified.
ninja: build stopped: subcommand failed.scons: *** [build.ninja] CalledProcessError : Command 'ninja' returned non-zero exit status 1.
Traceback (most recent call last):
  File "C:\Tools\DEV\scons\scripts\..\SCons\Action.py", line 1310, in execute
    result = self.execfunction(target=target, source=rsources, env=env)
  File "C:\Tools\DEV\scons\scripts\..\SCons\Tool\ninja\__init__.py", line 123, in ninja_builder
    for output in execute_ninja():
  File "C:\Tools\DEV\scons\scripts\..\SCons\Tool\ninja\__init__.py", line 120, in execute_ninja
    raise subprocess.CalledProcessError(return_code, 'ninja')
subprocess.CalledProcessError: Command 'ninja' returned non-zero exit status 1.
scons: building terminated because of errors.

[mtribiere]

BTW, when I asked can you post more, I was hoping the sample pasted into the issue here was a working (broken) repro, and you could paste info from that.

I'm sorry I can't dedicate much more time for a sample repo right now, but if I can I will

[jcbrill]

@mwichmann Is it possible that the text after BUILD are terminal control characters?

The exact sequence shows up in (at least the first link):
https://stackoverflow.com/questions/66395499/sbt-has-lost-its-ability-to-format-a-screen-with-ctrl-characters-why
https://stackoverflow.com/questions/16755142/how-to-make-win32-console-recognize-ansi-vt100-escape-sequences-in-c#comment92954461_16799175

[mwichmann]

sure, it's possible... since an external tool is involved (ninja) it's kind of hard to see what's going on, so failure to find a file and a display that shows extra chars possibly attached to a filename raised suspicions. Just guessing at the moment.

[bdbaddog]

How did you install ninja on your system?
Where is the binary located on your system?

[bdbaddog]

From the error, looks like either ninja couldn't be found, or ccppc couldn't be found.

Can you try specifing ccppc in your example SConstruct with it's full path rather than just as ccppc ?

[dmoody256]

ninja produces this error when the binary it's trying to execute is not in the path:

CreateProcess failed: The system cannot find the file specified.

It is trying to create a subprocess to perform some task and the binary it tried to use in that process creation can not be found.

SCons does not generally rely on shell environment from the shell its launched from (unless you manually forced it do so) and so SCons is constructing a path internally for the commands it needs to run. Ninja however requires you to setup the shell, because it is launching the process with the given binary from the current shell environment.

[dmoody256]

looking at your SConstruct a little closer I see its setting the PLATFORM to 'posix', this is causing ninja to use export to set things in ninja commands:
export PATHEXT='.COM;.EXE;.BAT;.CMD';ccppc ...
The export command is not in the default windows shell environment (cmd.exe), so it could be that the The system cannot find the file specified. message is actually from export.

[mtribiere]

How did you install ninja on your system? Where is the binary located on your system?

I tried two ways, one with the executable inside the root folder of the project and one with the pip install ninja. Both gave the same output.

Can you try specifing ccppc in your example SConstruct with it's full path rather than just as ccppc ?

I will try a little later.

looking at your SConstruct a little closer I see its setting the PLATFORM to 'posix', [,,,]

Yes sadly it's a bit of a tricky environment where I'm on windows but I need posix-style commands/arguments. I tried export and it indeed doesn't work.

[bdbaddog]

How did you install ninja on your system? Where is the binary located on your system?

I tried two ways, one with the executable inside the root folder of the project and one with the pip install ninja. Both gave the same output.

Can you try specifing ccppc in your example SConstruct with it's full path rather than just as ccppc ?

I will try a little later.

looking at your SConstruct a little closer I see its setting the PLATFORM to 'posix', [,,,]

Yes sadly it's a bit of a tricky environment where I'm on windows but I need posix-style commands/arguments. I tried export and it indeed doesn't work.

Sounds more like mingw type environment?