Any idea why I have the error : The system cannot find the path specified: '.sconsign.tmp' -> '.sconsign.dblite':

[peterphonic]

Hello,

I am using a Docker container to build our windows program with 14.1. When I compile on release, everything is working fine, then when I compile debug, I got this:

FileNotFoundError: [WinError 3] The system cannot find the path specified: '.sconsign.tmp' -> '.sconsign.dblite':
  File "c:\python38\lib\site-packages\SCons\Script\Main.py", line 1412:
    _exec_main(parser, values)
  File "c:\python38\lib\site-packages\SCons\Script\Main.py", line 1374:
    _main(parser)
  File "c:\python38\lib\site-packages\SCons\Script\Main.py", line 1154:
    nodes = _build_targets(fs, options, targets, target_top)
  File "c:\python38\lib\site-packages\SCons\Script\Main.py", line 1349:
    jobs.run(postfunc = jobs_postfunc)
  File "c:\python38\lib\site-packages\SCons\Taskmaster\Job.py", line 125:
    postfunc()
  File "c:\python38\lib\site-packages\SCons\Script\Main.py", line 1346:
    SCons.SConsign.write()
  File "c:\python38\lib\site-packages\SCons\SConsign.py", line 133:
    syncmethod()
  File "c:\python38\lib\site-packages\SCons\dblite.py", line 152:
    self._os_replace(self._tmp_name, self._file_name)
Exception ignored in: <function dblite.__del__ at 0x00000277A00E69D0>
Traceback (most recent call last):
  File "c:\python38\lib\site-packages\SCons\dblite.py", line 144, in __del__
  File "c:\python38\lib\site-packages\SCons\dblite.py", line 141, in close
  File "c:\python38\lib\site-packages\SCons\dblite.py", line 152, in sync
FileNotFoundError: [WinError 3] The system cannot find the path specified: '.sconsign.tmp' -> '.sconsign.dblite'
PS C:\mount>

It does the the same thing the other way around (compiling first debug, then release). So, it looks like scons is not able to copy .sconsign.tmp to .sconsign.dblite, which is weird, because I can clearly see that those 2 files are present in the folder.

This problem doesn't occur when I compile directly on the host (outside the container). I am wondering if it could be a user permission issue.

[mwichmann]

Is this an old SCons version? The line numbers don't line up with what I'm looking at, and there has been some work in this module recently.

Yes, it smells a bit like a permission error, but in fact in the sync() method the file rename from temporary to permanent is wrapped in a try block that catches PermissionError and tries to fix up - but you're getting FileNotFoundError. So this is a bit of a head scratcher. Almost like some kind of race in the closedown...

[peterphonic]

I am using 4.5.2.

I did multiple test and looks like everything is fine when I am using -j8 instead of -j12. Weird. So, the first -j12 in debug is working fine. Then, the next -j12 generate the error above. If compiling with -j8, no error. I'll do further test to make sure...

[bdbaddog]

does your build fire off another scons at the same time? do you have multiple docker builds running which could be trying to write to the same .sconsign file?

[mwichmann]

Someday we'll get off this hack of dumping the whole in-memory signature database to a file all in one go on process shutdown, and depending on __del__ to do the right thing, which it's kind of notorious for not always doing in Python. But that doesn't help you today. Yeah, changing the #jobs does make it sound like something racy.

[peterphonic]

@bdbaddog Yes, I am 100% sure that I have just one container and one compilation running.
@mwichmann I did a LOT of test inside and outside of the container. I wanted to know if the problem could be one the following :

1. racy condition
2. antivirus on host system
3. permissions

Config :
OS: Windows 11
system: I5, 32 gig of ram
permission: I think I have kind of admin permission because I can install software, but for sure, my pc is on the corporate network

1. Test 1, compiling source code on host system

Everything is compiling fine with -j16, both debug and release, vice-versa, no problems

2. Test 2, copying source code inside container, compiling inside the container

Everything is compiling fine with -j16, both debug and release, vice-versa, no problems

3. Test 3 source code on host, compiling inside the container, with a mount between container and source code

I have the problem that I mentioned above. The only way to make debug and release to compile with no problem is by doing
scons -j4
scons -j1 --opt
but then if I do another scons -j16 without any changes, I have the error. Soon as I give more cores, I have the error.

4. Ultimate test 4 on another pc, source code on host, compiling inside the container, with a mount between container and source code

Other pc config:
OS: Windows 11
system: I9, 32 gig of ram
permission: I have an admin account, pc is NOT on the corporate network

And now, Everything is compiling fine with -j16, both debug and release, vice-versa, no problems!!!

---

So, the error message I have on my I5 looks like a race condition, but I don`t think it is the problem. I suspect a permission problem with the user of the container that tries to replace a file on the mount that is also located on the host.

[mwichmann]

That detailed work is useful. Not quite sure how mounts work that have you reaching outside the container, and if there might be some special considerations, either performance-wise or permission-wise.

[peterphonic]

Not quite sure how mounts work that have you reaching outside the container

Actually, I think it should be transparent for the container, I start the container with --process=isolation, so, the container use same kernel as the host (Without this flag, I cannot compile at all with scons btw when inside a windows container).

.sconsing.dblite is created on the root of the source code, root located on the host (but shared via the mount with the container). If I right click on the file, I can see the owner is S-1-5-93-2-1, which is the container user. I have to find out why it causes problem on my I5 (located on corporate network), and not on my I9 that is outside the corporate network.

[mwichmann]

You could experiment with putting the sconsign file somewhere else - inside the container. I think you can only do this programmatically (SConsignFile), not via a CLI option. "That shouldn't make a difference", but it would be an interesting data point.

[peterphonic]

Nice, I did not know about SConsignFile! I added do this in my SConstruct for testing purpose:

SConsignFile("C:/TEMP/.sconsign")

C:/TEMP is a folder in my container only, not accessible outside to the host. Now everything is compiling fine within the container with -j16 on both debug and release. Problem fixed!

Thx!

[bdbaddog]

Great! So as you suspected some issue with permission on filesystems outside the container and the sconsign file!

[mwichmann]

FWIW, your usage might, or might not, be appropriate for Repository(). Not very commonly used these days (as far as we know, since we don't do embed any telemetry in SCons :-) ), it allows a directory elsewhere - NFS or other network share, host-shared mount, etc. to operate as a kind of backing store, everything that can be filled from there is, but any necessary building happens in the local directory.