Exception in an import callback leaves the compiler in an inconsistent state, leading to You shall not have another CompilerStack aside me error on next compilation

[neocho]

Hi, I'm getting an internal exception error but I'm not too sure why. I'm running solc on an express server.

Steps to getting this error:

1. Compile a contract that has invalid imports
2. Try to compile a contract that has valid imports
3. solc.compile() throws error

Error i'm getting:

{
  errors: [
    {
      component: 'general',
      formattedMessage: 'Internal exception in StandardCompiler::compile: /solidity/libsolidity/interface/CompilerStack.cpp(107): Throw in function solidity::frontend::CompilerStack::CompilerStack(ReadCallback::Callback)\n' +
        'Dynamic exception type: boost::wrapexcept<solidity::langutil::InternalCompilerError>\n' +
        'std::exception::what: You shall not have another CompilerStack aside me.\n' +
        '[solidity::util::tag_comment*] = You shall not have another CompilerStack aside me.\n',
      message: 'Internal exception in StandardCompiler::compile: /solidity/libsolidity/interface/CompilerStack.cpp(107): Throw in function solidity::frontend::CompilerStack::CompilerStack(ReadCallback::Callback)\n' +
        'Dynamic exception type: boost::wrapexcept<solidity::langutil::InternalCompilerError>\n' +
        'std::exception::what: You shall not have another CompilerStack aside me.\n' +
        '[solidity::util::tag_comment*] = You shall not have another CompilerStack aside me.\n',
      severity: 'error',
      type: 'InternalCompilerError'
    }
  ]
}

[cameel]

May be the same thing as #575. Could you create a small, self-contained JS snippet that reproduces this?

[neocho]

Yes, here's a script of it:
https://replit.com/@datboineo/solc-test#index.js

[cameel]

Thanks! I managed to reproduce it. Minimal repro:

import solc from 'solc'

function importCallback(filePath) {
  throw new Error("IMPORT CALLBACK ERROR")
}

let input = {
  language: 'Solidity',
  sources: {'C.sol': {content: 'import "A.sol";'}},
}

try {
  solc.compile(JSON.stringify(input), {import: importCallback})
}
catch {
}

const output = solc.compile(JSON.stringify(input))
console.log(JSON.parse(output))

Looks like the problem happens when there is an error in the import callback, but you catch it and continue to use the compiler. I think that it must be leaving the compiler in an inconsistent state. The native binary has no import callback so it would just exit with an error. In JS you can catch the error and retry, which triggers this.

If that's the case then a workaround for this would probably be to unload the binary and load it again if you detect a crash in the callback.

A proper fix will require some debugging in the compiler to find out what's actually going on in there.

[neocho]

cool thanks for the feedback! will try that out

[alexreyes]

Also having this error

[ekpyron]

As stated in the docs, the import callbacks have to return be it successfully or with an explicit error. Throwing exceptions in the callbacks is invalid - i.e. you need to make sure that you catch exceptions in the callback implementation. Given that, I'm inclined to close this issue.

[cameel]

Good point.

Ok then, I think we should close this but before we do, we should emphasize that point in the README more. I missed it myself when reading it. It should explicitly say that throwing an exception is not a valid thing to do. Might be a good idea to change the example we have there so that it has a try/catch block that would convert any error to { error: ... }.

Actually, I wonder if we couldn't wrap the callback in our own try/catch block and print a warning if any exception escapes. This seems to be a common mistake.

[ekpyron]

Not sure how common of a mistake it is - the problem in wrapping it ourselves is that it's technically overhead for anyone who uses a proper callback as intended - and we'd need to decide what to return after catching... So yeah, changing the README and making this more explicit sounds good - but maybe then we can first wait and see if this still pops up again before trying to wrap this ourselves?

[cameel]

we'd need to decide what to return after catching...

We can simply rethrow, just printing a warning beforehand. This won't fix the problem but will give a better clue as to what the issue is. This report shows that the messages you get currently do not make it easy to figure out :)

but maybe then we can first wait and see if this still pops up again before trying to wrap this ourselves?

I think the overhead would not be that significant (come on, the whole callback is JS, that's already slow :)). I think that just adding a warning would not hurt to do right away, but documenting it properly is the most important here.