Rebuild from Bitcode fails on watchOS

[daltonclaybrook]

Describe the bug

Xcode version: 12.1 (12A7403)
swift-nio-ssl version: 2.10.0

When linking a watchOS app with swift-nio-ssl, the app compiles and runs successfully. But attempting to distribute the app and rebuild from Bitcode (required for App Store release) results in a failure:

ipatool failed with an exception: #<CmdSpec::NonZeroExitException

(more output in attached log below)

To reproduce

1. Create a new watchOS project in Xcode. This can be a standalone Watch app or iOS companion. It doesn't matter.
2. Add swift-nio-ssl as a dependency of the Watch extension with File -> Swift Packages -> Add Package Dependency... Important: When selecting the "NIOSSL" library, you will need to change the target on the righthand side from the iOS app to the WatchKit extension. The Watch extension is not selected by default.
3. Archive the app with Product -> Archive
4. After the archive is finished, the Xcode Organizer window will appear. Select "Distribute App"
5. Select "Ad Hoc" as the method of distribution
6. Important: On the next screen, make sure "Rebuild from Bitcode" is checked. The other fields don't matter. Select "Next"
7. Proceed through the next screen until the app starts rebuilding from Bitcode.
8. Observe the error

Expected behavior

Xcode should be able to rebuild the app successfully from Bitcode, and you should be able to export the Ad Hoc IPA file afterward.

Actual behavior

The Rebuild from Xcode step fails and the IPA file cannot be exported. I've attached the build log file that contains additional output from the build tool:
nio-ssl.log

Additional notes

This issue was originally created for the grpc-swift repo here, but it appears to actually be an issue with swift-nio-ssl instead, which is a dependency of grpc-swift. This discussion under that issue may provide more context.

Another important detail: I started to discover this issue when I uploaded a build to App Store Connect. The upload succeeded, but a few minutes later, I received an email from Apple with this message:

ITMS-90562: Invalid Bundle - Bitcode failed to compile for your watchOS binary because it includes assembly source code or inline assembly. To resolve this issue, rewrite the corresponding code in a higher-level language such as Objective-C or Swift and redeliver your app.

As noted by @Lukasa, we may be able to remove the assembly code from this project when compiling for watchOS.

[daltonclaybrook]

Glancing at the assemblies a bit, I see checks for the OPENSSL_NO_ASM preprocessor macro. Would the fix for this be to set that macro when building on watchOS?

[Lukasa]

Yes, broadly. We need to do a bit more than that though, as we also need to add conditions to the guards in the .S files for the ARM platforms. The ideal trigger for this would not be building for watchOS but instead would be targeting bitcode. That future-proofs the possibility that bitcode becomes the preferred target on other Apple platforms as well, and allows people to use swift-nio-ssl in bitcode-based applications. But we may not be able to do that.

I'm chatting with some colleagues to work out what the best strategy will be here.

[Lukasa]

Ah, apologies, OPENSSL_NO_ASM will be sufficient. We just need to get the right trigger condition for it.

[daltonclaybrook]

Some additional context in case it's helpful: Testing on my own fork using Swift Package Manager, it wasn't enough for me to define the OPENSSL_NO_ASM macro (I still got the error when rebuilding from Bitcode). I needed to actually remove all the .S files from the package before I stopped seeing the error.

[daltonclaybrook]

Any update on this? I may have some bandwidth to attempt a PR for this in the next few weeks, but I don't know if I have all the necessary context.

[Lukasa]

Yeah, this one is going to be very hard. Removing all the .S files is essentially not practical. Rendering them empty is, but removing them entirely is very challenging. We can't move the assembly files to a separate target without busting compilation, but a separate target is the only way that SwiftPM can apply its target-specific versioning.

I think we need a SwiftPM enhancement here first, to make it possible to conditionally exclude some file types based on platform. I've filed a request for it here: https://bugs.swift.org/browse/SR-14109

In the meantime, I'll investigate one other possibility that might work and see how it goes.

[daltonclaybrook]

Hey @Lukasa, just checking in here. Any luck on the "other possibility"?

I understand that SwiftPM doesn't have the necessary support to remove files for a given target, but since CocoaPods does support this, would it be possible to publish a new Podspec that specializes the source_files directive in order to remove the S files for watchOS?

[Lukasa]

Yes, I think this is a reasonable idea in the short term. I’m continuing to pursue this issue with teams internally.

[dweekly]

FYI, I believe this is blocking being able to use gRPC on watchOS in an app to be submitted to the App Store.