Create CrossCompilationModel.md #73581
Conversation
Add some documentation to describe the proposed model for cross-compilation, associated flags, and the reasoning for the structure. This should allow us to have a reference for the design allowing us to evolve the model.
|
CC: @etcwilde |
There was a problem hiding this comment.
Many editorial requests.
Contentwise, I think we're missing some details on the behavior of the flags, specifically, where the standard library and other libraries live. It's not really an overlay, it's the standard library. Neither are Foundation nor Dispatch, which are also parts of the SDK. If I had the energy and time, I could totally build a Linux distro with the Swift standard library installed natively into /usr/lib, which would then make it part of the sysroot at /.
At least the way I understand it, the C/C++ headers should go under -sysroot. Any and all Swift bits should go under -sdk (except the bits that are specific to a given compiler, which are compiler-resources. I'm aware that I'm ignoring the binary swiftmodules right now.). If -sysroot is left unspecified, it seems like it should default to whatever -sdk was set to so that existing builds continue to work as they used to.
| code against. | ||
|
|
||
| The compiler resources are content for the compiler that the compiler provides. | ||
| In the case of the Swift compiler, this includes the Swift shims. Whilst this is |
There was a problem hiding this comment.
Standard library macros are new! However, I think that they should be going into the SDK - they are part of the standard library which goes into the SDK.
There was a problem hiding this comment.
Yes... but.... what triple do we compile them for?
I'm being somewhat facetious here.
There was a problem hiding this comment.
I think that the standard library macros should go into the SDK, which is technically a change and something we should try to get in for 6.0.
| These pieces compose in a particular manner to build a system image to build | ||
| code against. | ||
|
|
||
| The compiler resources are content for the compiler that the compiler provides. |
There was a problem hiding this comment.
something something "matches that specific compiler" something something "not really meant to be shared across different compilers" something something.
There was a problem hiding this comment.
Yeah, but also there is the compatibility required for the standard library. The Swift standard library depends on some of the compiler resources. The reason for calling out the packging here is that at least on Windows, we package the SwiftShims into the SDK to maintain the compatibility with the standard library. But I agree with you that we should make this bit explicit.
There was a problem hiding this comment.
Yeah... SwiftShims is unfortunate since it combines headers that are definitely compiler-ABI details with headers that are for the stdlib. So for that one, you can flip a coin to choose whether it's compiler-specific or stdlib-specific and you'd be equally right and wrong at the same time.
| These pieces compose in a particular manner to build a system image to build | ||
| code against. | ||
|
|
||
| The compiler resources are content for the compiler that the compiler provides. |
There was a problem hiding this comment.
Yeah, but also there is the compatibility required for the standard library. The Swift standard library depends on some of the compiler resources. The reason for calling out the packging here is that at least on Windows, we package the SwiftShims into the SDK to maintain the compatibility with the standard library. But I agree with you that we should make this bit explicit.
| code against. | ||
|
|
||
| The compiler resources are content for the compiler that the compiler provides. | ||
| In the case of the Swift compiler, this includes the Swift shims. Whilst this is |
There was a problem hiding this comment.
Standard library macros are new! However, I think that they should be going into the SDK - they are part of the standard library which goes into the SDK.
Co-authored-by: Evan Wilde <ewilde@apple.com>
Co-authored-by: Alexander Smarus <bender@readdle.com>
|
Great to finally see this documented. 🥳 |
The standard library is part of the "SDK Overlay", which seems odd, but makes sense after realization of what is going on. The "SDK Overlay" overlays the Darwin SDK - aka the sysroot. I was trying to avoid creating a new term as an existing term of art exists. The "SDK Overlay" is generally disjoint and what you end up with is a union of two filesystems (a la unionfs). This provides all the binary swiftmodules and auxiliary content required for Swift compilation for a platform. It would contain the standard library, dispatch, foundation, and potentially XCTest. The Swift shims also are provided through this union. |
| These pieces compose in a particular manner to build a system image to build | ||
| code against. | ||
|
|
||
| The compiler resources are content for the compiler that the compiler provides. |
There was a problem hiding this comment.
Yeah... SwiftShims is unfortunate since it combines headers that are definitely compiler-ABI details with headers that are for the stdlib. So for that one, you can flip a coin to choose whether it's compiler-specific or stdlib-specific and you'd be equally right and wrong at the same time.
| code against. | ||
|
|
||
| The compiler resources are content for the compiler that the compiler provides. | ||
| In the case of the Swift compiler, this includes the Swift shims. Whilst this is |
There was a problem hiding this comment.
Yes... but.... what triple do we compile them for?
I'm being somewhat facetious here.
There was a problem hiding this comment.
Could we put this file in a /docs/CrossCompilation or similar directory for better organization? Also, please consider adding a mapping for the new file/directory to /.github/CODEOWNERS.
There was a problem hiding this comment.
I don't think that the proposed organisation is that much better - ultimately, this is about how the compiler works, what the flags do, and how they enable cross-compilation. I'm happy to rename the file though if you have a suggestion.
Co-authored-by: Danielle <dani@builds.terrible.systems>
|
|
||
| Currently, we do not have a good way to isolate the SDK overlay from the | ||
| remainder of the required content. On Darwin platforms, this content is shipped | ||
| as part of the SDK. As a result the singular `-sdk` flag allows control over the |
There was a problem hiding this comment.
Note that the new swift-driver does not look in -sdk for the SDK overlay, only for the C/C++ headers, apple/swift-driver#1562.
|
I'm fine with shifting the cross-compilation model over to this new set of flags, but it's misleading to say only "compiler resources are controlled via the driver flag I get that you guys don't like the way that flag was named and feel it was misused, but given the way it has long been used, including by |
I don't think that the change really changes the existing behaviour per se. It does however try to homogenise the behaviour between the various platforms. The behaviour on Windows and Darwin is already pretty similar, at least from the user's experience. The equivalent of the The default value for This would make all the platforms behave similarly making the experience a bit better. |
|
I'm not talking about those other flags: I'm talking about My point is that I am fine with changing what |
|
I think that for having a road to migrate we should be incrementally changing the behaviour. What this means is that we should preserve the existing behaviour of |
|
As I said, I'm all for making these changes, but I think we should mention that |
|
Ah, I see what you are asking for - sure, making a remark about the historical behaviour is reasonable. |
Add a note about the historical behaviour of the `-resource-dir` flag and the preference for `-sdk`.
Add some documentation to describe the proposed model for cross-compilation, associated flags, and the reasoning for the structure. This should allow us to have a reference for the design allowing us to evolve the model.