[macOS] Generate universal gen_snapshots

[cbracken]

Previously, the gen_snapshot_arm64 and gen_snapshot_x64 binaries used by the tool were all built for x64 architecture. As such, developers building apps with Flutter rely on Rosetta translation with every build.

This refactors the gen_snapshot build rules on macOS hosts to consistently produce gen_snapshot_arm64 and gen_snapshot_x64 binaries with the target architecture of the build but with as universal binaries with both host architectures.

arm64 host build

Prior to this patch we emitted:

• gen_snapshot_arm64 (arch: x64, target_arch: simarm64)

After this patch, we emit:

• artifacts_x64/gen_snapshot_arm64 (arch: x64, target_arch: simarm64)
• artifacts_arm64/gen_snapshot_arm64 (arch: arm64, target_arch: arm64)
• gen_snapshot_arm64 (universal binary composed of both of the above)

x64 host build

Prior to this patch we emitted:

• gen_snapshot_x64 (arch: x64, target_arch: x64)

After this patch, we emit:

• artifacts_x64/gen_snapshot_x64 (arch: x64, target_arch: x64)
• artifacts_arm64/gen_snapshot_x64 (arch: arm64, target_arch: simx64)
• gen_snapshot_x64 (universal binary composed of both of the above)

Note that host builds on macOS currently default to a host architecture of x64 (can be overridden via --force-mac-arm64) regardless of host architecture and thus, the build itself relies on Rosetta translation when invoked on Apple Silicon arm64 hardware. This is to ensure a consistent build in CI regardless of bot architecture. See:

engine/tools/gn

Lines 502 to 505 in 6fa734d

	# TODO(cbracken):
	# https://github.com/flutter/flutter/issues/103386
	if get_host_os() == 'mac' and not args.force_mac_arm64:
	gn_args['host_cpu'] = 'x64'

Issue: flutter/flutter#101138
Issue: flutter/flutter#69157

Related issue: flutter/flutter#103386

Pre-launch Checklist

• I read the Contributor Guide and followed the process outlined there for submitting PRs.
• I read the Tree Hygiene wiki page, which explains my responsibilities.
• I read and followed the Flutter Style Guide and the C++, Objective-C, Java style guides.
• I listed at least one issue that this PR fixes in the description above.
• I added new tests to check the change I am making or feature I am adding, or the PR is test-exempt. See testing the engine for instructions on writing and running engine tests.
• I updated/added relevant documentation (doc comments with ///).
• I promise to revert this when it inevitably breaks the build.
• I signed the CLA.
• All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel on Discord.

[cbracken]

There's a bunch of common logic between this and the universal framework scripts that we should really extract into a library. I'll take a look and send a followup patch for the relevant bits.

[jmagman]

LGTM once the tests are happy

[zanderso]

DBC

What's the difference between create_macos_binary.py and create_macos_gen_snapshots.py? Do we need both of them?

[cbracken]

Oh interesting -- create_macos_gen_snapshots is old and was used for iOS (and is hardcoded full of armv7 stuff); I guess we never cleaned it up?

I'll send out a patch to kill it.

[jmagman]

create_macos_gen_snapshots is used in the generator which processes the output of the builds

engine/ci/builders/mac_host_engine.json

Line 560 in 7a309e5

"script": "flutter/sky/tools/create_macos_gen_snapshots.py"

To avoid the timeout and because an hour is really quite long to wait for the builds to clear, maybe the snapshots should be created in different builds, and then processed with the generator at the end?

[zanderso]

Yes, there should be examples of using global generators already in the ios/macos build json files, and I think that can work here if we know how to split the different targets across different builders.

[cbracken]

Reason for revert: while this patch worked fine, it pushed the mac build bot over its time limit. Previous builds were just squeaking under the wire with a minute or two to spare, but I'm seeing timeouts on the mac_host_engine host_release shard after this commit. e.g. https://ci.chromium.org/ui/p/flutter/builders/prod/Mac%20mac_host_engine/10667/overview

Looking at ci.yaml I see the timeout is set to 240, but the timeout on the failing shard is clearly 60 mins. See: https://ci.chromium.org/ui/p/flutter/builders/prod/Mac%20Production%20Engine%20Drone/344592/overview

[zanderso]

Ooof. Unfortunately, I think the subbuild timeout is hardcoded in the recipes. We need a case like this but for no rbe: https://flutter.googlesource.com/recipes/+/refs/heads/main/recipe_modules/shard_util/api.py#404.