Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add mechanism for configuring system image builds #54387

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

JeffBezanson
Copy link
Sponsor Member

@JeffBezanson JeffBezanson commented May 6, 2024

This adds the option to pass a filename of configuration settings when building the Core/compiler system image (from base/compiler/compiler.jl). This makes it easier to build different flavors of images, for example it can replace the hack that PackageCompiler uses to edit the list of included stdlibs, and makes it easy to change knobs you might want like max_methods.

Will need corresponding changes to PackageCompiler.

@JeffBezanson JeffBezanson added the domain:building Build system, or building Julia or its dependencies label May 6, 2024
@Seelengrab
Copy link
Contributor

Kind of tangential, but having full-on Julia source code as the configuration file format seems VERY dangerous, especially if people start sharing these. Would using a TOML be an option instead? That would also have the advantage of making the configuration file easier to document in terms of what the compiler actually looks for in the configuration options.

@KristofferC
Copy link
Sponsor Member

It would at least be good to have support for setting these from e.g. Make.user (similar to how one sets JULIA_PRECOMPILE, JULIA_CPU_TARGET etc).

@JeffBezanson
Copy link
Sponsor Member Author

TOML would be good, we just don't yet have a parser available at this stage. It will be awkward, since we don't even have arrays or dicts at this stage.

The target user for this is code like PackageCompiler that is building custom system images. I don't expect anyone to change these settings just when building julia with Make.user. For comparison, currently PackageCompiler literally rewrites the file sysimg.jl to make changes like this. So the threat model is like somebody sending you a file and telling you to replace sysimg.jl with it --- obviously that is dangerous and we can't prevent it.

@Seelengrab
Copy link
Contributor

TOML would be good, we just don't yet have a parser available at this stage. It will be awkward, since we don't even have arrays or dicts at this stage.

Yeah, I figured.. It's a bootstrapping thing all over again!

base/settings.jl Outdated
Comment on lines 8 to 10
if Intrinsics.slt_int(1, getfield(getfield(ARGS, :size), 1))
include(Settings, memoryrefget(memoryref(ARGS.ref, 2, false), :not_atomic, false))
end
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's this magic here?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is an array getindex that is hard-coded presumably due to bootstraping issues.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

specifically, this is

if 1 <= length(ARGS)
    include(Settings, ARGS[2])
end

base/settings.jl Outdated Show resolved Hide resolved
@JeffBezanson
Copy link
Sponsor Member Author

OK I have a better implementation:

  • Core.Compiler and Base each have their own settings, since they are separate libraries. You can pass --settings to either.
  • In the process, DRY the buildroot argument to the Base image build
  • No more unreadable intrinsics code
  • Backwards compatible with existing PackageCompiler

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
domain:building Build system, or building Julia or its dependencies
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants