v0.22.0 (2024-05-12) #44141
tgamblin
announced in
Announcements
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
v0.22.0 (2024-05-12)
v0.22.0
is a major feature release.Features in this release
Compiler dependencies
We are in the process of making compilers proper dependencies in Spack, and a number
of changes in
v0.22
support that effort. You may notice nodes in your dependencygraphs for compiler runtime libraries like
gcc-runtime
orlibgfortran
, and youmay notice that Spack graphs now include
libc
. We've also begun moving compilerconfiguration from
compilers.yaml
topackages.yaml
to make it consistent withother externals. We are trying to do this with the least disruption possible, so
your existing
compilers.yaml
files should still work. We expect to be done withthis transition by the
v0.23
release in November.gcc-runtime: add separate package for gcc runtime libs #41104: Packages compiled with
%gcc
on Linux, macOS and FreeBSD now depend on anew package
gcc-runtime
, which contains a copy of the shared compiler runtimelibraries. This enables gcc runtime libraries to be installed and relocated when
using a build cache. When building minimal Spack-generated container images it is
no longer necessary to install libgfortran, libgomp etc. using the system package
manager.
Add
intel-oneapi-runtime
, allow injecting virtual dependencies #42062: Packages compiled with%oneapi
now depend on a new packageintel-oneapi-runtime
. This is similar togcc-runtime
, and the runtimes canprovide virtuals and compilers can inject dependencies on virtuals into compiled
packages. This allows us to model library soname compatibility and allows
compilers like
%oneapi
to provide virtuals likesycl
(which can also beprovided by standalone libraries). Note that until we have an agreement in place
with intel, Intel packages are marked
redistribute(source=False, binary=False)
and must be downloaded outside of Spack.
Improve hit-rate on buildcaches #43272: changes to the optimization criteria of the solver improve the hit-rate of
buildcaches by a fair amount. The solver more relaxed compatibility rules and will
not try to strictly match compilers or targets of reused specs. Users can still
enforce the previous strict behavior with
require:
sections inpackages.yaml
.Note that to enforce correct linking, Spack will not reuse old
%gcc
and%oneapi
specs that do not have the runtime libraries as a dependency.Allow reusing specs built with compilers not in config #43539: Spack will reuse specs built with compilers that are not explicitly
configured in
compilers.yaml
. Because we can now keep runtime libraries in buildcache, we do not require you to also have a local configured compiler to use the
runtime libraries. This improves reuse in buildcaches and avoids conflicts with OS
updates that happen underneath Spack.
libc runtime #43190: binary compatibility on
linux
is now based on thelibc
version,instead of on the
os
tag. Spack builds now detect the hostlibc
(glibc
ormusl
) and add it as an implicit external node in the dependency graph. Binarieswith a
libc
with the same name and a version less than or equal to that of thedetected
libc
can be reused. This is only onlinux
, notmacos
orWindows
.External package detection for compilers #43464: each package that can provide a compiler is now detectable using
spack external find
. External packages defining compiler paths are effectively used ascompilers, and
spack external find -t compiler
can be used as a substitute forspack compiler find
. More details on this transition are inthe docs
Improved
spack find
UI for EnvironmentsIf you're working in an enviroment, you likely care about:
We've tweaked
spack find
in environments to show this information much moreclearly. Installation status is shown next to each root, so you can see what is
installed. Roots are also shown in bold in the list of installed packages. There is
also a new option for
spack find -r
/--only-roots
that will only show envroots, if you don't want to look at all the installed specs.
More details in Improve
spack find
output in environments #42334.Improved command-line string quoting
We are making some breaking changes to how Spack parses specs on the CLI in order to
respect shell quoting instead of trying to fight it. If you (sadly) had to write
something like this on the command line:
That will now result in an error, but you can now write what you probably expected
to work in the first place:
Quoted can also now include special characters, so you can supply flags like:
To reduce ambiguity in parsing, we now require that you not put spaces around
=
and
==
when for flags or variants. This would not have broken before but will nowresult in an error:
More details and discussion in Upcoming high-impact changes #30634.
Revert default
spack install
behavior to--reuse
We changed the default concretizer behavior from
--reuse
to--reuse-deps
inconcretizer: add mode to reuse dependencies only #30990 (in
v0.20
), which meant that everyspack install
invocation wouldattempt to build a new version of the requested package / any environment roots.
While this is a common ask for upgrading and for developer workflows, we don't
think it should be the default for a package manager.
We are going to try to stick to this policy:
is a known security issue that necessitates an upgrade.
With the install command you now have three options:
--reuse
(default): reuse as many existing installations as possible.--reuse-deps
/--fresh-roots
: upgrade (freshen) roots but reuse dependencies if possible.--fresh
: install fresh versions of requested packages (roots) and their dependencies.We've also introduced
--fresh-roots
as an alias for--reuse-deps
to make it more clearthat it may give you fresh versions. More details in concretizer: update
reuse:
default to True #41302 and concretizer args: --fresh-roots == --reuse-deps #43988.More control over reused specs
You can now control which packages to reuse and how. There is a new
concretizer:reuse
config option, which accepts the following properties:roots
:true
to reuse roots,false
to reuse just dependenciesexclude
: list of constraints used to select which specs not to reuseinclude
: list of constraints used to select which specs to reusefrom
: list of sources for reused specs (some combination oflocal
,buildcache
, orexternal
)For example, to reuse only specs compiled with GCC, you could write:
Or, if
openmpi
must be used from externals, and it must be the only external used:New
redistribute()
directiveSome packages can't be redistributed in source or binary form. We need an explicit
way to say that in a package.
Now there is a
redistribute()
directive so that package authors can write:Like other directives, this works with
when=
:More in Add new
redistribute()
directive #20185.New
conflict:
andprefer:
syntax for package preferencesPreviously, you could express conflicts and preferences in
packages.yaml
throughsome contortions with
require:
:You can now use
require:
andprefer:
for a much more readable configuration:See the documentation
and Add syntactic sugar for "strong preferences" and "conflicts" #41832 for more details.
include_concrete
in environmentsYou may want to build on the concrete contents of another environment without
changing that environment. You can now include the concrete specs from another
environment's
spack.lock
withinclude_concrete
:Now, when this environment is concretized, it will bring in the already concrete
specs from
environment1
andenvironment2
, and build on top of them withoutchanging them. This is useful if you have phased deployments, where old deployments
should not be modified but you want to use as many of them as possible. More details
in Include concrete environments with
include_concrete
#33768.python-venv
isolationSpack has unique requirements for Python because it:
External installations may contain their own installed packages that can interfere
with Spack installations, and some distributions (Debian and Ubuntu) even change the
sysconfig
in ways that alter the installation layout of installed Python packages(e.g., with the addition of a
/local
prefix on Debian or Ubuntu). To isolate Spackfrom these and other issues, we now insert a small
python-venv
package in betweenpython
and packages that need to install Python code. This isolates Spack's buildenvironment, isolates Spack from any issues with an external python, and resolves a
large number of issues we've had with Python installations.
See python: always use a venv #40773 for further details.
New commands, options, and directives
spack develop
: stage build artifacts in same root as non-dev buildsspack develop
: stage build artifacts in same root as non-dev builds #41373spack develop
build artifacts after install (Don't delete "spack develop" build artifacts after install #43424)spack find
: add options for local/upstream only (spack find: add options for local/upstream only #42999)spack logs
: print log files for packages (either partially built or installed) ("spack logs": print log files for packages (either partially built or installed) #42202)patch
: support reversing patches (spack.patch: support reversing patches #43040)develop
: Add -b/--build-directory option to set build_directory package attribute (develop: Add -b/--build-directory option to set build_directory package attribute #39606)spack list
: add--namesapce
/--repo
option (spack list
: add--namesapce
/--repo
option #41948)checked_by
field tolicense()
, add some license checksspack gc
: add options for environments and build dependencies (spack gc
: add options for environments and build dependencies #41731)--create
tospack env activate
(Add--create
tospack env activate
#40896)Performance improvements
spec format
speed (Refactor to improvespec format
speed #43712)spec.copy()
(asp.py: fewer calls to spec.copy() #43715)__str__
jinja2
import at startup unless needed (performance: avoidjinja2
import at startup unless needed #43237)Other new features of note
archspec
: update tov0.2.4
: support for Windows, bugfixes forneoverse-v1
andneoverse-v2
detection.spack config get
/blame
: with no args, show entire configspack env create <env>
: dir if dir-like (spack env create <env>
: dir if dir-like #44024)Binary caches
Removals, deprecations, and syntax changes
dpcpp
compiler and package (remove dpcpp compiler and package #43418)Notable Bugfixes
+
in module file names (allow+
in module file names #41999)cmd/python
: use runpy to allow multiprocessing in scripts (cmd/python: use runpy to allow multiprocessing in scripts #41789)Spack community stats
v0.21.0
This discussion was created from the release v0.22.0 (2024-05-12).
Beta Was this translation helpful? Give feedback.
All reactions