Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Revisit support for Spack target architectures, for best fit with Wave requirements #4523

Open
marcodelapierre opened this issue Nov 20, 2023 · 6 comments · May be fixed by #4701
Open

Revisit support for Spack target architectures, for best fit with Wave requirements #4523

marcodelapierre opened this issue Nov 20, 2023 · 6 comments · May be fixed by #4701
Assignees
@marcodelapierre
Labels
software/wave

Comments

@marcodelapierre
Copy link
Member

marcodelapierre commented Nov 20, 2023 •
edited

We want to make sure we are able to build Wave Containers with target micro-architectures that are relevant/useful to the service.

Current status:

TO-DO:

For reference, list of Spack supported targets as of today: https://spack.readthedocs.io/en/latest/basic_usage.html#support-for-specific-microarchitectures
@pditommaso
Copy link
Member

What's the difference of x86_64 vs x86_64_v3?

@marcodelapierre marcodelapierre self-assigned this Nov 20, 2023
@marcodelapierre marcodelapierre mentioned this issue Nov 20, 2023
Open
@marcodelapierre
Copy link
Member Author

x86_64_v3 is a CPU family that corresponds to newer CPUs; it features a number of additional features, such as AVX2 (256 bit floating point registers for vector arithmetics). Compiling for it results in more performant binaries than for x86_64. Pretty much all CPUs marketed after 2015 fall in this family, so there is very, very little risk of incompatibility.

@pditommaso
Copy link
Member

OK, but still there's the need to support the target microarchitecture then

@marcodelapierre
Copy link
Member Author

Yes, in Spack jargon: name: 'linux/x86_64', target: 'x86_64_v3'

I have updated the original post above - indeed at first I had missed the point you have just made

@pditommaso
Copy link
Member

I see. Main problem there's no support for target arch in the current request object

nextflow/plugins/nf-wave/src/main/io/seqera/wave/plugin/SubmitContainerTokenRequest.groovy

Lines 34 to 122 in d2dd216

/**
* Tower access token required to enable the service
*/
String towerAccessToken
/**
* Tower refresh token
*/
String towerRefreshToken
/**
* Tower workspace id
*/
Long towerWorkspaceId
/**
* Tower endpoint
*/
String towerEndpoint
/**
* The ID of the workflow that submitted this container request
*/
String workflowId
/**
* Container image to be pulled
*/
String containerImage
/**
* Container build file i.g. Dockerfile of the container to be build
*/
String containerFile
/**
* List of layers to be added in the pulled image
*/
ContainerConfig containerConfig
/**
* Conda recipe file used to build the container
*/
String condaFile
/**
* Spack recipe file used to build the container
*/
String spackFile
/**
* The request container platform
*/
String containerPlatform
/**
* The target repository where the built container needs to be stored
*/
String buildRepository
/**
* The container repository to cache build layers
*/
String cacheRepository
/**
* Request
*/
String timestamp
/**
* Request unique fingerprint
*/
String fingerprint
/**
* Enable freeze container mode
*/
boolean freeze
/**
* Specify the format of the container file
*/
String format
/**
* When {@code true} build requests are carried out in dry-run mode.
*/
Boolean dryRun

@marcodelapierre
Copy link
Member Author

exactly, we will need to add it

@marcodelapierre marcodelapierre linked a pull request Jan 18, 2024 that will close this issue
Spack version & target #4554
Closed
@marcodelapierre marcodelapierre removed a link to a pull request Jan 30, 2024
Spack version & target #4554
Closed
@marcodelapierre marcodelapierre linked a pull request Jan 30, 2024 that will close this issue
Update: support for target architecture for Spack Wave #4701
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@marcodelapierre marcodelapierre

Labels
software/wave
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Update: support for target architecture for Spack Wave nextflow-io/nextflow
3 participants
@pditommaso @bentsherman @marcodelapierre