Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support specifying disk image for additional disk with Google Batch #4535

Open
wants to merge 7 commits into
base: master
Choose a base branch
from

Conversation

hnawar
Copy link
Contributor

@hnawar hnawar commented Nov 22, 2023

New feature

Support specifying a disk image for additional disk when using Google Batch. This would be useful in many cases but particularly with large reference files which can be both time consuming and expensive.
Using DIsk image is faster and cheaper as the Disk is created from the image before the provisioning the VM

Usage scenario

User creates a Disk image containing all reference files
User specifies the image as part of the disk directive in the process

#4534

Signed-off-by: Hatem Nawar <hnawar@google.com>
Signed-off-by: Hatem Nawar <hnawar@google.com>
Copy link

netlify bot commented Nov 22, 2023

Deploy Preview for nextflow-docs-staging ready!

Name Link
🔨 Latest commit d6fca2b
🔍 Latest deploy log https://app.netlify.com/sites/nextflow-docs-staging/deploys/656dcfc713970700084c7bb0
😎 Deploy Preview https://deploy-preview-4535--nextflow-docs-staging.netlify.app/google
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

hnawar and others added 2 commits November 22, 2023 11:02
Signed-off-by: Ben Sherman <bentshermann@gmail.com>
@bentsherman bentsherman self-requested a review November 22, 2023 11:05
@bentsherman
Copy link
Member

This change by itself is fine, but the disk directive now has three purposes (resize boot disk, attach empty disk, attach disk with image) but can be specified only once. That means that e.g. you couldn't attach a disk with reference inputs and also attack a local SSD to use as scratch storage.

To solve this issue we just need to allow the disk directive to be specified multiple times. But I am fine with merging this PR as it is and implementing multiple disk directives in a separate PR.

@hnawar
Copy link
Contributor Author

hnawar commented Nov 22, 2023

Thanks Ben :)
Note, that the disk can still be used as scratch space if it is not completely full and is mounted in the same path with read/write permissions

I did think about the multiple disk directive, but did not have an answer for that

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants